Senior Big Data Platform Administrator

Senior Big Data Platform Administrator

Experience Required:

5–8 years of overall IT experience, with at least 3–5 years of hands-on experience in Big Data platform administration and Linux systems.

Job Summary

We are seeking an experienced Senior Big Data Platform Administrator to manage and support our enterprise Big Data infrastructure. The ideal candidate will have strong expertise in the Hadoop ecosystem, Linux administration, distributed systems, and platform monitoring tools. This role involves ensuring platform performance, availability, scalability, and security across production and non-production environments.

Key Responsibilities

• Administer and support Hadoop ecosystem components (HDFS, YARN) across Production, Test, and Development environments
• Manage and maintain Apache Spark and Airflow infrastructure
• Monitor cluster health, troubleshoot performance issues, and optimize system performance
• Implement and maintain High Availability (HA), Disaster Recovery (DR), backup, and restore strategies
• Manage logging, monitoring, and alerting systems using Prometheus, Grafana, and ELK Stack
• Perform Linux (Ubuntu) system administration, including patching, upgrades, and security hardening
• Develop and maintain automation scripts using Shell and/or Python
• Handle Kerberos authentication setup and maintenance
• Support incident management, root cause analysis (RCA), and ensure SLA adherence
• Manage capacity planning and system scalability
• Follow CI/CD and DevOps best practices for platform improvements and deployments

Skills Required

Mandatory Skills: 

• Strong experience with Hadoop ecosystem (HDFS, YARN)
• Hands-on experience with Apache Spark and Airflow
• Expertise in Linux (Ubuntu) administration
• Experience in implementing High Availability (HA) and Disaster Recovery (DR)

Optional Skills: 

• Experience with Prometheus, Grafana, and ELK Stack (Elasticsearch, Logstash, Kibana)
• Knowledge of Shell/Python scripting for automation
• Experience with Kerberos authentication
• Strong understanding of distributed systems, cluster management, and performance tuning
• Familiarity with CI/CD pipelines and DevOps practices

Preferred Qualifications

• Experience in managing large-scale on-premises Big Data environments
• Certification in Hadoop, Linux, or Cloud/DevOps technologies will be an added advantage

Educational Qualification

• Bachelor’s Degree in Engineering (B.E) – Computer Science / Information Technology

Key Competencies

• Strong problem-solving and analytical skills
• Excellent troubleshooting and debugging abilities
• Good communication and collaboration skills
• Ability to work in high-pressure, production-critical environments
• Proactive and self-driven mindset