Senior Backend Developer - Big Data and Workflow Orchestration

Process Manager

Description

The ideal candidate must possess in-depth functional knowledge of the process area and apply it to operational scenarios to provide effective solutions. The role enables to identify discrepancies and propose optimal solutions by using a logical, systematic, and sequential methodology. It is vital to be open-minded towards inputs and views from team members and to effectively lead, control, and motivate groups towards company objects. Additionally, candidate must be self-directed, proactive, and seize every opportunity to meet internal and external customer needs and achieve customer satisfaction by effectively auditing processes, implementing best practices and process improvements, and utilizing the frameworks and tools available. Goals and thoughts must be clearly and concisely articulated and conveyed, verbally and in writing, to clients, colleagues, subordinates, and supervisors.

Senior Backend Developer – Big Data & Orchestration (10+ Years)

Role Overview
We are seeking a highly experienced Senior Backend Developer with strong expertise in Big Data engineering, Apache Spark, Scala, and workflow orchestration. The ideal candidate will design scalable data pipelines, optimize Spark workloads, and support scheduling platforms such as Airflow and Control-M while leading technical initiatives within the team.
Key Responsibilities
•    Build and optimize large-scale distributed applications using Apache Spark
•    Develop backend components using Scala
•    Design and maintain complex ETL/ELT data pipelines
•    Work across the Hadoop/Cloudera ecosystem (HDFS, YARN, Hive)
•    Develop, schedule, and optimize workflows using Airflow
•    Configure and support scheduling workflows in Control-M
•    Contribute to architecture, design documents, and technical solutions
•    Mentor junior developers and lead technical delivery
•    Participate in CI/CD processes and production support activities
Must-Have Skills
•    Apache Spark (Core, SQL, Streaming, performance tuning)
•    Scala
•    Workflow orchestration using Apache Airflow
•    Job scheduling and operations using Control-M
•    Strong experience with Hadoop/Cloudera components (HDFS, YARN, Hive)
•    Solid understanding of distributed systems and scalable data architectures
•    Proven leadership in backend and Big Data engineering teams
Good-to-Have Skills
•    Python and PySpark
•    IAM (AWS identity and access basics)
•    Delta Lake
•    Databricks notebooks and cluster setup
•    Kafka or real-time streaming
•    Docker and Kubernetes
•    Other orchestration tools such as ADF, Step Functions, or Luigi
•    Airflow cloud providers (AWS, GCP, Azure)
Soft Skills
•    Excellent communication and collaboration skills
•    Strong analytical and troubleshooting abilities
•    High ownership and proactive mindset
•    Ability to manage multiple priorities and lead delivery
Work Experience
•    10+ years as a Backend Developer or Big Data Engineer
•    Demonstrated experience building scalable and reliable data solutions

About eClerx

eClerx is a global leader in productized services, bringing together people, technology and domain expertise to amplify business results. Our mission is to set the benchmark for client service and success in our industry. Our vision is to be the innovation partner of choice for technology, data analytics and process management services. Since our inception in 2000, we've partnered with top companies across various industries, including financial services, telecommunications, retail, and high-tech. Our innovative solutions and domain expertise help businesses optimize operations, improve efficiency, and drive growth. With over 18,000 employees worldwide, eClerx is dedicated to delivering excellence through smart automation and data-driven insights. At eClerx, we believe in nurturing talent and providing hands-on experience.

About eClerx Digital

eClerx Digital is the trusted partner of choice to many of the world’s largest global brands for creative production, e-commerce/web operations, and analytics and insights services. We help clients maximize the results of their digital activities across the customer journey. Global industry leaders partner with us to cost-effectively scale their digital activities by leveraging our highly skilled blended resource model.

To know more about us, visit https://eclerx.com

eClerx is an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability or protected veteran status, or any other legally protected basis, per applicable law.

A Trusted Innovation Partner, Delivering Business Results | eClerx

eClerx is a global leader in productized services, enhancing business outcomes through technology, Artificial Intelligence, and deep domain expertise.