Senior Data Engineer - Spark and Scala Specialist

Power Programmer - Senior Technologist. Data Engineering (Q2-FY-26)

Indepth hands on experience of developing, testing, deployment and debugging of Spark Jobs using Scala in Hadoop Platform In depth knowledge of Spark Core, working with RDDs, Spark SQL In depth knowledge on Spark Optimization Techniques and Best practices Good Knowledge of Scala Functional Programming: Try, Option, Future, Collections Good Knowledge of Scala OOPS: Classes, Traits and Objects (Singleton and Companion), Case Classes. Scala Language Features: Type System, Implicit/Givens Hands on experience of working in Hadoop Environment (HDFS/Hive), AWS S3, EMR, Workflow Orchestration tools like Airflow, Oozie Understanding and exposure to file formats such as Apache AVRO, Parquet, JSON Writing Test cases using frameworks such as scalatest. Experience on using GIT, resolving conflicts, working with branches. Strong programming skills using data structures and algorithms. Mandatory AWS Glue, AWS Redshift/Spectrum, S3, API Gateway, Athena, Step and Lambda functions Experience in Extract Transform Load (ETL) and Extract Load & Transform (ELT) data integration pattern. Experience in designing and building data pipelines. Development experience in one or more object-oriented programming languages, preferably Python • They will be ahead of the technology curve and will be constantly enabled and trained to be Polyglots., • They are Go-Getters with a drive to solve end customer challenges and will spend most of their time in designing and coding, End to End contribution to technology oriented development projects., • Providing solutions with minimum system requirements and in Agile Mode., • Collaborate with Power Programmers. • Open Source community and Tech User group. • Custom Development of new Platforms & Solutions, Opportunities. • Work on Large Scale Digital Platforms and marketplaces. • Work on Complex Engineering Projects using cloud native architecture • Work with innovative Fortune 500 companies in cutting edge technologies.