Data Engineer
Virtusa Consulting Services
Data Engineer
Virtusa Consulting Services
Hyderabad
Not disclosed
Job Details
Job Description
Pyspark, AWS Engineer Lead
Description
6-8 years of relevant work experience showing growth as a Data Engineer.
Hands On programming experience
Implementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation.
Experience of performance optimization in Batch and Real time processing applications
Expertise in Data Governance and Data Security Implementation
Worked on Scheduling tools like Airflow
Good hands-on design and programming skills building reusable tools and products
Experience developing in AWS or similar cloud platforms. Preferred:, ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight or similar.
Good knowledge of Python. Good to have Java/Scala.
Familiarity with systems with very high volume of transactions, micro service design, or data processing pipelines (Spark).
Experience in efficient data processing using open table format Delta, Iceberg.
Knowledge and hands-on experience with server-less technologies such as Lambda, MSK, MWAA, Kinesis Analytics a plus.
Expertise in practices like Agile, Peer reviews, Continuous Integration
Hands On programming experience
Implementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation.
Experience of performance optimization in Batch and Real time processing applications
Expertise in Data Governance and Data Security Implementation
Worked on Scheduling tools like Airflow
Good hands-on design and programming skills building reusable tools and products
Experience developing in AWS or similar cloud platforms. Preferred:, ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight or similar.
Good knowledge of Python. Good to have Java/Scala.
Familiarity with systems with very high volume of transactions, micro service design, or data processing pipelines (Spark).
Experience in efficient data processing using open table format Delta, Iceberg.
Knowledge and hands-on experience with server-less technologies such as Lambda, MSK, MWAA, Kinesis Analytics a plus.
Expertise in practices like Agile, Peer reviews, Continuous Integration
Job role
Work location
Hyderabad
Department
Data Science & Analytics
Role / Category
Data Science & Machine Learning
Employment type
Full Time
Shift
Day Shift
Job requirements
Experience
Min. 8 years
About company
Name
Virtusa Consulting Services
Job posted by Virtusa Consulting Services
This job has expired