Data Engineer

Virtusa Consulting Services

Pune

Not disclosed

Work from Office

Full Time

Min. 6 years

Job Details

Job Description

Pyspark AWS Data Engineer

Description
6-8 years of relevant work experience showing growth as a Data Engineer. Hands On programming experience Implementation Experience on Kafka, Kinesis, Spark, AWS Glue, AWS LakeFormation. Experience of performance optimization in Batch and Real time processing applications Expertise in Data Governance and Data Security Implementation Worked on Scheduling tools like Airflow Good hands-on design and programming skills building reusable tools and products Experience developing in AWS or similar cloud platforms. Preferred:, ECS, EKS, S3, EMR, DynamoDB, Aurora, Redshift, QuickSight or similar. Good knowledge of Python. Good to have Java/Scala. Familiarity with systems with very high volume of transactions, micro service design, or data processing pipelines (Spark). Experience in efficient data processing using open table format Delta, Iceberg. Knowledge and hands-on experience with server-less technologies such as Lambda, MSK, MWAA, Kinesis Analytics a plus. Expertise in practices like Agile, Peer reviews, Continuous Integration

Job role

Work location

Pune

Department

Data Science & Analytics

Role / Category

Data Science & Machine Learning

Employment type

Full Time

Shift

Day Shift

Job requirements

Experience

Min. 6 years

About company

Name

Virtusa Consulting Services

Job posted by Virtusa Consulting Services

Apply on company website

Follow us on social media


© 2024 Apna | All rights reserved Privacy Policy Terms & Conditions