Data Engineer

Data Engineer | Python Development

Data Engineer with Python development experience

Experience: 3+ Years

Mode: Hybrid (2-3 days/week)

Location: Hyderabad

Key Responsibilities

• Develop, test, and deploy data processing pipelines using AWS Serverless technologies such as AWS Lambda, Step Functions, DynamoDB, and S3.
• Implement ETL processes to transform and process structured and unstructured data efficiently.
• Collaborate with business analysts and other developers to understand requirements and deliver solutions that meet business needs.
• Write clean, maintainable, and well-documented code following best practices.
  
• Monitor and optimize the performance and cost of serverless applications.
• Ensure high availability and reliability of the pipeline through proper design and error handling mechanisms.
• Troubleshoot and debug issues in serverless applications and data workflows.
• Stay up-to-date with emerging technologies in the AWS and serverless ecosystem to recommend improvements.
  
  
  Required Skills and Experience
• 3-5 years of hands-on Python development experience, including experience with libraries like boto3, Pandas, or similar tools for data processing.
• Strong knowledge of AWS services, especially Lambda, S3, DynamoDB, Step Functions, SNS, SQS, and API Gateway.
• Experience building data pipelines or workflows to process and transform large datasets.
• Familiarity with serverless architecture and event-driven programming.
• Knowledge of best practices for designing secure and scalable serverless applications.
• Proficiency in version control systems (e.g., Git) and collaboration tools.
• Understanding of CI/CD pipelines and DevOps practices.
  Strong debugging and problem-solving skills.
• Familiarity with database systems, both SQL (e.g., RDS) and NoSQL (e.g., DynamoDB).
  
  
  Preferred Qualifications
• AWS certifications (e.g., AWS Certified Developer Associate or AWS Certified Solutions Architect Associate).
• Familiarity with testing frameworks (e.g., pytest) and ensuring test coverage for Python applications.
• Experience with Infrastructure as Code (IaC) tools such as AWS CDK, CloudFormation.
• Knowledge of monitoring and logging tools .

Experience Level

Mid Level