Python Data Engineer

Specialist - Architecture

Job Title Python Data Engineer

Role Overview

The Python Data Engineer is responsible for designing building and maintaining scalable data pipelines and infrastructure This role involves working closely with data scientists analysts and business stakeholders to ensure reliable efficient and secure data flow across systems

Key Responsibilities

Design develop and maintain ETLELT pipelines for structured and unstructured data

Integrate data from multiple sources databases APIs streaming platforms

Optimize data storage and retrieval for performance and cost efficiency

Implement data quality checks validation and monitoring

Collaborate with crossfunctional teams to define data models and schemas

Ensure compliance with data governance security and privacy standards

Automate workflows and processes for data ingestion and transformation

Troubleshoot and resolve datarelated issues in production environments

Key Skills Competencies

Technical Skills

Programming Advanced Python Pandas NumPy PySpark etc

Databases Strong SQL skills PostgreSQL MySQL SQL Server and NoSQL MongoDB Cassandra

Big Data Tools Apache Spark Hadoop Kafka Flink

Data Warehousing Snowflake Amazon Redshift Google BigQuery Azure Synapse

Cloud Platforms AWS Glue S3 Lambda GCP Dataflow BigQuery Azure Data Factory Databricks

ETL Tools Airflow Luigi Prefect

APIs Integration REST GraphQL message queues

Version Control GitGitHubGitLab

Data Engineering Fundamentals

Data modeling OLTP OLAP starsnowflake schema

Data partitioning indexing and query optimization

Batch and realtime data processing

Data security encryption and compliance GDPR HIPAA

Soft Skills

Strong problemsolving and analytical thinking

Excellent communication and collaboration skills

Ability to work in agile fastpaced environments

Attention to detail and commitment to data accuracy

Preferred Qualifications

Bachelors or Masters degree in Computer Science Data Engineering or related field

3 years of experience in data engineering or backend development