GCP Data Engineer

GCP Data Engineer

1. GCP BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, batch and real-time streaming, Databricks , Pyspark 2. Python, strong SQL Databricks with Delta Lake and Structured Streaming Responsibilities and JD in brief along with additional criteria to be considered: • Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms. • Build batch and real-time pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark and Databricks. • Develop and optimize Databricks, Delta Lake, Spark and Structured Streaming workloads for large-scale processing. • Implement high-throughput streaming solutions using Pub/Sub and Dataflow, including windowing, watermarking, late-arriving data, duplicate handling and dead-letter queues. • Handle high-volume TB/PB-scale data with focus on performance, scalability, reliability and cost optimization. • Develop complex SQL transformations, data models and analytics-ready BigQuery datasets using partitioning and clustering. • Orchestrate pipelines using Cloud Composer / Airflow and implement CI/CD and Terraform-based Infrastructure as Code. • Ensure data quality, governance, security, monitoring, alerting and production support. • Collaborate with architects, stakeholders and cross-functional teams in an Agile delivery model. • Troubleshoot and optimize end-to-end GCP, Databricks and streaming data pipelines. Infosys is a global leader in next-generation digital services and consulting. Within the Data & Analytics unit, you will work on cutting-edge data engineering and analytics initiatives for global clients across industries. The role offers opportunities to work with cloud-native data platforms, modern analytics ecosystems, AI-driven solutions, and large-scale enterprise data modernization programs. Employees benefit from continuous learning programs, certifications, and career growth opportunities within Infosys' data and analytics practice Responsibilities and JD in brief along with additional criteria to be considered: • Design and develop scalable GCP Data Engineering solutions and cloud-native data platforms. • Build batch and real-time pipelines using BigQuery, Cloud Storage, Dataflow, Pub/Sub, Dataproc, Python, PySpark and Databricks. • Develop and optimize Databricks, Delta Lake, Spark and Structured Streaming workloads for large-scale processing. • Implement high-throughput streaming solutions using Pub/Sub and Dataflow, including windowing, watermarking, late-arriving data, duplicate handling and dead-letter queues. • Handle high-volume TB/PB-scale data with focus on performance, scalability, reliability and cost optimization. • Develop complex SQL transformations, data models and analytics-ready BigQuery datasets using partitioning and clustering. • Orchestrate pipelines using Cloud Composer / Airflow and implement CI/CD and Terraform-based Infrastructure as Code. • Ensure data quality, governance, security, monitoring, alerting and production support. • Collaborate with architects, stakeholders and cross-functional teams in an Agile delivery model. • Troubleshoot and optimize end-to-end GCP, Databricks and streaming data pipelines.