Databricks Developer

Key Responsibilities:

🔹 Databricks Stack Expertise:

Hands-on experience working with the Databricks stack and utilizing its features for data engineering and processing.

Proficiency in building and managing Databricks Lakehouse Platform solutions for scalable data pipelines.

🔹 Data Engineering Technologies:

Extensive experience with Spark, Hadoop, Kafka, and other data engineering technologies.

Expertise in implementing Data Warehousing solutions using Databricks, Spark, and Python.

🔹 Streaming Technologies:

Proficient in streaming technologies and working with real-time data processing using Apache Spark Streaming, Kafka, and related tools.

🔹 ETL Tools Expertise:

Proficient in using ETL tools (e.g., SSIS, Redwood, etc.) to create and optimize ETL pipelines for data ingestion and transformation.

🔹 Production Pipelines & Job Management:

Experience in building production-grade data pipelines in Databricks, using Spark SQL with Python for incremental data processing.

Strong understanding of submitting jobs via Workflows, API, and CLI for streamlined data processing.

🔹 Data Governance & Optimization:

Experience with Unity Catalog for data governance.

Focus on query/job optimization to improve performance and efficiency in production environments.

Required Skills & Qualifications:

• Databricks Experience: Strong hands-on experience with Databricks stack for building scalable and efficient data solutions.
• Data Engineering: Expertise in Spark, Hadoop, and Kafka, with a focus on real-time streaming.
• Programming Skills: Proficient in Python and SQL for data processing and transformation tasks.
• ETL & Data Warehousing: Experience with ETL tools like SSIS or Redwood, and solid understanding of data warehousing concepts.
• Job Management & Data Governance: Experience in using Databricks Workflows, API, CLI for job submission and Unity Catalog for data governance.

Why Join Us?

• Work with a cutting-edge Databricks platform in a fast-paced, collaborative environment.
• Be part of a team driving the future of data engineering and real-time data processing.
• Competitive compensation and opportunities for career growth and development.

Location: Bangalore, Work from office, ITPL.

Experience: 5+ Years

Industry: Technology / Data Engineering

Please share your updated resume to Devaraj.k@gyansys.com