Pratice Lead Data Engineer

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and ETL processes to support downstream analytics and AI applications.
• Collaborate with ML Engineers to integrate data solutions into machine learning models and workflows.
• Work closely with clients to understand their data requirements and deliver tailored data solutions.
• Ensure data quality, integrity, and security across all projects.
• Optimize and manage data storage solutions in cloud environments (AWS, Azure, GCP).
• Utilize Databricks for data processing and analytics tasks, leveraging its capabilities to enhance data workflows.
• Monitor the performance of data pipelines, identify bottlenecks or failures, and implement improvements to enhance efficiency and reliability.
• Implement best practices for data engineering, including documentation, testing, and version control.
• Troubleshoot and resolve data-related issues in a timely manner.

Qualifications

• Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field.
• 10+ years of experience as a Data Engineer or in a similar role.
• Strong proficiency in SQL, Python, and other relevant programming languages.
• Hands-on experience with Databricks and its ecosystem.
• Familiarity with major cloud environments (AWS, Azure, GCP) and their data services.
• Experience with data warehousing solutions like Snowflake, Redshift, or BigQuery.
• Comfortable working with a variety of SQL, NoSQL and graph databases like PostgreSQL and MongoDB;
• Knowledge of data integration tools.
• Understanding of data modeling, data architecture, and database design.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.

Highly Desirable Skills

• Experience with real-time data processing frameworks (e.g., Apache Kafka, Spark Streaming).
• Knowledge of data visualization tools (e.g., Tableau, Power BI).
• Familiarity with machine learning concepts and frameworks.
• Experience working in a client-facing role.