Data Platform Engineer - Databricks Performance Optimization Specialist

Data Platform Engineer

Project Role : Data Platform Engineer
Project Role Description : Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
Must have skills : Databricks Unified Data Analytics Platform, Python (Programming Language), Machine Learning, Data Building Tool
Good to have skills : NA
Minimum 3 year(s) of experience is required
Educational Qualification : 15 years full time education

Summary: As a databricks workload optimization Engineer, you will engage with data engineers to optimize and enhance the performance of large-scale data processing systems on the Databricks platform. Focus on improving query efficiency, scalability, and cost-effectiveness across distributed data environments. Roles & Responsibilities: - Analyze Databricks Performance Optimization Metrics and Thresholds to help identify optimization opportunities across workloads - Implement Ad-hoc Analytics Optimization Tactics(Broadcast joins, window functions with partitions) - Engage in code reviews to ensure quality and adherence to best practices. Professional & Technical Skills: - Must To Have Skills: Proficiency in Databricks Unified Data Analytics Platform. - Deep expertise in performance tuning, benchmarking, and troubleshooting within data ecosystems such as but not limited to: - ETL/Streaming Optimization Tactics(Z-order, change data feed, auto compaction) - Understanding of adaptive Query Execution for Dynamic Optimization, Predictive I/O for Large Scan Operations Additional Information: - The candidate should have minimum 5 years of experience in Databricks Unified Data Analytics Platform. - This position is based at our Mumbai office. - A 15 years full time education is required. - Candidate should be ready to work in Rotational shifts.