Database Engineer

Here's a professional Job Description (JD) for your Databricks Solution Architect opening:

Job Description – Databricks Solution ArchitectPosition DetailsJob Title: Databricks Solution Architect

Experience: 6–8 Years

Location: Remote

Open Positions: 7

CTC: Up to ₹22 LPA

Shift Timing: Flexible (Any 9 Hours between 11:00 AM – 12:00 Midnight)

Digital Technologies is a leading technology solutions company specializing in Data Science, Big Data, Enterprise Cloud, and Automation services. We help organizations leverage cutting-edge technologies to drive innovation, optimize operations, and accelerate digital transformation.

We are looking for an experienced and highly skilled Databricks Solution Architect to design, build, and optimize modern Lakehouse architectures while delivering scalable data and AI solutions on the Azure ecosystem.

• Key ResponsibilitiesNext-Generation Architecture DeliveryDesign and implement modern Data Lakehouse solutions using Databricks Serverless Compute.
• Architect scalable data platforms using Databricks SQL, Jobs, and Notebooks.
• Optimize infrastructure costs and eliminate operational overhead through serverless architecture adoption.
• Production EngineeringDevelop, optimize, and deploy production-grade PySpark and SQL solutions.
• Troubleshoot and resolve Spark performance issues, CI/CD failures, and complex Databricks workloads.
• Build and maintain reliable data pipelines and AI-powered analytics solutions.
• Performance OptimizationDiagnose and resolve bottlenecks such as Data Skew, Out-of-Memory (OOM) issues, and disk spills.
• Apply advanced optimization techniques including:
• 
  
• Adaptive Query Execution (AQE)
• Catalyst Optimizer
• Liquid Clustering
• Z-Ordering
• Partitioning Strategies
• Governance & SecurityImplement enterprise-wide governance using Unity Catalog.
• Configure Lakehouse Federation for seamless external data access.
• Enforce Attribute-Based Access Control (ABAC) and security best practices.
• Utilize System Tables for monitoring and observability.
• Stakeholder EngagementAct as a trusted technical advisor for customers and internal teams.
• Translate complex technical concepts into business-friendly language.
• Provide architecture recommendations and solution guidance.
• Must-Have SkillsDatabricks & Spark ExpertiseStrong expertise in Apache Spark architecture, Catalyst Optimizer, and AQE.
• Deep understanding of Liquid Clustering and modern data optimization strategies.
• ProgrammingAdvanced proficiency in:
• 
  
• Python (PySpark)
• SQL
• Ability to write complex transformation logic and UDFs independently.
• Delta Lake & LakehouseHands-on experience with:
• 
  
• Delta Lake
• ACID Transactions
• Time Travel
• Delta Live Tables (DLT)
• Unity Catalog & Data SecurityExperience implementing:
• 
  
• Unity Catalog
• Lakehouse Federation
• System Tables
• Volume-based Access Controls
• DevOps & Operational ExcellenceExperience with:
• 
  
• Terraform
• Databricks Asset Bundles (DABs)
• Databricks Connect v2
• CI/CD implementation and automation
• Preferred / Nice-to-Have SkillsSAP Data IntegrationExperience integrating SAP ECC and SAP S/4HANA data into Databricks.
• Familiarity with:
• 
  
• SAP Datasphere
• SAP SLT
• Fivetran
• Qlik Connectors
• Understanding of SAP IDOCs and BAPIs.
• AI & Machine LearningHands-on experience with:
• 
  
• MLflow
• Feature Store
• Model Registry
• Experience with Mosaic AI and Generative AI solutions.
• Familiarity with:
• 
  
• Scikit-learn
• XGBoost
• PyTorch
• Serverless ArchitectureExperience migrating workloads from classic compute environments to serverless architectures.
• Strong understanding of performance optimization and cost management.

CertificationsStrongly Preferred:

• Databricks Certified Data Engineer Professional
• Why Join Us?Work on cutting-edge Data, AI, and Cloud technologies.
• Opportunity to architect enterprise-scale Lakehouse solutions.
• Remote work flexibility.
• Collaborative and innovation-driven environment.
• Exposure to modern AI, ML, and GenAI initiatives.

Interested CandidatesShare your updated resume with the recruitment team at RC Digital Technologies.

Position: Databricks Solution Architect

Location: Remote

Experience: 6–8 Years

CTC: Up to ₹22 LPA

This version is recruiter-ready and can be posted directly on LinkedIn, Naukri, Indeed, or shared with staffing partners.

Send me the your cv 8433030548