Databricks Administrator

Databricks Admin

2–5 years of experience in Databricks Administration / Big Data Platforms. Strong hands-on experience with Databricks (Workspace, Clusters, Jobs, Notebooks). Experience with Apache Spark (PySpark/Scala) and distributed data processing. Knowledge of cloud platforms (Azure preferred, AWS/GCP also acceptable). Experience with data storage systems like ADLS, S3, Blob Storage. Good understanding of data engineering concepts, ETL/ELT pipelines. Familiarity with security best practices (IAM roles, data encryption, private endpoints). Proficiency in SQL and scripting languages (Python, Shell, etc.). Experience in performance tuning and cost optimization for clusters and workloads. Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem. Infosys provides equal employment opportunities to applicants and employees without regard to race, color, sex, gender identity; sexual orientation, religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability. Administer and manage Azure/AWS Databricks workspaces. Configure and maintain clusters, including creation, scaling, and optimization. Manage user access, roles, and permissions using RBAC and integration with identity providers (Azure AD, etc.). Monitor cluster performance and optimize cost, resource utilization, and job execution. Implement and manage job scheduling, workflows, and pipelines. Ensure data security and governance, including data encryption and access policies. Integrate Databricks with data sources such as ADLS, S3, Snowflake, or SQL databases. Troubleshoot issues related to jobs, clusters, notebooks, and integrations. Support CI/CD pipelines and DevOps processes for Databricks deployments. Monitor usage through logs, metrics, and audit trails. Maintain documentation for architecture, configurations, and processes. Collaborate with data engineers, data scientists, and DevOps teams.