Senior Data Engineer

Senior Data Engineer

Career Category

Information Systems

Job Description

What you will do

Let’s do this. Let’s change the world. In this vital role as Specialist / Senior Data Engineer at Amgen you will be responsible for designing, building, maintaining, analyzing, and interpreting data to provide actionable insights that drive business decisions. This role involves working with large datasets, developing reports, supporting and executing data initiatives and, visualizing data to ensure data is accessible, reliable, and efficiently managed. The ideal candidate has strong technical skills, experience with big data technologies, and a deep understanding of data architecture and ETL processes

• Lead and mentor a team of data engineers, fostering a culture of innovation, collaboration, and continuous learning for solving complex problems of R&D division.
• Oversee the development of data extraction, validation, and transformation techniques, ensuring ingested data is of high quality and compatible with downstream systems.
• Guide the team in writing and validating high-quality code for data ingestion, processing, and transformation, ensuring resiliency and fault tolerance.
• Drive the development of data tools and frameworks for managing and accessing data efficiently across the organization.
• Oversee the implementation of performance monitoring protocols across data pipelines, ensuring real-time visibility, alerts, and automated recovery mechanisms.
• Coach engineers in building dashboards and aggregations to monitor pipeline health and detect inefficiencies, ensuring optimal performance and cost-effectiveness.
• Lead the implementation of self-healing solutions, reducing failure points and improving pipeline stability and efficiency across multiple product features.
• Oversee data governance strategies, ensuring compliance with security policies, regulations, and data accessibility best practices.
• Guide engineers in data modeling, metadata management, and access control, ensuring structured data handling across various business use cases.
• Collaborate with business leaders, product owners, and cross-functional teams to ensure alignment of data architecture with product requirements and business objectives.
• Prepare team members for stakeholder discussions by helping assess data costs, access requirements, dependencies, and availability for business scenarios.
• Drive Agile and Scaled Agile (SAFe) methodologies, managing sprint backlogs, prioritization, and iterative improvements to enhance team velocity and project delivery.
• Stay up-to-date with emerging data technologies, industry trends, and best practices, ensuring the organization leverages the latest innovations in data engineering and architecture.
• Design, develop, and optimize data pipelines/workflows using Databricks (Spark, Delta Lake) for ingestion, transformation, and processing of large-scale data. A knowledge of Medallion Architecture will be an added advantage.
• Collaborate with Data Architects, Business SMEs, and Data Scientists to design and develop end-to-end data pipelines to meet fast paced business needs across geographic regions
• Identify and resolve complex data-related challenges
• Adhere to best practices for coding, testing, and designing reusable code/component
• Analyze business and technical requirements and begin translating them into simple development tasks
• Execute unit and integration tests, and contribute to maintaining software quality
• Contribute to the maintenance and support of applications by monitoring performance and reporting issues
• Use CI/CD pipelines as part of DevOps practices and assist in the release process

What we expect of you

We are all different, yet we all use our unique contributions to serve patients.

Basic Qualifications:

• Master’s/Bachelor’s degree and 8 to 13 years of Computer Science, IT or related field experience

Preferred Qualifications:

• Experience with Software engineering best-practices, including but not limited to version control, infrastructure-as-code, CI/CD, and automated testing
• Knowledge of Python/R, Databricks, cloud data platforms
• Strong understanding of data governance frameworks, tools, and best practices.
• Knowledge of data protection regulations and compliance requirements (e.g., GDPR, CCPA)

Must-Have Skills:

• Experience managing a team of data engineers.
• Experience architecting and building data and analytics solutions that extract, transform, and load data from multiple source systems.
• Demonstrated hands-on experience with cloud platforms (AWS) and the ability to architect cost-effective and scalable data solutions.
• Proficiency in Python, PySpark, SQL.
• Experience with dimensional data modeling.
• Experience working with Apache Spark, Apache Airflow.
• Experienced with software engineering best-practices, including but not limited to version control (Git, Subversion, etc.), CI/CD (Jenkins, Maven etc.), automated unit testing, and Dev Ops.
• Experienced with AWS or GCP or Azure cloud services.
• Understanding of end to end project/product life cycle
• Well versed with full stack development & DataOps automation, logging frameworks, and pipeline orchestration tools.
• Strong analytical and problem-solving skills to address complex data challenges.
• Effective communication and interpersonal skills to collaborate with cross-functional teams.

Professional Certifications

• AWS Certified Data Engineer (must have)
• Databricks Certificate (preferred)

Soft Skills:

• Excellent problem-solving and analytical skills
• Strong communication and interpersonal abilities
• High attention to detail and commitment to quality
• Ability to prioritize tasks and work under pressure
• Team-oriented with a proactive and collaborative mindset
• Willingness to mentor junior developers and promote best practices
• Adaptable to changing project requirements and evolving technology

.

Experience Level

Senior Level