Data & Analytics Data Pipelines Lead

Data & Analytics Data Pipelines Lead

Key Responsibilities
Data Pipeline Leadership
•    Lead the design, development, deployment, and support of enterprise data pipelines and integration solutions.
•    Establish standards, patterns, and best practices for data ingestion, transformation, orchestration, and delivery.
•    Oversee data movement across the full analytics lifecycle: 
o    Source systems and external data providers
o    Landing/staging databases
o    Enterprise data warehouses
o    Data lake environments
o    Analytics platforms
o    Reporting databases and data marts
•    Ensure scalable, secure, and high-performing data integration architectures.
•    Drive automation and operational efficiency within data pipeline environments.
Data Integration & Engineering
•    Manage batch, near-real-time, and streaming data ingestion processes.
•    Coordinate data onboarding and integration activities for new source systems and external data suppliers.
•    Define and maintain ETL, ELT, and data replication standards.
•    Support cloud and on-premises data integration platforms.
•    Collaborate with enterprise architects to align data movement solutions with strategic architecture standards.
Data Quality & Governance
•    Establish and maintain enterprise data quality controls and monitoring processes.
•    Define data validation, reconciliation, exception handling, and alerting frameworks.
•    Monitor pipeline performance and proactively identify data integrity issues.
•    Partner with data governance teams to ensure compliance with organizational standards.
•    Implement and maintain data lineage documentation across data movement processes.
•    Support audit, compliance, and regulatory reporting requirements related to data traceability.
Data Lineage & Metadata Management
•    Document end-to-end data flows across all pipeline stages.
•    Maintain lineage mapping between source systems, transformation processes, and downstream reporting assets.
•    Ensure metadata accuracy and availability for analytical consumers.
•    Support impact assessments related to upstream and downstream changes.
•    Drive adoption of data catalog and metadata management capabilities.
Production Support & Operations
•    Lead operational support for production data integration and analytics pipelines.
•    Manage ServiceNow incidents, service requests, problem records, and change activities related to data pipeline operations.
•    Coordinate incident triage, root cause analysis, issue resolution, and stakeholder communications.
•    Establish and monitor service level agreements (SLAs) and operational metrics.
•    Ensure rapid resolution of critical data availability and quality issues.
•    Coordinate production releases and change management activities.
Monitoring & Reliability
•    Implement pipeline monitoring, observability, and alerting solutions.
•    Track pipeline health, throughput, latency, failure rates, and data quality metrics.
•    Develop operational dashboards and reporting for platform performance.
•    Lead efforts to improve platform reliability, resiliency, and recoverability.
•    Support disaster recovery and business continuity processes for critical data assets.
Stakeholder & Team Leadership
•    Serve as the primary point of contact for data pipeline operations and support.
•    Partner with business intelligence, analytics, reporting, application, and infrastructure teams.
•    Mentor data engineers and analysts on integration standards and best practices.
•    Facilitate prioritization of enhancements, technical debt reduction, and operational improvements.
•    Communicate risks, issues, and performance metrics to leadership.
________________________________________
Required Qualifications
•    Bachelor's degree in Computer Science, Information Systems, Engineering, Data Analytics, or related field.
•    7+ years of experience in data engineering, data integration, ETL/ELT development, or data platform operations.
•    3+ years of experience leading enterprise-scale data pipeline and integration initiatives.
•    Experience supporting production data environments and operational processes.
•    Experience managing incident, problem, and change management processes within ServiceNow or similar ITSM platforms.
•    Strong understanding of data warehousing, dimensional modeling, and data lake architectures.
•    Experience implementing data quality and data governance practices.
•    Strong analytical, troubleshooting, and problem-solving skills.
________________________________________
Preferred Qualifications
•    Experience with cloud data platforms such as Oracle, Azure, AWS, or Google Cloud.
•    Experience with the Teradata platform and Teradata Data Mover (TDM)
•    Experience with modern data engineering platforms including: 
o    Azure Data Factory
o    Databricks
o    Synapse Analytics
o    Snowflake
o    Informatica
o    Talend
o    SSIS
o    Kafka
o    Fivetran
o    dbt
•    Experience with metadata management and data lineage tools.
•    Knowledge of DevOps, CI/CD, infrastructure automation, and data observability platforms.
•    Familiarity with Agile delivery methodologies and product operating models.
________________________________________
Key Competencies
Technical Competencies
•    Data Engineering
•    ETL/ELT Architecture
•    Data Warehousing
•    Data Lake Architectures
•    Data Quality Management
•    Metadata Management
•    Data Lineage
•    Production Support Operations
•    Monitoring and Observability
•    ServiceNow Administration and Workflow Processes
Leadership Competencies
•    Operational Excellence
•    Stakeholder Management
•    Team Leadership
•    Incident Management
•    Strategic Planning
•    Continuous Improvement
•    Risk Management
•    Communication and Influence

Experience Level

Senior Level