Assistant Vice President

Senior Data Engineer (T-SQL, Snowflake, DBT, Prefect, CI/CD, DevOps)

 Position Overview:
As a Senior Data Engineer at EXL, you will leverage your expertise in T-SQL, Snowflake, DBT, and Prefect to design and implement data pipelines and workflows that power critical business insights. You will play a key role in the replication of stored procedures from SQL Server to Snowflake and will also be responsible for integrating CI/CD practices using Jenkins and GitLab. Strong communication skills and the ability to collaborate with cross-functional teams will be key to success in this role. If you have a passion for DevOps and data engineering, this is the perfect opportunity for you.

Key Responsibilities:

• Data Pipeline Development & Optimization:
  Design, develop, and maintain robust, scalable, and efficient data pipelines to move, transform, and load data into Snowflake from various data sources. Ensure data pipelines are optimized for performance and cost.
• Stored Procedure Replication:
  Leverage your expertise in T-SQL to replicate complex stored procedures from SQL Server to Snowflake, ensuring that the replicated logic works seamlessly and performs at scale in the cloud environment.
• DBT (Data Build Tool) Management:
  Use DBT to create and manage data transformations within Snowflake. Build reusable, efficient, and reliable data models and transformations that are aligned with business needs.
• Workflow Automation with Prefect:
  Implement and manage automated workflows using Prefect Scheduler to orchestrate data pipelines and ETL/ELT processes. Ensure reliability, monitoring, and alerting for critical data workflows.
• CI/CD and DevOps Integration:
  Design, develop, and maintain CI/CD pipelines for data projects using tools such as Jenkins, GitLab, or similar. Integrate these pipelines into the development lifecycle to automate testing, deployment, and version control for data workflows.
• Collaboration & Communication:
  Work closely with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements and deliver data solutions that meet organizational goals. Communicate complex technical concepts to non-technical stakeholders.
• Code Versioning & Management:
  Use GitLab or similar version control systems to manage the development and deployment of data engineering solutions. Foster best practices in collaborative code management, branching, and version control.
• Performance Tuning:
  Continuously monitor, troubleshoot, and optimize data pipelines for performance, ensuring they scale efficiently with large data volumes and meet SLAs.
• Data Quality & Documentation:
  Ensure data integrity and quality across the pipeline. Maintain clear, detailed documentation for code, architecture, and processes to enable knowledge sharing within the team.

Qualifications:

• Experience:
  • 8+ years of hands-on experience as a Data Engineer, with deep expertise in T-SQL, Snowflake, DBT, Prefect, and CI/CD practices.
  • Proven experience replicating SQL Server stored procedures to Snowflake, ensuring optimized performance and functional accuracy.
  • Strong experience with CI/CD pipelines using Jenkins, GitLab, or similar tools.
  • Solid understanding of DevOps principles and the ability to integrate CI/CD into data engineering workflows.
• Technical Skills:
  • Advanced proficiency in T-SQL for writing complex queries, stored procedures, and performance optimization.
  • Experience with Snowflake, including data modeling, schema design, and performance tuning.
  • Experience with DBT for creating and managing data transformations.
  • Familiarity with Prefect for workflow orchestration and data pipeline scheduling.
  • Strong background in CI/CD and DevOps practices using Jenkins, GitLab, or other similar platforms.
  • Python experience is preferred but not mandatory, especially for data processing or orchestration tasks.
• Database & ETL Skills:
  • Extensive experience with SQL Server and Snowflake, with the ability to replicate, optimize, and integrate stored procedures and data pipelines.
  • Proficient in building, deploying, and maintaining ETL/ELT pipelines for large-scale data processing.
• Collaboration & Communication:
  • Strong communication and interpersonal skills, with the ability to work effectively with cross-functional teams and convey complex technical information clearly.
  • Proven ability to manage multiple projects simultaneously, with a focus on quality, timeliness, and business value.
• Education:
  • A bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field is required.
  • Advanced certifications or training in data engineering, DevOps, or cloud technologies is a plus.
• Soft Skills:
  • Strong problem-solving abilities and an analytical mindset.
  • Ability to work in a fast-paced, collaborative, and evolving environment.
  • A self-starter with a strong attention to detail and the ability to work independently and in teams.