Associate Manager - Data Engineering

Associate Manager Data Engineering

Job Description:

The Technology Foundations, Data and Transformation Solutions (DTS) team provides ETL solutions and data to BI applications, products, and services. We are looking for an experienced Big Data Developer who loves solving complex problems for a full spectrum of technologies. The person in this role will develop and implement data pipelines for sources and downstream systems.

Role

• Highly capable in learning new technologies & frameworks and implementing them as per the project requirements by adhering to quality standards

• Experience in all phases of data warehouse development lifecycle, from gathering requirements to testing, implementation, and support

• Adept in analysing information system needs, evaluating end-user requirements, custom designing solutions and troubleshooting information systems

• Develop and implement data pipelines that extracts, transforms, and loads data into an information product that helps to inform the organization in reaching strategic goals

• Investigate and analyse alternative solutions for data storage, processing etc. to ensure most streamlined approaches are implemented

• Ensure operational resiliency of existing data pipelines by monitoring and resolving any issues.

• Communicate, collaborate and work effectively in a global environment.

• Lead projects through design, implementation, automation, and maintenance for large scale ETL processes supporting multiple business units

• Leverage industry best practices including proper use of source control, participation in code reviews, data validation and testing

• Implement best practices in Data Governance to ensure the data is available, usable and secure according to internal policies

• Mentor other Data Engineers on the team and ensure the efficient execution of their duties

• Assist in leading the development team and serve as a technical resource for team members

• Leverage new technologies and approaches to innovating with increasingly large data sets

• Ability to write algorithms with different rules

• Data warehousing principles & concepts and modification of existing data warehouse structures

All about you

• Must have experience deploying and working with big data technologies like Hadoop, Spark, and Sqoop

• Experience with streaming frameworks like Kafka.

• Experience designing and building ETL pipeline using NiFi

• Highly proficient in OO programming (Python, PySpark Java, and Scala)

• Experience with the Hadoop Ecosystem (HDFS, Yarn, MapReduce, Spark, Hive, Impala)

• Proficiency on Linux, Unix command line, Unix Shell Scripting, SQL and any Scripting language

• Proficiency on Linux, Unix command line, Unix Shell Scripting, SQL and any Scripting language

• Experience designing and implementing large, scalable distributed systems

• Ability to debug production issues using standard command line tools

• Create design documentation and maintain process documents

• Ability to debug Hadoop / Hive job failures

• Ability to use Cloudera in administering Hadoop

Optional: Cloud technologies like Databricks, AWS, Azure and GCP.

At DXC Technology, we believe strong connections and community are key to our success. Our work model prioritizes in-person collaboration while offering flexibility to support wellbeing, productivity, individual work styles, and life circumstances. We’re committed to fostering an inclusive environment where everyone can thrive.

Recruitment fraud is a scheme in which fictitious job opportunities are offered to job seekers typically through online services, such as false websites, or through unsolicited emails claiming to be from the company. These emails may request recipients to provide personal information or to make payments as part of their illegitimate recruiting process. DXC does not make offers of employment via social media networks and DXC never asks for any money or payments from applicants at any point in the recruitment process, nor ask a job seeker to purchase IT or other equipment on our behalf. More information on employment scams is available here.

Experience Level

Mid Level