Data Integration Developer

Purpose of Position:

Primary role of this position is to develop robust data integration pipelines based on the agreed specifications. Responsibilities under this role would be to develop the DI processes based on mapping document, perform unit testing, fixing the bugs raised by QA team, provide UAT support and prepare for deployment of the code after UAT signoff and provide warranty support post production deployment. 

Key Result Areas and Activities:

Data Integration and Implementation: Oversee the complete lifecycle of data integration processes, including design, development, and deployment. Engage in sessions with Business Analysts (BAs) or directly with clients to understand data integration needs and requirements. Incorporate and propose best practices for the design and development of data integration solutions to ensure quality and efficiency. Estimate the effort required for implementing data integration processes to manage resources and timelines effectively.

Documentation and Validation: Assist BAs in creating detailed mapping and specification documents to guide data integration efforts. Perform thorough data analysis to validate mapping documents and ensure accurate data integration and transformation. Develop prototypes and proof-of-concepts (POCs) for complex data integration use cases to test and refine solutions.

Design, Development, and Testing: Design and develop data integration processes using the chosen technology stack to ensure robust data flow and transformation. Develop test cases and perform unit and performance testing to validate the functionality and performance of data integration solutions. Offer deployment and post-production support to address any issues and ensure the smooth operation of data integration processes.

Automation and Innovation: Identify and propose automation opportunities using CLI/API to enhance efficiency and reduce manual effort in data integration processes. Contribute to data culture initiatives, including data cataloging, governance, and user adoption, to foster effective data management practices. Explore and propose innovative solutions to improve data integration practices and leverage emerging technologies.

Work and Technical Expertise:

Essential Skills:

• Ability to take ownership and drive end-to-end activities alone or as part of team in agile teams
• Deep experience with traditional/modern DI techniques. Understanding of data literacy, data catalogs, data culture is a plus
• Experience with writing and optimizing SQL queries is must
• Experience with formal dimensional modelling techniques, cloud native databases like Snowflake, Google Big Query is a big plus
• Experience in deployment & integration which includes setting up configuration management, developing components, peer reviews

Desirable Skills:

• Knowledge of shell or batch scripting
• Knowledge of monitoring tools, auditing, and administration
• Knowledge of incorporating data privacy and security solutions in the implementation
• Should have good knowledge of RDBMS concepts, MPP systems, etc.
• Should have knowledge of ABC framework (Audit, Balance, Control)

Qualifications:

• Bachelor’s degree in computer science, Information Technology, or a related field
• 3 to 5 years of IT experience
• 3+ years of experience on any of these DI / ETL Tools (Informatica PowerCenter / Talend / Data stage)

Qualities:

• Excellent verbal and written communication
• Collaboration skills to work in a self-organized and cross-functional teams
• Strong troubleshooting and problem-solving abilities
• Excellent analytical, presentation, reporting, documentation, and interactive skills