Data Engineer

Data Engineer

Project Role : Data Engineer
Project Role Description : Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
Must have skills : Microsoft Azure Data Services
Good to have skills : NA
Minimum 3 year(s) of experience is required
Educational Qualification : 15 years full time education

Summary:
As a Data Engineer, your typical day involves designing, developing, and maintaining comprehensive data solutions that support the generation, collection, and processing of data. You will be responsible for creating efficient data pipelines that facilitate smooth data flow and ensure the integrity and quality of data throughout its lifecycle. Your role includes implementing processes to extract, transform, and load data, enabling seamless migration and deployment across various systems. This position requires a proactive approach to managing data infrastructure and collaborating with teams to support organizational data needs effectively.

Key Responsibilities:
A. Function as the Junior Data Architect for a small, simple project/proposal or as a team lead for small/medium sized project
B. Discuss specific data architecture and data related issues with data architect/team (in area of expertise)
C. Experienced in Data Agents creation and Gen AI /Agentic AI Use case implementation
D. Github Copilot experience is an added advantage
E. Knowledge in LLM and Prompt engineering, AI foundry
F. candidate should have worked in Data governance Solution
G. Analyze and assess the impact of the requirements on the data and its lifecycle
H. Lead the data architecture and design of small database systems
I. Breadth of experience in various client scenarios and situations
J. Thought leadership and innovation
K. Experience in creation of new data assets & offerings
L. Experience in handling OLTP data workloads

Technical Experience:
A. Strong experience in Azure is preferred with hands-on experience in 2 or more of these skills : Azure SQL DB ,Azure SQL Managed Instance ,Azure Data Lake Store, Azure Cosmos DB, Azure Database for PostgreSQL, Azure Database for MySQL
B. Candidate must have around 1-3 years of extensive database experience
C. Candidate should have deep knowledge in Python, LLM
D. Experience in Azure purview/Profisee/Unity Catalog
E. Candidate must have 3-7 years of IT experience and around 1-3 years of extensive database experience
G. Experience working in medium to large database projects
H. Experience in using the required best practices and collaborate with Data Architect as required.

Professional Experience:
A. Team player
B. Should have excellent client communication skills
C. Should have good analytical and problem-solving skills

Educational Qualification:
A. Must have: BE/Btech/MCA
B. Good to have: ME/MTech