Data Engineer

Data Engineer 1

Responsibilities-

• Contributing to business applications and POCs using Python and big data technologies (e.g., Spark).
• Development of Data Analysis, Data Processing engines using Python
• Development of REST API using Python
• Develop and maintain ETL/ELT workflows for batch and real-time data ingestion
• Development & Deployment of applications on AWS/Azure
• Ability to do POCs on new tech stack and to integrate the same in the applications on functional level
• Ability to understand the business requirements, system architecture and process guidelines
• Ability to code for multiple projects at a time
• Ability to code for complex requirements under tight timelines
• Ability to work with cross functional / domain teams
• Ability to handle end to end projects on individual basis
• Ability to work in a fast-paced and agile development environment
• Solve complex analytic challenges using analytic algorithms and AI
• Build and deploy predictive and prescriptive models using statistical modeling, machine learning, and optimization
• Create structured prompts for LLMs to perform tasks like summarization or classification.
• Develop routines to parse and integrate LLM outputs into downstream systems.
• Prepare and execute test plans and scripts for data pipelines.

Qualification -

• Qualification: B.E or Post graduate in –Computer Science, Electronics & Instrumentation.

Experience & Skill Sets:

• Exp 2 to 4 Yrs. with minimum 2 yrs. In python & SQL development (Mandatory)

• Minimum 1 year of experience working with python > 3.x

• Minimum 1 year of implementing and delivering projects using CI/CD tools such as git, Jenkins, docker, Kubernetes, Kubeflow Pipelines, Airflow, Spark etc. 

• Minimum of 1-year experience in building microservices and web API

• Hands on experience using numpy and pandas 

• Experience working with PostgreSQL database will be added advantage.

• Experience working with Node JS and Node-Red will be added advantage.

• Devops experience and mindset is a big plus.

• Some experience using Google-OR tools would be an added advantage

• Exposure to building ETL pipelines and data integration; machine learning and NLP would be an added advantage.

• Good communication skill

• Must be flexible, independent and self-motivated.

• Team player and learning ability.

• Result orientation and focus on details.

• Strong analytical and problem-solving skills.

• Practical understanding of Object-Oriented Design Principles and Design Patterns
• Experience in secure product development lifecycle practices
• Proficiency in Visual Studio Code or Visual Studio with strong debugging skills such as watch, threads, modules, memory view, disassembly view etc.
• Experience working with Azure DevOps 
• Software Development best practices like using GIT, collaborative platforms like Github or bitbucket
• Experience working in an Agile/Scrum development process
• Behavior Driven (BDD) or Test-Driven Development (TDD)
• Experience building apps and solutions in Windows and Linux

Industry-Specific Experience 

• Experience/understanding of available opensource software and tooling 
• Experience in setting up app containerization (Docker and Kubernetes) 
• Past hands-on experience and understanding of Angular, Bootstrap 4, HTML 5, CSS3, Responsive Design
• Good verbal & written English communication
• Ability to work in a complex team structure, including distributed (I.e. global) teams.
• Readiness to work in flexible work timings.
• Interpersonal savvy. A very good team player.
• Quick and enthusiastic learner and a team player. Willingness to explore and adapt
• Willingness to learn new technologies and ability to build quick demonstrable proof of concepts.
• Experience working with data science teams and contributions to projects would be a huge plus.

Experience Level

Mid Level