Integration and Data Engineer

Integration Engineer

Project Role : Integration Engineer
Project Role Description : Provide consultative Business and System Integration services to help clients implement effective solutions. Understand and translate customer needs into business and technology solutions. Drive discussions and consult on transformation, the customer journey, functional/application designs and ensure technology and business solutions represent business requirements.
Must have skills : Python Frameworks
Good to have skills : NA
Minimum 5 year(s) of experience is required
Educational Qualification : 15 years full time education

Project Role : Data Engineer

Project Role Description : Seasoned Python Developer with strong hands-on expertise in Python programming, cloud integrations, database development, and data engineering and cloud computing platforms such as Databricks, AWS, Azure & Google. Build and operate very large data warehouses or data lakes, ETL optimization, designing, coding, tuning big data processes using Big Data and Apache Spark. Build data pipelines applications to stream and process datasets at low latencies. Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.

Technical Experience :
1. Design, develop and maintain data solutions for data generation, collection, and processing. Create data pipelines, ensure data quality, and implement ETL (extract, transform and load) processes to migrate and deploy data across systems.
2. Detailed experience working on large databases (SQL, NoSQL, query optimization). Working experience in programming languages Python, Java, Scala Experience using airflow or orchestration platforms to build data pipelines in 2-3 projects.
3. 2-3 years of experience developing CICD pipelines using GIT, Jenkins, Docker, Kubernetes, Shell Scripting, Terraform.
4. Must be able to understand ETL technologies and translate into Cloud (AWS, Azure, Google Cloud) native tools, Python or PySpark.