Lead Architect - Data

About the role

• Participate in all areas of the data Engineering life cycle and lead the team End to End Development to deployment.
• Lead the design, development, and maintenance of scalable data pipelines and ETL processes.
• Ensure the reliability, efficiency, and performance of data systems.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions.
• Implement data governance and security measures to protect sensitive information.
• Optimize and tune data systems for performance and scalability.
• Develop and maintain documentation for data processes and systems.
• Troubleshoot and resolve complex data-related issues in a timely manner.
• Mentor and guide junior data engineers, fostering a culture of continuous learning and improvement.
• Stay updated with the latest industry trends and technologies in data engineering.

Knowledge and Skills Required

• System Design & Engineering: Ability to create scalable, reliable, and efficient IT architectures
• Programming Languages: Proficiency in Python, Java, or Scala for data processing and scripting.
• SQL and Database Management: Strong experience with SQL and database management systems (e.g., MySQL, PostgreSQL, Oracle).
• Big Data Technologies: Familiarity with big data technologies such as Hadoop, Spark, and Kafka.
• Messaging Systems: Good understanding of Kafka and RabbitMQ for message brokering.
• Orchestration: Experience with Apache Airflow.
• API Concepts: Good understanding of RESTful APIs and microservices architecture.
• DevOps Tooling: Experience with Jenkins, Maven, GitLab, SonarQube, and Docker.
• Cloud Platforms: Experience with other cloud platforms (e.g., Google Cloud) is a plus.
• Data Warehousing: Knowledge of data warehousing solutions like Redshift, and Snowflake.
• Problem-Solving Skills: Strong analytical and problem-solving skills.
• Communication and Collaboration: Excellent communication and collaboration abilities.
• Agile Development: Experience working with Agile development methodologies.
• Monitoring: Understanding of Monitoring tools like Prometheus, Grafana etc is a plus
• Cloud Computing: Understanding cloud platforms like AWS, Azure, or Google Cloud

Requirements

• Bachelor’s or master’s degree in computer science, Engineering, or a related field.
• 10+ years of experience in data engineering or a related role.
• Demonstrated ability to design and implement complex data systems.
• Knowledge of data governance and security best practices.