Major Accountabilities: Collaborate with the CIO on application Architecture and Design of our ETL (Extract, Transform, Load) and other aspects of Data Pipelines. Our stack is built on top of the well-known Spark Ecosystem (e.g. Scala, Python, etc.) Periodically evaluate architectural landscape for efficiencies in our Data Pipelines and define current state, target state architecture and transition plans, road maps to achieve desired architectural state Conducts/leads and implements proof of concepts to prove new technologies in support of architecture vision and guiding principles (e.g. Flink) Assist in the ideation and execution of architectural principles, guidelines and technology standards that can be leveraged across the team and organization. Specially around ETL & Data Pipelines Promotes consistency between all applications leveraging enterprise automation capabilities Provide architectural consultation, support, mentoring, and guidance to project teams, e.g. architects, data scientist, developers, etc. Collaborate with the DevOps Lead on technical features Define and manage work items using Agile methodologies (Kanban, Azure boards, etc) Leads Data Engineering efforts (e.g. Scala Spark, PySpark, etc) Knowledge & Experience Experienced with Spark, Delta Lake, and Scala to work with Petabytes of data (to work with Batch and Streaming flows) Knowledge of a wide variety of open source technologies including but not limited to; NiFi, Kubernetes, Docker, Hive, Oozie, YARN, Zookeeper, PostgreSQL, RabbitMQ, Elasticsearch A strong understanding of AWS/Azure and/or technology as a service (Iaas, SaaS, PaaS) Strong verbal and written communications skills are a must, as well as the ability to work effectively across internal and external organizations and virtual teams Appreciation of building high volume, low latency systems for the API flow Core Dev skills (SOLID principles, IOC, 12-factor app, CI-CD, GIT) Messaging, Microservice Architecture, Caching (Redis), Containerization, Performance, and Load testing, REST APIs Knowledge of HTML, JavaScript frameworks (preferably Angular 2+), Typescript Appreciation of Python and C# .NET Core or Java Appreciation of global data privacy requirements and cryptography Experience in System Testing and experience of automated testing e.g. unit tests, integration tests, mocking/stubbing Relevant industry and other professional qualifications Tertiary qualifications (degree level) We are an inclusive employer and welcome applicants from all backgrounds. We pride ourselves on our commitment to Equality and Diversity and are committed to removing barriers throughout our hiring process. Key Requirements: Extensive data engineering development experience (e.g., ETL), using well known stacks (e.g., Scala Spark) Experience in Technical Leadership positions (or looking to gain experience) Background software engineering The ability to write technical documentation Solid understanding of virtualization and/or cloud computing technologies (e.g., docker, Kubernetes) Experience in designing software solutions and enjoys UML and the odd sequence diagram Experience operating within an Agile environment Ability to work independently and with minimum supervision Strong project development management skills, with the ability to successfully manage and prioritize numerous time pressured analytical projects/work tasks simultaneously Able to pivot quickly and make rapid decisions based on changing needs in a fast-paced environment Works constructively with teams and acts with high integrity Passionate team player with an inquisitive, creative mindset and ability to think outside the box.

More about this Big Data Developer job

ATF Labs is aggressively hiring for the job profile of Big Data Developer at Agra in MG Road locality. Kindly go through the FAQs below to get all answers related to the given job. 1. How much salary can I expect? Ans. You can expect a minimum salary of 50,000 INR and can go up to 70,000 INR. The salary offered will depend on your skills, experience and performance in the interview. 2. What is the eligibility criteria to apply for this job? Ans. The candidate should have completed Graduate degree and people who have 3 to 31 years are eligible to apply for this job. You can apply for more jobs in Agra to get hired quickly. 3. Is there any specific skill required for this job? Ans. The candidate should have Good (Intermediate / Advanced) English skills and sound communication skills for this job. 4. What are the timings of the job and how many working days are there? Ans. It is a Full Time job having timing from 09:30 AM to 06:30 PM and will have 6 working days i.e., monday - saturday. 5. Is it a work from home job? Ans. No, it’s not a work from home job and can’t be done online. You can explore and apply for other work from home jobs in Agra at apna. 6. Are there any charges or deposits required while applying for the role or while joining? Ans. No work-related deposit needs to be made during your employment with the company. 7. How can I apply for this job? Ans. Go to the apna app and apply for this job. Click on the apply button and call HR directly to schedule your interview. 8. What is the last date to apply? Ans. The last date to apply for this job is 02-Feb-2022. For more details, download apna app and find Full Time jobs in Agra. Through apna, you can find jobs in 74 cities across India. Join NOW!