Desired Competencies (Technical/Behavioral Competency)

Must-Have

Hadoop, Scala/Spark

Good-to-Have

Pyspark , Python

SN

Responsibility of / Expectations from the Role

1

Create Scala/Spark jobs for data transformation and aggregation

·         Produce unit tests for Spark transformations and helper methods

·         Write Scaladoc-style documentation with all code

·         Design data processing pipelines

·         Scala (with a focus on the functional programming paradigm)

·         Spark query tuning and performance optimization

·         Experience working with HDFS, Hive , Oozie

·         Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)

2

Seeking experienced ETL developer to work in Hadoop technologies , participating in design, development, implementation, troubleshooting and optimizing distributed solutions. Excellent analytical capabilities with knowledge of databases, SQL, ETL is desired.

3

Good written and oral communications skills. Willing to learn in a fast paced highly demanding environment.