Data Bricks Module Lead

Experience

7+ Years

Job Description

Tech Stack:Databricks,PySpark,Python,SQL and NoSQL databases,Scala,Spark-SQL
Experience with Azure: ADLS, Databricks, Stream Analytics, SQL DW, COSMOS DB, Analysis Services, Azure Functions, Serverless Architecture, ARM Templates

Skills:
Working knowledge of Azure Analytics features such as Stream Analytics, Machine Learning and Application Insights.
Modelling and ETL knowledge with Erwin, Azure Databricks, Data Factory, SSIS and Azure Synapse.
Develop and maintain system integrations using, Azure Logic Apps, Integration Services, Power Automate, PowerApps, etc.
Microsoft Azure data platform experience with Power BI, Azure Data Lake, data warehouse and Data Factory.
Knowledge on migrating on-premises applications to Azure PaaS and Azure LaaS.
Attention to detail and ability to coordinate multiple tasks, set priorities and meet deadlines
Should be well versed with Data Structures & algorithms
Excellent analytical and problem-solving skills.
Ability to work independently as a self-starter, and within a team environment.
Good Communication skills- Written and Verbal

Work with business to understand their current state architecture and contributing data sources, technologies, interfaces, performance issues and system configurations.
Work closely with the team across the overall data warehousing program including data acquisition, data curation and data syndication.
Apply knowledge of the Microsoft platform tooling involved in successful Azure implementations.
Serve as the subject matter expert with respect to Azure, cloud applications and system administration best practices.
Experience designing data lakes, database schemas and data models for large scale data platform implementation on Azure.
Develop system integrations using, Azure Logic Apps, Integration services, Power Automate, PowerApps, etc.
Perform project management activities including project documentation, business requirements and project tracking.
Ensuring data quality and consistency through data cleaning, transformation, and integration processes.
Monitoring and troubleshooting data-related issues within the Azure environment to maintain high availability and performance.
Collaborating with data scientists, business analysts, and other stakeholders to understand data requirements and implement appropriate data solutions.
Implementing data security measures, including encryption, access controls, and auditing, to protect sensitive information.
Automating data pipelines and workflows to streamline data ingestion, processing, and distribution tasks.
Utilizing Azure's analytics services, such as Azure Synapse Analytics, to provide insights and support data-driven decision-making.
Keeping abreast of the latest Azure features and technologies to enhance data engineering processes and capabilities.
Documenting data procedures, systems, and architectures to maintain clarity and ensure compliance with regulatory standards.
Providing guidance and support for data governance, including metadata management, data lineage, and data cataloguing.

Required Behavioral Competencies

Accountability:

• Takes responsibility for and ensures accuracy of own work, as well as the work and deadlines of the team.

Collaboration:

• Participates in team activities and reaches out to others in team to achieve common goals.

Agility:

• Demonstrates a willingness to accept and embrace differing ideas or perceptions which are beneficial to the organization.

Customer Focus:

• Displays awareness of customers stated needs and gives priority to meeting and exceeding customer expectations at or above expected quality within stipulated time.

Communication:

• Targets communications for the appropriate audience, clearly articulating and presenting his/her position or decision.

Drives Results:

• Sets realistic stretch goals for self & others to achieve and exceed defined goals/targets.

Certifications