Senior Architect

Position Overview

We are seeking a highly motivated Data Platform Engineer to join our Platform Engineering team responsible for the roadmap, and evolution of enterprise-scale hybrid data platforms.

The Platform Engineering team manages large-scale data and analytics ecosystems that enable data engineering, data warehousing, AI/ML, governance, observability, and operational intelligence workloads across on-premises and cloud environments.

The role offers the opportunity to work with modern data platform technologies including Cloudera CDP base platform, Kubernetes, cloud-native infrastructure, AI/ML platforms, Elastic stack, and advanced automation frameworks.

The ideal candidate combines strong Linux and infrastructure engineering expertise with a passion for learning, automation, scalability, reliability, and continuous improvement.

Key Responsibilities

Platform Engineering

• Design, deploy, configure, and manage large-scale enterprise data platforms.
• Support Engineering lifecycle management activities including provisioning, IaC, upgrades, patching, capacity expansion, delivering on engineering roadmap
• Ensure platform reliability, availability, resiliency, and operational excellence across our environments.
• Infrastructure Automation & DevOps
• Develop and maintain Infrastructure-as-Code (IaC) solutions using Ansible, Terraform, Gitlab, and CI/CD pipelines.
• Create reusable automation frameworks and engineering tools to improve operational efficiency and reduce manual effort.
• Build and operate hybrid data platform services spanning on-premises infrastructure and cloud environments.
• Support modern data platform technologies including:
  • Cloudera CDP Private Cloud
  • Kubernetes
  • Apache Spark
  • Hive
  • Impala
  • Iceberg
  • Ozone
  • NiFi
  • Kafka
  • AI/ML enablement
  • Hybrid Cloud setup (Azure/AWS, on-premises)
• Assist with platform modernization and migration initiatives.

Security & Compliance

• Implement and maintain enterprise-grade security controls.
• Manage authentication, authorization, encryption, and certificate-based security models.
• Support Kerberos, LDAP, SSO, Ranger, TLS, RBAC, and regulatory compliance requirements.
• Conduct security reviews and platform hardening activities.

Performance Engineering

• Perform performance analysis and tuning of platform services.
• Optimize compute, storage, network, and application layer configurations.
• Conduct capacity planning, forecasting, and scalability assessments.
• Assist application teams with performance troubleshooting and platform best practices.

Platform Integration & Customer Enablement

• Integrate data platforms with enterprise services such as Active Directory, monitoring tools, observability platforms, cloud storage, and Kubernetes ecosystems.
• Collaborate with architecture, application, security, and infrastructure teams.
• Provide technical guidance to internal customers leveraging the platform.

Required Qualifications

• Bachelor's degree in Computer Science, Information Technology, or related field.
• 10+ years of experience in platform engineering, Systems Engineering role
• Strong Linux administration experience.
• Experience supporting distributed systems and enterprise infrastructure on multi-petabytes scale, managing clusters spanning 1000s of servers.
• Experience with scripting and automation using Python, Bash, or similar languages.
• Experience with Gitlab and CI/CD methodologies.
• Strong troubleshooting and problem-solving skills.
• Excellent communication and collaboration abilities.

Desired Technical Skills

Technologies

• Cloudera CDP
• Elasticsearch
• Microsoft Azure
• Spark
• Hive
• Impala/Trino
• Kafka
• NiFi
• Iceberg
• Ozone
• Python & Shell Scripting

Container & Cloud Technologies

• Kubernetes
• Docker
• Microsoft Azure/AWS

Automation & Infrastructure

• Ansible
• Terraform
• Shell Scripting & Python
• GitLab CI/CD

Security

• Kerberos
• LDAP
• SSO
• Ranger
• TLS/SSL - Certificate Management

Databases

• PostgreSQL
• NoSQL Platforms

Observability

• Elasticsearch
• Kibana
• Grafana
• Prometheus
• Operational Monitoring Platforms

Preferred Qualifications

• Experience operating enterprise-scale production platforms.
• Experience with AI/ML platforms and MLOps ecosystems.
• Experience with hybrid cloud architectures.
• Understanding of Data Lakehouse, Data Mesh, and modern data platform concepts.
• Experience supporting highly regulated environments (HIPAA, GDPR, GxP, etc.).
• Experience working with multi petabyte-scale data ecosystems.

What You'll Work On

• Large-scale hybrid data platforms supporting critical business workloads.
• Hybrid Cloud platform modernization and migration initiatives.
• AI/ML enablement platforms.
• Infrastructure automation and operational excellence programs.
• Platform observability, governance, and security enhancements.
• Next-generation technologies including Iceberg, Kubernetes/Docker, object storage, and AI-assisted operations.