Sr. Technology Engineer - (DevOps | Hadoop | Cloudera)

We are looking for an experienced Senior Technology Engineer with strong expertise in DevOps, Big Data Platforms, CI/CD automation, and container orchestration technologies. The ideal candidate will have hands-on experience managing enterprise-scale data platforms, Hadoop ecosystems, Cloudera environments, Kubernetes/OpenShift platforms, and automation frameworks.

The role focuses on designing, implementing, and supporting highly available technology platforms, enabling self-service automation, improving platform reliability, and collaborating with engineering, data, security, and architecture teams in an Agile DevOps environment.

The successful candidate will have strong experience in production environments, platform engineering, automation, infrastructure optimization, and supporting large-scale enterprise data platforms.

Key Responsibilities:

DevOps & CI/CD Engineering:
• Design, implement, and manage CI/CD pipelines using tools such as Jenkins, GitHub Actions, GitOps, and ArgoCD.
• Automate software delivery workflows and streamline development lifecycle processes.
• Build and maintain automated deployment pipelines for data and analytics projects.
• Develop automation scripts using Python, Shell, Groovy, Terraform, and Ansible.
• Automate security checks, compliance validations, patching processes, and operational activities.

Big Data Platform Engineering:
• Manage and support enterprise Big Data platforms using Hadoop ecosystem technologies.
• Provide hands-on Tier-2 production support for Cloudera Data Platform (CDP) environments.
• Perform installation, configuration, maintenance, and troubleshooting of Cloudera platforms.
• Optimize cluster performance and improve reliability of distributed data platforms.
• Work closely with Data Engineers to optimize Spark jobs, Hive workloads, and Yarn-based processing.
• Support Cloudera Data Services and containerized data workloads.

Containerization & Platform Engineering:
• Deploy and manage containerized applications using Docker, Podman, Kubernetes, and OpenShift.
• Support OpenShift Container Platform (OCP) environments ensuring scalability, availability, and operational efficiency.
• Implement container management best practices across enterprise platforms.
• Support cloud-native application and platform modernization initiatives.

Infrastructure Automation:
• Develop and maintain Infrastructure as Code (IaC) solutions using Terraform and Ansible.
• Automate infrastructure provisioning, configuration management, and operational processes.
• Build reusable automation frameworks to improve platform efficiency.
• Support automated patching and platform maintenance activities.

Monitoring & Performance Optimization:
• Implement monitoring and observability solutions for enterprise platforms.
• Troubleshoot performance issues across Hadoop, Spark, Hive, and distributed systems.
• Improve platform stability, scalability, and operational efficiency.
• Ensure high availability and reliability of technology services.

Collaboration & Delivery:
• Work closely with Data Engineering, Application Development, Security, Architecture, and Infrastructure teams.
• Collaborate with product and platform teams to onboard applications and users onto technology platforms.
• Participate in Agile ceremonies including sprint planning, backlog management, stand-ups, and retrospectives.
• Deliver solutions from inception through implementation and operational support.
• Provide technical guidance and support for platform adoption.

Required Technical Skills:

Mandatory:
• 8+ years of hands-on Technology Engineering / Platform Engineering experience.
• Minimum 4 years of production experience with Hadoop ecosystem technologies.
• Strong hands-on experience with Cloudera Data Platform (CDP).
• Experience with Cloudera installation, configuration, maintenance, and troubleshooting.
• Experience optimizing Spark jobs, Hive workloads, and Yarn processes.
• Strong experience with CI/CD automation and GitOps practices.
• Hands-on experience with:

• Jenkins
• GitHub Actions
• ArgoCD
• Git
  • Strong scripting experience:
• Python
• Shell scripting
• Groovy
  • Experience with:
• Terraform
• Ansible
  • Strong containerization experience:
• Docker
• Podman
  • Strong Kubernetes and OpenShift experience.
  • Strong Linux OS and networking fundamentals.
  • Experience supporting enterprise-scale, highly available technology platforms.

Preferred / Good to Have:
• Public cloud experience (AWS/Azure/GCP).
• Experience working on Data Science platforms or ML engineering environments.
• Experience with Kafka and distributed data architectures.
• Exposure to security automation and regulatory compliance checks.
• Experience working in banking, financial services, or large enterprise environments.

Required Experience:
• Experience working with large and complex enterprise environments requiring high availability, reliability, scalability, and observability.
• Experience delivering technology solutions from design through production implementation.
• Experience working in Agile + DevOps delivery models.
• Strong understanding of platform engineering practices.

Education:
• Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field.
• Equivalent professional experience will also be considered.

Key Competencies:
• Strong problem-solving and troubleshooting skills.
• Excellent communication and stakeholder management skills.
• Self-starter with strong ownership mindset.
• Ability to work under pressure in production environments.
• Innovative mindset with focus on automation and continuous improvement.
• Strong collaboration skills across engineering teams.