Experience: 10+ Years
Location: Bangalore
Role: Full-time / Technical Lead
Role Summary
We are seeking a highly capable Technical Lead with deep expertise in Big Data (Spark, Hadoop, Hive), Kubernetes platforms, Azure cloud, PostgreSQL, CI/CD (GitHub Actions/Jenkins), and DevOps automation. The ideal candidate will be a strong hands-on engineer who also brings mature leadership capabilities, sound architectural judgment, and exceptional software craftsmanship practices.
Key Responsibilities
1. Software Engineering & Craftsmanship
- Develop high‑quality, maintainable code using Software Craftsmanship principles such as:
- Test-Driven Development (TDD)
- Continuous Integration / Continuous Delivery / Continuous Deployment
- Clean code, refactoring, SOLID principles
- Participate actively in design reviews and code reviews to ensure engineering excellence.
- Implement quick POCs using the latest tech stack to validate ideas and technical feasibility.
- Apply design patterns and engineering best practices for scalable and maintainable solutions.
- Collaborate in all phases of the SDLC , with strong understanding of Agile/Scrum or continuous delivery environments.
2. Big Data Engineering & Distributed Systems
- Lead development of large-scale ETL/ELT pipelines using Apache Spark (Java/Scala) .
- Oversee data processing frameworks in Hadoop and Hive , especially within Azure-based ecosystems .
- Optimize Spark/Hive workloads for cost, performance, and reliability.
3. Cloud, Kubernetes & Infrastructure Engineering
- Lead architecture and operations of distributed applications running on:
- Kubernetes (On-Prem + AKS)
- Docker
- Drive adoption of Helm , container standards, and cluster best practices.
- Build and maintain Azure infrastructure using Terraform (IaC) with reusable, scalable modules.
- Ensure end-to-end observability of clusters using Prometheus, ELK, APM tools, and custom Python scripts.
4. DevOps Engineering & CI/CD Leadership
- Define and own the platform CI/CD strategy using:
- GitHub Actions
- Jenkins
- Terraform automation
- Build standardized CI/CD frameworks for 50+ microservices, leveraging Helm and automated build workflows.
- Automate DNS creation, certificate management, APM integration, user management, Vault policy creation, and repository migrations (200+ repos).
- Champion an automation-first mindset across the engineering and SRE teams.
- Coordinate with developers and testers for smooth deployment and production readiness.
5. Database Engineering
- Provide leadership on PostgreSQL cluster management:
- HA setup
- Performance tuning
- Failover mechanisms
- Monitoring dashboards
- Automate PostgreSQL operations using Python and infrastructure scripts.
6. Observability, Stability & Support
- Ensure reliability and availability of services across stacks (Java, Angular, .NET).
- Implement monitoring and alerting using Prometheus, Grafana, ELK, Elastic APM.
- Oversee production stability through proactive dashboards and automated health checks.
- Provide guidance for L2/L3 support and participate in incident review and resolution.
7. Leadership & Collaboration
- Lead a cross-functional team of developers, DevOps engineers, and data engineers.
- Translate business requirements into technical designs and actionable backlogs.
- Mentor team members on coding practices, DevOps processes, and cloud-native patterns.
- Collaborate with distributed teams and communicate effectively with technical and non-technical stakeholders.
Required Skills & Qualifications
- 10+ years of experience in software engineering / data engineering / DevOps roles.
- Strong expertise in:
- Apache Spark (Java/Scala)
- Hadoop, Hive on Azure
- Kubernetes, Docker
- Azure Kubernetes Service (AKS)
- Ansible
- Jenkins & GitHub Actions
- PostgreSQL (HA + automation)
- Strong understanding of CI/CD, SDLC, Agile, and continuous delivery.
- Good understanding of networking fundamentals , DNS, LBs, ingress controllers.
- Experience working on production support and high-availability systems.
Soft Skills
- Strong leadership and mentoring capabilities.
- Excellent communication skills with distributed teams.
- Ability to analyze complex problems and deliver quick, practical solutions.
- High sense of ownership and accountability.
Nice-to-Have
- Experience with Azure Data Lake, Databricks, or Synapse.
- Exposure to microservices architecture and domain-driven design.
