Role description
Who we are
At UST, we help the world's best organizations grow and succeed through transformation. Bringing together the right talent, tools, and ideas, we work with our clients to co-create lasting change. Together, with over 30,000 employees in 30+ countries, we build for boundless impact—touching billions of lives in the process. Visit us at UST.com.
Summary
UST is looking for a skilled Kafka Administrator / Developer to join our Data Engineering team. The ideal candidate will be responsible for designing, deploying, administering, and optimizing Apache Kafka and Confluent Kafka platforms that power enterprise-scale real-time event streaming, messaging, and distributed data processing systems. This role requires expertise in Kafka administration, stream processing, cloud platforms, monitoring, security, and automation.
The Opportunity
- Design, deploy, configure, and administer Apache Kafka and Confluent Kafka clusters in production environments.
- Manage Kafka brokers, topics, partitions, replication, consumer groups, and cluster configurations.
- Develop Kafka producers and consumers using Java, Python, or Scala.
- Build and maintain real-time stream processing applications using Kafka Streams and ksqlDB.
- Implement and manage Schema Registry using Avro, JSON, and Protobuf schemas for data serialization.
- Design and maintain Kafka Connect pipelines for integrating databases, cloud storage, data lakes, and enterprise applications.
- Monitor Kafka cluster health, throughput, latency, broker performance, and consumer lag.
- Perform Kafka capacity planning, performance tuning, and infrastructure scaling.
- Ensure high availability, fault tolerance, disaster recovery, and business continuity of Kafka environments.
- Implement and enforce Kafka security using TLS/SSL, SASL Authentication, RBAC, and ACL-based authorization.
- Collaborate with Data Engineers, Application Developers, and Solution Architects to implement event-driven architectures and streaming solutions.
- Automate Kafka administration and deployment activities using Terraform, Ansible, and scripting.
- Develop operational documentation, runbooks, deployment guides, and standard operating procedures.
- Participate in production support, troubleshooting, root cause analysis, and performance optimization activities.
- Promote best practices in Kafka administration, event streaming, monitoring, and platform governance.
What You Need
- 3+ years of hands-on experience with Apache Kafka administration and/or development.
- Strong understanding of Kafka architecture, including:
- Brokers
- Topics
- Partitions
- Consumer Groups
- Offsets
- ISR (In-Sync Replicas)
- Replication
- ZooKeeper and/or KRaft
- Hands-on experience with:
- Kafka Connect
- Kafka Streams
- Schema Registry
- ksqlDB
- Strong programming skills in Java, Python, or Scala.
- Experience working in Linux/Unix environments with shell scripting.
- Strong understanding of distributed systems concepts including:
- Event-Driven Architecture
- CAP Theorem
- Eventual Consistency
- CQRS
- Event Sourcing
- Experience deploying Kafka on cloud platforms such as:
- AWS MSK
- Confluent Cloud
- Azure Event Hubs
- Google Cloud Pub/Sub
- Strong knowledge of Kafka security including:
- TLS/SSL
- SASL Authentication
- RBAC
- ACLs
- Excellent troubleshooting, debugging, and root cause analysis skills.
- Experience with monitoring and observability tools:
- Prometheus
- Grafana
- Kibana
- ELK/EFK Stack
- Experience with Confluent Platform and Confluent Control Center.
- Experience with Docker and Kubernetes for containerized deployments.
- Knowledge of CI/CD pipelines using:
- Jenkins
- GitHub Actions
- GitLab CI
- Experience with Infrastructure as Code (IaC) tools:
- Experience with enterprise messaging and real-time analytics platforms.
- Confluent Certified Developer or Confluent Certified Administrator certification is an added advantage.
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related technical discipline.
- Relevant certifications in Kafka, Cloud, or DevOps technologies are preferred.
- Minimum 3+ years of experience in Apache Kafka Administration and/or Kafka Development.
- Experience designing and supporting enterprise-scale event streaming platforms.
- Proven experience with cloud-native, distributed, and high-availability messaging systems.
What We Believe
We're proud to embrace the same values that have shaped UST since the beginning. Since day one, we've been building enduring relationships and a culture of integrity. Today, those same values inspire us to encourage innovation, champion diversity and inclusion, and place people at the center of everything we do.
We will listen, learn, be empathetic, and help selflessly in our interactions with everyone.
Through business, we will better the lives of those less fortunate than ourselves.
We honor our commitments and act with responsibility in all our relationships.
Equal Employment Opportunity Statement
UST is an Equal Opportunity Employer. We believe that no one should be discriminated against because of their differences, such as age, disability, ethnicity, gender, gender identity and expression, religion, or sexual orientation.
All employment decisions shall be made without regard to age, race, creed, color, religion, sex, national origin, ancestry, disability status, veteran status, citizenship status, or any other basis protected by applicable federal, state, or local law.
UST reserves the right to periodically redefine your roles and responsibilities based on the requirements of the organization and/or your performance.
- To support and promote the values of UST.
- Comply with all Company policies and procedures.
Skills
Apache Kafka, Python, Java, Scala, Prometheus, TLS, SASL, Kubernetes, Docker, Terraform, Schema Registry
About UST
UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world’s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients’ organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact—touching billions of lives in the process.
