About Us:
At apexanalytix, we’re lifelong innovators! Since the date of our founding nearly four decades ago we’ve been consistently growing, profitable, and delivering the best procure-to-pay solutions to the world. We’re the perfect balance of established company and start-up. You will find a unique home here.
And you’ll recognize the names of our clients. Most of them are on The Global 2000. They trust us to give them the latest in controls, audit and analytics software every day. Industry analysts consistently rank us as a top supplier management solution, and you’ll be helping build that reputation.
Read more about apexanalytix - https://www.apexanalytix.com/about/
Quick Take
If you like infrastructure you can see all the way down, this is that role. apexanalytix runs its own private cloud: more than 50 Kubernetes clusters across two datacenters, every one of them declared in Git and reconciled by Flux, networked with Cilium's eBPF datapath, and backed by Ceph. Very little here is a black box — you can follow a request from a BGP-advertised service IP, through the eBPF datapath, to a pod running on a node you helped bring online.
As Senior Cloud Engineer you will own that platform. You will design, build, and upgrade clusters; run the networking and storage layers; operate the services everything else depends on — Harbor, Keycloak, OpenBao; and keep the entire fleet reproducible through GitOps. You will also work close to the metal, automating the path from a new server to a production Kubernetes node. It is a deep, hands-on engineering role with genuine ownership: you are the person who makes the platform work, and the final escalation point when it does not.
The stack you will work with
Kubernetes (kubeadm, Cluster API, Kamaji, AKS) · Cilium + Hubble · Rook-Ceph and cephadm · FluxCD, Kustomize, Helm · KubeVirt · Harbor · Keycloak · OpenBao · OpenTofu and Ansible · Prometheus, Thanos, Grafana, Loki, Tempo · AlmaLinux, Ubuntu, Flatcar Container Linux · Gitea Actions · Go, Python, Bash
The Work
GitOps and platform automation
- Run FluxCD as the only path to production — Kustomize layering, HelmReleases, and automated image updates — so that the fleet's desired state always lives in Git, not in someone's shell history.
- Author and maintain Helm charts and distribute them through OCI registries.
- Build infrastructure as code with OpenTofu/Terraform modules, Ansible roles, and cloud-init/Ignition templates.
- Design and maintain CI/CD pipelines (Gitea Actions) and the Linux and Windows runner fleet behind them.
- Automate the operational work that would otherwise be toil, in Go, Python, Bash, or PowerShell — favouring deterministic, re-runnable tooling over documented manual procedure.
- Bring change discipline to a large fleet: a written blast-radius assessment before production changes, canary-then-fleet rollout, and a rollback path that has been thought through in advance.
Kubernetes at fleet scale
- Design, build, upgrade, and operate Kubernetes across bare-metal (kubeadm), virtualised (KubeVirt with Cluster API), and managed cloud (AKS) footprints.
- Own the control plane: etcd backup, restore and defragmentation, API server tuning and audit logging, certificate rotation, and version upgrades under a clear support policy.
- Operate Cluster API providers, machine deployments, MachineHealthChecks, and Kamaji hosted control planes.
- Get workload topology right: PodDisruptionBudgets, anti-affinity and topology spread, requests and limits, autoscaling, and priority classes.
- Build multi-tenancy that holds: namespaces, RBAC, admission webhooks, and Pod Security standards.
- Diagnose the genuinely hard failures — kubelet and PLEG stalls, containerd faults, OOMKills, eviction storms, scheduler pressure — and turn each one into a runbook.
eBPF networking with Cilium
- Operate Cilium as the fleet CNI: eBPF datapath, kube-proxy replacement, and endpoint and identity debugging.
- Run the Cilium BGP Control Plane and LB-IPAM, advertising service IPs to the physical fabric and across clusters.
- Build and troubleshoot ClusterMesh multi-cluster service discovery and its datapath.
- Author CiliumNetworkPolicy at L3/L4/L7 and use Hubble to see east-west traffic as it actually flows.
- Take on the advanced work: pod CIDR planning and renumbering, dual-stack IPv6 conversion, VRF-based pod isolation, and Multus secondary networks.
- Own service exposure end to end — Gateway API and HTTPRoute, ingress, kube-vip, and CoreDNS tuning.
Software-defined storage with Ceph
- Operate Rook-Ceph and external cephadm clusters: OSD lifecycle, CRUSH and placement-group management, CephFS and RBD, rebalancing, and capacity planning.
- Run the full CSI driver set — ceph-csi, NFS, SMB, NVMe-TCP, KubeVirt CSI, Portworx, and Pure Storage — and know which one belongs on which workload.
- Manage StorageClasses, volume expansion, snapshots and clones, and RWO versus RWX topology decisions.
- Own backup and disaster recovery for stateful workloads with Velero, Kopia, and etcd snapshots — and prove it by restoring, not by reading a green job status.
- Investigate storage performance properly: PVC benchmarking, I/O latency analysis, and noisy-neighbour isolation on shared arrays.
Platform services and identity
- Own Harbor as the enterprise registry: projects and quotas, retention, cross-site replication, garbage collection, Trivy scanning, robot accounts, and high availability.
- Own Keycloak as the identity provider: realms, clients, scopes and mappers, OIDC for kubectl, Grafana and Headlamp, identity brokering and IdP federation, group-to-role mapping, and Infinispan multi-site caching.
- Run the secrets platform: OpenBao raft clusters, External Secrets Operator, rotation, and SSH certificate authority issuance.
- Automate certificates and supply-chain trust: cert-manager and Let's Encrypt, wildcard rotation, registry mirroring, and image signing and policy enforcement.
- Support the data and middleware tier the platform depends on: PostgreSQL, OceanBase, Valkey/Redis Sentinel, and RabbitMQ.
Observability and reliability engineering
- Operate Prometheus, Thanos, Grafana, Loki, Tempo, and Alloy, and author the dashboards and alert rules the team actually relies on.
- Take part in the on-call rotation, act as incident commander during platform outages, and write the root cause analysis afterwards.
- Design and test disaster recovery and failover — including the maintenance window, the rollback plan, and the report that follows.
- Own capacity planning and hardware and software end-of-life tracking, and lead OEM and vendor escalations.
- Work within a compliance-regulated environment (FedRAMP, SOC 2): segmentation, audit logging, and change evidence.
- Mentor engineers earlier in their careers, lead design reviews, and raise the team's operating standard.
Infrastructure foundations — from bare metal to node
- Automate the path from a new server to a production Kubernetes node so provisioning is repeatable rather than manual — PXE, kickstart, cloud-init, and Ignition.
- Provision and harden node operating systems: AlmaLinux 9, Ubuntu 24.04, Flatcar Container Linux, and Windows Server, including kernel and sysctl tuning, NUMA alignment, and container runtime configuration.
- Configure host networking and storage for new capacity: LACP bonding, VLAN trunks, nmstate policies, MTU consistency, LVM, multipath, and NVMe-TCP.
- Bring nodes into the cluster with kubeadm or Cluster API, then validate CNI, CSI, and workload readiness before promoting them to production.
- Partner with datacenter operations and remote hands on new hardware — commissioning, out-of-band management (iDRAC, iLO, IPMI, UCSM), and firmware and BIOS lifecycle — and confirm fabric readiness before rack-in.
- Retire and rotate nodes safely: HA-floor analysis, controlled drain, reimage, and return to the pool with no workload impact.
The Must-Haves
More than 8 years in infrastructure engineering, including at least 5 years running production Kubernetes at scale.- Deep Kubernetes knowledge: control-plane internals, etcd operations, cluster upgrades, RBAC and admission control, scheduling, and real failure diagnosis rather than restart-and-hope.
- A strong GitOps practice — FluxCD or ArgoCD with Kustomize and Helm — and the conviction that Git should be the only path to production.
- Substantial CNI experience with Cilium: eBPF datapath, BGP control plane, LB-IPAM, ClusterMesh, network policy, and Hubble. Experience with other CNIs is welcome, but Cilium is our standard.
- Substantial CSI and storage experience, ideally with Ceph (Rook or cephadm): OSD and CRUSH management, CephFS and RBD, snapshots, and performance troubleshooting.
- Production ownership of a container registry (Harbor or equivalent) and an identity provider (Keycloak or equivalent) — or the depth and appetite to take both on.
- Strong Linux fundamentals across RHEL/AlmaLinux, Ubuntu, and Flatcar: systemd, kernel and sysctl tuning, the storage and network stacks, and performance debugging.
- Comfort working close to the hardware — bare-metal provisioning, out-of-band management, and firmware lifecycle — and the instinct to automate it rather than repeat it.
- Automation skill in OpenTofu/Terraform, Ansible, and Bash, plus at least one of Python or Go.
- Hands-on experience with Cluster API and KubeVirt for automated cluster and virtual-machine provisioning.
- Working knowledge of datacenter networking — VLANs, LACP, BGP, VXLAN, MTU and jumbo frames — enough to work fluently alongside the network team.
- Experience with backup and disaster recovery tooling (Velero, Kopia, etcd snapshots), including having actually executed restores.
- Experience running observability at scale: Prometheus, Grafana, Thanos, Loki, and sensible alert tuning.
- Certified Kubernetes Administrator (CKA), or equivalent demonstrable depth; CKS or CKAD is a plus.
- A bachelor's degree in Computer Science, Information Technology, or equivalent practical experience.
- Clear writing: design documents, runbooks, and root cause analyses that a colleague can act on without asking you first.
- An ownership mindset, accountability for uptime, and the generosity to bring other engineers up with you.
Nice to have
- Experience building and operating bare-metal Kubernetes, not managed cloud only.
- Azure experience (AKS and hybrid connectivity); AWS or GCP also useful.
- Experience supporting Windows Server worker nodes in Kubernetes.
- Experience in a FedRAMP, SOC 2, or comparable regulated environment.
- Red Hat (RHCSA/RHCE), Ceph, Isovalent/Cilium, HashiCorp Vault, or Azure (AZ-104 / AZ-305) certification.
- Contributions to open-source infrastructure projects.
Over the years, we’ve discovered that the most effective and successful associates at apexanalytix are people who have a specific combination of values, skills, and behaviors that we call “The apex Way”. Read more about The apex Way - https://www.apexanalytix.com/careers/
Benefits
At apexanalytix we know that our associates are the reason behind our successes. We truly value you as an associate and part of our professional family. Our goal is to offer the very best benefits possible to you and your loved ones. When it comes to benefits, whether for yourself or your family the most important aspect is choice. And we get that. apexanalytix offers competitive benefits for the countries that we serve, in addition to our BeWell@apex initiative that encourages employees’ growth in six key wellness areas: Emotional, Physical, Community, Financial, Social, and Intelligence.
With resources such as a strong Mentor Program, Internal Training Portal, plus Education, Tuition, and Certification Assistance, we provide tools for our associates to grow and develop.
