Job Description
We are seeking a highly skilled and hands-on AI Infrastructure Engineer to design, build, and maintain our next-generation, self-hosted AI infrastructure. In this role, you will bridge the gap between enterprise infrastructure and advanced AI engineering. You will be responsible for deploying, optimizing, and orchestrating multi-model Large Language Models (LLMs), Vision-Language Models (VLMs), and Agentic AI frameworks on both raw bare-metal hardware and enterprise container platforms.
The ideal candidate thrives in a high-performance computing environment, understands the nuances of bare-metal GPU clustering, and has a proven track record of moving open-source AI stacks from experimental setups into secure, scalable production environments.
Key Responsibilities
1. AI Infrastructure & Deployment Orchestration
- Design, provision, and manage high-performance compute clusters on bare-metal (Rocky Linux / Ubuntu) and enterprise platforms (Red Hat OpenShift / Kubernetes).
- Deploy, optimize, and maintain scalable LLM/VLM serving engines using vLLM and Ollama, ensuring low-latency and high-throughput inference.
- Configure and scale distributed training and inference workloads using Ray framework, specifically optimizing for multi-GPU, multi-modal applications (Text, Image, and Audio).
- Set up, secure, and customize enterprise-grade user interfaces like Open WebUI for cross-functional team access.
2. Advanced RAG & Model Lifecycle Management
- Architect and maintain end-to-end Retrieval-Augmented Generation (RAG) pipelines with advanced context management, chunking strategies, and hybrid search capabilities.
- Integrate and tune vector databases (e.g., Milvus, Qdrant, PGVector) to support high-context RAG operations.
- Establish reproducible pipelines for model fine-tuning (PEFT, LoRA, QLoRA) on private datasets, optimizing memory efficiency and training times.
3. Agentic AI & Autonomic Systems
- Design and deploy scalable Agentic AI frameworks (e.g., CrewAI, AutoGen, LangGraph) capable of executing multi-step complex workflows, tool use, and self-correction.
- Develop custom agent "skills", API integrations, and secure sandboxed execution environments for autonomous agents.
Required Technical Skills
Core AI & MLOps Stack
- Inference Engines: vLLM (including tensor-parallel/pipeline-parallel setups), Ollama, Hugging Face TGI.
- Distributed Computing: Ray (Ray Train, Ray Serve, Ray Core).
- Frameworks & UIs: Open WebUI, LangChain, LangGraph, CrewAI, AutoGen.
- Data & RAG: Advanced RAG architecture, semantic chunking, metadata filtering, and vector database administration.
- Fine-Tuning: DeepSpeed, Axolotl, or standard PyTorch-based fine-tuning workflows utilizing LoRA/QLoRA.
Infrastructure & System Engineering
- Orchestration: Advanced Kubernetes administration, Red Hat OpenShift, Helm charting, and K8s operators (specifically NVIDIA GPU Operator).
- Operating Systems: Expert-level administration of Linux distributions (Rocky Linux / RHEL, Ubuntu Server).
- Hardware & Networking: Deep understanding of bare-metal GPU nodes (NVIDIA H100, L40S, A100), PCIe topologies, NVLink, and high-speed networking (InfiniBand/RoCE, SR-IOV).
- Containers: Docker, Podman, and secure container runtimes (CRI-O, containerd).
Pay: ₹15,000.00 - ₹40,000.00 per month
Benefits:
Work Location: In person
