AI Inference & Engine Optimization Intern (vLLM / PyTorch/Python 3.11+)
About Siellium Global Labs
Siellium Global Labs Private Limited is a startup scaling a zero-retention infrastructure layer across sovereign enterprise networks. We are engineering a local-first execution layer that utilizes a dynamic Mixture-of-Agents (MoA) orchestration framework.
The Role:
We are looking for a highly capable AI Inference & Optimization Intern to build and manage our local inference pipelines. This is a pure backend infrastructure role. You will not be building standard UI wrappers; you will be directly manipulating GPU memory, optimizing token throughput, and deploying a high-performance Mixture-of-Agents (MoA) execution layer.
Your mission is to squeeze maximum computational power out of our hardware. You will be responsible for pinning base models into VRAM and executing the mathematical compression needed to allow multiple domain-specific expert models to swap in and out seamlessly on a per-request basis.
Core Responsibilities
- Engine Configuration & Deployment: Stand up and operate the core execution engine using vLLM, explicitly utilizing custom PagedAttention layers to maximize GPU concurrency and throughput.
- Memory Optimization: Execute mathematical compression on neural networks to fit them into constrained hardware budgets. You will apply structural matrix pruning algorithms and execute Activation-Aware Weight Quantization (AWQ) to compress 16-bit floating-point weights down to 4-bit and 8 bit Integers.
- Dynamic Adapter Swapping/Orchestration: Pin a foundational base model (Llama-3-8B) securely into VRAM. Configure the asynchronous, dynamic hot-swapping of specialized Low-Rank Adapters (LoRA) for domain experts like DeepSeek-Coder (for syntax) and Phi-4 (for logic)
- Continuous Batching: Tune the inference server to handle high-volume asynchronous requests, preventing memory bottlenecks during multi-model orchestration loops.
- Pipeline Integration: Ensure seamless integration between the VRAM-optimized execution layer and the high-concurrency Python/FastAPI routing gateway and fine tune the AI model.
- Concurrency Optimization: Maximize hardware TFLOPS utilization by tuning continuous batching parameters and managing non-blocking event loops for parallel multi-model execution
- VRAM Compression: Execute structural matrix pruning algorithms to minimize memory overhead without degrading logical accuracy.
Preferred Qualifications
- Quantization: Apply Activation-Aware Weight Quantization (AWQ) to compress massive 16-bit floating-point weights down to 4-bit integers to fit within constrained hardware budgets.
Required Technical Stack
- Python (3.11+) and advanced asynchronous programming (asyncio).
- Deep familiarity with PyTorch and the Hugging Face ecosystem.
- Strong understanding of vLLM and how continuous batching and PagedAttention function at a system level.
- Working knowledge of Parameter-Efficient Fine-Tuning (PEFT / LoRA) and how adapter weights interact with base models.
- Clear comprehension of GPU VRAM allocation, CUDA mechanics, and quantization formats (AWQ, GPTQ, or GGUF)
College Student enthusiastic for AI & ML are preferred
- Experience running or fine-tuning open-weights (Llama, Qwen, DeepSeek) on local hardware or cloud GPU instances (Vast.ai, RunPod, AWS).
- Familiarity with structured pruning libraries or research implementations (e.g., SparseML, Wanda).
- Experience exposing backend inference engines via high-concurrency frameworks like FastAPI.
What This Role Offers
This position provides deep exposure to low-level AI engineering. You will gain hands-on experience building, compressing, and deploying production-grade AI clusters from scratch, with complete ownership over the underlying infrastructure and execution layer.
Pay: ₹2,000.00 - ₹4,000.00 per month
Benefits:
- Provident Fund
- Work from home
Work Location: Remote
