We're looking for someone genuinely strong on Google Cloud Platform - not necessarily years of work experience, but real depth: you've built things on GCP, you understand its IAM, quota, and networking model cold, and you can reason about Cloud Run, Vertex AI, and multi-project setups from first principles. You'll design and build Zeqo's multi-provider LLM router - the internal system that decides which model/provider handles each request (Google Gemini, OpenAI, Anthropic, and self-hosted open-weight models), optimizes for cost/latency/quality, and aggressively reduces redundant token spend through prompt caching and smart request management. This is a foundational infrastructure role: get it right and every downstream product gets faster and cheaper automatically.
This is intentionally open to early-career engineers — we care far more about depth of GCP knowledge and evidence of things you've actually built than a resume with years on it.
Location: Fully Remote
What You'll Do
- Design and build a unified LLM router that abstracts Google, OpenAI, Anthropic, and self-hosted model endpoints behind a single internal API, so product teams never hardcode a provider.
- Implement intelligent routing logic — route by task type, cost ceiling, latency budget, context length, or model capability (e.g., OCR evaluation vs. conversational tutoring vs. embedding generation).
- Build fallback and failover chains so a rate limit, timeout, or outage on one provider automatically retries on another without breaking the user experience.
- Design prompt/response caching (semantic and exact-match) to cut redundant token usage and reduce rate-limit pressure — particularly for repeated question-bank queries, common doubt patterns in Mophy, and OCR grading rubrics.
- Own rate-limit and quota management across providers and GCP projects, including request queuing, backpressure, and graceful degradation under load.
- Build cost observability: per-request cost attribution, per-product/per-school cost dashboards, and alerting on spend anomalies.
- Benchmark providers and models on cost, latency, and quality for Zeqo's specific workloads (OCR grading, RAG retrieval via Mophy, test generation) and recommend routing policy changes.
- Collaborate with the ML/infra work already in flight — our self-hosted BGE-large-en-v1.5 embedding service on Cloud Run, GCP multi-project quota isolation, and DPO training data pipelines — and integrate the router cleanly into that architecture.
What We're Looking For
- 0–2 years of experience — this is deliberately open to freshers and early-career engineers. What matters is depth, not tenure.
- Real GCP depth, demonstrated through projects, coursework, certifications, or personal builds: Cloud Run, IAM, service accounts, multi-project/quota architecture, Vertex AI. A GCP certification (Associate Cloud Engineer or Professional Cloud Architect) is a strong plus but not required if you can show equivalent hands-on work.
- Solid backend fundamentals — API design, working with external APIs, basic distributed-systems thinking (retries, timeouts, rate limits).
- Comfort with or eagerness to learn caching strategies (Redis or similar); exposure to LLM APIs (OpenAI, Gemini, Anthropic) is a plus but not a hard requirement — we'll teach you the LLM-specific parts.
- A cost-conscious, first-principles mindset — you enjoy understanding why a system is expensive or slow, not just making it work.
- Startup mindset: comfortable figuring things out with a small team, moving fast, and owning a system end-to-end even without a large playbook to follow.
- Bonus: personal projects or coursework touching RAG, embeddings, or prompt engineering.
To Apply
Send your resume and any GCP-related projects (GitHub, coursework, certifications, personal builds — anything that shows depth) to [email protected] or WhatsApp +91-9136734595. We'd rather see a GCP project you built and can explain in detail than a polished resume with no substance behind it.
Along with your resume, please answer these three questions:
- Describe a GCP project or setup you've built (personal, academic, or professional). What service(s) did you use, and what was the trickiest problem you had to solve?
- If an API you're calling suddenly starts rate-limiting you mid-request, what's your first instinct for handling it? (No wrong answers — we want to see how you think about this.)
- What's one thing about cloud costs (GCP or otherwise) that you think most engineers get wrong or don't think about enough?
Pay: ₹20,000.00 - ₹35,000.00 per month
Benefits:
- Flexible schedule
- Work from home
Work Location: Remote
