Role Summary
We're looking for a Principal Search/AI Engineer to join our team of search experts and serve as technical lead on large-scale search modernization projects and programs for Fortune-1000 clients. You will collaborate with our established search practice to design, build, and optimize AI-powered search platforms that deliver rich content and product discovery experiences. In this role, you'll drive technical excellence across multi-year digital transformation initiatives, lead solution architecture, and mentor engineering teams while implementing cutting-edge search solutions enhanced with AI and agentic capabilities for our retail, e-commerce, and enterprise clients.
Key Responsibilities
Search Platform Leadership
- Lead technical architecture and strategy for complex search modernization programs spanning multiple years and teams
- Drive end-to-end ownership of Lucene-based search platforms (Solr/Elasticsearch/OpenSearch) including schema design, index pipelines, scaling, and monitoring
- Architect high-throughput, low-latency Java services that expose search functionality via REST/gRPC APIs
- Design and implement advanced search relevance strategies including semantic search, hybrid approaches, and AI-enhanced ranking
- Perform sophisticated relevance tuning leveraging BM25, TF-IDF, vector similarity, and learning-to-rank methodologies
AI and Agentic Search Innovation
- Prototype and productionize next-generation search features including faceted navigation, autocomplete/type-ahead, spell-checking, vector search, and hybrid retrieval
- Lead integration of AI-powered conversational search, intelligent query understanding, and agentic workflows to enable natural-language and multi-turn search experiences
- Architect and implement agentic AI frameworks (LangGraph, LangChain) for multi-agent search workflows and intelligent content discovery
Technical Leadership and Collaboration
- Partner with data scientists and ML engineers to deploy and optimize PyTorch-based ranking models for query understanding, embeddings, and re-ranking
- Lead design and implementation of data pipelines (Python/Java/Spark/Dataflow) to generate training signals from user behavior analytics
- Mentor and guide engineering teams on search best practices, performance optimization, and emerging AI technologies
- Drive technical decision-making across cross-functional teams and stakeholder groups
Quality and Operational Excellence
- Establish comprehensive metrics, alerting, and automated testing frameworks to guarantee SLA compliance at enterprise scale
- Lead performance profiling and capacity planning for indexes serving hundreds of millions of documents or SKUs
- Drive continuous improvement initiatives for search relevance, performance, and user experience metrics
Required Qualifications
- 8+ years building enterprise-scale back-end systems in Java (Spring Boot or similar) with demonstrated focus on performance, scalability, and reliability
- 5+ years hands-on leadership experience with Lucene-based search engines (Apache Solr, Elasticsearch, or OpenSearch) including advanced index design, relevance tuning, and cluster operations
- 3+ years’ experience as technical lead on large-scale search or platform modernization programs for enterprise clients
- Proven track record delivering AI-enhanced search solutions handling 100M+ documents/SKUs at Fortune-1000 scale (e-commerce, media, enterprise content, or similar)
- Deep understanding of information retrieval concepts including tokenization, analyzers, BM25, TF-IDF, vector similarity, semantic search, and machine learning integration
- Strong proficiency in Python for data manipulation, ML model integration, and prototyping
- Extensive experience with cloud-native architectures, Docker/Kubernetes, CI/CD, and enterprise deployment (AWS, GCP, or Azure)
- Demonstrated leadership in mentoring engineers and driving technical excellence across teams
Preferred / Bonus Skills
- Expert-level understanding of Lucene-based search engines and their underlying architecture, performance characteristics, and optimization strategies
- Hands-on experience with modern search platform ecosystem including Algolia, Coveo, Lucidworks, Constructor.io, or Vertex AI Search
- Proven experience implementing agentic AI frameworks (LangGraph, LangChain) and orchestrating multi-agent search workflows
- Advanced experience embedding and serving PyTorch models for Learning-to-Rank, embeddings, conversational AI, and generative re-ranking
- Deep knowledge of vector databases and similarity search (OpenSearch/ES K-NN, Milvus, Pinecone, Weaviate, Chroma)
- Experience designing streaming data pipelines (Apache Kafka, Pub/Sub, Apache Flink) for real-time indexing and personalization
- Active contributions to open-source information retrieval/ML projects, published research, or recognized technical thought leadership
- Experience with cloud-native search services (AWS CloudSearch, GCP Vertex AI Search, Azure Cognitive Search) and hybrid architectures
Background in natural language processing, transformer models, and large language model integration
