Designation: Senior Analyst
Level: L2
Experience: 4 to 6 years
Location: Chennai, TamilNadu, India.
Job Description
We are seeking a Vector Developer to design, implement, and optimize high-performance data pipelines using Vector. The role involves migrating and managing log/metric pipelines from platforms like Splunk, ensuring efficient data collection, transformation, and routing to downstream systems such as Datadog or other storage/analytics platforms.
The ideal candidate will have strong experience in observability systems, log processing, and pipeline optimization, with a focus on reliability, performance, and cost efficiency.
Key Responsibilities
- Design and build data pipelines using Vector (sources transforms sinks)
- Migrate existing pipelines from Splunk or similar tools to Vector
- Implement data transformations using Vector Remap Language
- Filter, enrich, and route logs/metrics efficiently to downstream systems
- Handle edge cases such as: Oversized records, Malformed data, High-throughput ingestion scenarios
- Optimize pipelines for: Performance (low latency, high throughput), Cost (reduce unnecessary log ingestion)
- Configure buffering, retries, and backpressure handling for reliability
- Monitor and troubleshoot pipeline failures and data inconsistencies
- Collaborate with data engineering and platform teams to enforce data contracts
- Good to have worked in Rust and any cybersecurity domain
