**Job Title:** Senior Data Engineer (Data Delivery)
**Location:**Remote
**Project:** Compliance Store – Data Delivery Workstream (WS3)
**Domain:** Healthcare / Compliance Data Platform
### Job Summary
We are seeking an experienced Senior Data Engineer to join a high-impact Compliance Store Data Delivery team supporting a regulated healthcare environment. The ideal candidate will be responsible for designing, developing, and optimizing scalable data extraction, retrieval, and delivery solutions leveraging Snowflake, Amazon S3, Python, and modern data engineering practices.
This role requires expertise in large-scale batch processing, index-based data discovery, API-driven data access, microservices architecture, and compliance-focused data handling. Candidates with healthcare, insurance, or other regulated industry experience will be highly preferred.
### Key Responsibilities
* Design and develop scalable data extraction and delivery solutions using Snowflake, Amazon S3, and Python.
* Build and optimize data retrieval mechanisms for batch and API-driven delivery requests.
* Implement index-based data discovery strategies (L1/L2/L3 indexing models).
* Develop reusable microservices for data extraction, transformation, validation, and enrichment.
* Support both asynchronous batch workflows and synchronous API-based data access patterns.
* Optimize query performance, partitioning strategies, and extraction efficiency.
* Integrate extraction services with FastAPI-based Data Access APIs.
* Implement audit logging and tracking for search, extraction, and delivery activities.
* Collaborate with architects, API teams, compliance teams, and QA teams throughout the delivery lifecycle.
* Ensure compliance with data governance, security, and PII/PHI handling standards.
### Required Skills & Experience
* 7–10+ years of experience in Data Engineering or Data Platform Development.
* Strong hands-on experience with Snowflake, including:
* Query Optimization
* External Tables
* Iceberg Tables
* Experience working with Amazon S3 and Parquet file formats.
* Strong Python programming and pipeline development expertise.
* Experience building large-scale batch extraction and delivery pipelines.
* Strong understanding of:
* Indexing Strategies
* Partitioning
* Data Retrieval Architectures
* Experience with REST APIs and Data Access Services.
* Experience with Microservices Architecture and Containerized Deployments.
* Strong problem-solving and troubleshooting skills.
### Preferred Skills
* Experience with FastAPI or similar Python-based API frameworks.
* Knowledge of PII/PHI data handling and compliance requirements.
* Healthcare, Insurance, or other regulated industry experience.
* Familiarity with ECG or SFTP-based batch delivery mechanisms.
* Experience with Spark, Distributed Processing, or Serverless Architectures.
* Exposure to Audit Logging and Metadata-Driven Platforms.
* Experience working in Agile delivery environments.
### Key Stakeholder Interactions
* Solution Architects
* Backend/API Engineering Teams
* Security & Compliance Teams
* QA/Test Engineers
* Business & Compliance Stakeholders
### What We're Looking For
* Strong ownership mindset and ability to work independently.
* Excellent communication and stakeholder management skills.
* Experience delivering enterprise-grade data solutions in highly regulated environments.
* Ability to thrive in fast-paced, collaborative project teams.
Pay: ₹70,000.00 - ₹90,000.00 per month
Experience:
- Senior Data Engineer (Data Delivery): 7 years (Required)
Work Location: Remote
