Job Description: Data Engineer (Agentic AI – AWS)- 6+ Years.
Role: Data Engineer (Agentic AI – AWS)
Experience: 6+ Years
Work Hours: 11:00 to 8:00 pm IST
Duration: 6 months (Extension possible)
Key Responsibilities
· Design, build, and maintain ETL/ELT pipelines for structured and unstructured data in AWS.
· Work with AWS services (e.g., S3, Glue, EMR, Lambda, Kinesis, Step Functions, Redshift, Athena, DynamoDB, RDS) to manage data ingestion, transformation, and storage.
· Ensure data quality, governance, and security across the full lifecycle.
· Enable real-time and batch data processing to support AI-driven workflows.
· Collaborate with AI/ML teams to prepare datasets for model training, inference, and fine-tuning.
· Optimize data infrastructure for scalability, cost-efficiency, and performance.
· Implement CI/CD pipelines and best practices for data engineering in a cloud-native environment.
· Monitor and troubleshoot data pipelines to ensure high availability and reliability.
Required Skills & Qualifications
· 3–7 years of experience in data engineering (adjust based on seniority).
· Strong programming skills in Python, SQL, and/or Scala/Java.
· Expertise in AWS cloud ecosystem (S3, Glue, EMR, Redshift, Athena, Kinesis, Lambda, etc.).
· Experience with data pipeline orchestration tools (Airflow, Step Functions, Dagster, or similar).
· Proficiency with big data frameworks (Spark, Hadoop, or Flink).
· Familiarity with data modeling, warehousing, and schema design.
· Solid understanding of data governance, lineage, and security (IAM, Lake Formation, encryption).
· Experience with real-time streaming data (Kafka, Kinesis, or equivalent).
· Knowledge of DevOps practices (Terraform/CloudFormation, CI/CD, Git, Docker).
Pay: ₹90,000.00 - ₹130,000.00 per month
Work Location: Remote
