Job Title: Senior Data Engineer
Experience: 1.5 to 4 Years
Job Summary
We are looking for a Data Engineer with 1.5 to 4 years of experience in designing, developing, and optimizing scalable data platforms, ETL/ELT pipelines, and cloud-native data solutions. The ideal candidate should have strong expertise in Snowflake, AWS, Python, SQL, Spark, workflow orchestration, and modern data engineering practices.
Key Responsibilities
- Design and develop scalable batch and real-time data pipelines.
- Build and optimize ETL/ELT pipelines using Python, SQL, DBT, Airflow, Luigi, Snowpipe, Fivetran, and Matillion.
- Design, implement, and optimize Snowflake data warehouse solutions, including migrations, schema design, performance tuning, RBAC, and data governance.
- Develop distributed data processing solutions using Apache Spark, Hadoop, Hive, and Kafka.
- Build cloud-native data platforms using AWS (S3, EC2, EMR, Glue, Lambda, RDS).
- Develop REST APIs using FastAPI and automate deployments using Terraform, Docker, Kubernetes, and CI/CD.
- Implement monitoring, logging, alerting, data quality, and security (OAuth, JWT, RBAC, Data Masking).
- Collaborate with cross-functional teams, mentor junior engineers, and deliver scalable, high-performance data solutions.
Required Skills
- Programming: Python, SQL, Linux Bash
- Data Warehouse: Snowflake, Snowpipe, Snowpark, BigQuery
- Cloud: AWS (EC2, S3, RDS, EMR, Glue, Lambda)
- Big Data: Apache Spark, Hadoop, Hive, Kafka, HBase
- ETL/Orchestration: DBT, Airflow, Luigi, Fivetran, Matillion
- Databases: PostgreSQL, MySQL, MongoDB, Cloud SQL, BigTable
- DevOps: Terraform, Docker, Kubernetes, Git, CI/CD
- Monitoring: Prometheus, Grafana, Cloud Monitoring
- APIs & Security: FastAPI, REST APIs, OAuth 2.0, JWT, RBAC, Data Masking
- Good to Have: Exposure to Generative AI, LangChain, OpenAI, and RAG.
Job Types: Full-time, Permanent
Pay: ₹400,000.00 - ₹700,000.00 per year
Benefits:
- Flexible schedule
- Provident Fund
- Work from home
Work Location: In person
