About the Company: Leading technology firm
Experience:5 years
Location :Hyderabad
Job Title: Data Engineer
Roles and Responsibilities:
- Data Engineer builds and maintains systems that collect, store, and process large volumes of data
- Designs medallions data architectures like Lakehouse with Bronze, Silver, Gold layers for scalable data processing.
- Uses Apache Spark to process large datasets efficiently
- Experience with Azure/AWS/GCP data platforms
- Optimizes data pipelines for speed, performance, and cost
- Combines data from multiple sources into clean, consistent formats.
- Develops batch and real-time (streaming) data pipelines
- Implements incremental data processing and data versioning
- Automates workflows using CI/CD tools and DevOps practices
- Monitors pipelines with logging, alerts, and error handling
- Works with cloud platforms and orchestration tools to manage data systems
Education:Bachelor's Degree
