Roles and Responsibilities-
● Design and develop scalable data pipelines and workflows for data ingestion, transformation, and integration.
● Build and maintain data storage systems, including data warehouses, data lakes, and
relational databases.
● Ensure data accuracy, integrity, and consistency through validation and quality
assurance processes.
● Collaborate with data scientists, analysts, and business teams to understand data
needs and deliver tailored solutions.
● Optimize database performance and manage large-scale datasets for efficient
processing.
● Leverage cloud platforms (AWS, Azure, or GCP) and big data technologies
(Hadoop, Spark, Kafka) for building robust data solutions.
● Automate and monitor data workflows using orchestration frameworks such as
Apache Airflow.
● Implement and enforce data governance policies to ensure compliance and data
security.
● Troubleshoot and resolve data-related issues to maintain seamless operations.
● Stay updated on emerging tools, technologies, and trends in data engineering.
Skills and Knowledge-
1. Core Skills:
● Proficient in Python (libraries: Pandas, NumPy) and SQL.
● Knowledge of data modeling techniques, including:
○ Entity-Relationship (ER) Diagrams
○ Dimensional Modeling
○ Data Normalization
● Familiarity with ETL processes and tools like:
○ Azure Data Factory (ADF)
○ SSIS (SQL Server Integration Services)
2. Cloud Expertise:
● AWS Services: Glue, Redshift, Lambda, EKS, RDS, Athena
● Azure Services: Databricks, Key Vault, ADLS Gen2, ADF, Azure SQL
● Snowflake
3. Big Data and Workflow Automation:
● Hands-on experience with big data technologies like Hadoop, Spark, and Kafka.
● Experience with workflow automation tools like Apache Airflow (or similar).
Qualifications and Requirements-
● Education: Bachelor’s degree (or equivalent) in Computer Science, Information
Technology, Engineering, or a related field.
● Experience:
○ Freshers with strong understanding, internships and relevant academic
projects are welcome.
○ 2–3 years of experience working with Python, SQL, and data integration or
visualization tools is preferred.
● Other Skills:
○ Strong communication skills, especially the ability to explain technical
concepts to non-technical stakeholders.
○ Ability to work in a dynamic, research-oriented team with concurrent projects.
Pay: ₹20,000.00 - ₹50,000.00 per month
Benefits:
Work Location: In person
