Project Role : Custom Software Engineer
Project Role Description : Lead the effort to design, build and configure applications, acting as the primary point of contact.
Must have skills : Data Engineering
Good to have skills : NA
Minimum 5 year(s) of experience is required
Educational Qualification : 15 years full time education
Summary:
Mandatory Requirement
1. Location – Bengaluru
2. Work from client office (RMZ Ecoworld) 4 days – every Monday WFH
3. Shift B – 1:30 to 10:30 PM IST
Experienced Data Engineer with hands-on expertise in building automated data ingestion pipelines using Azure technologies. Proficient in end-to-end design, development, deployment, and monitoring of data workflows aligned with analytics and AI-driven use cases.
Roles and Responsibilities:
Build automated data ingestion pipelines from Azure Data Lake Storage (ADLS) and Azure SQL.
Develop and maintain Azure Data Factory (ADF) pipelines with linked services, triggers, and parameterization.
Containerize ETL workflows using Azure Container Apps for scalability and portability.
Configure scheduling and message-based execution using Azure Service Bus.
Utilize Ansible playbooks for infrastructure provisioning and ensuring deployment consistency.
Maintain Redis cache for managing intermediate LLM (Large Language Model) processing states.
Implement monitoring, alerting, and logging through Azure Application Insights.
Collaborate with analytics and AI teams to align ingestion schemas and infrastructure with downstream workflows.
Technical Experience & Professional Attributes:
Strong knowledge of Azure Data Services (ADF, ADLS, Azure SQL, Azure Service Bus, Application Insights).
Experience with containerization and CI/CD using Azure Container Apps and Ansible.
Solid understanding of ETL design principles, data orchestration, and workflow automation.
Familiarity with caching mechanisms (Redis) and their use in AI/ML pipelines.
Excellent problem-solving, debugging, and analytical skills.
Strong communication and collaboration skills with cross-functional teams.
Educational Qualifications:
Bachelor s degree in Computer Science, Information Technology, or a related field.
(Optional: Add certifications such as Microsoft Certified: Azure Data Engineer Associate if applicable.)
Additional Information:
Available for immediate or planned deployment based on project timelines.
Committed to delivering high-quality, scalable, and efficient data solutions.
Primary location preference: Bangalore.
15 years full time education
