Architect Data Products(M/F/D)

BASF Digital Solutions Private Limited -
Andhra Pradesh

Apply Now

Job details

Permanent

Qualifications

BCS
CI/CD
Power BI
Azure
Computer Science
Software troubleshooting
Big data
R
PaaS
Git
English
Bachelor's degree
Continuous integration
Scripting
Unity
GitHub
APIs
Agile
Metadata
Communication skills
Python
Information Technology

Full job description

Objective of the position

Describe your Product Mission here / objective of the role:

At our unit “Data Foundation - Big Data Management” we aim to offer organizations a robust and scalable solution for managing and deriving insights from vast quantities of data. By utilizing Azure's PaaS components, our platform streamlines the deployment and handling of Big Data workloads, empowering our clients to make data-driven decisions and propel business expansion. Our team is accountable for the creation and upkeep of the Big Data platform built on Azure PaaS components EDL (Enterprise Data Lake). We collaborate intensively with stakeholders to comprehend their needs and devise solutions that fulfill their expectations. The team is also in charge of guaranteeing the platform's scalability, dependability, and security, and for staying abreast with the newest technologies and trends in the big data domain

About the Job:

We are seeking a highly motivated and detail-oriented candidate who will be responsible for all data engineering aspects of the data product layer of our Enterprise Datalake, working with Azure and Databricks, maintaining/developing our framework and CI/CD pipelines and processes in an ambitious team.

Main Task

Develop, extend and optimize the data product layer of our Enterprise Datalake with high code standards, tests and suitable documentation to meet both user and internal needs.

Collaborate with the product owner and architect to understand data requirements and provide suitable solutions.

Troubleshoot data and software related issues and optimize pipelines for performance and scalability.

Work with Azure Databricks, Azure Data Factory/Synapse keeping up to date with the latest features and suggest their implementation if beneficial.

Demonstrate fluent communication skills in English (spoken and written)

Job Requirements

T e c hn i c a l & P r o f e ss i o n a l Knowledge (mandatory)

Experienc e in Agile way of working with a DevOps mindset

In-depth know how with Lakehouse concepts , Databricks and Unity Catalog .

Strong data engineering and development skills to maintain and extend the data product layer of our datalake

Strong knowledge in automation & scripting with tools like Github actions , Azure DevOps pipelines, Azure Data Factory, etc.

T e c hn i c a l & P r o f e ss i o n a l Knowledge ( additional plus/nice to have skills)

Experience with Azure Datafactory /Synapse data flows and pipelines, which are the basis of a legacy application for data products

Ideally, automating code generation and code management with the help of AI tools

PowerBI as one of the main tools of our users

Integration knowledge , how metadata can be exchanged with other applications via API s e.g. with Colibra datacatalog or an inhouse solution for data product definition

Ideally, backed up by vendor certifications (e.g. Microsoft, Databricks ).

Team player with strong interpersonal, written, and verbal communication skills

Edu c a t i o n

Bachelor's degree in computer science, Information Technology, Engineering, Business, or related fields.

Work experience

Minimum 7-8 years related working experience as a Dataengineer using Azure, Databricks, Python, Azure Data Factory or Synapse

Apply Now

Jobseeker tools

Employer Tools

Browse

Stay Connected