Skip to content

Data Lead (Databricks & Power BI) | Offshore

Photon -
India

Apply Now

Job details

Qualifications

Power BI
Data modeling
Cloud infrastructure
Azure
Spark
R
Data analysis skills
Git
SQL
AWS
Quality assurance
Machine learning
Data management
APIs
Data visualization
AI
Data warehouse
Python

Full job description

Responsibilities
API Integration & Data Extraction

Lead the integration of Fenergo APIs to extract relevant KYC and AML data, ensuring seamless connectivity and data flow between systems

Design, develop, and maintain scalable data pipelines and ETL processes to support data ingestion from various sources, including databases, APIs, and flat files

Ensure robust data extraction processes that maintain data quality and compliance with regulatory requirements

Data Processing & Pipeline Development

Utilize Databricks and Apache Spark to design and implement robust data processing pipelines, ensuring high data quality and performance

Work with DataFrames for transforming data and implementing the Medallion Architecture

Execute SQL queries for data extraction, manipulation, and complex data operations

Join datasets and add fields to reports to provide comprehensive analytical insights

Leverage AI tools in Databricks to assist with data workflows and optimization

Use notebooks as data transformation pipelines for efficient data processing

Data Analysis & Interpretation

Analyze and interpret complex data sets to identify trends, patterns, and anomalies that can inform business decisions related to client and investor lifecycle management

Understand and navigate the data model to ensure accurate data representation and reporting

Conduct regular data quality assessments and audits to ensure data integrity and compliance with industry standards

Perform root cause analysis to swiftly identify data issues and collaborate with relevant teams to implement effective solutions

Data Visualisation & Reporting

Architect and develop interactive dashboards and reports in Power BI, translating complex data into clear, actionable insights for clients, leadership, and stakeholders

Develop and maintain dashboards and reports to provide insights into key performance indicators (KPIs) and operational metrics for KYC/AML processes

Create visual representations that highlight critical data points for regular reporting to clients and senior management

Ensure reports meet the needs of both technical and non-technical stakeholders

Collaboration & Stakeholder Management

Collaborate with cross-functional teams, including IT, Compliance, Risk Management, business analysts, and senior management, to gather data requirements and deliver strategic insights

Engage with clients and internal stakeholders to understand their reporting needs and ensure alignment with business objectives

Work closely with KYC/AML operations teams to ensure data solutions support compliance and regulatory requirements

Act as a bridge between technical teams and business users, translating complex data concepts into actionable business insights

Documentation & Governance

Maintain comprehensive documentation of data processes, API integrations, data flows, data management processes, and reporting solutions for future reference and compliance

Document data governance practices and ensure adherence to data quality best practices

Ensure all data handling complies with regulatory standards and internal policies

Continuous Improvement & Problem-Solving

Recommend long-term product solutions to enhance data quality, accessibility, and usability

Identify opportunities for process optimization and automation in data workflows

Stay up-to-date with industry trends and best practices in data engineering, analysis, and management

Proactively identify and resolve data-related issues, ensuring timely and accurate reporting

Demonstrate creativity and insightfulness in developing dynamic approaches to complex data challenges

Quality Assurance

Ensure data integrity throughout all pipelines and reporting mechanisms

Implement data validation and quality control measures

Monitor data processes and implement control mechanisms to ensure reliability

Skills

Core Data Engineering Skills (Required):

Proficiency in Databricks and Apache Spark for data processing and pipeline development

Strong knowledge of Power BI for data visualization and reporting, with ability to create executive-level dashboards

Expert-level proficiency in SQL for data querying, manipulation, and complex analytical operations

Experience with programming languages such as Python or R for data analysis and automation

Strong understanding of data warehousing concepts and ETL processes

Knowledge of data modeling concepts and best practices for data management

Understanding of the Medallion Architecture and data lakehouse principles

Experience working with DataFrames for data transformation

Ability to leverage AI tools in Databricks to optimize data workflows

API & Integration Skills (Required):

Strong experience in API integration for data extraction and system connectivity

Ability to ensure seamless data flow between multiple systems

Cloud & Infrastructure (Required):

Experience with cloud platforms, particularly Azure or AWS

Knowledge of Git connection to Databricks for version control

Experience with AWS/Azure and Databricks integration/mounting

Understanding of data governance and data quality best practices

Additional Technical Skills (Preferred):

Databricks administration skills

Familiarity with machine learning concepts and their application in data analysis

Experience with graph data models

Understanding of data governance and compliance standards (GDPR, AML regulations, etc.)

Knowledge of secure data handling practices

Apply Now