Senior Azure Solution Architect/Engineering Lead
Core Skills: Azure Databricks, PySpark, Spark, SQL, Azure Synapse, Kafka, Big Data, Python, Java, Azure Data Factory (ADF)
Professional Summary
Senior Data Architect with 15+ years of experience designing, building, and scaling enterprise-grade data platforms. Proven expertise in cloud-native big data architectures on Azure, real-time and batch data processing, and high-performance analytics systems. Strong background in data modeling, distributed computing, and end-to-end data pipeline orchestration, with hands-on leadership in large-scale enterprise and mission-critical projects.
Technical Expertise
Cloud & Data Platforms
Azure Databricks (Lakehouse architecture, Delta Lake, Unity Catalog)
Azure Synapse Analytics (Dedicated & Serverless Pools)
Azure Data Factory (ADF) for ELT/ETL orchestration
Azure Data Lake Storage Gen2
Big Data & Distributed Processing
Apache Spark (Core, SQL, Structured Streaming)
PySpark performance tuning & optimization
Apache Kafka (real-time ingestion, streaming pipelines)
Large-scale batch and streaming data architectures
Programming & Query Languages
Python (advanced data engineering, automation, frameworks)
Java (Spark internals, Kafka consumers/producers, microservices)
SQL (complex analytics, query optimization, warehouse design)
Data Architecture & Design
Lakehouse, Lambda, and Kappa architectures
Dimensional modeling & data warehouse design
Data governance, security, and lineage
Scalability, fault tolerance, and cost optimization
- Technical depth in designing and building data platforms for high volume data processing with low latency
- Strong Expertise in Azure, Databricks, Python, Pyspark, SparkSQL, CI-CD (Jenkins, Github), Obs (Dynatrace),
- Good to have : Java, Kafka , Casandra, Microservices, API, Flink, ZeroMQ, Protocol Buffers
- Experience in recruiting , people leadership for managing team of 40+ , govern delivery and also act as technical mentor to set processes to bring innovation, goals to derive business outcomes
- Act as Module lead for India operations, liaise with NA teams to drive the objectives
Key Responsibilities & Achievements
Architected and delivered enterprise-scale Azure Lakehouse platforms supporting petabyte-scale data.
Designed real-time streaming solutions using Kafka and Spark Structured Streaming.
Led migration of on-premise data warehouses to Azure Synapse + Databricks , improving performance and reducing cost.
Built reusable PySpark frameworks for data ingestion, transformation, and validation.
Optimized Spark jobs through partitioning, caching, and memory tuning to achieve significant runtime improvements.
Implemented end-to-end ADF pipelines with CI/CD and parameter-driven orchestration.
Mentored senior and junior engineers; provided architectural guidance across multiple teams.
Collaborated with business stakeholders to translate analytics requirements into scalable technical designs.
Leadership & Soft Skills
Technical leadership and architectural decision-making
Cross-team collaboration and stakeholder communication
Design reviews, code quality standards, and best practices
Agile/Scrum execution in large enterprise environments
- Functional experience of capital markets, securities processing will be advantage
