Tamil Nadu Quality catalog Jobs – 125 Open Jobs Today

JD19 - Consultant (DBT)
GainInsights —Chennai, Tamil Nadu
- Azure
- Computer Science
- SQL
Custom Software Engineer
Accenture —Chennai, Tamil Nadu
- Full-time
- Azure
- Computer Science
- Java
8d
Graphic Designer
Wedtree eStore Private Limited —T Nagar, Chennai, Tamil Nadu
- ₹15,000 - ₹25,000 a month
- Full-time
- In-person
- Health insurance
- Food provided
Quick apply
14d
Senior AWS Data Engineer
Agilisium —Saidapet, Chennai, Tamil Nadu
- Full-time
- DevOps
- SQL
- AWS
Senior Data Engineer
WPP —Chennai, Tamil Nadu
- Hybrid work
- SQL
- APIs
- AI
10d
Digital Content Associate, Prime Video Publishing Operations
ADCI MAA 15 SEZ - K20 —Chennai, Tamil Nadu
- Full-time
- SQL
- Bachelor's degree
- Communication skills
Quality - Catalog
Saaki Argus & Averil Consulting —Chennai, Tamil Nadu
Quick apply
Videography Intern
PicklemonkMedia —Chennai, Tamil Nadu
- Full-time
- In-person
- Diploma
Quick apply
ServiceNow Senior AI Developer
InfoBeans —Chennai, Tamil Nadu
- Azure
- Customer service
- DevOps
18h
Corel draw
Vanitha Fireworks Industries Pvt Ltd., —Sivakasi, Tamil Nadu
- ₹18,000 - ₹20,000 a month
- Full-time
- Sales
- Customer service
- In-person
Quick apply
Data Architect, Common Data Products
Kone —Chennai, Tamil Nadu
- Full-time
- Computer Science
- Engineering
- English
13d
Enterprise Data Migration Architect
RCG Global Services —Chennai, Tamil Nadu
- Azure
- Oracle
- Databases
Quick apply
Lead Software Engineer, Data Engineering
ValGenesis —Chennai, Tamil Nadu
- Full-time
- Azure
- DevOps
- Java
Quick apply
Cloud & Data Platform Architect
Guidehouse —Chennai, Tamil Nadu
- Azure
- Computer Science
- AWS
Senior Technical Product Owner
Toppan Merrill —Chennai, Tamil Nadu
- Full-time
- APIs
- Agile
- Communication skills
2d
Senior Engineer - Mechanical Design
Hubbell Incorporated —Chennai, Tamil Nadu
- AutoCAD
- Assembly
- Electrical experience
Manager, Site Merchandizing , RBS Retail Efficiency
ADCI - BLR 14 SEZ —Chennai, Tamil Nadu
- Full-time
- Microsoft Excel
- Marketing
Engineer II - Mechanical Design
Hubbell Incorporated —Chennai, Tamil Nadu
- AutoCAD
- Assembly
- Electrical experience
Senior Software Engineer, Data Engineering
ValGenesis —Chennai, Tamil Nadu
- Full-time
- Azure
- DevOps
- Java
Quick apply
Lead Functional Consultant - SAP MDG
EPAM Systems, Inc. —Coimbatore, Tamil Nadu
- APIs
- Health insurance
- Paid time off

I want to receive the latest job alerts for quality catalog in tamil nadu

By signing in to your account, you agree to SimplyHired's Terms of Service and consent to our Cookie and Privacy Policy.

Explore jobs in more locations

quality catalog jobs near tamil nadu

Software Engineer jobs in Chennai, Tamil Nadu

Mechanical Designer jobs in Chennai, Tamil Nadu

Enterprise Data Architect jobs in Chennai, Tamil Nadu

Technical Consultant jobs in Coimbatore, Tamil Nadu

Graphic Designer jobs in T Nagar, Chennai, Tamil Nadu

Data Engineer jobs in Chennai, Tamil Nadu

Data Architect jobs in Chennai, Tamil Nadu

Site Manager jobs in Chennai, Tamil Nadu

Servicenow Developer jobs in Chennai, Tamil Nadu

Senior Software Engineer jobs in Chennai, Tamil Nadu

Software Architect jobs in Chennai, Tamil Nadu

Senior Accountant jobs in Vadodara, Gujarat

Associate Product Manager jobs in Bengaluru, Karnataka

Technical Writing jobs in Technopark, Thiruvananthapuram, Kerala

Senior Mechanical Designer jobs in Greater Noida, Uttar Pradesh

Mechanical jobs in Nagpur, Maharashtra

Mechanical Design Engineer jobs in Nashik, Maharashtra

Senior Mechanical Designer jobs in Vasanth Nagar, Bengaluru, Karnataka

Senior Mechanical Designer jobs in Nacharam, Hyderabad, Telangana

Senior Mechanical Designer jobs in Whitefield, Bengaluru, Karnataka

JD19 - Consultant (DBT)

GainInsights -
Chennai, Tamil Nadu

Apply Now

Job details

Qualifications

BCS
CI/CD
Performance tuning
Data modelling
Azure
Computer Science
Spark
Encryption
Disaster recovery
Git
SQL
RBAC
Cloud security
Bachelor's degree
Continuous integration
Macros
Business continuity planning
GitHub
APIs
JSON
Kafka
Python
Information Technology

Full job description

Experience: 3 to 5 years

Location: Chennai / Bangalore

Job purpose:

Client-facing consultant responsible for designing and delivering secure, scalable, and cost-efficient data platforms using Python, PySpark (Apache Spark), dbt (ELT), and Apache Kafka for streaming. Owns end-to-end engineering across ingestion (batch + streaming), transformation, modelling, governance, performance tuning, CI/CD, and operations. Partners with stakeholders to translate business requirements into technical solutions, produce design artifacts, and ensure reliable delivery.

Duties and responsibilities of position:

General areas of responsibility relate to:

Lead/co-lead requirements workshops; translate business needs into user stories and architecture patterns combining PySpark, dbt, and Kafka.

Design secure, scalable, cost-optimized data solutions across lakehouse + warehouse and streaming paradigms.

Prepare HLD/LLD, architecture diagrams, data flow/mapping specifications, estimations, and runbooks.

Ensure compliance and governance (e.g., GDPR, HIPAA, SOC 2) and enterprise standards across batch and streaming.

Support production reliability through monitoring, incident response, and continuous improvement.

Collaborate with cross-functional teams (BI, Cloud Ops, Data Science) and mentor junior engineers.

Specific areas of responsibility include but are not limited to:

Design and implement Kafka producers/consumers (Python or JVM-based) with exactlyonce/at-least-once semantics per use case.

Configure topics, partitions, replication, schemas (Avro/JSON/Protobuf) with Schema Registry and compatibility rules.

Implement stream processing via Kafka Streams, Spark Structured Streaming (PySpark), or Flink (if applicable).

Build CDC pipelines from databases using Debezium/CDC connectors; manage offsets, replays, and dead-letter queues.

Enforce resilience patterns: idempotency, retries/backoff, circuit breakers, exactly-once sinks (Delta/Snowflake/etc.).

Build ingestion frameworks using Python (files/APIs) and PySpark for distributed processing; parameterize environments.

Standardize medallion zones (bronze/silver/gold) with file formats (Parquet/Delta/CSV/JSON) and metadata conventions.

Implement incremental loads and hybrid micro-batch strategies for streaming/batch convergence.

Develop scalable ETL/ELT using PySpark (DataFrames, Spark SQL) and dbt for modular SQL models (sources staging marts).

Implement SCD Type 1 & Type 2:

PySpark/Delta Lake MERGE with surrogate keys, effective/expiry dates, current flags, audit columns.

dbt snapshots and incremental models with materializations (table/incremental).

Optimize jobs: partitioning, Z-ordering/bucketing, broadcast joins, predicate pushdown, and shuffle tuning.

Enforce coding standards: modular Python packages, parameterized notebooks/jobs, dbt macros and packages.

Design dimensional models (star/snowflake) with conformed dimensions and KPI-aligned fact tables.

Build dbt semantic layers with tests (unique/not null/relationships), docs, exposures, and data contracts.

Manage schema evolution and late-arriving data; ensure reproducibility with versioned models.

Apply RBAC in data platforms; manage secrets via Key Vault/Secrets Manager and environment variables.

Implement encryption in transit/at rest, data masking, and row/column-level policies in target systems.

Maintain lineage and catalog (dbt docs/manifest; Purview/Collibra/Atrium where available); steward metadata and tagging.

Profile PySpark jobs (stage/task metrics); fix data skew, excessive shuffles, small-file problems; tune cluster configs.

Tune dbt models (query plans, indices/distributions, materializations, incremental strategies).

Optimize Kafka workloads: topic partitioning, consumer group balancing, backpressure, batch size/linger.ms tuning.

Version control with Git; implement CI/CD for Python packages, PySpark jobs, dbt projects, and Kafka artifacts (schemas/config).

Parameterize dev/test/prod; manage secrets/variables; approvals; maintain rollback/DR strategies.

Automate quality gates: unit tests, data tests (dbt), schema compatibility checks, linting, and release notes.

Implement observability: job/app telemetry, logs, metrics dashboards for SLAs/SLOs, freshness, and latency (batch + streaming).

Set alerts for failures/anomalies; conduct RCA and remediation plans; track data downtime.

Optimize costs: autoscaling/serverless (where applicable), warehouse/cluster rightsizing, storage lifecycle, usage tagging/reporting.

Other duties:

Lead solution walkthroughs and trade-off analysis (PySpark vs. dbt, streaming vs. batch, Delta vs. warehouse).

Build POCs (CDC with Debezium, SCD2 with Delta/dbt snapshots, Kafka throughput/latency benchmarks) to de-risk delivery.

Manage risks/assumptions, change requests, and stakeholder communications; maintain decisions & actions log.

Contribute to proposals, estimations, and SOWs for Python/PySpark/dbt/Kafka engagements.

Prepare executive reports on pipeline health, governance posture, streaming reliability, and cost optimization.

Conduct internal knowledge-sharing; build accelerators (dbt macros, PySpark templates, Kafka schema standards); mentor associates.

Qualifications:

Bachelor’s degree in Computer Science, Information Technology, or a related field; or equivalent professional experience.

Experience: 4+ years in data engineering/BI consulting with strong hands-on work in Python, PySpark, dbt, and Kafka.

Python: data processing, packaging, logging, testing; API/file ingestion; CLI utilities.

PySpark: DataFrame APIs, Spark SQL, performance tuning (partitions, caching, joins, shuffle), structured streaming.

dbt: sources/staging/marts, materializations (table/incremental), snapshots, tests, docs; Jinja/macros; packages.

Kafka: producers/consumers, topics/partitions/replication, Schema Registry (Avro/JSON/Protobuf), Streams/Structured Streaming, offset management, DLQs.

CDC/SCD: incremental loads, watermarks, SCD1/SCD2 via Delta MERGE or dbt snapshots/incremental patterns; Debezium/CDC connectors (nice-to-have).

Modelling: star/snowflake, conformed dimensions, semantic consistency.

Orchestration/Tooling: Airflow/ADF/Prefect (any), Git, CI/CD (GitHub Actions/Azure DevOps), secrets management.

Storage/Compute: Lakehouse/warehouse (Delta/Parquet; Snowflake/Synapse/BigQuery/Databricks— any preferred).

Security/Governance: RBAC, secrets, encryption, masking/row policies; catalogue/lineage basics.

Hands-on experience in data modeling, indexing, and query performance tuning.

Experience with backup, disaster recovery, and business continuity planning.

Strong understanding of cloud security risks, including data breaches, malware, and social engineering.

Excellent collaboration skills for working with cross-functional teams and providing leadership in cloud security projects.

Experience working in a team, able to lead, prioritize and execute tasks in a high-pressure environment and provide or make sound decisions in emergencies.

Apply Now

Refine Your Search

quality catalog jobs in tamil nadu

I want to receive the latest job alerts for quality catalog in tamil nadu

Explore jobs in more locations

Job purpose:

Duties and responsibilities of position:

General areas of responsibility relate to:

Lead/co-lead requirements workshops; translate business needs into user stories and architecture patterns combining PySpark, dbt, and Kafka.

Design secure, scalable, cost-optimized data solutions across lakehouse + warehouse and streaming paradigms.

Prepare HLD/LLD, architecture diagrams, data flow/mapping specifications, estimations, and runbooks.

Ensure compliance and governance (e.g., GDPR, HIPAA, SOC 2) and enterprise standards across batch and streaming.

Support production reliability through monitoring, incident response, and continuous improvement.

Collaborate with cross-functional teams (BI, Cloud Ops, Data Science) and mentor junior engineers.

Specific areas of responsibility include but are not limited to:

Design and implement Kafka producers/consumers (Python or JVM-based) with exactlyonce/at-least-once semantics per use case.

Configure topics, partitions, replication, schemas (Avro/JSON/Protobuf) with Schema Registry and compatibility rules.

Implement stream processing via Kafka Streams, Spark Structured Streaming (PySpark), or Flink (if applicable).

Build CDC pipelines from databases using Debezium/CDC connectors; manage offsets, replays, and dead-letter queues.

Enforce resilience patterns: idempotency, retries/backoff, circuit breakers, exactly-once sinks (Delta/Snowflake/etc.).

Build ingestion frameworks using Python (files/APIs) and PySpark for distributed processing; parameterize environments.

Standardize medallion zones (bronze/silver/gold) with file formats (Parquet/Delta/CSV/JSON) and metadata conventions.

Implement incremental loads and hybrid micro-batch strategies for streaming/batch convergence.

Develop scalable ETL/ELT using PySpark (DataFrames, Spark SQL) and dbt for modular SQL models (sources staging marts).

Implement SCD Type 1 & Type 2:

PySpark/Delta Lake MERGE with surrogate keys, effective/expiry dates, current flags, audit columns.

dbt snapshots and incremental models with materializations (table/incremental).

Optimize jobs: partitioning, Z-ordering/bucketing, broadcast joins, predicate pushdown, and shuffle tuning.

Enforce coding standards: modular Python packages, parameterized notebooks/jobs, dbt macros and packages.

Design dimensional models (star/snowflake) with conformed dimensions and KPI-aligned fact tables.

Build dbt semantic layers with tests (unique/not null/relationships), docs, exposures, and data contracts.

Manage schema evolution and late-arriving data; ensure reproducibility with versioned models.

Apply RBAC in data platforms; manage secrets via Key Vault/Secrets Manager and environment variables.

Implement encryption in transit/at rest, data masking, and row/column-level policies in target systems.

Maintain lineage and catalog (dbt docs/manifest; Purview/Collibra/Atrium where available); steward metadata and tagging.

Profile PySpark jobs (stage/task metrics); fix data skew, excessive shuffles, small-file problems; tune cluster configs.

Tune dbt models (query plans, indices/distributions, materializations, incremental strategies).

Optimize Kafka workloads: topic partitioning, consumer group balancing, backpressure, batch size/linger.ms tuning.

Version control with Git; implement CI/CD for Python packages, PySpark jobs, dbt projects, and Kafka artifacts (schemas/config).

Parameterize dev/test/prod; manage secrets/variables; approvals; maintain rollback/DR strategies.

Automate quality gates: unit tests, data tests (dbt), schema compatibility checks, linting, and release notes.

Implement observability: job/app telemetry, logs, metrics dashboards for SLAs/SLOs, freshness, and latency (batch + streaming).

Set alerts for failures/anomalies; conduct RCA and remediation plans; track data downtime.

Optimize costs: autoscaling/serverless (where applicable), warehouse/cluster rightsizing, storage lifecycle, usage tagging/reporting.

Other duties:

Lead solution walkthroughs and trade-off analysis (PySpark vs. dbt, streaming vs. batch, Delta vs. warehouse).

Build POCs (CDC with Debezium, SCD2 with Delta/dbt snapshots, Kafka throughput/latency benchmarks) to de-risk delivery.

Manage risks/assumptions, change requests, and stakeholder communications; maintain decisions & actions log.

Contribute to proposals, estimations, and SOWs for Python/PySpark/dbt/Kafka engagements.

Prepare executive reports on pipeline health, governance posture, streaming reliability, and cost optimization.

Prepare executive reports on pipeline health, governance posture, streaming reliability, and cost optimization.

Conduct internal knowledge-sharing; build accelerators (dbt macros, PySpark templates, Kafka schema standards); mentor associates.

Qualifications:

Bachelor’s degree in Computer Science, Information Technology, or a related field; or equivalent professional experience.

Experience: 4+ years in data engineering/BI consulting with strong hands-on work in Python, PySpark, dbt, and Kafka.

Python: data processing, packaging, logging, testing; API/file ingestion; CLI utilities.

PySpark: DataFrame APIs, Spark SQL, performance tuning (partitions, caching, joins, shuffle), structured streaming.

dbt: sources/staging/marts, materializations (table/incremental), snapshots, tests, docs; Jinja/macros; packages.

Kafka: producers/consumers, topics/partitions/replication, Schema Registry (Avro/JSON/Protobuf), Streams/Structured Streaming, offset management, DLQs.

CDC/SCD: incremental loads, watermarks, SCD1/SCD2 via Delta MERGE or dbt snapshots/incremental patterns; Debezium/CDC connectors (nice-to-have).

Modelling: star/snowflake, conformed dimensions, semantic consistency.

Orchestration/Tooling: Airflow/ADF/Prefect (any), Git, CI/CD (GitHub Actions/Azure DevOps), secrets management.

Storage/Compute: Lakehouse/warehouse (Delta/Parquet; Snowflake/Synapse/BigQuery/Databricks— any preferred).

Security/Governance: RBAC, secrets, encryption, masking/row policies; catalogue/lineage basics.

Hands-on experience in data modeling, indexing, and query performance tuning.

Experience with backup, disaster recovery, and business continuity planning.

Strong understanding of cloud security risks, including data breaches, malware, and social engineering.

Excellent collaboration skills for working with cross-functional teams and providing leadership in cloud security projects.

Experience working in a team, able to lead, prioritize and execute tasks in a high-pressure environment and provide or make sound decisions in emergencies.