Experience: 4–6 Years Location: Noida, India Employment Type: Full-Time
We are looking for an experienced Data Engineer to design, build, and optimize our modern data lakehouse infrastructure. The ideal candidate will have hands-on expertise with PostgreSQL, MinIO, Dremio, and Apache Iceberg, and will play a key role in building scalable, high-performance data pipelines and enabling self-service analytics across the organization.
-
Design, implement, and maintain Apache Iceberg table formats for large-scale analytical datasets, ensuring schema evolution, partitioning, and time-travel capabilities are optimally configured.
-
Manage and optimize object storage on MinIO, including bucket policies, lifecycle rules, versioning, and access controls for the data lake.
-
Architect and maintain the lakehouse layer to ensure ACID compliance, data consistency, and efficient query performance.
-
Configure, administer, and optimize Dremio as the semantic and query acceleration layer, including reflections, data source connections, and virtual datasets.
-
Tune Dremio query performance for BI tools and ad-hoc analytics, ensuring low-latency access to Iceberg tables.
-
Build and maintain virtual datasets, views, and data curation layers to support self-service analytics for business teams.
-
Design, develop, and optimize PostgreSQL schemas, queries, indexes, and stored procedures for transactional and metadata-driven workloads.
-
Monitor and tune PostgreSQL performance (query plans, indexing strategies, vacuuming, replication) to ensure high availability and reliability.
-
Implement backup, recovery, and disaster recovery strategies for PostgreSQL instances.
-
Build and maintain robust ETL/ELT pipelines to ingest data from multiple sources into the Iceberg-based lakehouse on MinIO.
-
Ensure data quality, validation, and governance across ingestion and transformation pipelines.
-
Collaborate with data analysts, BI developers, and data scientists to understand data requirements and deliver reliable datasets.
-
Monitor system health, storage utilization, and query performance across MinIO, Dremio, and PostgreSQL environments.
-
Implement security best practices including access control, encryption, and audit logging across the data stack.
-
Document architecture, data flows, and operational runbooks; contribute to establishing engineering best practices.
-
Troubleshoot and resolve production issues related to storage, query engines, and databases.
-
4–6 years of experience in Data Engineering or a related field.
-
Strong hands-on experience with Apache Iceberg (table maintenance, compaction, partitioning, schema evolution).
-
Practical experience with MinIO or other S3-compatible object storage solutions.
-
Proven experience administering and optimizing Dremio (or similar query engines like Trino/Presto/Starburst) is a strong plus.
-
Solid expertise in PostgreSQL — schema design, query optimization, indexing, and performance tuning.
-
Proficiency in SQL and at least one scripting/programming language (Python preferred).
-
Experience with data lakehouse architecture and modern data stack concepts (data lake, data warehouse, ELT/ETL).
-
Familiarity with containerization (Docker/Kubernetes) and cloud/on-prem infrastructure is a plus.
-
Understanding of data governance, security, and access control principles.
-
Strong problem-solving skills and ability to work independently in a fast-paced environment.
