The worldwide data management software market is massive (According to IDC, the worldwide database software market, which it refers to as the database management systems software market, was forecasted to be approximately $82 billion in 2023 growing to approximately $137 billion in 2027. This represents a 14% compound annual growth rate). At MongoDB we are transforming industries and empowering developers to build amazing apps that people use every day. We are the leading developer data platform and the first database provider to IPO in over 20 years. Join our team and be at the forefront of innovation and creativity.
The Data Pipelines Engineering team is responsible for building ETL pipelines that populate the Internal Data Platform, which drives analytics that help the company run more efficiently. Our team builds highly performant and scalable processes that extract massive datasets and makes those datasets available for querying in an optimal way. We are also building a Generative AI framework that will help teams within the company tap into the data that we store in their Retrieval-Augmented Generation (RAG)-based applications.
We are looking to speak to candidates who are based in Gurgaon, India for our hybrid working model.
- Design, build and maintain efficient, scalable ETL/ELT pipelines using Python and Spark across batch, file-based, and streaming architectures
- Ensure data quality, reliability and timeliness across pipelines by following established data engineering best practices
- Model and store large datasets using modern file formats (Parquet, JSON, Avro) and table formats (Iceberg, Hive)
- Deploy pipelines on cloud infrastructure, leveraging Cloud-based technologies (mostly AWS, some GCP) to build and deploy data pipelines
- Partner with Data Analysts and Data Scientists to understand their needs and deliver the datasets that drive their work
- Work with Security and Compliance teams to ensure that datasets have appropriate permissions and regulations in place
- Work with our Data Platform, and Governance sibling teams to make data scalable, consumable, and discoverable
- 4+ years of building ETL pipelines for a Data Lake/Warehouse
- Expertise in Python, Spark, SQL, Airflow
- Experience with data warehousing and engineering concepts, analytical data modeling, data quality validation, monitoring, and pipeline reliability practices
- Hive, Iceberg, Glue, or other technologies that expose big data as tables
- Familiarity with different big data file types such as Parquet, Avro, JSON etc
- Background in building data platforms on Cloud (e.g. AWS, GCP, Azure)
- Exposure to real-time or streaming data technologies is a plus
- In 3 months, you'll have collaborated with stakeholders in Data Analytics and Data Science to build your first ETL pipeline
- In 6 months, you'll have owned the delivery of a large project from start (scoping, design) to finish (delivery)
- In 12 months, you'll have designed new features, led development work, and become a go-to expert on parts of the system
MongoDB is built for change, empowering our customers and our people to innovate at the speed of the market. We have redefined the data platform for the AI era, enabling builders to create, transform, and disrupt industries with software. MongoDB's unified data platform, the most widely available, globally distributed data platform on the market, helps organizations modernize legacy workloads, embrace innovation, and unleash AI. Our cloud-native platform, MongoDB Atlas, is the only globally distributed, multi-cloud data platform and is available across AWS, Google Cloud, and Microsoft Azure.
With offices worldwide and over 67,000 customers, including 75% of the Fortune 100 and AI-native startups, relying on MongoDB for their most important applications, we're powering the next era of software.
Our compass at MongoDB is our Leadership Commitment, guiding how and why we make decisions, show up for each other, and win. It's what makes us MongoDB.
To drive the personal growth and business impact of our employees, we're committed to developing a supportive and enriching culture for everyone. From employee affinity groups, to fertility assistance and a generous parental leave policy, we value our employees' wellbeing and want to support them along every step of their professional and personal journeys. Learn more about what it's like to work at MongoDB, and help us make an impact on the world!
MongoDB is committed to providing any necessary accommodations for individuals with disabilities within our application and interview process. To request an accommodation due to a disability, please inform your recruiter.
MongoDB is an equal opportunities employer.
Req ID - 2273437556
