Roel: Senior Data Engineer
Location: India, Remote
Experience: 5 - 10 Years
Algoworks
www.algoworks.com
About the company
Algoworks is an award-winning artificial intelligence, engineering services and experience transformation firm with offices across the United States, Europe, South America and India. We bring together a global team of engineers, architects, designers, researchers and operators united by rigor, accountability and a commitment to delivering measurable results.
For over 20 years, Algoworks has partnered with Fortune 500 organizations across the Americas, Europe and Asia to define, build and run technology that drives meaningful business outcomes. Our work combines human-centered design, engineering excellence and AI-powered capabilities to solve complex challenges with clarity and precision. Innovation, particularly in the responsible application of AI, is embedded in how teams approach problem-solving and continuous improvement.
At Algoworks, growth is continuous and closely tied to impact. Teams collaborate across geographies and disciplines, strengthening outcomes through shared insight and collective expertise. The culture values transparency, open dialogue and an environment where every voice is heard and contribution is recognized.
Through collaboration, accountability and a focus on results, Algoworks operates at the intersection of technology and people, building not only advanced systems but strong global teams that elevate performance and create lasting impact.
Follow the video below to know about us! Clipchamp
Role overview
We are seeking a Senior Data Engineer to design and build scalable data pipelines for supply chain transaction data. This role focuses on ingesting raw data (EDI/XML), transforming it into structured datasets, and enabling downstream reporting and analytics. This is a hands-on engineering role requiring strong expertise in data processing, pipeline development, and performance optimization. The ideal candidate will also leverage AI-assisted tools to accelerate development while ensuring high standards of data quality, accuracy, and scalability.
Key responsibilities:
1.Data pipeline development
-
Build and maintain batch ingestion pipelines for transaction data from multiple systems.
-
Design scalable ELT pipelines to process large volumes of structured and semi-structured data.
-
Ensure reliability, scalability, and efficiency of data pipelines.
2.Data transformation and modelling
-
Develop transformations to convert raw data into structured analytics datasets.
-
Implement data layers including Bronze (raw), Silver (cleaned), and Gold (reporting).
-
Design data models optimized for reporting, analytics, and business insights.
3.Data processing and schema management
-
Handle schema variations across trading partners and systems.
-
Process EDI, XML, and other semi-structured data formats.
-
Ensure data consistency and integrity across pipelines.
4. Performance optimization
-
Optimize SQL queries and data pipelines for performance and cost efficiency.
-
Monitor and improve processing times and system performance.
-
Implement best practices for scalable data engineering.
5.AI-assisted development
-
Leverage AI tools (e.g., Cursor, Copilot) to accelerate ELT pipeline development and SQL generation.
-
Use AI-assisted techniques for handling schema variations and transformation logic.
-
Validate AI-generated code for correctness, performance, and scalability.
-
Utilize AI for debugging, optimization, and productivity improvements.
6.Collaboration and data quality
-
Work closely with QA and BI teams to ensure data accuracy and reliability.
-
Implement data validation checks and quality frameworks.
-
Support downstream analytics and reporting requirements.
Required skills and qualifications:
-
Bachelor’s degree in Computer Science, Engineering, or related field.
-
5–10 years of experience in data engineering and pipeline development.
Strong expertise in:
-
SQL and data transformation.
-
Data warehousing solutions (Snowflake, Azure Data Services, or similar).
-
Handling large-scale transactional datasets.
-
Experience working with EDI, XML, or semi-structured data formats.
-
Strong understanding of batch processing and pipeline orchestration.
-
Hands-on experience using AI tools for SQL development, pipeline creation, and debugging.
-
Ability to validate AI-generated outputs for correctness and efficiency.
Nice to have skills:
-
Experience in supply chain, logistics, or financial transaction systems.
-
Familiarity with data quality frameworks and validation techniques.
-
Exposure to modern data architectures (Lakehouse, Medallion architecture).
-
Experience with cloud platforms such as Azure or AWS.
Must have skills:
-
5+ years of experience in data engineering and pipeline development.
-
Strong expertise in SQL, ELT/ETL pipelines, and data warehousing platforms.
-
Experience handling large-scale transactional data, batch processing, and semi-structured formats like EDI/XML.
-
Hands-on experience with AI-assisted development tools for pipeline creation, SQL generation, and debugging.
Desired attributes:
-
Strong problem-solving and analytical mindset.
-
Ability to work in fast-paced, data-intensive environments.
-
Proactive and ownership-driven approach.
-
Strong collaboration and communication skills.
Interview process
2 rounds of discussion.
