Job purpose
Looking for a Test Lead / Data QA Engineer to drive and oversee data validation, manual testing, and regression testing activities across capital
markets data pipelines spanning both legacy and modern platforms. The role is responsible for ensuring data accuracy, completeness,
consistency, integrity, and adherence to business rules through source-to-target validation, schema verification, data reconciliation, lineage
analysis, and automation-led regression testing.
The successful candidate will lead test delivery, manage and coordinate offshore QA resources, and engage closely with business and
technology stakeholders to maintain high-quality data across post-trade and enterprise data ecosystems. Key responsibilities include building
reusable Python- and SQL-based validation frameworks, supporting batch and cloud-based execution models, identifying data anomalies,
validating end-to-end data flows, and embedding robust data quality controls throughout the development lifecycle.
The ideal candidate combines strong data QA and testing expertise, hands-on automation skills, capital markets domain knowledge, and a
proactive ownership mindset to ensure data quality, consistency, and reliability across environments.
Key responsibilities
- Lead end-to-end test planning, execution, and delivery for data validation and regression testing initiatives, ensuring
quality, schedule, and stakeholder expectations are met.
- Manage and mentor offshore/onshore QA teams, including resource planning, task allocation, deliverable reviews,
and adherence to testing best practices.
- Drive test governance, including test strategy, defect management, risk tracking, quality metrics, status reporting, and
release readiness assessments.
- Collaborate with business, engineering, and product teams to define requirements, resolve issues, ensure test coverage, and
embed quality controls across the delivery lifecycle.
- Perform source-to-target data validation across legacy and modern systems, including SQL Server and cloud-based
data environments.
- Validate data accuracy, completeness, consistency, timeliness, duplication handling, referential integrity, schema
structures, metadata, lineage, and business-rule conformance.
- Build and maintain automation-first regression testing frameworks using Python, PySpark, SQL, and reusable
validation utilities.
- Execute field-level, dataset-level, transformation-level, and aggregate-level data checks across source, staging,
warehouse, data lake, and downstream systems.
- Validate ETL/ELT pipelines, post-trade data flows, stored procedures, batch jobs, and cloud-based execution
workflows.
- Design reusable test cases, validation scenarios, reconciliation checks, automated scripts, and regression packs for
repeatable data-quality assurance.
- Write, optimize, and troubleshoot complex SQL queries and stored procedures across SQL Server, Oracle, PostgreSQL,
or similar platforms.
- Translate business rules, stored procedure logic, and post-trade process flows into automated validation scripts and
assertions.
- Support capital markets and post-trade validation across allocations, clearing, settlement, confirmations,
reconciliations, market data, reference data, and downstream reporting.
- Build Python-based utilities, APIs, CLI tools, and reusable frameworks to support data validation, orchestration, API
integrations, and reporting workflows.
- Integrate regression tests, data validation checks, SQL deployables, and quality gates into CI/CD pipelines using Azure
DevOps, Jenkins, GitLab, or equivalent tools.
- Automate execution of AutoSys jobs, batch processes, stored procedures, and validation scripts across development,
UAT, and production environments.
- Work with engineering teams to embed data quality controls into pipelines, releases, and operational workflows.
- Identify data anomalies, schema mismatches, duplicate records, null-handling issues, transformation failures,
reconciliation breaks, and integrity violations.
- Log defects with clear evidence, impact analysis, root-cause observations, and business context.
Key competencies
- 8–12 years of hands-on experience in Data QA, Data Engineering QA, or Python-based automation roles and proven
experience leading QA teams, test planning, estimation, execution, defect management, and end-to-end test delivery across complex data programs.
- Strong stakeholder management skills with the ability to communicate test status, risks, quality metrics, and release readiness to business and technology leadership.
- Experience establishing test strategies, governance frameworks, quality standards, resource planning, and mentoring distributed QA teams.
- Demonstrated ability to drive cross-functional collaboration, manage competing priorities, mitigate delivery risks, and ensure successful project outcomes in Agile environments.
- Experience validating metadata, schema structures, data types, constraints, lineage, transformation logic, and downstream outputs.
- Strong Python skills for data validation, automation framework development, regression testing, and pipeline verification.
- Working knowledge of PySpark for validating large-scale data pipelines and distributed datasets.
- Solid SQL skills for source-to-target validation, reconciliation, complex queries, stored procedures, schema checks, and defect analysis.
- Experience in data QA, ETL/ELT testing, data pipeline testing, and regression testing across legacy and modern platforms.
- Proven experience building automation-first regression frameworks, reusable validation scripts, test cases, and data verification utilities.
- Strong understanding of data quality dimensions including accuracy, completeness, consistency, timeliness, uniqueness, referential integrity, and business-rule conformance.
- Exposure to CI/CD tools such as Azure DevOps, Jenkins, or GitLab, with the ability to integrate automated tests and quality
gates into deployment pipelines.
- Familiarity with batch orchestration tools such as AutoSys, Control-M, or Airflow.
- Good analytical skills to understand post-trade, financial, market data, or reference data and translate them into repeatable
validation scenarios.
- Experience with JIRA, XRay, agile delivery practices, defect management, and QA process governance.
- Proficiency in leveraging AI-assisted tools such as Claude or GitHub Copilot to accelerate test case generation, SQL
development, data validation, defect analysis, documentation, and test data creation.
- Strong communication, problem-solving, attention to detail, and cross-functional collaboration skills.
Nice to Have
- Exposure to Azure, AWS, ADF, containers, Kubernetes, or cloud-based data platforms.
- Experience with relational and NoSQL databases such as SQL Server, MongoDB, Cassandra, DynamoDB, or similar platforms.
- Exposure to Hadoop, Spark, Databricks, cloud data lakes, and Big Data validation.
- Experience validating batch, streaming, real-time, and Kafka-based ingestion pipelines.
- Capital markets domain exposure, especially post-trade, market data, reference data, trade data, or ION/blotter platform
integrations.
- Exposure to DevSecOps, infrastructure as code, and code quality tools such as Terraform, Bicep, Sonar, Mend, or ZAP.
