Responsibilities
Collaborating with stakeholders to understand business requirements and developing optimized data transformation workflows for multiple data sources and destinations.
Optimizing Database jobs for performance by collaborating with DBAs and implementing SQL coding and design changes within the job structure.
Actively participated in troubleshooting and resolving issues related to data loading delays and database server outages, ensuring minimal disruption to data flow and system operations.
Engaged in periodic reviews of job logs and performance metrics to identify trends, proactively addressing potential issues.
Troubleshooting production issues through meticulous job log reviews, implementing necessary design changes via change management, and adhering to the code promotion process.
Contributed to the development and maintenance of documentation for troubleshooting procedures, best practices, and system configurations, facilitating knowledge sharing within the team and managing Change Requests
Resolving data issues arising from load delays or database server downtimes, with a focus on ensuring customer satisfaction.
Qualifications
4+ years of experience with data pipeline design and ETL jobs development using prominent tools such as Informatica, Talend, pySpark, Scala and other open-source ETL frameworks.
Extracting data from various data sources, transforming, and loading it into target systems.
Ensuring data quality, integrity, and consistency across different systems.
Optimizing ETL processes for performance, scalability, and reliability.
SQL proficiency (intermediate skill level minimum)
Ability to manage complex data mappings and parallel process workflows.
Troubleshooting and resolving data integration issues.
Solid skills with writing/translating SQL queries and SQL data validation.
Performance tuning of SQL queries.
Defining, managing, and monitoring job schedules, as well as troubleshooting issues within the Autosys environment.
Understanding and implementing job schedules, dependencies, and workflows within Autosys.
Analyzing and resolving issues with ETL jobs and processes
informatica,talend,pyspark,scala,
