Software Developer is needed to perform the following duties: Designed and implemented end-to-end data architecture on AWS, processing data across 350+ tables from on-prem systems into a scalable cloud data platform. Led migration of enterprise data pipelines from DB2 (on-prem) to Amazon Redshift, improving scalability, performance, and analytics capabilities. Built PySpark-based data ingestion framework to extract data from DB2 and store in optimized Parquet format in Amazon S3. Implemented batch and near real-time ingestion using Qlik Replicate with CDC for continuous data synchronization. Designed multi-layer architecture: Qlik Amazon Aurora (staging & batch control) AWS Glue Amazon Redshift. Developed scalable ETL pipelines using AWS Glue and PySpark for data transformation, cleansing, and enrichment. Orchestrated workflows using AWS Step Functions and AWS Lambda, ensuring reliable and fault-tolerant pipelines. Automated scheduling and event-driven processing with Amazon EventBridge. Designed batch monitoring framework using Amazon Aurora by generating batch IDs, enabling end-to-end tracking, data lineage, and reconciliation. Built data capture and provisioning workflows to extract production data, apply scrubbing/masking, and load into test environments for consumer testing. Implemented monitoring, alerting, and dashboards using Amazon CloudWatch and Amazon SNS; integrated ServiceNow (SNOW) for automated incident ticketing. Developed operational dashboards and data quality reports to track pipeline health, SLA adherence, and data accuracy. Built MCP-based data access services integrated with Amazon Bedrock Knowledge Bases to enable querying of data reports and documents using natural language. Enabled self-service analytics by allowing users to query structured and unstructured datasets via AI-powered knowledge base solutions, reducing dependency on engineering teams. Optimized storage, partitioning, and query performance in Redshift, reducing costs and improving execution times. Provisioned infrastructure and automated deployments using Terraform and GitLab CI/CD pipelines across dev, test, and production environments.Bachelor's Degree is required in Computer Science or Computer Engineering or Information Technology
.
