We are seeking a highly skilled Platform Engineer with strong expertise in Python, Apache Airflow/Astronomer Astro, Databricks, and AWS cloud technologies. The ideal candidate will be responsible for designing, automating, administering, and supporting modern cloud-based data and platform engineering solutions. This role requires handson technical expertise, strong troubleshooting capabilities, and the ability to collaborate effectively with architects, client stakeholders, and distributed delivery teams.
- Design, build, automate, and support scalable platform engineering solutions using Python
- Develop, maintain, and troubleshoot Apache Airflow DAGs within Astronomer Astro environments
- Administer and integrate Databricks environments, including:
o Workspaces
o Clusters
o Jobs
o Notebooks
o Unity Catalog
- Manage and support AWS cloud infrastructure and services, including:
o IAM
o EKS
o S3
o Lambda
o Secrets Manager
o CloudWatch
o Networking components
- Implement Infrastructure as Code (IaC) solutions using Terraform for provisioning and managing cloud resources
- Build and maintain CI/CD pipelines and deployment automation frameworks using tools such as Harness and GitOps methodologies
- Design reusable self-service onboarding capabilities and platform automation patterns
- Ensure platform security, governance, and compliance best practices across cloud environments
- Troubleshoot issues across infrastructure, platform, networking, integrations, and application layers
- Collaborate closely with onshore architects, engineering teams, and client stakeholders to deliver reliable platform solutions
- Contribute to operational excellence, documentation, monitoring, and continuous platform improvements
- Strong hands-on experience in Python development for platform engineering, automation, and integrations
- Experience working with Apache Airflow and/or Astronomer Astro
- Hands-on experience with Databricks administration and integrations
- Strong AWS cloud experience, including IAM, EKS, S3, Lambda, Secrets Manager, CloudWatch, and networking
- Experience with Infrastructure as Code tools, preferably Terraform
- Experience with CI/CD automation and deployment practices
- Familiarity with Harness and GitOps methodologies
- Strong understanding of cloud security, IAM roles/policies, secrets management, and governance
- Experience creating reusable platform services and self-service onboarding workflows
- Strong analytical and troubleshooting skills
- Excellent communication and stakeholder management skills
- Experience working in collaborative global delivery environments
- Experience supporting enterprise-scale cloud data platforms
- Exposure to Kubernetes and containerized deployments
- Experience with observability and monitoring tools
- Knowledge of DevOps and Platform Engineering best practices
- AWS or Databricks certifications are a plus
