Project Role : Custom Software Engineer
Project Role Description : Develop custom software solutions to design, code, and enhance components across systems or applications. Use modern frameworks and agile practices to deliver scalable, high-performing solutions tailored to specific business needs.
Must have skills : Data Engineering
Good to have skills : NA
Minimum 2 year(s) of experience is required
Educational Qualification : Bachelors degree in Computer Science Engineering Information Technology or related field
Summary:
As a Custom Software Engineer, you will engage in the development of custom software solutions that involve designing, coding, and enhancing various components across systems or applications. Your typical day will include collaborating with team members to utilize modern frameworks and agile practices, ensuring the delivery of scalable and high-performing solutions that are tailored to meet specific business needs. You will also be involved in troubleshooting and optimizing existing software components to improve functionality and performance, while continuously seeking innovative approaches to enhance the overall software development process.
Roles & Responsibilities:
1. Design, develop, and maintain scalable data pipelines and ETL processes to collect, process, and store large volumes of structured and unstructured data.
2. Collaborate with data scientists, analysts, and business teams to understand data requirements and deliver reliable data solutions.
3. Optimize data architecture for performance, reliability, and scalability.
4. Implement data quality checks and monitor data integrity throughout the pipeline.
5. Work with cloud platforms as AWS, GCP, Azure and big data technologies Spark, Hadoop, Kafka, etc.
6. Manage and optimize databases and data warehouses. e.g: Redshift, BigQuery, Snowflake.
7. Develop automation tools for data ingestion, transformation, and integration.
8. Document data flows, architecture, and processes.
9. Ensure compliance with data governance and security policies.
Professional & Technical Skills:
1. Proven experience as a Data Engineer or in a similar role.
2. Strong programming skills in Python, Pyspark, Java, or Scala.
3. Experience with SQL and NoSQL databases.
4. Familiarity with ETL tools and data pipeline frameworks. e.g: Apache Airflow, Luigi.
5. Hands-on experience with cloud data services AWS Glue, Google Dataflow, Azure Data Factory.
6. Knowledge of big data technologies such as Hadoop, Spark, Kafka.
7. Strong problem-solving skills and attention to detail.
8. Excellent communication and collaboration skills.
9. Experience with containerization and orchestration tools Docker, Kubernetes.
10. Knowledge of machine learning pipelines and data science workflows.
11. Familiarity with data visualization tools Tableau, Power BI.
Additional Information:
1. The candidate should have minimum 3 years of experience in Data Engineering.
2. This position is based at our Bengaluru office.
3. A Bachelors degree in Computer Science Engineering, Information Technology, or related field is required.
Bachelors degree in Computer Science Engineering Information Technology or related field
