Associate Data Engineer (178805)

Job Summary We are looking for a passionate Data Engineer with years of hands-on experience in building and maintaining scalable data pipelines. The ideal candidate should have strong expertise in Databricks, PySpark, and SQL, with experience in creating and managing Databricks Jobs and Workflows/Pipelines. Exposure to Azure cloud services will be an added advantage.

Key Responsibilities • Develop, maintain, and optimize ETL/ELT pipelines using Databricks, PySpark, and SQL. • Create, schedule, monitor, and troubleshoot Databricks Jobs, Workflows, and Pipelines. • Design and implement scalable data processing solutions for large datasets. • Perform data cleansing, transformation, validation, and quality checks. • Collaborate with business and analytics teams to understand requirements and deliver data solutions. • Support production deployments, monitoring, and issue resolution. • Follow best practices for code versioning, documentation, and performance optimization.

Required Skills • 2–3 years of experience in Data Engineering. • Strong proficiency in SQL and PySpark. • Hands-on experience with Azure Databricks. • Experience in creating and managing Databricks Jobs, Workflows, and Pipelines. • Good understanding of ETL/ELT processes and data warehousing concepts. • Familiarity with Git/version control and basic CI/CD practices. • Strong analytical and problem-solving skills. 

Preferred Skills • Experience with Azure Data Factory (ADF), ADLS, Synapse, or other Azure services. • Exposure to Delta Lake and Lakehouse architecture. • Knowledge of performance tuning and optimization of Spark workloads.

Education • Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.