Data Engineer (6–7 Years Experience)
Location: Remote (Required to visit client office 1–2 days/month – any location)
Working Hours: 11:00 AM – 9:00 PM IST
Role Overview
We are looking for an experienced Data Engineer with 6–7 years of hands-on experience in building data pipelines, working with Azure Databricks, ADF, Python, Apache Spark, and SQL. The candidate will collaborate with business and technical teams to develop scalable data solutions and support analytics initiatives.
Key Responsibilities
Design and develop scalable data pipelines using Azure Databricks, Apache Spark, and related Azure services.
Create and manage workflows using Azure Data Factory (ADF) and automation platforms.
Integrate structured, semi-structured, and unstructured data into Databricks environments.
Develop and maintain ETL/ELT pipelines using Delta Lake and Azure Data Lake.
Optimize Spark jobs for performance, reliability, and cost efficiency.
Troubleshoot and resolve issues in distributed data workflows.
Collaborate with data analysts, business users, and teams for self-service analytics support.
Automate data ingestion, transformations, deployments, and workflow orchestration.
Implement CI/CD pipelines using GitHub Actions or similar tools.
Ensure adherence to data governance, security, and privacy requirements.
Utilize data cataloging and lineage tools to maintain metadata governance.
Required Skills (Must-Have)
Azure Databricks
Azure Data Factory (ADF)
Python
Apache Spark
SQL
Good-to-Have Skills
Power Apps
Power Automate
Azure Logic Apps
ETL process knowledge
GitHub Actions / CI-CD pipelines
MLflow
Machine learning workflows
Experience Requirements
6–7 years of experience in data engineering.
Strong understanding of ETL/ELT methodologies and data modeling.
Hands-on experience with Azure ecosystem: Databricks, Data Lake, Synapse, and ADF.
Preferred Qualifications
Experience building CI/CD pipelines for Databricks.
Knowledge of MLflow, Delta Lake, and machine learning lifecycle workflows.
Familiarity with Azure Purview or other governance platforms.
Experience working in Agile environments.
Soft Skills
Strong communication and stakeholder management capabilities.
Ability to work in fast-paced, dynamic environments.
Excellent problem-solving skills and attention to detail.
Job Type: Contractual / Temporary
Contract length: 6-12 months
Pay: ₹70,000.00 - ₹90,000.00 per month
Work Location: Remote