Minimum Criteria
Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory.
Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing.
Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions, Azure DevOps).
Strong understanding of data architecture principles and cloud-native design patterns.
Essential Criteria
Demonstrated ability to lead technical delivery, mentor engineering teams and collaborate with stakeholders to ensure alignment between data solutions and business strategy.
Proficiency in Linux/Unix environments and shell scripting.
Deep understanding of source control, testing strategies, and agile development practices.
Self-motivated with a strategic mindset and a passion for driving innovation in data engineering.
Desirable Criteria
Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives.
Familiarity with:
Apache Airflow
Data modelling and metadata management
Experience influencing enterprise data strategy and contributing to architectural governance.