Azure Data engineer – Freelance – 2-3 days per week on side in Eindhoven - €500pd
As a member of the IT Big Data \& Analytics team you are part of a team of best-in-class engineers, organized in Agile teams. Our mission is to translate our client’s business challenges into Data \& Analytics solutions. You will be part of one of our DevOps teams, working together in a Scaled Agile (SAFe) environment. We share a mission to deliver business value for our stakeholders using modern data analytics technology. We work in an SAFe Agile Release Train with 7 DevOps teams.
Our Client is a crucial player in a fast moving semiconductor industry, which means change constantly around. You will work with your team to capture the constant and dynamic refinement of business priorities and data \& analytics user stories. You are looked upon, and rewarded for, being able constantly switch between business value and technical implementation. Together with your team you will work constantly to drive our client’s ambition by building a modern Data \& Analytics platform, using modern Cloud components and leveraging the Agile mindset fully.
The Data engineer works in an agile team and align with an architect and team members on solution design challenges and choices made to address the business requirements. They collaborate with solution architects, competence, development team and usiness to ensure consistency in the overall big-data landscape.
The developer / Data engineer translates business demand in solution designs, leveraging the big data design patterns and architectural guiderails. And aligns, identifies dependencies and overlap with other business demand and ensures that the team executes according to business requirements.
Experience \& Skills:
You have a strong software engineering background and for example a MSc. in Computer Science or equivalent. You should have at least 2 - 4 years of experience in working in an DevOps Developer / Data engineer role, preferably working in a highly complex environment like ours.
Data modelling experience on non-business process data domains (e.g. IOT, machine data)
Data modelling experience on business process data domains (e.g. corporate data, process data)
Data Operations (CI/CD) with Azure DevOps
Hands on experience with Azure (OData) API development
Experience with Terraform (Infrastructure as Code)
Python 4+ years, pre if you know other languages as well (Rust, Java, Scala)
Hands on experience with cloud native service Databricks, Azure Data Factory and Azure SQL
SQL (T-SQL or other dialect) 4+ years: SQL background, understanding the concepts of OLAP/BI
querying patterns, and has practical knowledge in creating these solutions to support this.
2+ years (hands-on) experience with columnar storage formats, and backends (Parquet, Delta
lake)
PySpark 4+ years: Deep knowledge of spark runtime internals
2+ years (hands-on) experience in designing and building both batch and streaming
architectures
2+ years (hands-on) experience with big data (hybrid) cloud concepts and implementations
Hands on and leading proof of concepts and securing learning in the industrialized solutions.
conceptual thinker, has the ability to think ahead and share the lessons learnt with both
technology and business stakeholders.
Experience working with Agile Release trains.
Experience with GIT