Data Engineer - 3 months+ - £400-450pd Inside IR35 - Fully Remote
As a Senior Data Engineer your primary focus will be the data, the platform we handle data with, the integrity of data and the security of data. You will be instrumental in the analysis, design, build and repeatable testing of tools, frameworks and delivery for data transformation, data migrations and other data services artefacts.
Responsibilities:
. Document the data processing logic according to agreed documentation standards. This is to include requirements, solutions, reconciliation, execution, reporting, low level ETL logic, as well as other surrounding data services artefacts or infrastructure.
. Fully understand business data requirements and seamlessly convert these into real solutions which can be utilised across the organisation
. Work with the delivery team and data team providing advice and guidance to help size effort, value and create great designs
. Assist in project reporting by providing updates, reporting risks, and issues and any other reasonable request for information as prioritised by management
. Advise on roadmap extensions to the base platform to take advantage of new cloud-based capabilities being released by the vendor
. Working with the wider engineering teams to monitor, update, evolve and drive improvement in our Azure platform - from scalability, performance, availability, access and cost
. Follow and enhance (where necessary) established data quality and data governance approaches as relevant to the project/effort you are performing, as well as providing clear and concise information to interested stakeholders and the projects engaged with.
Key Skills
. Strong technical ability and understanding of how databases work as well as the use of data software (ETL packages) and programming languages (usually SQL \& Python) to develop tools to extract, transform and load data, as well as to document the creation of these tools.
. Proficient capability to understand business processes in detail to apply knowledge, understand and create requirements, apply them to a tool design and execute its build
. Experience in some or all of the following: Azure Data Lake Azure Data Factory Azure Databricks (inc. Python and SQL) Azure SQL Server Azure DevOps/Git
. Excellent communication skills as you will need to discuss technical principles and business processes in simple language to people at all levels
. The ability to invent and innovate in the determination of creative solutions to problems, and to take on work for which there is no set of rules or industry-standard answer
Skills:
Azure Data Factory
Azure Data Lake
Azure DevOps
MS SQL Server
Python