Lead Data Engineer

Details

Reference number

430548

Salary

£59,586

The basic pay for this role will be between £64979 - £79011 (London), £59586- £73641 (National) depending on the proficiency level assessment. An additional digital allowance may be payable depending on the level of assessed capability, in order to meet the MHCLG overall targeted remuneration rate for the specific grade, location and capability rating. For applicants in receipt of existing allowances, we will assess each case individually, but we would seek not to exceed the MHCLG overall targeted remuneration rate for the specific grade, location and capability rating for existing civil servants, the usual policy on level transfer and promotion will apply and is non-negotiable, subject to the statements above.

A Civil Service Pension with an employer contribution of 28.97%

GBP

Job grade

Grade 7### Contract type

Permanent### Type of role

Digital### Working pattern

Flexible working, Full-time, Job share, Part-time### Number of jobs available

2

Contents

Location

About the job

Benefits

Things you need to know

Apply and further information

Location

Bristol, Darlington, London, Manchester, WolverhamptonAbout the job

Job summary

Here at the Ministry of Housing, Communities and Local Government (MHCLG), we work on things that make a real difference to people’s lives. Whether it's through the homes we live in, the work of our local councils, or the communities we’re all a part of, our work is at the top of the political agenda.

We have ambitious and far-reaching outcomes to achieve this year and, if you’re thinking of joining us, there’s never been a more exciting time. We have circa 3,500 staff who are based in 20 offices across the UK.

The Digital Planning Programme is delivering a once-in-a-generation transformation of England’s planning system. The Common Data Platform is a federated data platform enabling teams to get to work with data faster. Working with large datasets form a variety of sources, we are creating modern, digital, and data-driven tools and services to support local authorities, developers, communities, and central government.

As a Lead Data Engineer, you’ll play a key role in building and managing the data infrastructure that underpins this transformation. You’ll lead the design of reusable data pipelines, open data services, and making planning and land data more transparent, accessible, and interoperable. This role combines hands-on engineering with technical leadership, supporting cross-government efforts to make planning data more open, standardised, and integrated across the planning lifecycle.

You can explore our platform here Planning Data, and as we work in the open, you can find out what we do here at Digital Land.### Job description

As a Lead Data Engineer You'll;

Lead the design, development, and maintenance of modern data pipelines and platforms that ingest, process, and publish planning data from local authorities and external providers

Collaborate with product managers, policy specialists, developers, and analysts to turn complex planning data into usable, accessible services for multiple user groups

Model and transform data in line with open standards, working toward consistent, high-quality data structures

Build and maintain cloud-native data infrastructure using platforms such as AWS or Azure (e.g., Glue, Lambda, S3, Redshift, or equivalent)

Ensure robust metadata management, version control, and data lineage to support transparency and auditability

Coach junior data engineers and help build internal capability in data engineering and best practice

Ensure compliance with data governance, protection, and security standards, and work in line with FAIR data principles and open data expectations

Contribute to the department’s open-source and open-data commitments by developing public-facing datasets and APIs

Manage and maintain key pipelines supporting our platform to ensure service reliability.

Person specification

As a Lead Data Engineer You'll have;

Proven experience designing and implementing end-to-end data pipelines using modern programming languages and tools such as DuckDB, Parquet, Python, SQL, Airflow, PostGIS, Spark, Git

Strong experience with cloud-based data architecture and infrastructure (e.g. AWS Glue, S3, Lambda, Redshift, or Azure equivalents)

Experience working with large, structured, semi-structured, and geospatial datasets, and understanding data quality and standardisation challenges

Ability to lead technical discussions, influence data design decisions, and communicate effectively with non-technical stakeholders

Experience working in agile, cross-disciplinary delivery teams

Strong understanding of data ethics, privacy, and security frameworks, including experience with anonymisation and responsible data use.

Behaviours

We'll assess you against these behaviours during the selection process:

Leadership

Delivering at Pace

Changing and Improving

Working Together

Benefits

Alongside your salary of £59,586, Ministry of Housing, Communities and Local Government contributes £17,262 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.* Learning and development tailored to your role

An environment with flexible working options

A culture encouraging inclusion and diversity

A Civil Service pension with an employer contribution of 28.97%

Things you need to know

Selection process details

This vacancy is using Success Profiles , and will assess your Behaviours, Strengths and Experience.

MHCLG want to bring in a diverse workforce at all levels.

Our application system is designed to remove as much bias as possible from the recruitment system – this means that a hiring manager does not know your name, your details, see your whole application in one go (or have your CV at review stage unless stated otherwise).

Artificial Intelligence

Artificial Intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance for more information on appropriate and inappropriate use.

CV and Cover Letter Declaration

We recruit based on your knowledge and skills, and not background, gender or ethnicity - this is called anonymous recruitment.

Please remove references to your:

name/title

educational institutions

age

gender

email address

postal address

phone number

nationality/immigration status

You will need to merge your CV and covering letter into one document. When submitting your application Applied will ask you to upload a CV, when doing this you should upload your merged document of your CV and covering letter.

Most of our campaigns utilise multiple assessors and so it is possible that your application would be viewed by different assessors.

In the event of a high number of applications, we will conduct an initial Lead Sift assessing the following question (Technical): Tell us about a time when you made large volumes of varied data more open, accessible, or interoperable. How did you approach designing and implementing technical solutions, applying data standards, and engaging with stakeholders to ensure the data was usable and valuable beyond your immediate team?

At sift, through your CV and covering letter we will be assessing:

Experience

Technical

In your CV and covering letter, please include: a. why you are interested in the role; b. how you meet the essential skills and experience required’

The interview will be of a blended nature consisting of the following success profiles elements:

Behaviour (Leadership, Delivering at Pace, Changing and Improving, Working Together)

Experience

Technical

Strength

The strength based questions will require natural responses from the candidates.

There will be a technical test task provided to candidates invited to interview.

In full the campaign will test the below Success Profile Elements:

Experience: Yes - Experience questions will be based around the essential skills and criteria as listed in the person specification

Technical: Yes – Technical questions will be based around the essential skills and criteria as listed in the person specification

Behaviours: (Leadership, Delivering at Pace, Changing and Improving, Working Together)

Strengths: Yes

Group 1 Digital and Data roles

MHCLG has implemented the Digital and Data capability framework for Group 1 roles. Applicants that are successful and have been offered a position will be required to complete a capability assessment after the interview.

MHCLG will honour completed capability assessments for this role from other Government Departments for existing civil servants on level transfer only. Please provide a copy of your capability assessment to the Hiring Manager when applying. If you have any queries on pay, please contact the Hiring Manager.

Each experience or technical skill is assessed between 1-3, representing working towards, at or above the job level requirements. You are awarded a proficiency level accordingly, and you will be given opportunity to annually re assess your capability and potentially increase your overall remuneration, through payment of allowances, depending on level of assessed capability.

Candidates moving from another government department have the option to retain their current basic pay if this is within our pay band for the relevant grade.

We do not consider direct CV applications – you must apply for this role via the application link on Civil Service Jobs:

Please note that near miss offers may be made at the lower grade to candidates who do not meet the grade criteria for this campaign.

Grade 7 Group 1 Digital and Data salary

The basic pay for this role will be between £64979 - £79011 (London), £59586- £73641 (National) depending on the proficiency level assessment.

An additional digital allowance may be payable depending on the level of assessed capability, in order to meet the MHCLG overall targeted remuneration rate for the specific grade, location and capability rating

For applicants in receipt of existing allowances, we will assess each case individually, but we would seek not to exceed the MHCLG overall targeted remuneration rate for the specific grade, location and capability rating for existing civil servants, the usual policy on level transfer and promotion will apply and is non-negotiable, subject to the statements above.

Please note that the average employer pension contribution is based upon the National minimum salary for this role. Should your agreed starting salary for this role be different, the average employer pension contribution will be calculated accordingly. If you are a Secondee, this will not apply as you will remain on your home organisation’s terms and conditions.

Benefits

Transfers across the Civil Service on or after 4 October 2018:

Any move to MHCLG from another employer will mean you can no longer access childcare vouchers. This includes moves between government departments. You may however be eligible for other government schemes, including Tax-Free Childcare. Determine your eligibility at https://www.childcarechoices.gov.uk.

For further information about the benefits available to MHCLG employees, please see the attached Candidate Pack.

Geographical Location:

London

Bristol

Manchester

Wolverhampton

Darlington*

There may be opportunities for candidates to work flexibly depending on the business needs. This will be discussed with the vacancy manager on a case-by-case basis if you are successful for the role.

Please note: The Darlington Economic Campus is a pioneering new cross-government hub which will bring together people across departments and public organisations to play an active role in the most important economic issues of the day. The work of the Campus will make a real difference to people both across the UK and internationally. There will be substantial career opportunities and exciting prospects - a career at the Campus means you will be working at the heart of Government, with access to the benefits and fantastic opportunities offered by the civil service. This role is based at MHCLG and we will be joined on the campus by:

HM Treasury

Department for International Trade

Department for Business, Energy and Industrial Strategy

Office for National Statistics

Department for Education

For further information on the DEC, please take a look at the attached DEC candidate pack.

Sift and Interview dates

Sifting is envisaged to take place W/C 20th October 2025 with interview dates to be confirmed. All interviews are currently being held remotely via videocall.

Reserve List

In the event that we identify more appointable candidates than we currently have posts available, we will hold applicant details on a reserve list for a period of 6 months from which further appointments can be made. This may include roles at a lower grade. Candidates placed on a reserve list will be informed of this. Those candidates who do not wish to remain on the reserve list should contact recruitment@communities.gov.uk to be removed from the reserve list.

SC (Security Check):

Important note

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is Security Check and the process can take up to 8 weeks to complete.

Please note that successful candidates will need to pass the Security Check – this requires you to have been resident in the UK for the past 5 years. Please refer to the MHCLG Notes on Security Clearance section of our Candidate Pack for further information on Security Check (SC).

Candidates should also note that with effect from 1st August 2018 the department will also check all applicants who are successful at interview, against the Internal Fraud Database (IFD) held by the Cabinet Office. In accordance with the Civil Service Internal Fraud Policy, any applicant who is included on the IFD will be refused employment by MHCLG Please see the Candidate Pack for further information on the Internal Fraud Database.

Candidate Pack Information

Please see attached Candidate pack for further information.

Before starting your application it’s very important to make sure that you are eligible to apply and meet the Civil Service nationality requirements. All candidates are expected to read the information provided in the MHCLG candidate pack regarding nationality requirements and rules

Internal Fraud Database

The Internal Fraud function of the Fraud, Error, Debt and Grants Function at the Cabinet Office processes details of civil servants who have been dismissed for committing internal fraud, or who would have been dismissed had they not resigned. The Cabinet Office receives the details from participating government organisations of civil servants who have been dismissed, or who would have been dismissed had they not resigned, for internal fraud. In instances such as this, civil servants are then banned for 5 years from further employment in the civil service. The Cabinet Office then processes this data and discloses a limited dataset back to MHCLG as a participating government organisations. MHCLG then carry out the pre employment checks so as to detect instances where known fraudsters are attempting to reapply for roles in the civil service. In this way, the policy is ensured and the repetition of internal fraud is prevented.

For more information please see- Internal Fraud Register

Feedback will only be provided if you attend an interview or assessment.### Security

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check .

See our vetting charter .

People working with government assets must complete baseline personnel security standard (opens in new window) checks.### Nationality requirements

This job is broadly open to the following groups:

UK nationals

nationals of the Republic of Ireland

nationals of Commonwealth countries who have the right to work in the UK

nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS)

nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)

individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020

Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements

Working for the Civil Service

The Civil Service Code sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles .

The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.### Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan and the Civil Service Diversity and Inclusion Strategy .Apply and further information

Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.### Contact point for applicants

Name : Digital Data Jobs

Email : digitaldatajobs@communities.gov.uk

Email : digitaldatajobs@communities.gov.uk