Lead Data Engineer

Details

Reference number

424721

Salary

£57,670 - £67,500

London: £60,670 - £67,500 / National: £57,670 - £64,500. This role typically attracts a Government Digital and Data (GDaD) allowance of £2,500 - £10,000 for eligible individuals. Confirmation of eligibility will be provided to the successful applicant on appointment.

If the successful candidate is a current Civil Servant joining on level transfer, HM Treasury will honour their current substantive salary if higher.

A Civil Service Pension with an employer contribution of 28.97%

GBP

Job grade

Grade 7### Contract type

Permanent### Business area

HMT - Economics### Type of role

Architecture and Data### Working pattern

Flexible working, Full-time, Part-time### Number of jobs available

1

Contents

Location

About the job

Benefits

Things you need to know

Apply and further information

Location

This post can be based in London (1 Horse Guards Road) or Darlington (Feethams House).About the job

Job summary

If you’re interested in making a difference to people’s lives, the Treasury can offer you an exciting opportunity to influence decision making that affects the whole of the UK. Working at the heart of government, we collaborate across government to promote responsible public spending and drive strong and sustainable economic growth.

Our work ranges from protecting customers through the regulation of the financial sector, helping to reduce carbon emissions and creating a greener economy, to promoting British trade around the world and supporting people across the country on jobs, growth and more.

We are part of the Darlington Economic Campus, a pioneering new cross-government hub which brings people together to play an active role in the most important issues of the day whilst working closer to the communities we serve. The campus provides the opportunity for people from all over the UK to help shape the future of the country, and our flexible working practices ensure you can collaborate effectively with our partners. It’s central government, made more accessible to you!

Job description

About the Group

Economics Group

The Economics Group leads HM Treasury’s analysis of developments in the UK economy and advises ministers on the implications for the Government’s economic strategy. It provides expert economic analysis and advice for Ministers. The Group’s data hub is transforming how the Treasury uses data across its entire business. The Group collaborates with external experts on the UK economy, including the Bank of England, Office for Budget Responsibility and leading academics. Economics Group provides support for the UK Government’s economics and social research professions, including running centralised recruitment and the annual conferences.

About the team

The Data Hub is a centre of excellence for data management and advanced analytics in HM Treasury. Working closely partners inside and outside of government, the Hub is responsible for the implementation of the Treasury’s data strategy, developing the organisation’s data capability and driving data-led transformation across the department. The Hub also provides strategic coordination and support to teams across the department, bringing data tools and technologies to bear on new and existing data sources. It apples innovative methods to enable rich analysis and support HMT’s departmental objectives, delivering new insights and improving decision-making. To do this, the Hub is organised into two functions:

Data Management – leads the department in developing its data management architecture, engineering, and data literacy. This includes developing systematic and structural approaches using data science techniques and ensuring that officials at all levels have access to data-related learning and development opportunities.

Advanced Analytics –a flexible pool of data scientists who are responsible for leading the development of the department’s AI capabilities, and can identify, initiate, and support high-value data science projects across the department.

Abut the Job

As a lead data engineer, you will work on a variety of data engineering projects and own several AI products, maintaining and developing them in response to user feedback. These AI products are used by most HMT staff: improving them will increase productivity across the department. You will also work in a team or data engineers and scientists, supporting HMT through a wide portfolio of projects, and hold relationships with customers within HMT and beyond.

Key accountabilities of this role include:

You will be acquiring data sets from multiple sources at varying levels of maturity. You will have a demonstrable understanding of how to expose data from systems (e.g. from APIs), link data from multiple systems, and deliver streaming services. You will collaborate with others to identify and plan these services, and to review specifications.

You will maintain and develop AI products used internally and develop new products in line with HMT’s AI roadmap. You will leverage your expertise in Python, cloud systems and AI (specifically LLMs and embeddings) to achieve this. You will ensure the products run reliably and react to any issues rapidly.

You can produce data models and understand where to use different types of data models. You have experience of a wide range of data engineering tools. You understand industry-recognised data-modelling patterns, how to establish and use agreed standards and tools, design, code, and test programs and scripts for data engineering. You can develop resilient, scalable, and future-proof services for to meet user needs.

You will actively scope and carry out other projects including web app development and data engineering. You will explore opportunities to work innovatively, recognising, identifying and exploiting opportunities to ensure efficient and effective performance of the organisation, and to support and enable that you will influence the data architecture in place. You can explore new ways of conducting business and organisational processes.

You will promote best practice in data engineering including acquisition, storage and sharing of data for the benefit of modelling and policy development, working with the Hub to provide overall technical leadership and development opportunities across HMT in data and data science.

You will contribute to the growth of the Data Hub through sharing work, pair programming, contributing process ideas.

Essential criteria:

Strong experience using Python for data engineering and web app development

Expertise in SQL

Considerable experience at least one major Cloud provider (Azure is preferred, however Google Cloud or AWS are acceptable)

Experience successfully implementing a product and managing customer relationships, acting on feedback and other good product management practices

Experience using LLM and Embedding APIs, with a strong understanding of how both work

Desirable criteria:

Experience making pipelines and data warehousing with Microsoft Fabric

Experience making apps with Plotly Dash, Streamlit and/or Chainlit

Candidate Drop-In Session

We will be running a candidate drop-in session for this role to give you greater insight about the role as well as the chance to learn more about HM Treasury and the recruitment process. If you would like to join us, then use the appropriate link below to join the call at the right time.

Tuesday 14th October 2025 : 16:30

Join the meeting now

Meeting ID: 392 285 196 672 1

Passcode: Gx3Gb2xL

If you would like to speak to the hiring manager informally prior to the closing date for applications to find out more about the job, please contact Molly Adamat at Molly.adamat@hmtreasury.gov.uk

Person specification

Your application will consist of a CV and a 750-word personal statement.

Your CV should consist of your career history, including any key achievements in each role. Your CV will not be scored, but instead will be used to provide further context on your personal statement, and support conversations at interview.

Your personal statement should demonstrate evidence of how you meet the criteria listed below:

Evidence of experience in python for data engineering and web app development, SQL, using major cloud providers. Strong understanding of LLMs and embedding APIs.

Evidence of ability to communicate between the technical and non-technical: ability to identify, tailor and prioritize diverse user needs and consider ways to improve service delivery. Consider the full impact of implementing changes on the processes, behaviours and morale of diverse end users

Successful track record of service delivery - From scoping and identifying problems to building, integrating, testing and problem management, deliver objectives with professional excellence, expertise and efficiency, taking account of diverse customer needs.

These criteria have been identified as the essential skills or experience for the role, so it is important that you include evidence to demonstrate you have the skills or experience we are looking for. You should not include any personal identifying information within your CV or personal statement.

Please review the Candidate FAQ document that is attached to the advert for more guidance on how to complete your application form.

Artificial Intelligence (AI)

Artificial Intelligence can be a useful tool to support your application, however, all examples and statements provided must be truthful, factually accurate and taken directly from your own experience. Where plagiarism has been identified (presenting the ideas and experiences of others, or generated by artificial intelligence, as your own) applications may be withdrawn and internal candidates may be subject to disciplinary action. Please see our candidate guidance for more information on appropriate and inappropriate use.

Behaviours

We'll assess you against these behaviours during the selection process:

Managing a Quality Service

Changing and Improving

Working Together

Technical skills

We'll assess you against these technical skills during the selection process:

Experience of python for data engineering and web app development, SQL, using major cloud providers.

Benefits

Alongside your salary of £57,670, HM Treasury contributes £16,706 towards you being a member of the Civil Service Defined Benefit Pension scheme. Find out what benefits a Civil Service Pension provides.* 25 days’ annual leave (rising to 30 after 5 years), plus 8 public holidays and the King's birthday (unless you have a legacy arrangement as an existing Civil Servant)

Flexible working patterns (part-time, job-share, condensed hours)

Generous parental and adoption leave package.

A Civil Service Pension which provides an attractive pension, benefits for dependants and average employer contributions of 28%

Onsite restaurant and coffee bar. The London office also offers a gym, showers and prayer room

Access to a cycle-to-work salary sacrifice scheme, season ticket advances and payroll giving

Access to a retail discounts and cashback site

A Rental Deposit Advance Scheme to help meet the total costs of deposits for privately rented homes

A range of active staff networks, based around interests and diversity

Flexible Working Arrangements

HM Treasury views flexible working as essential in enabling us to recruit and retain talented people, ensuring that they can enjoy a long-lasting career with us. All employees have the right to apply for flexible working and there are a range of options available including; part-time, compressed hours and job sharing. Additionally, we operate flexitime systems, allowing employees the flexibility to adjust their working patterns throughout the week which is subject to operational needs and line management approval.

At HM Treasury we have an incredibly broad remit; our work touches every citizen of the country. So, it’s important our employees come from the widest possible range of backgrounds, bringing us the widest possible range of perspectives and ways of thinking. We are committed to ensuring that all staff can realise their potential and achieve a healthy work-life balance.

HM Treasury operates an office based working approach across all Treasury sites - Darlington, London, and Norwich, and along with the rest of the Civil Service, has an expectation of a minimum of 60% attendance in the office, along with working remotely. This blended working approach allows you to work collaboratively, meet stakeholders face to face, support others and promotes a healthy work life balance (please be aware that this role can only be worked in the UK and not overseas). Some roles will not be suitable for Hybrid Working. Similarly, Hybrid Working will not suit everyone’s circumstances. Arrangements will be discussed and agreed with the successful candidate(s) and subject to regular review. All our offices have been recently modernised and designed to collaborate and connect with colleagues as well as desk and quiet space to allow a range of ways to work.

The office working expectation is linked to the location of the role. If you apply to a post in a single location, then you will not be able to meet this expectation at any of our other sites or move your role to another location.

Things you need to know

Selection process details

This vacancy is using Success Profiles , and will assess your Behaviours, Experience and Technical skills.Recruitment Timeline

Closing date: Sunday 19th October 2025

Shortlisting: w/c 20th October 2025

Interviews: w/c 27th October 2025

This timeline is indicative and may be subject to change. We will inform you if there is a substantial change to the recruitment timeline.

If your contact details change at any time during the selection process, please ensure you update your Civil Service Jobs Profile.

Please note that only applications submitted through Civil Service Jobs will be accepted.

Reserve Lists

You will be placed on a reserve list if your application is successful, but we cannot offer you a post immediately. Where more than one location has been advertised, candidates will be approached for roles in merit order according to the first preferred location stated on your application form.

Please note that a place on the reserve list does not guarantee an offer. We would still encourage you to apply for other HM Treasury opportunities that you are interested in. If you are offered a role in your first preferred location and you decline the offer or you are unable to take up the post within a reasonable timeframe, you will be removed from the reserve list, other than in exceptional circumstances.

Those on the reserve list may also be approached for similar roles at a lower grade. If you are offered a role at a lower grade and choose to decline, you can remain on the reserve list.

Eligibility Statement

A candidate is not eligible to apply for a role in the Civil Service if the application is made within a 5 year period following a dismissal for carrying out internal fraud against government. Checks will be performed as part of pre-employment checks in line with this. Please refer to the Candidate FAQ document attached to the advert for more information.

Everyone working with government assets must complete Baseline Personnel Security Standard (BPSS) checks. Additionally, individuals appointed to the Treasury group will be subject to National Security Vetting. The level of security vetting required for this role is Security Check (SC)

To allow for meaningful checks to be carried out, applicants will need to have lived in the UK for at least 3 out of the past 5 years. In exceptional circumstances, for example if you have been working abroad on a government posting, a lack of residency would not be a bar to security clearance however the Department will need to consider eligibility on a case-by-case basis once the advert closing date has passed.

Please read the Vetting Charter for information on what to expect during the vetting process and what will be expected from you. Many areas of your life may be explored during your vetting journey, and it is important that every individual, regardless of their background and experiences, should feel comfortable going through this personal process, whilst having confidence that it is fair, proportionate, and inclusive.

These short videos address common concerns and preconceptions which applicants may have about national security vetting. If you have questions relating to security clearances, please contact HMTSecurityVetting@hmtreasury.gov.uk

Feedback will only be provided if you attend an interview or assessment.### Security

Successful candidates must undergo a criminal record check.

Successful candidates must meet the security requirements before they can be appointed. The level of security needed is security check .

See our vetting charter .

People working with government assets must complete baseline personnel security standard (opens in new window) checks.### Nationality requirements

This job is broadly open to the following groups:

UK nationals

nationals of the Republic of Ireland

nationals of Commonwealth countries who have the right to work in the UK

nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities with settled or pre-settled status under the European Union Settlement Scheme (EUSS)

nationals of the EU, Switzerland, Norway, Iceland or Liechtenstein and family members of those nationalities who have made a valid application for settled or pre-settled status under the European Union Settlement Scheme (EUSS)

individuals with limited leave to remain or indefinite leave to remain who were eligible to apply for EUSS on or before 31 December 2020

Turkish nationals, and certain family members of Turkish nationals, who have accrued the right to work in the Civil Service

Further information on nationality requirements

Working for the Civil Service

The Civil Service Code sets out the standards of behaviour expected of civil servants.

We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission's recruitment principles .

The Civil Service embraces diversity and promotes equal opportunities. As such, we run a Disability Confident Scheme (DCS) for candidates with disabilities who meet the minimum selection criteria.

The Civil Service also offers a Redeployment Interview Scheme to civil servants who are at risk of redundancy, and who meet the minimum requirements for the advertised vacancy.### Diversity and Inclusion

The Civil Service is committed to attract, retain and invest in talent wherever it is found. To learn more please see the Civil Service People Plan and the Civil Service Diversity and Inclusion Strategy .Apply and further information

This vacancy is part of the Great Place to Work for Veterans initiative.

The Civil Service welcomes applications from people who have recently left prison or have an unspent conviction. Read more about prison leaver recruitment (opens in new window).

Once this job has closed, the job advert will no longer be available. You may want to save a copy for your records.### Contact point for applicants

Name : Molly Adamat

Email : molly.adamat@hmtreasury.gov.uk

Email : hrrecruitment@hmtreasury.gov.uk

Further information

Complaints Process: Our recruitment process is underpinned by the principle of appointment based on fair and open competition and appointment on merit, as outlined in the Civil Service Commissioners’ Recruitment Principles. If you feel your application has not been treated in accordance with these principles and you wish to make a complaint, you should in the first instance contact HMT by email at: hrrecruitment@hmtreasury.gov.uk. If you are not satisfied with the response you receive, you can contact the Civil Service Commission, which regulates all Civil Service recruitment.