The Institute's Defence & National Security programme builds capability that gets used, not just published. We are creating a new Applied Engineering team to add to the delivery engine behind that: a cross-functional group of software engineers, data scientists, and data engineers, who take research from across the Institute and academia to turn it into deployable tools that analysts and decision-makers rely on. The team's work will make sense of disparate data streams and produce the pipelines and tooling that let people make real-world decisions with confidence.
ROLE PURPOSE
The team will work with large, varied and often poorly structured datasets, including documents, structured records, and imagery. These data will come from multiple sources, in inconsistent formats and with varying provenance. Making them reliable and usable will be central to the team's work, and this role will lead that effort.
As the team's first Senior Data Engineer, you will design and build the pipelines and infrastructure needed to turn heterogeneous data into reliable, well-structured resources that analysts and engineers can use with confidence. The immediate priority will be the data layer supporting analyst-facing tools: curated datasets, clear data models, serving layers and query interfaces. As the team's machine learning work develops, this foundation will also need to support feature pipelines, dataset versioning and evaluation data. You will establish and maintain the team's data platform, including storage, compute, orchestration and access control, across cloud and on-premise environments. Where systems are owned by partners, you will design within their technical and security constraints. You will join the team's founding cohort of [N], reporting to the Lead Applied Engineer and working closely with applied engineers, data scientists and Defence stakeholders.
This is a senior individual-contributor role with substantial technical responsibility. It would suit an experienced data engineer who has built and operated data systems at scale, can set clear technical direction, and is able to adapt their approach to the practical constraints of the deployment environment.
HOW YOU WILL MAKE AN IMPACT
- Design, build and operate pipelines that ingest, validate, clean and structure large volumes of varied, mixed-modality data from Defence partners, with data quality, lineage and reproducibility built in from the outset.
- Build the data products that support the team's tools, including data models, curated datasets, serving layers and query interfaces that applied engineers can build against and analysts can use directly.
- Design the data layer to support future machine learning work, including feature pipelines, dataset versioning and evaluation datasets, while maintaining a clear focus on current analyst needs.
- Establish and maintain the team's data platform across cloud and on-premise environments, covering storage, compute, orchestration, access control and infrastructure as code, using well-understood components that the team can operate reliably.
- Work with the team’s applied engineers and partner staff to understand how data will be used, and design within the technical and security constraints of partner environments.
- Set and maintain the team's data engineering standards, including testing, schema and contract management, documentation and secure data handling, supported by effective code review.
- Help shape the team's data architecture and working practices from the outset, mentor colleagues as the team grows, and advise on how data and platform engineering responsibilities should develop.
ABOUT YOU
- Substantial experience designing, building and operating production data pipelines over large, heterogeneous and poorly structured data – multiple modalities, batch and streaming – with attention to data quality, lineage and reproducibility.
- Strong Python software engineering: testing, packaging, code review, CI/CD, and the judgement to know when a notebook needs to become a service.
- Experience standing up and operating data infrastructure – storage, compute, orchestration, access control – using infrastructure-as-code, in cloud and/or on-premise environments.
- Experience designing data models and building data products for others to consume: curated datasets, serving layers and query interfaces that downstream engineers and analysts can use and trust.
- Experience handling data under security, access-control and audit constraints, and designing systems that respect them by default.
- Ability to work with downstream engineers, data scientists, analysts and partner engineers to understand how data will be used, and to make pragmatic trade-offs when partner environments constrain the design
Please see the Job Description for a full breakdown of the role and person specification
APPLICATION PROCEDURE
If you are interested in this opportunity, please click the apply button below. You will need to register on the applicant portal and complete the application form, information required is outlined clearly in the full Job Description. If you have questions about the role or would like to apply using a different format, please contact us at [email protected].
As this role requires eligibility for Developed Vetting (DV) clearance, it is an essential part of the application process that you include the following information as part of your cover letter:
- Your current nationality
- Your nationality at birth
- Other nationality (include dual nationality if applicable)
- Confirmation that you have been residing in the UK for the past 5 years (if you haven’t, please provide details of when and where you resided and the reason)
- Country where you were born.
- County in which you were born.
- Town where you were born.
Please note, if these details are not provided, we will be unable to progress with your application for this role.
TERMS AND CONDITIONS
This full-time post is offered on a permanent basis. The annual salary range is £57,000 - £69,000 plus excellent benefits, including flexible working and family friendly policies, Employee-only benefits guide | The Alan Turing Institute
Given the close working relationship with government partners, this role requires regular on-site presence with Defence partners in London 3 days per week, with possible occasional travel to other sites in the UK, in addition to the Institute's standard office attendance expectations.
Eligibility for DV clearance is an essential requirement for this role. Eligibility criteria and further information on the process can be found on the UK Government security vetting website. Please note that we require you to provide essential information in your cover letter in order to progress your application. Details of this are in the Application Procedure section.
Please note, if these details are not provided, we will be unable to progress with your application for this role.
EQUALITY, DIVERSITY AND INCLUSION
We value diversity of background, experience, and perspective, and are proud to be an inclusive employer. We warmly encourage applications from all backgrounds, particularly from groups currently under-represented in our sector. If you feel passionate about this role but don’t meet every single requirement, please apply – we recognise that great candidates may bring strengths beyond the criteria listed.
We are committed to making sure our recruitment process is accessible and inclusive. This includes making reasonable adjustments for candidates who have a disability or long-term condition. Please contact us at [email protected] to advise us how we can assist you.
Please note all offers of employment are subject to obtaining and retaining the right to work in the UK and satisfactory pre-employment security screening which includes a DBS Check. Full details on the pre-employment screening process can be requested from [email protected].