Guidelines for AI Procurement

What is AI and how can it be used in government?

Defines AI as digital technology creating systems that perform tasks typically requiring intelligence, usually involving machines finding patterns in data to complete repetitive work. Most government applications use machine learning, a subset of AI where systems improve performance through experience.

What is the aim of the guidelines?

These guidelines help government departments establish effective AI purchasing practices, offering principles for acquiring AI technology and addressing procurement challenges. The guidance was developed collaboratively and will be updated as government AI use advances.

Who should use the guidelines?

The guidelines target central government departments and other public sector bodies, useful for procurement professionals, technology officers, transformation teams, data specialists, and suppliers seeking to understand government AI standards. Readers should understand procurement rules and use commercial judgment.

How should the guidelines be used?

These guidelines outline themes for assessing AI viability and procurement implementation, emphasizing transparency about tools, data and algorithms. Procurement teams should use these alongside existing digital service, technology and data standards, applying commercial judgment when AI is embedded in non-AI requirements.

1 Include your procurement within a strategy for AI adoption

Organisations should update their technology and data strategies to incorporate AI and use procurement to advance government-wide AI adoption through collaboration and sharing knowledge across departments.

2 Make decisions in a diverse multidisciplinary team

AI projects should involve teams with varied expertise including domain knowledge, commercial skills, engineering, model development, data ethics, and design. Successful suppliers must assemble teams with appropriate skills and diversity to reduce algorithmic bias.

3 Conduct a data assessment before starting your procurement process

Before procuring an AI system, organisations must ensure relevant data will be available and establish governance mechanisms. They should address data flaws and bias beforehand or plan to manage them, and determine how to share data with vendors.

4 Assess the benefits and risks of AI deployment

Procurement documents should clearly define public benefit goals and explain how they drive decision-making, considering human and socio-economic impacts in line with social value principles. Organisations must justify why AI suits their problem, conduct initial impact assessments early, and revisit them at key points.

5 Engage effectively with the market from the outset

Government procurement can foster a fair, competitive market for AI systems. Engage with vendors early during planning, reach out to diverse suppliers including startups and SMEs, and create an open competitive environment that does not impose unnecessary burdens on potential bidders.

6 Establish the right route to market and focus on the challenge rather than a specific solution

7 Develop a plan for governance and information assurance

Establish oversight mechanisms throughout the AI system lifecycle tailored to the risk profile of the project. Refer to existing codes of practice, relevant standards and regulations in your procurement documents, and maximise transparency in decision-making to build user confidence.

8 Avoid Black Box algorithms and vendor lock in

Include explainability and interpretability of algorithms as design criteria, allowing your team and other suppliers to understand system results. Highly interpretable outputs reduce the risk of vendor lock-in and enable engagement with alternative suppliers for future development.

9 Focus on the need to address technical and ethical limitations of AI deployment during your evaluation

Use a multidisciplinary team with broad expertise to evaluate tenders. Assess whether suppliers have identified and addressed bias in data, considered integration with existing services, met governance requirements, and adhered to technical standards.

10 Consider the lifecycle management of the AI system

Recognise that AI system functionality and consequences may only emerge during deployment, requiring ongoing evaluation and communication between buyer and supplier. Include lifespan testing, knowledge transfer, training for non-specialists, and appropriate ongoing support in your requirements.

Multidisciplinary teams

Assemble a team with diverse skills and experience to assess whether AI is a viable solution and to support implementation. Consider roles such as data architect, data scientist, technical architect, security architect and delivery manager, and consult specialists or discovery exercises if the team lacks expertise.

Data assessment and governance

Conduct a discovery into your data before procurement to understand its complexities and limitations. Establish data governance covering access, storage and consent; consider sharing data with suppliers during procurement while protecting privacy through NDAs or anonymisation techniques; and encourage innovative approaches that use less sensitive data.

AI Impact assessment

Organisations must conduct iterative impact assessments starting at the project design stage to identify and mitigate risks. The assessment should cover user needs, social benefits, human impacts, data quality concerns, bias potential, costs, and unintended consequences, with decision points for ongoing review.

Preliminary Market Engagement

Organisations should engage with suppliers including SMEs and community organisations early to understand whether and how AI can address their needs. This engagement must follow public procurement principles, treat all suppliers fairly, and not favour particular solutions or vendors.

Procurement approach and vehicle

Organisations can choose from various routes to acquire AI systems, including established frameworks and innovation-focused procedures. Options range from agile procurement with proof-of-concepts and challenge-based competitions to specialist AI procurement frameworks that include ethical requirements.

Drafting your requirement

When drafting AI procurement specifications, organizations should use output-based requirements supported by detailed problem statements grounded in user needs and performance targets, including clear definition of the challenge, explanation of AI relevance, fit within data strategy and governance, known data limitations with supplier mitigation strategies, and evaluation information about algorithms, variables, techniques, training data sources and model constraints; they should prioritize transparent and interpretable AI development through open standards and licensing agreements to prevent single-supplier dependency. Procurers should also clarify fair and mutually beneficial intellectual property ownership, define integration requirements, specify training and knowledge transfer needs, consider ongoing support and maintenance beyond the initial contract, allocate risk appropriately with clarity on department liability for usage and data handling and supplier liability for technical matters, ensure the system enables reporting of suspected unauthorised behaviour, and keep all requirements transparent and non-discriminatory towards SMEs, VCSEs or suppliers from countries with UK trade agreements.

Process-based governance and auditability

Organisations should implement process-based governance frameworks that define team roles, workflow stages requiring intervention, evaluation timeframes, and clear logging protocols. End-to-end auditability should be enabled through process logs capturing data across all project phases to support interpretable and justifiable AI.

Model testing

Organisations must establish with suppliers how they will continuously monitor model accuracy after deployment, as inaccurate systems can produce erroneous decisions harming citizens.

Knowledge transfer and training

Organisations should assess whether the supplier's knowledge transfer plan will enable staff to use the AI system independently, or establish alternative in-house capacity. Staff must receive sufficient training to operate the system appropriately, and the system should enable reporting of suspected misuse to relevant authorities.

End-of-life

Organisations should plan how to handle decommissioning of AI systems and associated data, including establishing clear responsibilities between themselves and suppliers when contracts end, and must verify their contract management procedures can adequately support these processes. Reliable and auditable methods for data handling throughout the system lifecycle are essential for following these guidelines.