Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile
United States · Framework · National Institute of Standards and Technology
High impact. A reviewer confirms each change before it is listed here.
Our summary
This document provides practical guidance for organizations managing risks specific to generative AI systems, based on the earlier AI Risk Management Framework released in 2023. It identifies risks that are new to or intensified by generative AI, such as confabulation, dangerous content generation, and data privacy concerns, and recommends management approaches focused on governance, training data transparency, testing before deployment, and incident reporting. The profile applies across sectors and uses cases that rely on large language models and similar foundational models.
Written by the Index in our own words. It is not legal advice; the official text is the authority.
Inside this policy
Every section of this policy in order. Our notes summarise each one in plain language and do not quote the official text, which is the authority.
Drafted by the Index An automated summary of the official text, not yet reviewed by a person.Reviewed Checked by a person against the official text.
5 sections
Jump to a chapter
Introduction
This companion resource to the AI Risk Management Framework supports voluntary adoption by organizations seeking to incorporate trustworthiness throughout all stages of generative AI lifecycle activities, from initial conception through operational deployment. The profile offers guidance tailored to generative AI risks based on organizational goals, applicable regulations, and risk management priorities. Future versions of this profile will add more AI Risk Management Framework categories and actions as risks become clearer, and NIST will develop a glossary of generative AI risk management terms for its Trustworthy and Responsible AI Resource Center.
Overview of Risks Unique to or Exacerbated by GAI
Generative AI risks span lifecycle stages from model design to outputs and human behavior, including harmful bias and output homogenization, human-AI configuration issues, confabulation and false information, information integrity threats, information security vulnerabilities from automated attacks and prompt injection, intellectual property violations through unauthorized replication, privacy violations from training on personal data and information memorization or inference, generation of obscene or abusive imagery including synthetic child sexual abuse material and nonconsensual intimate images, assistance to malicious actors in accessing weapons knowledge, violent or hateful content creation, disparate performance across population subgroups or languages, over-reliance or under-reliance on systems, unintentional or deliberate spread of false or misleading information, cybersecurity vulnerabilities and lowered attack barriers, and facilitation of illegal intimate imagery and child sexual abuse material production. Environmental and equity harms include substantial energy consumption and carbon emissions from training, and amplification of harmful biases at scale. Opacity and third-party component integration reduce traceability and accountability.
Suggested Actions to Manage GAI Risks
Organizations should establish policies and processes across the generative AI lifecycle to document training data provenance, evaluate model capabilities and safety, define risk tiers and performance thresholds, prevent illegal content generation, maintain system inventories and incident response protocols with diverse team composition, and conduct regular red-teaming and adversarial testing. For third-party risks, organizations must implement supplier risk assessment frameworks, update acquisition processes to assess intellectual property and data privacy, establish incident response plans with regular rehearsals, and ensure contract terms specify liability assignment and service level agreements. Organizations should also allocate resources for feedback processes, monitor generated content for privacy and harmful impacts in real-time, apply fairness metrics across demographic groups, conduct impact assessments on affected communities, document training data quality and content provenance, implement content filters, establish post-deployment monitoring with incident procedures, and verify personnel can effectively evaluate performance and escalate issues promptly.
Appendix A Primary GAI Considerations
Organizations should manage generative AI risks through twenty-five key considerations including accessibility, data protection, auditing, monitoring, and workforce diversity, with acceptable use policies for human-AI teamwork and third-party risk controls. They should address measurement gaps in pre-deployment testing through structured public feedback, participatory engagement, field testing, and AI red-teaming with diverse teams, and track provenance through digital watermarking, metadata recording, and fingerprinting to distinguish AI-generated from human-generated content. Organizations must document system objectives and limitations, establish incident reporting guidelines with documented personnel responsibilities, maintain incident records and logs, track version history and metadata, and document third-party plugins to enable incident identification and prevention.
Appendix B References
Controls
What this policy requires, forbids, grants or penalises, one control at a time and in our own words. Each control links to the official text, which is the authority. This is not legal advice.
Drafted by the Index Drafted automatically from the official text, not yet reviewed by a person.Reviewed Checked by a person against the official text.
Themes covered 0 of 35
Choose a theme to see only its controls. About the themes
Risk and scope 0 of 4
- Risk classification, not covered
- Prohibited practices, not covered
- Risk management system, not covered
- Impact assessment, not covered
Data and privacy 0 of 4
- Data governance and quality, not covered
- Data subject rights and consent, not covered
- Cross-border data transfer, not covered
- Copyright and training data, not covered
Transparency and records 0 of 4
- Transparency and disclosure to users, not covered
- Labelling of synthetic content, not covered
- Technical documentation, not covered
- Record keeping and logging, not covered
Technical safeguards 0 of 4
- Human oversight, not covered
- Accuracy, robustness and cybersecurity, not covered
- Bias and non-discrimination, not covered
- Testing and evaluation, not covered
Assurance and lifecycle 0 of 4
- Conformity assessment and certification, not covered
- Registration and notification, not covered
- Incident and breach reporting, not covered
- Post-market monitoring and corrective action, not covered
Rights and protected groups 0 of 4
- Automated decision-making rights, not covered
- Children and vulnerable groups, not covered
- Biometric identification and surveillance, not covered
- Whistleblowing, complaints and redress, not covered
Roles and organisation 0 of 6
- Supply chain and provider-deployer duties, not covered
- General purpose and foundation models, not covered
- Accountability and governance roles, not covered
- Training and AI literacy, not covered
- Public sector use and procurement, not covered
- Innovation support and sandboxes, not covered
Enforcement and sectors 0 of 5
- Enforcement, penalties and regulators, not covered
- Sector rules: health, not covered
- Sector rules: finance and insurance, not covered
- Sector rules: employment, not covered
- Sector rules: education, not covered
The full list of controls could not be loaded, so only the first 0 are shown.
Change history
No changes recorded since we began monitoring this policy.