Model AI Governance Framework for Generative AI

Executive summary

Generative AI presents opportunities and risks including bias, misuse, hallucination, copyright concerns and value alignment issues. This framework establishes a systematic and balanced approach across nine dimensions requiring coordination of policymakers, industry, researchers and the public: accountability through allocation of responsibility across the development chain; data quality and fair treatment of training data; trusted development practices with transparency on safety measures; incident reporting systems; third-party testing and independent verification; security measures; content provenance; accelerated R&D on model safety and alignment through global cooperation; and AI for public good including democratised access and sustainable development. The framework builds on prior policy discussions and international insights and will continue to evolve as technology and policy discussions develop.

Accountability accountability

Different players in the generative AI development chain, including model developers, application deployers and cloud service providers, must be responsible to end-users with responsibilities allocated based on control, considering this allocation upfront during development and providing guidance on remedies if problems are discovered later. Model developers should require deployers to obtain models from trusted sources to reduce tampering risks and bear responsibility for their models and deployment, with shared responsibility models clarifying redress when problems occur. Additional measures such as indemnity and insurance should cover unanticipated issues and protect end-users, and legal frameworks may need updating to address emerging risks fairly with no-fault insurance potentially serving as a safety net for residual issues.

Data data

Data is essential for training AI models but its use raises personal data protection and copyright compliance questions; policymakers should clarify how personal data legislation applies to generative AI to permit legitimate use while safeguarding rights through consent guidance and business practices, and privacy-enhancing technologies from anonymisation to emerging approaches can enable development while maintaining confidentiality. Copyright owners and stakeholders have competing concerns about training dataset use including remuneration, licensing, and opt-out systems, with some arguing training does not infringe copyright; policymakers should foster stakeholder dialogue for balanced solutions while AI developers implement data quality control and governance practices including consistent annotation and debiasing. Governments should expand trusted dataset pools and curate representative training data repositories for local contexts including low-resource languages to support safer and culturally representative models.

Trusted development and deployment trusted development and deployment

Model developers and deployers should implement safety best practices across the development lifecycle including fine-tuning techniques like RLHF, risk assessment, RAG and few-shot learning, with transparent disclosure comparable to food labels across seven standardised areas: training data sources and processing, training infrastructure and environmental impact, evaluation outcomes, implemented safety measures, known risks and mitigation steps, intended use scope, and user data handling and protection. Additional disclosure detail can be customised for advanced models while balancing transparency with proprietary information protection, and governments may define risk thresholds above which further oversight applies. Core safety evaluations for large language models must cover robustness, accuracy, bias, toxic outputs and data governance as identified by AI Verify Foundation and IMDA; sectors may require additional or stricter evaluations for high-risk applications and individual deployers may conduct domain-specific assessments, while industry and sectoral authorities must collaboratively enhance evaluation standards and methods ensuring compatibility between universal standards and sector-particular rules.

Incident reporting incident reporting

Before incidents occur, developers should enable vulnerability reporting through channels and security research programmes, applying industry practices like providing 90 days to patch before public disclosure. After incidents occur, organisations need internal processes for timely notification and remediation to relevant authorities and the public where proportionate and required by law, with reporting calibrated to the specific local context. Providers of high-risk artificial intelligence systems must report serious incidents to market surveillance authorities within 15 days of awareness, where serious incident means any malfunction causing death, serious health damage, critical infrastructure disruption, fundamental rights breaches, or serious property or environmental harm.

Testing and assurance testing and assurance

Third-party testing should define reliable and consistent methodologies with clear scope to complement internal testing, and identify independent entities with qualified capabilities to conduct testing. Standards organisations like ISO/IEC and IEEE can codify testing approaches, while accreditation mechanisms can verify independence and competency, with robustness and fairness forming a baseline for third-party assessment.

Security security

Organisations developing generative AI should embed security considerations throughout the development life cycle, adapting traditional security-by-design principles to address unique challenges such as natural language input and probabilistic model behaviour. They should develop new safeguards including input filters to detect unsafe prompts and digital forensics tools to identify malicious code within models, with reference to resources like MITRE's Adversarial Threat Landscape.

Content provenance content provenance

Digital watermarking and cryptographic provenance are technical approaches to label and verify AI-generated content to address concerns about deepfakes and misinformation, though current watermarking solutions lack interoperable standards and the C2PA standard for open content provenance tracking may be insufficient without complementary enforcement mechanisms. Publishers and platforms should support embedding and displaying watermarks and provenance details to help users verify content authenticity, and standards should specify what types of edits trigger labelling. Key stakeholders should work with policymakers to increase public understanding of content provenance tools and simplified provenance displays.

Safety and alignment R&D safety and alignment R&D

Accelerated research and development in model safety and alignment is needed to keep pace with generative AI advancements, including forward alignment techniques like RLAIF and backward alignment approaches such as emergent capability testing and mechanistic interpretability. Establishing safety research institutes and fostering global cooperation can prioritise impactful research areas and leverage limited talent and resources. Singapore's Digital Trust Centre addresses trustworthy AI technologies and broader digital trust concerns through research funding of S$50 million from IMDA and the National Research Foundation, established in June 2022 to direct the nation's research on trusted AI and related technologies.

AI for public good AI for public good

Governments should partner with companies and communities to democratize access to generative AI through digital literacy initiatives on safe use and deepfake identification, support SMEs in AI adoption, and coordinate with AI developers to deploy AI in public services such as education and healthcare through responsible data sharing and access to computing resources. Workforce upskilling across industry, government and educational institutions is needed to enable workers to effectively use AI tools and mitigate job displacement, with organizations adopting generative AI providing training to help employees navigate job changes. The generative AI ecosystem must address sustainability by developing energy-efficient technology, tracking carbon footprint during model training and inference, and hosting workloads in data centres using green energy, with AI developers and equipment manufacturers leading research on green computing.

Conclusion

The Framework's nine dimensions provide a foundation for global policy discussions on generative AI governance; policymakers must collaborate with industry, researchers and aligned jurisdictions to advance accountability, transparency, fairness, robustness and security, enabling a trusted AI ecosystem where AI serves the public good.

Acknowledgements

• KPMG

• a*star

Further development

Implementation guidelines and resources will be developed to apply the Framework's nine dimensions through ongoing stakeholder engagement. Singapore established the AI Verify Foundation to coordinate a global open-source community in developing AI governance testing tools, standards and best practices to foster trusted AI development; IMDA serves as architect of Singapore's digital future by integrating AI governance with economic development and social inclusion.