Artificial Intelligence and Privacy Report

About this report

This report addresses how artificial intelligence development and use affect privacy by examining four key data protection concerns: fairness and discrimination, purpose limitation, data minimisation, and transparency. The report targets people working with or interested in artificial intelligence, including engineers, social scientists, and lawyers, and also discusses the supervisory role of the Norwegian Data Protection Authority regarding AI applications. This section acknowledges the organisations and experts who contributed to developing the report through their experience and assessment of artificial intelligence and data protection matters.

Legal sources and use of terminology

This section defines AI as covering machine learning and deep learning, establishes the EU General Data Protection Regulation and the Norwegian Personal Data Act as the legal basis, and references guidance from the Article 29 Working Party on automated decisions and profiling.

Artificial intelligence and data protection

Artificial intelligence systems learn from often personal data to solve complex problems through techniques such as machine learning and deep learning, which enable computers to build algorithms and self-learning systems from accumulated data. New data protection regulations entering into force in May 2018 strengthen privacy rights and increase responsibilities for organisations processing personal data, creating tension between growing demand for data and intensified transparency requirements. As AI development is still in early stages, it is possible and necessary to ensure AI technologies comply with data protection rules while safeguarding fundamental personal data protection rights.

How does artificial intelligence work?

AI systems have two key privacy-relevant features: they make decisions independently, and they learn from experience. Systems acquire learning experience through input data in various formats such as images, text, speech or numbers, which may or may not be linked to individuals.

Machine learning

Machine learning operates in three steps: selecting data containing patterns, identifying those patterns, and generating a model that recognises patterns in new data. Supervised learning uses labelled data split typically into 80 percent for training and 20 percent for verification, with domain expertise important for selecting relevant features and generalisation required to ensure the model performs accurately on unknown data rather than only on training data, with feature quality potentially mattering more than data quantity. Unsupervised learning groups unlabelled data by identifying patterns rather than predefined categories, and reinforcement learning improves through trial and error with minimal or no pre-labelled data, but unsupervised models cannot classify data into categories beyond those discovered during training, making representative training data essential.

Results of learning

Defines a model as an aggregate representation of training data that produces outputs such as labels or probability scores, rather than storing source data directly. Describes static models that do not change during use, requiring all retraining in test environments before deployment, and dynamic models that improve from input data during operation, offering better accuracy but less control over development.

The more training data the better?

Training data quality, feature selection, and representativeness often matter more than quantity; too many features create noise and excessive data needs, while deep learning requires less careful feature selection but larger volumes; feature engineering transforms data presentation through techniques such as extracting information from dates, grouping time periods, converting to consistent formats and scales, and removing imbalances from extreme values. Starting with limited data and monitoring model accuracy as new data is added helps determine when additional data ceases to improve performance, consistent with data minimisation; the required amount varies by machine learning type, data quality, selected features, and the accuracy level the application demands, with different standards for diagnosing fatal illnesses versus targeting advertisements.

The Black Box

Machine learning often produces results without explanation of how specific outcomes were derived, making it difficult to identify which features drove decisions. Decision trees offer high transparency when based on manageable data, whereas neural networks with multiple layers, loops that recirculate data, or components with short-term memory present greater difficulty in understanding and verifying results. The complexity and size of neural networks, exemplified by 152-layer networks used in image recognition, far exceeds what humans can comprehend or examine without specialised tools.

Artificial intelligence meets the GDPR

The GDPR applies to AI development and use involving personal data, whether directly identifying individuals or indirectly linked to them through combinations of identifying characteristics. Data controllers have duties under the GDPR when developing or using artificial intelligence systems.

Fundamental principles of data protection

GDPR requires that personal data be processed lawfully, fairly and transparently; used only for specified justified purposes; limited in scope to necessary information; kept accurate and current; retained only as long as needed; and protected with appropriate security. Data controllers must demonstrate accountability for compliance with these principles.

Algorithmic bias meets the fairness principle

AI systems may produce biased or discriminatory outcomes if training data reflects real-world bias or lacks relevance to the application. The fairness principle requires controllers to prevent arbitrary discrimination through appropriate data selection, algorithm development and testing, with enforcement through investigative powers of data protection authorities.

Artificial intelligence meets the principle of purpose limitation

When previously collected personal data is repurposed for AI development or other new uses, controllers must assess whether the new purpose aligns with the original purpose; if not, fresh consent or a different processing basis is required, though the principle can be set aside for scientific or historical research, statistical purposes, or public archival use. When assessing compatibility between original and new purposes, controllers must consider the connection between purposes, collection context, data subject's reasonable expectations, nature of the data, consequences for the data subject, and whether appropriate safeguards apply, with different considerations for static models with clear separation between development and use versus continuously improving models where development and use blur.

Artificial intelligence meets data minimisation

Controllers must ensure collected data is fitting, pertinent, and not in excess of what the processing purpose requires, though AI development often makes it difficult to predict what data the algorithm will need or learn. Data minimisation also demands proportionality in how far personal data processing intrudes on privacy, achievable through pseudonymisation or encryption; developers must document their assessment of relevant data and least-invasive methods, and conduct ongoing review rather than relying on initial predictions.

The black box meets the principle of transparent processing

Controllers must be transparent about how they use personal data by informing data subjects of processing purposes, methods, legal basis and their rights in clear language readily available to them, and must provide specific information about automated decision-making including the controller's identity, legal basis, data categories and inspection rights. Solely automated decisions are generally prohibited unless necessary to conclude a contract, authorised by law, or made with explicit data subject consent, and when they occur, controllers must safeguard the data subject's rights and interests, enable a human to make the final decision, and provide a right of appeal; automated decisions using sensitive personal data require consent or legal permission. Data subjects must be informed before automated processing begins that their data is collected for automated decision-making, including the model's underlying logic, significance and anticipated impacts at a level they can understand, and controllers must supply sufficient information to enable data subjects to understand decisions and what would change the outcome, along with information about how to oppose decisions through appeals or by requesting human review; the transparency principle requires controllers to provide similar explanations as for automated decisions when a human decides using an automated tool's output without automated decision-making conditions being met.

Controlling the algorithms

As artificial intelligence increasingly makes decisions about individuals' access to services, the complexity of these systems makes it difficult for people to understand or verify how decisions are made, creating a dependency on organisations to comply with data protection rules.

The DPA’s supervisory competence

The Data Protection Authority has the power to conduct investigations to verify compliance with regulations, and may request documentation, access to premises and equipment, and access to personal data being processed, with national procedural rules governing such access.

Investigating the use of AI

The Data Protection Authority inspects artificial intelligence systems under the same rules as other data processing, with particular focus on compliance with data protection principles, proper processing bases, data minimisation, fair treatment, and subject notification. The Authority may examine training data and its use, test results and audits for unlawful or discriminatory processing, and whether privacy by design principles were applied.

How deep can an investigation go?

Organisations must document and demonstrate that their data processing follows regulatory requirements and be able to explain how systems handle personal information. The DPA can verify details, ask for demonstrations, and impose fines or processing bans if organisations cannot account for their practices. As automated decision-making increases, the DPA expects to conduct more technical system inspections, while organisations face greater responsibility for internal controls.

How to inspect a “black box”?

While rule-based algorithms are relatively transparent, deep learning and neural networks lack clarity about their internal processes, requiring substantial technical expertise to understand and inspect. The DPA may need to hire external specialists to conduct thorough reviews of AI-based systems to ensure they do not reinforce social inequalities, cause discriminatory outcomes, or enable unlawful data reuse.

Solutions and recommendations

Accountability is a core principle requiring data controllers and processors to ensure compliant processing. The section introduces important tools including data protection impact assessment and privacy by design, both central to GDPR compliance when developing or deploying AI systems.

Assess the data protection impact, and build privacy into your system!

Organisations must conduct data protection impact assessments when using automated decision-making, processing sensitive data at scale, or monitoring public areas, considering the nature, scope and context of processing. They must also embed privacy protection into system design from the start, including protective standard settings and data minimisation measures, with DPA involvement if high risks cannot be reduced.

Tools and methods for good data protection in AI

Methods for good data protection in AI include reducing training data requirements through Generative Adversarial Networks for synthetic data generation, federated learning for distributed model improvement without sharing user data, and matrix capsules; protecting privacy without reducing dataset size through differential privacy, homomorphic encryption, transfer learning, and the RAIRD system for aggregated metadata and data queries; and addressing lack of transparency through Explainable AI and LIME, which produces human-understandable explanations by showing which parts of input data are relevant to decisions.

Recommendations for privacy friendly development and use of AI

AI developers, system suppliers, and purchasing organisations should conduct research on privacy-protective design, adopt multidisciplinary teams, understand GDPR obligations, limit training data to what is necessary, document privacy by design compliance, perform risk assessments and DPIAs, test systems for regulatory compliance and discrimination, and establish protections for data subject rights including information, access, deletion, rectification, and data portability. End users have rights to comprehensible information about processing, to give and withdraw consent, to access their data, to rectify or delete inaccurate information, to object to processing which must stop for direct marketing without justification, and to demand processing be limited. Authorities should ensure the public sector exemplifies responsible AI use with awareness of ethical and privacy consequences and expertise in privacy by design, allocate research funding for data protection-compliant technology, ensure enforcement authorities have relevant expertise with cross-sectoral knowledge sharing, and keep legislation aligned with technological developments.