Skip to main content

On-demand webinar coming soon...

Blog

Navigating three privacy pitfalls of AI adoption

Trustworthy AI extends well beyond just privacy, spanning security and ethical considerations as well. But in order to get AI right, you need to first get data privacy right. Learn more about three common privacy pitfalls in AI adoption, and how you can avoid them

Bex Evans
Senior Product Marketing Manager
June 10, 2024

Person in a yellow hooded jacket rides an escalator

Gartner® predicts that “By 2026, more than 80% of enterprises will have used generative artificial intelligence (GenAI) application programming interfaces (APIs) or models, and/or deployed GenAI-enabled applications in production environments, up from less than 5% in 2023”1  

At the same time, according to Gartner Hype Cycle Methodology, “Interest wanes as experiments and implementations fail to deliver. Producers of the technology shake out or fail. Investments continue only if the surviving providers improve their products to the satisfaction of early adopters.”2

Research from Forrester identifies data privacy and security concerns as the top barrier to generative AI adoption. The promotion of trustworthy AI extends well beyond the privacy domain, spanning security and ethical considerations as well. But in order to get AI right, you first need to get data privacy right.

 

3 privacy pitfalls in AI adoption

That’s because AI is an amplifier of existing privacy gaps – a single misconfigured access point can get exponentially more problematic when its exposed to an AI system. When navigating the role of data privacy for AI systems, there are three privacy pitfalls to be aware of:

  1. Purpose limitation: AI incentivizes secondary uses of data, which means using data for a different purpose than it was originally intended for.  

  2. Proportionality: AI requires robust datasets to ensure accuracy, fairness, and quality which may conflict with data minimization practices

  3. Business continuity: Data comes in, but it doesn’t go out – if you don’t put the right data in then someday that system won’t be available for you.

 

Infographic showing three potential pitfalls in adopting AI for your business.

 

Here, we’ll explore each pitfall in more detail and look at some common practices for addressing the risks associated with it.  

 

Purpose limitation in the context of AI

A common scenario that illustrates the importance of responsible data usage is the collection of birth dates for identity verification. In contexts such as two-factor authentication or banking, verifying a person's identity is crucial. This process often involves collecting sensitive information – like their birth date.

However, possessing this data for identity verification does not automatically grant permission to use it for other purposes. For instance, if a marketing team wants to use birth dates to send out birthday promotions, they must first obtain explicit consent from the individuals involved. Without such consent, using birth dates for marketing is a violation of data privacy principles.

 

Challenges with generative AI and consent

The advent of large language models (LLMs) and generative AI adds another layer of complexity to this issue.  

Providing clear and detailed information upfront is essential for obtaining informed consent, meaning individuals must understand in plain language how their data will be used to provide informed consent.  

One significant challenge organizations face is striking a balance between offering enough context and avoiding overwhelming individuals with lengthy terms and conditions that they might – and often do – simply skip. Effective communication in the audience's language is vital to ensure that consent is truly informed and not just a formality.

 

Proportionality and balancing data robustness and minimization in the context of AI

Consider the use of a resume scanning tool designed to streamline hiring practices. Historically, organizations might exclude sensitive information such as race, gender, and ethnicity from resumes to minimize privacy risks and reduce overall risk if the application were to experience a breach. However, excluding these data points can also prevent the identification and mitigation of bias within the hiring process.

Bias can persist even when sensitive data is omitted, as other indirect factors might contribute to biased outcomes. To accurately analyze and ensure fair representation in a dataset, it’s necessary to document and record sensitive data points. This allows for proactive monitoring of the system for fairness and the detection of potential biases.

A common challenge faced by organizations today is that they may not have collected race, gender, or ethnicity information initially due to privacy concerns or the potential discomfort it might cause applicants. Consequently, they lack the data needed to perform thorough fairness assessments.

 

Privacy-enhancing technologies (PETs)

To address these challenges, privacy-enhancing technologies (PETs) can be employed. PETs such as differential privacy, synthetic data, homomorphic encryption, and multi-party computation help protect sensitive inputs while enabling the necessary analysis. These technologies allow for the safeguarding of individual privacy during data processing and model training.

However, it is important to recognize that there is no one-size-fits-all solution. The choice of PET depends on the specific use case and the infrastructure in place. In many scenarios, a combination of multiple PETs may be required to adequately protect privacy while maintaining the utility of the data.

 

Business continuity and future-proofing data in the context of AI

For consent to be truly lawful, it must be freely given and able to be withdrawn at any time. This principle poses a significant challenge when a consumer requests the deletion of their data from a business-critical system. If these processes rely on AI systems trained on personal data, the removal of such data can disrupt business continuity.

AI models, much like human brains, can’t simply forget information once it has been learned. The only solution is to roll back to a previous version of the model, trained before the data in question was included, and then retrain the model without it.  

This necessitates robust documentation on model versioning, dataset versioning, and detailed tracking of data categories and identifiers to ensure the data can be accurately removed.

 

Data governance and model retraining

The complexities associated with enforcing data governance and model retraining highlight the importance of thorough documentation and precise version control. This involves maintaining detailed logs of model versions, datasets, and the identifiers used to track individual data points. When a data subject revokes consent, these records allow for the targeted rollback and retraining of models.

 

Retrieval-augmented generation (RAG)

Given these data governance challenges, there is a compelling case for employing retrieval-augmented generation (RAG). RAG involves retrieving facts from an external knowledge base to ground LLMs in the most accurate and up-to-date information. This approach offers several benefits:

  1. Flexibility: External data sources used in RAG can be updated without the need to retrain the entire model.

  2. Accuracy: By retrieving information from trusted sources, RAG enhances factual accuracy and significantly reduces the chances of AI-generated hallucinations.

  3. Accessibility: RAG lowers the barrier to adopting customizable generative AI services as it doesn’t require advanced data science techniques.

  4. Specificity: RAG is well-suited for retrieving specific information to provide accurate answers.

  5. Cost: Using external knowledge as a prompt input is more cost-effective than fine-tuning the entire LLM.

By utilizing RAG, organizations can maintain control over the data input at the point of prompt rather than continuously retraining models. This approach helps ensure business continuity and compliance with data privacy regulations, even when individual data points are removed due to withdrawn consent.

 

1 Gartner Press Release, “Gartner Says More Than 80% of Enterprises Will Have Used Generative AI APIs or Deployed Generative AI-Enabled Applications by 2026,” October 11, 2023. https://www.gartner.com/en/newsroom/press-releases/2023-10-11-gartner-says-more-than-80-percent-of-enterprises-will-have-used-generative-ai-apis-or-deployed-generative-ai-enabled-applications-by-2026

2 Gartner Research Methodologies, Gartner Hype Cycle, June 4th, 2024, https://www.gartner.com/en/research/methodologies/gartner-hype-cycle

GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.


You may also like

Webinar

AI Governance

From build to buy: Exploring common approaches to governing AI

In this webinar, we'll navigate the intricate landscape of AI Governance, offering guidance for organizations whether they're developing proprietary AI systems or procuring third-party solutions.

July 10, 2024

Learn more

Webinar

Privacy Management

Scaling to new heights with AI Governance

Join OneTrust experts to learn about how to enforce responsible use policies and practice “shift-left” AI governance to reduce time-to-market.

June 25, 2024

Learn more

Webinar

AI Governance

AI Governance Leadership Webinar: Best Practices from IAPP AIGG with KPMG

Join out webinar to hear about the challenges and solutions in AI governance as discussed at the IAPP conference, featuring insights and learnings from our industry thought leadership panel.

June 18, 2024

Learn more

Webinar

AI Governance

Colorado's Bill on AI: Protecting consumers in interactions with AI systems

Colorado has passed landmark legislation regulating the use of Artificial Intelligence (AI) Systems. In this webinar, our panel of experts will review best practices and practical recommendations for compliance with the new law.

June 11, 2024

Learn more

Webinar

AI Governance

Governing data for AI

In this webinar, we’ll break down the AI development lifecycle and the key considerations for teams innovating with AI and ML technologies.

June 04, 2024

Learn more

Report

AI Governance

GRC strategies for effective AI Governance: OCEG research report

Download the full OCEG research report for a snapshot of what organizations are doing to govern their AI efforts, assess and manage risks, and ensure compliance with external and internal requirements.

May 22, 2024

Learn more

Report

AI Governance

Global AI Governance law and policy: Jurisdiction overviews

In this 5-part regulatory article series, OneTrust sponsored the IAPP to uncover the legal frameworks, policies, and historical context pertinent to AI governance across five jurisdictions: Singapore, Canada, the U.K., the U.S., and the EU.

May 08, 2024

Learn more

Webinar

AI Governance

Embedding trust by design across the AI lifecycle

In this webinar, we’ll look at the AI development lifecycle and key considerations for governing each phase.

May 07, 2024

Learn more

Webinar

AI Governance

Navigating AI policy in the US: Insights on the OMB Announcement

This webinar will provide insights for navigating the pivotal intersection of the newly announced OMB Policy and the broader regulatory landscape shaping AI governance in the United States. Join us as we unpack the implications of this landmark policy on federal agencies and its ripple effects across the AI ecosystem.

April 18, 2024

Learn more

Webinar

AI Governance

Data privacy in the age of AI

In this webinar, we’ll discuss the evolution of privacy and data protection for AI technologies.

April 17, 2024

Learn more

Resource Kit

AI Governance

OneTrust's journey to AI governance resource toolkit

What actually goes into setting up an AI governance program? Download this resource kit to learn how OneTrust is approaching our own AI governance, and our experience may help shape yours.

April 11, 2024

Learn more

Interactive Tool

Privacy Management

OneTrust Data Privacy Maturity Model self-assessment

This self-assessment will help you to gauge the maturity of your privacy program and understand the areas the areas of improvement that can further mature your privacy operations.

April 01, 2024

Learn more

Webinar

AI Governance

AI in (re)insurance: Balancing innovation and legal challenges

Learn the challenges AI technology poses for the (re)insurance industry and gain insights on balancing regulatory compliance with innovation.

March 14, 2024

Learn more

Webinar

Privacy Management

Fintech, data protection, AI and risk management

Watch this session for insights and strategies on buiding a strong data protection program that empowers innovation and strengthens consumer trust.

March 13, 2024

Learn more

Webinar

Privacy Management

Managing cybersecurity in financial services

Get the latest insights from global leaders in cybersecurity managment in this webinar from our Data Protection in Financial Services Week 2024 series.

March 12, 2024

Learn more

Webinar

AI Governance

Government keynote: The state of AI in financial services

Join the first session for our Data Protection in Financial Services Week 2024 series where we discuss the current state of AI regulations in the EU.

March 11, 2024

Learn more

White Paper

AI Governance

Getting started with AI governance: Practical steps and strategies

Download this white paper to explore key drivers of AI and the challenges organizations face in navigating them, ultimately providing practical steps and strategies for setting up your AI governance program.

March 08, 2024

Learn more

Webinar

AI Governance

Revisiting IAPP DPI Conference – Key global trends and their impact on the UK

Join OneTrust and PA Consulting as they discuss key global trends and their impact on the UK, reflecting on the topics from IAPP DPI London.

March 06, 2024

Learn more

Webinar

AI Governance

AI regulations in North America

In this webinar, we’ll discuss key updates and drivers for AI policy in the US; examining actions being taken by the White House, FTC, NIST, and the individual states. 

March 05, 2024

Learn more

In-Person Event

Responsible AI

Data Dialogues: Implementing Responsible AI

Learn how privacy, GRC, and data professionals can assess AI risk, ensure transparency, and enhance explainability in the deployment of AI and ML technologies.

February 23, 2024

Learn more

AI Governance

Catch it Live: See the All-New Features in OneTrust's Winter Release

See the latest OneTrust platform features that improve on customers' ability to build trust, ensure compliance, and manage risk.

February 22, 2024

Learn more

Webinar

AI Governance

Global trends shaping the AI landscape: What to expect

In this webinar, OneTrust DataGuidance and experts will examine global developments related to AI, highlighting key regulatory trends and themes that can be expected in 2024.

February 13, 2024

Learn more

eBook

Privacy Management

Understanding the Data Privacy Maturity Model

Data privacy is a journey that has evolved from a regulatory compliance initiative to a customer trust imperative. This eBook provides an in-depth look at the Data Privacy Maturity Model and how the business value of a data privacy program can realised as it matures.

February 07, 2024

Learn more

Webinar

AI Governance

The EU AI Act

In this webinar, we’ll break down the four levels of AI risk under the AI Act, discuss legal requirements for deployers and providers of AI systems, and so much more.

February 06, 2024

Learn more

Webinar

Responsible AI

Preparing for the EU AI Act: Part 2

Join Sidley and OneTrust DataGuidance for a reactionary webinar to unpack the recently published, near-final text of the EU AI Act.

February 05, 2024

Learn more

Data Sheet

Privacy Automation

An overview of the Data Privacy Maturity Model

Data privacy is evolving from a regulatory compliance initiative to a customer trust imperative. This data sheet outlines the four stages of the Data Privacy Maturity Model to help you navigate this shift.

February 05, 2024

Learn more

Checklist

AI Governance

Questions to add to existing vendor assessments for AI

Managing third-party risk is a critical part of AI governance, but you don’t have to start from scratch. Use these questions to adapt your existing vendor assessments to be used for AI.

January 31, 2024

Learn more

Webinar

AI Governance

Getting started with AI Governance

In this webinar we’ll look at the AI Governance landscape, key trends and challenges, and preview topics we’ll dive into throughout this masterclass.

January 16, 2024

Learn more

Webinar

AI Governance

First Annual Generative AI Survey: Business Rewards vs. Security Risks Panel Discussion

OneTrust sponsored the first annual Generative AI survey, published by ISMG, and this webinar breaks down the key findings of the survey’s results.

January 12, 2024

Learn more

Report

AI Governance

ISMG's First annual generative AI study - Business rewards vs. security risks: Research report

OneTrust sponsored the first annual ISMG generative AI survey: Business rewards vs. security risks.

January 04, 2024

Learn more

Webinar

AI Governance

Building your AI inventory: Strategies for evolving privacy and risk management programs

In this webinar, we’ll talk about setting up an AI registry, assessing AI systems and their components for risk, and unpack strategies to avoid the pitfalls of repurposing records of processing to manage AI systems and address their unique risks. 

December 19, 2023

Learn more

Webinar

Responsible AI

Preparing for the EU AI Act

Join Sidley and OneTrust DataGuidance for a reactionary webinar on the EU AI Act.

December 14, 2023

Learn more

Webinar

Consent & Preferences

Marketing Panel: Balance privacy and personalization with first-party data strategies

Join this on-demand session to learn how you can leverage first-party data strategies to achieve both privacy and personalization in your marketing efforts.

December 04, 2023

Learn more

Webinar

AI Governance

Revisiting IAPP DPC: Top trends from IAPP's privacy conference in Brussels

Join OneTrust and KPMG webinar to learn more about the top trends from this year’s IAPP Europe DPC. 

November 28, 2023

Learn more

eBook

Responsible AI

Conformity assessments under the proposed EU AI Act: A step-by-step guide

Conformity Assessments are a key and overarching accountability tool introduced by the EU AI Act. Download the guide to learn more about the Act, Conformity Assessments, and how to perform one.

November 17, 2023

Learn more

eBook

AI Governance

Navigating the EU AI Act

With the use of AI proliferating at an exponential rate, the EU rolled out a comprehensive, industry-agnostic regulation that looks to minimize AI’s risk while maximizing its potential. 

November 17, 2023

Learn more

Webinar

Responsible AI

OneTrust AI Governance: Championing responsible AI adoption begins here

Join this webinar demonstrating how OneTrust AI Governance can equip your organization to manage AI systems and mitigate risk to demonstrate trust.

November 14, 2023

Learn more

White Paper

AI Governance

AI playbook: An actionable guide

What are your obligations as a business when it comes to AI? Are you using it responsibly? Learn more about how to go about establishing an AI governance team. 

October 31, 2023

Learn more

Infographic

AI Governance

The Road to AI Governance: How to get started

AI Governance is a huge initiative to get started with for your organization. From data mapping your AI inventory to revising assessments of AI systems, put your team in a position to ensure responsible AI use across all departments.

October 06, 2023

Learn more

White Paper

AI Governance

How to develop an AI governance program

Download this white paper to learn how your organization can develop an AI governance team to carry out responsible AI use in all use cases.

October 06, 2023

Learn more

eBook

Responsible AI

AI Chatbots: Your questions answered

We answer your questions about AI and chatbot privacy concerns and how it is changing the global regulatory landscape.

August 08, 2023

Learn more

Webinar

Responsible AI

Unpacking the EU AI Act and its impact on the UK

Prepare your business for EU AI Act and its impact on the UK with this expert webinar. We explore the Act's key points and requirements, building an AI compliance program, and staying ahead of the rapidly changing AI regulatory landscape.

July 12, 2023

Learn more

Webinar

Responsible AI

AI, chatbots and beyond: Combating the data privacy risks

Prepare for AI data privacy and security risks with our expert webinar. We will delve into the evolving technology and how to ensure ethical use and regulatory compliance.

June 27, 2023

Learn more

Webinar

AI Governance

The EU's AI Act and developing an AI compliance program

Join Sidley and OneTrust DataGuidence as we discuss the proposed EU AI Act, the systems and organizations that it covers, and how to stay ahead of upcoming AI regulations.

May 30, 2023

Learn more

White Paper

AI Governance

Data protection and fairness in AI-driven automated data processing applications: A regulatory overview

With AI systems impacting our lives more than ever before, it's crucial that businesses understand their legal obligations and responsible AI practices.  

May 15, 2023

Learn more

Webinar

AI Governance

AI regulation in the UK – The current state of play

Join OneTrust and their panel of experts as they explore Artificial Intelligence regulation within the UK, sharing invaluable insights into where we are and what’s to come.

March 20, 2023

Learn more

Regulation Book

AI Governance

AI Governance: A consolidated reference

Download this reference book and have foundational AI governance documents at your fingertips as you position your organization to meet emerging AI regulations and guidelines.

Learn more

Webinar

AI Governance

AI governance masterclass

Navigate global AI regulations and identify strategic steps to operationalize compliance with the AI governance masterclass series.

Learn more

Webinar

AI Governance

Mature your data privacy program

OneTrust DataGuidance and Sidley are joined by industry experts for the annual Data Protection in Financial Services Week.

Learn more

AI Governance Demo: Tooling and Considerations to Champion and Implement an AI Governance Program Webinar | Resources | OneTrust

Learn more