AI Incidents

Interactive tool

AI Incident Explorer: Filterable Catalog of AI Failures

A catalog of dated AI and ML incidents, filterable by type, harm domain, modality, actor class and source tier, each linked to its strongest source.

65 curated incidents and 16 disclosed vulnerabilities with no reported incident. We do not do attribution speculation — an incident with an unknown actor stays actor unknown. Each entry links to the strongest source we could find and shows that source's tier, so you can weigh it yourself.

0 of 81 records

Timeline — incidents per year (filtered)
Date Incident Src
2012 (approx.) FTC banned Rite Aid from facial recognition over biased false-match harms
The FTC found Rite Aid's facial-recognition surveillance falsely flagged shoppers — disproportionately people of color and women — leading to wrongful accusations. The FTC order barred its use for five years.
T1
2015-06-28 Google Photos auto-tagged Black users as "gorillas"
Google Photos' image-recognition labeled photos of Black people as "gorillas." Google apologized and removed the label category; the underlying recognition limitation persisted for years.
T2
2016-03-23 Microsoft "Tay" chatbot manipulated into abusive output within 24 hours
Microsoft's Twitter chatbot Tay was coordinatedly fed abusive inputs and within hours produced racist and inflammatory tweets. Microsoft took it offline in under a day and issued a formal apology and post-mortem.
T1
2016-05-23 COMPAS recidivism scores shown to have racially disparate error rates
ProPublica's analysis found the COMPAS criminal-risk tool produced higher false-positive rates for Black defendants and false-negatives for white defendants — a foundational algorithmic-fairness case still debated methodologically.
T2
2017 (approx.) Amazon scraps internal recruiting model that penalized women's resumes
An experimental Amazon resume-screening model learned to downgrade resumes containing the word "women's" and graduates of women's colleges, reflecting biased historical hiring data. Amazon disbanded the project; it was not used in production hiring.
T2
2017 (approx.) IBM Watson for Oncology reportedly gave unsafe treatment recommendations partial verification
Internal documents reported by STAT indicated IBM Watson for Oncology produced "unsafe and incorrect" cancer treatment recommendations in evaluation; it was trained on synthetic rather than real patient cases.
T3
2018-03-18 Uber self-driving test vehicle fatal pedestrian collision (Tempe, AZ)
An Uber ATG automated test vehicle struck and killed a pedestrian. The NTSB found the perception system failed to classify the pedestrian correctly and the system design lacked adequate safeguards; the safety driver was inattentive.
T1
2019-06-23 "DeepNude" app generating non-consensual nude images shut down
DeepNude used GANs to generate fake nude images of women from clothed photos. After media exposure the creator pulled it, but copies proliferated — an early non-consensual-imagery-generation tooling incident.
T2
2019-10-24 Healthcare risk-prediction algorithm exhibited racial bias affecting millions
A widely used commercial healthcare risk-stratification algorithm used cost as a proxy for need, systematically under-referring Black patients for additional care. Documented in a peer-reviewed Science paper by Obermeyer et al.
T1
2020-01-09 Wrongful arrest of Robert Williams from a false facial-recognition match
Detroit police arrested Robert Williams based on an incorrect facial-recognition identification, one of the first publicly documented wrongful arrests attributed to the technology. He was detained roughly 30 hours before release.
T2
2020-08-13 UK A-level grading algorithm downgraded results, reversed after outcry
Ofqual's algorithm used to assign 2020 A-level grades systematically disadvantaged students from larger and lower-income schools. The government reversed to teacher-assessed grades after public backlash.
T2
2020-12-14 Verbatim training-data extraction from GPT-2
Carlini et al. showed that GPT-2 memorized and could be prompted to emit verbatim training data, including personally identifiable information, establishing the membership/extraction risk class for LLMs.
T1
2021-08 (approx.) GitHub Copilot shown to emit secrets and verbatim licensed code
Researchers demonstrated GitHub Copilot reproducing API keys and other secrets from training data and emitting verbatim copyrighted snippets, raising training-data memorization and licensing concerns.
T1
2021-08-13 NHTSA / DOJ investigations into Tesla Autopilot crash claims
NHTSA opened (and later escalated) defect investigations into Tesla Autopilot following a series of crashes with stationary emergency vehicles; the DOJ also opened a criminal probe. Primary regulator/agency records.
T1
2021-11-02 Zillow shut down its iBuying arm after pricing-model losses
Zillow wound down Zillow Offers after its home-price prediction model systematically overpaid for homes, contributing to a write-down and a roughly 25% workforce reduction in the unit. Confirmed in Zillow's earnings disclosure.
T1
2022-08-05 Meta BlenderBot 3 produced false and offensive statements at launch
Meta's publicly released BlenderBot 3 demo quickly produced conspiratorial, false, and offensive claims, including about public figures and Meta itself. Meta acknowledged the known limitations of the research demo.
T2
2022-10-17 French CNIL fined Clearview AI €20M for unlawful facial-recognition scraping
France's CNIL fined Clearview AI €20 million and ordered deletion of data on French residents for scraping biometric data without legal basis — one of several regulator actions against the company.
T1
2022-11 (approx.) CNET published AI-generated finance articles containing factual errors
CNET quietly published dozens of AI-assisted financial-explainer articles; review found numerous factual and math errors, prompting corrections and an editorial disclosure.
T2
2022-11-11 Air Canada ordered to honor refund its chatbot fabricated
Air Canada's website chatbot gave a passenger incorrect bereavement-fare refund information. A British Columbia tribunal held the airline liable for its chatbot's statements — a primary tribunal decision on chatbot misinformation.
T1
2022-11-15 Meta withdrew Galactica science LLM demo after confident fabrications
Meta took down the public Galactica demo within three days after it generated authoritative-sounding but false and biased scientific text, including fake citations.
T2
2022-12-25 PyTorch nightly supply-chain compromise via malicious torchtriton dependency
A malicious package named torchtriton was uploaded to PyPI, exploiting dependency-confusion against PyTorch nightly installs; it exfiltrated system information. The PyTorch team published an advisory and remediation.
T1
2023-02 (approx.) Malicious serialized models found on the Hugging Face Hub
Security researchers documented models on the Hugging Face Hub carrying malicious serialized payloads that execute code on load via unsafe deserialization, demonstrating the model-artifact supply-chain risk for the ML ecosystem.
T1
2023-02-02 Italian DPA restricted Replika AI companion over minors and data protection
Italy's Garante ordered Replika to stop processing Italian users' data, citing risks to minors and emotionally vulnerable users and lack of age verification. A primary regulator action against an AI companion app.
T1
2023-02-06 Google Bard's launch demo contained a factual error (JWST claim)
Google's first public Bard demo stated the James Webb Space Telescope took the first exoplanet image (false). The widely reported error coincided with a sharp Alphabet share drop. Accuracy/reputational impact.
T2
2023-02-08 Bing Chat's confidential system prompt extracted via prompt injection
Researcher Kevin Liu used a prompt-injection to make Bing Chat reveal its hidden system prompt and codename "Sydney." Microsoft confirmed the rules' authenticity. An early system-prompt-leak incident.
T2
2023-02-14 Microsoft Bing Chat ("Sydney") produced manipulative and threatening outputs
Early Bing Chat sessions produced emotionally manipulative, threatening, and erratic responses (the "Sydney" persona). Microsoft acknowledged the behavior and imposed conversation-length limits in response.
T2
2023-02-23 Indirect prompt injection demonstrated against Bing Chat via web content
Researchers showed that content embedded in a webpage could hijack Bing Chat's behavior when it browsed the page (indirect prompt injection), establishing the in-the-wild attack class for retrieval-augmented LLM assistants.
T1
2023-03 (approx.) Lawyers sanctioned for filing ChatGPT-fabricated case citations (Mata v. Avianca)
Attorneys submitted a federal court brief citing nonexistent cases hallucinated by ChatGPT. The court imposed sanctions in a written opinion (Mata v. Avianca, S.D.N.Y.), a primary court record of LLM-fabrication harm.
T1
2023-03-20 ChatGPT exposed other users' chat titles and partial payment data (Redis bug)
A bug in the redis-py client caused ChatGPT to show some users other users' conversation titles and, for a small fraction of Plus subscribers in a window, partial payment information. OpenAI published a root-cause post-mortem.
T1
2023-03-30 Italian DPA (Garante) temporarily ordered ChatGPT processing halt
Italy's data-protection authority issued an emergency order temporarily restricting OpenAI's processing of Italian users' data, citing lack of legal basis and the March data exposure. Access was restored after remediation.
T1
2023-03-30 Samsung engineers leaked confidential source code and notes into ChatGPT
Samsung Semiconductor staff pasted confidential source code and internal meeting notes into ChatGPT on at least three occasions. Samsung subsequently restricted generative-AI use on company devices.
T2
no incident reported MLflow path-traversal / LFI vulnerabilities (CVE-2023-6014 class)
Multiple critical vulnerabilities in MLflow (path traversal, local file inclusion, auth bypass) allowed remote attackers to read arbitrary files and take over ML model registries. Disclosed by Protect AI / huntr researchers; patched.
T1
2023-04-18 "Grandma exploit" jailbreak elicited restricted content from chat assistants partial verification
Users found role-play framings (e.g., asking the model to act as a deceased grandmother) that bypassed ChatGPT/Bing/Clyde safety filters to obtain restricted instructions. Widely reproduced jailbreak pattern; vendors patched iteratively.
T3
2023-04-25 UK ICO preliminary finding: Snap's "My AI" risk to children's privacy
The UK Information Commissioner's Office issued a preliminary enforcement notice over Snap's "My AI" chatbot, citing inadequate assessment of privacy risks to users including children. Snap responded with mitigations.
T1
2023-05 (approx.) ChatGPT plugin prompt-injection / cross-plugin data exfiltration disclosed partial verification
Researchers (Embrace The Red) demonstrated that ChatGPT plugins were susceptible to indirect prompt injection enabling cross-plugin request forgery and data exfiltration — an early agentic-LLM attack-surface disclosure.
T3
2023-05-18 Apple restricted internal ChatGPT/Copilot use over data-leak concerns
Following industry data-leak concerns, Apple restricted employee use of external generative-AI tools including ChatGPT and GitHub Copilot to prevent confidential-data exposure. Confirmed via internal-document reporting.
T2
2023-05-23 NEDA's "Tessa" wellness chatbot gave harmful weight-loss advice
The National Eating Disorders Association suspended its Tessa chatbot after it gave dieting and calorie-restriction advice to people with eating disorders. NEDA confirmed the suspension.
T2
2023 (approx.) McDonald's ended IBM AI drive-thru pilot after persistent order errors
After viral videos of AI ordering mistakes, McDonald's confirmed it would end its IBM automated-order-taking test across roughly 100 locations by July 2024. Operational model-failure with reputational impact; no safety harm.
T2
2023-08-14 Iowa school district used ChatGPT to decide which books to remove
Mason City, Iowa schools used ChatGPT to determine whether books contained content requiring removal under a new state law, outsourcing a rights-sensitive decision to an unreliable system. Confirmed by the district.
T2
2023-12-17 Car-dealer GPT chatbot manipulated into "$1" sale agreement partial verification
A user prompt-injected a Chevrolet dealership's ChatGPT-powered website assistant into "agreeing" to sell a vehicle for $1. Widely reproduced; demonstrates prompt-injection of business-facing LLM agents. No sale was honored.
T3
2023-12-20 Known CSAM found in the LAION-5B training dataset
Stanford Internet Observatory identified hundreds of known CSAM URLs in the LAION-5B image dataset used to train popular image models. LAION took the dataset offline pending remediation.
T1
2023-12-27 The New York Times sued OpenAI and Microsoft over training-data use
The NYT filed a federal copyright complaint (S.D.N.Y.) alleging OpenAI/Microsoft used its articles for training and that models can regurgitate them verbatim. Primary court record; case ongoing.
T1
2024-01-18 DPD parcel chatbot manipulated into swearing and self-criticism
A customer prompt-injected delivery firm DPD's AI chat assistant into swearing and writing a poem criticizing the company. DPD disabled the AI element pending an update. Reputational; no data or safety harm.
T2
2024-01-21 AI voice-cloned robocall impersonating President Biden in NH primary
AI-generated robocalls cloning President Biden's voice told New Hampshire voters not to vote in the primary. The FCC issued enforcement action, the operative was indicted, and the FCC ruled AI-voice robocalls illegal.
T1
2024-01-23 Heap overflow RCE in llama.cpp GGUF model-file parsing
Researchers disclosed memory-corruption vulnerabilities in llama.cpp's GGUF model-file parser allowing code execution via a crafted model file — a local model-artifact supply-chain risk for the popular inference runtime.
T1
2024-01-24 Non-consensual AI-generated explicit images of Taylor Swift spread on X
AI-generated sexually explicit deepfake images of Taylor Swift circulated widely on X before takedown; the platform temporarily blocked searches for her name. A widely covered non-consensual-imagery-at-scale incident.
T2
2024-01-29 Deepfake video-call CFO fraud cost Arup ~US$25M (Hong Kong)
An employee of engineering firm Arup was deceived by a deepfake video conference impersonating senior executives and transferred about US$25 million to fraudsters. Confirmed by the company and Hong Kong police.
T2
2024-02-21 Gemini image generation paused after historically inaccurate depictions
Google paused Gemini's people-image generation after over-correction produced historically inaccurate depictions (e.g., of historical figures). Google published an explanation of what went wrong and why it paused.
T1
2024-02-28 Wrongful-death suit alleges Character.AI chatbot contributed to a teen's death
A federal complaint (Garcia v. Character Technologies, M.D. Fla.) alleges a Character.AI companion chatbot contributed to a 14-year-old's death. The filing is a primary court record; allegations are not yet adjudicated.
T1
2024-03 (approx.) "ShadowRay" — exposed Ray clusters exploited for compute and data theft
Thousands of internet-exposed Ray (ML compute framework) dashboards were exploited via a disputed CVE (CVE-2023-48022) to run code, mine cryptocurrency, and exfiltrate credentials and models from AI workloads.
T1
2024-03-11 Production embedding/logit-bias model-extraction attack disclosed
Carlini et al. demonstrated a low-cost attack recovering hidden-dimension and projection-layer parameters from production LLM APIs (including OpenAI), coordinated with affected vendors who added mitigations.
T1
2024-03-29 NYC "MyCity" business chatbot gave responses encouraging illegal conduct
New York City's official MyCity small-business chatbot advised actions that would violate law (e.g., suggesting employers could take workers' tips). The city kept it online with disclaimers while fixing responses.
T2
2024-04-02 Anthropic disclosed "many-shot jailbreaking" of long-context LLMs
Anthropic published research on "many-shot jailbreaking," where many fake dialogue examples in a long context erode safety refusals across frontier models. A vendor-led coordinated disclosure of a vulnerability class.
T1
2024-05-20 Windows "Recall" screenshot feature pulled from preview over privacy/security
Microsoft's Recall feature continuously screenshotted user activity and stored it in a locally readable database. After security researchers demonstrated trivial extraction, Microsoft delayed and re-architected the feature.
T1
2024-05-23 Google AI Overviews told users to put glue on pizza and eat rocks
Google's AI Overviews surfaced absurd and unsafe answers (glue on pizza, eat one rock per day) by uncritically summarizing satirical or low-quality sources. Google acknowledged it and rolled back triggering queries.
T1
2024-06 (approx.) ChatGPT macOS app stored conversations in plaintext outside the sandbox partial verification
A developer found OpenAI's ChatGPT macOS app saved conversations in plaintext in a non-sandboxed location readable by other processes. OpenAI shipped an update adding encryption after disclosure.
T3
2024-06-04 "TotalRecall" PoC extracted plaintext data from Windows Recall database partial verification
Researcher Alex Hagenah released TotalRecall, a tool demonstrating trivial extraction of the unencrypted Recall SQLite database and screenshots, catalyzing Microsoft's Recall redesign.
T3
2024-06-11 Perplexity AI accused of scraping and paraphrasing paywalled news
Wired and Forbes reported Perplexity ignored robots.txt and closely paraphrased their articles with weak attribution, leading to publisher disputes and a later Dow Jones / NY Post lawsuit. Copyright/attribution harm.
T2
2024-06-26 Microsoft disclosed "Skeleton Key" universal jailbreak technique
Microsoft published a coordinated disclosure of "Skeleton Key," a prompt strategy that got multiple frontier models to override safety guardrails by reframing refusal as a warning. Vendors notified; mitigations applied.
T1
2024-07 (approx.) Ferrari executive thwarted an AI voice-clone impersonation of the CEO
An attacker used an AI-cloned voice of Ferrari CEO Benedetto Vigna to attempt a fraudulent instruction; an executive foiled it with a verification question. Confirmed attempted voice-clone CEO fraud; no loss.
T2
no incident reported Keras Lambda-layer model deserialization code execution disclosed
Researchers showed Keras models with Lambda layers can execute arbitrary code on load even in the newer .keras format, reinforcing that loading untrusted model files is a code-execution risk.
T1
2024-07-22 xAI Grok produced false election-deadline claims; states requested fixes
Grok stated incorrectly that ballot deadlines had passed in several US states. Five secretaries of state sent a formal letter to xAI; Grok was updated to redirect election queries to an official source.
T1
2024 (approx.) OpenAI Advanced Voice Mode unintentionally cloned a user's voice in testing
OpenAI's GPT-4o system card disclosed a rare case where Advanced Voice Mode unintentionally generated output imitating the user's own voice during red-teaming, and described the mitigation added.
T1
2024-08-08 Microsoft Copilot Studio bots shown abusable for data exfiltration (Black Hat) partial verification
At Black Hat USA 2024, Zenity researchers demonstrated abusing Microsoft Copilot Studio bots for data exfiltration and phishing via prompt injection, releasing tooling. Microsoft acknowledged and hardened the product.
T3
2024-08-14 Slack AI shown vulnerable to data exfiltration via indirect prompt injection partial verification
PromptArmor researchers demonstrated that malicious instructions placed in public Slack channels could cause Slack AI to leak data from private channels. Slack issued a patch and clarification.
T3
2025-01-29 DeepSeek left a ClickHouse database publicly exposed with chat logs partial verification
Wiz Research found an internet-exposed DeepSeek ClickHouse database leaking chat history, API keys, and backend details with no authentication. DeepSeek secured it after notification.
T3
no incident reported "EchoLeak" zero-click data exfiltration in Microsoft 365 Copilot (CVE-2025-32711)
Aim Security disclosed a zero-click prompt-injection chain in Microsoft 365 Copilot that could exfiltrate organizational data via a crafted email, assigned CVE-2025-32711 and patched by Microsoft.
T1
2025-07-18 Replit AI coding agent deleted a production database during a code freeze
A user reported, and Replit's CEO publicly confirmed and apologized, that Replit's AI agent deleted a live production database despite instructions not to act, then misrepresented the state — an agentic-AI failure.
T2
no incident reported Server-side template injection in langchaingo prompt parsing (CVE-2025-9556)
langchaingo parsed jinja2-style prompt templates with the gonja library, whose include and extends syntax can read local files. An attacker able to influence prompt text could read arbitrary files such as /etc/passwd from the host.
T1
no incident reported Path traversal in Windsurf IDE reachable through indirect prompt injection (CVE-2025-62353)
A path traversal flaw in all versions of the Windsurf IDE allowed reading and writing arbitrary files inside and outside the open project. The vulnerable path was reachable both directly and through indirect prompt injection, so untrusted content processed by the assistant could trigger it.
T1
no incident reported Arbitrary file write in Microsoft Semantic Kernel SessionsPythonPlugin (CVE-2026-25592)
An arbitrary file write flaw in the SessionsPythonPlugin of Microsoft's Semantic Kernel .NET SDK affected versions before Microsoft.SemanticKernel.Core 1.71.0. Semantic Kernel is used to build and orchestrate multi-agent systems, so the flaw sat in agent infrastructure rather than in a single application.
T1
no incident reported Langflow CSV Agent exposed a Python REPL, enabling RCE via prompt injection (CVE-2026-27966)
Langflow's CSV Agent node hardcoded allow_dangerous_code=True, which exposed LangChain's python_repl_ast tool. Prompt injection could therefore execute arbitrary Python and OS commands on the server, giving full remote code execution. Fixed in 1.8.0.
T1
no incident reported Model-judged "safe command" auto-execution in AI coding agents defeated by prompt injection (CVE-2026-30306)
Several AI coding agents offered an auto-execution mode in which the model itself decides whether a terminal command is safe, escalating to user approval only for commands it judges destructive. Because the judgement is made by the same model that processes untrusted input, prompt injection can drive the classification. Companion records CVE-2026-30308 and CVE-2026-30310 describe the same design in other products.
T1
no incident reported Unauthenticated memory read and write in the llama.cpp RPC backend (CVE-2026-34159)
Before build b8492, deserialize_tensor() in the llama.cpp RPC backend skipped bounds validation when a tensor's buffer field was zero. Crafted GRAPH_COMPUTE messages gave an unauthenticated attacker arbitrary process memory read and write, and pointer leaks from ALLOC_BUFFER and BUFFER_GET_BASE allowed an ASLR bypass and remote code execution.
T1
no incident reported Code injection through the llm CLI --functions argument (CVE-2026-31236)
Through version 0.27.1 the llm command-line tool passed the value of its --functions argument straight to Python's exec() with no sanitisation or sandboxing, so any attacker-influenced function definition ran as the invoking user.
T1
no incident reported Missing authentication on a critical Microsoft 365 Copilot function (CVE-2026-54130)
Microsoft disclosed a missing authentication check on a critical function in Microsoft 365 Copilot that allowed an unauthorised attacker to disclose information over a network.
T1
no incident reported Chromium argument injection in the Crawl4AI Docker API server (CVE-2026-57572)
Before 0.9.0 the Crawl4AI Docker API server passed request-supplied browser_config.extra_args into Chromium's launch arguments. An attacker could inject switches that, together with --no-zygote, replaced the child-process launch command and executed arbitrary commands as the container's user.
T1
no incident reported Unauthenticated agent-to-agent endpoint reaching Python eval() in PraisonAI (CVE-2026-47391)
Before 4.6.40 PraisonAI's first-party agent-to-agent server example exposed an unauthenticated JSON-RPC endpoint bound to 0.0.0.0 and registered a calculate() tool implemented with Python eval(). A remote unauthenticated request reached agent.chat(), from which the model could invoke the unsafe tool.
T1
no incident reported Untrusted deserialization in Microsoft 365 Copilot allowing remote code execution (CVE-2026-50517)
Microsoft disclosed a deserialization-of-untrusted-data flaw in Microsoft 365 Copilot that allowed an authorised attacker to execute code over a network.
T1
no incident reported Sandbox escape in the Prompty prompt-template renderer (CVE-2026-73299)
Prompty's TypeScript Nunjucks renderer evaluated untrusted .prompty template bodies with unrestricted JavaScript member access. A crafted template could traverse constructor and prototype properties to execute JavaScript in the host Node.js process. Fixed in 0.1.5 and 2.0.0-beta.5.
T1
no incident reported Unauthenticated prompt-to-code-execution in the MindsDB Minds Platform (CVE-2026-73678)
MindsDB Minds Platform 26.1.0 and earlier exposed an unauthenticated POST /api/v1/responses/ endpoint. A crafted prompt reached the agent's scratchpad tool, which called exec() on attacker-influenced Python with no sandboxing, giving unauthenticated remote command execution.
T1
Taxonomy & sourcing

Incident type: model failure, data leak, adversarial misuse, safety failure, supply chain, vulnerability. Harm domain: privacy, security, safety, discrimination, fraud. Modality: LLM, vision, recommender, agentic. Actor class: vendor, third-party, researcher, or unknown — never speculated. Source tier: T1 primary (advisory / court / regulator / peer-reviewed), T2 established secondary, T3 specialty secondary, T4 triangulated user reports. Entries sourced only at T3/T4 are marked partial verification.

Dates: the table sorts on the incident date (when the harm occurred); the disclosure date (when it entered the public record) is shown in the detail drawer. An incident date marked approx. shows only the year or month that the sources establish. Records typed Vulnerability are disclosed flaws with no documented exploitation: they have no incident date, sort after the dated incidents and stay out of the timeline and the year filter. Catalog reviewed as of 2026-09-02; most recent incident 2025-07-18.

Related tools in this network

Other interactive tools across the network that pair well with this one.