![Italian PM Meloni Denies Trump Claim She ‘Begged’ for G7 Photo]](https://i0.wp.com/s.france24.com/media/display/0c7efee0-6bfa-11f1-b317-005056a97e36/w:1280/p:16x9/capture-11412217636a356b0eb17678-43159022.jpg?w=1024&resize=1024,1024&ssl=1 "Italian PM Meloni Denies Trump Claim She ‘Begged’ for G7 Photo]")
In a recent security investigation, researcher Andi Ahmeti uncovered that ChatGPT can’t distinguish between content it generates and malicious Markdown pulled from external websites. This flaw means that if a user asks the chatbot to summarize a page containing hidden instructions, the attacker’s payload can be embedded directly in the response.
Threat hunter Ahmeti warned that attackers might exploit this blind trust to inject phishing URLs or fabricate fake security alerts styled in ChatGPT’s voice. In a demonstrated scenario, an attacker injected a faux security notification into a GitHub page. When the user prompted ChatGPT to summarize the page, the model produced a legitimate summary followed by an alarming “new device added” message with a “click here” link. That link directed users to an attacker-controlled domain, potentially facilitating credential theft.
Additionally, Ahmeti showed how the same technique could embed an inline QR code in the chatbot’s output. Scanning the QR code with a phone transports the victim to a malicious site hosted in an attacker-controlled S3 bucket, bypassing typical desktop URL defenses such as blocklists and password‑manager checks.
“AI systems increasingly render untrusted content directly inside browsers, which expands risk significantly,” Ahmeti explained. “The bigger issue is that AI products are starting to resemble browser or operating system environments, creating a much larger security surface.”
OpenAI has not confirmed whether the vulnerability, termed “ChatGPhish,” has been patched. Ahmeti reported the issue to OpenAI via Bugcrowd on April 29 and again on May 1, but received mixed responses—initially marked as not reproducible, then as a duplicate. No official update from OpenAI has been received.
To mitigate this risk, Ahmeti recommends stringent sandboxing, isolating model‑generated content, and thorough filtering of Markdown, HTML, embeds, and previews. He emphasized that “AI-generated content should always be treated as untrusted” and that prompt injection will likely continue to surface as a security threat in AI systems.
