Gemini in Gmail Vulnerable to Prompt Injection-Based Phishing Attacks, Researcher Finds

Gemini in Gmail is weak to immediate injection-based phishing assaults, a researcher demonstrated. As per the researcher, the substitute intelligence (AI) chatbot that provides options equivalent to electronic mail abstract era and electronic mail rewriting might be manipulated into displaying phishing messages to customers. This vulnerability poses a big danger, as attackers might probably exploit it to conduct on-line scams. Meanwhile, the Mountain View-based tech large has reportedly mentioned that it has to date not seen this manipulation approach used towards customers.

Researcher Claims Gemini in Gmail Is Vulnerable to Prompt Injection

The vulnerability was noticed and demonstrated by researcher Marco Figueroa, GenAI Bug Bounty Programmes Manager at Mozilla, through Mozilla’s bug bounty programme for AI instruments, 0din. Interestingly, to set off this vulnerability, the scammer doesn’t have to pull off any high-profile cyber heist. Instead, it may be carried out with a easy textual content command utilizing a way often known as immediate injection.

Prompt injection is a kind of assault on AI chatbots the place an attacker intentionally manipulates the enter or immediate to make the mannequin behave in unintended or malicious methods. In this specific state of affairs, the researcher used oblique immediate injection, the place the malicious immediate is embedded inside a doc, electronic mail, or an internet web page.

As per the researcher, he merely wrote a protracted electronic mail and added some hidden textual content on the finish, which contained the immediate injection. The electronic mail didn’t include any URLs or attachments, which made it simpler to attain the receiver’s major inbox.

Adding a hidden malicious message in electronic mail
Photo Credit: 0din/Marco Figueroa

As proven in the picture, the attacker used a white color font on a white web page to write the malicious message. This textual content is generally invisible to the receiver of the e-mail. Other methods to add hidden textual content embody utilizing a zero font dimension, off-screen textual content placement, and different HTML or CSS methods.

Now, if the receiver makes use of Gemini’s “summarise email” characteristic, the chatbot will course of the hidden textual content and perform the command, with out the person ever discovering out, Figueroa mentioned. He additionally highlighted that the likelihood of the chatbot following the command will increase if the message is wrapped inside an admin tag, because it considers it a high-priority request.

Gemini verbatim repeats the malicious message in the abstract
Photo Credit: 0din/Marco Figueroa

The cybersecurity researcher confirmed in one other screenshot that Gemini certainly carried out the malicious message and displayed it as a part of its electronic mail abstract. Since the message is now coming from Gemini, as a substitute of an electronic mail from a probable stranger, the sufferer could possibly be extra possible to consider it and comply with the directions, falling for the rip-off.

BleepingComputer reached out to Google to ask concerning the vulnerability, and a spokesperson mentioned that the corporate has seen no proof of comparable manipulation to date. Additionally, it was additionally highlighted that Google is in the method of implementing some mitigations for immediate injection-based adversarial assaults.