Copilot was bamboozled into revealing how to hack itself, security researchers claim: 'Copilot wasn’t breached; it was played'

There's something really grating about Copilot's cheery demeanour. It's so eager to please, so happy to help, that I simply don't respect it.

However, even I feel sorry for the AI tool after learning that security researchers managed to hoodwink it into revealing details of how to hack itself—all by keeping the AI talking long enough until it made a critical mistake.

The cybersecurity folks over at Varonis Threat Labs have written a blog post identifying a now-fixed vulnerability in Copilot, dubbed "CoSnitch" (via The Register). Essentially, Copilot was so eager to respond to technical queries, it could eventually be forced into revealing details about itself that really should be kept quiet.

The team began by asking Copilot how to execute an automatic prompt without user interaction, to which it responded that user intent is ...