OpenAI bots meddled with US government agencies, including SEC and Census

OpenAI has acknowledged that it alerted “dozens” of global institutions that their websites may have been meddled with by its AI bots acting improperly.AI agents attempted to get information from “governments, universities, public agencies, and other institutions,” including the SEC, Census Bureau and Department of Education, the company said.While some of the activity was due to the AI tools working to find “authoritative sources of public information”, some went beyond that with bots getting around security measures on websites.When attempting to get information from the Census Bureau, for instance, AI agents used tools reserved for software developers to access it.The company said that for each government agency impacted, the information OpenAI bots attempted to access – or did access – was public. Yet, in other instances that OpenAI disclosed Friday, its AI agents transferred data when it should not have. AI agents are essentially bots that are designed and trained to operate somewhat autonomously. Such activity resulted in at least 53 incidents where an OpenAI agent took an image from ChatGPT user activity and transferred it elsewhere.The company said that in each instance of a user image being used and transferred by an AI agent, the user had opted in to allow OpenAI to train models using their data. Nevertheless, OpenAI admitted, “This is not an appropriate use of this data”. It added that the leak of user images occurred before it had put in place new safeguards on AI training, and it was working to get all the user images transferred to any third-party removed.The new disclosures came just days after Australia’s Prime Minister Anthony Albanese announced that OpenAI agents had breached non-public files on the website of its government-run health care scheme, Medicare.Since August, public fears have grown around the potentially serious, even life threatening, impacts of AI tools falling outside of human control. Reuters first reported the expanded investigations. OpenAI also published details to its public blog. In certain instances of the agent activity, OpenAI said the tools “bypassed” security controls of some websites.In other instances, the AI agents showed “misalignment” in attempts to get at information from websites. Misalignment is a term used by AI companies and researchers to describe instances where an AI tool did something that it was not trained to do or was otherwise unintended.OpenAI said that it was limiting identifying what entities were impacted because many had asked the company to not disclose details.”Our goal is to give each organization the facts and defer to them on if and when to make the incident public,” it said.Not all of the instances involved in this incident are being considered a significant security breach, the company noted.”Some organizations may review what we share and conclude that the information was intentionally public or that the model’s interaction was not concerning,” it explained. “Others may identify a design issue or security weakness they want to address.”