The White House has collaborated with top AI developers, including OpenAI, Google, and Microsoft, to participate in a public evaluation of their generative AI systems at DEF CON 31, a hacker convention taking place in Las Vegas in August 2023. The evaluation aims to push the new generative AI models to their limits and uncover potential flaws and vulnerabilities, such as confabulations, jailbreaks, and biases, which pose challenges for security professionals and the public. The White House Office of Science, Technology, and Policy endorses this exercise, which aligns with the Biden administration's AI Bill of Rights and the National Institute of Standards and Technology's AI Risk Management Framework.

Participants will have timed access to multiple large language models (LLMs), including ChatGPT, through laptops provided by the organisers. AI Village aims to grow the community of researchers equipped to handle vulnerabilities in AI systems by conducting the largest red-teaming exercise for any group of AI models.