As Google Gemini becomes a familiar tool for your staff and (possibly) students, a critical question emerges: How will you ensure responsible AI use within your organization? While the thought of becoming the “AI police” might not be appealing, proactive auditing is essential for maintaining accountability.
Earlier this year, Google announced that Vault now offers partial support for auditing Gemini conversations.
Related Post: Understanding Gemini data privacy for EDU domains
Remind me…what is Google Vault?
Google Vault is an important tool for complying with state and federal data retention policies and for use in criminal investigations or Freedom of Information Act (FOIA) requests.
Adding support for Gemini means that you can now perform a Gemini audit using the familiar features of Vault.
How to search Vault
Performing a Gemini search with Google Vault is pretty straightforward. Keep in mind that only designated users (typically super admins) have access to Vault.
- Visit vault.google.com
- Create a new matter
- Select Vault from the list of available services
- Fill out the search fields (users, date, etc)
- View and export data as desired
While the availability of Gemini support in Google Vault is a positive development, there are some pretty significant limitations:
- No data retention: At this time, Google Vault does not retain Gemini search data that has been deleted by the user.
- Limited search options: The search options are limited to user, organizational unit, and date. Keyword searches are not currently supported.
- Limited conversation visibility: Currently, you can only see a short excerpt of a user search. To see the entire thread, you must export the conversation as an XLM file.
- Limited data export options: At this time, XML is the only export option available for Gemini searches. This format provides lots of data, but is difficult to sift through.
In the sample query below, you can see the entire Vault interface for Gemini. At this time, you can’t even click on a search result to see the full conversation. The file export option is limited to XML which provides a bit more detail, but is very difficult to sift through.
More features coming soon
While I am happy that Gemini support for Vault is available, it appears that Google raced to launch this update as quickly as possible without fully building out standard features such as robust search and filtering options, data export capabilities, and support for data retention.
Google’s support articles (linked below) indicate that all of these missing features are under development and will be introduced at a later date. At the moment, your ability to use Gemini to audit district Gemini usage is limited and a bit clunky.
Additional resources
Reader Interactions