National-AI Enabled Repository for Wales: AI Prototype Testing Sessions Report

This report provides an overview of the AI prototype testing sessions carried out as part of the National AI-Enabled Repository for Wales Project which formed part of the initial scoping grants stage of the AHRC’s National Infrastructure for Digital Innovation and Curation in Arts and Humanities (iDAH). It outlines how the activities in each session were carried out, the results and feedback generated, and key findings and recommendations for future testing.

Two testing sessions were conducted in September 2022 to analyse how the outputs of our AI-Enabled Repository prototype compared against human-generated responses, with a focus on the machine-learning based functions of image analysis, text summarisation, and ontological text classification. The first session took place during Aberystwyth University’s AI Research Hub Symposium (Pushing Humanity Forward: Our AI-Enabled World) and participants included symposium attendees from a range of academic disciplines. The second was conducted online and was attended chiefly by participants of focus groups held previously during the project including research data managers, GLAM (Galleries, Museums, Libraries and Archives) specialists, and academic researchers. The differences in the professional backgrounds and assumed knowledge bases of participants across the two sessions offered an opportunity to understand how different types of project stakeholder might engage with AI testing activities. The sessions also allowed us to examine the extent to which such activities can generate discussion and engagement, facilitate a better understanding of the functions of a National AI-Enabled Repository, and inform both the development of our AI prototype and the design of further testing activities in the future.