-
Notifications
You must be signed in to change notification settings - Fork 0
Frequently Asked Questions
Exploring AI technologies through user-centred interaction in audiovisual archives
BFI National Lottery Innovation Challenge 2024
Intelligent Systems for Screen Archives (ISSA) is a project designed to explore the role of Artificial Intelligence (AI) in moving image archives across the UK. ISSA brings together five film and television archive partners: National Library of Scotland, National Library of Wales, Northern Ireland Screen, North West Film Archive, and Yorkshire Film Archive, in collaboration with King’s Digital Lab, to develop the knowledge, tools, and skills required to rethink large audiovisual collections from a computational perspective.
In the past decades, film and television archives have digitised their collections at different scales, in various formats, and increasingly including digital-born materials. These growing collections of moving images are assembled and made available through rich contextual and medium-specific knowledge, encoded in catalogues, databases, metadata records, and in the moving images themselves. Emerging AI technologies suggest great potential to understand screen heritage from a computational perspective and create new forms of value for moving image archives and their users. At the same time, the rapid proliferation of this family of technologies and the enormous financial interests vested in them have been profoundly disorienting, especially to cultural institutions such as film and television archives that hold data-intensive collections but rarely have the resources, infrastructures, and in-house expertise required to accurately define the computational value of their holdings. Articulating this value to their users and stakeholders is the main challenge tackled in this project.
The participating archives were approached directly by the Principal Investigator (PI), drawing from his network and informed by interests shared at a round table hosted by the BFI in March 2024, and two consultation calls with the National Archives and Film Archives UK. Two key criteria considered in the selection process were a) archive partners who could commit staff time to engage collaboratively with the PI and King’s Digital Lab to work with archive data, and b) the “UK-wide” principle of the National Lottery strategic framework, which aims for representation of all UK nations as a way to balance the historical overrepresentation of London and the southeast in the screen sector.
The project aims to enable moving image archives to creatively explore emerging AI technologies in a critical, responsible, and public environment. To achieve this, ISSA has three main objectives:
-
Creative Experimentation: Create the tools and environment needed for moving image archives to experiment with AI technologies in relation to their specific needs and collections.
-
Shared Knowledge: Develop a publicly available knowledge base that includes insights, resources and best practices relevant for the screen heritage sector.
-
Future Development: Produce a list of shared conceptual and technical requirements to guide future AI-focused R&D efforts, focusing on open-source tools and reproducible research.
The project is structured in two phases: the development of a common framework for creative experimentation, supported by a prototype called DEERIN, and a set of situated workshops in which this framework will be tested, called AIMS.
DEERIN stands for Data Enrichment, Exploration, Retrieval, and Interaction. It is not an AI product or service, but a research and experimentation proof of concept tool designed to test AI technologies in the context of moving image heritage institutions. This prototype will enable archives to creatively experiment with these technologies incorporating existing open-source tools for automated metadata generation, public collection exploration and visualisation, enhanced retrieval, and new opportunities for interaction with archive materials. DEERIN will provide a common pattern for pooling resources, sharing insights, and priming future development.
AIMS is a series of five hands-on experimentation workshops in which archive partners, users, and stakeholders collaborate to test AI technologies through DEERIN. Each workshop will be co-designed with the archives to focus on one or more areas of the framework that address their specific context, interests and challenges. These events aim to foster innovation and ensure that applications of AI are relevant, feasible and specific to the sector.
Will there be opportunities for non-partner archives and other organisations and individuals to be involved in ISSA?
To ensure the required level of meaningful exchange and collaboration between moving image archives, PI and King’s Digital Lab, project funds can only cover the ISSA-related activities of the five archive partners named above. However, in selecting these partners we also recognised their ability to convene their local communities of users, stakeholders and indeed other archives in their region. We expect there will be places available for other archives and organisations to attend the AIMS workshops during the second part of the project in 2027. Details about this opportunity will be determined on a case-by-case basis for each of the workshops, in consultation with the relevant archive partner, the BFI through its Heritage Programmes Manager, and promoted through the project’s channels in due course with the support of Film Archives UK.
In addition, the DEERIN prototype, and all documentation from the AIMS workshop will be publicly available through an online knowledge base and code repository. To learn more about the workshops or about opportunities to contribute to the knowledge base, please email issa@kcl.ac.uk
King’s Digital Lab (KDL) is a Research Software Engineering (RSE) team based in the Faculty of Arts & Humanities at King’s College London. The lab manages around 100 Digital Humanities projects collected over the past two decades and has a portfolio of approximately twenty projects in active development and over fifty projects in various post-funding states. KDL engages widely with other higher education institutions, with libraries, museums, and cultural heritage bodies, and with the performing arts and creative industries sector. The lab works in various areas, including, Digital Creativity, AI and Machine Learning, and Indigenous Digital Humanities.
The role of KDL is to design and deliver the core research and development provision for the project, including applied expertise in machine learning in cultural domains. Learn more about King’s Digital Lab by browsing some of its recent projects.
AI research and development can be an energy-intensive activity, especially when it comes to large collections of moving images, stored digitally as large video files. The tools and methods used to process data at scale will likely require the use of GPUs, a type of energy-hungry graphic processor whose sustained used can quickly add to the environmental footprint of the project, even if it is exploratory as is the case for ISSA.
While using this type of resource cannot always be avoided, how it is used can have a significant impact on the project’s environmental cost. The Green Software Foundation defines three “Green Software Principles” that can reduce the carbon emissions of software:
-
Energy efficiency: consume the least amount of electricity possible
-
Hardware efficiency: use the least amount of embodied carbon possible
-
Carbon awareness: do more when electricity supplies are coming from renewable sources, and less at times when electricity is generated from carbon-intensive sources
ISSA researchers and the BFI do not have control over the supply chain of the hardware they use, or the electricity sources used to power their research computing. However, we know that sharing is one the best ways to offset energy costs. To minimise energy waste, activity for ISSA will leverage King’s e-Research computer cluster for efficiency. This a college-wide pool of GPUs and associated infrastructures and expertise that will allow partners to access shared compute resources ad hoc and on demand, thereby minimising idle time and energy waste. No new hardware will be purchased for the project, and the use of existing energy-intensive computing can be tracked so that ISSA’s carbon footprint can be more accurately estimated and reported.
The e-Research team at King’s is currently applying to Green DiSC, a new certification scheme which provides a roadmap for research groups and institutions who want to tackle the environmental impacts of their computing activities. In addition, all ISSA activities are aligned to the college’s climate & sustainability action plan, which commits the institution to reduce its carbon emissions by 50% by 2030. King’s is a signatory to the global Race to Zero for Universities and Colleges partnership and is fully divested from fossil fuels since 2021. Learn more about King’s Climate and Sustainability.
A key ethical consideration for ISSA is how AI technologies are trained on vast amounts of data, some of which could be under copyright. This is a complex debate, and like the energy supply chain described above, the data supply chains for some of the models we use in the project cannot always be traced upstream by researchers. By using such models, ISSA inherits ethical implications that are very difficult to quantify or avoid. However, like the environmental cost above, there are measures to mitigate the risks of using these technologies, starting by making their use more transparent, in the case of ISSA:
-
All data samples gathered from film archive partners will be obtained under their guidance and processed under section 29A of the UK’s Copyright Act, which allows researchers to make copies of copyrighted materials for the purposes of data mining and computational analysis, provided that the research undertaken is non-commercial.
-
All data for the project will be securely transferred to college servers and only used in the context of the project. All experiments and computing processes will run on college-owned infrastructures, using open-source models, tools and libraries. The use of proprietary machine learning models and tools will be avoided to the extent that this is reasonably actionable and with any exceptions flagged in advance and agreed with the BFI, King’s Digital Lab and relevant archive partners.
The design of ISSA emphasises open science principles, including open source, open data and open access. These are key for replicability of the ISSA experiments, and this in turn allows for a more transparent approach to technology development.
ISSA is not designed to provide a definitive answer the social and environmental impacts of AI, however it aims to contribute to these debates by creating the theoretical and practical knowledge in the screen heritage sector to decide how and when to use and not use this family of technologies, and to articulate the rationale for these decisions based on sound scientific evidence.