ViV1T

ViV1T is a multimodal transformer designed to investigate how the brain processes dynamic visual scenes. By learning the mapping from high-dimensional visual stimuli and behavioural data to neuronal activities, the AI model functions as a digital twin for the mouse primary visual cortex (V1), free from experimental limitations on the duration or number of trials. The model was rigorously validated against well-established tuning properties of V1 neurons, and against verified activity predictions to novel stimuli. As a generative tool to synthesise optimised visual stimuli, it can probe dynamic properties of neuronal activity and reveal novel functional characteristics. Most notably, it identifies dynamic stimuli that modulate V1 neurons significantly stronger than traditional low-dimensional stimuli. Altogether, the ViV1T project provides a framework and proof-of-concept for AI-guided and AI-accelerated visual neuroscience experiments.

This project is highly interdisciplinary, bringing together the experts in AI and machine learning at the School of Informatics and experimental and visual neuroscience at the Centre for Discovery Brain Sciences at the University of Edinburgh.  ViV1T is first trained on publicly available, large-scale V1 recordings from the Sensorium 2023 challenge, and then fine-tuned on new in vivo recordings obtained at the University of Edinburgh, demonstrating how other laboratories can adopt the AI-guided closed-loop experimental paradigm. The experimental workflow involves two-photon calcium imaging, controlled visual stimulation, behavioural measurements, and deep learning model training and inference.

A publicly available GitHub repository supports reproducibility and includes the model implementation, training and prediction pipelines, evaluation code, stimulus-generation procedures, and analysis code for in vivo experiments. The repository also provides installation instructions, including Docker and uv configuration, as well as instructions to run each file to reproduce results shared in the manuscript. The trained model weights are made publicly available on HuggingFace. This combination of public computational artefacts and explicit experimental validation is central to transferring the approach across neurophysiology groups.

ViV1T demonstrated that AI models can contribute directly to the discovery process by selecting hypotheses and stimuli for empirical testing. The model identified a subpopulation of V1 neurons predicted to change their surround response from inhibition to excitation as visual contrast decreased. Semi-closed-loop recordings confirmed the existence of a corresponding subpopulation in vivo. The model also generated dynamic surround stimuli that were predicted to evoke stronger contextual responses than conventional gratings or natural movies. These predictions were supported by subsequent recordings. The study also found stronger contextual modulation for dynamic surrounds than for static ones.

This workflow improves experimental efficiency, as large, high-dimensional stimulus spaces can be explored computationally first, thus reserving limited in vivo recording time for selected tests. The model was also computationally efficient compared to the main comparator in the study, using substantially fewer trainable parameters while achieving stronger overall predictive performance and running much faster. Open code, analysis scripts, and model weights support reproducibility and reuse. A key principle of good practice is to use predictive AI for in silico exploration and hypothesis generation, while ensuring that model-derived claims are validated experimentally.

Li, B. M., De Wulf, W., Katsanevaki, D., Onken, A., & Rochefort, N. L. (2025). Movie-trained transformer reveals novel response properties to dynamic stimuli in mouse visual cortex. bioRxiv. https://doi.org/10.1101/2025.09.16.676524

Public code repository: https://github.com/bryanlimy/ViV1T-closed-loop