CMS FlashSim

Detailed simulations and reconstructions of particle collisions incur significant computing costs in high-energy physics. In CMS, the conventional workflow uses Geant4 to model interactions between particles and the detector. This is followed by experiment-specific reconstruction algorithms that convert the simulated detector response into physics objects and analysis datasets. Full event simulation alone accounts for around half of the experiment’s CPU budget, and billions of simulated events are required yearly to enable comparison with data collected by the collider. This demand is set to increase further at the High-Luminosity LHC, starting in 2030.

FlashSim uses generative machine learning to shorten this process. A Conditional Flow Matching model is trained on events produced by the FullSim workflow, learning to map collisions at the generator level directly into the NanoAOD data format used for CMS physics analysis. This requires the model to generate a large set of analysis variables while preserving the distributions and correlations necessary for physics analysis. Therefore, the aim is not simply faster event generation, but a much faster approximation of the full simulation and reconstruction chain with sufficient fidelity for downstream scientific use.

FlashSim emerged from the growing mismatch between the computational cost of detailed detector simulation and reconstruction, and the volume of Monte Carlo data required by CMS. A key enabling condition is the experiment’s adoption of NanoAOD as a common analysis-level data format. It gives the generative model a defined target that is compact enough for end-to-end modelling while retaining the variables required by many physics analyses. FlashSim can therefore bypass intermediate stages of detector simulation and reconstruction and learn the relationship between generator-level events and their analysis-level representation. Finally, the use of NanoAOD data format is essential for the FlashSim to be transparent to users, who already use the format for their analysis activities.

The approach depends on FullSim events for training and on close integration of machine-learning and particle-physics expertise, which is difficult to find in the same group of researchers, and happened somehow naturally among the huge user base in CMS. Its accuracy is assessed through the quantities that matter for downstream analysis, including reconstructed-object distributions, correlations and signal/background discrimination. Testing in a realistic physics workflow has also been used to identify model failures, and to guarantee the usability throughout the full analysis space of interest for CMS.

FlashSim development was initially supported through specific research programs in Italy (funded by national and European funds), and is now a recognized R&D path in the CMS experiment.  The initial development was also supported by the capability to access large scale processing facilities, like those deployed by INFN internally and via the ICSC NRRP EU funded program. Crucial to the development of FlashSim has been the large availability of “Full Simulation” simulation datasets in CMS, which are routinely produced to allow data analysis. The access to those datasets is restricted to CMS members, even if a fraction of them is released through the CERN Open Data Portal.

FlashSim shows that an end-to-end generative model can produce analysis-quality CMS events at a fraction of the computational cost of conventional simulation. Current benchmarks report exceeds 1 kHz when using with GPU acceleration, and 100 Hz with CPU-only inference. Depending on the configuration and hardware used, FlashSim is said to be between up to five orders of magnitude faster than Full Simulation.

The scientific value of this acceleration has been tested in a realistic CMS search for the Higgs boson decaying to two muons. FlashSim showed good agreement with Full Simulation for dimuon and dijet variables, as well as for global event properties, while preserving the complex correlations employed by the analysis’s machine-learning signal/background discriminator. These tests also revealed modelling weaknesses, including the fake-jet problem, which has made downstream analysis part of the model-validation process.

The generative formulation creates additional possibilities for increasing simulation statistics. As FlashSim produces stochastic detector responses, the same generator event can be simulated repeatedly. The CMS collaboration tested nine-fold oversampling to generate 828 million Drell–Yan events and reduce statistical fluctuations in sparsely populated regions of the distributions.

All in all, FlashSim, if / when used in the majorities on CMS analyses, allows fro a much faster turnaround of physics result publication, at a much reduced computing cost. Its use allows for a larger number of published analysis results, since they are often limited by the prompt availability of Full Simulation samples. It also allows to meet easily external deadlines, such as PhD dissertation preparation, cutting the time while waiting for the availability of Full Simulation datasets.

Its structure, while designed for a specific HEP-related use case, shows Conditional Flow Matching as a viable technique to reproduce complexly correlated data samples.

Conference presentations:

https://indico.cern.ch/event/1471803/contributions/6968002/
https://cds.cern.ch/record/2913372/files/DP2024_080.pdf