Benchmarking
Probabilistic Safety Assessment with a Small Tree Model Input
In safety-critical industries like nuclear energy, probabilistic safety assessment (PSA) is both a safety imperative and a computational burden. Engineers rely on the fault-tree method to quantify the risk of a facility-level failure, but evaluating fault tree models for complex systems using traditional Monte Carlo methods can be slow. This benchmark runs the SCRAM [1] PSA analysis tool hosted on the Signaloid Compute Engine to demonstrate the speedups that running over Signaloid’s compute platform enables. The input to SCRAM, for this benchmark, is a small fault tree model comprising four basic events, in the Open-PSA model exchange format.
Figure 1. Monte Carlo UQ implementation
When running on the Signaloid Cloud Compute Engine (SCCE), the implementation of SCRAM replaces the usual approach of sampling followed by evaluation of each sample, with a direct computation on a representation of the probability distribution across samples. Thus, in a single computation, SCRAM running on SCCE can compute the same type of output distribution that would take a Monte Carlo simulation thousands or hundreds of thousands of iterations.
Figure 2. Signaloid UQ implementation
SCRAM, evaluating the SmallTree top-event PSA model, and running on a single-threaded Signaloid C0Pro-L+ core, achieves runtimes 221× faster runtime than an optimized C-language Monte-Carlo-based implementation (i.e., the industry standard and status quo) of the same SCRAM tool running on an Amazon EC2 R7iz instance. This acceleration transforms a potentially-tedious Monte Carlo computation into a short real-time task, enabling, e.g., instantaneous risk assessment rather than retrospective analysis. With 95% confidence, a SCRAM implementation using a 2.1M-iteration Monte Carlo evaluation of this particular PSA fault tree will have the same accuracy as the Signaloid UxHw-based execution of SCRAM; if requiring higher confidence in the accuracy of the Monte Carlo compared to UxHw, the speedups of UxHw are even greater.
Key Performance Indicator
Signaloid Platform Solution
Competing Solution
Signaloid Benefit
Speed for the same uncertainty quantification accuracy.
Run existing non-Monte-Carlo code and use either the Signaloid Compute Engine's automated ingestion of distribution information, or use the Signaloid UxHw API to set program variables as probability distributions.
Run existing Monte Carlo code, or, starting from non-Monte-Carlo code, modify code to implement Monte Carlo sampling, iteration, and aggregation of the results from the Monte Carlo iterations of the computation.
221x faster execution time than 2.18M-iteration Monte Carlo, while achieving same fidelity of full distribution result.
Plot of output distribution when running on a Signaloid C0Pro-L+ core that provides the 221× speedup.
Plot of the output of an 2.18M-iteration Monte Carlo for this use case. Signaloid C0Pro-L+ core is 221× faster than this Monte Carlo, while achieving the same or better Wasserstein distance to the ground-truth (20M-iteration) Monte Carlo.
Plot of ground truth (20M-iteration) Monte Carlo.
Benchmarking Methodology
Monte Carlo simulations work by statistical sampling and therefore each multi-iteration Monte Carlo run will result in a slightly different output distribution. By contrast, Signaloid's platform is deterministic and each run produces the same distribution for a given Signaloid C0 core type.
The performance improvements are calculated by comparing Signaloid's platform with a Monte Carlo simulation of a similar quality of distribution. First, we run a large Monte Carlo simulation (about 50M iterations) on an AWS r7iz high-performance AWS instance: We use this distribution result as a baseline or ground truth reference of distribution quality. Then we calculate the performance of Signaloid's technology, and compare it with the performance of a Monte Carlo iteration count where the output distribution's Wasserstein distance (to the output distribution of the ground truth reference) is as accurate as the Signaloid-core-executed algorithm's output distribution, with 95% confidence level.
Performance data based on Spring 2026 release of Signaloid's technology.