Best-fit Assessment for Models (BAM): an open-source Python-based statistical tool for assessing the performance of numerical models in volcanology

Abstract

The Best-Fit Assessment for Numerical Models (BAM) is a Python-based, open-source, modular, and versatile statistical tool designed to primarily evaluate the performance of numerical models in volcanology. BAM was developed to assess models that simulate the transport and deposition of volcanic mass flows, namely pyroclastic density currents, lava flows, lahars, and debris avalanches. BAM makes use of matrix arrays in the form of raster pairs, chiefly meant to compare the footprint of flow model outputs against user-provided observed geological evidence, such as mapped deposits. This comparison is achieved via a best-fit assessment, which, firstly, includes the computation of length metrics (e.g., percent-length ratio), a confusion matrix (i.e., statistical contingency table), and traditional similarity metrics (e.g., the Jaccard similarity coefficient, Dice-Sørensen coefficient, precision, and sensitivity). Secondly, BAM introduces an approach to incorporate any branching of the footprint geometry, termed the skeleton-aggregated percent-length ratio, and a method to more strictly evaluate areas of overestimation or underestimation, the function-transformed false positives and false negatives, respectively. These transformed results are reincorporated into the traditional similarity metrics to yield innovative and insightful function-transformed similarity metrics, completing the best-fit assessment procedure. This collection of measures establishes BAM as a robust framework to validate, calibrate, and benchmark numerical models focused on inundation areas for volcanic mass flows.

Bibliographic Information