Skip to main content

Lund University Publications

LUND UNIVERSITY LIBRARIES

The InterModel Vigorish (IMV) as a flexible and portable approach for quantifying predictive accuracy with binary outcomes

Domingue, Benjamin W. ; Rahal, Charles ; Faul, Jessica ; Freese, Jeremy ; Kanopka, Klint ; Rigos, Alexandros LU ; Stenhaug, Ben and Tripathi, Ajay Shanker (2025) In PLoS ONE 20(3 March).
Abstract

Understanding the “fit” of models designed to predict binary outcomes has been a longstanding problem across the social sciences. We propose a flexible, portable, and intuitive metric for quantifying the change in accuracy between two predictive systems in the case of a binary outcome: the InterModel Vigorish (IMV). The IMV is based on an analogy to weighted coins, well-characterized physical systems with tractable probabilities. The IMV is always a statement about the change in fit relative to some baseline model— which can be as simple as the prevalence—whereas other metrics are stand-alone measures that need to be further manipulated to yield indices related to differences in fit across models. Moreover, the IMV is consistently... (More)

Understanding the “fit” of models designed to predict binary outcomes has been a longstanding problem across the social sciences. We propose a flexible, portable, and intuitive metric for quantifying the change in accuracy between two predictive systems in the case of a binary outcome: the InterModel Vigorish (IMV). The IMV is based on an analogy to weighted coins, well-characterized physical systems with tractable probabilities. The IMV is always a statement about the change in fit relative to some baseline model— which can be as simple as the prevalence—whereas other metrics are stand-alone measures that need to be further manipulated to yield indices related to differences in fit across models. Moreover, the IMV is consistently interpretable independent of baseline prevalence. We contrast this metric with alternatives in numerous simulations. The IMV is more sensitive to estimation error than many alternatives and also shows distinctive sensitivity to prevalence. We consider its performance using examples spanning the social and natural sciences. The IMV allows for precise answers to questions about changes in model fit in a variety of settings in a manner that will be useful for furthering research and the understanding of social outcomes.

(Less)
Please use this url to cite or link to this publication:
author
; ; ; ; ; ; and
organization
publishing date
type
Contribution to journal
publication status
published
subject
in
PLoS ONE
volume
20
issue
3 March
article number
e0316491
publisher
Public Library of Science (PLoS)
external identifiers
  • scopus:105001003865
  • pmid:40117240
ISSN
1932-6203
DOI
10.1371/journal.pone.0316491
language
English
LU publication?
yes
additional info
Publisher Copyright: © 2025 Domingue et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
id
b890e9a1-a214-4ffe-bb64-fdf843ee7d6c
date added to LUP
2025-04-04 07:45:10
date last changed
2025-07-11 15:38:51
@article{b890e9a1-a214-4ffe-bb64-fdf843ee7d6c,
  abstract     = {{<p>Understanding the “fit” of models designed to predict binary outcomes has been a longstanding problem across the social sciences. We propose a flexible, portable, and intuitive metric for quantifying the change in accuracy between two predictive systems in the case of a binary outcome: the InterModel Vigorish (IMV). The IMV is based on an analogy to weighted coins, well-characterized physical systems with tractable probabilities. The IMV is always a statement about the change in fit relative to some baseline model— which can be as simple as the prevalence—whereas other metrics are stand-alone measures that need to be further manipulated to yield indices related to differences in fit across models. Moreover, the IMV is consistently interpretable independent of baseline prevalence. We contrast this metric with alternatives in numerous simulations. The IMV is more sensitive to estimation error than many alternatives and also shows distinctive sensitivity to prevalence. We consider its performance using examples spanning the social and natural sciences. The IMV allows for precise answers to questions about changes in model fit in a variety of settings in a manner that will be useful for furthering research and the understanding of social outcomes.</p>}},
  author       = {{Domingue, Benjamin W. and Rahal, Charles and Faul, Jessica and Freese, Jeremy and Kanopka, Klint and Rigos, Alexandros and Stenhaug, Ben and Tripathi, Ajay Shanker}},
  issn         = {{1932-6203}},
  language     = {{eng}},
  number       = {{3 March}},
  publisher    = {{Public Library of Science (PLoS)}},
  series       = {{PLoS ONE}},
  title        = {{The InterModel Vigorish (IMV) as a flexible and portable approach for quantifying predictive accuracy with binary outcomes}},
  url          = {{http://dx.doi.org/10.1371/journal.pone.0316491}},
  doi          = {{10.1371/journal.pone.0316491}},
  volume       = {{20}},
  year         = {{2025}},
}