Relevance-Based Importance
By Megan Czasonis, Mark Kritzman, and David Turkington
As the race to design sophisticated data analytics continues, we show why relevance-based prediction offers an ideal way to measure the importance of an input variable to a prediction.
T-statistics act as a hallmark for rigor by pinpointing the effect of a single variable and distinguishing signal from noise. However, they have significant limitations: (1) t-stats do not capture ‘shared’ information, (2) t-stats are not prediction-specific, and (3) t-stats only consider linear relationships. In a recent paper, we introduce an alternative method, called Relevance-Based Importance (RBI), which measures the importance of every variable to the reliability of every individual prediction. RBI recognizes that it is almost never the case that a variable is always important, or that it is never important. Rather, it's more likely that variables are sometimes important, depending on the circumstance. We show that RBI brings the virtues of t-statistics but also adapts to each unique situation, making it robust to complexities where t-stats fall short.
