Feature Importance

A useful by-product of Weight-of-Evidence transformation is information value, which is a way to measure feature importance. The iv() function provides a convenient way to calculate this statistic.

# Reverse levels of `default_status`; see ?woe "Details" section
loans$default_status <- factor(loans$default_status, levels = c("good", "bad"))

loans |>
  iv(
    outcome = default_status,
    predictors = c(amount_of_existing_debt, industry),
    verbose = FALSE
  )
#> # A tibble: 2 × 2
#>   variable                   iv
#>   <chr>                   <dbl>
#> 1 amount_of_existing_debt 0.666
#> 2 industry                0.169

Siddiqi (2017) advocates for interpreting the information value statistic as follows:

Information Value	Interpretation
< 0.02	Not predictive
[0.02, 0.1)	Weakly predictive
[0.1, 0.3)	Moderately predictive
[0.3, 0.5)	Strongly predictive
> 0.5	Likely overfit

We can add these interpretations directly to the data frame by setting labels = TRUE.

(iv <- loans |>
  iv(
    outcome = default_status,
    predictors = c(amount_of_existing_debt, industry),
    labels = TRUE,
    verbose = FALSE
  )
)
#> # A tibble: 2 × 3
#>   variable                   iv label                
#>   <chr>                   <dbl> <chr>                
#> 1 amount_of_existing_debt 0.666 Likely overfit       
#> 2 industry                0.169 Moderately predictive

We can see that amount_of_existing_debt is likely overfit, while industry is moderately predictive. You may want to consider omitting a variable whose information value indicates it is “Not predictive” or “Likely overfit”.