Analyst: Jennifer Fouquier

EXPLANA Description and Analysis Summary

EXPLANA Description

EXPLANA uses established machine-learning methods combined with innovative techniques to identify the most relevant features, from a variety of input features, that correlate with a response variable.

Input features and response variables can be numerical, categorical, or from non-normal data distributions. The relationships between selected features and the response can be linear or complex, non-linear relationships.

For longitudinal datasets, changes in features for each study identifier, often subjects, are calculated using different reference points to obtain delta datasets (First, Previous and Pairwise delta datasets). This is important because features in longitudinal studies can carry varying degrees of importance between models built using different reference points. EXPLANA automates feature selection from several models built using these datasets. This report was generated to textually and graphically summarize exploratory analysis and aid hypothesis generation.

Please ensure you understand the workflow, parameters/decisions, and that the percent variation explained (using out-of-bag (OOB) scores) is adequate for your purposes.

When you are using data without prior hypotheses, you are performing exploratory analysis and should make this clear when communicating results.

Response Variable: Crude_Rate_per_100000

Analysis Notes:

Methods

Feel free to use the following text, including citation information, for use in methods to ensure reproducibility:

EXPLANA was used for exploratory analysis to identify important features related to the response variable, Crude_Rate_per_100000.

A random effect of State was used to adjust for non-independence (repeated measurements) if needed. There were 1000 trees used per Random Forest model with a max feature fraction of 0.2 of the input features for each split per decision tree in the forest. If mixed effects Random Forests were needed, 20 iterations were performed. BorutaSHAP was used to find features that perform repeatedly better than shuffled versions of all input features. Features were considered important if they performed better than 97% of the SHAP importance score of the best shuffled feature using 100 trials, p=0.05. Categorical variables were binary encoded and low occuring categorical values were not removed.

These methods are from an EXPLANA feature selection report (version: 2025.05.09) created on 2026-09-08. Additional information can be found at https://github.com/JTFouquier/explana/.

Config File

analyst: Jennifer Fouquier
response_var: Crude_Rate_per_100000
include_time: 'yes'
random_effect: State
sample_id: sample_id
timepoint: Year
out: workflow-results/EXPLANA-crc-Crude_Rate_per_100000-mixed-model-2012-2016-complete-columns-20260908/
iterations: '20'
n_estimators: '1000'
max_features: '0.2'
borutashap_trials: '100'
borutashap_threshold: '97'
borutashap_p: '0.05'
analyze_original: 'yes'
analyze_first: 'yes'
analyze_previous: 'yes'
analyze_pairwise: 'yes'
absolute_values: 'no'
include_reference_values: 'no'
analysis_notes: ''
enc_percent_threshold: '0'
distance_matrices: list()
df_mod: ''
delta_df_mod: |
  delta_df <- delta_df
input_datasets:
  metadata:
    file_path: /Users/jenniferfouquier/repos/snakemake-merf/data/crc/explana_colorectal_complete_case_columns.csv
    df_mod: |
      df <- df %>%
        select(-Total_Count_Known) %>%
        filter(Year %in% c(2012, 2014, 2016)) %>%
        select(
          -N_Age_Groups_Suppressed,
          -N_Age_Groups,
          -Current_Smoker_Status__No,
          -Alcohol_Consumption__No,
          -Binge_Drinking__No,
          -Diabetes__No,
          -Exercise__No,
          -Fair_or_Poor_Health__Fair_or_Poor_Health
          )
    dim_method: ''
    dim_param_dict:
      method: none

Results Summary

Selected Features by Dataset

Data Original First Previous Pairwise
% Variance Explained 54.3% (57.7%) -5.1% -11.7% -6.3%
N Trees 1000 1000 1000 1000
Feature fraction/split 0.2 0.2 0.2 0.2
Max Depth 7 7 7 7
MERF Iters. 20 20 20 20
BorutaSHAP Trials 100 100 100 100
BorutaSHAP Threshold 97 97 97 97
P-value 0.05 0.05 0.05 0.05
N Study IDs 51 51 51 51
N Samples 153 102 102 153
Input Features 21 NA NA NA
Accepted Features 13 NA NA NA
Tentative Features 2 NA NA NA
Rejected Features 5 NA NA NA
Model Type (Pass/Fail) MERF PASS; Boruta PASS MERF FAIL (low variance) MERF FAIL (low variance) MERF FAIL (low variance)

Selected feature ranks from models built using Original and, for longitudinal analyses, First, Previous and Pairwise delta datasets. Selected features are shown in black and labeled with feature rank. For true/positive instances of categorical variables (indicated with “ENC” after encoding), average impact on response is shown after the rank. For numerical features, impact is not shown because the feature relationship to response can be complex, requiring further post-hoc tests or inspection of SHAP dependence plots for additional insight. Empty grey boxes indicate features included in the model for a dataset, but not selected. Long feature names may be truncated and indicated with ellipses. A comprehensive list of input features can be found in model details below.

Original Dataset

⇨ Open File Directory

Model Summary

Data Original
Model Type (Pass/Fail) MERF PASS; Boruta PASS
% Variance Explained 54.3% (57.7%)
N Trees 1000
Feature fraction/split 0.2
Max Depth 7
MERF Iters. 20
BorutaSHAP Trials 100
BorutaSHAP Threshold 97
P-value 0.05
N Study IDs 51
N Samples 153
Input Features 21
Accepted Features 13
Tentative Features 2
Rejected Features 5

SHAP summary beeswarm plots of feature influence on the machine learning prediction of response values. Each point represents one sample, and the horizontal position indicates impact on the response as indicated on the x-axis. Points to the left indicate a negative impact, and points to the right indicate a positive impact. The colors represent the selected feature values, where red is larger and blue is smaller. For binary encoded features (‘ENC’) red is yes[1] and blue is no[0]. SHAP is generally an improvement upon other importance scores because it provides information about both rank (how helpful the feature was compared to other features [y-axis]) and impact (a positive or negative impact on response values [x-axis]). If multiple figures are shown, scales may vary with a maximum of ten features per plot.

Selected Features

important_features decoded_features feature_importance_vals
Diabetes__Yes Diabetes__Yes 0.3490
Total_Population Total_Population 0.2932
Fair_or_Poor_Health__Good_or_Better_Health Fair_or_Poor_Health__Good_or_Better_Health 0.2711
Overall_Health__Fair Overall_Health__Fair 0.2606
Exercise__Yes Exercise__Yes 0.2583
BMI_Categories__Obese_BMI_30_0_99_8 BMI_Categories__Obese_BMI_30_0_99_8 0.1758
Overall_Health__Poor Overall_Health__Poor 0.1410
BMI_Categories__Normal_Weight_BMI_18_5_24_9 BMI_Categories__Normal_Weight_BMI_18_5_24_9 0.1050
Overall_Health__Excellent Overall_Health__Excellent 0.0988
Overall_Health__Very_good Overall_Health__Very_good 0.0885
Current_Smoker_Status__Yes Current_Smoker_Status__Yes 0.0758
BMI_Categories__Overweight_BMI_25_0_29_9 BMI_Categories__Overweight_BMI_25_0_29_9 0.0645
Smoker_Status__Smoke_some_days Smoker_Status__Smoke_some_days 0.0306

Feature Stats

important_features feature_importance_vals mean std min 25% 50% 75% max
Diabetes__Yes 0.3490 10.087 1.710 6.6 8.9 9.9 11.2 15.0
Total_Population 0.2932 2509411.928 2930373.749 224263.0 641358.0 1741848.0 2846929.0 16481311.0
Fair_or_Poor_Health__Good_or_Better_Health 0.2711 82.863 3.414 73.7 81.0 83.4 85.6 88.4
Overall_Health__Fair 0.2606 12.396 2.028 8.7 10.9 12.3 13.6 17.6
Exercise__Yes 0.2583 76.886 3.929 67.5 74.7 76.9 79.8 84.3
BMI_Categories__Obese_BMI_30_0_99_8 0.1758 28.987 3.575 20.5 26.6 29.1 31.4 37.7
Overall_Health__Poor 0.1410 4.750 1.539 2.6 3.6 4.4 5.5 9.1
BMI_Categories__Normal_Weight_BMI_18_5_24_9 0.1050 33.735 3.376 26.8 31.4 33.4 35.7 46.0
Overall_Health__Excellent 0.0988 18.824 2.487 12.5 17.4 18.8 20.4 27.3
Overall_Health__Very_good 0.0885 33.042 2.956 27.2 30.7 33.7 35.4 38.9
Current_Smoker_Status__Yes 0.0758 18.590 3.626 8.8 16.3 18.1 21.0 28.3
BMI_Categories__Overweight_BMI_25_0_29_9 0.0645 35.443 1.330 30.0 34.7 35.5 36.3 39.1
Smoker_Status__Smoke_some_days 0.0306 5.433 0.899 3.2 4.8 5.4 6.0 8.2

Input Features

input_features was_selected
Total_Population yes
Alcohol_Consumption__Yes no
BMI_Categories__Normal_Weight_BMI_18_5_24_9 yes
BMI_Categories__Obese_BMI_30_0_99_8 yes
BMI_Categories__Overweight_BMI_25_0_29_9 yes
Binge_Drinking__Yes no
Current_Smoker_Status__Yes yes
Diabetes__No_pre_diabetes_or_borderline_diabetes no
Diabetes__Yes yes
Exercise__Yes yes
Fair_or_Poor_Health__Good_or_Better_Health yes
Overall_Health__Excellent yes
Overall_Health__Fair yes
Overall_Health__Good no
Overall_Health__Poor yes
Overall_Health__Very_good yes
Smoker_Status__Former_smoker no
Smoker_Status__Never_smoked no
Smoker_Status__Smoke_everyday no
Smoker_Status__Smoke_some_days yes
timepoint_explana no

Interpretation/Literature Search

The following links can help with hypothesis generation. Names of variables likely need modification.

important_features url
Diabetes__Yes https://pubmed.ncbi.nlm.nih.gov/?term=Diabetes__Yes%20AND%20Crude_Rate_per_100000
Total_Population https://pubmed.ncbi.nlm.nih.gov/?term=Total_Population%20AND%20Crude_Rate_per_100000
Fair_or_Poor_Health__Good_or_Better_Health https://pubmed.ncbi.nlm.nih.gov/?term=Fair_or_Poor_Health__Good_or_Better_Health%20AND%20Crude_Rate_per_100000
Overall_Health__Fair https://pubmed.ncbi.nlm.nih.gov/?term=Overall_Health__Fair%20AND%20Crude_Rate_per_100000
Exercise__Yes https://pubmed.ncbi.nlm.nih.gov/?term=Exercise__Yes%20AND%20Crude_Rate_per_100000
BMI_Categories__Obese_BMI_30_0_99_8 https://pubmed.ncbi.nlm.nih.gov/?term=BMI_Categories__Obese_BMI_30_0_99_8%20AND%20Crude_Rate_per_100000
Overall_Health__Poor https://pubmed.ncbi.nlm.nih.gov/?term=Overall_Health__Poor%20AND%20Crude_Rate_per_100000
BMI_Categories__Normal_Weight_BMI_18_5_24_9 https://pubmed.ncbi.nlm.nih.gov/?term=BMI_Categories__Normal_Weight_BMI_18_5_24_9%20AND%20Crude_Rate_per_100000
Overall_Health__Excellent https://pubmed.ncbi.nlm.nih.gov/?term=Overall_Health__Excellent%20AND%20Crude_Rate_per_100000
Overall_Health__Very_good https://pubmed.ncbi.nlm.nih.gov/?term=Overall_Health__Very_good%20AND%20Crude_Rate_per_100000
Current_Smoker_Status__Yes https://pubmed.ncbi.nlm.nih.gov/?term=Current_Smoker_Status__Yes%20AND%20Crude_Rate_per_100000
BMI_Categories__Overweight_BMI_25_0_29_9 https://pubmed.ncbi.nlm.nih.gov/?term=BMI_Categories__Overweight_BMI_25_0_29_9%20AND%20Crude_Rate_per_100000
Smoker_Status__Smoke_some_days https://pubmed.ncbi.nlm.nih.gov/?term=Smoker_Status__Smoke_some_days%20AND%20Crude_Rate_per_100000

Log


Binary encoded columns created for categorical input variables:

BorutaSHAP Figures

⇨ Open PDF in new window

First Delta Dataset

⇨ Open File Directory

Results

Model Summary

Data First
Model Type (Pass/Fail) MERF FAIL (low variance)
% Variance Explained -5.1%
N Trees 1000
Feature fraction/split 0.2
Max Depth 7
MERF Iters. 20
BorutaSHAP Trials 100
BorutaSHAP Threshold 97
P-value 0.05
N Study IDs 51
N Samples 102
Input Features NA
Accepted Features NA
Tentative Features NA
Rejected Features NA

Selected Features

important_features decoded_features feature_importance_vals
no_selected_features NA -100

Feature Stats

Analysis not completed

Input Features

Analysis not completed

Interpretation/Literature Search

The following links can help with hypothesis generation. Names of variables likely need modification.

important_features url
no_selected_features https://pubmed.ncbi.nlm.nih.gov/?term=no_selected_features%20AND%20Crude_Rate_per_100000

Log


Binary encoded columns created for categorical input variables:

 ->  timepoint_explana: ['ENC_timepoint_explana_is_1__2', 'ENC_timepoint_explana_is_1__3']

BorutaSHAP Figures

⇨ Open PDF in new window

Previous Delta Dataset

⇨ Open File Directory

Model Summary

Data Previous
Model Type (Pass/Fail) MERF FAIL (low variance)
% Variance Explained -11.7%
N Trees 1000
Feature fraction/split 0.2
Max Depth 7
MERF Iters. 20
BorutaSHAP Trials 100
BorutaSHAP Threshold 97
P-value 0.05
N Study IDs 51
N Samples 102
Input Features NA
Accepted Features NA
Tentative Features NA
Rejected Features NA

Selected Features

important_features decoded_features feature_importance_vals
no_selected_features NA -100

Feature Stats

Analysis not completed

Input Features

Analysis not completed

Interpretation/Literature Search

The following links can help with hypothesis generation. Names of variables likely need modification.

important_features url
no_selected_features https://pubmed.ncbi.nlm.nih.gov/?term=no_selected_features%20AND%20Crude_Rate_per_100000

Log


Binary encoded columns created for categorical input variables:

 ->  timepoint_explana: ['ENC_timepoint_explana_is_1__2', 'ENC_timepoint_explana_is_2__3']

BorutaSHAP Figures

⇨ Open PDF in new window

Pairwise Delta Dataset

⇨ Open File Directory

Model Summary

Data Pairwise
Model Type (Pass/Fail) MERF FAIL (low variance)
% Variance Explained -6.3%
N Trees 1000
Feature fraction/split 0.2
Max Depth 7
MERF Iters. 20
BorutaSHAP Trials 100
BorutaSHAP Threshold 97
P-value 0.05
N Study IDs 51
N Samples 153
Input Features NA
Accepted Features NA
Tentative Features NA
Rejected Features NA

Selected Features

important_features decoded_features feature_importance_vals
no_selected_features NA -100

Feature Stats

Analysis not completed

Input Features

Analysis not completed

Interpretation/Literature Search

The following links can help with hypothesis generation. Names of variables likely need modification.

important_features url
no_selected_features https://pubmed.ncbi.nlm.nih.gov/?term=no_selected_features%20AND%20Crude_Rate_per_100000

Log


Binary encoded columns created for categorical input variables:

 ->  timepoint_explana: ['ENC_timepoint_explana_is_1__2', 'ENC_timepoint_explana_is_1__3', 'ENC_timepoint_explana_is_2__3']

BorutaSHAP Figures

⇨ Open PDF in new window

See the Github repository for more information.