Introspecting Models

Introspecting the model takes two steps: The first step is to extract logical formulae from the machine learning models; the second step is to perform automated reasoning on the logical formulae.

Settings For Formula Extraction

There are a few parameters for the logical formula extraction process, and they are organised into a settings file in JSON format. The parameters include:

  • tree_sampling: Sample a subset of trees from the ensemble tree model for logical analysis.
    • method: The method used to sample the trees. The options are:

      • uniform: sample uniformly regardless of the weights.

      • best: sample the best (sub-)models with regard to their weights.

      • worst: sample the worst (sub-)models with regard to their weights.

    • proportion: The proportion for sampling. For instance, 0.2 means that only 20% of trees in the ensemble tree model are sampled.

  • branch_sampling: Sample a subset of branches from each decision tree.

    • method: Same as above.

    • proportion: Same as above.

  • node_sampling: Sample a subset of nodes from each branch.

    • method: Same as above.

    • proportion: Same as above.

To generate a formula extraction settings file with default values, run the following command:

silas gen-extract-settings [OPTIONS]

where OPTIONS include:

  • -h: Print help message and exit.

  • -o file_path: Output the settings file in the directory file_path. If this option is not supplied, the extraction settings will be stored in extract-settings.json in the directory where the command is issued.

To extract formulae from machine learning model(s) specified in a settings file, run the following command:

silas extract [OPTIONS] [settings_file] [model_dir] [class_label]

where settings_file is the path of the formula extraction settings file, model_dir is the directory where the machine learning models are stored, and class_label (string) is the outcome class to be analysed. The OPTIONS include:

  • -h: Print help message and exit.

  • -o formula_dir: Output the formulae in the directory formula_dir. If this option is not supplied, the output will be in extracted-formulae/tmp/.

For instance, to extract formulae using formulae/model1/extract-settings1.json from models in models/model1/ for class “Positive” and output the formulae in formulae/model1/, run the following command:

silas extract -o formulae/model1/ formulae/model1/extract-settings1.json models/model1/ Positive

Introspecting The Extracted Formulae

To introspect the extracted formulae and obtain explanations of the machine learning models, use the following command:

silas introspect [formula_dir]

where formula_dir is the directory that stores the extracted formulae. For instance, if the formulae are stored in formulae/model1/, run the following instance:

silas introspect formulae/model1/