KINS - psykei/psyki-python Wiki

Theory

KINS: knowledge injection via network structuring.

KINS is a SKI algorithm based on structuring. This means that the injection step consists in creating ad-hoc layers (modules) from the prior symbolic knowledge which mimic the behavior of the provided formulae. One can decide at which layer from the starting predictor inject the new modules.

kins architecture

X is the network input, Hi are hidden layers of the network, Y is the output layer. Blue boxes are modules Rj created from logic rules representing the prior knowledge. Modules can be trainable or not, depending on the user's choice.

API

To create a new KINS injector the user must provide a prior neural network built with tensorflow and a mapping between the names of the variables used in the logic rules and their corresponding feature index:

injector = NetworkComposer(my_neural_network, feature_variable_map)

For instance:

feature_variable_map = {'PL': 0, 'PW': 1, 'SL': 2, 'SW': 3}

is a mapping for the well known iris dataset (PL is petal length, etc.).

By default the fuzzifier of this injector is SubNetworkBuilder. This means that the provided formulae will be encoded into ad-hoc layers following a well defined mapping from crisp logic rules into a continuous interpretation (in range of [0, 1]) in such a way that each module outputs a value close to 1 if the evaluation of the formula is true, 0 otherwise. For more details about this fuzzification we suggest to read the paper. User can also specify the layer where to perform the injection.

The user can inject a set of Formula(e) and obtain a new predictor:

new_predictor = injector.inject(my_formulae)

kins api

Running example

Knowledge provided in form of textual Datalog rules.

class(PL,PW,SL,SW,setosa) ← PL ≤ 2.28
class(PL,PW,SL,SW,virginica) ← PL > 2.28 ∧ PW > 1.64
class(PL,PW,SL,SW,versicolor) ← PL > 2.28 ∧ PW ≤ 1.64
import unittest
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import OneHotEncoder
from tensorflow.keras import Input, Model
from tensorflow.python.framework.random_seed import set_random_seed
from psyki.logic.datalog.grammar.adapters.antlr4 import get_formula_from_string
from psyki.ski.injectors import NetworkComposer
from test.resources.rules import get_rules
from test.utils import get_mlp


EPOCHS = 50
BATCH_SIZE = 8
VERBOSE = 0
x, y = load_iris(return_X_y=True, as_frame=True)
encoder = OneHotEncoder(sparse=False)
encoder.fit_transform([y])
dataset = x.join(y)
train, test = train_test_split(dataset, test_size=0.5, random_state=0)
train_x, train_y = train.iloc[:, :-1], train.iloc[:, -1]
test_x, test_y = test.iloc[:, :-1], test.iloc[:, -1]
variable_mapping = {'PL': 0, 'PW': 1, 'SL': 2, 'SW': 3}

set_random_seed(0)
formulae = [get_formula_from_string(rule) for rule in get_rules('iris')]
input_layer = Input((4,))
predictor = get_mlp(input_layer, 3, 3, 32, 'relu', 'softmax')
predictor = Model(input_layer, predictor)
injector = NetworkComposer(predictor, variable_mapping, layer=2)
new_predictor = injector.inject(formulae)
new_predictor.compile('adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
new_predictor.fit(train_x, train_y, batch_size=BATCH_SIZE, epochs=EPOCHS, verbose=VERBOSE)
⚠️ **GitHub.com Fallback** ⚠️