Research Journal - GeorgeIniatis/Blood_Brain_Barrier_Drug_Prediction GitHub Wiki

Singh et al. (A classification model for blood brain barrier penetration)

Summary

This study aimed to produce improved models to predict the BBB permeability of drugs since traditional methods are time consuming and expensive. It used a dataset of 605 compounds with 2 classification thresholds using the Brain/Plasma ratio. The B/P ratio is the most appropriate method to determine the extent of brain penetration when analysing brain pharmacokinetics (Threshold-1: Brain/Plasma ≥ 0.6 as BBB+ and Brain/Plasma < 0.6 as BBB- and Threshold-2: Brain/Plasma > 0.6 as BBB+ and Brain/Plasma<0.3 as BBB-). The dataset was curated to remove duplicates, inorganic material, mixtures and compounds with both BBB+ and BBB- natures. It was then divided into training and testing sets in a 4:1 ratio although the sets varied a bit for the 2 thresholds and multiple classification models were trained.

A single model could not produce good results for both thresholds and therefore a consensus model was build that gave similar results for both of them with an accuracy of 86% for threshold 1 and 87% for threshold 2. A consensus model trains and combines multiple classifiers and therefore mitigates overfitting problem associated with a single classifier, however, it naturally requires more computational power. This consensus model was proved to be superior to models built in the past by other studies after the Accuracy, Matthew's Correlation Coefficient, Sensitivity, Specificity, and Correct Classification Ratio were compared.

Finally the study discovered a list of corroborated substructures that were more prevalent in BBB+ compounds and drugs.

Context

This is the first paper I read while researching the project topic. I learned how other people tackled the problem and what methods they used and I gained access to a dataset that I could then use to construct my own.

Useful Information

Multiple classification models have been built in the past by previous studies for the same purpose however they used small datasets that couldn't generalise effectively to other external datasets, since they didn't have access to them or they didn't exist at the time. Therefore these models are not suitable for high-throughput screening of new drugs.

Shortcomings of models:

  • "Neural network model development requires higher computational time. This shortcoming can be rectified by removing each input variable node of the neural network and then observing its performance. Some regression-like methods are used to understand the specific connection weights related to input variables that can be removed without altering the performance."
  • "Training neural network models with many hidden layers may lead to overfitting which in turn causes poor performance in external test data sets. Additionally, this potential pitfall can be overcome by reducing the amount of training, using cross validation and introducing bootstrapping method. The bootstrapping method uses all the data to train the network and evaluate performance [20]. The use of external test/validation data sets is a fool-proof method to test for over-training or over-fitting."

Contributions

  • Useful vocabulary of abbreviations
  • Useful background information
  • Large curated dataset
  • Identification of substructures (Table 13) that seem to influence brain permeability
  • Creation of superior consensus model

Related Work

Methods

  • Created 3 different classification models.
    • One using Random Forest
    • One using Multilayer Perceptron
    • One using Sequential Minimal Optimization
  • Combined these models to produce 4 consensus models
  • All models were validated using a 10-fold cross validation, external testing sets and y-scrambling
  • The models performance was measured by using the following metrics:
    • Accuracy
    • Matthew's Correlation Coefficient
    • Sensitivity
    • Specificity
    • Correct Classification Ratio
    • For exact calculation see Section 2.5

Outcomes

  • The greater number of BBB+ compounds and CNS drugs are in the range
    • 200 to 400 for Molecular Weight
    • -1 to +1 for AlogP
    • 0 to 2 for LogD
    • 0 to 100 for PSA
    • 0 to 8 for HBAs
    • 0 to 2 for HBDs
    • 0 – 6 for RBs
    • Charge property is also important (Don't really understand it. Will have another look if needed)

Conclusions

This study created a superior consensus model using a new curated dataset that can be used in the early screening for brain permeability and discovered a list of corroborated substructures that were more prevalent in BBB+ compounds and drugs.

Saber et al. (A machine learning model for the prediction of drug permeability across the Blood-Brain Barrier: a comparative approach)

Summary

This study aimed to produce improved models to predict the BBB permeability of drugs through the use of sequential feature selection (SFS) and genetic algorithms (GA). These algorithms select the most relevant descriptors and therefore should improve the predictive ability of the models. Six different classifiers were initially trained on the Zhao's et al. dataset using a subset of 19 molecular descriptors ,again taken from the same study, and would act as the baseline for the study. Then SFS and GA were performed independently and the same classifiers were trained again using the subset of descriptors chosen by each algorithm and compared with the baseline.

The highest overall accuracy without feature selection was 93.35%. This accuracy increased with sequential feature selection and genetic algorithms on multiple classifiers. The highest accuracy 96.23% was obtained after performing the genetic algorithms on the feature vector. Genetic algorithms with fitness function based on the performance of a support vector machine (SVM) led to an increase in accuracy of all the classifiers unlike sequential feature selection which led the study to conclude that GA is a more robust approach than SFS in choosing the most relevant molecular drug descriptors for brain permeability.

Finally the study discovered the importance of the polar surface area of drugs in BBB permeability

Context

I learned about feature selection techniques and discovered some important descriptors that I could make us of when training my models.

Useful Information

  • Binary models generally have a better predictionary accuracy than Quantitative models
  • Quantitative models attempt to quantify the brain permeability of a given drug by computing the logarithm of the ratio of the concentration of the drug in the brain to that in blood (logBB) or its penetration rate (PR)
    • Models based on logBB values are prone to biases introduced by the threshold setting
  • Classifiers can be trained using molecular descriptors, fingerprints of molecules and even the drug side effects

Contributions

  • Useful background information
  • Creation of models with high accuracy
  • Highlight important of polar surface area

Related Work

Methods

  • Six different classifiers were trained
    • Support Vector Machines (SVM) with Linear, Polynomial and RBF kernels
    • Linear Discriminant Analysis (LDA)
    • k-Nearest Neighbours (k-NN)
    • Quadratic Discriminant Analysis (QDA)
  • The classifiers were trained on the Zhao's et al. dataset using a subset of 8 molecular descriptors ,again taken from the same study.
    • Molecular weight (MW)
    • Polar surface area (PSA) Extremely Important
    • Octanol/water partition (logP)
    • Number of hydrogen bond acceptors (HA)
    • Number of hydrogen bond donors (HD)
    • pKa (Strongest acid)
    • pKa (Strongest base)
    • Number of rotatable bonds (NRB)
  • Then sequential feature selection and genetic algorithms were used to select the most relevant descriptors and the classifiers were trained using the descriptors chosen by each algorithm. The selection of the most relevant features is crucial since it guarantees an improved prediction performance on one hand and a faster computation on the other by reducing the size of the features vectors
  • The performance of each classifier was individually evaluated by using a confusion matrix, ROC curve, and a test set (4:1 ratio used on the dataset)
  • The best classifiers after descriptor selection by GA were QDA and SVM with Polynomial kernel

Conclusions

This study concluded that GA is a more robust approach than SFS in choosing the most relevant molecular drug descriptors and reiterated the importance of the polar surface area of drugs in BBB permeability.

Zhao et al. (Predicting Penetration Across the Blood-Brain Barrier from Simple Descriptors and Fragmentation Schemes)

Summary

This study aimed to reduce the high number of descriptors needed to train classification models for brain permeability. Using Algorithm Builder and fragmentation schemes, 19 molecular descriptors and a chain fragmentation scheme were calculated which were then used to build prediction models using binomial partial least square regression (PLS) and recursive partitioning (RP). The models used a subset of the 19 descriptors or the chain fragmentation scheme and were trained using a modified Adenot's and Lahana's dataset and were tested on two different sets, the first one being a subset of the the modified Adenot's and Lahana's dataset simulating the same chemical space and the second one being a modified Li's et al. dataset simulating a different chemical space.

The analysis revealed that hydrogen-bonding properties such as hydrogen bonding acidity and basicity (pKa), the polar surface area (PSA) and the number of hydrogen bonding donors (NHD) and acceptors (NHA) played a huge part in modelling brain permeability. The models built had an accuracy of roughly 90% for the training, 95% for the testing set within the same chemical space and 75% for the testing set with the different chemical space.

Context

Gained access to two great datasets and discovered important descriptors that I could make us of when training my models.

Useful Information

  • LogBB is a variable used to commonly express the extend of a drug passing through the BBB. Even though it is a very direct approach, the process needed to extract it is very expensive and time consuming and therefore not suitable for high throughput.
  • Drug CNS activity implies BBB+ but the opposite cannot be said for CNS inactive drugs. Drugs that are CNS inactive can still cross into the brain
  • Small datasets lead to models not generalising
  • Past models used up to 199 descriptors to train their models

Cruciani et al. categorised compounds as:

  • BBB+ if LogBB > 0.0
  • BBB is Uncertain if LogBB was between -0.3 and 0.0
  • BBB- if LogBB < -0.3

Li et al. categorised compounds as:

  • BBB+ if LogBB >= -1
  • BBB- if LogBB < -1

Contributions

  • Modified Adenot's and Lahana's dataset. The compounds whose structure could not be found and the compounds that were permanently charged were removed.
  • Modified Li's et al. dataset. The compounds whose structure was the same and the compounds that were permanently charged were removed

Related Work

  • Pointed me to Cruciani et al. paper and dataset of 97 compounds
  • Pointed me to Li et al. paper and dataset of 415 compounds
  • Pointed me to Algorithm Builder developed by PharmaAlgorithms which calculates molecular descriptors based on SMILES

Methods

  • Calculated 19 molecular descriptors, Abraham and Hydrogen-Bonding (Have a look at Table 1)
  • Using a subset of the descriptors RC and binomial PLS models were trained
  • Fragmentation scheme approach developed by PharmaAlgorithms, Inc also used to train PLS models (No.12 and 17 in Table 2)
  • Used a 2:1 ratio for the sets, taken from the modified Adenot's and Lahana's using the Kennard-Stone method. "The Kennard-Stone method selects the test and training sets according to the compound descriptors, so that the test set occupies the same chemical space as the training set".

Outcomes

  • Correlation study was conducted to see which descriptors were correlated
    • A with NHD and PCA
    • B with NHA and PCA
    • S with NHA
    • PSA with NHD, NHA, A and B
    • V with MW
  • Important comparisons in Table 2

Conclusions

The study concluded that hydrogen-bonding properties such as hydrogen bonding acidity and basicity (pKa), the polar surface area (PSA) and the number of hydrogen bonding donors (NHD) and acceptors (NHA) played a huge part in modelling brain permeability.

Gao et al. (Predict drug permeability to blood–brain-barrier from clinical phenotypes drug side effects and drug indications)

Summary

This study aimed to produce models to predict BBB permeability of drugs and compounds by using their well recorded side effects and indications, instead of their chemical structures. It used a training dataset of 213 drugs with known BBB and extracted their side effects and indications, using the SIDER database, and mapped them to 43 subgroups, to train SVM models with multiple kernels (Polynomial, Normalised-Polynomial, Radial Basis Function). The model with the polynomial kernel had the best performance.

When both chemical features and clinical phenotypes(side effects and indications) were available and combined the model achieved significantly better performance than solely chemical feature based approaches.

Context

Presented a new interesting approach, the usage of drug side effects and indications, that I could make us of when training my models.

Useful Information

  • BBB prevents 98% of external compounds from entering the brain
  • Models using chemical descriptors only, generally work for small compounds/drugs that use passive diffusion to pass the BBB
  • Many molecules such as glucose and insulin use other methods to pass the BBB which are hard to be described by chemical features

Contributions

New novel approach for BBB prediction

Related Work

  • Pointed me to the SIDER database

Methods

  • Curated a training dataset of 213 drugs with known BBB (See Table 1 of Supplementary Data) and an independent testing set (See Table 2 of Supplementary Data)
  • For each drug the side effects and indications were extracted as features using the SIDER database
  • Multiple SVM models with different kernels were trained (Polynomial, Normalised-Polynomial, Radial Basis Function) and their performances were evaluated through Monte Carlo cross validation.
  • Their performances were compared (See Table 4 of Supplementary Data. Note: Only Side Effects were used for predictions) and the SVM model with the polynomial kernel was proven to have the best performance
  • The model was trained on the training set, using the side effects, indications, and side effects and indications combined, as features and its performance was evaluated again using Montel Carlo cross validation (See Table 1 of Paper)
  • The trained model was applied to the independent training set. (See Table 3 in Paper for results)
  • Another model was trained on the testing set, using clinical phenotypes, chemical descriptors, and clinical phenotypes and chemical descriptors combined, and its performance was evaluated again using cross validation. (See Table 4 of Paper)
  • The SVM model with the polynomial kernel was then used to make predictions on the SIDER database, after removing the drugs from the training set and the independent testing set and extracting their side effects and indications. (See Table 6 of Supplementary Data)

GitHub Link for Implementation

Conclusions

The study concluded that brain permeability can be predicted by creating models that use drug side effects and indications as features and that these models can be further improved by integrating the chemical structure of drugs as additional features.

Zhang et al. (QSAR Modelling of the Blood–Brain Barrier Permeability for Diverse Organic Compounds)

Summary

This study aimed to develop QSAR models to predict BBB permeability. Support Vector Machines (SVM) and k-Nearest-Neighbour (kNN) models were trained using a training set of 144 compounds and all 6 combinations of 3 collections of descriptors (MolconnZ, MOE, Dragon). The models were evaluated on 3 external sets and a consensus model was built at the end from all statistically significant models.

Context

Gained access to three great datasets and discovered important descriptors that I could make us of when training my models.

Useful Information

  • The ability of a drug or compound to enter the brain can be estimated by measure the brain-to-blood concentration ratio (BB). This is defined as the ratio of the drug concentration in brain tissue to the drug concentration in blood
  • Models often try to predict logBB using chemical descriptors
  • Uneven distribution of BBB+/- compounds and drugs could introduce biases into our prediction models

Contributions

  • 3 Unique, curated and interesting datasets
  • Discovered some important descriptors

Methods

  • 3 Datasets were used
    • Dataset 1 was compiled from various sources and included logBB values. It was split into the Training set (144 Compounds) and the External Validation Set 1 (15 Compounds)
    • Dataset 2 was used as the External Validation Set 2 and it included the BBB permeability of 99 drugs
    • Dataset 3 was used as the External Validation Set 3 and it included the BBB permeability of 267 organic compounds
    • Any duplicates and inorganic compounds found in the external datasets were removed
  • The training set was divided into multiple internal training and test sets using the sphere-exclusion algorithm (Similar to cross validation?). Then the models were trained on the training sets and validated on the test sets.
  • The statistical significance of the models created was evaluated using multiple parameters (See Validation of QSAR Models of the Paper. P.1906)
  • Y-Scrambling was used to test the robustness of the models
  • The models were also evaluated using the 3 external validation sets
  • 3 SVM and 3 kNN models were built. 6 combinations of 3 collections of descriptors (MolconnZ, MOE, Dragon). (See Table 3 of Paper)
  • Application of Applicability Domain (AD) increased the performance of the models

Outcomes

  • Top 10 descriptors with their frequencies from each type/category of descriptors (See Table 4 of Paper)
  • Polar surface area (PSA) , Octanol/Water partition coefficient (logP) and the number of hydrogen bond donors and acceptor atoms dominated were found to dominate the models
  • E-state indices and VSA descriptors were also found to be important

Conclusions

The study compiled the largest dataset of diverse organic compounds with their logBB values, for its time, and proved that a BBB permeability consensus model can have high predictive performance in detecting drug's or compounds' ability to enter the brain. It also discovered important chemical descriptors that seem to play an important role in BBB permeability.