Home - GregLinthicum/From-Logistic-Regression-to-Long-short-term-memory-RNN GitHub Wiki

Introduction

The technology emerges from the state where the machine did not even get the words you were uttering and responded in a choppy robotic way to the open space of the creativity unimaginable ever before.

The time has arrived to venture into improving the attention span of Amazon's Alexa, Google's Assistant, and Apple's Siri. Alexa services exploded into over seventy thousand of skills. Companies and governments use speech recognition and speech synthesis in telephony, in the browser or in in the field applications. It all got traction in 2017 when Deep Learning became irreversibly commercial.

Now we're talking.

The text you are reading on this Wiki page right now is just a graphical representation of the soundtrack. Let's cut out this impediment. Typing skills just like a calligraphy a century ago would become obliviated. Not so quick. Writing provides a spectrum of services perhaps underestimated before. First, the written text allows us to synchronize on pronunciation - or communicating despite different pronunciations ( ask Chinese). Second, the written text provides an access to the treasury of historical records. At this time the written text is still a better form of data representation for conducting searches and for more advanced forms of information processing than soundtracks allow to achieve.

Taxonomy

Supervised Learning

  • Artificial Neural networks ( ANN )
  • Multilayer Perceptrons (MLPs)
  • Convolutional Neural Networks (CNNs)
  • Recurrent Neural Networks (RNNs)
  • Long short-term memory (LSTM)

Unsupervised Learning

  • Self-Organizing Maps (SOM)
  • Restricted Boltzman Machines (RBM)
  • Deep Boltzmann Machines (DBM)
  • Deep Believe Networks (DBN)
  • Auto-Encoders (AE)
    • Sparse Auto-Encoder (SAE)
    • Stacked Auto-Encoder (SAE)
    • Stacked Sparse Auto-Encoder (SSAE)
    • Variational Autoencoder (VAE)
  • Generative Pre-trained Transformer (GPT)

Libraries that dominate the market

., * CUDA CUDA CUDA CUDA CUDA CUDA

Cloud and Meta Libraries

Application

Supervised Learning ( Discriminative )

Typical discriminative models include logistic regression (LR), support vector machines (SVM), conditional random fields (CRFs) (specified over an undirected graph), decision trees, neural networks.

Artificial Neural networks ( ANN ) (predicting client decision)

Used mainly for Regression and Classification "Many studies have shown that ANNs can better model heteroskedasticity i.e. data with high volatility and non-constant variance, given its ability to learn hidden relationships in the data without imposing any fixed relationships in the data. This is something very useful in financial time series forecasting (e.g. stock prices) where data volatility is very high." [TDS]. Alternatively, finds application where simpler technologies would suffice (AWS Predictive Scaling), but once AI puts its foot in it inspires and expands towards tackling harder tasks such as DevOps with Kubernetes, or MLOps + + + all the way to AIOps.

Multilayer Perceptrons (MLPs)

Use Hiddenlayer to render a graph of your neural network in Jupyter Notebook, or to a pdf or png file. See Jupyter notebook examples for TensorFlow, PyTorch, and Keras (Phil Ferriere).

Convolutional Neural Networks (CNNs) (image classification)

Used Mainly for Machine Vision JiaxiangZheng, Py

Recurrent Neural Networks (RNNs) (stock prediction)

Used mainly for Time Series Analysis Bruno Gonçalves (predicting opening stock price)

Long short-term memory (LSTM)

Natural Language Processing, Alex Müller

Unsupervised Learning ( Generative )

Typical generative model approaches include naïve Bayes classifiers, Gaussian mixture models, variational autoencoders.

Self-Organizing Maps (SOM) (fraud detection)

Hadelin de Ponteves: "According to a recent report published by Markets & Markets the Fraud Detection and Prevention Market is going to be worth $33.19 Billion USD by 2021(...) Unsupervised Deep Learning Models (...) will literally come up with an explicit list of customers who potentially cheated on their applications."

Restricted Boltzmann Machines (RBM) (stochastic Hopfield network)(PyTorch)

Used mainly for dimensionality reduction, classification, regression, collaborative filtering, feature learning, and topic modelling. An Introductionto Restricted Boltzmann Machines Christian Igel

Deep Boltzmann Machines (DBM) Used Mainly for Recommendation Systems.

Deep Believe Networks (DBN)

Difference between Deep Belief and Deep Boltzman is that Deep Boltzman is bidirectional on every level.

Auto-Encoders (AE)(PyTorch Udemy)

Used Mainly for Recommendation Systems. sparse-stacked-and-variational-autoencoder

Transformers

NLP The Transformer is a deep learning model introduced in 2017, used primarily in the field of natural language processing (NLP). Like recurrent neural networks (RNNs), Transformers are designed to handle sequential data, such as natural language, for tasks such as translation and text summarization. However, unlike RNNs, Transformers do not require that the sequential data be processed in the order.