Some practical usacases of autoencoder in deep learning - abdussamad007/deep-learning-autoencoder GitHub Wiki
Feature learning or Representation learning
A machine learning model can't directly see, hear, or sense input examples. Instead, you must create a representation of the data to provide the model with a useful vantage point into the data's key qualities. That is, in order to train a model, you must choose the set of features that best represent the data.
Feature learning is motivated by the fact that machine learning tasks such as classification often require input that is mathematically and computationally convenient to process. However, real-world data such as images, video, and sensor data has not yielded to attempts to algorithmically define specific features. An alternative is to discover such features or representations through examination, without relying on explicit algorithms.
Example of features- A feature is an individual measurable property or characteristic of a phenomenon being observed.
-
In spam detection algorithms, features may include the presence or absence of certain email headers, the email structure, the language, the frequency of specific terms, the grammatical correctness of the text.
-
In character recognition, features may include histograms counting the number of black pixels along horizontal and vertical directions, number of internal holes, stroke detection and many others.
-
In speech recognition, features for recognizing phonemes can include noise ratios, length of sounds, relative power, filter matches and many others.
-
In computer vision, there are a large number of possible features, such as edges and objects.
Autoencoder:
An autoencoder is a type of artificial neural network used to learn efficient data codings in an unsupervised manner.The aim of an autoencoder is to learn a representation (encoding) for a set of data, typically for dimensionality reduction. Along with the reduction side, a reconstructing side is learnt, where the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input, hence its name.
Why we need autoencoder?
Auto encoders have an input layer, hidden layer, and an output layer. The input is forced to be as identical to the output, so its the hidden layer we are interested in.
The hidden layer form a kind of encoding of the input. "The aim of an auto-encoder is to learn a compressed, distributed representation (encoding) for a set of data." If input is a 100 dimensional vector, and you have 60 neurons in the hidden layer, then the auto encoder algorithm will replicate the input as a 100 dimensional vector in the output layer, in the process giving you a 60 dimensional vector that encodes your input.
So the purpose of auto encoders is dimensionality reduction, amongst many others.
Different types of autoencoder
- Denoising autoencoder
- Sparse autoencoder
- Variational autoencoder (VAE)
- Contractive autoencoder (CAE)
- Vanilla autoencoder
- Multilayer autoencoder
- Convolutional autoencoder
- Regularized autoencoder
Autoencode Usecases
1.Handwritten Digit Recognition Using Stacked Autoencoders
Optical character recognition(OCR) is the interpretation of handwritten or printed text into a format that can be processed mechanically or electronically. OCR has been researched and used since the early 1900s to convert printed text into machine processable data . The ability to process fixed font characters is well established but accuracy is hampered by the amount of noise in the image. Hand written or hand printed characters are a much more difficult problem than fixed font due to the noise and lack of consistency. In this paper we are exploring the use of a stacked autoencoders to function as an OCR. Details is given in the below source.
Source: http://csis.pace.edu/~ctappert/srd2017/2017PDF/c2.pdf https://www.researchgate.net/publication/313955743_Bangla_handwritten_digit_recognition_using_autoencoder_and_deep_convolutional_neural_network
2 Natural Language Processing
Word Embeddings, Document Clustering, Machine Translation, Paraphrase Detection and many more.
Source: https://www.doc.ic.ac.uk/~js4416/163/website/nlp/
3. Sentiment Analysis Using Semi-Supervised Recursive Autoencoder
Sentiment analysis is a growing field targeting to extract insights or subjective conclusions from the sources like text or large amount of data. Identifying the sentiment from online text gives businesses and organizations insights into what are main issues hitting the public opinion and what what they should focus on improving or changing. Getting significant attention from both business and research communities, sentiment analysis has many potential applications like summarizing user-reviews, brand-management and public relations management of business organizations and governments. Most of the past work has been focused on classifying the data in two classes: positive and negative. In my project, I aim to classify the data in five different classes: very positive, positive, neutral, negative and very negative.
Source: https://pdfs.semanticscholar.org/f8a9/8d14b726fa9f71e308ac72c9458d82dfad96.pdf https://pdfs.semanticscholar.org/6a14/92fe5022733078402612e3de2048ce85dce6.pdf
4. Bilingual Representations - Bilingual Autoencoders
Source: https://www.doc.ic.ac.uk/~js4416/163/website/nlp/bilingual.html
5. Stacked Autoencoders for Unsupervised Feature Learning and Multiple Organ Detection in a Pilot Study Using 4D Patient Data
Medical image analysis remains a challenging application area for artificial intelligence. When applying machine learning, obtaining ground-truth labels for supervised learning is more difficult than in many more common applications of machine learning. This is especially so for datasets with abnormalities, as tissue types and the shapes of the organs in these datasets differ widely. However, organ detection in such an abnormal dataset may have many promising potential real-world applications, such as automatic diagnosis, automated radiotherapy planning, and medical image retrieval, where new multimodal medical images provide more information about the imaged tissues for diagnosis. Here, we test the application of deep learning methods to organ identification in magnetic resonance medical images, with visual and temporal hierarchical features learned to categorize object classes from an unlabeled multimodal DCE-MRI dataset so that only a weakly supervised training is required for a classifier. A probabilistic patch-based method was employed for multiple organ detection, with the features learned from the deep learning model. This shows the potential of the deep learning model for application to medical images, despite the difficulty of obtaining libraries of correctly labeled training datasets and despite the intrinsic abnormalities present in patient datasets.
Reference: https://ieeexplore.ieee.org/document/6399478/authors#authors
6. Unsupervised Feature Learning for Audio Analysis
Identifying acoustic events from a continuously streaming audio source is of interest for many applications including environmental monitoring for basic research. In this scenario neither different event classes are known nor what distinguishes one class from another. Therefore, an unsupervised feature learning method for exploration of audio data is presented in this paper. It incorporates the two following novel contributions: First, an audio frame predictor based on a Convolutional LSTM autoencoder is demonstrated, which is used for unsupervised feature extraction. Second, a training method for autoencoders is presented, which leads to distinct features by amplifying event similarities. In comparison to standard approaches, the features extracted from the audio frame predictor trained with the novel approach show 13 % better results when used with a classifier and 36 % better results when used for clustering.
Reference: https://arxiv.org/abs/1712.03835
7. FEATURE LEARNING USING STACKED AUTOENCODERS TO PREDICT THE ACTIVITY OF ANTIMICROBIAL PEPTIDES
Reference: https://cmsb2015.sciencesconf.org/data/program/04_11h30_presentation_cmsb.pdf
8. Unsupervised Feature Learning for Heart Sounds Classification Using Autoencoder
Cardiovascular disease seriously threatens the health of many people. It is usually diagnosed during cardiac auscultation, which is a fast and efficient method of cardiovascular disease diagnosis. In recent years, deep learning approach using unsupervised learning has made significant breakthroughs in many fields. However, to our knowledge, deep learning has not yet been used for heart sound classification. In this paper, we first use the average Shannon energy to extract the envelope of the heart sounds, then find the highest point of S1 to extract the cardiac cycle. We convert the time-domain signals of the cardiac cycle into spectrograms and apply principal component analysis whitening to reduce the dimensionality of the spectrogram. Finally, we apply a two-layer autoencoder to extract the features of the spectrogram. The experimental results demonstrate that the features from the autoencoder are suitable for heart sound classification.
Reference: http://iopscience.iop.org/article/10.1088/1742-6596/1004/1/012002
Other Reference:
http://robotics.stanford.edu/~quocle/tutorial2.pdf
Andrew NG Notes: https://web.stanford.edu/class/cs294a/sparseAutoencoder_2011new.pdf
http://www.academia.edu/19712443/Auto-Encoder_Motivation_architechture_and_Applications
https://www.analyticsvidhya.com/blog/2018/06/unsupervised-deep-learning-computer-vision/
https://towardsdatascience.com/applied-deep-learning-part-3-autoencoders-1c083af4d798