Results - HoldenCaulfieldRye/DeepLearnNLP GitHub Wiki

See what DeepLearnNLP has learned

Words as Vectors

In order to learn the relationships between words, the first hidden layer turns them into multi-dimensional vectors, for which each dimension represents a characteristic. As the network's performance rises, locally optimal characteristics are chosen, without any human interference. The similarity between words can then be measured as the Euclidean distance between their vector counterparts.
This is similar to - though of course much less elaborate than - Google's [word2vec] (https://code.google.com/p/word2vec/) open-source deep learning tool, described [here] (http://gigaom.com/2013/08/16/were-on-the-cusp-of-deep-learning-for-the-masses-you-can-thank-google-later/).

display_nearest_words() can be used to get an idea of the vector space created by DeepLearnNLP.

>> display_nearest_words('could', model1, 5)
can 7.20
would 7.59
will 7.78
should 7.88
might 8.01

Nearest words are also verbs, conjugated in the present tense, whose spelling is identical for all personal pronouns (I, you, he, she, we, they).

>> display_nearest_words('i', model1, 5)
we 6.34
they 6.58
you 6.66
he 6.87
she 6.92

Nearest words are also [personal pronouns] (http://en.wikipedia.org/wiki/Personal_pronoun).

>> display_nearest_words('for', model1, 5)
with 8.09
about 8.20
to 8.31
own 8.80
's 8.81

Nearest words are also [prepositions] (http://en.wikipedia.org/wiki/Preposition_and_postposition).

>> display_nearest_words('but', model1, 5)
and 7.37
, 7.38
; 7.53
-- 7.68
it 7.81

Nearest words are also [conjunctions] (http://en.wikipedia.org/wiki/Conjunction_(grammar)) or punctuation which act as conjunctions.

>> display_nearest_words('where', model1, 3)
what 8.68
when 8.70
who 8.76

Nearest words are also interrogative pro-adverbs (why is absent because it isn't part of the learned vocabulary).

>> display_nearest_words('never', model1, 2)
nt 7.86
not 8.15

Nearest words are also [adverbs] (http://en.wikipedia.org/wiki/Adverb) which connote negation.

Predicting the Next Word

The machine learning task is to predict the 4th word in a phrase, given the first 3 words. I personally find the means with which this task is achieved more interesting than the end, but it's still interesting to look at the resulting performance.

predict_next_word() can be used to evaluate the performance.

>> predict_next_word('john', 'might', 'be', model1, 3);
john might be . Prob: 0.17911
john might be right Prob: 0.16564
john might be like Prob: 0.11167


>> predict_next_word('take','many','of', model1, 3)
take many of us Prob: 0.42744
take many of these Prob: 0.18398
take many of them Prob: 0.08584


>> predict_next_word('her', 'school', 'was', model1, 4)
her school was nt Prob: 0.64191
her school was like Prob: 0.14731
her school was her Prob: 0.02892
her school was good Prob: 0.02468


>> predict_next_word('go','make','more', model1, 4)
go make more money Prob: 0.79201
go make more than Prob: 0.10646
go make more work Prob: 0.01444
go make more music Prob: 0.01274


>> predict_next_word('just','like','those', model1, 4)
just like those who Prob: 0.44391
just like those . Prob: 0.34331
just like those people Prob: 0.10128
just like those days Prob: 0.02841


>> predict_next_word('my','american','school', model1, 5)
my american school . Prob: 0.39638
my american school ? Prob: 0.18532
my american school , Prob: 0.11178
my american school officials Prob: 0.07184
my american school and Prob: 0.04623