Data Science, NLP

Glove – word to vector representation

Congratulations on your new baby boy!

Glove is an unsupervised learning algorithm, intends to map words into a meaningful space based on distance.
The idea is ratio between the probability of words appearing next to each other. This objective is to take corpus, divide it into sub structure and predict the words by taking middle word in each iteration.

For example we have a statement “i like to play football in evening”. Here we have “play” as our middle word and our algorithm needs to predict the outside words. Before that we need to understand the substructure of words. Each alphabet can be arranged using distributed representation. where we have unique numeric value for each word.

Screenshot (19)

Markov model is inspiration for our problem. As the word prediction is stochastic process and we only can predict the next state (t+1) only influenced by the current.

1_IKT1tpW5bgH7y9B08BfLfw

We have two vectors (u, v). u is used for learning and v is taken for predicting the outside vectors.

Screenshot (17)

All the vectors are random and we take derivative w.r.t vector in order to increase the probability. As in each iteration we have vocabulary vector which will be processed in each iteration and can be slow down our algorithm. Stochastic gradient descent is used for optimization.

Screenshot (18)

End Notes.

We get the basic idea what Glove is and how it works. More can be learned on official website.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s