From N-gram to Hidden Markov: An Exploration of Basic Language Models

N-gram A basic language model is called an n-gram. An n-gram is a set of words or phrases taken from a sentence, which are then used as a baseline for predicting what words or phrases could come afterwards. A basic example would be a three-word n-gram, also known as a trigram. Trigrams are sets of three words that are often found together in a sentence. For example, “He went to” could be a trigram. It predicts that the next word in the sentence would likely be a verb. N-grams can be used to predict a variety of language patterns from sentences to entire paragraphs.

Hidden Markov The Hidden Markov model (HMM) is an extension of the n-gram model. It’s a probabilistic model that takes into account the probability of different words or phrases appearing in a sentence. HMMs also consider the probability of one word or phrase leading to the next. The model works by assigning probabilities to specific transitions between different words or phrases. For example, if a sentence starts with the phrase “He went to” there is likely a higher probability that the next word will be either “the” or “a” as opposed to any other word. By assigning these probabilities, HMMs can be used to predict the next word or phrase in a sentence.

Probabilistic The probabilistic language model is an extension of the HMM. It takes theHidden Markov model one step further by incorporating additional context and information. Probabilistic models take into account the probability of sequences of words or phrases appearing in a sentence. For example, it is likely that if a sentence starts with “He went to” that the next word will be either “the” or “a”. However, if the sentence starts with “He went to the store” it is more likely that the next word will be either “and” or “but”. The probabilistic model is able to take these probabilities into account and make more accurate predictions.

Overall, basic language models can be used to accurately predict the next word or phrase in a sentence. N-grams are sets of words that are often found together in a sentence and are used to predict what comes next. HMMs take into account the probability of words and phrases appearing in a sentence while the probabilistic model takes the HMM one step further by incorporating additional context and information. All three of these language models can be used to accurately predict what language will most likely come next in a sentence.