Sign in to continue learning
Natural Language Processing with Probabilistic Models
30h35m
Welcome to the second week of this course. In this video, you're going to learn about parts-of-speech tagging. Specifically, you'll see the different applications that it is used for. You will also learn how to compute the accuracy of a parts-of-speech tagger. First, I'll go over what parts-of-speech tagging is. Then, I'll show you how to use so-called Markov chains and hidden Markov models to create parts-of-speech tags for your text corpus. Next, I will introduce the Viterbi algorithm and demonstrate how it's used in hidden Markov models. You'll get to try this on your own with an example. You're going to apply all of these skills in this week's coding assignments. It's a big week, so let's get started. Parts-of-speech refers to the category of words or the lexical terms in a language. Examples of these lexical terms in the English language would be noun, verb, adjective, adverb, pronoun, preposition, although there are many others. Let's take a look at this sentence. Why not learn something? Writing out the names of these terms can quickly become cumbersome during text analysis, so you're going to use a short representation called tags to represent these categories. The process of assigning these tags to the words of a sentence or your corpus is referred to as parts-of-speech tagging, or POS tagging for short. Because POS tags describe the characteristic structure of lexical terms in a sentence or text, you can use them to make assumptions about semantics. They're used for identifying named entities too. In a sentence such as the Eiffel Tower is located in Paris, Eiffel Tower and Paris are both named entities. Tags are also used for coreference resolution. If you have the two sentences, the Eiffel Tower is located in Paris, it is 324 meters high. You can use parts-of-speech tagging to infer that it refers in this context to the Eiffel Tower. Cool, right? Another application is speech recognition, where you use parts-of-speech tags to check if a sequence of words has a high probability or not. You now know the different applications of parts-of-speech tagging. In the next video, you're going to learn about Markov chains, and you'll see how you can use them to decode the parts-of-speech of a certain sentence.