Sign in to continue learning
Natural Language Processing with Sequence Models
21h14m
There are many types of RNN architectures. Sometimes you're trying to build a sentiment classifier of a tweet. In that case, you might want to use one type of model. Another time, you might want to generate a caption for an image. And then, you might want to use a different architecture. So there are many different types of architectures, and in this video, we will show you all the different types. There are many different types of tasks in AI. Here, I'll group them according to the nature of their inputs and outputs. For instance, a one-to-one task takes as input a set of low or non-correlated features x and returns a single output y. Say you had a list of scores from your favorite team in the European soccer tournament, La Liga Santander. As inputs, you could have a recurrent neural network that predicted the position of your team on the leaderboard. However, notice that this recurrent neural network isn't much different from a conventional neural network. It only has an additional hidden state h superscript t subscript 0. So for this type of task, RNNs aren't all that useful. But if you want a neural network that takes an arbitrary image and generates a caption describing the content of that image in English, in this case, a brown puppy, you can build an RNN. This type of task is known as one-to-many because your RNN takes a single image and generates multiple words to describe it. Another type of task is many-to-one. You may already be familiar with one practical example of this type, sentiment analysis. In that instance, if you have a sequence of words, like this tweet that says, I'm very happy as input, and your model should output whether the tweet has a negative or positive sentiment, your RNN would take every word from the sequence as input in different steps, propagate information from the beginning to the end, and output the sentiment. Finally, many-to-many tasks involve multiple inputs and multiple outputs. For example, in machine translation, you have a sequence of words in French. You want to get its equivalent in another language, such as English. An RNN would do well for this task because they propagate information from the beginning to end, and this is what makes them able to capture the general meaning of word sequences. This particular architecture, encoder-decoder, is very popular for machine translation. The first part of this neural network doesn't return any outputs, y hat, and is called an encoder because it encodes sequences of words in a single representation. This captures the overall meaning of the sentence that is decoded later to a sequence of words in the other language. RNNs are powerful architectures. This can be used to solve a lot of interesting problems. In NLP, there are multiple applications where recurrent neural networks are useful, such as machine translation and caption generation. You can think of RNNs as versatile tools that can be shaped according to the situation. Depending on the task you're trying to solve, you can choose one of these architectures. In the next video, I'll talk about a simple recurrent neural network, and then you can apply it to all these architectures.