Sign in to continue learning
AI for Good
Beginner
Topics
Computer Vision
Data Processing
GenAI Applications
NLP
Supervised Learning
In the previous lab, you prepared the text data for modeling by converting the words in each message into tokens, and then counting up the tokens to model the entire corpus of text messages. In this lab, you'll apply a technique known as latent Dirichlet allocation, or LDA for short, and you can think of LDA as a method for running a statistical comparison across your corpus of documents, in this case, text messages. With this method, you'll be able to group messages that are statistically similar in terms of the relevant words, or more specifically, the tokens that they contain. So let's jump into the lab and see how this works. First off, at the top here, you can see there's a blog post explaining the LDA algorithm, if you're interested to learn about it in more detail. You'll start, as always, by running this first cell to import all the packages required for this lab. When you run this next cell, you'll load the dataset, including the processed tokens that you prepared in the previous lab. And next, you will again convert your corpus of tokens into a bag of words representation, and this just means assigning a number to each unique token in your dataset, so that the LDA algorithm can take advantage of it. With this next cell, you'll run the LDA method on your data, in this case, specifying that you want to model the data as containing two topics. With LDA, you typically hard-code the number of topics, but we'll also be looking at how we can work out what an optimal number of topics might be. The output here is showing the most common words that occur in the topics that the model identified. Note that you haven't specified anything about what you want the content of these topics to be about. You've just run a model to separate the data into two topics, based on the statistical similarities and differences between the messages in the dataset. Next, you'll look at something called the coherence score for different collections of words. There are some links here to learn more about the coherence score, if you're interested, but the takeaway is that this is a measure of how consistent or how good the topics are that you've identified, and there's a long literature about the pros and cons of using coherence for measuring the quality of topic modeling that I encourage you to read if you're interested. The way that coherence is measured at a high level is by computing an estimate of the similarity between the most frequently occurring words. So in general, the score is a negative number, and numbers closer to zero indicate a higher degree of coherence, or in other words, a better model. Also, you get just one measure of the coherence, no matter how many topics you have. So in this case, two topics, but you could run this for more as well, and you'll do that below. To visualize how this works, you'll compute the coherence for the top 10 words of each of the two topics you identified in the previous modeling step, and then compare that with the coherence score for a random list of words from the data set. Each time you run this, you'll compare against a different random set of words, and what you'll find is that the coherence of the top words in your model topics is much closer to zero than for random words. So this indicates your model topics are much more coherent than a random collection of words. And this is just a good sanity check to make sure that you've built your data pipelines correctly, and you are creating meaningful and not semi-random topics. As I mentioned, you can also run models for a greater number of topics, and that's what you'll do with this next step. So here, you'll generate models ranging from two to six topics. And like with previous labs, you can see that we've sped this up a little for you in the video. It'll probably take about a minute in total. Once that's finished, you can visualize the results of top words for each topic and the coherence score for each model. So here, I can see that for a model with two topics, the coherence score is about negative 2.43, and the top words for each of those two topics are listed here in the order of relative frequency. You can then use this pull-down menu to look at the coherence score and the top words for models with a different number of topics. When you run this next cell, you'll generate a plot of coherence scores versus number of topics for each of the models that you generated above. Again, better models will have a higher, less negative score. So here, you can see that it looks like two topics remains the best model in this case. To proceed with the analysis below, you can indicate a number of topics and select your final model. So here, I'll select two topics since that had the best coherence score, but you can also look at other numbers of topics. And then here again, this is showing the top words for the two topic models. To get a better sense of how individual messages break down between topics, run this next cell to compare the relevance across topics of a single message. So here, what you're looking at is a plot of how strongly a particular message is associated with each topic in your model. In this case, the message is about people in a certain area who lack resources, and it's much more closely related to topic one than to topic two. Try this for some different message types to get a sense of how they relate to the different topics. Next, you can plot how these topics change over time by running a cell here. This plot is showing the relative importance of each of the two topics over time. So on the x-axis here, you have days after the first messages were received, and on the vertical axis, you have the relative importance of the two topics. The way this is calculated is that messages are grouped together by week. You could do this per day, but grouping per week provides better smoothing in the statistics. Then you're evaluating all the messages with your model to determine how much they're related to topic one versus topic two, just like you did above, but now for the whole week of messages. For each week of messages, the average relevance or importance of each of the two topics is plotted here, and the shaded error margin shows two standard deviations around that mean. So here, it looks like you have a clear signal that immediately after the disaster, topic two was much more relevant than topic one. Looking back up at the top words for these topics, this starts to make some intuitive sense, as in topic two, the most common words were things like need, help, food, water, aid, and shelter, while in topic one, you have things like no, information, job, school, and work. Looking back down at the plot over time, it looks like topic two became less the focus over time, while topic one grows. Somewhere around maybe a month after the earthquake, topic one actually overtakes topic two in relevance, and by three months later, topic one is much more in people's mind than topic two. What do you think might be good names or descriptions for these topics? So I think you could maybe classify these into something like long-term needs versus urgent needs, or response-related topics in topics two versus long-term recovery topics in topic one. This is, of course, just a first pass at modeling this out, but having this sort of basic information about how needs are evolving in the weeks and months after a disaster can be very helpful to responders and organizations working in disaster response. And for me personally, while I'm not surprised that the information changed in this way, this is actually a more accurate indication of how the needs from the population has changed than the one that I've seen before, before working on the lab for this course. So hopefully this is also a good indication that you might be able to discover some of these topics in data that people have missed before in such a way that you can help disaster responders better understand the needs of the communities that they're trying to help. Now, as an optional extra piece here, you might have noticed some words in the top words above that didn't seem particularly relevant. So I'm going to show you a couple of examples here. Now, as an optional extra piece here, you might have noticed some words in the top words above that didn't seem particularly relevant. So if you're interested in exploring further, you can use the list here, where I've already suggested a couple of words, like and would, that you might consider adding to the stop words, that is, might consider removing. And you can add more words here in the list. Just put them in single quotes like this and separate them by a comma. Then run this code cell to remove those words from your dictionary. After that, you can go back up to step six here and run the code again to train a new set of models. When that runs, just keep running down through the notebook cell by cell and see what you find. You might discover that by excluding certain words, you can discover that a different number of topics seem optimal and potentially identify different kinds of trends over time. And if you want to start again from scratch, you can start all the way back up at step three to recalculate your bag of words from the original data set. That's it for the labs of this project. Here, you applied LDA to model topics within your data set. And with the topics that you identified, you were able to classify the messages from the data as related to things like urgent needs versus long-term recovery. And you were able to see how the volume of messages regarding each of those topics relative to each other evolved over time. Join me in the next video where we will wrap up the design phase of this project.
