Sign in to continue learning
In the previous video, you started a project where your task is to monitor biodiversity and animal population trends in Karo National Park in South Africa. You identified key stakeholders and defined a problem statement, and the next step is to determine whether AI can add value as part of your solution. Now, of course, we've already pointed out many examples of how AI is being used in biodiversity monitoring projects, but to determine whether AI can add value in your specific project, you need to begin by looking at the data, and so that's where we're going next. Let's head over to the lab and dig into the data. When you first open this lab, you will see this description of the project at the top. If you follow this link, you can see more information about the Karo project and about access to the data. If you follow this other link, you'll arrive at the zooniverse.org platform, where you can learn a little bit more about the citizen science project that helped label the images that you'll be working with. And now, before getting started with running any code, I'd like you to head up here and click on the Jupyter icon in the upper left here to open the folder where your lab is. And from there, I'd like you to open the data folder. As always, you'll find a data sheet in here that describes where the data came from. And then what you have here inside the data folder are more folders containing images of animals, and each folder is labeled with the name of the animal that has been identified in the images in that folder. So, for example, here you have a folder with images of baboons, and you can click on each of the images to see what these pictures look like. Or you can go to another folder like this one for gemsbok, which are also known as oryx, and you can have a look at the images to see what these pictures look like. And so I'd like you to spend some time in here just manually exploring the data to see what you find. See if you can get an intuitive sense of what some of the challenges might be in identifying the animals in these images. And you'll probably see things like some animals are occluded or barely visible at all, and some animals might be far away, or because the pictures are taken at night, it's not so clear what animal it really is. Once you've explored the data a little bit, head back over to the notebook. And get started by running this first cell up at the top to import all the Python packages that you'll need for this lab. Run this next cell to print out a list of the folder names inside your data folder. And here you will see the names of the animals again, one for each of the folders of images that you have. Run this next command here to see an example of printing out the list of image names inside a particular folder. In this case, for the black rhinoceros folder. And the spot check will also tell you that there are only 10 images of rhinos here. In this case, the file name of each image contains information about where and when the image was recorded. In this case, these KAR underscore S1 and R1 tags just indicate that this is from the Keroo dataset, season 1, repetition 1, and the A01, B03, etc. numbers indicate the location of the camera trap. So you can see that these pictures were taken from, let me count, 1, 2, 3, 4 different cameras. So not a whole lot of variety. So you can see here that there were four different camera traps which caught images across these 10 in total. And finally, this image tag just gives the number of the image, a unique identifier across the entire dataset. When you run this next cell, you'll be running a command that searches through the file structure and extracts a data frame of information for all the images. Where now, using the file path of each image, you've extracted the camera location, which is part of the file name for each image, the name of the animal pictured in that image, and the file path to that image. You'll use this information throughout the lab to access images automatically. With this next cell, you can take a high-level look at the data frame you generated with the last cell to see that it included all the animal names in different camera locations. Run this next cell here to count up the number of images for each camera location. What you can see is that some locations have hundreds of images, while others only have a few. And with this next cell, you can count up the number of images for each animal. Here again, you'll find that for some animals, there are hundreds of images, while others have relatively few. And this is something that we'll come back to throughout this lab. Where we have few images, we would expect the machine learning algorithm to be less accurate. If you're not familiar with many of these animal names, don't worry. You'll become familiar with them throughout these labs. Just note that the names as listed here are sometimes out of order. Like, for example, this one is Red Hartebeest, and this one is Mountain Zebra. And this one here is Black-backed Jackal. And this is Curry Bustard, which is a kind of bird. Up here, this one is a Gemsbuck, which is also known as an Oryx. Another way to visualize the distribution of animals in your data set is by percentage of the total, and that's what you'll do when you run this next cell. Now, this chart shows the relative percentages of the total number of images that you have of each animal. The main thing to take away at this point in your data exploration is that this is an imbalanced data set. Like we said, your data is not distributed evenly across all the classes of objects, types of animals in this case, that you hope to identify. So if you try to develop models with an imbalanced data set, you will run into problems. Like in this case, for example, you can see that even a model that just always guessed Gemsbuck Oryx would be right more than 25% of the time. In the design phase of this project, you'll see some approaches to balance it out your data before modeling. Taking this investigation one step further, you can run this next cell to look at the relative distributions of animals imaged at each individual camera location. Now here on the horizontal axis, you have the list of camera location identifiers. And on the vertical axis, you have the fraction of each type of animal shown here in the plot legend that appears in images taken at each location. What you can see is that not only is the relative distribution of animals in your data set imbalanced, but not surprisingly, the distribution changes quite dramatically as well based on the camera location. Those of you who are conservation biologists, you might already be able to draw important information from these relative distributions about where the animals are in the park, which animals are occupied in the same spaces, and so on. We're going to move on to looking at the actual images and thinking about an AI solution for an automatic animal detector. So please join me in the next video to start looking at the images in this data set.