Sign in to continue learning
AI and Climate Change
15h40m
Hello, everyone. My name is Sarah Beery, and today I'm going to be talking about using computer vision to monitor biodiversity at a global scale. So why do we want to monitor biodiversity? Unfortunately, we're in the middle of what's being called the sixth mass extinction. We're losing plants and animal species at a faster rate now than we were during the last extinction of the dinosaurs. And on average, we've lost around 68% of our animal population since just 1970. To try to understand this loss so that we can mitigate it, biologists and ecologists collect data from a vast, diverse set of sensors. This is everything from satellite data to GPS collars placed on animals, to networks of static cameras or acoustic sensors that are placed in the wild and monitor over time, to actually relying on interested community members to go out with their cell phone and take pictures or audio recordings of species. All of these are being pulled together to try to build a global picture of where and when species exist and how we can help them survive in the face of climate change and human encroachment on their habitats. Unfortunately, because of the sheer volume of data that's currently being collected, manual data processing, going through all of that data by hand and labeling species so that then you can take that and turn it into scientific insight, that doesn't scale. For example, I placed a network of 100 camera traps in Kenya last year, and that network has collected 10 million images just in the last year alone. There's no way that I'm going to be able to label all of those images myself. Community scientists have collected over 64 million species observations that all need to be identified to species in iNaturalist, which is a community science platform. And an aerial survey, a single aerial survey, that would be used, for example, to monitor the population size of elephants in Kenya, for example, that can generate up to 200 terabytes of video. So all of this really points to the fact that we need automation. We need to help the ecologists that we work with be as efficient as possible so they can focus on the hard examples, and that we can hopefully scale up our ability to monitor these species to match the amount of data that we're collecting. One example of this is a detection model that I built while I was working at Microsoft AI for Earth. We call the model the Megadetector. And it's quite simple. It just detects humans, animals, and vehicles. But it does it in a really robust way that works off the shelf for most ecologists. And this model is being used by 40 to 50 organizations worldwide. One example is the Idaho Department of Fish and Game uses it to help monitor and manage their wolf population. It meant that they had to label less than 15% of their images. Another example is Wildlife Protection Solutions uses this model in 18 countries worldwide. They're using it to provide real-time alerts to land managers when there's potential poaching threats. Now that's amazing, but unfortunately, a lot of machine learning models struggle with some of this data. And the reason that they struggle is that biodiversity data, unlike the data we commonly work with in computer vision systems that's very well curated and independently and identically distributed, biodiversity data is both noisy and biased. So here on the left, you can see a map of global biodiversity. And you can see that there's much, much more diversity of species in the sub-equatorial tropics. But actually, the data that we have available that we could use for training a machine learning model, for example, is significantly biased towards America and Europe. So that means that we're actually trying to ask machine learning models to predict in areas that they're not well trained on. And as a result, we see that some tasks just don't generalize well. So for example, here what we're looking at is if you train a machine learning model or a computer vision model to predict animal species on one set of fixed camera traps, and then you ask it to predict on camera traps that weren't seen in its training set, you see a significant gap in performance. You see that error is significantly higher when you're asking it to generalize to out of domain. And so what we want to try to figure out how to do is close that gap. And so one of the challenges is that because the data can be quite noisy, your model struggles to understand in a new frame of reference what's going on. For example, a challenging image like this one, human experts are able to label this. But I bet if you just look at this image, and let's focus on one potential object here. For this object, I don't know if it's foreground or background, let alone what species of animal it is. But experts don't look at a single image in a vacuum. They look at an image in context of other images taken at the same camera location. And they can sometimes look at images over a very large time horizon. So here you can see images over an entire month. And what that allows them to do is allows them to understand for a fixed camera, what is foreground, what is background, and what are the species that are common are likely to show up and what shape and size are they and, and what that all of that context is providing is the ability for a human to label something that's very challenging with high confidence. And so this hard identification of a wildebeest in this very foggy image was possible because of that very long term context. And what we're seeking to do here is provide this same contextual approach to improve species identification really at a global scale.