Sign in to continue learning
Convolutional Neural Networks
35 hours 41 mins
In order to build deep neural networks, one modification to the basic convolutional operation that you need to really use is padding. Let's see how it works. What we saw in earlier videos is that if you take a 6x6 image and convolve it with a 3x3 filter, you end up with a 4x4 output, with a 4x4 matrix. And that's because the number of possible positions for your 3x3 filter, there are only 4x4 possible positions for the 3x3 filter to fit in your 6x6 matrix. And the math for this turns out to be that if you have an n by n image and convolve that with an f by f filter, then the dimension of the output will be n minus f plus 1 by n minus f plus 1. And in this example, 6 minus e plus 1 is equal to 4, which is why you wind up with a 4x4 output. So there are two downsides to this. One is that every time you apply a convolutional operator, your image shrinks. So we've gone from 6x6 down to 4x4, then you can only do this a few times before your image starts getting really small. Maybe it shrinks down to 1x1 or something. So maybe you don't want your image to shrink every time you detect edges or detect other features on it. So that's one downside. And the second downside is that if you look at a pixel at a corner or at an edge, this little pixel is touchless used only in one of the outputs. Because it touches that 3x3 region. Whereas if you take a pixel in the middle, say this pixel, then there are a lot of 3x3 regions that overlap that pixel. And so it's as if pixels on the corners or on the edges are used much less in the output. So you're throwing away a lot of the information near the edge of the image. So to solve both of these problems, both the shrinking output and when we build really deep neural networks, you see why you don't want the image to shrink on every step. Because if you have maybe a hundred layer deep net, then if it shrinks a bit on every layer, then after a hundred layers you end up with a very small image. So that was one problem. The other is throwing away a lot of the information from the edges of the image. So in order to fix both of these problems, what you can do is, before applying the convolutional operation, you can pad the image. So in this case, let's say you pad the image with an additional border of one pixel all around the edges. So if you do that, then instead of a 6x6 image, you've now padded this to an 8x8 image. And if you convolve an 8x8 image with a 3x3 image, you now get back out, not a 4x4, but a 6x6 image. So you've managed to preserve the original input size of 6x6. So by convention, when you pad, you pad it with zeros. And if P is the padding amount, so in this case, P is equal to 1. Because we're padding all around with an extra border of one pixel, then the output becomes n plus 2P minus F plus 1 by n plus 2P minus F by 1. So this becomes 6 plus 2 times 1 minus 3 plus 1 by the same thing and that. So 6 plus 2 minus 3 plus 1, that's equal to 6. So you end up with a 6x6 image that's preserving the size of the original input. So this green pixel actually influences all of these cells of the output. And so this effect of, you know, maybe not quite throwing away, but counting less the information from the edge or from the corner of the edge of the image is reduced. And I've shown here the effect of padding the border with just one pixel. If you want, you can also pad the border with two pixels, in which case I guess you add on another border here. And then you can pad it with even more pixels if you choose. So I guess what I'm drawing here, this would be a pad of equal 2 or P equals 2. In terms of how much to pad, it turns out there are two common choices that I call valid convolutions and sane convolutions. I'm not sure these are great names, but in a valid convolution, this basically means no padding. And so in this case, you might have an n by n image convolved with an f by f filter, and this would give you an n minus f plus 1 by n minus f plus 1 dimensional output. So this is like the example we had previously from the previous videos, where we had an n by n image convolved with a 3 by 3 filter, and that gave you a 4 by 4 output. The other most common choice of padding is called the sane convolution, and that means when you pad, so the output size is the same as the input size. So if you actually look at this formula, when you pad by P pixels, then it's as if n goes to n plus 2P, and then you have from the rest of this, minus f plus 1. So when you have an n by n image and a padding of a border of P pixels all around, then the output size ends up being this dimension, n plus 2P minus f plus 1. And so if you want n plus 2P minus f plus 1 to be equal to 1, so that the output size is the same as the input size, if you take this and solve for, I guess n cancels out on both sides, and if you solve for P, this implies that P is equal to f minus 1 over 2. So when f is odd, by choosing the padding size to be as follows, you can make sure that the output size is the same as the input size. And that's why, for example, when the filter was 3 by 3, as in the example in the previous slide, the padding that would make the output size the same as the input size was 3 minus 1 over 2, which is 1. And as another example, if your filter was 5 by 5, so if f is equal to 5, then if you plug it into that equation, you find that a padding of 2 is required to keep the output size the same as the input size, when the filter is 5 by 5. And by convention, in computer vision, f is usually odd. It's actually almost always odd. And you rarely see even numbered filter widths used in computer vision. And I think there are two reasons for that. One is that if f was even, then you'd need some asymmetric padding. It's only if f is odd that this type of same convolution gives a natural padding. We can pad the same dimension all around, rather than pad more on the left and pad less on the right or something asymmetric. And then second, when you have an odd dimension filter, such as 3 by 3 or 5 by 5, then it has a central position. And sometimes in computer vision, it's nice to have a pixel. It's called a central pixel, so you can talk about the position of the filter. Maybe none of these is a great reason for using f to be pretty much always odd. But if you look at the computer vision literature, you see 3 by 3 filters are very common. You see some 5 by 5, 7 by 7s. And actually, sometimes later, we'll also talk about 1 by 1 filters and when that makes sense. But just by convention, I recommend you just use odd-numbered filters as well. I think that you can probably get just fine performance even if you were to use an even-numbered value for f. But if you stick to the common computer vision convention, I usually just use odd-numbered f. So, you've now seen how to use padded convolutions. To specify the padding for a convolution operation, you can either specify the value for p, or you can just say that this is a valid convolution, which means p equals 0. Or you can say this is a same convolution, which means pad as much as you need to make sure that the output has the same dimension as the input. So, that's it for padding. In the next video, let's talk about how you can implement strided convolutions.