Automated image processing

The PhenoCam network has more than 200 cameras taking pictures of landscapes every day. These are the images we’re asking you to look at in Season Spotter, because there is information in them that a computer cannot easily pull out. However, there is some information that we can get out with computer algorithms.

Our cameras record images just like your digital camera or your smartphone: they divide up the scene into many pixels, and then the color for each pixel is stored as a set of three numbers. These colors represent red, green, and blue, which can produce any color when mixed together (as light).

Each of the three numbers ranges from 0 to 255. When all the numbers are 0, the color you get is black. When all the numbers are 255, the color you get is white. When all the numbers have the same value, but it’s between 0 and 255, you get a gray – a darker gray for lower numbers and a lighter gray for higher numbers. If the red and blue numbers are both 0, but the green number is 255, you get a bright green. And the same thing is true for blue if it’s the one with 255 and the other two are 0. Same for red. (There are online color mixers where you can play around with these numbers to see what color you get with different values.)

Now, we care about green. We care about green because it’s the color of chlorophyll – the color of leaves. If we can get a sense of the “greenness” of a northern forest landscape, then we can figure out when the leaves come onto the trees and when they change color and fall off again.

Spring, summer, and fall in Bartlett Forest, New Hampshire

Well, great! One of those numbers the camera is recording is for green. We have a green number for each pixel, so we can combine all those green numbers somehow – average them, for example – and then we know how green the image is.

The only problem is that this doesn’t work. Remember that if all three numbers are 255, the image is white? Well, if you have a very snowy scene, your green number may be high, despite there being no leaves in the picture. The opposite thing happens during the summer on overcast days. The picture might be rather dark and so the green number is small despite there being lots of green leaves on the trees.

The solution is to figure out how the green number is related to the blue and red numbers. If the green number is higher than the other two numbers, then there is quite a bit of green in the image – whether it’s a dark image or bright one. And if the green number is similar to the red and blue numbers, then there’s little green in the image.

To explain this relationship, we use a simple formula that we call the “Green Chromatic Coordinate” (or GCC for short). It is written:

In this formula, R is the red number, G is the green number, and B is the blue number. The formula itself simply means that we look at the fraction of total brightness (R+G+B) represented by the green number. If GCC = 0.33, then it means the green number is about equal to the red and blue numbers and there’s not much green in the image. If GCC = 1.00, then the whole image is a solid green, without any other colors. (Real images fall in between these two extremes.)

It turns out that this simple formula works. If we take a year’s worth of images, average the R, G, and B numbers over all the pixels in each image, and then calculate the GCC for each image, we can plot the “greenness” of that forest over the year.

GCC graph for 2014 at Bartlett Forest, New Hampshire.

Here’s how to read this graph. The dates for 2014 are along the bottom. The GCC is on the left side. Each dot represents one day. So dots higher up have larger GCC and mean more green in the image. When the dots go up, that’s when the trees are getting their leaves. When the dots go down, that’s when the leaves are turning color and falling off the trees. By doing this processing, we can learn what’s going on in the forest without ever having to look at the original images!