ClimateHack.AI 2022
ClimateHack.AI 2022 was a satellite imagery generation challenge that ran between January & March 2022.
Loading...
ClimateHack.AI 2022 was a satellite imagery generation challenge that ran between January & March 2022.
Traditional solar PV forecasting methodologies have almost universally only used numerical weather predictions and solar electricity readings to forecast solar electricity output; however, numerical weather predictions are not always especially useful for predicting solar irradiance. Climate Hack.AI looks to add satellite imagery to the input data set, and we briefly review the state of the art in this new domain.
A good baseline model is "optical flow", which infers the movement of each pixel in the image from the most recent pair of images. It then uses those movement vectors to "warp" the most recent image to predict the future. It does surprisingly well! See this Python code for computing optical flow and a video of optical flow working on the Climate Hack.AI satellite dataset below.
There is also an interesting introduction to optical flow on the Computerphile YouTube channel.
Google Research's "MetNet" paper (Sønderby et al., 2020) describes a deep neural network designed to predict precipitation up to eight hours into the future. MetNet performs better than state-of-the-art numerical weather prediction models and produces probabilistic outputs.
The model does not use any numerical weather predictions in its inputs: instead, it takes as input a high-resolution image of the area of interest, as well as a larger, lower resolution, 1,024 km x 1,024 km context image that is large enough to capture any clouds that might cross the area of interest, from both precipitation radar and satellite imagery. Topographic maps are also included as extra channels in these images. This is then passed through a convolutional LSTM model, followed by a few axial attention layers to allow the model to learn what parts of the images are the most important.
On 15th November 2021, Google Research announced MetNet-2, which extends the prediction horizon to 12 hours by using input data with a significantly larger spatial extent (2,048 km × 2,048 km). This is a huge model that runs across 128 Google Cloud TPU v3 cores. To get the model to fit onto the hardware, the authors abandoned the axial-attention module that was present in MetNet-1.
DeepMind has also been working on precipitation nowcasting. In 2021, DeepMind released their "skilful precipitation nowcasting" paper (Ravuri et al., 2021), which uses a generative adversarial network (GAN) to create realistic-looking precipitation nowcasts. Like the MetNet papers, the "skilful nowcasting" model does not look at numerical weather predictions.
This model takes as input the last 20 minutes of imagery and outputs the next hour and a half of future imagery. It does this by creating four context stacks of images at different spatial resolutions, and passing them through a set of convolutional gated recurrent unit (GRU) layers that combine the context stacks at each level, starting from the smallest set of images, and working up to the largest images. These convolutional GRU layers are used to predict each timestep one at a time. As part of this process, a random vector is drawn from a uniform distribution and used as the initial hidden state of the bottom GRU layer, ensuring that each prediction is slightly different even with the same inputs, creating its probabilistic forecasts.
Unlike the MetNet papers, the DeepMind authors put a lot of thought into figuring out if expert human weather forecasters "believe" the GAN's forecasts. They conclude that the DeepMind GAN produces predictions which humans find very believable.
Open Climate Fix has implemented both MetNet and the model in the skilful nowcasting paper. See the "Architecture of OCF's ML models" section of their "Solar PV Nowcasting Using Deep Learning" report for more details.
In the second half of 2021, DeepMind released two related papers, which have been highly influential on Open Climate Fix's approach to short-term solar forecasting: The "Perceiver" paper (Jaegle et al., 2021), and the "Perceiver IO" paper (Jaegle et al., 2021b).
The Perceiver is based on self-attention, which has proven to be an extremely powerful model for many domains. Recent breakthroughs such as OpenAI's GPT-3 and DeepMind's AlphaFold-2 use self-attention extensively.
Conventional self-attention models suffer because their computational complexity goes up with the square of the length of the input (in this case the length of the input is linked to the number of pixels in the satellite and/or NWP data). This makes working on inputs such as large images intractable. The Perceiver introduces a beautifully simple way to limit the computational complexity of self-attention, thereby allowing these models to be applied to images and even videos.
Open Climate Fix are particularly excited about the Perceiver because it excels at "multi-modal" tasks, such as being able to combine multiple different types of input (satellite imagery, numerical weather predictions, etc), which may be on different spatial or time grids; and also to perform multiple tasks (such as predicting GSP-level PV, PV for single PV systems or future satellite imagery).
Furthermore, the input modalities do not need to be perfectly aligned in space and time. So, for example, hourly numerical weather predictions on a 2 km grid and 5-minutely satellite images on a 2-6 km grid can be input natively into the model without preprocessing or interpolating the data.
The following are two implementations of the Perceiver:
It is also possible to create a custom Perceiver IO model in just a few lines by using PyTorch's in-built Transformer building blocks.