Bias and error validation of Solcast data against surface measurements

During early and mid-2019, extensive analysis was performed comparing Solcast data with publicly available surface measurements from around the world, across all climate types except polar climates.

The internal Solcast analysis presented here can be referenced against a growing list of validation papers listed on the publications page. If you would like a copy of the validation timeseries data, a sample for one site is provided here on our website, and you are welcome to make a request for us to send you the data for all sites (be sure to tell us about your intended use case). This validation and accuracy information is correct to the best of our knowledge, but should not be interpreted as any form of guarantee or warranty. If you’d like help with further analysis of uncertainty at your specific site(s), please reach out to us to speak about a consulting study.

Site Selection

For validation of satellite-derived irradiance estimates, granular time series data from uniform, high quality, well maintained, long-period surface measurement sites are required. The Baseline Surface Radiation Network (BSRN) was selected to ensure the consistency of sensor specifications and the measurement calibration and quality control approach. A number of BSRN stations are equipped to measure solar and atmospheric radiation, using sensors of the highest available accuracy and a high sample rate. The BSRN network was set up specifically to provide high-quality observational data to the scientific community, making it ideal for validating satellite-based estimates of solar radiation. The use of a single, research-grade dataset for validation ensures the below statistics are of the highest possible quality and an accurate representation of the Solcast database.

Within the total set of BSRN stations, sites outside Solcast data coverage areas were excluded (i.e. removing oceanic and polar sites), and comparison was limited to sites with a sufficient record in time during the Solcast data availability period of approximately 12 years from 2007 (Solcast finds that prior to the mid-2000’s, satellite data was of substantially lower quality in general to warrant inclusion, even with quality control and other steps applied). Two mountain-top sites were excluded, Sonnblick, Austria (SON), and Izaña, Spain (IZA), which have extreme local topography that is not representative of solar development sites. Solcast data is not recommended for use in steep mountain terrain with local relief of more than 1000m.

Indicators

The most common statistical indicators in the solar industry used to evaluate solar radiation models fall into two categories; bias measures and error measures.

Bias measures

Bias measures are important for long term energy yield calculations, providing a base to estimate the accuracy of energy simulations by understanding a possible error in the long-term estimate. Mean bias error (MBE) provides information about the aggregate difference between the Solcast dataset compared against ground observations. It is a measure of the dataset’s systematic tendency to over- or underestimate the solar resource. Deviation of bias error indicates the spread of this error. It indicates the likelihood of the error at a specific site occurring within a confidence interval.

Error measures

Error measures are important for short term variability work. They help to develop an understanding of the accuracy of operational calculations based on energy simulations, covering recent performance and evaluation, equipment calibration, plant monitoring and forecasting. Mean absolute error (MAE) measures the average magnitude of the difference between the ground station and the models. Root mean squared error (RMSE) is another measure of these differences, but uses quadratic weighting. This means larger errors have a disproportionately larger effect on RMSE, and consequently this measure is very sensitive to large errors.

Results Overview
The following tables present the global total summary statistics of the Solcast dataset compared to the surface measurements, at the hourly average time granularity.

Global bias statistics - Hourly GHI :

Mean bias error (MBE) : -0.02 W/m2
Relative mean bias error (rMBE) : 0.00%
Standard deviation of bias error : ±2.02%
80% of sites have bias smaller than : ±1.93%
90% of sites have bias smaller than : ±2.50%

Global mean error statistics - Hourly GHI :

Mean absolute error (MAE) : 47.44 W/m2
Relative mean absolute error (rMAE) : 10.89%
Root mean square error (RMSE) : 72.38W/m2
Relative root mean square error (rRMSE) : 16.5%
Correlation coefficient : 0.91

Solcast - Historical and TMY Software

Proven, Accurate, Bankable Data:

Data and Specifications:

Inputs and Algorithms

Methodology

Validation and Accuracy