Cloud mask
Important note
If you plan to use our cloudmask dataset, please contact us by sending an E-Mail to Veronika Pörtge.
The cloud mask was derived from radiance measurements of the SWIR camera of the specMACS instrument. The evaluation of the CIRRUS HL measurements are complicated by two factors. 1.) The measurements were partly taken above land surfaces. 2.) The measurements were partly taken inside clouds. This complicates the preparation of a cloudmask as our original cloudmask was generated to evaluate measurements above clouds and above the ocean. The original cloudmask is described in the Master’s Thesis of Felix Gödde. We tried to adapt the original cloudmask algorithm to better fit the needs of the CIRRUS-HL measurements but we are aware that there is still room for improvement.
As the original algorithm was developed for clouds above ocean, the main problem with these measurements was the occurence of the bright sunglint on the ocean surface. To overcome the effects of the sunglint, spectral water vapor absorption was used to identify clouds in front of a bright background. For scenes that were not affected by the sunglint, a mask based on the brightness of the pixels was used. For CIRRUS-HL we decided to change the work flow and combine these two masks at all times, regardless of whether sunglint could occur or not. If both masks indicate a pixel as cloudy it is defined as “most likely cloudy” in the final mask. If only one mask indicates a pixel as cloudy, the pixel is marked “probably cloudy” in the final mask and if neither mask indicates a cloud, this pixel is marked as “probably_clear_sky”.
The cloud mask for each research flight is stored in a separate NETCDF4 file. Each file contains the following variables:
- ”cloud_mask”: It is generally recommended to work with this cloud mask! This cloud mask is the result of merging the brightness cloud mask and the combined brightness water vapor mask. It is created in the following way: In the presence of sunglint the combined brightness water vapor cloud mask is used, while in scenes without sunglint the pure brightness mask is preferred. To decide whether sunglint is present in the scene or not the ocean reflectance was simulated with libRadtran. Three classes are discriminated: 0 indicates probably clear, 1 probably cloudy pixels and 2 most likely cloudy pixels. This mask is located at the bottom of the flowchart and is called “best estimate cloud mask”.
- ”swir_radiance”: radiance measured by the SWIR camera at a wavelength of around 1601 nm in units of [mW m-2 nm-1 sr-1] (data was smoothed by a gaussian 3x3 kernel to minimize inconsistencies in the measurements and obtain a decent result for the cloud mask)
- ”vza”: angle between the local vertical axis with respect to the WGS 84 ellipsoid and the line of sight to each pixel of the sensor. The angle is given in [degree]
- ”vaz”: horizontal angle between the line of sight to each pixel of the sensor and due north. The angle is given in [degree]
- ”sza”: angle between the local vertical axis with respect to the WGS 84 ellipsoid and the line of sight to the sun. The angle is given in [degree]
- ”saz”: horizontal angle between the line of sight to the sun and due north. The angle is given in [degree]
- CF_min: Minimal possible cloud fraction derived from the ratio between the number of pixels classified as “most likely cloudy” and the total number of pixels in one measurement. Measurements in which at least one pixel is classified as “unknown” are excluded from the calculation of the cloud fraction and set to “nan”.
- CF_max: Maximal possible cloud fraction derived from the ratio between the number of pixels classified as “most likely cloudy” plus the number of pixels classified as “probably cloudy” and the total number of pixels in one measurement. Measurements in which at least one pixel is classified as “unknown” are excluded from the calculation of the cloud fraction and set to “nan”.
Important notes and known problems:
- Measurements within clouds are often not identified correctly, especially in thin clouds where you can see the land surface through the cloud.
- Bright land surfaces are sometimes marked as cloudy pixels.
- For measurements with large solar zenith angle the cloud mask only identifies the brightest pixels as cloudy which results in an underestimation of the total cloud fraction.
- Clouds that are covered by a shadow are sometimes not identified as cloud.
- The cloud mask only discriminates between probably clear, probably cloudy and most likely cloudy pixels. It should be noted that there is no class which contains the most likely clear sky pixels!
- Even if the thresholds are chosen carefully there are certain situations where the cloud mask fails. In cases with sunglint accompanied by high aerosol concentrations it occurred that small areas were incorrectly identified as clouds. Furthermore, especially in scenes with low sun where only little light was reaching the sensor, artifacts of the sensor occured in the measurements. This results in the incorrect classification of single pixels as clouds. Therefore it was decided to apply a binary opening operation on the cloud mask with a 3x3 kernel. This operation was already carried out and should not be repeated. It removes all clouds which are smaller than 3x3 pixels. So if you are interested in analysing the cloud size distribution, you must be aware that very small clouds are not detected.
Changes
The algorithm developed to extract the cloud mask from the specMACS data in the presence of sunglint is described in the Master’s Thesis of Felix Gödde. The Pdf document of the thesis can be downloaded here. For the EUREC4A cloud mask product some minor modifications of the algorithm were done:
- In the original cloud detection algorithm three water vapor absorption bands were used (total spectral range: 1015 nm - 1910 nm). For the CIRRUS-HL data we decided to reduce the spectral range of the fitting algorithm to 1015 nm - 1300 nm. This only includes the first absorption band which has the advantage of not being affected by liquid water absorption. In the other two bands liquid water absorption occurs.
- The brightness threshold is no longer constant but a function of the solar zenith angle. This was done by obtaining the best matching brightness threshold for different scenes with varying solar zenith angles and fitting a function through these datapoints.
- The cloud mask is no longer a binary cloud mask but is made up of three different classes (probably clear, probably cloudy and most likely cloudy pixels). Technically this was done by combining the water vapor mask and the brightness mask.
- The meteorological data for the water vapor profiles now comes from ERA-5 data (previously: ERA-Interim).
Download
Help needed?
If you have any questions concerning the cloud mask please feel free to contact us! This could be, e.g., by sending an e-mail to Veronika Pörtge.
Versions
- 2023-02-13: V1.0 uploaded.
Literature
- The algorithm developed to extract the cloud mask from the specMACS data in the presence of sunglint is described in the Master’s Thesis of Felix Gödde. The Pdf document of the thesis can be downloaded here.
- A comparison of cloud masks derived from different instruments during NARVAL-II is shown in the Bachelor’s Thesis of Sabrina Pavicic. The Pdf document of the thesis can be downloaded here.
- The specMACS cloudmask which was produced for the NARVAL-II measurements was used in the following paper: Höppler, L., Gödde, F., Gutleben, M., Kölling, T., Mayer, B., and Zinner, T.: Synergy of Active- and Passive Remote Sensing: An Approach to Reconstruct Three-Dimensional Cloud Macro- and Microphysics, Atmos. Meas. Tech. Discuss., https://doi.org/10.5194/amt-2020-49, in review, 2020.