User Tools

Site Tools


intercomparisons:c2_cubiccloud

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Next revision
Previous revision
intercomparisons:c2_cubiccloud [2016/11/15 16:22]
127.0.0.1 external edit
intercomparisons:c2_cubiccloud [2016/11/16 11:51]
claudia
Line 68: Line 68:
  
  
-==== Results ====+==== Results ​without atmosphere ​====
  
-Example result: ​+The following plots show the results obtained with MYSTIC. For all other models the patterns look the same. 
  
-{{:​intercomparisons:​phase_c:​c2_cubic_cloud:​mystic_case6_noatm.png|Example resultcubic cloud - case6, no atmosphere}} +{{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_mystic_noatm.png|C2 without atmosphereMYSTIC results}}
  
 ** General ** ** General **
  
   * Generally we find a very good agreement between all Monte Carlo models, only for a few cases they do not agree within 2 standard deviations.   * Generally we find a very good agreement between all Monte Carlo models, only for a few cases they do not agree within 2 standard deviations.
 +  * The polarization patterns are the same in all models, also the pattern for V when variance reduction methods are applied
   * For SHDOM a quantitative comparison is difficult because it does not provide a standard deviation and systematic differences are expected (different treatment of cloud boundary. Smaller differences due to different solution method of VRTE). ​   * For SHDOM a quantitative comparison is difficult because it does not provide a standard deviation and systematic differences are expected (different treatment of cloud boundary. Smaller differences due to different solution method of VRTE). ​
-  * MSCART (backward mode) shows sytematic ​differences to the forward calculations,​ Wang Zhen already mentioned this. MYSTIC gives the same results in forward and backward mode.+  * MSCART (backward mode) shows systematic ​differences to the forward calculations,​ Wang Zhen already mentioned this. MYSTIC gives the same results in forward and backward mode.
   * The agreed number of photons for this case is 49e9, this number has been used for 3DMCPOL, MSCART and MYSTIC. For SPARTA, 10e11 photons were used   * The agreed number of photons for this case is 49e9, this number has been used for 3DMCPOL, MSCART and MYSTIC. For SPARTA, 10e11 photons were used
 +  * Circular polarization (Stokes component V) is very noisy in Monte Carlo results, especially for models without variance reduction. ​
  
-The following plots show the mean radiance (for each Stokes component), mean absolute difference with respect to MYSTIC (forward Monte Carlo), mean standard deviation, fraction of pixels which agree within 2 standard deviations. Only pixels with non-zero radiances are included into the statistics.+The following plots show the mean radiance (for each Stokes component), mean absolute difference with respect to MYSTIC (forward Monte Carlo), mean standard deviation, fraction of pixels which agree within 2 standard deviations ​(2σ), please note the non-linear scale of the colorbar. Only pixels with non-zero radiances are included into the statistics.
    
 {{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_statistics_1.png|Summary plots, statistics C2 without atmosphere}} {{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_statistics_1.png|Summary plots, statistics C2 without atmosphere}}
 {{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_statistics_1.pdf|Summary plots, statistics C2 without atmosphere}} {{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_statistics_1.pdf|Summary plots, statistics C2 without atmosphere}}
  
-work in progress ​... +  * Stokes component I (upper row): The mean radiance (average over all non-zero pixels) is very similar for all models. The mean absolute difference is smaller than 1\% of the mean radiance. The mean differences are largest for SHDOM. Models without variance reduction (3DMCPOL, SPARTA) show larger differences than models with variance reduction (MSCART, MYSTIC-backward). Also the standard deviations are much smaller when variance reduction techniques are used (as expected). \\  The match fraction shows that all Monte Carlo models agree within 2σ for cases 5-9 (down-looking). The match fraction is between 0.9 and 0.95 for some up-looking cases 1-4, which might indicate a small bias between other models and MYSTIC. MYSTIC (forward and backward mode) agree perfectly.  
 +  * Stokes component Q (second row): The first column shows the mean absolute radiance (since Q can be positive or negative). The only visible difference ​in mean radiance is seen for case 5, here SHDOM is smaller than the Monte Carlo models, the difference is about 10%. \\ Mean standard deviations again are larger for models without variance reduction. The match fraction shows obvious systematic differences for mscart_bmc (all cases), mscart_fmc (cases 5 and 7) and SPARTA (case 6).    
 +  * Stokes component U (third row): Largest difference for case 7, mean difference between SHDOM and MYSTIC is about 5% of mean radiance. The match fraction shows obvious systematic differences for mscart_bmc (all cases), mscart_fmc (case 7). 
 +  * Stokes component V (bottom): Very noisy, without variance reduction (SPARTA, 3DMCPOL) noise is much larger than mean radiance. The match fraction shows systematic differences for mscart_bmc and mscart_fmc (all cases). These differences might be numerical because the mean radiance is very small (about 2e-6). Since the standard deviation is an order of magnitude larger than the radiance for SPARTA and 3DMCPOL, the results mostly match within 2σ, but this does not show whether the models really agree.  
 +  * Generally the standard deviation for SPARTA is below 3DMCPOL because for SPARTA more photons were used (10e11 vs49e9)MYSTIC and MSCART used 49e9 photons and variance reduction techniques which decrease the standard deviation approximately by almost one order of magnitude. 
 +  
 +==== Results with atmosphere ====
  
-  * Stokes component I (upper row): The mean radiance (average over all non-zero pixels) is very similar for all models. The mean absolute difference is smaller than 1\% of the mean radiance. The mean differences are largest for SHDOM. Models without variance reduction (3DMCPOL, SPARTA) show larger differences than models ​with variance reduction (MSCART, ​MYSTIC-backward)Also the standard deviations are much smaller when variance reduction techniques are used (as expected). \\  The match fraction shows that all Monte Carlo models agree within 2 <​m>​sigma<​\m>​ for cases 5-9 (down-looking). The match fraction is between 0.9 and 0.95 for some up-looking cases 1-4, which indicates a small bias between other models ​and MYSTIC. MYSTIC (forward and backward mode) agree perfectly.  +The following plots show the results obtained ​with MYSTIC. ​As for the cases without atmosphere the patterns look the same for all models.  
-  * Stokes component Q (second row)The first column shows the mean absolute radiance (since Q can be positive or negative)The only visible difference in mean radiance is seen for case 5, here SHDOM is smaller than the Monte Carlo models, the difference is about 10%. \\ Mean standard deviations again are larger for models ​without ​variance reduction. The match fraction shows obvious sytematic differences for mscart_bmc (all cases)mscart_fmc (cases 5 and 7) and SPARTA (case 6).    +{{:​intercomparisons:phase_c:​c2_cubic_cloud:​c2_mystic_atm.png|C2 ​without ​atmosphere, MYSTIC ​results}}
-  * Stokes component U (third row): Largest difference for case 7, mean difference between SHDOM and MYSTIC ​is about 5% of mean radiance. The match fraction shows obvious sytematic differences for mscart_bmc (all cases), mscart_fmc (case 7). +
-  * Stokes component V (bottom): Very noisy, without variance reduction radiance has same order of magnitude as standard deviation. ​ The match fraction shows sytematic differences for mscart_bmc and mscart_fmc (all cases).+
  
 +The following plots show the mean radiance (for each Stokes component), mean absolute difference with respect to MYSTIC (forward Monte Carlo), mean standard deviation, fraction of pixels which agree within 2 standard deviations (2σ), please note the non-linear scale of the colorbar.
 +{{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_statistics_2.png|Summary plots, statistics C2 with atmosphere}}
 +{{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_statistics_2.pdf|Summary plots, statistics C2 with atmosphere}}
 +  * Stokes component I (upper row): Mean radiance agrees very well for all models. Mean difference is largest for SHDOM (about 5e-4), for all other models below 1e-4. The mean standard deviations are in the range from 2e-5 to 5e-5, here the difference between models with variance reduction and without is not as large as for the cases without atmosphere. The reason is that for cases with atmosphere, there are no pixels with zero radiance and all pixels are now included in the statistics. In the clearsky region the standard deviation is not reduced by variance reduction. The match fraction shows a perfect agreement between all Monte Carlo codes over the full image. ​
 +  * Stokes component Q (second row): Mean absolute radiance agrees well for all models, again largest mean difference for SHDOM As for I the standard deviations are similar for all codes. The match fraction shows a perfect agreement between all Monte Carlo codes, except MSCART (backward mode). MYSTIC (backward mode) agrees perfectly to MYSTIC run in forward mode. 
 +  * Stokes component U (third row): Mean absolute radiance agrees well. Match fraction again shows perfect agreement for all Monte Carlo codes except MSCART (backward mode). ​
 +  * Stokes component V (fourth row): Mean radiances agree within noise. Largest differences (still below 5e-6) for MSCART. Standard deviation is significantly reduced by variance reduction, because V is only non-zero in cloud (only cloud scattering causes circular polarization,​ molecular scattering does not). Pixels with zero V are not included in statistics, thus standard deviation is more influenced by variance reduction. Very small systematic differences (may be numerical) between MYSTIC and MSCART (forward and backward).
 +  * Standard deviations are smaller in forward than in backward mode, when the same number of photons/​pixel (i.e. 1e7) is used. This can be seed for MYSTIC and also for MSCART.
  
-{{:​intercomparisons:​phase_c:​c2_cubic_cloud:​c2_statistics_1.pdf|Summary plots, statistics C2 with atmosphere}} +==== Detailed plots for all models, no atmosphere ====
- +
- +
-==== Results without ​atmosphere ====+
  
 ** Statistics plots **  ** Statistics plots ** 
Line 144: Line 155:
  
  
-==== Results ​with atmosphere ====+==== Detailed plots for all models - with atmosphere ====
  
 ** Statistics plots **  ** Statistics plots ** 
intercomparisons/c2_cubiccloud.txt · Last modified: 2018/01/08 12:22 (external edit)