Version 5.1.1 of SCC

The new SCC version (released on 06/06/2019) brings many important updates and enhancements:

1) SCC product file format

The SCC v5.1 generates products which are:

• CF compliant (CF-1.7).
• optical products (ELDA outputs) are fully compatible with the format required by the new EARLINET database (which will be released quite soon).
• all SCC products are compatible with Panoply (https://www.giss.nasa.gov/tools/panoply/).
  This means that it is possible to plot the contents of any SCC product using Panoply without the need to write custom tools.

2) Quicklook web interface

A new color was adopted to distinguish between cloud-free regions (white regions) from no-data region (grey regions) in cloudmask quicklooks.

Please, keep in mind that the quicklook web interface (https://quicklooks.earlinet.org) will be password protected until the 31 of July 2019. After this date the password will be removed and it will become accessible to anybody.

IMPORTANT: We recommend to take advantage from this period for checking that everything is fine for your station on EARLINET quicklook web interface (https://quicklooks.earlinet.org).
In particular, you should check that:

• no testing or calibration lidar systems show up there by enabling for quicklook (on https://scc.imaa.cnr.it) only trusted lidar/configurations
• the name of the lidar systems that show up are somehow meaningful (avoiding for example names like "test system" or "my lidar" and so on)
• the quicklooks images are ok

3) Other enhancements

• All the stations are now identified by 3 digits code according to the list provided by Lucia. This is going to be the same code that identifies the stations of the new EARLINET database.
• It is now possible to use 15 digits to define the  measurement ID (the previous SCC version allowed to use only 12 digits). This has been done to allow the usage of the following format for the measurement ID (which we recommend!):

HHHHMMDDssshhmm

where 'sss' is the new 3 digits station code and HHHHMMDD hhmm is the start of the measurement.

• the stability of all the modules has been improved

In making the update we tried to not break any existing configuration.
So you should be able to log-in with your old username and password and find on the server all the configurations and measurements you have already uploaded before the upgrade.

As a consequence all the products on the servers are still the ones generated by you running the previous SCC version (v5.0). If you like you can re-calculate those products using the new version of the SCC (actually we recommend to do so). Please, keep in mind that in this case you are going to overwrite all your products! So, if you like to keep your old products you should make a back-up of them before trying to re-run the new SCC version.

Please, always remember that in order to generate the new products (HiRELPP, Cloumask, ELIC and ELQUICK), it is necessary to define for the configuration(s) an high resolution product, following some basic instructions on how to do that.
Please report problems or suggestions on the SCC forum available at: https://earlinetforum.imaa.cnr.it/viewforum.php?f=21

Version 5.0 of SCC

The new SCC version (released on 08/11/2018) brings many important updates and enhancements:

1) New SCC workflow

The SCC workflow has been modified by including 4 new calculus modules: HiRELPP, Cloudmask, ELIC and ELQUICK

• 1.a) HiRELPP module ingests raw input data and generate high resolution RCS timeseries (un-calibrated)
• 1.b) Cloudmask module ingests the output of HiRELPP module and generates cloudmask as output
• 1.c) ELPP module uses the generated cloudmask to skip (or label as cloud) in the time/vertical integration cloud contaminated signals
• 1.d) ELDA module takes into account cloud information in calculating optical products and provides the constants used to calibrate the optical products. Please note that calibration constants are calculated by ELDA only for certain product types (more information: https://earlinetforum.imaa.cnr.it/viewtopic.php?f=30&t=275)
• 1.e) ELIC module calibrates HiRELPP and ELPP products using the constants calculated by ELDA. It produces high and low calibrated high resolution products consisting in total attenuated backscatter timeseries
• 1.f) ELQUICK module generates quicklooks images from the high resolution ELIC products.The generated quicklook images will be available on a separated interface that will be available soon.

2) SCC product file format

All the SCC products are in NetCDF format and are CF compliant. To meet the CF compliance we had to modify the NetCDF structure of all the SCC products with respect to the old ones. This means that your reading software for the SCC products should be changed accordingly.In particular, the optical products generated by the SCC v5.0 are different from the EARLINET files we have used up to now. The new format is the same we have used for the publication of the data on CERA database. For the moment such files CANNOT be uploaded on the EARLINET database because it still requires the old format. Anyway we are going to release the new version of EARLINET database (presumably in January 2019) that will accept the new format only.

IMPORTANT: only the SCC product file format is changed. The raw input file format is still the same. This means no specific modifications are needed in your own converter to generate input file for the new SCC version.

3) Other enhancements

• 3.a) Web interface has been improved in terms of user-friendliness. Now it is possible to define different versions of a lidar system and to each lidar version it is possible to link several lidar configurations.
• 3.b) It is possible to calculate molecular atmosphere using model data (ECMWF, ICON_IGLO, GDAS). Model data are taken from CloudNET. To use this new option you should first check (using the SCC web inteface) that your station the field "Cloudnet Station ID" is set. If it is set the calculation of molecular atmosphere using model data can be activated by setting in the input file: Molecular_Calc=2.I would like to thanks Ewan O'Connor that offered us this very interesting option. The update does not break any existing configuration.All the configurations and measurements already uploaded before the updating should be found on the server. In order to no start the re-analysis of all the submitted measurements with the new SCC version automatically, the HiRELPP flag has been manually set to failed status for all the measurements. As a consequence, all the products on the servers are still the ones previously generated running the previous SCC version (v4.0). It is possible to re-calculate those products using the new version of the SCC (recommended). In this case, all the previous products will be overwritten! In order to keep the own old products a back-up of them is necessary before trying to re-run the new SCC version.

Please, note that in order to generate the new products (HiRELPP, Cloumask, ELIC and ELQUICK), it is necessary to define for the configuration(s) an high resolution product, following some basic instructions on how to do that.
Please report problems or suggestions on the SCC forum available at: https://earlinetforum.imaa.cnr.it/viewforum.php?f=21