FREQUENTly ASKED QUESTIONs (FAQ)

Last update: May 2020

 

Output data

O1. "I launched the data example script, but I can't retrieve the output file?"

In this case, you probably receive a "time out message".

  1. Network shortage in OUR side: to check this point, try the mirror SoDa server ("http://pro.soda-is.com/" or "http://pro.soda-is.com/").
  2. Network shortage on YOUR side: just try to go on the internet.
  3. Problem of proxy: your internet browser knows which "door" (proxy, firewall...) to use to get out of your local network, but you need to tell wget the information relative to this proxy.
    For wget, you need to specify the proxy information in your .bat file by adding:
    set http_proxy=machine:port
    before the wget command. 1-2-3... launch!
    The proxy information (machine and port) are available:
        Internet Explorer: Tools / Options / Connections / Network parameters
        Firefox: Tools / Options / Advanced / Network / Parameters
    Alternatively, you may ask for the proxy information to your IT person.

O2. "The data appear in Excel in a single column, and I would like that the data are separated and fits in each column. How can I do that?"

Solution 1 "manual": specify to EXCEL that your file is delimited with ";"

  • Select the whole column in which all the data are displayed (column A)
  • Go to the top menu "DATA"
  • Then click on "CONVERT"
  • Next step, select "DELIMITED" and click on ";"

Solution 2 "automatic":
In Windows: click here to download sed.exe + libraries. To replace automatically the " " by ";" in your test.txt file obtained using a wget command, you can use the following command:
sed "s/ /;/g" test.txt > test.csv
As a consequence, you can create a .bat file with the following command lines: wget -O test.txt "http://pro.soda-is.com/pub/hc3v5_request.php?geopoint=44.083,5.059&elevation=-999& ...
firstday=2017-01-01&lastday=2017-01-05&duration=15&time=TU&slope=15&azimuth=180&albedo=0.2"
sed "s/ /;/g" test.txt > test.csv
del test.txt

=> the file test.txt will be automatically deleted, and replaced by the test.csv file with ";" as separator.

In Unix, sed is a default tool, so you just have to copy and paste the previous example in a .sh file, by taking care of beginning the file by #!/bin/sh.

O3. "I face issues with decimal numbers when opening the CSV files in Excel (e.g. values are far too large)"

The decimal separator in all the CSV files provided by SoDa is the dot ".". As a consequence, these files can be opened without any problem with Excel configured in English.

If your version of Excel is configured in another language (e.g. French), open the Search and Replace dialog (Ctrl+H) and replace all dots by the decimal separator of your language (e.g. the comma).

O4. "I think I made 2 identical requests, and I retrieve 2 different results, why?"

The complexity of the input parameters might be a bit confusing for the users. Please check:

  • The HelioClim-3 version (4 or 5)
  • The elevation value: default (SRTM) or manually set by the user
  • If you take into account the shadowing effect of the far horizon ("true")
  • The plane orientations
  • The temporal coverage.

O5. "Why 2 HelioClim-3 requests inside the same pixel MSG could return different values?"

The HelioClim databases consist of Global Irradiance values over the horizontal plane. These values take into account a default value for the elevation for each Meteosat pixels, which is the values of TB5' (Terrain Base with 5 min of spatial resolution, i.e. 10 km) for both HelioClim-1 and -3.

Several years ago, new elevation databases (Digital Elevation Models-DEM) with higher spatial resolutions became available: GTOPO30'' (1 km) and finally SRTM (90 m) available worldwide. This was the opportunity for the HelioClim-3 database to exploit this new information "on-the-fly". A post-processing layer has thus been added to correct the elevation with the latest DEM available. That is why two HelioClim-3 requests made within the same Meteosat pixel return different results. A webservice of elevation (TB5, GTOPO and SRTM) is available at this link.

O6. "How to launch automatically my requests at night?"

In Windows, use the "scheduled tasks" to launch it automatically in a Windows environment. In Unix, launch automatically your script with a "cron".

O7. "How can I introduce a delay between 2 successive wget requests?"

Define the number of seconds to wait : "N"

  • In Windows: timeout [N]
  • In Unix: sleep [N]

O8. "What are the meaning of the automatic access error messages?"

Please find below the meaning of the error messages that you may receive using an automatic access to the following SoDa resources:

HelioClim-3 Archives + Meteo HC3 real time and forecasts + Meteo HelioClim-1 CAMS radiation CAMS McClear GFS MERRA-2

 

  • ERROR 1: No data available with these input parameters (generic message if the error is unknown)
  • ERROR -1: Incorrect tilted plane parameter
  • ERROR -2: Cannot open file
  • ERROR -3: Cannot write to file
  • ERROR -4: Cannot read file
  • ERROR -5: Not enough memory
  • ERROR -6: Cannot access remote data
  • ERROR -7: Helioclim service is corrupted
  • ERROR -8: Unregistered IP address
  • ERROR -9: Incorrect location parameter
  • ERROR -10: Incorrect time step parameter
  • ERROR -11: Incorrect date parameter
  • ERROR -12: Incorrect horizon parameter
  • ERROR -13: Incorrect elevation parameter
  • ERROR -14: Incorrect stations file
  • ERROR -15: Subprogram failed with no error message
  • ERROR -16: Incorrect calibration parameter
  • ERROR -17: Incorrect user/password parameters
  • ERROR -18: Incorrect albedo parameter
  • ERROR -19: Incorrect time reference parameter
  • ERROR -21: Incorrect rely parameter
  • ERROR -22: Incorrect header parameter
  • ERROR -23: HC3 database is not updated
  • ERROR -24: This service is out the scope of your SoDa subscription
  • ERROR -25: Incorrect gamma-sun-min parameter
  • ERROR -26: Mismatch dimensions of returned parameters
  • ERROR -27: Incorrect output format parameter
  • ERROR -28: Incorrect gfs grib parameter (internal error)
  • ERROR -29: Incorrect spectral band parameter
  • ERROR -30: Incorrect kato range
  • ERROR -31: Parameter does not exist or is not defined
  • ERROR -32: The date interval is too long for the specified time step parameter
  • ERROR -33: AI Forecast Invalid predictor name
  • ERROR -34: AI Forecast too many predictors with this name
  • ERROR -35: Invalid azimuth parameter
  • ERROR -36: AI Forecast web service is corrupted
  • ERROR -37: AI Forecast prediction parameters do not match the predictor parameters.
  • ERROR -38: Date shortcuts are not compatible with time reference TST

Maps

M1. "How to visualise unit16 Geotiff images?"

The maps are in unsigned integer 16 bits Georeferenced TIFF images. To visualize these images, you may either use a GIS software such as QGIS , or for instance with Matlab.
NB: To be immediately visible by just double-clicking on the file, the images should be in unsigned integer 8 bits.

M2. "Is there the phenomenon of spectral aliasing in the Meteosat images?"

The spatial resolution of the MSG satellite is 3 km at Nadir, and decreases as we get away from this point (lat, lon = 0°, 0°). All the objects smaller than the sampling step are thus invisible in the images. However, if no processing is performed beforehand of the sampling process, some spectral artefacts might appear, named "aliasing".

The processing applied on the MSG images enables to retrieve images with very little aliasing; The images are not acquired by a unique sensor at 3 km of spatial resolution, but by three sensors, which enables to artifically create a sensor with a higher spatial resolution. The high frequencies of this images are then smoothed out thanks to a low pass filter, which enables the provision of high quality MSG images after the final sampling.

M3. "When I compare the irradiation value at a given pixel location on the map, with the value retrieved through the website, the values are slightly different. What is the origin of this difference ?"

The maps are built with the raw HC3 pixel value from the database. The altitude is thus the one stored by default, i.e. Terrain Base 5 (approx. 10 km).

When a request is launched via the SoDa website, On the contrary, with a website request, the irradiation is corrected with the SRTM-v8 (90 m of spatial resolution) database altitude. As a consequence, hilly of mountainous areas, the differences between these values might be very inportant.

Data completion

D1. "I have ground measurements together with HelioClim data within a common period. What is the best way to fuse them?"

If the ground measurements are relevant, they are supposed to better represent the reality than HelioClim for a given site. The principle is to adjust HelioClim data onto ground measurements by computing a regression function (least-square, or first axis of inertia). We propose to our user a on-delivery service to provide time series of calibrated HelioClim-3 values. to receive a quotation and ask your question. A few publications also deal with this topic ; please refer to the publication page with the keyword "calibration".

D2. "I have data from 2 different HelioClim databases within a common period, typically HC-1 and HC-3, with a common period 2004-2005. What is the best way to fuse them?"

Various validation experiments between HC databases and ground measurements have shown that HC-3 data are closer to actual values than HC-1. Thus, HC-3 data have less uncertainty than HC-1. The principle is to adjust HC-1 data onto HC-3 data for the common period. One way to do that is to compute a regression function between HC-1 and HC-3 data. One may use least-square method or alternatively the first axis of inertia method. Then, the relationship may be applied to the whole series of HC-1 data, thus offering a consistent data set from 1985 till nowadays.

We propose to our user a on-delivery service to provide time series of calibrated HelioClim-3 values. to receive a quotation and ask your question. A few publications also deal with this topic ; please refer to the publication page with the keyword "calibration".