THREDDS Server Setup | CloudSat DPC

CIRA DPC THREDDS/OPeNDAP End User Instructions

This website/webservice provides OPeNDAP access to the public CloudSat data, and these instructions are intended to show you how to use it. Please send suggestions for improvement to cloudsat@colostate.edu.

Accessing the data in the CloudSat files requires authentication. Please send an email to cloudsat@colostate.edu requesting an account. In the message, please include the email address you use to log into the Cloudsat DPC website. A THREDDS username and password will be supplied via email.

Data Access

There are two ways to use the DPC THREDDS service to download or use CloudSat data files:

Browsing the DPC TDS website
Using a file's OPeNDAP URL to remotely access data within a program

The following sections describe these methods.

Browse the CloudSat DPC TDS Website

Point your browser at the CloudSat DPC TDS website. Starting with the "CloudSat Data" Dataset, navigate the folder structure to a file of interest. Select that file. This will bring up a page that shows details about the file and clicking on the "HTTPServer" Service will download the file to your computer. A dialog box will prompt you for your username and password. After signing in, the file will be downloaded to your computer. Note that once your credentials have been entered, it will not ask for them again until you restart your browser.

Selecting the "OpenDap" selector will also bring up the credentials dialog unless you have already entered them. It will then bring up an "OPeNDAP Dataset Access Form" that allows selecting subsets of the data for viewing with the "Get ASCII" button at the top, or downloading with the "Get Binary" button. "Get Binary" will download to a .dods file. The DPC does not provide support for using this file format.

Use an OPeNDAP URL to remotely access data within a program

After browsing to the file of interest as described in the previous section and selecting the "OpenDAP Service, there will be a "Data URL:" box which shows the OPeNDAP URL for the file. It can be copied, and after inserting your credentials into it as described below, can be used to open the file remotely from within a program.

This URL, and URLs for other files, can be constructed from the beginning of this string, the "product.version" and the file name as will be described below.

This example uses the Python netCDF4 module to get data remotely from a CloudSat file. The netCDF4 library, used from with a Fortran program say, should work the same way. Including the embedding of your credentials into the URL as shown in the program below.

Python example

Make sure the netCDF4 module is included in your Python environment.

pip install netCDF4

conda install conda-forge::netcdf4

The conda-forge is recommended but not required.

A working program:

import netCDF4

# The URL for the file you want to access
url = 'https://<username>:<password>@dpc-thredds.cira.colostate.edu/thredds/dodsC/cloudsat-data/2B-CLDCLASS.P1_R05/2007/001/2007001005141_03607_CS_2B-CLDCLASS_GRANULE_P1_R05_E02_F00.hdf'
dataset = netCDF4.Dataset(url, 'r')

# Get the list of field name keys
keylist = list(dataset.variables.keys())
print(f'keylist = {keylist}')

# Get a particular field name
field = [key for key in keylist if key.endswith('Profile_time')][0]
print(f'field = {field}')

# Use it access data
Profile_time = dataset[field][:]
print(f'Profile_time[:37]: {Profile_time.data[:37]}')

Its output is:

keylist = ['StructMetadata.0', '2B-CLDCLASS%2fGeolocation_Fields%2fProfile_time', '2B-CLDCLASS%2fGeolocation_Fields%2fUTC_start', '2B-CLDCLASS%2fGeolocation_Fields%2fTAI_start', '2B-CLDCLASS%2fGeolocation_Fields%2fLatitude', '2B-CLDCLASS%2fGeolocation_Fields%2fLongitude', '2B-CLDCLASS%2fGeolocation_Fields%2fHeight', '2B-CLDCLASS%2fGeolocation_Fields%2fRange_to_intercept', '2B-CLDCLASS%2fGeolocation_Fields%2fDEM_elevation', '2B-CLDCLASS%2fGeolocation_Fields%2fVertical_binsize', '2B-CLDCLASS%2fGeolocation_Fields%2fPitch_offset', '2B-CLDCLASS%2fGeolocation_Fields%2fRoll_offset', '2B-CLDCLASS%2fData_Fields%2fData_quality', '2B-CLDCLASS%2fData_Fields%2fData_status', '2B-CLDCLASS%2fData_Fields%2fData_targetID', '2B-CLDCLASS%2fData_Fields%2fRayStatus_validity', '2B-CLDCLASS%2fData_Fields%2fNavigation_land_sea_flag', '2B-CLDCLASS%2fData_Fields%2fPrecip_flag', '2B-CLDCLASS%2fData_Fields%2fSST', '2B-CLDCLASS%2fData_Fields%2fSurface_wind', '2B-CLDCLASS%2fData_Fields%2fcloud_scenario', '2B-CLDCLASS%2fData_Fields%2fCloudLayer', '2B-CLDCLASS%2fData_Fields%2fCloudLayerBase', '2B-CLDCLASS%2fData_Fields%2fCloudLayerTop', '2B-CLDCLASS%2fData_Fields%2fCloudLayerType']
field = 2B-CLDCLASS%2fGeolocation_Fields%2fProfile_time
Profile_time[:37]: [0.         0.16000003 0.32       0.48000002 0.64000005 0.8
 0.96000004 1.12       1.28       1.44       1.6        1.76
 1.9200001  2.08       2.24       2.4        2.56       2.72
 2.88       3.04       3.2        3.3600001  3.52       3.68
 3.8400002  4.         4.16       4.32       4.48       4.64
 4.8        4.96       5.12       5.28       5.44       5.6
 5.76      ]

Note that the field name paths are separated with the HTTP/URL coding for '/'. This is how it works for HDFEOS files in OPeNDAP. Similarly, when getting data from other OPeNDAP webservice sites the field name formats tend to be different than when opening local files. Even for NetCDF files. The

field = [key for key in keylist if key.endswith('Profile_time')][0]

construct is one way to work around that.

Constructing the CloudSat URLs

Suppose you want to access the 2017033223812_57288_CS_2C-ICE_GRANULE_P1_R05_E06_F01.hdf file. Maybe you got it from the CloudSat DPC website or saw it on the SFTP site. Either way, you will need both the product and version for this file: 2C-ICE and P1_R05. These strings are included in the file name. Starting with the first part of the URL in the Python code above you can construct the URL for the new file. Start with this:

https://<username>:<password>@dpc-thredds.cira.colostate.edu/thredds/dodsC/cloudsat-data

Add the product.version directory:

https://<username>:<password>@dpc-thredds.cira.colostate.edu/thredds/dodsC/cloudsat-data/2C-ICE.P1_R05

The year and day-of-year are the first four digits and next three digits of the file name:

https://<username>:<password>@dpc-thredds.cira.colostate.edu/thredds/dodsC/cloudsat-data/2C-ICE.P1_R05/2017/033

Then add the full file name:

https://<username>:<password>@dpc-thredds.cira.colostate.edu/thredds/dodsC/cloudsat-data/2C-ICE.P1_R05/2017/033/2017033223812_57288_CS_2C-ICE_GRANULE_P1_R05_E06_F01.hdf

This URL works when substituted into the above program.

Note that the Python pydap module can also be used to remotely access the CloudSat data. Look at the website's instructions on how to use it.

Note that neither the netCDF4 nor the pydap module will read downloaded HDFEOS CloudSat files. The netCDF4 module may be able to open them, but the resulting dataset objects will not work correctly.