Marine and Climate Data Discovery and Access Project
MACDDAP has integrated large marine and climate data sets, previously distributed across Australian research institutions, using web services technology. A "virtual database" provides for the collection, combination and analysis of data across scientific disciplines to facilitate data discovery for marine and climate related applications.
MACDDAP, comprised of seven components, utilises open scientific and geospatial data standards including OpeNDAP for data exchange and AAF for single sign on across federated databases. The translation services component translates data sets into standard vocabularies used in meteorology and oceanography. The translation services were developed by Arcitecta and are based on Mediaflux. Arcitecta is responsible for maintaining the translation services.
Figure 1 - MACDDAP Components
MACDDAP was funded by the National e-Research Architecture Taskforce (NeAT) under the National Collaborative Research Infrastructure Strategy (NCRIS).
Translation Services
Valuable datasets have been collected over decades in different formats and described using various metadata schemas. These are costly to conform manually to contemporary standards.
Translation services harvest remote THREDDS catalogues and data sets to generate a conformance statement against the NetCDF-CF conventions.
Figure 2 - MACDDAP translation services provide conformance guidance
A conformance statement includes the specification for a dynamically generated form indicating the changes that are required to achieve the required level of conformance. The form can be arbitrarily complex and its content will be dependent on the types of inputs and the types of conformance chosen. A form may contain controlled vocabularies.
The completed form can be saved with a profile. A profile is a named set of operations that can be applied to THREDDS catalogues. Any number of profiles may be created for different purposes. Profiles are saved on the server and may be recalled when next accessing the system.
Figure 3 - Conformance generator with controlled vocabulary
Controlled vocabularies, such as the CF Standard Names, can be loaded into the server to generate pull-down selections for users. When a profile for THREDDS data is executed, the conformance modifications will be used to generate modified NcML for every data set within one or more THREDDS catalogues. The output THREDDS driver will insert the NcML into each data set within the sets of catalogues processed. The enhanced catalogues are downloaded by the user to be re-inserted into the THREDDS server.
Translation services execute in the background, and send notification via e-mail when completed (translations of hundreds of thousands of files can take weeks to complete), allowing translated data sets to be downloaded at a later time.
Translation services are implemented within a general purpose framework that can be enhanced to incorporate additional and emerging metadata standards.
TPAC has already translated (and re-published) the following datasets: cars2009, Kerguelen GEM, AWAP Australia, and auscom. Next to be translated will be: cars2006, AWAP, argo, cmap, aspect, ace crc mooring and ctd.
Translation services can be accessed at: https://mediaflux.sf.utas.edu.au/translation
For further details please contact Jason Lohrey, jason.lohrey@arcitecta.com.
Other Portals
Other MACDDAP components include numerous enhancements to catalogues and portals, and an aggregation service, as illustrated in Figure 1.
Data is available from pre-existing sources such as the IMOS GeoNetwork MEST and the TPAC Digital Library portal and large, previously unmanageable datasets are easily accessible and usable. Examples include Argo, WOCE and MODIS satellite images.
TPAC Digital Library Portal: https://dl.tpac.org.au
The TPAC portal includes a geospatial and temporal search capability and a metadata access and notification service.
Resources and data held by the Antarctic Climate and Ecosystems CRC, the Australian Government Antarctic Division, the Bureau of Meteorology Research Centre, CSIRO Marine & Atmospheric Research, and the University of Tasmania are available.
For further details, please contact Peter Blain, peter.blain@utas.edu.au.
IMOS Ocean Portal: http://imos.aodn.org.au/webportal
The IMOS Ocean Portal allows users to discover and explore data streams from a number of sources including Argo floats, deep water moorings, ocean radar and others, some in near-real time, and all as delayed-mode quality-controlled data.
For further details, please contact Peter Blain, peter.blain@utas.edu.au.
IMOS-GeoNetwork MEST: http://imosmest.aodn.org.au/geonetwork/srv/en/main.home
The IMOS Metadata Entry and Search Tool is used by IMOS as the backend catalogue to the Ocean Portal and contains catalogue records harvested from all IMOS data hosts. It is typically used for "power" searching, where searching within the IMOS Ocean Portal isn't sufficient.
For further details, please contact Peter Blain, peter.blain@utas.edu.au.
Aggregation Portal: http://www.marine.csiro.au/remotesensing/imos/aggregator.html
The aggregation service improves usability of marine and terrestrial data. It supports aggregation of data subsets that match OPeNDAP constraint URLs into a single output file with a common set of axes. The aggregator includes polygon matching, visualisation capabilities, enhanced unit and metadata management and single file update.
For further details please contact Edward King, edward.king@csiro.au.
