6 Portals
6.1 Overview
CalCOFI data is available through various portals, each serving different purposes and user needs. This document outlines the main access points and their characteristics.
6.2 Data Flow
While it would be ideal for CalCOFI data to be available through a single portal, each portal has its strengths and limitations. The following diagram illustrates one possible realization of data flow between CalCOFI data and the portals: from raw data to the integrated database to portals and meta-portals.
In practice, CalCOFI is a partnership with various contributing members, so the authoritative dataset might flow differently, such as from EDI to the database to the other portals. The other portals, such as OBIS or ERDDAP, serve different audiences or purposes. The meta-portals like ODIS and Data.gov then index these portals to provide broader discovery of CalCOFI datasets.
6.3 Portals
While some portals serve as data repositories, others provide advanced data access and visualization tools. The following sections describe the main portals where CalCOFI data is available and their key features.
Full Archive | Versioning | DOI Issued | Query by xyt | Query by taxa | Multiple formats | API Access | |
---|---|---|---|---|---|---|---|
EDI | ✔ | ✔ | ✔ | ▲ | ▲ | ✖ | ▲ |
NCEI | ✔ | ✔ | ✔ | ✖ | ✖ | ✖ | ▲ |
OBIS | ▲ | ▲ | ▲ | ✔ | ✔ | ▲ | ✔ |
ERDDAP | ▲ | ✖ | ✖ | ✔ | ▲ | ✔ | ✔ |
Capability Legend: ✔ = full, ▲ = partial, ✖ = none |
6.3.1 EDI
Environmental Data Initiative
- Complete dataset archives using DataOne software and EML metadata
- DOIs issued for all datasets ensuring citability
- Full archive allowing for any data file types
- Basic spatial and temporal filtering through web interface
- Download in original formats with metadata
- Access through DataOne API
- Links:
- EDIrepository.org
- CalCOFI datasets: EDI query “CalCOFI”
6.3.2 NCEI
National Centers for Environmental Information
- Long-term archival of oceanographic data
- DOIs issued for dataset submissions
- Standardized metadata using ISO 19115-2
- Basic search interface with geographic and temporal filtering
- Data preserved in original submission formats
- Access through NCEI API services
- Links:
- NCEI Ocean Archive
- CalCOFI datasets: NCEI search “CalCOFI”
6.3.3 OBIS
Ocean Biodiversity Information System
- Specialized in marine biodiversity data
- Standardized using DarwinCore fields
- Extended measurements supported via extendedMeasurementOrFact
- Powerful filtering by space, time, and taxonomic parameters
- Multiple download formats (CSV, JSON, Darwin Core Archive)
- Full REST API access
- Links:
- OBIS.org
- CalCOFI datasets: obis.org/dataset + “calcofi” Keyword
6.3.4 ERDDAP
Environmental Research Division Data Access Program
- Tabular and gridded data server
- Advanced subsetting by space, time, and parameters
- Multiple output formats (CSV, JSON, NetCDF, etc.)
- RESTful API with direct data access
- Built-in data visualization tools
- No persistent identifiers but stable URLs
- Links:
6.4 Metadata
The Ecological Metadata Language (EML) (and using R package EML in workflows) serves as a key standard for describing ecological and environmental data. For CalCOFI, EML metadata files are generated alongside data files, providing structured documentation that enables interoperability across different data portals. This metadata-driven approach allows automated ingestion into various data systems while maintaining data integrity and provenance.
The EML specification provides detailed structure for describing datasets, including:
- Dataset identification and citation
- Geographic and temporal coverage
- Variable definitions and units
- Methods and protocols
- Quality control procedures
- Access and usage rights
This standardized metadata enables automated data transformation and ingestion into various portal systems while preserving the original data context and quality information.
6.5 Meta-Portals
6.5.1 Google Dataset Search
The JSON-LD metadata in the Portal dataset web pages get indexed by Google Dataset Search through schema.org metadata. This ensures that CalCOFI data is discoverable through Google search and other search engines.
6.5.2 ODIS
Ocean Data Information System
ODIS uses the same technology as Google Dataset Search (schema.org, JSON-LD), but focuses on ocean data. CalCOFI curates a sitemap of authoritative datasets to server to ODIS.org
This federated approach ensures that CalCOFI data remains:
- Discoverable through multiple channels
- Properly cited and attributed
- Integrated with global ocean data systems
6.6 CalCOFI.io Tools
CalCOFI is also developing an integrated database and tools that enable efficient data access and analysis:
6.6.1 APIs
- RESTful endpoints for programmatic access
- Filtering by space, time, and taxonomic parameters
- Relationship queries across tables
- Links:
6.6.2 Library
- Direct data access from R
- Built-in analysis functions
- Integration with tidyverse ecosystem
- Link:
6.6.3 Apps
- Interactive data exploration with Shiny applications
- User-friendly interfaces
- Subset and download data
- Link:
- calcofi.io, App button