6  Portals

6.1 Overview

CalCOFI data is available through various portals, each serving different purposes and user needs. This document outlines the main access points and their characteristics.

6.2 Data Flow

While it would be ideal for CalCOFI data to be available through a single portal, each portal has its strengths and limitations. The following diagram illustrates one possible realization of data flow between CalCOFI data and the portals: from raw data to the integrated database to portals and meta-portals.

graph TD
  %% nodes with styles
  raw([raw data]):::source

  subgraph calcofi[CalCOFI.io]
    db[(database)]:::database
    web[CalCOFI.org]:::website
    api[APIs]:::api
    lib[library]:::code
    app[apps]:::code
    flow[workflows]:::code
  end

  subgraph portals[Portals]
    edi[EDI]:::portal
    obis[OBIS]:::portal
    erddap[ERDDAP]:::portal
    ncei[NCEI]:::portal
  end

  subgraph meta[Meta-Portals]
    odis[ODIS]:::metaportal
    datagov[data.gov]:::metaportal
  end

  %% edges
  raw --> db
  db  --> api
  db  --> web
  api --> lib
  api --> app
  api --> flow
  flow  --> portals
  portals -->|sitemap| odis
  portals --> datagov

  %% Custom styles
  classDef source     fill:#E0E7FF,stroke:#6366F1,stroke-width:2px
  classDef database   fill:#FEF3C7,stroke:#D97706,stroke-width:2px
  classDef website    fill:#F3E8FF,stroke:#9333EA,stroke-width:2px
  classDef api        fill:#E0E7FF,stroke:#6366F1,stroke-width:2px
  classDef code       fill:#DBEAFE,stroke:#3B82F6,stroke-width:2px
  classDef portal     fill:#F0FDF4,stroke:#22C55E,stroke-width:2px
  classDef metaportal fill:#FEF2F2,stroke:#DC2626,stroke-width:2px

  %% Style subgraphs
  style calcofi fill:#F8FAFC,stroke:#CBD5E1,stroke-width:2px
  style portals fill:#F8FAFC,stroke:#CBD5E1,stroke-width:2px
  style meta fill:#F8FAFC,stroke:#CBD5E1,stroke-width:2px
Figure 6.1: Flow of data from raw to database to portals and meta-portals.

In practice, CalCOFI is a partnership with various contributing members, so the authoritative dataset might flow differently, such as from EDI to the database to the other portals. The other portals, such as OBIS or ERDDAP, serve different audiences or purposes. The meta-portals like ODIS and Data.gov then index these portals to provide broader discovery of CalCOFI datasets.

6.3 Portals

While some portals serve as data repositories, others provide advanced data access and visualization tools. The following sections describe the main portals where CalCOFI data is available and their key features.

Table 6.1: Portal Capabilities.
Full Archive Versioning DOI Issued Query by xyt Query by taxa Multiple formats API Access
EDI ✔ ✔ ✔ ▲ ▲ ✖ ▲
NCEI ✔ ✔ ✔ ✖ ✖ ✖ ▲
OBIS ▲ ▲ ▲ ✔ ✔ ▲ ✔
ERDDAP ▲ ✖ ✖ ✔ ▲ ✔ ✔
Capability Legend: ✔ = full, ▲ = partial, ✖ = none

6.3.1 EDI

Environmental Data Initiative

  • Complete dataset archives using DataOne software and EML metadata
  • DOIs issued for all datasets ensuring citability
  • Full archive allowing for any data file types
  • Basic spatial and temporal filtering through web interface
  • Download in original formats with metadata
  • Access through DataOne API
  • Links:

6.3.2 NCEI

National Centers for Environmental Information

  • Long-term archival of oceanographic data
  • DOIs issued for dataset submissions
  • Standardized metadata using ISO 19115-2
  • Basic search interface with geographic and temporal filtering
  • Data preserved in original submission formats
  • Access through NCEI API services
  • Links:

6.3.3 OBIS

Ocean Biodiversity Information System

  • Specialized in marine biodiversity data
  • Standardized using DarwinCore fields
  • Extended measurements supported via extendedMeasurementOrFact
  • Powerful filtering by space, time, and taxonomic parameters
  • Multiple download formats (CSV, JSON, Darwin Core Archive)
  • Full REST API access
  • Links:

6.3.4 ERDDAP

Environmental Research Division Data Access Program

6.4 Metadata

The Ecological Metadata Language (EML) (and using R package EML in workflows) serves as a key standard for describing ecological and environmental data. For CalCOFI, EML metadata files are generated alongside data files, providing structured documentation that enables interoperability across different data portals. This metadata-driven approach allows automated ingestion into various data systems while maintaining data integrity and provenance.

graph LR
    subgraph dataset[Dataset]
        data([data.csv]):::source
        eml[metadata.eml]:::database
    end

    subgraph portals[Portals]
        edi[EDI]:::portal
        erddap[ERDDAP]:::portal
        obis[OBIS]:::portal
        ncei[NCEI]:::portal
    end

    subgraph metaportals[Meta-Portals]
        odis[ODIS]:::metaportal
        datagov[data.gov]:::metaportal
    end

    dataset --> portals
    eml -->|json-ld| metaportals

    %% Custom styles
    classDef source     fill:#E0E7FF,stroke:#6366F1,stroke-width:2px
    classDef database   fill:#FEF3C7,stroke:#D97706,stroke-width:2px
    classDef portal     fill:#F0FDF4,stroke:#22C55E,stroke-width:2px
    classDef metaportal fill:#FEF2F2,stroke:#DC2626,stroke-width:2px

    %% Style subgraphs
    style dataset     fill:#F8FAFC,stroke:#CBD5E1,stroke-width:2px
    style portals     fill:#F8FAFC,stroke:#CBD5E1,stroke-width:2px
    style metaportals fill:#F8FAFC,stroke:#CBD5E1,stroke-width:2px
Figure 6.2: Metadata in the form of ecological metadata language (EML) is used to describe the dataset in a consistent manner that can be ingested by the portals.

The EML specification provides detailed structure for describing datasets, including:

  • Dataset identification and citation
  • Geographic and temporal coverage
  • Variable definitions and units
  • Methods and protocols
  • Quality control procedures
  • Access and usage rights

This standardized metadata enables automated data transformation and ingestion into various portal systems while preserving the original data context and quality information.

6.5 Meta-Portals

6.5.2 ODIS

Ocean Data Information System

ODIS uses the same technology as Google Dataset Search (schema.org, JSON-LD), but focuses on ocean data. CalCOFI curates a sitemap of authoritative datasets to server to ODIS.org

This federated approach ensures that CalCOFI data remains:

  • Discoverable through multiple channels
  • Properly cited and attributed
  • Integrated with global ocean data systems

6.6 CalCOFI.io Tools

CalCOFI is also developing an integrated database and tools that enable efficient data access and analysis:

6.6.1 APIs

  • RESTful endpoints for programmatic access
  • Filtering by space, time, and taxonomic parameters
  • Relationship queries across tables
  • Links:

6.6.2 Library

  • Direct data access from R
  • Built-in analysis functions
  • Integration with tidyverse ecosystem
  • Link:

6.6.3 Apps

  • Interactive data exploration with Shiny applications
  • User-friendly interfaces
  • Subset and download data
  • Link: