Reproducible Quarto / R notebooks that ingest source datasets, freeze versioned releases, and publish the CalCOFI integrated database. Rendered from CalCOFI/workflows.
45 notebooks · 19 metadata-described workflows · generated 2026-07-28
Acquire, standardize, and load a source dataset into the integrated database. Cards are grouped by data provider.
Bird and mammal observations along CalCOFI (and NMFS, CPR) cruise transects, 1987-2021. Transect effort + counts + species/behavior lookups. Curated via CCE-LTER DataZoo 255 (PI Bill Sydeman).
Hydrographic bottle data including temperature, salinity, oxygen, nutrients, pigments, and C14 from CalCOFI cruises.
CTD downcast profiles with temperature, salinity, oxygen, fluorescence, and other sensors at standard depths.
Discrete profile dissolved inorganic carbon (DIC) and total alkalinity (TA) from CalCOFI Niskin bottle samples, quality-controlled using WOCE flagging.
Spiny lobster (Panulirus interruptus) phyllosoma larvae counts by developmental stage from CalCOFI net tows, 1951-2008. Source: EDI knb-lter-cce.188.4 (Koslow).
Abundance and species composition of phytoplankton (385 taxonomic categories) in the California Current, 1996-2022, by inverted-microscope counts (E. Venrick). Samples from the near-surface "second depth" are pooled across stations into four regions (NE, SE, Alley, Offshore; Hayward & Venrick 1998) before counting, so the grain is cruise x region, not per-station. Source: EDI knb-lter-cce.254.4.
Continuous Underway Fish Egg Sampler (CUFES) egg counts (sardine, anchovy, jack mackerel, hake, squid, other) with underway environmental conditions, from CalCOFI cruises (1996-present). Source: NOAA CoastWatch ERDDAP erdCalCOFIcufes.
Ichthyoplankton (fish eggs and larvae) collected by bongo and manta net tows on CalCOFI cruises, processed by SWFSC.
Zooplankton net-tow sample registry from the SIO Pelagic Invertebrate Collection, filtered to the CalCOFI region. Position, time, depth, and gear metadata per tow. Biovolume measurements are not in the source file and are pending from the provider (see Questions for Data Providers).
Euphausiid (krill) abundance from CalCOFI / CCE-LTER net tows, 1951-2019, one row per tow. Provisional pending provider answers on abundance units and taxonomic scope (see Questions for Data Providers).
Holoplankton abundance and carbon biomass by 33 higher taxa (copepods to family, plus amphipods, chaetognaths, salps, pyrosomes, pteropods, ostracods, foraminifera, radiolaria and more) from CalCOFI net tows, 1951-2021, enumerated by microscopy (Ohman Lab / SIO Pelagic Invertebrate Collection). One row per (sample, taxon, measurement_type); unpooled tows are station-resolved, pooled samples are per-cruise regional composites. Raw (uncorrected) abundance per 1000 m3 and per m2, plus biomass mg C/m2.
Zooplankton abundance, carbon biomass, and size (Feret diameter, individual carbon content) from ZooScan optical imaging of CalCOFI / CCE-LTER PRPOOS net tows on lines 80, 87 and 90, 2005-present, machine-classified into 23 bioclasses (copepod groups, euphausiids, chaetognaths, appendicularians, salps, doliolids, pteropods, rhizaria and more). One row per (sample, taxon, measurement_type); per-station tows. Source: SIO Ocean Informatics ZooScan portal (Ohman Lab; interface by Marina Frants).
Push released tables out to external repositories (ERDDAP, OBIS).
Freeze a versioned release and the maintenance utilities that support the pipeline.
Planning and reference notebooks. (Candidate to fold into calcofi.io/docs/.)
Exploratory analyses and legacy load scripts, kept for reference.
The targets dependency graph parsed from each notebook's
calcofi: front-matter (the same fields
calcofi4db::build_targets_list() wires up), grouped into
subgraphs by workflow type and color-coded to match the sections above.
graph LR ingest_calcofi_bird_mammal_census(["ingest_calcofi_bird_mammal_census"]):::c_ingest_calcofi_bird_mammal_census ingest_calcofi_bottle(["ingest_calcofi_bottle"]):::c_ingest_calcofi_bottle ingest_calcofi_ctd_cast(["ingest_calcofi_ctd_cast"]):::c_ingest_calcofi_ctd_cast ingest_calcofi_dic(["ingest_calcofi_dic"]):::c_ingest_calcofi_dic ingest_calcofi_phyllosoma(["ingest_calcofi_phyllosoma"]):::c_ingest_calcofi_phyllosoma ingest_calcofi_phytoplankton(["ingest_calcofi_phytoplankton"]):::c_ingest_calcofi_phytoplankton ingest_cce_lter_euphausiids(["ingest_cce_lter_euphausiids"]):::c_ingest_cce_lter_euphausiids ingest_cce_lter_zoodb(["ingest_cce_lter_zoodb"]):::c_ingest_cce_lter_zoodb ingest_cce_lter_zooscan(["ingest_cce_lter_zooscan"]):::c_ingest_cce_lter_zooscan ingest_pic_zooplankton(["ingest_pic_zooplankton"]):::c_ingest_pic_zooplankton ingest_spatial(["ingest_spatial"]):::ingest ingest_swfsc_cufes(["ingest_swfsc_cufes"]):::c_ingest_swfsc_cufes ingest_swfsc_ichthyo(["ingest_swfsc_ichthyo"]):::c_ingest_swfsc_ichthyo publish_ctd_cast_to_netcdf(["publish_ctd_cast_to_netcdf"]):::publish publish_ichthyo_to_netcdf(["publish_ichthyo_to_netcdf"]):::publish publish_ichthyo_to_obis(["publish_ichthyo_to_obis"]):::publish release_database(["release_database"]):::release ingest_swfsc_ichthyo --> ingest_calcofi_bird_mammal_census ingest_swfsc_ichthyo --> ingest_calcofi_bottle ingest_swfsc_ichthyo --> ingest_calcofi_ctd_cast ingest_calcofi_bottle --> ingest_calcofi_dic ingest_swfsc_ichthyo --> ingest_calcofi_phyllosoma ingest_swfsc_ichthyo --> ingest_calcofi_phytoplankton ingest_swfsc_ichthyo --> ingest_cce_lter_euphausiids ingest_swfsc_ichthyo --> ingest_cce_lter_zoodb ingest_swfsc_ichthyo --> ingest_cce_lter_zooscan ingest_swfsc_ichthyo --> ingest_pic_zooplankton ingest_swfsc_ichthyo --> ingest_swfsc_cufes release_database --> publish_ctd_cast_to_netcdf release_database --> publish_ichthyo_to_netcdf release_database --> publish_ichthyo_to_obis ingest_calcofi_bird_mammal_census --> release_database ingest_calcofi_bottle --> release_database ingest_calcofi_ctd_cast --> release_database ingest_calcofi_dic --> release_database ingest_calcofi_phyllosoma --> release_database ingest_calcofi_phytoplankton --> release_database ingest_cce_lter_euphausiids --> release_database ingest_cce_lter_zoodb --> release_database ingest_cce_lter_zooscan --> release_database ingest_pic_zooplankton --> release_database ingest_spatial --> release_database ingest_swfsc_cufes --> release_database ingest_swfsc_ichthyo --> release_database classDef c_ingest_calcofi_bird_mammal_census fill:#f0d2e8,stroke:#00000033,color:#10161c; classDef c_ingest_calcofi_bottle fill:#f5cad9,stroke:#00000033,color:#10161c; classDef c_ingest_calcofi_ctd_cast fill:#e0ccf0,stroke:#00000033,color:#10161c; classDef c_ingest_calcofi_dic fill:#ffd2bb,stroke:#00000033,color:#10161c; classDef c_ingest_calcofi_phyllosoma fill:#e0bbf0,stroke:#00000033,color:#10161c; classDef c_ingest_calcofi_phytoplankton fill:#b5e8b0,stroke:#00000033,color:#10161c; classDef c_ingest_cce_lter_euphausiids fill:#d2f0bb,stroke:#00000033,color:#10161c; classDef c_ingest_cce_lter_zoodb fill:#d9c2f0,stroke:#00000033,color:#10161c; classDef c_ingest_cce_lter_zooscan fill:#c2e8f0,stroke:#00000033,color:#10161c; classDef c_ingest_pic_zooplankton fill:#bbe0f0,stroke:#00000033,color:#10161c; classDef ingest fill:#cdd9e5,stroke:#00000033,color:#10161c; classDef c_ingest_swfsc_cufes fill:#f0e8bb,stroke:#00000033,color:#10161c; classDef c_ingest_swfsc_ichthyo fill:#cdebc6,stroke:#00000033,color:#10161c; classDef publish fill:#20c997,stroke:#00000033,color:#10161c; classDef release fill:#f06595,stroke:#00000033,color:#10161c;