A tour of the CalCOFI.io database
The first recorded tour of CalCOFI's integrated database: where to start on calcofi.io, the Explorer and CTD transects apps, how the tables fit together, and pulling the data in R and Python. The people who collect the data were on the call, and said what is missing.
- Recording70 minutes, on Google Drive
- Slides25 slides, with a link to every page shown
- Notesthe agenda, the four polls and the wish list
- Scriptsthe R and Python walk-throughs, with a guide to the tables
In the recording
Times are approximate, to about a minute.
- Welcome, and where the slides, notes and scripts are
- The plan for the hour
- Why one database; every release frozen, with a DOI
- calcofi.io: datasets, species and measurements
- Poll: on a dataset page, what would you click first to get the data?
- A simpler Explorer, as a mock-up
- The Explorer: six views, a title that states the query, sharing and downloads
- CTD transects
- How the tables fit together
- Five tables to know, and the divisions in the data
- Derived products
- The data in R and Python, in Positron
- Discussion
What the tour showed
One database. Each dataset used to have tables of its own. To set a net tow beside the CTD cast of the same station visit, the datasets need one schema. CalCOFI's follows the Darwin Core extension for measurements that OBIS uses, so any kind of measurement fits. Twenty datasets go through one pipeline. Each release is frozen in the cloud with the code and source files that built it, and gets a DOI from Zenodo, so a result can be rerun against the release it used.
calcofi.io. Every dataset, species and measurement has its own page, saying who collected it, over which years and area, how to cite it, and where to get it.
The apps. The Explorer maps one organism or one ocean variable at a time, six ways: stations, hexagons, contours, cruises, regions and sections. Its title is a sentence that states the query, and its web address reproduces the view. CTD transects draws a section along any line for any cruise since 1993, as measured values or as departures from the 1993–2013 average.
The tables. Five are worth knowing. sample says where and when something was sampled.
obs_env and obs_bio hold one value at one depth, with species keyed to WoRMS (and
birds to ITIS). sample_measurement holds one value for a whole cast or tow, and
climatology is the baseline every anomaly subtracts. Datasets join on the station visit. Derived
values, such as density, spiciness and mixed-layer depth, are computed once, in the pipeline.
Code. The calcofi4r and calcofi4py packages connect to a release, list its tables and run SQL on them. A script that names its release returns the same rows next year. The tour's scripts run in Positron or VS Code with no login.
What we heard
- Where is the download button? Asked what they would click first on a dataset page, attendees found no download button, too many ways in, and words like Parquet, STAC and JSON. They wanted a button, and a CSV.
- One dataset or several? Someone after a single dataset can already get it from ERDDAP. The integrated database is for several kinds of data together, such as fish larvae beside temperature. The Explorer lost its second variable when an early side-by-side view was dropped, and it is coming back.
- Show the big divisions. Environment or biology, observed or derived, bottle or CTD cast (the bottle database ends in 2021; the casts go on), and the steps within a sample: cruise, station visit, cast, then bottles or net tows. A simple picture of these would help more than the full diagram.
- Two modes. A simple path to the data in about four clicks, and the full technical depth for those who want it. One attendee would put little effort into the layers in between, such as helpers for writing SQL, now that an AI assistant can fix a query.
- One front door. The new calcofi.org, now being built, should lead into calcofi.io's data with no boundary a visitor notices. calcofi.io was built apart because the WordPress site could not host the apps; the two are meant to join.
- Short videos. A recorded walk of about ten minutes from download to plot, with CTD data as the example, to follow at your own pace, with the clicks made visible.
Since the tour
- Every dataset page has one Download button: CSV first, Excel where it fits, and the citation and licence in the zip.
- The Explorer opens on a simpler Find data panel, the mock-up shown in the tour. Add
?ui=classicto its address for the old one. - The front door asks “What do you want to do?” and offers seven tasks in plain words.
- Short tutorials are planned for January and February 2027.
Requests and questions are welcome as an issue on the tour's repository, or through the feedback button in the Explorer.
Data shown: CalCOFI Integrated Database, release v2026.11.09 (doi:10.5281/zenodo.24083894). The scripts are under the MIT licence; the data keep their own licences and citations.