---
title: "Deploy consumers"
subtitle: "Bring every server-side consumer onto the promoted release"
author: "CalCOFI"
date: today
format:
html:
toc: true
toc-depth: 3
code-fold: true
code-tools: true
df-print: kable
calcofi:
target_name: deploy_consumers
workflow_type: deploy
dependency:
- test_release
output: _output/deploy_consumers.html
workflow_url: https://calcofi.io/workflows/deploy_consumers.html
description: >
Brings the server-side consumers (db-viz-hex, db-viz-cruise, the h3t tile
API) onto the promoted release, and reports which release each is actually
serving before and after. Runs after test_release, so it can only ever
deploy a version that passed the consumer-contract suite. Deploying is the
default; CALCOFI_DEPLOY=false makes it a read-only drift report.
editor_options:
chunk_output_type: console
---
## Why this is a target
Every consumer that follows a release has drifted at least once, and always the
same way: `latest.txt` was correct and something *derived* from it lagged, with
nothing erroring. The releases index was a static page nobody regenerated; the
h3t API held its old database file open across a symlink flip; db-viz-hex's tile
tag was hardcoded and sat three releases behind.
The common cause is that the deploy lived only as prose in a runbook. Making it
a **target** rather than a step inside `test_release.qmd` means:
- it is visible in `tar_visnetwork()` / `tar_outdated()`, so "the release shipped
but consumers were never updated" is a state you can *see*
- it can be re-run on its own (`tar_make(names = tidyselect::any_of("deploy_consumers"))`)
without re-running the query suite
- it depends on `test_release`, so it can only ever deploy a version that passed
the consumer-contract checks
GitHub-hosted consumers (`db-query`, `db-viz-station`) redeploy themselves on
dispatch from the promote step and are not handled here. ERDDAP has its own
target, `publish_to_erddap`, because its ~1.6 GB server-side parquet pull is the
slowest leg and is worth running independently.
```{r}
#| label: setup
#| message: false
#| warning: false
librarian::shelf(dplyr, glue, here, knitr, tibble, quiet = TRUE)
here <- here::here
# Deploying is the DEFAULT. A release that consumers never receive is not a
# release, and every drift this notebook exists to prevent came from a deploy
# step that had to be remembered. Opting OUT is the unusual case, so it is what
# takes a flag:
#
# CALCOFI_DEPLOY=false report consumer state and change nothing (dry run)
#
# The earlier design had this backwards -- deploy required CALCOFI_DEPLOY=true,
# which meant the normal path silently did nothing, AND (because targets marks a
# skipped render as done) setting the flag afterwards would not re-trigger it
# without a tar_invalidate. Two ways to get a stale consumer set while every
# target reported success.
DEPLOY <- !identical(tolower(Sys.getenv("CALCOFI_DEPLOY", "true")), "false")
SSH_HOST <- Sys.getenv("CALCOFI_SSH_HOST", "calcofi")
`%||%` <- function(x, y) if (is.null(x) || length(x) == 0 || !nzchar(x)) y else x
RELEASE <- tryCatch(
trimws(readLines(
"https://storage.googleapis.com/calcofi-db/ducklake/releases/latest.txt",
warn = FALSE)[1]),
error = function(e) NA_character_)
cat(glue("promoted release : {RELEASE}\n"))
cat(glue("deploy : {DEPLOY} ",
"({ifelse(DEPLOY, 'default; set CALCOFI_DEPLOY=false for a dry run', 'CALCOFI_DEPLOY=false')})\n"))
cat(glue("ssh host : {SSH_HOST}\n"))
```
## Which release is each consumer actually serving? ----
```{r}
#| label: status
# Read the state consumers actually expose, not what we believe we deployed.
# Each probe below is chosen because the OBVIOUS check is the one that lies:
# * the h3t API's table list looks current even when it is serving an old file
# — only `db_mtime` reveals which inode it has open (this is how a stale
# v2026.08.03 was caught while the symlink already said v2026.08.04)
# * an app returning HTTP 200 says nothing about which database it opened
ssh_lines <- function(cmd) {
out <- suppressWarnings(system2(
"ssh", c(SSH_HOST, shQuote(cmd)), stdout = TRUE, stderr = FALSE))
out[nzchar(out)]
}
ssh_out <- function(cmd) paste(ssh_lines(cmd), collapse = " ")
# -L: several of these endpoints redirect (ERDDAP's info index 302s), and a
# redirect is a healthy response, not a failure.
http_code <- function(url) suppressWarnings(system2(
"curl", c("-sL", "-o", "/dev/null", "-w", "%{http_code}", "--max-time", "45",
shQuote(url)), stdout = TRUE))
# Which file does the h3t API actually have open?
#
# Ask it for the mtime, then match that against the mtimes on disk — IN R. The
# obvious version does the comparison inside the ssh command, but that nests
# quotes through glue -> ssh -> shell and silently produces no match, which is
# indistinguishable from "no file matched". Fetch the list, compare here.
h3t_serving <- function() {
h <- suppressWarnings(system2("curl",
c("-s", "--max-time", "45", "https://h3t.calcofi.io/h3t/health"),
stdout = TRUE))
h <- paste(h, collapse = "")
mt <- sub('.*"mtime":"([0-9.]+)".*', "\\1", h)
if (!grepl("^[0-9.]+$", mt)) return("(unreachable)")
disk <- ssh_lines(
"cd /share/github/CalCOFI/db-viz-hex/data && stat -c '%.6Y %n' calcofi_v*.duckdb")
m <- regmatches(disk, regexpr("^[0-9.]+", disk))
f <- sub("^[0-9.]+ +", "", disk)
# compare to 3 decimals: the API reports the mtime it saw at open time, which
# can differ from stat's in the last digits of float formatting
hit <- which(substr(m, 1, nchar(mt) - 3) == substr(mt, 1, nchar(mt) - 3))
if (length(hit)) f[hit[1]] else glue("(unmatched mtime {mt})")
}
fetch <- function(url) paste(suppressWarnings(system2(
"curl", c("-sL", "--max-time", "60", shQuote(url)), stdout = TRUE)), collapse = "")
# db-query is GitHub Pages, and its bump workflow committed _config.yml with the
# default GITHUB_TOKEN -- which does NOT trigger other workflows, so the Pages
# deploy never fired. The repo said v2026.08.04 while the site served
# v2026.08.02, and the bump run was green. So the probe compares the LIVE PAGE
# against the config, not the config against latest.txt: the config was right.
dbquery_serving <- function() {
cfg <- fetch("https://raw.githubusercontent.com/CalCOFI/db-query/main/_config.yml")
want <- sub(".*default_version:[ ]*([^\n#]+).*", "\\1", cfg)
want <- trimws(sub("\n.*", "", want))
live <- fetch("https://calcofi.io/db-query/")
if (!nzchar(want) || !grepl("^v20", want)) return("(config unreadable)")
if (grepl(want, live, fixed = TRUE)) want else glue("{want} in config, NOT on the live page")
}
# the browsable release listing: a static page that is rebuilt on promote
idx_serving <- function() {
h <- fetch("https://storage.calcofi.io/calcofi-db/ducklake/releases/index.html")
v <- regmatches(h, gregexpr("v20[0-9]{2}\\.[0-9]{2}\\.[0-9]{2}", h))[[1]]
if (!length(v)) "(unreadable)" else max(v)
}
consumer_status <- function() {
tibble::tribble(
~consumer, ~serving, ~probe,
"db-viz-hex", ssh_out("readlink /share/github/CalCOFI/db-viz-hex/data/calcofi_latest.duckdb"),
"calcofi_latest.duckdb symlink",
"h3t API", h3t_serving(),
"/h3t/health db_mtime -> file on disk",
"ERDDAP", ssh_out("grep -om1 'release v2026[0-9.]*' /share/github/CalCOFI/erddap/content/datasets.xml"),
"datasets.xml summary text",
"db-viz-cruise", ssh_out("stat -c %y /share/data/db-viz-cruise/db-viz-cruise.duckdb 2>/dev/null | cut -c1-16"),
"database mtime (carries no version)",
"db-query site", dbquery_serving(),
"live page vs _config.yml default_version",
"releases index", idx_serving(),
"storage.calcofi.io releases/index.html") |>
mutate(http = c(http_code("https://app.calcofi.io/db-viz-hex/"),
http_code("https://h3t.calcofi.io/h3t/health"),
http_code("https://erddap.calcofi.io/erddap/info/index.json"),
http_code("https://app.calcofi.io/db-viz-cruise/"),
http_code("https://calcofi.io/db-query/"),
http_code("https://storage.calcofi.io/calcofi-db/ducklake/releases/")))
}
before <- consumer_status()
kable(before, caption = glue("Consumer state BEFORE (promoted release: {RELEASE})"))
```
## Deploy ----
```{r}
#| label: deploy
#| results: asis
if (!DEPLOY) {
cat(glue(
"**Dry run — nothing deployed.** `CALCOFI_DEPLOY=false` is set, so this ",
"target reported the state above and changed nothing.\n\n",
"The server-side consumers may therefore be serving a previous release. ",
"Unset the variable (deploying is the default) and re-run:\n\n",
"```r\n",
"targets::tar_invalidate(names = tidyselect::any_of(\"deploy_consumers\"))\n",
"targets::tar_make(names = tidyselect::any_of(\"deploy_consumers\"))\n",
"```\n\n",
"or run the script directly: `bash scripts/deploy_consumers.sh`.\n"))
} else {
cat("Running `scripts/deploy_consumers.sh`.\n\n```\n")
out <- system2("bash", c(here("scripts", "deploy_consumers.sh"),
"--release", RELEASE),
stdout = TRUE, stderr = TRUE)
cat(paste(out, collapse = "\n"), "\n```\n")
rc <- attr(out, "status") %||% 0L
if (!identical(as.integer(rc), 0L))
stop(glue(
"consumer deploy FAILED for {RELEASE}. latest.txt is promoted and correct, ",
"so calcofi4r and the hosted sites are fine — but db-viz-hex, db-viz-cruise ",
"or the h3t API may still be serving the previous release."))
}
```
```{r}
#| label: status-after
#| eval: !expr DEPLOY
after <- consumer_status()
kable(after, caption = glue("Consumer state AFTER (promoted release: {RELEASE})"))
# The h3t API is the one that fails silently, so assert on it rather than
# trusting the script's own exit status.
h3t_now <- after$serving[after$consumer == "h3t API"]
if (nzchar(RELEASE) && nzchar(h3t_now) && !grepl(RELEASE, h3t_now, fixed = TRUE))
warning(glue(
"h3t API is serving {h3t_now}, not {RELEASE}. It holds its database file ",
"open, so a symlink flip alone does not move it — the container restart in ",
"step 3 is what does."))
```
```{r}
#| label: write-output
#| include: false
# `output:` in the calcofi block is this HTML; quarto writes it. Nothing else to do.
```