Saving-results job runs on; a retry attempt failed earlier in the evening
Dynamic compositional representations ยท October 16, 2026
On 2026-10-16, a job saving results is running on a transfer node, and an earlier attempt to save results failed within a minute. The study stands at 4,377 of 6,068 runs finished (72.1%), with no runs running and none waiting.
What ran
Two jobs ran on a transfer node with 2 CPU cores. Job 8055 started at 22:57 UTC and ended with an error (code 1) in under a minute. Job 8056 started at 23:06 UTC and is still running, saving results.
What finished
No training runs finished on 2026-10-16. The latest grid activity on record is the training-regimes work, which finished in 3 jobs at 21:25 UTC.
The numbers
- Runs finished so far: 4,377 of 6,068 (72.1%)
- Runs running: 0; runs waiting: 0
- Runs failed: 3; runs interrupted: 20; runs not started: 1,668
- Node-hours used so far: 150.18
- Mean run length: 6.8 minutes
- Result cells with results: 584; cells complete: 578; cells in the grids: 821
- Job 8056 has run for 1 h 24 min of its 6 h limit, charged 0.01 node-hours
- Job 8055 was charged 0 node-hours
Failures and what was done
Job 8055, which was saving results, ended with an error (code 1). A second attempt, job 8056, started at 23:06 UTC and is running. Separately, 3 runs in ffn1-lm failed with exit code 1 (ffn1-lm__W4__s0, s1, s2 in job 3583877); nothing is queued for that grid and 9 of 12 runs are finished.
What was learned
Nothing new was recorded in the notebook or reviews for this date.
What was decided
One commit was made: keep the daily research news within the drafting input budget.
What is next
- saving results (job 8056) is running now.
- Several grids have nothing queued and unfinished runs: grid2-role (0 of 177), fold1 (0 of 36), fold1b (0 of 14), fold1c-lm (0 of 12), ctx3-lm (0 of 6), cut5 (157 of 288), cut8b (210 of 252), and weight-w4a (116 of 324).
- Several runs hold locks from jobs that have ended, in cut5, cut8b, interject1, cut4c, and weight-w4a.