ci: a job first stops what earlier jobs left running

On 22 Oct a cancelled job left its analysis server running in the
runner, 4 GB, until someone noticed. forgejo-runner v13.1.0 starts every
host step as a process group and sends the group SIGKILL on a cancel or
at timeout-minutes (act/common/cmdgroup.go), and after that no later step
runs, `if: always()` ones neither (they get the cancelled context; only
actions' post steps get a fresh one), so a cleanup at the job's end
can't be relied on. The check job's second step, after the sandbox's
proof, stops every process of a job's working folder (`$HOME/work/`,
also one removed since) that is neither the runner nor the step itself,
TERM then KILL after 5 s, and names each in the log. The runner takes
one job at a time (capacity: 1), so nothing of a running job is touched.
No change on the server.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Tizian 「ティツィアーン」 <raisondetredev@pm.me>

.forgejo/workflows/ci.yaml

@@ -55,6 +55,44 @@ jobs:
           fi
           echo "This job can neither see nor reach production."
 
+      # A job cancelled, or stopped at timeout-minutes, is killed with its step's process
+      # group (forgejo-runner v13.1.0 starts every host step as a group and sends it
+      # SIGKILL), and no later step runs, `if: always()` ones neither. Yet on 22 Oct a
+      # cancelled job left its analysis server running, 4 GB, until someone noticed. The
+      # runner takes one job at a time (capacity: 1), so a process here in a job's working
+      # folder that is neither the runner nor this step is such a leftover: it is stopped
+      # before anything else starts, and the log names it.
+      - name: Leftovers
+        run: |
+          own_group=$(ps -o pgid= -p $$ | tr -d ' ')
+          # This step and the runner above it.
+          keep=' '
+          pid=$$
+          while [ -n "$pid" ] && [ "$pid" -gt 1 ]; do
+            keep="$keep$pid "
+            pid=$(ps -o ppid= -p "$pid" | tr -d ' ')
+          done
+          left=$(ps -e -o pid=,pgid= | while read -r pid pgid; do
+            case "$keep" in *" $pid "*) continue ;; esac
+            [ "$pgid" != "$own_group" ] || continue
+            # Only what a job started: its working folder, or one since removed.
+            case "$(readlink "/proc/$pid/cwd" 2>/dev/null)" in
+              "$HOME/work/"*) echo "$pid" ;;
+            esac
+          done)
+          if [ -z "$left" ]; then
+            echo 'Nothing left behind by earlier jobs.'
+            exit 0
+          fi
+          echo 'Left behind by earlier jobs, stopped now:'
+          # shellcheck disable=SC2086 # one process ID a word
+          ps -o pid=,etime=,rss=,args= -p "$(echo $left | tr ' ' ,)" | cut -c1-200
+          # shellcheck disable=SC2086
+          kill -TERM $left 2>/dev/null || true
+          sleep 5
+          # shellcheck disable=SC2086
+          kill -KILL $left 2>/dev/null || true
+
       # A commit that passed this job before isn't checked again: a fast-forward merge
       # brings the commit a branch's push checked to main unchanged, and only its deploy
       # is new. The marks live in the runner's home; only pushes run this workflow, so