Backport #38616 by @bircni
Fixes jobs that get stuck in `cancelling` after Gitea is restarted while
a job is running.
Reproduction:
1. Run a job with `sleep 100`
2. Stop Gitea and wait 120s (> 100s)
3. Start Gitea — the job is still `running`; cancel it, and it stays in
`cancelling`
## Cause
A cancellation is only ever delivered to a runner as the *response* to
its `UpdateTask`
RPC. A runner that has already given up on the task — it crashed, or it
failed to report
the final state while Gitea was unreachable — never calls `UpdateTask`
again, so it never
learns about the cancellation. The task then sits in `cancelling` until
`stop_zombie_tasks`
reaps it, which needs `ZOMBIE_TASK_TIMEOUT` (10m) of silence and only
runs every 5 minutes.
Cancelling actually made this worse. xorm rewrites the `updated` column
on every `UPDATE`,
even when `Cols()` restricts the update to `status`, so persisting the
`cancelling` status
reset the zombie clock. Pressing cancel pushed the cleanup a full
`ZOMBIE_TASK_TIMEOUT`
into the future instead of bringing it forward.
## Change
`StopTask` now skips the `cancelling` handshake when the task has had no
state report from
its runner for longer than `TaskReportTimeout` (1 minute) and cancels it
directly. This
joins the two existing fallbacks — runner deleted, and runner without
cancelling support —
so every cancel path (web UI, API, concurrency, rerun) is covered.
Runners report the state of a running task every few seconds, so a
minute of silence means
the runner is gone. The value is a constant rather than a setting
because it only decides
whether the runner is still reachable, not whether a task should be
killed —
`ZOMBIE_TASK_TIMEOUT` still owns that.
### Tradeoff
If a runner is alive but has been silent for over a minute and is
cancelled in that window,
it skips the graceful post-step cleanup added in #37275. No work is
lost: the runner still
learns the outcome on its next report, because `UpdateTask` returns the
task status as the
result and the runner stops there. That is the behaviour that existed
before #37275.
Co-authored-by: bircni <bircni@icloud.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Backport #38586 by @bircni
Several small fixes to the Actions runner management UI.
### Runner task list links to the job, not the workflow run
Relabeled the first column from "Run" to "Job"; it now shows the job ID
and links to the specific job (`/actions/runs/{runID}/jobs/{jobID}`).
Renamed locale key `task_list.run` to `task_list.job`.
### Missing "Disabled" translation
The runner list rendered a grey label via `actions.runners.disabled`,
but that key
did not exist in `locale_en-US.json`, so the raw key string leaked into
the UI.
Replaced `"actions.runners.disabled"` with `"disabled"`.
### Status column sorting ignored active vs idle
Sorting by status ordered purely on `last_online`, but the displayed
status is
computed from both `last_online` (offline) and `last_active` (idle vs
active).
As a result idle runners were interleaved with active ones. Sorting now
ranks by
the computed status (active → idle → offline). Disabled runners sink to
the bottom
of their status group (`is_disabled` as a secondary key), with
`last_online`/`id`
as stable tiebreakers so pagination stays deterministic.
### Status label colors
Active and idle both rendered green. Idle is now yellow, active green,
and
offline/unknown grey; the separate grey "Disabled" badge is unchanged.
This keeps
connectivity visible even for disabled runners (e.g. a disabled runner
still shows
whether it is idle or offline).
<img width="715" height="406" alt="image"
src="https://github.com/user-attachments/assets/9ef06aa8-a870-4de5-9d94-603a58186908"
/>
Co-authored-by: bircni <bircni@icloud.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Backport #38281 by @bircni
3 reductions in the DB load generated by many runners polling `FetchTask`:
**1. Debounce runner heartbeat writes**
Every poll wrote `last_online`, and every `UpdateTask`/`UpdateLog` wrote
`last_active` — while a runner streams logs that is many writes per
second per runner. These are now persisted only when stale enough to
actually affect the active/offline status (`ShouldPersistLastOnline` /
`ShouldPersistLastActive`), using the existing columns.
**2. Throttle concurrent task picks**
A new in-process semaphore (`MAX_CONCURRENT_TASK_PICKS`) bounds how many
runners run the task-assignment transaction at once, so a fleet polling
together cannot stampede the query. Throttled polls retry on their next
poll without advancing the runner's tasks version.
**3. Paginate the task-pick query**
`CreateTaskForRunner` previously loaded every waiting job in the
runner's scope into memory on each poll (no `LIMIT`). Now it pages
through the waiting backlog oldest-first with `LIMIT`, claiming the
first label-matching job.
Co-authored-by: bircni <bircni@icloud.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
## Summary
Fixes two related bugs that cause jobs in large workflows (50+ parallel
jobs) to never get a runner assigned even though runners are free.
### Bug 1 — Concurrent runner race
When N runners all poll `FetchTask` with a stale `tasksVersion`
simultaneously, they all query the same waiting job list sorted by
`(updated, id)` and all pick **job #1**. Only one wins the `UPDATE WHERE
task_id=0` optimistic lock; the rest return empty-handed but still
receive `latestVersion` in the response. They then consider themselves
"up to date" and skip `PickTask` on every subsequent poll, leaving jobs
#2–50 permanently unassigned.
**Fix:** `CreateTaskForRunner` now iterates through all matching waiting
jobs. When the optimistic lock fails on job #1, it immediately tries job
#2, then #3, etc., each in its own independent transaction so a failed
attempt rolls back cleanly before the next candidate is tried.
`PickTask` no longer wraps this call in an outer `db.WithTx` (which
caused `halfCommitter` entanglement that prevented per-attempt
rollbacks).
### Bug 2 — Idle runner doesn't re-check after finishing a task
`tasks_version` only bumps when a job transitions **to** waiting (new
workflow triggered, blocked→unblocked). After a runner finishes its
current task it polls `FetchTask` with `tasksVersion == latestVersion`,
so the server skips `PickTask` entirely — the remaining 45 waiting jobs
are invisible to the now-idle runner.
**Fix:** Also call `IncreaseTaskVersion` in `UpdateRunJob` when a
(non-reusable-caller) job transitions to a **done** state. Idle runners
then see a version mismatch on their next poll and attempt `PickTask`,
picking up the remaining jobs.
## Fixes#36983
## Summary
1. Add transitional `Cancelling` status (between `Running` and
`Cancelled`); cancel flow marks active tasks `Cancelling`, runner
finalizes to `Cancelled` on terminal result.
2. Taskless jobs cancel directly (no runner to finalize).
3. Runner-protocol responses map `Cancelling` → `RESULT_CANCELLED`.
4. Run/job aggregation treats `Cancelling` as active.
5. Status mapping/aggregation tests + en-US locale added.
**Problem**
When a workflow was cancelled from the UI, jobs were marked cancelled
immediately, which could skip post-run cleanup behavior.
## Solution
Use a transitional status path:
Running → Cancelling → Cancelled
This allows runner finalization and cleanup path execution before final
terminal state.
**Testing**
> 1. go test -tags "sqlite sqlite_unlock_notify" ./models/actions -run
"TestAggregateJobStatus|TestStatusAsResult|TestStatusFromResult"
> 2. go run
github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.11.4 run
./models/actions/... ./routers/api/actions/runner/...
## Related
- act_runner: https://gitea.com/gitea/act_runner/pulls/825 —
independent; this PR's capability gate keeps legacy runners on the
immediate-cancel path. The new flow activates only for runners that
advertise the `cancelling` capability.
Co-authored-by: Nicolas <bircni@icloud.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Claude (Opus 4.7) <noreply@anthropic.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: Giteabot <teabot@gitea.io>
The jobparser sub package in act is only used by Gitea. Move it to Gitea
to make it more easier to maintain.
---------
Co-authored-by: Christopher Homberger <christopher.homberger@web.de>
Giteabot