Add support for `strategy.max-parallel` on Gitea Actions matrix jobs.
**How it works**
Jobs over the limit are inserted as `Blocked` instead of `Waiting`, so
runners never see them. When a job finishes, the job-status resolver
promotes one `Blocked` job per freed slot, in job order. Slots are
counted per `JobID` and scoped by reusable-workflow caller. A
`Cancelling` job still owns its runner, so it keeps its slot.
The cap is applied wherever a job can become `Waiting`: initial insert,
rerun, approval, and resolver promotion.
Best effort, not a hard invariant: two concurrent emitter passes can
each promote into the last slot, overshooting by one. It does not
compound, since every later pass recounts.
**Parsing**
Any YAML number, cast to an int as GitHub does (`1.5` → 1). `0` or
negative means unlimited. Expressions (`${{ ... }}`) are not evaluated
yet and fall back to unlimited.
**Migration**
Adds the `max_parallel` column on `action_run_job`. No index or
constraint changes.
**Compatibility**
Existing rows default to `0`, so behaviour is unchanged. No runner
changes needed: the runner protocol is untouched, and since the server
splits the matrix each runner still receives a single job.
Closes https://github.com/go-gitea/gitea/issues/35561
Signed-off-by: Pascal Zimmermann <pascal.zimmermann@theiotstudio.com>
Signed-off-by: ZPascal <pascal.zimmermann@theiotstudio.com>
Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
`RedisBroker.Subscribe` returns before the server acks `SUBSCRIBE`, so a
publish right after it can be dropped, making
`TestRedisBroker/CrossBroker` fail intermittently on loaded CI runners
([example](https://github.com/go-gitea/gitea/actions/runs/30257188479/job/89948425118)).
Each scenario now uses its own topic and waits for `PUBSUB NUMSUB`
before publishing. `MemoryBroker` registers synchronously and skips the
wait.
* Closes#36942
* Fixes#19265
Replaces the SSE-based push channel (`/user/events`) with a WebSocket
endpoint (`/-/ws`).
### What changes
- **New `/-/ws` endpoint** (authenticated). One WebSocket per origin,
shared across tabs via a single `SharedWorker`.
- **Pubsub broker** (`services/pubsub`) for fan-out by topic, behind a
`Broker` interface. `MemoryBroker` is the default (single process); a
Redis backend is available for multi-process setups, configured via
`[websocket].PUBSUB_TYPE` / `PUBSUB_CONN_STR`. The internal Gitea queue
was not usable here because it has FIFO/single-consumer semantics.
- **Push-only event production.** Events are emitted by write-triggered
notifiers — `NotificationCountChange`, `PublishStopwatchesForUser`, and
the logout publisher — wired into the existing `notify.Notifier`
interface. No server-side pollers.
- **Typed pub/sub on the client.** `web_src/js/modules/worker.ts` is a
singleton transport; features subscribe per event type via
`onUserEvent('notification-count', cb)` instead of branching on
`event.data.type`.
- **Wire contract** (`UserEventType` union) is shared between the worker
and consumers via `web_src/js/types.ts`, kept in sync with
`services/websocket/events.go`.
- **Client-side periodic polling fallback** kicks in only when the
WebSocket cannot be established (e.g. proxy blocks WS, browser lacks
module-SharedWorker support).
### What's removed
- `modules/eventsource` (SSE manager, run loop, messenger).
- `/user/events` route and `tests/integration/eventsource_test.go`.
- All server-side polling for stopwatches and notification counts.
### Stopwatch multi-tab fix
The navbar stopwatch icon was previously rendered conditionally on `{{if
$activeStopwatch}}`, so tabs loaded before the timer started had no DOM
element to update. The icon and popup are now always rendered (toggled
with `tw-hidden`), and the start/stop/cancel handlers POST silently so
all open tabs reflect the change in real time.
### Deployment note
WebSocket needs the upgrade headers to pass through a reverse proxy,
e.g. for nginx:
```nginx
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
```
Without them the WebSocket cannot be established, and after 3
consecutive failed opens the shared worker signals `push-unavailable`:
the notification count and stopwatch fall back to periodic polling on
the existing `[ui.notification]` timeouts. Real-time push is lost, the
features keep working. The reverse-proxy docs need the same note (see
the `docs-update-needed` label).
---------
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Epid <rexmrj@gmail.com>
Fixes a bug in Actions context generation for `pull_request_target`
workflows where `github.ref` / `gitea.ref` was incorrectly populated
with `refs/heads/owner:branch` instead of `refs/heads/branch`.
### Problem Statement
In `services/actions/context.go`, when constructing the `ref` string for
`pull_request_target` events:
```go
ref = git.BranchPrefix + pullPayload.PullRequest.Base.Name
Signed-off-by: Sudhanshu Singh <sudhanshuwriterblc@gmail.com>
Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
The handler of `/actions/runs/{run}/approve` doesn't check if the run is
already approved. If a run is re-approved, its jobs' status will be
reset to `StatusWaiting`, causing incorrect job status.
Various fixes to actions
1. **Cap total jobs per run in reusable-workflow expansion** — only
nesting depth was capped, so fan-out + nested reusable workflows could
explode job-row inserts and exhaust the DB from a single push. Now
enforces `MaxJobNumPerRun` in the insert path.
2. **Reject rerun-failed when a run has no failed jobs** — an empty job
list meant "re-run everything", so `rerun-failed` on a green run re-ran
all jobs. Now errors (web + API).
3. **Don't adopt external commit statuses into the legacy hash** — the
pre-#35699 Context-only hash matched API-posted statuses too, collapsing
two same-named workflows into one check. Now limited to Actions-user
rows.
4. **Don't cut post-cancel cleanup short in `StopEndlessTasks`** — the
sweep force-stopped just-cancelled jobs mid-cleanup. Now targets
`StatusRunning` only; stalled cancels stay covered by `StopZombieTasks`.
5. **Avoid redundant run reload in `GenerateGiteaContext`** — resolving
`github.triggering_actor` reloaded the run already passed in. Now loads
only the trigger user via new `ActionRunAttempt.LoadTriggerUser`.
---------
Co-authored-by: Zettat123 <zettat123@gmail.com>
Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Fixes#38494
`DeleteRepositoryDirectly` left rows behind in seven registered tables
carrying a repo-scoped key: `action_variable`, `action_run_attempt`,
`action_tasks_version`, `renamed_branch`, `commit_status_summary`,
`commit_status_index` and `repo_transfer`. Repository IDs are `pk
autoincr` and never reissued, so the orphaned rows were unreachable, but
they accumulated forever (unbounded table growth, referential
inconsistency; not a security issue, see the issue discussion).
This adds all seven to the `deleteBeans` cascade, each placed next to
its sibling bean
(`CommitStatus`/`CommitStatusIndex`/`CommitStatusSummary`,
`Branch`/`RenamedBranch`, `Secret`/`ActionVariable`,
`ActionRun`/`ActionRunAttempt`). `repo_transfer` goes through the same
cascade rather than `DeleteRepositoryTransfer` so teardown stays one
mechanism; the existing reaper remains for the transfer flows.
The test inserts one row per previously-orphaned table (the fixture
files ask test cases to prepare their own data), deletes the repository,
and asserts each table is purged. Without the fix it fails on all seven
tables.
---------
Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Signed-off-by: Luc <luuuc@users.noreply.github.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Users should know what they are doing, don't check everything,
especially for that we don't understand.
Fixes#23840
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Rename functions "util.Remove" (remove.go) to "util.RemoveWithRetry"
(file_retry.go) and add comments to clarify their behaviors, also add
tests.
Refactor callers: when no concurrent access (cmd cli, migration, app
init, test), use "os.Xxx" directly.
More details are in `modules/util/file_retry.go`
By the way, clean up OS (windows) detection, make FileURLToPath test
always run
## Changes
### 1. Handle reusable workflow expansion failures
If a reusable workflow caller job is invalid (e.g. uses a workflow with
syntax error), the job emitter should mark it as failed instead of
retrying.
Related:
https://github.com/go-gitea/gitea/pull/38518#discussion_r3608882095
### 2. No longer process concurrent run inline
**Before**: If a run(R1)'s status change unlocks another blocked run(R2)
via concurrency group, the job emitter will process R2 in R1's
transaction.
**Current**: No longer process R2 inline and emit the ID of R2 to let
another pass process it.
This PR adds support for file exclusion into `.gitea/template`
Docs: https://gitea.com/gitea/docs/pulls/470
### Usage
Template owners add a `.gitea/template` file to their template
repository:
```
# Files to include for variable expansion
*.go
text/*.txt
**/modules/*
# Files/directories to exclude from the generated repo
[exclude]
vendor/*
node_modules/
src/build/
```
Resolves#37202
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
by the way, remove some unnecessary "models" imports from model
migration package, fix migration test model init bug
(modelmigration/migrationtest/tests.go)
The "path" details should be hidden to other packages
By the way, fix a resource leaking in gogit's CommitNodeIndex
(the file was not closed in CacheCommit)
before: gitrepo vs git packages
after: git package fully handle all git operations
by the way, use `WithRepo(repo)` instead of `WithDir(repo.Path)` to hide
path details.
benefits:
1. remove all unnecessary wrappers, developers no need to struggle with
"which package should be used"
2. simplify code, RepositoryFacade can (will) be used everywhere, all
"path" details are (will be) hidden
Migrations should never use model structs directly, because the model
structs can be different in different releases. e.g. if one migration uses
"User" model, it works in the early releases, then one day, when the
User model changes, the migration breaks because it will use the
new (incorrect) User model, it should only use the old User model.
The same to "modules/structs".
---------
Signed-off-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: delvh <dev.lh@web.de>
Removed `gitrepo.RunCmd*` functions, because gitcmd.Command works with
Repository directly.
Move some "local filesystem" related function into "localfs.go"
Blob.Size requires a context after the repository context removal
(5b078f72aa).
Actually the template code should just render, it should not depend on
the fragile dynamic calls to backend functions.
---------
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
A `workflow_dispatch` job that has `needs:` **and** an `if:` comparing a
boolean
input against a boolean literal never runs — it stays `Blocked` forever
even
though its needs succeed. Minimal repro:
```yaml
on:
workflow_dispatch:
inputs:
deploy:
type: boolean
default: true
jobs:
build:
runs-on: ubuntu-latest
steps: [{ run: echo build }]
deploy:
needs: build
if: ${{ inputs.deploy == true }} # never true on Gitea
runs-on: ubuntu-latest
steps: [{ run: echo deploy }]
```
On GitHub this runs; on Gitea `deploy` is stuck. Jobs **without**
`needs` are
unaffected, which makes it look like a matrix/`needs` bug — it isn't.
## Root cause
`workflow_dispatch` stores boolean inputs as the strings
`"true"`/`"false"`
(`ctx.FormBool` → `strconv.FormatBool` in the web path, plain strings in
the API
path). Since #37478 (shipped in **v1.27.0**), `evaluateJobIf` runs
**server-side**
as part of the job-emitter resolver passes. For a `needs`-gated job the
server
therefore evaluates `inputs.deploy == true` with `inputs.deploy` being
the string
`"true"`; comparing a string to a boolean coerces to `NaN == 1` →
`false`, so the
job is never dispatched.
Jobs without `needs` skip this server-side gate and are evaluated by the
runner,
which coerces the string to a real bool — that's why they keep working,
and why
the same input yields opposite results in the two paths.
## Fix
Normalize `type: boolean` dispatch inputs to native JSON booleans in the
shared
`DispatchActionWorkflow` path, so it covers both the web and API entry
points in
one place. The coerced value flows into both the run's `EventPayload`
(read back
by the server-side `if` evaluation and by the runner) and the
creation-time
parse, so all consumers agree.
This matches GitHub, whose `inputs` context "preserves Boolean values as
Booleans
instead of converting them to strings", and mirrors the native JSON
types Gitea
already sends for `workflow_call`. Only booleans are coerced; other
input types
are left untouched, and a value that is already a bool (JSON API
request) passes
through.
Note: this makes dispatch payloads carry native booleans, which the
runner
consumes correctly as of gitea/runner#1087 (it accepts a native bool and
keeps
the string fallback) — the same expectation `workflow_call` already
relies on.
Fixes https://github.com/go-gitea/gitea/issues/38466
### Description
This PR addresses orphaned user-keyed tables during user deletion by
adding the missing models to the `db.DeleteBeans` call in
`services/user/delete.go`:
- `auth_model.TwoFactor` (TOTP secrets)
- `auth_model.WebAuthnCredential` (WebAuthn keys)
- `activities_model.Notification` (Notifications)
- `issues_model.IssueWatch` (Issue watches)
Additionally, it adds corresponding unit test coverage in
`services/user/user_test.go` to assert that these records are cleaned up
successfully.
### Related Issues
Fixes#38510
Signed-off-by: pranav718 <raypranav718@gmail.com>
1. use var names "reqOwnerName, reqRepoName", because these values are
from request and should not be really used for path construction
2. simplify enable-pprof logic, don't "log.Fatal"
3. don't make runServe command to guess the repo storage path, instead,
let server return RepoStoragePath
4. don't process lfs verbs when the repo is a wiki
5. construct the request URI path correctly for the lfs transfer backend
(moved to the caller)
6. don't call "owner, err := user_model.GetUserByName", the "owner
rename redirection" has been done before
7. fix incorrect "repo.OwnerName = ownerName", the real owner might have
been "redirected"
8. fix incorrect "inactive owner" check, it should be checked even if
the repo is redirected
Use general error functions to handle responses, error handling code is
hugely simplified.
Updating a branch by merge produced an unsigned commit even when merges
are configured to be signed. Update by rebase was unaffected.
`Update()` builds a fake reverse PR to switch head and base, and it has
no `Index`, so `pr.GetGitHeadRefName()` resolves `refs/pull/0/head`.
Since #36186 `SignMerge` looks that ref up in the base repository
instead of the temporary merge repo. That lookup fails. The caller
dropped the error, so `sign` stayed false.
Sync fork goes through the same fake-PR path.
Pass both sides of the merge to `SignMerge` as refs and evaluate them in
the temp repo, where `base` and `tracking` always exist.
Tests cover a signed and an unsigned update by merge.
Fixes#38066
Follow-ups from https://github.com/go-gitea/gitea/pull/37038:
- Render the OpenAPI 3.0 spec in the API viewer, it is richer than the
Swagger 2.0 rendering
- Replace the back link with gitea-styled buttons to view both specs and
return to Gitea, flowing above swagger-ui on viewports where they would
overlap the title
- Key `VisibilityModes` by the string enum type, now named
`VisibilityString`, removing the `string()` casts at API call sites
- Drop the raw visibility strings from the `Service` settings struct in
favor of the typed mode fields, which also removes the org default
visibility row from the admin config page
- Fix two pre-existing issues surfaced in review: an invalid
`DEFAULT_USER_VISIBILITY` was silently accepted as public, and the org
visibility error message showed the enum zero value instead of the
submitted input
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
## How to reproduce this bug
1. Create a user with `visibility=public`
2. Update `ALLOWED_USER_VISIBILITY_MODES` setting to `limited, private`
3. Modify any field other than "User visibility" (e.g. "Full Name")
4. UI shows 500 error
## Fix
Only update the visibility field when it actually changes.
1. Storing "ctx" in a long-living object is wrong
2. Make the commit & tree cacheable (for the future performance
optimization)
3. Also fix some bad designs like `// FIXME: bad design, this field can
be nil if the commit is from "last commit cache"`
ref:
* #33893
Scheduled runs stored a `null` event payload, so
`github.event.repository`/`sender`/`organization` were empty in
scheduled workflows. Every other event carries them, and so does GitHub
Actions. `CreateScheduleTask` now synthesizes them.
Addresses a batch of privately reported security issues, grouped by
area:
- **SSRF** - migration PR-patch/asset fetches, OAuth2 avatar & OpenID
discovery, pull-mirror URL re-validation, and the outbound proxy path.
- **Access-token scope** - prevent scope escalation on token creation;
keep public-only tokens confined (feeds, packages, Actions listings,
star/watch lists, limited/private owners).
- **Access control / disclosure** - go-get default-branch leak, webhook
authorization-header leak, watch clearing on private transitions,
label/attachment scoping.
- **Denial of service** - input bounds for npm dist-tags, Debian control
files, Arch file lists, and SSH keys.
### 📌 Attention for site admins
Not breaking - existing configs keep working - but two changes are worth
a look:
- **New SSRF protection** Outbound requests (migrations, OAuth2 avatars,
OpenID discovery, pull mirrors, proxy path) are now validated against
the allow/block host lists. If your instance legitimately reaches
internal hosts, you may need to add them to
`[security].ALLOWED_HOST_LIST` (and the relevant `ALLOW_LOCALNETWORKS`
settings).
- **Deprecation** `[webhook].ALLOWED_HOST_LIST` is deprecated and will
be removed in a future release. Use `[security].ALLOWED_HOST_LIST`
instead; the old key still works for now.
---------
Co-authored-by: TheFox0x7 <thefox0x7@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Zettat123 <zettat123@gmail.com>
The `official` flag of a pull request review is computed against the
target branch's protection rules at submit time and stored on the review
record. `ChangeTargetBranch` updated the base branch but never
re-evaluated it, so an `official` approval obtained against an
unprotected branch could be retargeted onto a protected branch and
satisfy its required approvals, bypassing the branch protection.
This re-evaluates the `official` flag of the latest approve/reject
reviews against the new base branch whenever the target branch changes.
The LFS batch and upload handlers linked an object that already existed
in the content store but was not linked to the current repo whenever the
token's user could access it in another repo. Deploy-key tokens carry
the repo owner's identity, so a single-repo write deploy key could link
and then download objects from any repo the owner can see.
This drops the cross-repo access check: the batch handler now makes the
client upload (hash-verified) any object not yet linked to the repo, and
the upload handler skips proof of possession only when the object is
already linked to the current repo.
Follow #38237#38237 posts "skipped" commit statuses for every workflow that is not
triggered due to a filter (e.g. `paths` or `branches`) mismatch.
However, for non-required workflows, creating "skipped" commit statuses
for them would generate a lot of noise.
To address this issue, this PR adds a check before creating commit
status:
- For the context that matches any required status check patterns, a
"skipped" commit status will be created. The `Required` label can inform
users that this status check is required, but has been skipped because
of a filter mismatch.
- For a non-required context, nothing will be created.
NOTE: Reducing noise is a best-effort approach and isn't entirely
accurate. When creating commit statuses, it is impossible to predict
which branch protection rule will take effect. Therefore, we have to
compare the commit status context against the required patterns from all
rules. If any rule matches, the context is considered "required".
3 reductions in the DB load generated by many runners polling
`FetchTask`:
**1. Debounce runner heartbeat writes**
Every poll wrote `last_online`, and every `UpdateTask`/`UpdateLog` wrote
`last_active` — while a runner streams logs that is many writes per
second per runner. These are now persisted only when stale enough to
actually affect the active/offline status (`ShouldPersistLastOnline` /
`ShouldPersistLastActive`), using the existing columns.
**2. Throttle concurrent task picks**
A new in-process semaphore (`MAX_CONCURRENT_TASK_PICKS`) bounds how many
runners run the task-assignment transaction at once, so a fleet polling
together cannot stampede the query. Throttled polls retry on their next
poll without advancing the runner's tasks version.
**3. Paginate the task-pick query**
`CreateTaskForRunner` previously loaded every waiting job in the
runner's scope into memory on each poll (no `LIMIT`). Now it pages
through the waiting backlog oldest-first with `LIMIT`, claiming the
first label-matching job.
---------
Co-authored-by: Zettat123 <zettat123@gmail.com>
Pascal Zimmermann