fix(indexer): search code as case-insensitive literal substrings (#40028)

Code search now matches case-insensitive literal substrings like GitHub,
in every script. Word tokenizers can't do this: the ES pattern dropped
every non-ASCII word, and the `standard` tokenizer keeps `console.log`
as one token.

- Elasticsearch indexes positional trigrams, plus 1-2 grams for short
terms
- Bleve indexes distinct 1-3 grams as a candidate filter and verifies
candidates against the content before paging, because phrase queries
over trigrams are quadratic in repeated grams
- Snippet offsets come from the original content, so `<em>` in sources
and width-changing lowercase no longer misplace them
- `search_mode=fuzzy` requests behave like Words, the UI never offered
fuzzy for code search
- The ES code indexer test now runs in CI, it was skipped before

There are no UI changes. Both code indexes rebuild after upgrade.

Fixes: https://github.com/go-gitea/gitea/issues/40027
Fixes: https://github.com/go-gitea/gitea/issues/16158
Fixes: https://github.com/go-gitea/gitea/issues/21359

---------

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
This commit is contained in:
authored and GitHub committed 2026-10-28 20:38:45 +08:00
1 parent 02c03d69d3
commit 1ebeade2c1
41 files changed
+330 -317

No files matched your search

-1
View File
@@ -215,7 +215,6 @@ jobs:
run: GITEA_TEST_DATABASE=mysql make test-integration
env:
TAGS: bindata
TEST_INDEXER_CODE_ES_URL: "http://elastic:changeme@elasticsearch:9200"
test-mssql:
if: needs.files-changed.outputs.backend == 'true'