Compare commits

...

21 Commits

Author SHA1 Message Date
dependabot[bot]
ec26888487 Bump github.com/labstack/echo-contrib from 0.17.4 to 0.18.0 (#667)
Bumps [github.com/labstack/echo-contrib](https://github.com/labstack/echo-contrib) from 0.17.4 to 0.18.0.
- [Release notes](https://github.com/labstack/echo-contrib/releases)
- [Commits](https://github.com/labstack/echo-contrib/compare/v0.17.4...v0.18.0)

---
updated-dependencies:
- dependency-name: github.com/labstack/echo-contrib
  dependency-version: 0.18.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-13 11:36:13 +08:00
dependabot[bot]
57d76151fd Bump golang.org/x/crypto from 0.48.0 to 0.49.0 (#669)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.48.0 to 0.49.0.
- [Commits](https://github.com/golang/crypto/compare/v0.48.0...v0.49.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-version: 0.49.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-13 11:33:38 +08:00
dependabot[bot]
c2ee390841 Bump github.com/go-webauthn/webauthn from 0.16.0 to 0.16.1 (#671)
Bumps [github.com/go-webauthn/webauthn](https://github.com/go-webauthn/webauthn) from 0.16.0 to 0.16.1.
- [Release notes](https://github.com/go-webauthn/webauthn/releases)
- [Commits](https://github.com/go-webauthn/webauthn/compare/v0.16.0...v0.16.1)

---
updated-dependencies:
- dependency-name: github.com/go-webauthn/webauthn
  dependency-version: 0.16.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-13 11:33:26 +08:00
dependabot[bot]
d6fc346e70 Bump @codemirror/view from 6.39.16 to 6.40.0 (#670)
Bumps [@codemirror/view](https://github.com/codemirror/view) from 6.39.16 to 6.40.0.
- [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/view/compare/6.39.16...6.40.0)

---
updated-dependencies:
- dependency-name: "@codemirror/view"
  dependency-version: 6.40.0
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-13 11:33:09 +08:00
dependabot[bot]
4e977077ba Bump nodemon from 3.1.11 to 3.1.14 (#672)
Bumps [nodemon](https://github.com/remy/nodemon) from 3.1.11 to 3.1.14.
- [Release notes](https://github.com/remy/nodemon/releases)
- [Commits](https://github.com/remy/nodemon/compare/v3.1.11...v3.1.14)

---
updated-dependencies:
- dependency-name: nodemon
  dependency-version: 3.1.14
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-13 11:32:51 +08:00
dependabot[bot]
f865b2b099 Bump @codemirror/commands from 6.10.1 to 6.10.3 (#664)
Bumps [@codemirror/commands](https://github.com/codemirror/commands) from 6.10.1 to 6.10.3.
- [Changelog](https://github.com/codemirror/commands/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/commands/compare/6.10.1...6.10.3)

---
updated-dependencies:
- dependency-name: "@codemirror/commands"
  dependency-version: 6.10.3
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-13 11:32:24 +08:00
Thomas Miceli
d26221de54 Translated using Weblate (Ukrainian) (#659)
Currently translated at 69.4% (243 of 350 strings)

Translation: Opengist/Opengist
Translate-URL: http://tr.opengist.io/projects/_/opengist/uk/

Co-authored-by: Anonymous <noreply@weblate.org>
2026-03-13 11:17:23 +08:00
Thomas Miceli
e91139d3ec Improve code search + tests (#663)
Signed-off-by: Thomas Miceli <tho.miceli@gmail.com>
Co-authored-by: Qiang Zhou <zhouqiang.loaded@bytedance.com>
Co-authored-by: theodoruszq <theodoruszq@gmail.com>
2026-03-13 11:16:10 +08:00
Webysther Sperandio
279da52899 feat: search all fields (#622)
*  feat(search): search all feature

- add Description field to Gist struct and index it
- extend SearchGistMetadata with Description and Content
- update Bleve and Meilisearch to index and search Description
- modify ParseSearchQueryStr to parse description: and content: keywords
- update templates and i18n for new search options

* Fix test

* Set content by default

Signed-off-by: Thomas Miceli <tho.miceli@gmail.com>

* Config to define default searchable fields

Signed-off-by: Thomas Miceli <tho.miceli@gmail.com>

---------

Signed-off-by: Thomas Miceli <tho.miceli@gmail.com>
Co-authored-by: Thomas Miceli <tho.miceli@gmail.com>
2026-03-12 00:55:23 +08:00
dependabot[bot]
5ad01a3304 Bump @codemirror/view from 6.39.11 to 6.39.16 (#653)
Bumps [@codemirror/view](https://github.com/codemirror/view) from 6.39.11 to 6.39.16.
- [Changelog](https://github.com/codemirror/view/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codemirror/view/compare/6.39.11...6.39.16)

---
updated-dependencies:
- dependency-name: "@codemirror/view"
  dependency-version: 6.39.16
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-12 00:52:53 +08:00
dependabot[bot]
1944502d14 Bump marked from 17.0.3 to 17.0.4 (#652)
Bumps [marked](https://github.com/markedjs/marked) from 17.0.3 to 17.0.4.
- [Release notes](https://github.com/markedjs/marked/releases)
- [Commits](https://github.com/markedjs/marked/compare/v17.0.3...v17.0.4)

---
updated-dependencies:
- dependency-name: marked
  dependency-version: 17.0.4
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-12 00:52:39 +08:00
dependabot[bot]
2d7261ac83 Bump docker/setup-qemu-action from 3 to 4 (#654)
Bumps [docker/setup-qemu-action](https://github.com/docker/setup-qemu-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-qemu-action/releases)
- [Commits](https://github.com/docker/setup-qemu-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-qemu-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-12 00:52:12 +08:00
dependabot[bot]
50f2980c10 Bump docker/login-action from 3 to 4 (#655)
Bumps [docker/login-action](https://github.com/docker/login-action) from 3 to 4.
- [Release notes](https://github.com/docker/login-action/releases)
- [Commits](https://github.com/docker/login-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/login-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-12 00:52:03 +08:00
dependabot[bot]
2e68b6893b Bump docker/build-push-action from 6 to 7 (#656)
Bumps [docker/build-push-action](https://github.com/docker/build-push-action) from 6 to 7.
- [Release notes](https://github.com/docker/build-push-action/releases)
- [Commits](https://github.com/docker/build-push-action/compare/v6...v7)

---
updated-dependencies:
- dependency-name: docker/build-push-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-12 00:51:52 +08:00
dependabot[bot]
9b68f08c62 Bump docker/metadata-action from 5 to 6 (#657)
Bumps [docker/metadata-action](https://github.com/docker/metadata-action) from 5 to 6.
- [Release notes](https://github.com/docker/metadata-action/releases)
- [Commits](https://github.com/docker/metadata-action/compare/v5...v6)

---
updated-dependencies:
- dependency-name: docker/metadata-action
  dependency-version: '6'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-12 00:51:21 +08:00
dependabot[bot]
dfabdb403a Bump docker/setup-buildx-action from 3 to 4 (#658)
Bumps [docker/setup-buildx-action](https://github.com/docker/setup-buildx-action) from 3 to 4.
- [Release notes](https://github.com/docker/setup-buildx-action/releases)
- [Commits](https://github.com/docker/setup-buildx-action/compare/v3...v4)

---
updated-dependencies:
- dependency-name: docker/setup-buildx-action
  dependency-version: '4'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-03-12 00:51:09 +08:00
Thomas Miceli
4da067ab60 Rebuild search index in admin options (#661)
Signed-off-by: Thomas Miceli <tho.miceli@gmail.com>
2026-03-12 00:50:44 +08:00
dependabot[bot]
a8339ff6bd Bump github-markdown-css from 5.8.1 to 5.9.0 (#651) 2026-03-11 23:07:34 +07:00
dependabot[bot]
7a5cdd1565 Bump @codemirror/lang-javascript from 6.2.4 to 6.2.5 (#650) 2026-03-11 23:07:02 +07:00
dependabot[bot]
00dcb53e3a Bump katex from 0.16.33 to 0.16.38 (#649) 2026-03-11 23:06:40 +07:00
Thomas Miceli
f8b3bbce6a Rebuild search index in admin options (#647)
Signed-off-by: Thomas Miceli <tho.miceli@gmail.com>
2026-03-09 07:30:28 +08:00
36 changed files with 1453 additions and 2298 deletions

View File

@@ -83,6 +83,18 @@ jobs:
--health-interval 10s --health-interval 10s
--health-timeout 5s --health-timeout 5s
--health-retries 5 --health-retries 5
meilisearch:
image: getmeili/meilisearch:latest
ports:
- 47700:7700
env:
MEILI_NO_ANALYTICS: true
MEILI_ENV: development
options: >-
--health-cmd "curl -sf http://localhost:7700/health"
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v6 uses: actions/checkout@v6
@@ -94,6 +106,8 @@ jobs:
- name: Run tests - name: Run tests
run: make test TEST_DB_TYPE=${{ matrix.database }} run: make test TEST_DB_TYPE=${{ matrix.database }}
env:
OG_TEST_MEILI_HOST: http://localhost:47700
test: test:
name: Test name: Test

View File

@@ -42,7 +42,7 @@ jobs:
- name: Docker meta - name: Docker meta
id: meta id: meta
uses: docker/metadata-action@v5 uses: docker/metadata-action@v6
with: with:
images: | images: |
ghcr.io/thomiceli/opengist ghcr.io/thomiceli/opengist
@@ -54,26 +54,26 @@ jobs:
type=semver,pattern={{version}} type=semver,pattern={{version}}
- name: Set up QEMU - name: Set up QEMU
uses: docker/setup-qemu-action@v3 uses: docker/setup-qemu-action@v4
- name: Set up Docker Buildx - name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v4
- name: Login to GitHub Container Registry - name: Login to GitHub Container Registry
uses: docker/login-action@v3 uses: docker/login-action@v4
with: with:
registry: ghcr.io registry: ghcr.io
username: ${{ github.repository_owner }} username: ${{ github.repository_owner }}
password: ${{ secrets.GITHUB_TOKEN }} password: ${{ secrets.GITHUB_TOKEN }}
- name: Login to Docker Hub - name: Login to Docker Hub
uses: docker/login-action@v3 uses: docker/login-action@v4
with: with:
username: ${{ secrets.DOCKERHUB_USERNAME }} username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_TOKEN }} password: ${{ secrets.DOCKERHUB_TOKEN }}
- name: Build and push - name: Build and push
uses: docker/build-push-action@v6 uses: docker/build-push-action@v7
with: with:
context: . context: .
platforms: linux/amd64,linux/arm64 platforms: linux/amd64,linux/arm64

View File

@@ -32,6 +32,10 @@ index.meili.host:
# Set the API key for the Meiliseach server # Set the API key for the Meiliseach server
index.meili.api-key: index.meili.api-key:
# Set the default search fields. Can contain multiple fields (e.g., `content,username`).
# Fields: content,user,title,description,filename,extension,language,topic. Default: content
search.default: content
# Default branch name used by Opengist when initializing Git repositories. # Default branch name used by Opengist when initializing Git repositories.
# If not set, uses the Git default branch name. See https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup#_new_default_branch # If not set, uses the Git default branch name. See https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup#_new_default_branch
git.default-branch: git.default-branch:

View File

@@ -15,6 +15,7 @@ aside: false
| index | OG_INDEX | `bleve` | Define the code indexer (either `bleve`, `meilisearch`, or empty for no index). | | index | OG_INDEX | `bleve` | Define the code indexer (either `bleve`, `meilisearch`, or empty for no index). |
| index.meili.host | OG_MEILI_HOST | none | Set the host for the Meiliseach server. | | index.meili.host | OG_MEILI_HOST | none | Set the host for the Meiliseach server. |
| index.meili.api-key | OG_MEILI_API_KEY | none | Set the API key for the Meiliseach server. | | index.meili.api-key | OG_MEILI_API_KEY | none | Set the API key for the Meiliseach server. |
| search.default | OG_SEARCH_DEFAULT | `content` | Set the default search fields. Can contain multiple fields (e.g., `content,username`). Fields: `content,user,title,description,filename,extension,language,topic`. |
| git.default-branch | OG_GIT_DEFAULT_BRANCH | none | Default branch name used by Opengist when initializing Git repositories. If not set, uses the Git default branch name. More info [here](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup#_new_default_branch) | | git.default-branch | OG_GIT_DEFAULT_BRANCH | none | Default branch name used by Opengist when initializing Git repositories. If not set, uses the Git default branch name. More info [here](https://git-scm.com/book/en/v2/Getting-Started-First-Time-Git-Setup#_new_default_branch) |
| sqlite.journal-mode | OG_SQLITE_JOURNAL_MODE | `WAL` | Set the journal mode for SQLite. More info [here](https://www.sqlite.org/pragma.html#pragma_journal_mode) | | sqlite.journal-mode | OG_SQLITE_JOURNAL_MODE | `WAL` | Set the journal mode for SQLite. More info [here](https://www.sqlite.org/pragma.html#pragma_journal_mode) |
| http.host | OG_HTTP_HOST | `0.0.0.0` | The host on which the HTTP server should bind. Use an IP address for network binding. Use a path for Unix socket binding (e.g. /run/opengist.sock) | | http.host | OG_HTTP_HOST | `0.0.0.0` | The host on which the HTTP server should bind. Use an IP address for network binding. Use a path for Unix socket binding (e.g. /run/opengist.sock) |

16
go.mod
View File

@@ -11,12 +11,12 @@ require (
github.com/glebarez/sqlite v1.11.0 github.com/glebarez/sqlite v1.11.0
github.com/go-ldap/ldap/v3 v3.4.12 github.com/go-ldap/ldap/v3 v3.4.12
github.com/go-playground/validator/v10 v10.30.1 github.com/go-playground/validator/v10 v10.30.1
github.com/go-webauthn/webauthn v0.16.0 github.com/go-webauthn/webauthn v0.16.1
github.com/google/uuid v1.6.0 github.com/google/uuid v1.6.0
github.com/gorilla/schema v1.4.1 github.com/gorilla/schema v1.4.1
github.com/gorilla/securecookie v1.1.2 github.com/gorilla/securecookie v1.1.2
github.com/gorilla/sessions v1.4.0 github.com/gorilla/sessions v1.4.0
github.com/labstack/echo-contrib v0.17.4 github.com/labstack/echo-contrib v0.18.0
github.com/labstack/echo/v4 v4.15.1 github.com/labstack/echo/v4 v4.15.1
github.com/markbates/goth v1.82.0 github.com/markbates/goth v1.82.0
github.com/meilisearch/meilisearch-go v0.36.1 github.com/meilisearch/meilisearch-go v0.36.1
@@ -29,8 +29,8 @@ require (
github.com/yuin/goldmark-emoji v1.0.6 github.com/yuin/goldmark-emoji v1.0.6
github.com/yuin/goldmark-highlighting/v2 v2.0.0-20230729083705-37449abec8cc github.com/yuin/goldmark-highlighting/v2 v2.0.0-20230729083705-37449abec8cc
go.abhg.dev/goldmark/mermaid v0.6.0 go.abhg.dev/goldmark/mermaid v0.6.0
golang.org/x/crypto v0.48.0 golang.org/x/crypto v0.49.0
golang.org/x/text v0.34.0 golang.org/x/text v0.35.0
gopkg.in/yaml.v3 v3.0.1 gopkg.in/yaml.v3 v3.0.1
gorm.io/driver/mysql v1.6.0 gorm.io/driver/mysql v1.6.0
gorm.io/driver/postgres v1.6.0 gorm.io/driver/postgres v1.6.0
@@ -76,7 +76,7 @@ require (
github.com/go-playground/universal-translator v0.18.1 // indirect github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-sql-driver/mysql v1.9.3 // indirect github.com/go-sql-driver/mysql v1.9.3 // indirect
github.com/go-viper/mapstructure/v2 v2.5.0 // indirect github.com/go-viper/mapstructure/v2 v2.5.0 // indirect
github.com/go-webauthn/x v0.2.1 // indirect github.com/go-webauthn/x v0.2.2 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/golang/snappy v1.0.0 // indirect github.com/golang/snappy v1.0.0 // indirect
github.com/google/go-tpm v0.9.8 // indirect github.com/google/go-tpm v0.9.8 // indirect
@@ -112,10 +112,10 @@ require (
go.etcd.io/bbolt v1.4.3 // indirect go.etcd.io/bbolt v1.4.3 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect golang.org/x/exp v0.0.0-20260112195511-716be5621a96 // indirect
golang.org/x/net v0.49.0 // indirect golang.org/x/net v0.51.0 // indirect
golang.org/x/oauth2 v0.34.0 // indirect golang.org/x/oauth2 v0.34.0 // indirect
golang.org/x/sync v0.19.0 // indirect golang.org/x/sync v0.20.0 // indirect
golang.org/x/sys v0.41.0 // indirect golang.org/x/sys v0.42.0 // indirect
golang.org/x/time v0.14.0 // indirect golang.org/x/time v0.14.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect google.golang.org/protobuf v1.36.11 // indirect
modernc.org/libc v1.67.7 // indirect modernc.org/libc v1.67.7 // indirect

44
go.sum
View File

@@ -114,10 +114,10 @@ github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU= github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro= github.com/go-viper/mapstructure/v2 v2.5.0 h1:vM5IJoUAy3d7zRSVtIwQgBj7BiWtMPfmPEgAXnvj1Ro=
github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM= github.com/go-viper/mapstructure/v2 v2.5.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/go-webauthn/webauthn v0.16.0 h1:A9BkfYIwWAMPSQCbM2HoWqo6JO5LFI8aqYAzo6nW7AY= github.com/go-webauthn/webauthn v0.16.1 h1:x5/SSki5/aIfogaRukqvbg/RXa3Sgxy/9vU7UfFPHKU=
github.com/go-webauthn/webauthn v0.16.0/go.mod h1:hm9RS/JNYeUu3KqGbzqlnHClhDGCZzTZlABjathwnN0= github.com/go-webauthn/webauthn v0.16.1/go.mod h1:RBS+rtQJMkE5VfMQ4diDA2VNrEL8OeUhp4Srz37FHbQ=
github.com/go-webauthn/x v0.2.1 h1:/oB8i0FhSANuoN+YJF5XHMtppa7zGEYaQrrf6ytotjc= github.com/go-webauthn/x v0.2.2 h1:zIiipvMbr48CXi5RG0XdBJR94kd8I5LfzHPb/q+YYmk=
github.com/go-webauthn/x v0.2.1/go.mod h1:Wm0X0zXkzznit4gHj4m82GiBZRMEm+TDUIoJWIQLsE4= github.com/go-webauthn/x v0.2.2/go.mod h1:IpJ5qyWB9NRhLX3C7gIfjTU7RZLXEP6kzFkoVSE7Fz4=
github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU= github.com/gobwas/httphead v0.1.0 h1:exrUm0f4YX0L7EBwZHuCF4GDp8aJfVeBrlLQrs6NqWU=
github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM= github.com/gobwas/httphead v0.1.0/go.mod h1:O/RXo79gxV8G+RqlR/otEwx4Q36zl9rqC5u12GKvMCM=
github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og= github.com/gobwas/pool v0.2.1 h1:xfeeEhW7pwmX8nuLVlqbzVc7udMDrwetjEv+TZIz1og=
@@ -190,8 +190,8 @@ github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/labstack/echo-contrib v0.17.4 h1:g5mfsrJfJTKv+F5uNKCyrjLK7js+ZW6HTjg4FnDxxgk= github.com/labstack/echo-contrib v0.18.0 h1:mgn4sdnEUUxH7E3bDGplIKkZ+syDZvafbIhuf7hPs3I=
github.com/labstack/echo-contrib v0.17.4/go.mod h1:9O7ZPAHUeMGTOAfg80YqQduHzt0CzLak36PZRldYrZ0= github.com/labstack/echo-contrib v0.18.0/go.mod h1:8r/++U/Fw/QniApFnzunLanKaviPfBX7fX7/2QX0qOk=
github.com/labstack/echo/v4 v4.15.1 h1:S9keusg26gZpjMmPqB5hOEvNKnmd1lNmcHrbbH2lnFs= github.com/labstack/echo/v4 v4.15.1 h1:S9keusg26gZpjMmPqB5hOEvNKnmd1lNmcHrbbH2lnFs=
github.com/labstack/echo/v4 v4.15.1/go.mod h1:xmw1clThob0BSVRX1CRQkGQ/vjwcpOMjQZSZa9fKA/c= github.com/labstack/echo/v4 v4.15.1/go.mod h1:xmw1clThob0BSVRX1CRQkGQ/vjwcpOMjQZSZa9fKA/c=
github.com/labstack/gommon v0.4.2 h1:F8qTUNXgG1+6WQmqoUWnz8WiEU60mXVVw0P4ht1WRA0= github.com/labstack/gommon v0.4.2 h1:F8qTUNXgG1+6WQmqoUWnz8WiEU60mXVVw0P4ht1WRA0=
@@ -278,31 +278,31 @@ go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU= go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0= go.yaml.in/yaml/v2 v2.4.3 h1:6gvOSjQoTB3vt1l+CU+tSyi/HOjfOjRLJ4YwYZGwRO0=
go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8= go.yaml.in/yaml/v2 v2.4.3/go.mod h1:zSxWcmIDjOzPXpjlTTbAsKokqkDNAVtZO0WOMiT90s8=
golang.org/x/crypto v0.48.0 h1:/VRzVqiRSggnhY7gNRxPauEQ5Drw9haKdM0jqfcCFts= golang.org/x/crypto v0.49.0 h1:+Ng2ULVvLHnJ/ZFEq4KdcDd/cfjrrjjNSXNzxg0Y4U4=
golang.org/x/crypto v0.48.0/go.mod h1:r0kV5h3qnFPlQnBSrULhlsRfryS2pmewsg+XfMgkVos= golang.org/x/crypto v0.49.0/go.mod h1:ErX4dUh2UM+CFYiXZRTcMpEcN8b/1gxEuv3nODoYtCA=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU= golang.org/x/exp v0.0.0-20260112195511-716be5621a96 h1:Z/6YuSHTLOHfNFdb8zVZomZr7cqNgTJvA8+Qz75D8gU=
golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU= golang.org/x/exp v0.0.0-20260112195511-716be5621a96/go.mod h1:nzimsREAkjBCIEFtHiYkrJyT+2uy9YZJB7H1k68CXZU=
golang.org/x/mod v0.32.0 h1:9F4d3PHLljb6x//jOyokMv3eX+YDeepZSEo3mFJy93c= golang.org/x/mod v0.33.0 h1:tHFzIWbBifEmbwtGz65eaWyGiGZatSrT9prnU8DbVL8=
golang.org/x/mod v0.32.0/go.mod h1:SgipZ/3h2Ci89DlEtEXWUk/HteuRin+HHhN+WbNhguU= golang.org/x/mod v0.33.0/go.mod h1:swjeQEj+6r7fODbD2cqrnje9PnziFuw4bmLbBZFrQ5w=
golang.org/x/net v0.49.0 h1:eeHFmOGUTtaaPSGNmjBKpbng9MulQsJURQUAfUwY++o= golang.org/x/net v0.51.0 h1:94R/GTO7mt3/4wIKpcR5gkGmRLOuE/2hNGeWq/GBIFo=
golang.org/x/net v0.49.0/go.mod h1:/ysNB2EvaqvesRkuLAyjI1ycPZlQHM3q01F02UY/MV8= golang.org/x/net v0.51.0/go.mod h1:aamm+2QF5ogm02fjy5Bb7CQ0WMt1/WVM7FtyaTLlA9Y=
golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw= golang.org/x/oauth2 v0.34.0 h1:hqK/t4AKgbqWkdkcAeI8XLmbK+4m4G5YeQRrmiotGlw=
golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA= golang.org/x/oauth2 v0.34.0/go.mod h1:lzm5WQJQwKZ3nwavOZ3IS5Aulzxi68dUSgRHujetwEA=
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4= golang.org/x/sync v0.20.0 h1:e0PTpb7pjO8GAtTs2dQ6jYa5BWYlMuX047Dco/pItO4=
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI= golang.org/x/sync v0.20.0/go.mod h1:9xrNwdLfx4jkKbNva9FpL6vEN7evnE43NNNJQ2LF3+0=
golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.0.0-20220811171246-fbc7d0a398ab/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.41.0 h1:Ivj+2Cp/ylzLiEU89QhWblYnOE9zerudt9Ftecq2C6k= golang.org/x/sys v0.42.0 h1:omrd2nAlyT5ESRdCLYdm3+fMfNFE/+Rf4bDIQImRJeo=
golang.org/x/sys v0.41.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks= golang.org/x/sys v0.42.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.40.0 h1:36e4zGLqU4yhjlmxEaagx2KuYbJq3EwY8K943ZsHcvg= golang.org/x/term v0.41.0 h1:QCgPso/Q3RTJx2Th4bDLqML4W6iJiaXFq2/ftQF13YU=
golang.org/x/term v0.40.0/go.mod h1:w2P8uVp06p2iyKKuvXIm7N/y0UCRt3UfJTfZ7oOpglM= golang.org/x/term v0.41.0/go.mod h1:3pfBgksrReYfZ5lvYM0kSO0LIkAl4Yl2bXOkKP7Ec2A=
golang.org/x/text v0.34.0 h1:oL/Qq0Kdaqxa1KbNeMKwQq0reLCCaFtqu2eNuSeNHbk= golang.org/x/text v0.35.0 h1:JOVx6vVDFokkpaq1AEptVzLTpDe9KGpj5tR4/X+ybL8=
golang.org/x/text v0.34.0/go.mod h1:homfLqTYRFyVYemLBFl5GgL/DWEiH5wcsQ5gSh1yziA= golang.org/x/text v0.35.0/go.mod h1:khi/HExzZJ2pGnjenulevKNX1W67CUy0AsXcNubPGCA=
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI= golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4= golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
golang.org/x/tools v0.41.0 h1:a9b8iMweWG+S0OBnlU36rzLp20z1Rp10w+IY2czHTQc= golang.org/x/tools v0.42.0 h1:uNgphsn75Tdz5Ji2q36v/nsFSfR/9BRFvqhGBaJGd5k=
golang.org/x/tools v0.41.0/go.mod h1:XSY6eDqxVNiYgezAVqqCeihT4j1U2CCsqvH3WhQpnlg= golang.org/x/tools v0.42.0/go.mod h1:Ma6lCIwGZvHK6XtgbswSoWroEkhugApmsXyrUmBhfr0=
google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE= google.golang.org/protobuf v1.36.11 h1:fV6ZwhNocDyBLK0dj+fg8ektcVegBBuEolpbTQyBNVE=
google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco= google.golang.org/protobuf v1.36.11/go.mod h1:HTf+CrKn2C3g5S8VImy6tdcUvCska2kB7j23XfzDpco=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=

View File

@@ -150,7 +150,12 @@ func resetHooks() {
} }
func indexGists() { func indexGists() {
log.Info().Msg("Indexing all Gists...") log.Info().Msg("Rebuilding index from scratch...")
if err := index.ResetIndex(); err != nil {
log.Error().Err(err).Msg("Cannot reset index")
return
}
gists, err := db.GetAllGistsRows() gists, err := db.GetAllGistsRows()
if err != nil { if err != nil {
log.Error().Err(err).Msg("Cannot get gists") log.Error().Err(err).Msg("Cannot get gists")

View File

@@ -43,6 +43,7 @@ type config struct {
BleveDirname string `yaml:"index.dirname" env:"OG_INDEX_DIRNAME"` // deprecated BleveDirname string `yaml:"index.dirname" env:"OG_INDEX_DIRNAME"` // deprecated
MeiliHost string `yaml:"index.meili.host" env:"OG_MEILI_HOST"` MeiliHost string `yaml:"index.meili.host" env:"OG_MEILI_HOST"`
MeiliAPIKey string `yaml:"index.meili.api-key" env:"OG_MEILI_API_KEY"` MeiliAPIKey string `yaml:"index.meili.api-key" env:"OG_MEILI_API_KEY"`
SearchDefault string `yaml:"search.default" env:"OG_SEARCH_DEFAULT"`
GitDefaultBranch string `yaml:"git.default-branch" env:"OG_GIT_DEFAULT_BRANCH"` GitDefaultBranch string `yaml:"git.default-branch" env:"OG_GIT_DEFAULT_BRANCH"`
@@ -111,6 +112,7 @@ func configWithDefaults() (*config, error) {
c.OpengistHome = "" c.OpengistHome = ""
c.DBUri = "opengist.db" c.DBUri = "opengist.db"
c.Index = "bleve" c.Index = "bleve"
c.SearchDefault = "content"
c.SqliteJournalMode = "WAL" c.SqliteJournalMode = "WAL"

View File

@@ -821,6 +821,7 @@ func (gist *Gist) ToIndexedGist() (*index.Gist, error) {
UserID: gist.UserID, UserID: gist.UserID,
Visibility: gist.Private.Uint(), Visibility: gist.Private.Uint(),
Username: gist.User.Username, Username: gist.User.Username,
Description: gist.Description,
Title: gist.Title, Title: gist.Title,
Content: wholeContent, Content: wholeContent,
Filenames: fileNames, Filenames: fileNames,

View File

@@ -192,7 +192,7 @@ admin.actions.sync-db: 'Gists von der Datenbank synchronisieren'
admin.actions.git-gc: '„garbage collection“ bei allen git Repositories ausführen' admin.actions.git-gc: '„garbage collection“ bei allen git Repositories ausführen'
admin.actions.sync-previews: 'Alle Gist Vorschauen synchronisieren' admin.actions.sync-previews: 'Alle Gist Vorschauen synchronisieren'
admin.actions.reset-hooks: 'Alle Git server Hooks für alle Repositories synchronisieren' admin.actions.reset-hooks: 'Alle Git server Hooks für alle Repositories synchronisieren'
admin.actions.index-gists: 'Alle Gists Indexieren' admin.actions.index-gists: 'Suchindex neu aufbauen'
admin.id: 'ID' admin.id: 'ID'
admin.user: 'Benutzer' admin.user: 'Benutzer'
admin.delete: 'Löschen' admin.delete: 'Löschen'
@@ -236,7 +236,7 @@ flash.admin.sync-db: 'Synchronisiere Repositories aus der Datenbank...'
flash.admin.git-gc: 'Sammle Repositories...' flash.admin.git-gc: 'Sammle Repositories...'
flash.admin.sync-previews: 'Synchronisiere Gist-Vorschauen...' flash.admin.sync-previews: 'Synchronisiere Gist-Vorschauen...'
flash.admin.reset-hooks: 'Setze Git-Server-Hooks für alle Repositories zurück...' flash.admin.reset-hooks: 'Setze Git-Server-Hooks für alle Repositories zurück...'
flash.admin.index-gists: 'Indiziere alle Gists...' flash.admin.index-gists: 'Suchindex wird neu aufgebaut...'
flash.auth.username-exists: 'Benutzername existiert bereits' flash.auth.username-exists: 'Benutzername existiert bereits'
flash.auth.invalid-credentials: 'Ungültige Anmeldeinformationen' flash.auth.invalid-credentials: 'Ungültige Anmeldeinformationen'

View File

@@ -88,10 +88,12 @@ gist.search.found: gists found
gist.search.no-results: No gists found gist.search.no-results: No gists found
gist.search.help.user: gists created by user gist.search.help.user: gists created by user
gist.search.help.title: gists with given title gist.search.help.title: gists with given title
gist.search.help.description: gists with given description
gist.search.help.filename: gists having files with given name gist.search.help.filename: gists having files with given name
gist.search.help.extension: gists having files with given extension gist.search.help.extension: gists having files with given extension
gist.search.help.language: gists having files with given language gist.search.help.language: gists having files with given language
gist.search.help.topic: gists with given topic gist.search.help.topic: gists with given topic
gist.search.help.all: search all fields
gist.search.placeholder.title: Title gist.search.placeholder.title: Title
gist.search.placeholder.visibility: Visibility gist.search.placeholder.visibility: Visibility
gist.search.placeholder.public: Public gist.search.placeholder.public: Public
@@ -292,7 +294,7 @@ admin.actions.sync-db: Synchronize gists from database
admin.actions.git-gc: Garbage collect all git repositories admin.actions.git-gc: Garbage collect all git repositories
admin.actions.sync-previews: Synchronize all gists previews admin.actions.sync-previews: Synchronize all gists previews
admin.actions.reset-hooks: Reset Git server hooks for all repositories admin.actions.reset-hooks: Reset Git server hooks for all repositories
admin.actions.index-gists: Index all gists admin.actions.index-gists: Rebuild search index
admin.actions.sync-gist-languages: Synchronize all gists languages admin.actions.sync-gist-languages: Synchronize all gists languages
admin.id: ID admin.id: ID
admin.user: User admin.user: User
@@ -338,7 +340,7 @@ flash.admin.sync-db: Syncing repositories from database...
flash.admin.git-gc: Garbage collecting repositories... flash.admin.git-gc: Garbage collecting repositories...
flash.admin.sync-previews: Syncing Gist previews... flash.admin.sync-previews: Syncing Gist previews...
flash.admin.reset-hooks: Resetting Git server hooks for all repositories... flash.admin.reset-hooks: Resetting Git server hooks for all repositories...
flash.admin.index-gists: Indexing all gists... flash.admin.index-gists: Rebuilding search index...
flash.admin.sync-gist-languages: Syncing Gist languages... flash.admin.sync-gist-languages: Syncing Gist languages...
flash.auth.username-exists: Username already exists flash.auth.username-exists: Username already exists

View File

@@ -213,7 +213,7 @@ admin.invitations: 'Invitaciones'
admin.invitations.create: 'Crear invitación' admin.invitations.create: 'Crear invitación'
admin.actions.sync-previews: 'Sincronizar todas las vistas previas de gists' admin.actions.sync-previews: 'Sincronizar todas las vistas previas de gists'
admin.actions.reset-hooks: 'Resetear los hooks de Git en todos los repositorios' admin.actions.reset-hooks: 'Resetear los hooks de Git en todos los repositorios'
admin.actions.index-gists: 'Indexar todos los gists' admin.actions.index-gists: 'Reconstruir índice de búsqueda'
admin.config-link-overriden: 'sobrescrito' admin.config-link-overriden: 'sobrescrito'
admin.invitations.help: 'Las invitaciones se pueden usar para crear una cuenta aunque el registro esté deshabilitado.' admin.invitations.help: 'Las invitaciones se pueden usar para crear una cuenta aunque el registro esté deshabilitado.'
admin.invitations.max_uses: 'Cantidad máxima de usos' admin.invitations.max_uses: 'Cantidad máxima de usos'
@@ -231,7 +231,7 @@ flash.admin.sync-db: 'Sincronizando repositorios desde la base de datos...'
flash.admin.git-gc: 'Recolectando basura en los repositorios...' flash.admin.git-gc: 'Recolectando basura en los repositorios...'
flash.admin.sync-previews: 'Sincronizando vistas previas de gists...' flash.admin.sync-previews: 'Sincronizando vistas previas de gists...'
flash.admin.reset-hooks: 'Reseteando hooks del servidor Git en todos los repositorios...' flash.admin.reset-hooks: 'Reseteando hooks del servidor Git en todos los repositorios...'
flash.admin.index-gists: 'Indexando todos los gists...' flash.admin.index-gists: 'Reconstruyendo índice de búsqueda...'
flash.auth.username-exists: 'El nombre de usuario ya existe' flash.auth.username-exists: 'El nombre de usuario ya existe'
flash.auth.invalid-credentials: 'Credenciales incorrectas' flash.auth.invalid-credentials: 'Credenciales incorrectas'
flash.auth.account-linked-oauth: 'Cuenta vinculada a %s' flash.auth.account-linked-oauth: 'Cuenta vinculada a %s'

View File

@@ -193,7 +193,7 @@ admin.actions.reset-hooks: Réinitialiser les hooks de Git pour tous les dépôt
gist.new.url: URL gist.new.url: URL
gist.search.no-results: Aucun gist trouvé gist.search.no-results: Aucun gist trouvé
settings.unlink-gitlab-account: Détacher le compte GitLab settings.unlink-gitlab-account: Détacher le compte GitLab
admin.actions.index-gists: Indexer tous les gists admin.actions.index-gists: Reconstruire l'index de recherche
gist.new.preview: 'Aperçu' gist.new.preview: 'Aperçu'
gist.new.create-a-new-gist: 'Créer un nouveau gist' gist.new.create-a-new-gist: 'Créer un nouveau gist'
gist.edit.edit-gist: 'Modifier %s' gist.edit.edit-gist: 'Modifier %s'
@@ -231,7 +231,7 @@ flash.admin.sync-db: 'Synchronisation des dépôts à partir de la base de donn
flash.admin.git-gc: 'Nettoyage des dépôts...' flash.admin.git-gc: 'Nettoyage des dépôts...'
flash.admin.sync-previews: 'Synchronisation des aperçus du Gist...' flash.admin.sync-previews: 'Synchronisation des aperçus du Gist...'
flash.admin.reset-hooks: 'Réinitialisation des hooks du serveur Git pour tous les dépôts...' flash.admin.reset-hooks: 'Réinitialisation des hooks du serveur Git pour tous les dépôts...'
flash.admin.index-gists: 'Indexation de tous les gists...' flash.admin.index-gists: 'Reconstruction de l''index de recherche...'
flash.auth.username-exists: 'Nom d''utilisateur déjà utilisé' flash.auth.username-exists: 'Nom d''utilisateur déjà utilisé'
flash.auth.invalid-credentials: 'Identifiants non valides' flash.auth.invalid-credentials: 'Identifiants non valides'
flash.auth.account-linked-oauth: 'Compte lié à %s' flash.auth.account-linked-oauth: 'Compte lié à %s'

View File

@@ -170,7 +170,7 @@ admin.actions.sync-db: Gistek szinkronizálása az adatbázissal
admin.actions.git-gc: Használatlan git repository-k eltávolítása admin.actions.git-gc: Használatlan git repository-k eltávolítása
admin.actions.sync-previews: Gist előnézetek szinkronizálása admin.actions.sync-previews: Gist előnézetek szinkronizálása
admin.actions.reset-hooks: Git server hook-ok alaphelyzetbe állítása minden repository-nál admin.actions.reset-hooks: Git server hook-ok alaphelyzetbe állítása minden repository-nál
admin.actions.index-gists: Gistek indexelése admin.actions.index-gists: Keresési index újraépítése
admin.id: Azonosító admin.id: Azonosító
admin.user: Felhasználó admin.user: Felhasználó
admin.delete: Törlés admin.delete: Törlés

View File

@@ -191,7 +191,7 @@ admin.actions.sync-db: 'Sincronizza gists dal database'
admin.actions.git-gc: 'Esegui la garbage collection da tutti i repositories' admin.actions.git-gc: 'Esegui la garbage collection da tutti i repositories'
admin.actions.sync-previews: 'Sincronizza tutte le anteprime dei gists' admin.actions.sync-previews: 'Sincronizza tutte le anteprime dei gists'
admin.actions.reset-hooks: 'Resetta tutti gli hook del server Git per tutti i repositories' admin.actions.reset-hooks: 'Resetta tutti gli hook del server Git per tutti i repositories'
admin.actions.index-gists: 'Indicizza tutti i gists' admin.actions.index-gists: 'Ricostruisci indice di ricerca'
admin.id: 'ID' admin.id: 'ID'
admin.user: 'Utente' admin.user: 'Utente'
admin.delete: 'Elimina' admin.delete: 'Elimina'
@@ -235,7 +235,7 @@ flash.admin.sync-db: 'Sincronizzando i repositories dal database...'
flash.admin.git-gc: 'Eseguendo il garbage collector dei repositories...' flash.admin.git-gc: 'Eseguendo il garbage collector dei repositories...'
flash.admin.sync-previews: 'Sincronizzando le anteprime dei gists...' flash.admin.sync-previews: 'Sincronizzando le anteprime dei gists...'
flash.admin.reset-hooks: 'Resettando gli hook di Git per tutti i repositories...' flash.admin.reset-hooks: 'Resettando gli hook di Git per tutti i repositories...'
flash.admin.index-gists: 'Indicizzando tutti i gists...' flash.admin.index-gists: 'Ricostruzione indice di ricerca...'
flash.auth.username-exists: 'Il nome utente esiste già' flash.auth.username-exists: 'Il nome utente esiste già'
flash.auth.invalid-credentials: 'Credenziali errate' flash.auth.invalid-credentials: 'Credenziali errate'

View File

@@ -227,7 +227,7 @@ admin.actions.sync-db: 'Synchronizuj Gisty z bazy danych'
admin.actions.git-gc: 'Zbierz śmieci we wszystkich repozytoriach Git' admin.actions.git-gc: 'Zbierz śmieci we wszystkich repozytoriach Git'
admin.actions.sync-previews: 'Synchronizuj podglądy wszystkich Gistów' admin.actions.sync-previews: 'Synchronizuj podglądy wszystkich Gistów'
admin.actions.reset-hooks: 'Zresetuj hooki serwera Git dla wszystkich repozytoriów' admin.actions.reset-hooks: 'Zresetuj hooki serwera Git dla wszystkich repozytoriów'
admin.actions.index-gists: 'Indeksuj wszystkie Gisty' admin.actions.index-gists: 'Przebuduj indeks wyszukiwania'
admin.id: 'ID' admin.id: 'ID'
admin.user: 'Użytkownik' admin.user: 'Użytkownik'
admin.delete: 'Usuń' admin.delete: 'Usuń'
@@ -271,7 +271,7 @@ flash.admin.sync-db: 'Synchronizowanie repozytoriów z bazy danych...'
flash.admin.git-gc: 'Zbieranie śmieci w repozytoriach...' flash.admin.git-gc: 'Zbieranie śmieci w repozytoriach...'
flash.admin.sync-previews: 'Synchronizowanie podglądów Gistów...' flash.admin.sync-previews: 'Synchronizowanie podglądów Gistów...'
flash.admin.reset-hooks: 'Resetowanie hooków serwera Git dla wszystkich repozytoriów...' flash.admin.reset-hooks: 'Resetowanie hooków serwera Git dla wszystkich repozytoriów...'
flash.admin.index-gists: 'Indeksowanie wszystkich Gistów...' flash.admin.index-gists: 'Przebudowywanie indeksu wyszukiwania...'
flash.auth.username-exists: 'Nazwa użytkownika już istnieje' flash.auth.username-exists: 'Nazwa użytkownika już istnieje'
flash.auth.invalid-credentials: 'Niepoprawne dane logowania' flash.auth.invalid-credentials: 'Niepoprawne dane logowania'

View File

@@ -214,7 +214,7 @@ admin.invitations: 'Инвайты'
admin.invitations.create: 'Создать инвайт' admin.invitations.create: 'Создать инвайт'
admin.actions.sync-previews: 'Обновить предпросмотры всех фрагментов' admin.actions.sync-previews: 'Обновить предпросмотры всех фрагментов'
admin.actions.reset-hooks: 'Сбросить хуки Git-сервера для всех репозиториев' admin.actions.reset-hooks: 'Сбросить хуки Git-сервера для всех репозиториев'
admin.actions.index-gists: роиндексировать все фрагменты' admin.actions.index-gists: ерестроить поисковый индекс'
validation.should-not-be-empty: 'Поле %s не должно быть пустым' validation.should-not-be-empty: 'Поле %s не должно быть пустым'
admin.invitations.help: 'Инвайты используются для создания аккаунта, даже когда регистрация запрещена.' admin.invitations.help: 'Инвайты используются для создания аккаунта, даже когда регистрация запрещена.'
admin.invitations.max_uses: 'Максимальное количество использований' admin.invitations.max_uses: 'Максимальное количество использований'
@@ -232,7 +232,7 @@ flash.admin.sync-db: 'Выполняется синхронизация репо
flash.admin.git-gc: 'Сборка мусора в репозиториях…' flash.admin.git-gc: 'Сборка мусора в репозиториях…'
flash.admin.sync-previews: 'Обновление предпросмотров фрагментов…' flash.admin.sync-previews: 'Обновление предпросмотров фрагментов…'
flash.admin.reset-hooks: 'Пересоздание Git-хуков для всех репозиториев…' flash.admin.reset-hooks: 'Пересоздание Git-хуков для всех репозиториев…'
flash.admin.index-gists: 'Выполняется индексация фрагментов…' flash.admin.index-gists: 'Перестроение поискового индекса…'
flash.auth.username-exists: 'Такое имя пользователя уже занято' flash.auth.username-exists: 'Такое имя пользователя уже занято'
flash.auth.invalid-credentials: 'Некорректные данные для входа' flash.auth.invalid-credentials: 'Некорректные данные для входа'
flash.auth.account-linked-oauth: 'Учётная запись связана с %s' flash.auth.account-linked-oauth: 'Учётная запись связана с %s'

View File

@@ -191,7 +191,7 @@ admin.actions.sync-db: Gistleri veri tabanından senkronize et
admin.actions.git-gc: Tüm Git depolarındaki gereksiz verileri temizle admin.actions.git-gc: Tüm Git depolarındaki gereksiz verileri temizle
admin.actions.sync-previews: Tüm gist önizlemelerini senkronize et admin.actions.sync-previews: Tüm gist önizlemelerini senkronize et
admin.actions.reset-hooks: Tüm depolar için Git sunucu kancalarını sıfırla admin.actions.reset-hooks: Tüm depolar için Git sunucu kancalarını sıfırla
admin.actions.index-gists: Tüm gistleri indeksle admin.actions.index-gists: Arama dizinini yeniden oluştur
admin.id: ID admin.id: ID
admin.user: Kullanıcı admin.user: Kullanıcı
admin.delete: Sil admin.delete: Sil
@@ -234,7 +234,7 @@ flash.admin.sync-db: Depolar veri tabanından senkronize ediliyor...
flash.admin.git-gc: Depolardan gereksiz veriler temizleniyor... flash.admin.git-gc: Depolardan gereksiz veriler temizleniyor...
flash.admin.sync-previews: Gist önizlemeleri senkronize ediliyor... flash.admin.sync-previews: Gist önizlemeleri senkronize ediliyor...
flash.admin.reset-hooks: Tüm depolar için Git sunucusu kancaları sıfırlanıyor... flash.admin.reset-hooks: Tüm depolar için Git sunucusu kancaları sıfırlanıyor...
flash.admin.index-gists: Tüm gistler indeksleniyor... flash.admin.index-gists: Arama dizini yeniden oluşturuluyor...
flash.auth.username-exists: Kullanıcı adı zaten mevcut flash.auth.username-exists: Kullanıcı adı zaten mevcut
flash.auth.invalid-credentials: Geçersiz kimlik bilgileri flash.auth.invalid-credentials: Geçersiz kimlik bilgileri

View File

@@ -192,7 +192,7 @@ admin.actions.sync-db: Синхронізувати gists з базою дани
admin.actions.git-gc: Збір сміття з репозиторіїв Git admin.actions.git-gc: Збір сміття з репозиторіїв Git
admin.actions.sync-previews: Синхронізувати всі gists перегляди admin.actions.sync-previews: Синхронізувати всі gists перегляди
admin.actions.reset-hooks: Скинути серверні Git hooks для всіх репозиторіїв admin.actions.reset-hooks: Скинути серверні Git hooks для всіх репозиторіїв
admin.actions.index-gists: Проіндексувати всі gists admin.actions.index-gists: Перебудувати пошуковий індекс
admin.id: ID admin.id: ID
admin.user: Користувач admin.user: Користувач
admin.delete: Видалити admin.delete: Видалити
@@ -236,7 +236,7 @@ flash.admin.sync-db: Синхронізація репозиторіїв за б
flash.admin.git-gc: Збір сміття з репозиторіїв... flash.admin.git-gc: Збір сміття з репозиторіїв...
flash.admin.sync-previews: Синхронізація Gist переглядів... flash.admin.sync-previews: Синхронізація Gist переглядів...
flash.admin.reset-hooks: Скидання cерверниз Git hooks для всіх репозиторіїв... flash.admin.reset-hooks: Скидання cерверниз Git hooks для всіх репозиторіїв...
flash.admin.index-gists: Індексація всіх gists... flash.admin.index-gists: Перебудова пошукового індексу...
flash.auth.username-exists: Це ім'я користувача вже існує flash.auth.username-exists: Це ім'я користувача вже існує
flash.auth.invalid-credentials: Недійсні облікові дані flash.auth.invalid-credentials: Недійсні облікові дані

View File

@@ -214,7 +214,7 @@ admin.invitations: '邀请'
admin.invitations.create: '创建邀请' admin.invitations.create: '创建邀请'
admin.actions.sync-previews: '同步所有 Gists 预览' admin.actions.sync-previews: '同步所有 Gists 预览'
admin.actions.reset-hooks: '重置所有存储库的 Git 服务 hooks' admin.actions.reset-hooks: '重置所有存储库的 Git 服务 hooks'
admin.actions.index-gists: '索引所有 Gists' admin.actions.index-gists: '重建搜索索引'
admin.invitations.help: '即使在禁用注册功能的情况下,邀请功能也可用于创建帐户。' admin.invitations.help: '即使在禁用注册功能的情况下,邀请功能也可用于创建帐户。'
admin.invitations.max_uses: '最多使用次数' admin.invitations.max_uses: '最多使用次数'
admin.invitations.expires_at: '过期时间' admin.invitations.expires_at: '过期时间'
@@ -231,7 +231,7 @@ flash.admin.sync-db: '正在从数据库同步存储库...'
flash.admin.git-gc: '正在进行存储库垃圾回收...' flash.admin.git-gc: '正在进行存储库垃圾回收...'
flash.admin.sync-previews: '正在同步 Gist 预览...' flash.admin.sync-previews: '正在同步 Gist 预览...'
flash.admin.reset-hooks: '正在重置所有存储库的 Git 服务挂钩...' flash.admin.reset-hooks: '正在重置所有存储库的 Git 服务挂钩...'
flash.admin.index-gists: '正在索引所有 Gists...' flash.admin.index-gists: '正在重建搜索索引...'
flash.auth.username-exists: '用户名已存在' flash.auth.username-exists: '用户名已存在'
flash.auth.invalid-credentials: '无效的凭证' flash.auth.invalid-credentials: '无效的凭证'
flash.auth.account-linked-oauth: '帐户已关联到 %s' flash.auth.account-linked-oauth: '帐户已关联到 %s'

View File

@@ -190,7 +190,7 @@ gist.search.no-results: 沒有找到任何 Gists
gist.search.help.title: Gists 的標題 gist.search.help.title: Gists 的標題
gist.search.help.filename: Gists 的檔案名稱 gist.search.help.filename: Gists 的檔案名稱
gist.search.help.language: Gists 的程式語言 gist.search.help.language: Gists 的程式語言
admin.actions.index-gists: 索引所有的 Gists admin.actions.index-gists: 重建搜尋索引
gist.search.help.user: 由使用者建立的 Gists gist.search.help.user: 由使用者建立的 Gists
gist.search.found: 已找到 Gists gist.search.found: 已找到 Gists
gist.search.help.extension: Gists 的副檔名 gist.search.help.extension: Gists 的副檔名

View File

@@ -2,16 +2,21 @@ package index
import ( import (
"errors" "errors"
"fmt"
"os"
"strconv" "strconv"
"strings"
"github.com/blevesearch/bleve/v2" "github.com/blevesearch/bleve/v2"
"github.com/blevesearch/bleve/v2/analysis/analyzer/custom" "github.com/blevesearch/bleve/v2/analysis/analyzer/custom"
"github.com/blevesearch/bleve/v2/analysis/token/camelcase" "github.com/blevesearch/bleve/v2/analysis/token/camelcase"
"github.com/blevesearch/bleve/v2/analysis/token/length"
"github.com/blevesearch/bleve/v2/analysis/token/lowercase" "github.com/blevesearch/bleve/v2/analysis/token/lowercase"
"github.com/blevesearch/bleve/v2/analysis/token/unicodenorm" "github.com/blevesearch/bleve/v2/analysis/token/unicodenorm"
"github.com/blevesearch/bleve/v2/analysis/tokenizer/unicode" "github.com/blevesearch/bleve/v2/analysis/tokenizer/unicode"
"github.com/blevesearch/bleve/v2/search/query" "github.com/blevesearch/bleve/v2/search/query"
"github.com/rs/zerolog/log" "github.com/rs/zerolog/log"
"github.com/thomiceli/opengist/internal/config"
) )
type BleveIndexer struct { type BleveIndexer struct {
@@ -52,14 +57,9 @@ func (i *BleveIndexer) open() (bleve.Index, error) {
return nil, err return nil, err
} }
docMapping := bleve.NewDocumentMapping()
docMapping.AddFieldMappingsAt("GistID", bleve.NewNumericFieldMapping())
docMapping.AddFieldMappingsAt("UserID", bleve.NewNumericFieldMapping())
docMapping.AddFieldMappingsAt("Visibility", bleve.NewNumericFieldMapping())
docMapping.AddFieldMappingsAt("Content", bleve.NewTextFieldMapping())
mapping := bleve.NewIndexMapping() mapping := bleve.NewIndexMapping()
// Token filters
if err = mapping.AddCustomTokenFilter("unicodeNormalize", map[string]any{ if err = mapping.AddCustomTokenFilter("unicodeNormalize", map[string]any{
"type": unicodenorm.Name, "type": unicodenorm.Name,
"form": unicodenorm.NFC, "form": unicodenorm.NFC,
@@ -67,21 +67,88 @@ func (i *BleveIndexer) open() (bleve.Index, error) {
return nil, err return nil, err
} }
if err = mapping.AddCustomAnalyzer("gistAnalyser", map[string]interface{}{ if err = mapping.AddCustomTokenFilter("lengthMin2", map[string]interface{}{
"type": custom.Name, "type": length.Name,
"char_filters": []string{}, "min": 2.0,
"tokenizer": unicode.Name,
"token_filters": []string{"unicodeNormalize", camelcase.Name, lowercase.Name},
}); err != nil { }); err != nil {
return nil, err return nil, err
} }
docMapping.DefaultAnalyzer = "gistAnalyser" // Analyzer: split mode (camelCase splitting for partial search)
// "CPUCard" -> ["cpu", "card"]
if err = mapping.AddCustomAnalyzer("codeSplit", map[string]interface{}{
"type": custom.Name,
"char_filters": []string{},
"tokenizer": unicode.Name,
"token_filters": []string{"unicodeNormalize", camelcase.Name, lowercase.Name, "lengthMin2"},
}); err != nil {
return nil, err
}
// Analyzer: exact mode (no camelCase splitting for full-word search)
// "CPUCard" -> ["cpucard"]
if err = mapping.AddCustomAnalyzer("codeExact", map[string]interface{}{
"type": custom.Name,
"char_filters": []string{},
"tokenizer": unicode.Name,
"token_filters": []string{"unicodeNormalize", lowercase.Name},
}); err != nil {
return nil, err
}
// Analyzer: keyword with lowercase (for Languages - single token, no splitting)
if err = mapping.AddCustomAnalyzer("lowercaseKeyword", map[string]interface{}{
"type": custom.Name,
"char_filters": []string{},
"tokenizer": "single",
"token_filters": []string{lowercase.Name},
}); err != nil {
return nil, err
}
// Document mapping
docMapping := bleve.NewDocumentMapping()
docMapping.AddFieldMappingsAt("GistID", bleve.NewNumericFieldMapping())
docMapping.AddFieldMappingsAt("UserID", bleve.NewNumericFieldMapping())
docMapping.AddFieldMappingsAt("Visibility", bleve.NewNumericFieldMapping())
// Content: dual indexing (exact + split)
// "Content" uses the property name so Bleve resolves its analyzer correctly
contentExact := bleve.NewTextFieldMapping()
contentExact.Name = "Content"
contentExact.Analyzer = "codeExact"
contentExact.Store = false
contentExact.IncludeTermVectors = true
contentSplit := bleve.NewTextFieldMapping()
contentSplit.Name = "ContentSplit"
contentSplit.Analyzer = "codeSplit"
contentSplit.Store = false
contentSplit.IncludeTermVectors = true
docMapping.AddFieldMappingsAt("Content", contentExact, contentSplit)
// Languages: keyword analyzer (preserves as single token)
languageFieldMapping := bleve.NewTextFieldMapping()
languageFieldMapping.Analyzer = "lowercaseKeyword"
docMapping.AddFieldMappingsAt("Languages", languageFieldMapping)
// All other text fields use codeSplit as default
docMapping.DefaultAnalyzer = "codeSplit"
mapping.DefaultMapping = docMapping mapping.DefaultMapping = docMapping
return bleve.New(i.path, mapping) return bleve.New(i.path, mapping)
} }
func (i *BleveIndexer) Reset() error {
i.Close()
if err := os.RemoveAll(i.path); err != nil {
return fmt.Errorf("failed to remove Bleve index directory: %w", err)
}
log.Info().Msg("Bleve index directory removed, re-creating index")
return i.Init()
}
func (i *BleveIndexer) Close() { func (i *BleveIndexer) Close() {
if i == nil || i.index == nil { if i == nil || i.index == nil {
return return
@@ -105,18 +172,111 @@ func (i *BleveIndexer) Remove(gistID uint) error {
return (*atomicIndexer.Load()).(*BleveIndexer).index.Delete(strconv.Itoa(int(gistID))) return (*atomicIndexer.Load()).(*BleveIndexer).index.Delete(strconv.Itoa(int(gistID)))
} }
func (i *BleveIndexer) Search(queryStr string, queryMetadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error) { // Search returns a list of Gist IDs that match the given search metadata.
// The method returns an error if any.
//
// The queryMetadata parameter is used to filter the search results.
// For example, passing a non-empty Username will search for gists whose
// username matches the given string.
//
// If the "All" field in queryMetadata is non-empty, the method will
// search across all metadata fields with OR logic. Otherwise, the method
// will add each metadata field with AND logic.
//
// The page parameter is used to paginate the search results.
// The method returns the total number of search results in the second return
// value.
//
// The third return value is a map of language counts for the search results.
// The language counts are computed by asking Bleve to return the top 10
// facets for the "Languages" field.
func (i *BleveIndexer) Search(metadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error) {
var err error var err error
var indexerQuery query.Query var indexerQuery query.Query = bleve.NewMatchAllQuery()
if queryStr != "" {
// Use match query with fuzzy matching for more flexible content search // Query factory
contentQuery := bleve.NewMatchQuery(queryStr) factoryQuery := func(field, value string) query.Query {
contentQuery.SetField("Content") query := bleve.NewMatchPhraseQuery(value)
contentQuery.SetFuzziness(2) query.SetField(field)
indexerQuery = contentQuery return query
} else { }
contentQuery := bleve.NewMatchAllQuery()
indexerQuery = contentQuery // Exact search
addQuery := func(field, value string) {
if value != "" && value != "." {
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, factoryQuery(field, value))
}
}
// Query factory for text fields: exact match boosted + match query + prefix
factoryTextQuery := func(field, value string) query.Query {
exact := bleve.NewMatchPhraseQuery(value)
exact.SetField(field)
exact.SetBoost(2.0)
fuzzy := bleve.NewMatchQuery(value)
fuzzy.SetField(field)
fuzzy.SetFuzziness(1)
fuzzy.SetOperator(query.MatchQueryOperatorAnd)
queries := []query.Query{exact, fuzzy}
if len([]rune(value)) >= 2 {
prefix := bleve.NewPrefixQuery(strings.ToLower(value))
prefix.SetField(field)
prefix.SetBoost(1.5)
queries = append(queries, prefix)
}
if len([]rune(value)) >= 4 {
wildcard := bleve.NewWildcardQuery("*" + strings.ToLower(value) + "*")
wildcard.SetField(field)
wildcard.SetBoost(0.5)
queries = append(queries, wildcard)
}
return bleve.NewDisjunctionQuery(queries...)
}
// Query factory for Content: searches both exact (Content) and split (ContentSplit) fields
factoryContentQuery := func(value string) query.Query {
// Exact field (no camelCase split): matches "cpucard"
exactMatch := bleve.NewMatchQuery(value)
exactMatch.SetField("Content")
exactMatch.SetOperator(query.MatchQueryOperatorAnd)
exactMatch.SetBoost(2.0)
// Split field (camelCase split): matches "cpu", "card"
splitMatch := bleve.NewMatchQuery(value)
splitMatch.SetField("ContentSplit")
splitMatch.SetFuzziness(1)
splitMatch.SetOperator(query.MatchQueryOperatorAnd)
splitMatch.SetBoost(1.0)
queries := []query.Query{exactMatch, splitMatch}
if len([]rune(value)) >= 2 {
prefix := bleve.NewPrefixQuery(strings.ToLower(value))
prefix.SetField("Content")
prefix.SetBoost(1.5)
queries = append(queries, prefix)
}
if len([]rune(value)) >= 4 {
wildcard := bleve.NewWildcardQuery("*" + strings.ToLower(value) + "*")
wildcard.SetField("Content")
wildcard.SetBoost(0.5)
queries = append(queries, wildcard)
}
return bleve.NewDisjunctionQuery(queries...)
}
// Text field search
addTextQuery := func(field, value string) {
if value != "" && value != "." {
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, factoryTextQuery(field, value))
}
} }
// Visibility filtering: show public gists (Visibility=0) OR user's own gists // Visibility filtering: show public gists (Visibility=0) OR user's own gists
@@ -132,48 +292,62 @@ func (i *BleveIndexer) Search(queryStr string, queryMetadata SearchGistMetadata,
accessQuery := bleve.NewDisjunctionQuery(publicQuery, userIdQuery) accessQuery := bleve.NewDisjunctionQuery(publicQuery, userIdQuery)
indexerQuery = bleve.NewConjunctionQuery(accessQuery, indexerQuery) indexerQuery = bleve.NewConjunctionQuery(accessQuery, indexerQuery)
buildFieldQuery := func(field, value string) query.Query {
switch field {
case "Content":
return factoryContentQuery(value)
case "Title", "Description", "Filenames":
return factoryTextQuery(field, value)
case "Extensions":
return factoryQuery(field, "."+value)
default: // Username, Languages, Topics
return factoryQuery(field, value)
}
}
// Handle "All" field - search across all metadata fields with OR logic // Handle "All" field - search across all metadata fields with OR logic
if queryMetadata.All != "" { if metadata.All != "" {
allQueries := make([]query.Query, 0) allQueries := make([]query.Query, 0, len(AllSearchFields))
for _, field := range AllSearchFields {
// Create match phrase queries for each field allQueries = append(allQueries, buildFieldQuery(field, metadata.All))
fields := []struct {
field string
value string
}{
{"Username", queryMetadata.All},
{"Title", queryMetadata.All},
{"Extensions", "." + queryMetadata.All},
{"Filenames", queryMetadata.All},
{"Languages", queryMetadata.All},
{"Topics", queryMetadata.All},
} }
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, bleve.NewDisjunctionQuery(allQueries...))
for _, f := range fields {
q := bleve.NewMatchPhraseQuery(f.value)
q.FieldVal = f.field
allQueries = append(allQueries, q)
}
// Combine all field queries with OR (disjunction)
allDisjunction := bleve.NewDisjunctionQuery(allQueries...)
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, allDisjunction)
} else { } else {
// Original behavior: add each metadata field with AND logic // Original behavior: add each metadata field with AND logic
addQuery := func(field, value string) { addQuery("Username", metadata.Username)
if value != "" && value != "." { addTextQuery("Title", metadata.Title)
q := bleve.NewMatchPhraseQuery(value) addTextQuery("Description", metadata.Description)
q.FieldVal = field addQuery("Extensions", "."+metadata.Extension)
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, q) addTextQuery("Filenames", metadata.Filename)
} addQuery("Languages", metadata.Language)
addQuery("Topics", metadata.Topic)
if metadata.Content != "" {
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, factoryContentQuery(metadata.Content))
} }
addQuery("Username", queryMetadata.Username) // Handle default search fields from config with OR logic
addQuery("Title", queryMetadata.Title) if metadata.Default != "" {
addQuery("Extensions", "."+queryMetadata.Extension) var fields []string
addQuery("Filenames", queryMetadata.Filename) for _, f := range strings.Split(config.C.SearchDefault, ",") {
addQuery("Languages", queryMetadata.Language) f = strings.TrimSpace(f)
addQuery("Topics", queryMetadata.Topic) if f == "all" {
fields = AllSearchFields
break
}
if indexField, ok := SearchFieldMap[f]; ok {
fields = append(fields, indexField)
}
}
if len(fields) == 1 {
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, buildFieldQuery(fields[0], metadata.Default))
} else if len(fields) > 1 {
defaultQueries := make([]query.Query, 0, len(fields))
for _, field := range fields {
defaultQueries = append(defaultQueries, buildFieldQuery(field, metadata.Default))
}
indexerQuery = bleve.NewConjunctionQuery(indexerQuery, bleve.NewDisjunctionQuery(defaultQueries...))
}
}
} }
languageFacet := bleve.NewFacetRequest("Languages", 10) languageFacet := bleve.NewFacetRequest("Languages", 10)
@@ -186,6 +360,8 @@ func (i *BleveIndexer) Search(queryStr string, queryMetadata SearchGistMetadata,
s.Fields = []string{"GistID"} s.Fields = []string{"GistID"}
s.IncludeLocations = false s.IncludeLocations = false
log.Debug().Interface("searchRequest", s).Msg("Bleve search request")
results, err := (*atomicIndexer.Load()).(*BleveIndexer).index.Search(s) results, err := (*atomicIndexer.Load()).(*BleveIndexer).index.Search(s)
if err != nil { if err != nil {
return nil, 0, nil, err return nil, 0, nil, err

View File

@@ -4,33 +4,31 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"testing" "testing"
"github.com/rs/zerolog"
"github.com/stretchr/testify/require"
) )
// setupBleveIndexer creates a new BleveIndexer for testing // setupBleveIndexer creates a new BleveIndexer for testing
func setupBleveIndexer(t *testing.T) (*BleveIndexer, func()) { func setupBleveIndexer(t *testing.T) (Indexer, func()) {
zerolog.SetGlobalLevel(zerolog.Disabled)
t.Helper() t.Helper()
// Create a temporary directory for the test index
tmpDir, err := os.MkdirTemp("", "bleve-test-*") tmpDir, err := os.MkdirTemp("", "bleve-test-*")
if err != nil { require.NoError(t, err)
t.Fatalf("Failed to create temp directory: %v", err)
}
indexPath := filepath.Join(tmpDir, "test.index") indexPath := filepath.Join(tmpDir, "test.index")
indexer := NewBleveIndexer(indexPath) indexer := NewBleveIndexer(indexPath)
// Initialize the indexer
err = indexer.Init() err = indexer.Init()
if err != nil { if err != nil {
os.RemoveAll(tmpDir) os.RemoveAll(tmpDir)
t.Fatalf("Failed to initialize BleveIndexer: %v", err) t.Fatalf("Failed to initialize BleveIndexer: %v", err)
} }
// Store in the global atomicIndexer since Add/Remove use it
var idx Indexer = indexer var idx Indexer = indexer
atomicIndexer.Store(&idx) atomicIndexer.Store(&idx)
// Return cleanup function
cleanup := func() { cleanup := func() {
atomicIndexer.Store(nil) atomicIndexer.Store(nil)
indexer.Close() indexer.Close()
@@ -40,123 +38,50 @@ func setupBleveIndexer(t *testing.T) (*BleveIndexer, func()) {
return indexer, cleanup return indexer, cleanup
} }
func TestBleveIndexerAddGist(t *testing.T) { func TestBleveAddAndSearch(t *testing.T) { testAddAndSearch(t, setupBleveIndexer) }
indexer, cleanup := setupBleveIndexer(t) func TestBleveAccessControl(t *testing.T) { testAccessControl(t, setupBleveIndexer) }
defer cleanup() func TestBleveMetadataFilters(t *testing.T) { testMetadataFilters(t, setupBleveIndexer) }
func TestBleveAllFieldSearch(t *testing.T) { testAllFieldSearch(t, setupBleveIndexer) }
func TestBleveFuzzySearch(t *testing.T) { testFuzzySearch(t, setupBleveIndexer) }
func TestBleveContentSearch(t *testing.T) { testContentSearch(t, setupBleveIndexer) }
func TestBlevePagination(t *testing.T) { testPagination(t, setupBleveIndexer) }
func TestBleveLanguageFacets(t *testing.T) { testLanguageFacets(t, setupBleveIndexer) }
func TestBleveWildcardSearch(t *testing.T) { testWildcardSearch(t, setupBleveIndexer) }
func TestBleveMetadataOnlySearch(t *testing.T) { testMetadataOnlySearch(t, setupBleveIndexer) }
func TestBleveTitleFuzzySearch(t *testing.T) { testTitleFuzzySearch(t, setupBleveIndexer) }
func TestBleveMultiLanguageFacets(t *testing.T) { testMultiLanguageFacets(t, setupBleveIndexer) }
testIndexerAddGist(t, indexer) func TestBlevePersistence(t *testing.T) {
} tmpDir, err := os.MkdirTemp("", "bleve-persist-test-*")
require.NoError(t, err)
func TestBleveIndexerAllFieldSearch(t *testing.T) {
indexer, cleanup := setupBleveIndexer(t)
defer cleanup()
testIndexerAllFieldSearch(t, indexer)
}
func TestBleveIndexerFuzzySearch(t *testing.T) {
indexer, cleanup := setupBleveIndexer(t)
defer cleanup()
testIndexerFuzzySearch(t, indexer)
}
func TestBleveIndexerSearchBasic(t *testing.T) {
indexer, cleanup := setupBleveIndexer(t)
defer cleanup()
testIndexerSearchBasic(t, indexer)
}
func TestBleveIndexerPagination(t *testing.T) {
indexer, cleanup := setupBleveIndexer(t)
defer cleanup()
testIndexerPagination(t, indexer)
}
// TestBleveIndexerInitAndClose tests Bleve-specific initialization and closing
func TestBleveIndexerInitAndClose(t *testing.T) {
tmpDir, err := os.MkdirTemp("", "bleve-init-test-*")
if err != nil {
t.Fatalf("Failed to create temp directory: %v", err)
}
defer os.RemoveAll(tmpDir) defer os.RemoveAll(tmpDir)
indexPath := filepath.Join(tmpDir, "test.index") indexPath := filepath.Join(tmpDir, "test.index")
indexer := NewBleveIndexer(indexPath)
// Test initialization // Create and populate index
err = indexer.Init() indexer1 := NewBleveIndexer(indexPath)
if err != nil { require.NoError(t, indexer1.Init())
t.Fatalf("Failed to initialize BleveIndexer: %v", err)
}
if indexer.index == nil { var idx Indexer = indexer1
t.Fatal("Expected index to be initialized, got nil") atomicIndexer.Store(&idx)
}
// Test closing g := newGist(1, 1, 0, "persistent data survives restart")
indexer.Close() require.NoError(t, indexer1.Add(g))
// Test reopening the same index indexer1.Close()
atomicIndexer.Store(nil)
// Reopen at same path
indexer2 := NewBleveIndexer(indexPath) indexer2 := NewBleveIndexer(indexPath)
err = indexer2.Init() require.NoError(t, indexer2.Init())
if err != nil {
t.Fatalf("Failed to reopen BleveIndexer: %v", err)
}
defer indexer2.Close() defer indexer2.Close()
if indexer2.index == nil { idx = indexer2
t.Fatal("Expected reopened index to be initialized, got nil") atomicIndexer.Store(&idx)
} defer atomicIndexer.Store(nil)
}
// TestBleveIndexerUnicodeSearch tests that Unicode content can be indexed and searched ids, total, _, err := indexer2.Search(SearchGistMetadata{Content: "persistent"}, 1, 1)
func TestBleveIndexerUnicodeSearch(t *testing.T) { require.NoError(t, err)
indexer, cleanup := setupBleveIndexer(t) require.Equal(t, uint64(1), total, "data should survive close+reopen")
defer cleanup() require.Equal(t, uint(1), ids[0])
// Add a gist with Unicode content
gist := &Gist{
GistID: 100,
UserID: 100,
Visibility: 0,
Username: "testuser",
Title: "Unicode Test",
Content: "Hello world with unicode characters: café résumé naïve",
Filenames: []string{"test.txt"},
Extensions: []string{".txt"},
Languages: []string{"Text"},
Topics: []string{"unicode"},
CreatedAt: 1234567890,
UpdatedAt: 1234567890,
}
err := indexer.Add(gist)
if err != nil {
t.Fatalf("Failed to add gist: %v", err)
}
// Search for unicode content
gistIDs, total, _, err := indexer.Search("café", SearchGistMetadata{}, 100, 1)
if err != nil {
t.Fatalf("Search failed: %v", err)
}
if total == 0 {
t.Skip("Unicode search may require specific index configuration")
return
}
found := false
for _, id := range gistIDs {
if id == 100 {
found = true
break
}
}
if !found {
t.Log("Unicode gist not found in search results, but other results were returned")
}
} }

View File

@@ -1,10 +1,24 @@
package index package index
var AllSearchFields = []string{"Username", "Title", "Description", "Filenames", "Extensions", "Languages", "Topics", "Content"}
var SearchFieldMap = map[string]string{
"user": "Username",
"title": "Title",
"description": "Description",
"filename": "Filenames",
"extension": "Extensions",
"language": "Languages",
"topic": "Topics",
"content": "Content",
}
type Gist struct { type Gist struct {
GistID uint GistID uint
UserID uint UserID uint
Visibility uint Visibility uint
Username string Username string
Description string
Title string Title string
Content string Content string
Filenames []string Filenames []string
@@ -18,9 +32,12 @@ type Gist struct {
type SearchGistMetadata struct { type SearchGistMetadata struct {
Username string Username string
Title string Title string
Description string
Content string
Filename string Filename string
Extension string Extension string
Language string Language string
Topic string Topic string
All string All string
Default string
} }

View File

@@ -2,10 +2,11 @@ package index
import ( import (
"fmt" "fmt"
"github.com/rs/zerolog/log"
"github.com/thomiceli/opengist/internal/config"
"path/filepath" "path/filepath"
"sync/atomic" "sync/atomic"
"github.com/rs/zerolog/log"
"github.com/thomiceli/opengist/internal/config"
) )
var atomicIndexer atomic.Pointer[Indexer] var atomicIndexer atomic.Pointer[Indexer]
@@ -13,9 +14,10 @@ var atomicIndexer atomic.Pointer[Indexer]
type Indexer interface { type Indexer interface {
Init() error Init() error
Close() Close()
Reset() error
Add(gist *Gist) error Add(gist *Gist) error
Remove(gistID uint) error Remove(gistID uint) error
Search(query string, metadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error) Search(metadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error)
} }
type IndexerType string type IndexerType string
@@ -84,6 +86,19 @@ func Close() {
atomicIndexer.Store(nil) atomicIndexer.Store(nil)
} }
func ResetIndex() error {
if !IndexEnabled() {
return nil
}
idx := atomicIndexer.Load()
if idx == nil {
return fmt.Errorf("indexer is not initialized")
}
return (*idx).Reset()
}
func AddInIndex(gist *Gist) error { func AddInIndex(gist *Gist) error {
if !IndexEnabled() { if !IndexEnabled() {
return nil return nil
@@ -110,7 +125,11 @@ func RemoveFromIndex(gistID uint) error {
return (*idx).Remove(gistID) return (*idx).Remove(gistID)
} }
func SearchGists(query string, metadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error) { // SearchGists returns a list of Gist IDs that match the given search metadata.
// If the indexer is not enabled, it returns nil, 0, nil, nil.
// If the indexer is not initialized, it returns nil, 0, nil, fmt.Errorf("indexer is not initialized").
// The function returns an error if any.
func SearchGists(metadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error) {
if !IndexEnabled() { if !IndexEnabled() {
return nil, 0, nil, nil return nil, 0, nil, nil
} }
@@ -120,7 +139,7 @@ func SearchGists(query string, metadata SearchGistMetadata, userId uint, page in
return nil, 0, nil, fmt.Errorf("indexer is not initialized") return nil, 0, nil, fmt.Errorf("indexer is not initialized")
} }
return (*idx).Search(query, metadata, userId, page) return (*idx).Search(metadata, userId, page)
} }
func DepreactionIndexDirname() { func DepreactionIndexDirname() {

File diff suppressed because it is too large Load Diff

View File

@@ -6,9 +6,11 @@ import (
"fmt" "fmt"
"strconv" "strconv"
"strings" "strings"
"unicode"
"github.com/meilisearch/meilisearch-go" "github.com/meilisearch/meilisearch-go"
"github.com/rs/zerolog/log" "github.com/rs/zerolog/log"
"github.com/thomiceli/opengist/internal/config"
) )
type MeiliIndexer struct { type MeiliIndexer struct {
@@ -50,10 +52,7 @@ func (i *MeiliIndexer) open() (meilisearch.IndexManager, error) {
i.client = meilisearch.New(i.host, meilisearch.WithAPIKey(i.apikey)) i.client = meilisearch.New(i.host, meilisearch.WithAPIKey(i.apikey))
indexResult, err := i.client.GetIndex(i.indexName) indexResult, err := i.client.GetIndex(i.indexName)
if indexResult != nil && err == nil { if indexResult == nil || err != nil {
return indexResult.IndexManager, nil
}
_, err = i.client.CreateIndex(&meilisearch.IndexConfig{ _, err = i.client.CreateIndex(&meilisearch.IndexConfig{
Uid: i.indexName, Uid: i.indexName,
PrimaryKey: "GistID", PrimaryKey: "GistID",
@@ -61,17 +60,37 @@ func (i *MeiliIndexer) open() (meilisearch.IndexManager, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
}
_, _ = i.client.Index(i.indexName).UpdateSettings(&meilisearch.Settings{ _, _ = i.client.Index(i.indexName).UpdateSettings(&meilisearch.Settings{
FilterableAttributes: []string{"GistID", "UserID", "Visibility", "Username", "Title", "Filenames", "Extensions", "Languages", "Topics"}, FilterableAttributes: []string{"GistID", "UserID", "Visibility", "Username", "Extensions", "Languages", "Topics"},
DisplayedAttributes: []string{"GistID"}, SearchableAttributes: []string{"Content", "ContentSplit", "Username", "Title", "Description", "Filenames", "Extensions", "Languages", "Topics"},
SearchableAttributes: []string{"Content", "Username", "Title", "Filenames", "Extensions", "Languages", "Topics"}, RankingRules: []string{"words", "typo", "proximity", "attribute", "sort", "exactness"},
RankingRules: []string{"words"}, TypoTolerance: &meilisearch.TypoTolerance{
Enabled: true,
DisableOnNumbers: true,
MinWordSizeForTypos: meilisearch.MinWordSizeForTypos{OneTypo: 4, TwoTypos: 10},
},
}) })
return i.client.Index(i.indexName), nil return i.client.Index(i.indexName), nil
} }
func (i *MeiliIndexer) Reset() error {
if i.client != nil {
taskInfo, err := i.client.DeleteIndex(i.indexName)
if err != nil {
return fmt.Errorf("failed to delete Meilisearch index: %w", err)
}
_, err = i.client.WaitForTask(taskInfo.TaskUID, 0)
if err != nil {
return fmt.Errorf("failed to wait for Meilisearch index deletion: %w", err)
}
log.Info().Msg("Meilisearch index deleted, re-creating index")
}
return i.Init()
}
func (i *MeiliIndexer) Close() { func (i *MeiliIndexer) Close() {
if i.client != nil { if i.client != nil {
i.client.Close() i.client.Close()
@@ -80,12 +99,21 @@ func (i *MeiliIndexer) Close() {
i.client = nil i.client = nil
} }
type meiliGist struct {
Gist
ContentSplit string
}
func (i *MeiliIndexer) Add(gist *Gist) error { func (i *MeiliIndexer) Add(gist *Gist) error {
if gist == nil { if gist == nil {
return errors.New("failed to add nil gist to index") return errors.New("failed to add nil gist to index")
} }
doc := &meiliGist{
Gist: *gist,
ContentSplit: splitCamelCase(gist.Content),
}
primaryKey := "GistID" primaryKey := "GistID"
_, err := (*atomicIndexer.Load()).(*MeiliIndexer).index.AddDocuments(gist, &meilisearch.DocumentOptions{PrimaryKey: &primaryKey}) _, err := (*atomicIndexer.Load()).(*MeiliIndexer).index.AddDocuments(doc, &meilisearch.DocumentOptions{PrimaryKey: &primaryKey})
return err return err
} }
@@ -94,13 +122,14 @@ func (i *MeiliIndexer) Remove(gistID uint) error {
return err return err
} }
func (i *MeiliIndexer) Search(queryStr string, queryMetadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error) { func (i *MeiliIndexer) Search(queryMetadata SearchGistMetadata, userId uint, page int) ([]uint, uint64, map[string]int, error) {
searchRequest := &meilisearch.SearchRequest{ searchRequest := &meilisearch.SearchRequest{
Offset: int64((page - 1) * 10), Offset: int64((page - 1) * 10),
Limit: 11, Limit: 11,
AttributesToRetrieve: []string{"GistID", "Languages"}, AttributesToRetrieve: []string{"GistID", "Languages"},
Facets: []string{"Languages"}, Facets: []string{"Languages"},
AttributesToSearchOn: []string{"Content"}, AttributesToSearchOn: []string{"Content", "ContentSplit"},
MatchingStrategy: meilisearch.All,
} }
var filters []string var filters []string
@@ -111,23 +140,83 @@ func (i *MeiliIndexer) Search(queryStr string, queryMetadata SearchGistMetadata,
filters = append(filters, fmt.Sprintf("%s = \"%s\"", field, escapeFilterValue(value))) filters = append(filters, fmt.Sprintf("%s = \"%s\"", field, escapeFilterValue(value)))
} }
} }
var query string
if queryMetadata.All != "" {
query = queryMetadata.All
searchRequest.AttributesToSearchOn = append(AllSearchFields, "ContentSplit")
} else {
// Exact-match fields stay as filters
addFilter("Username", queryMetadata.Username) addFilter("Username", queryMetadata.Username)
addFilter("Title", queryMetadata.Title) if queryMetadata.Extension != "" {
addFilter("Filenames", queryMetadata.Filename) ext := queryMetadata.Extension
addFilter("Extensions", queryMetadata.Extension) if !strings.HasPrefix(ext, ".") {
ext = "." + ext
}
addFilter("Extensions", ext)
}
addFilter("Languages", queryMetadata.Language) addFilter("Languages", queryMetadata.Language)
addFilter("Topics", queryMetadata.Topic) addFilter("Topics", queryMetadata.Topic)
if queryMetadata.Default != "" {
query = queryMetadata.Default
var fields []string
for _, f := range strings.Split(config.C.SearchDefault, ",") {
f = strings.TrimSpace(f)
if f == "all" {
fields = AllSearchFields
break
}
if indexField, ok := SearchFieldMap[f]; ok {
fields = append(fields, indexField)
}
}
if len(fields) > 0 {
for _, f := range fields {
if f == "Content" {
fields = append(fields, "ContentSplit")
break
}
}
searchRequest.AttributesToSearchOn = fields
}
} else {
// Fuzzy-matchable fields become part of the query
var queryParts []string
var searchFields []string
if queryMetadata.Content != "" {
queryParts = append(queryParts, queryMetadata.Content)
searchFields = append(searchFields, "Content", "ContentSplit")
}
if queryMetadata.Title != "" {
queryParts = append(queryParts, queryMetadata.Title)
searchFields = append(searchFields, "Title")
}
if queryMetadata.Description != "" {
queryParts = append(queryParts, queryMetadata.Description)
searchFields = append(searchFields, "Description")
}
if queryMetadata.Filename != "" {
queryParts = append(queryParts, queryMetadata.Filename)
searchFields = append(searchFields, "Filenames")
}
query = strings.Join(queryParts, " ")
if len(searchFields) > 0 {
searchRequest.AttributesToSearchOn = searchFields
}
}
}
if len(filters) > 0 { if len(filters) > 0 {
searchRequest.Filter = strings.Join(filters, " AND ") searchRequest.Filter = strings.Join(filters, " AND ")
} }
response, err := (*atomicIndexer.Load()).(*MeiliIndexer).index.Search(queryStr, searchRequest) response, err := (*atomicIndexer.Load()).(*MeiliIndexer).index.Search(query, searchRequest)
if err != nil { if err != nil {
log.Error().Err(err).Msg("Failed to search Meilisearch index") log.Error().Err(err).Msg("Failed to search Meilisearch index")
return nil, 0, nil, err return nil, 0, nil, err
} }
gistIds := make([]uint, 0, len(response.Hits)) gistIds := make([]uint, 0, len(response.Hits))
for _, hit := range response.Hits { for _, hit := range response.Hits {
if gistIDRaw, ok := hit["GistID"]; ok { if gistIDRaw, ok := hit["GistID"]; ok {
@@ -143,7 +232,9 @@ func (i *MeiliIndexer) Search(queryStr string, queryMetadata SearchGistMetadata,
var facetDist map[string]map[string]int var facetDist map[string]map[string]int
if err := json.Unmarshal(response.FacetDistribution, &facetDist); err == nil { if err := json.Unmarshal(response.FacetDistribution, &facetDist); err == nil {
if facets, ok := facetDist["Languages"]; ok { if facets, ok := facetDist["Languages"]; ok {
languageCounts = facets for lang, count := range facets {
languageCounts[strings.ToLower(lang)] += count
}
} }
} }
} }
@@ -151,6 +242,30 @@ func (i *MeiliIndexer) Search(queryStr string, queryMetadata SearchGistMetadata,
return gistIds, uint64(response.EstimatedTotalHits), languageCounts, nil return gistIds, uint64(response.EstimatedTotalHits), languageCounts, nil
} }
func splitCamelCase(text string) string {
var result strings.Builder
runes := []rune(text)
for i := 0; i < len(runes); i++ {
r := runes[i]
if i > 0 {
prev := runes[i-1]
if unicode.IsUpper(r) {
if unicode.IsLower(prev) || unicode.IsDigit(prev) {
result.WriteRune(' ')
} else if unicode.IsUpper(prev) && i+1 < len(runes) && unicode.IsLower(runes[i+1]) {
result.WriteRune(' ')
}
} else if unicode.IsDigit(r) && !unicode.IsDigit(prev) {
result.WriteRune(' ')
} else if !unicode.IsDigit(r) && unicode.IsDigit(prev) {
result.WriteRune(' ')
}
}
result.WriteRune(r)
}
return result.String()
}
func escapeFilterValue(value string) string { func escapeFilterValue(value string) string {
escaped := strings.ReplaceAll(value, "\\", "\\\\") escaped := strings.ReplaceAll(value, "\\", "\\\\")
escaped = strings.ReplaceAll(escaped, "\"", "\\\"") escaped = strings.ReplaceAll(escaped, "\"", "\\\"")

View File

@@ -0,0 +1,88 @@
package index
import (
"fmt"
"os"
"strconv"
"testing"
"github.com/meilisearch/meilisearch-go"
"github.com/rs/zerolog"
)
// syncMeiliIndexer wraps MeiliIndexer to make Add/Remove synchronous for tests.
type syncMeiliIndexer struct {
*MeiliIndexer
}
func (s *syncMeiliIndexer) Add(gist *Gist) error {
if gist == nil {
return fmt.Errorf("failed to add nil gist to index")
}
doc := &meiliGist{
Gist: *gist,
ContentSplit: splitCamelCase(gist.Content),
}
primaryKey := "GistID"
taskInfo, err := s.index.AddDocuments(doc, &meilisearch.DocumentOptions{PrimaryKey: &primaryKey})
if err != nil {
return err
}
_, err = s.client.WaitForTask(taskInfo.TaskUID, 0)
return err
}
func (s *syncMeiliIndexer) Remove(gistID uint) error {
taskInfo, err := s.index.DeleteDocument(strconv.Itoa(int(gistID)), nil)
if err != nil {
return err
}
_, err = s.client.WaitForTask(taskInfo.TaskUID, 0)
return err
}
func setupMeiliIndexer(t *testing.T) (Indexer, func()) {
zerolog.SetGlobalLevel(zerolog.Disabled)
t.Helper()
host := os.Getenv("OG_TEST_MEILI_HOST")
if host == "" {
host = "http://localhost:47700"
}
apiKey := os.Getenv("OG_TEST_MEILI_API_KEY")
indexName := fmt.Sprintf("test_%d", os.Getpid())
inner := NewMeiliIndexer(host, apiKey, indexName)
err := inner.Init()
if err != nil {
t.Skipf("MeiliSearch not available at %s: %v", host, err)
}
wrapped := &syncMeiliIndexer{MeiliIndexer: inner}
// Store the inner MeiliIndexer in atomicIndexer, because MeiliIndexer.Search
// type-asserts the global to *MeiliIndexer.
var idx Indexer = inner
atomicIndexer.Store(&idx)
cleanup := func() {
atomicIndexer.Store(nil)
inner.Reset()
inner.Close()
}
return wrapped, cleanup
}
func TestMeiliAddAndSearch(t *testing.T) { testAddAndSearch(t, setupMeiliIndexer) }
func TestMeiliAccessControl(t *testing.T) { testAccessControl(t, setupMeiliIndexer) }
func TestMeiliMetadataFilters(t *testing.T) { testMetadataFilters(t, setupMeiliIndexer) }
func TestMeiliAllFieldSearch(t *testing.T) { testAllFieldSearch(t, setupMeiliIndexer) }
func TestMeiliFuzzySearch(t *testing.T) { testFuzzySearch(t, setupMeiliIndexer) }
func TestMeiliContentSearch(t *testing.T) { testContentSearch(t, setupMeiliIndexer) }
func TestMeiliPagination(t *testing.T) { testPagination(t, setupMeiliIndexer) }
func TestMeiliLanguageFacets(t *testing.T) { testLanguageFacets(t, setupMeiliIndexer) }
func TestMeiliMetadataOnlySearch(t *testing.T) { testMetadataOnlySearch(t, setupMeiliIndexer) }
func TestMeiliTitleFuzzySearch(t *testing.T) { testTitleFuzzySearch(t, setupMeiliIndexer) }
func TestMeiliMultiLanguageFacets(t *testing.T) { testMultiLanguageFacets(t, setupMeiliIndexer) }

View File

@@ -164,6 +164,18 @@ func AllGists(ctx *context.Context) error {
return ctx.Html("all.html") return ctx.Html("all.html")
} }
// Search handles the search page for gists.
//
// It takes a query parameter "q" which is a search query in the format:
// "user:username title:title description:description filename:filename language:language topic:topic"
//
// It also takes a page parameter "page" which is the page number to display.
//
// It returns an error if the search query is invalid or if the page number is invalid.
//
// It returns the search results as a list of rendered gists, along with the total number of results, the languages found, and the search query.
//
// The search results are paginated, with 10 results per page.
func Search(ctx *context.Context) error { func Search(ctx *context.Context) error {
var err error var err error
@@ -171,7 +183,7 @@ func Search(ctx *context.Context) error {
Query: ctx.QueryParam("q"), Query: ctx.QueryParam("q"),
} }
content, meta := handlers.ParseSearchQueryStr(ctx.QueryParam("q")) metadata := handlers.ParseSearchQueryStr(ctx.QueryParam("q"))
pageInt := handlers.GetPage(ctx) pageInt := handlers.GetPage(ctx)
var currentUserId uint var currentUserId uint
@@ -182,14 +194,18 @@ func Search(ctx *context.Context) error {
currentUserId = 0 currentUserId = 0
} }
gistsIds, nbHits, langs, err := index.SearchGists(content, index.SearchGistMetadata{ // Search gists in the index and fetch the gists IDs from the database
Username: meta["user"], gistsIds, nbHits, langs, err := index.SearchGists(index.SearchGistMetadata{
Title: meta["title"], Username: metadata["user"],
Filename: meta["filename"], Title: metadata["title"],
Extension: meta["extension"], Description: metadata["description"],
Language: meta["language"], Filename: metadata["filename"],
Topic: meta["topic"], Extension: metadata["extension"],
All: meta["all"], Language: metadata["language"],
Topic: metadata["topic"],
Content: metadata["content"],
All: metadata["all"],
Default: metadata["default"],
}, currentUserId, pageInt) }, currentUserId, pageInt)
if err != nil { if err != nil {
return ctx.ErrorRes(500, "Error searching gists", err) return ctx.ErrorRes(500, "Error searching gists", err)

View File

@@ -8,7 +8,7 @@ import (
"strings" "strings"
"github.com/gorilla/schema" "github.com/gorilla/schema"
"github.com/rs/zerolog/log"
"github.com/thomiceli/opengist/internal/web/context" "github.com/thomiceli/opengist/internal/web/context"
) )
@@ -119,10 +119,16 @@ func Paginate[T any](ctx *context.Context, data []*T, pageInt int, perPage int,
return nil return nil
} }
func ParseSearchQueryStr(query string) (string, map[string]string) { // ParseSearchQueryStr parses a search query string and returns a map of metadata.
// The query string is split into words and each word is checked if it contains a colon (:).
// If a word contains a colon, it is split into a key-value pair and added to the metadata map.
// If a word does not contain a colon, it is added to an "all" key in the metadata map.
// The "all" key is used to search all fields in the index.
// The function returns the metadata map.
func ParseSearchQueryStr(query string) map[string]string {
words := strings.Fields(query) words := strings.Fields(query)
metadata := make(map[string]string) metadata := make(map[string]string)
var contentBuilder strings.Builder var allFieldsBuilder strings.Builder
for _, word := range words { for _, word := range words {
if strings.Contains(word, ":") { if strings.Contains(word, ":") {
@@ -133,10 +139,18 @@ func ParseSearchQueryStr(query string) (string, map[string]string) {
metadata[key] = value metadata[key] = value
} }
} else { } else {
contentBuilder.WriteString(word + " ") // Add to content search by default
allFieldsBuilder.WriteString(word + " ")
} }
} }
content := strings.TrimSpace(contentBuilder.String()) // Set the default search field
return content, metadata allContent := strings.TrimSpace(allFieldsBuilder.String())
if allContent != "" {
metadata["default"] = allContent
}
log.Debug().Msgf("Metadata: %v", metadata)
return metadata
} }

View File

@@ -147,7 +147,10 @@ func (s *Server) setFuncMap() {
return dict, nil return dict, nil
}, },
"addMetadataToSearchQuery": func(input, key, value string) string { "addMetadataToSearchQuery": func(input, key, value string) string {
content, metadata := handlers.ParseSearchQueryStr(input) metadata := handlers.ParseSearchQueryStr(input)
// extract free-text content (stored under "all") and remove it from metadata
content := metadata["all"]
delete(metadata, "all")
metadata[key] = value metadata[key] = value

113
package-lock.json generated
View File

@@ -7,22 +7,22 @@
"name": "opengist", "name": "opengist",
"devDependencies": { "devDependencies": {
"@catppuccin/highlightjs": "^1.0.1", "@catppuccin/highlightjs": "^1.0.1",
"@codemirror/commands": "^6.10.1", "@codemirror/commands": "^6.10.3",
"@codemirror/lang-javascript": "^6.2.4", "@codemirror/lang-javascript": "^6.2.5",
"@codemirror/language": "^6.12.2", "@codemirror/language": "^6.12.2",
"@codemirror/state": "^6.5.4", "@codemirror/state": "^6.5.4",
"@codemirror/text": "^0.19.6", "@codemirror/text": "^0.19.6",
"@codemirror/view": "^6.39.11", "@codemirror/view": "^6.40.0",
"@tailwindcss/forms": "^0.5.11", "@tailwindcss/forms": "^0.5.11",
"@tailwindcss/typography": "^0.5.19", "@tailwindcss/typography": "^0.5.19",
"@tailwindcss/vite": "^4.2.1", "@tailwindcss/vite": "^4.2.1",
"codemirror": "^6.0.2", "codemirror": "^6.0.2",
"github-markdown-css": "^5.8.1", "github-markdown-css": "^5.9.0",
"highlight.js": "^11.11.1", "highlight.js": "^11.11.1",
"jdenticon": "^3.3.0", "jdenticon": "^3.3.0",
"katex": "^0.16.33", "katex": "^0.16.38",
"marked": "^17.0.3", "marked": "^17.0.4",
"nodemon": "^3.1.11", "nodemon": "^3.1.14",
"pdfobject": "^2.3.1", "pdfobject": "^2.3.1",
"tailwindcss": "^4.2.1", "tailwindcss": "^4.2.1",
"vite": "^7.3.1" "vite": "^7.3.1"
@@ -65,22 +65,22 @@
} }
}, },
"node_modules/@codemirror/commands": { "node_modules/@codemirror/commands": {
"version": "6.10.1", "version": "6.10.3",
"resolved": "https://registry.npmjs.org/@codemirror/commands/-/commands-6.10.1.tgz", "resolved": "https://registry.npmjs.org/@codemirror/commands/-/commands-6.10.3.tgz",
"integrity": "sha512-uWDWFypNdQmz2y1LaNJzK7fL7TYKLeUAU0npEC685OKTF3KcQ2Vu3klIM78D7I6wGhktme0lh3CuQLv0ZCrD9Q==", "integrity": "sha512-JFRiqhKu+bvSkDLI+rUhJwSxQxYb759W5GBezE8Uc8mHLqC9aV/9aTC7yJSqCtB3F00pylrLCwnyS91Ap5ej4Q==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@codemirror/language": "^6.0.0", "@codemirror/language": "^6.0.0",
"@codemirror/state": "^6.4.0", "@codemirror/state": "^6.6.0",
"@codemirror/view": "^6.27.0", "@codemirror/view": "^6.27.0",
"@lezer/common": "^1.1.0" "@lezer/common": "^1.1.0"
} }
}, },
"node_modules/@codemirror/lang-javascript": { "node_modules/@codemirror/lang-javascript": {
"version": "6.2.4", "version": "6.2.5",
"resolved": "https://registry.npmjs.org/@codemirror/lang-javascript/-/lang-javascript-6.2.4.tgz", "resolved": "https://registry.npmjs.org/@codemirror/lang-javascript/-/lang-javascript-6.2.5.tgz",
"integrity": "sha512-0WVmhp1QOqZ4Rt6GlVGwKJN3KW7Xh4H2q8ZZNGZaP6lRdxXJzmjm4FqvmOojVj6khWJHIb9sp7U/72W7xQgqAA==", "integrity": "sha512-zD4e5mS+50htS7F+TYjBPsiIFGanfVqg4HyUz6WNFikgOPf2BgKlx+TQedI1w6n/IqRBVBbBWmGFdLB/7uxO4A==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
@@ -133,9 +133,9 @@
} }
}, },
"node_modules/@codemirror/state": { "node_modules/@codemirror/state": {
"version": "6.5.4", "version": "6.6.0",
"resolved": "https://registry.npmjs.org/@codemirror/state/-/state-6.5.4.tgz", "resolved": "https://registry.npmjs.org/@codemirror/state/-/state-6.6.0.tgz",
"integrity": "sha512-8y7xqG/hpB53l25CIoit9/ngxdfoG+fx+V3SHBrinnhOtLvKHRyAJJuHzkWrR4YXXLX8eXBsejgAAxHUOdW1yw==", "integrity": "sha512-4nbvra5R5EtiCzr9BTHiTLc+MLXK2QGiAVYMyi8PkQd3SR+6ixar/Q/01Fa21TBIDOZXgeWV4WppsQolSreAPQ==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
@@ -150,13 +150,13 @@
"license": "MIT" "license": "MIT"
}, },
"node_modules/@codemirror/view": { "node_modules/@codemirror/view": {
"version": "6.39.11", "version": "6.40.0",
"resolved": "https://registry.npmjs.org/@codemirror/view/-/view-6.39.11.tgz", "resolved": "https://registry.npmjs.org/@codemirror/view/-/view-6.40.0.tgz",
"integrity": "sha512-bWdeR8gWM87l4DB/kYSF9A+dVackzDb/V56Tq7QVrQ7rn86W0rgZFtlL3g3pem6AeGcb9NQNoy3ao4WpW4h5tQ==", "integrity": "sha512-WA0zdU7xfF10+5I3HhUUq3kqOx3KjqmtQ9lqZjfK7jtYk4G72YW9rezcSywpaUMCWOMlq+6E0pO1IWg1TNIhtg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"@codemirror/state": "^6.5.0", "@codemirror/state": "^6.6.0",
"crelt": "^1.0.6", "crelt": "^1.0.6",
"style-mod": "^4.1.0", "style-mod": "^4.1.0",
"w3c-keyname": "^2.2.4" "w3c-keyname": "^2.2.4"
@@ -1402,11 +1402,14 @@
} }
}, },
"node_modules/balanced-match": { "node_modules/balanced-match": {
"version": "1.0.2", "version": "4.0.4",
"resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-1.0.2.tgz", "resolved": "https://registry.npmjs.org/balanced-match/-/balanced-match-4.0.4.tgz",
"integrity": "sha512-3oSeUO0TMV67hN1AmbXsK4yaqU7tjiHlbxRDZOpH0KW9+CeX4bRAaX0Anxt0tx2MrpRpWwQaPwIlISEJhYU5Pw==", "integrity": "sha512-BLrgEcRTwX2o6gGxGOCNyMvGSp35YofuYzw9h1IMTRmKqttAZZVU67bdb9Pr2vUHA8+j3i2tJfjO6C6+4myGTA==",
"dev": true, "dev": true,
"license": "MIT" "license": "MIT",
"engines": {
"node": "18 || 20 || >=22"
}
}, },
"node_modules/binary-extensions": { "node_modules/binary-extensions": {
"version": "2.3.0", "version": "2.3.0",
@@ -1422,14 +1425,16 @@
} }
}, },
"node_modules/brace-expansion": { "node_modules/brace-expansion": {
"version": "1.1.12", "version": "5.0.4",
"resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz", "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-5.0.4.tgz",
"integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==", "integrity": "sha512-h+DEnpVvxmfVefa4jFbCf5HdH5YMDXRsmKflpf1pILZWRFlTbJpxeU55nJl4Smt5HQaGzg1o6RHFPJaOqnmBDg==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"balanced-match": "^1.0.0", "balanced-match": "^4.0.2"
"concat-map": "0.0.1" },
"engines": {
"node": "18 || 20 || >=22"
} }
}, },
"node_modules/braces": { "node_modules/braces": {
@@ -1496,13 +1501,6 @@
"@codemirror/view": "^6.0.0" "@codemirror/view": "^6.0.0"
} }
}, },
"node_modules/concat-map": {
"version": "0.0.1",
"resolved": "https://registry.npmjs.org/concat-map/-/concat-map-0.0.1.tgz",
"integrity": "sha512-/Srv4dswyQNBfohGpz9o6Yb3Gz3SrUDqBH5rTuhGR7ahtlbYKnVxw2bCFMRljaA7EXHaXZ8wsHdodFvbkhKmqg==",
"dev": true,
"license": "MIT"
},
"node_modules/crelt": { "node_modules/crelt": {
"version": "1.0.6", "version": "1.0.6",
"resolved": "https://registry.npmjs.org/crelt/-/crelt-1.0.6.tgz", "resolved": "https://registry.npmjs.org/crelt/-/crelt-1.0.6.tgz",
@@ -1636,9 +1634,9 @@
} }
}, },
"node_modules/github-markdown-css": { "node_modules/github-markdown-css": {
"version": "5.8.1", "version": "5.9.0",
"resolved": "https://registry.npmjs.org/github-markdown-css/-/github-markdown-css-5.8.1.tgz", "resolved": "https://registry.npmjs.org/github-markdown-css/-/github-markdown-css-5.9.0.tgz",
"integrity": "sha512-8G+PFvqigBQSWLQjyzgpa2ThD9bo7+kDsriUIidGcRhXgmcaAWUIpCZf8DavJgc+xifjbCG+GvMyWr0XMXmc7g==", "integrity": "sha512-tmT5sY+zvg2302XLYEfH2mtkViIM1SWf2nvYoF5N1ZsO0V6B2qZTiw3GOzw4vpjLygK/KG35qRlPFweHqfzz5w==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"engines": { "engines": {
@@ -1768,9 +1766,9 @@
} }
}, },
"node_modules/katex": { "node_modules/katex": {
"version": "0.16.33", "version": "0.16.38",
"resolved": "https://registry.npmjs.org/katex/-/katex-0.16.33.tgz", "resolved": "https://registry.npmjs.org/katex/-/katex-0.16.38.tgz",
"integrity": "sha512-q3N5u+1sY9Bu7T4nlXoiRBXWfwSefNGoKeOwekV+gw0cAXQlz2Ww6BLcmBxVDeXBMUDQv6fK5bcNaJLxob3ZQA==", "integrity": "sha512-cjHooZUmIAUmDsHBN+1n8LaZdpmbj03LtYeYPyuYB7OuloiaeaV6N4LcfjcnHVzGWjVQmKrxxTrpDcmSzEZQwQ==",
"dev": true, "dev": true,
"funding": [ "funding": [
"https://opencollective.com/katex", "https://opencollective.com/katex",
@@ -2066,9 +2064,9 @@
} }
}, },
"node_modules/marked": { "node_modules/marked": {
"version": "17.0.3", "version": "17.0.4",
"resolved": "https://registry.npmjs.org/marked/-/marked-17.0.3.tgz", "resolved": "https://registry.npmjs.org/marked/-/marked-17.0.4.tgz",
"integrity": "sha512-jt1v2ObpyOKR8p4XaUJVk3YWRJ5n+i4+rjQopxvV32rSndTJXvIzuUdWWIy/1pFQMkQmvTXawzDNqOH/CUmx6A==", "integrity": "sha512-NOmVMM+KAokHMvjWmC5N/ZOvgmSWuqJB8FoYI019j4ogb/PeRMKoKIjReZ2w3376kkA8dSJIP8uD993Kxc0iRQ==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"bin": { "bin": {
@@ -2089,16 +2087,19 @@
} }
}, },
"node_modules/minimatch": { "node_modules/minimatch": {
"version": "3.1.2", "version": "10.2.4",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.2.tgz", "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-10.2.4.tgz",
"integrity": "sha512-J7p63hRiAjw1NDEww1W7i37+ByIrOWO5XQQAzZ3VOcL0PNybwpfmV/N05zFAzwQ9USyEcX6t3UO+K5aqBQOIHw==", "integrity": "sha512-oRjTw/97aTBN0RHbYCdtF1MQfvusSIBQM0IZEgzl6426+8jSC0nF1a/GmnVLpfB9yyr6g6FTqWqiZVbxrtaCIg==",
"dev": true, "dev": true,
"license": "ISC", "license": "BlueOak-1.0.0",
"dependencies": { "dependencies": {
"brace-expansion": "^1.1.7" "brace-expansion": "^5.0.2"
}, },
"engines": { "engines": {
"node": "*" "node": "18 || 20 || >=22"
},
"funding": {
"url": "https://github.com/sponsors/isaacs"
} }
}, },
"node_modules/ms": { "node_modules/ms": {
@@ -2128,16 +2129,16 @@
} }
}, },
"node_modules/nodemon": { "node_modules/nodemon": {
"version": "3.1.11", "version": "3.1.14",
"resolved": "https://registry.npmjs.org/nodemon/-/nodemon-3.1.11.tgz", "resolved": "https://registry.npmjs.org/nodemon/-/nodemon-3.1.14.tgz",
"integrity": "sha512-is96t8F/1//UHAjNPHpbsNY46ELPpftGUoSVNXwUfMk/qdjSylYrWSu1XavVTBOn526kFiOR733ATgNBCQyH0g==", "integrity": "sha512-jakjZi93UtB3jHMWsXL68FXSAosbLfY0In5gtKq3niLSkrWznrVBzXFNOEMJUfc9+Ke7SHWoAZsiMkNP3vq6Jw==",
"dev": true, "dev": true,
"license": "MIT", "license": "MIT",
"dependencies": { "dependencies": {
"chokidar": "^3.5.2", "chokidar": "^3.5.2",
"debug": "^4", "debug": "^4",
"ignore-by-default": "^1.0.1", "ignore-by-default": "^1.0.1",
"minimatch": "^3.1.2", "minimatch": "^10.2.1",
"pstree.remy": "^1.1.8", "pstree.remy": "^1.1.8",
"semver": "^7.5.3", "semver": "^7.5.3",
"simple-update-notifier": "^2.0.0", "simple-update-notifier": "^2.0.0",

View File

@@ -12,22 +12,22 @@
"type": "module", "type": "module",
"devDependencies": { "devDependencies": {
"@catppuccin/highlightjs": "^1.0.1", "@catppuccin/highlightjs": "^1.0.1",
"@codemirror/commands": "^6.10.1", "@codemirror/commands": "^6.10.3",
"@codemirror/lang-javascript": "^6.2.4", "@codemirror/lang-javascript": "^6.2.5",
"@codemirror/language": "^6.12.2", "@codemirror/language": "^6.12.2",
"@codemirror/state": "^6.5.4", "@codemirror/state": "^6.5.4",
"@codemirror/text": "^0.19.6", "@codemirror/text": "^0.19.6",
"@codemirror/view": "^6.39.11", "@codemirror/view": "^6.40.0",
"@tailwindcss/forms": "^0.5.11", "@tailwindcss/forms": "^0.5.11",
"@tailwindcss/typography": "^0.5.19", "@tailwindcss/typography": "^0.5.19",
"@tailwindcss/vite": "^4.2.1", "@tailwindcss/vite": "^4.2.1",
"codemirror": "^6.0.2", "codemirror": "^6.0.2",
"github-markdown-css": "^5.8.1", "github-markdown-css": "^5.9.0",
"highlight.js": "^11.11.1", "highlight.js": "^11.11.1",
"jdenticon": "^3.3.0", "jdenticon": "^3.3.0",
"katex": "^0.16.33", "katex": "^0.16.38",
"marked": "^17.0.3", "marked": "^17.0.4",
"nodemon": "^3.1.11", "nodemon": "^3.1.14",
"pdfobject": "^2.3.1", "pdfobject": "^2.3.1",
"tailwindcss": "^4.2.1", "tailwindcss": "^4.2.1",
"vite": "^7.3.1" "vite": "^7.3.1"

View File

@@ -116,10 +116,12 @@
<div class="p-4 text-xs space-y-1"> <div class="p-4 text-xs space-y-1">
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">user:thomas</code> {{ .locale.Tr "gist.search.help.user" }}</p> <p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">user:thomas</code> {{ .locale.Tr "gist.search.help.user" }}</p>
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">title:mygist</code> {{ .locale.Tr "gist.search.help.title" }}</p> <p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">title:mygist</code> {{ .locale.Tr "gist.search.help.title" }}</p>
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">description:sync</code> {{ .locale.Tr "gist.search.help.description" }}</p>
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">filename:myfile.txt</code> {{ .locale.Tr "gist.search.help.filename" }}</p> <p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">filename:myfile.txt</code> {{ .locale.Tr "gist.search.help.filename" }}</p>
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">extension:yml</code> {{ .locale.Tr "gist.search.help.extension" }}</p> <p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">extension:yml</code> {{ .locale.Tr "gist.search.help.extension" }}</p>
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">language:go</code> {{ .locale.Tr "gist.search.help.language" }}</p> <p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">language:go</code> {{ .locale.Tr "gist.search.help.language" }}</p>
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">topic:homelab</code> {{ .locale.Tr "gist.search.help.topic" }}</p> <p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">topic:homelab</code> {{ .locale.Tr "gist.search.help.topic" }}</p>
<p class="text-gray-400"><code class="text-slate-800 dark:text-slate-300 pr-1">all:systemctl</code> {{ .locale.Tr "gist.search.help.all" }}</p>
</div> </div>
</div> </div>
</div> </div>

277
test.md
View File

@@ -1,277 +0,0 @@
---
description: Testing handler and middleware
slug: /testing
sidebar_position: 13
---
# Testing
## Testing Handler
`GET` `/users/:id`
Handler below retrieves user by id from the database. If user is not found it returns
`404` error with a message.
### CreateUser
`POST` `/users`
- Accepts JSON payload
- On success `201 - Created`
- On error `500 - Internal Server Error`
### GetUser
`GET` `/users/:email`
- On success `200 - OK`
- On error `404 - Not Found` if user is not found otherwise `500 - Internal Server Error`
`handler.go`
```go
package handler
import (
"net/http"
"github.com/labstack/echo/v5"
)
type (
User struct {
Name string `json:"name" form:"name"`
Email string `json:"email" form:"email"`
}
handler struct {
db map[string]*User
}
)
func (h *handler) createUser(c *echo.Context) error {
u := new(User)
if err := c.Bind(u); err != nil {
return err
}
return c.JSON(http.StatusCreated, u)
}
func (h *handler) getUser(c *echo.Context) error {
email := c.Param("email")
user := h.db[email]
if user == nil {
return echo.NewHTTPError(http.StatusNotFound, "user not found")
}
return c.JSON(http.StatusOK, user)
}
```
`handler_test.go`
```go
package handler
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/labstack/echo/v5"
"github.com/labstack/echo/v5/echotest"
"github.com/stretchr/testify/assert"
)
var (
mockDB = map[string]*User{
"jon@labstack.com": &User{"Jon Snow", "jon@labstack.com"},
}
userJSON = `{"name":"Jon Snow","email":"jon@labstack.com"}`
)
func TestCreateUser(t *testing.T) {
// Setup
e := echo.New()
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(userJSON))
req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
h := &controller{mockDB}
// Assertions
if assert.NoError(t, h.createUser(c)) {
assert.Equal(t, http.StatusCreated, rec.Code)
assert.Equal(t, userJSON, rec.Body.String())
}
}
// Same test as above but using `echotest` package helpers
func TestCreateUserWithEchoTest(t *testing.T) {
c, rec := echotest.ContextConfig{
Headers: map[string][]string{
echo.HeaderContentType: {echo.MIMEApplicationJSON},
},
JSONBody: []byte(`{"name":"Jon Snow","email":"jon@labstack.com"}`),
}.ToContextRecorder(t)
h := &controller{mockDB}
// Assertions
if assert.NoError(t, h.createUser(c)) {
assert.Equal(t, http.StatusCreated, rec.Code)
assert.Equal(t, userJSON+"\n", rec.Body.String())
}
}
// Same test as above but even shorter
func TestCreateUserWithEchoTest2(t *testing.T) {
h := &controller{mockDB}
rec := echotest.ContextConfig{
Headers: map[string][]string{
echo.HeaderContentType: {echo.MIMEApplicationJSON},
},
JSONBody: []byte(`{"name":"Jon Snow","email":"jon@labstack.com"}`),
}.ServeWithHandler(t, h.createUser)
assert.Equal(t, http.StatusCreated, rec.Code)
assert.Equal(t, userJSON+"\n", rec.Body.String())
}
func TestGetUser(t *testing.T) {
// Setup
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/", nil)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
c.SetPath("/users/:email")
c.SetPathValues(echo.PathValues{
{Name: "email", Value: "jon@labstack.com"},
})
h := &controller{mockDB}
// Assertions
if assert.NoError(t, h.getUser(c)) {
assert.Equal(t, http.StatusOK, rec.Code)
assert.Equal(t, userJSON, rec.Body.String())
}
}
func TestGetUserWithEchoTest(t *testing.T) {
c, rec := echotest.ContextConfig{
PathValues: echo.PathValues{
{Name: "email", Value: "jon@labstack.com"},
},
Headers: map[string][]string{
echo.HeaderContentType: {echo.MIMEApplicationJSON},
},
JSONBody: []byte(userJSON),
}.ToContextRecorder(t)
h := &controller{mockDB}
// Assertions
if assert.NoError(t, h.getUser(c)) {
assert.Equal(t, http.StatusOK, rec.Code)
assert.Equal(t, userJSON+"\n", rec.Body.String())
}
}
```
### Using Form Payload
```go
// import "net/url"
f := make(url.Values)
f.Set("name", "Jon Snow")
f.Set("email", "jon@labstack.com")
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(f.Encode()))
req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationForm)
```
Multipart form payload:
```go
func TestContext_MultipartForm(t *testing.T) {
testConf := echotest.ContextConfig{
MultipartForm: &echotest.MultipartForm{
Fields: map[string]string{
"key": "value",
},
Files: []echotest.MultipartFormFile{
{
Fieldname: "file",
Filename: "test.json",
Content: echotest.LoadBytes(t, "testdata/test.json"),
},
},
},
}
c := testConf.ToContext(t)
assert.Equal(t, "value", c.FormValue("key"))
assert.Equal(t, http.MethodPost, c.Request().Method)
assert.Equal(t, true, strings.HasPrefix(c.Request().Header.Get(echo.HeaderContentType), "multipart/form-data; boundary="))
fv, err := c.FormFile("file")
if err != nil {
t.Fatal(err)
}
assert.Equal(t, "test.json", fv.Filename)
}
```
### Setting Path Params
```go
c.SetPathValues(echo.PathValues{
{Name: "id", Value: "1"},
{Name: "email", Value: "jon@labstack.com"},
})
```
### Setting Query Params
```go
// import "net/url"
q := make(url.Values)
q.Set("email", "jon@labstack.com")
req := httptest.NewRequest(http.MethodGet, "/?"+q.Encode(), nil)
```
## Testing Middleware
```go
func TestCreateUserWithEchoTest2(t *testing.T) {
handler := func(c *echo.Context) error {
return c.JSON(http.StatusTeapot, fmt.Sprintf("email: %s", c.Param("email")))
}
middleware := func(next echo.HandlerFunc) echo.HandlerFunc {
return func(c *echo.Context) error {
c.Set("user_id", int64(1234))
return next(c)
}
}
c, rec := echotest.ContextConfig{
PathValues: echo.PathValues{{Name: "email", Value: "jon@labstack.com"}},
}.ToContextRecorder(t)
err := middleware(handler)(c)
if err != nil {
t.Fatal(err)
}
// check that middleware set the value
userID, err := echo.ContextGet[int64](c, "user_id")
assert.NoError(t, err)
assert.Equal(t, int64(1234), userID)
// check that handler returned the correct response
assert.Equal(t, http.StatusTeapot, rec.Code)
}
```
For now you can look into built-in middleware [test cases](https://github.com/labstack/echo/tree/master/middleware).

158
test2.md
View File

@@ -1,158 +0,0 @@
---
description: Testing handler and middleware
slug: /testing
sidebar_position: 13
---
# Testing
## Testing Handler
`GET` `/users/:id`
Handler below retrieves user by id from the database. If user is not found it returns
`404` error with a message.
### CreateUser
`POST` `/users`
- Accepts JSON payload
- On success `201 - Created`
- On error `500 - Internal Server Error`
### GetUser
`GET` `/users/:email`
- On success `200 - OK`
- On error `404 - Not Found` if user is not found otherwise `500 - Internal Server Error`
`handler.go`
```go
package handler
import (
"net/http"
"github.com/labstack/echo/v4"
)
type (
User struct {
Name string `json:"name" form:"name"`
Email string `json:"email" form:"email"`
}
handler struct {
db map[string]*User
}
)
func (h *handler) createUser(c echo.Context) error {
u := new(User)
if err := c.Bind(u); err != nil {
return err
}
return c.JSON(http.StatusCreated, u)
}
func (h *handler) getUser(c echo.Context) error {
email := c.Param("email")
user := h.db[email]
if user == nil {
return echo.NewHTTPError(http.StatusNotFound, "user not found")
}
return c.JSON(http.StatusOK, user)
}
```
`handler_test.go`
```go
package handler
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
"github.com/labstack/echo/v4"
"github.com/stretchr/testify/assert"
)
var (
mockDB = map[string]*User{
"jon@labstack.com": &User{"Jon Snow", "jon@labstack.com"},
}
userJSON = `{"name":"Jon Snow","email":"jon@labstack.com"}`
)
func TestCreateUser(t *testing.T) {
// Setup
e := echo.New()
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(userJSON))
req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationJSON)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
h := &handler{mockDB}
// Assertions
if assert.NoError(t, h.createUser(c)) {
assert.Equal(t, http.StatusCreated, rec.Code)
assert.Equal(t, userJSON, rec.Body.String())
}
}
func TestGetUser(t *testing.T) {
// Setup
e := echo.New()
req := httptest.NewRequest(http.MethodGet, "/", nil)
rec := httptest.NewRecorder()
c := e.NewContext(req, rec)
c.SetPath("/users/:email")
c.SetParamNames("email")
c.SetParamValues("jon@labstack.com")
h := &handler{mockDB}
// Assertions
if assert.NoError(t, h.getUser(c)) {
assert.Equal(t, http.StatusOK, rec.Code)
assert.Equal(t, userJSON, rec.Body.String())
}
}
```
### Using Form Payload
```go
// import "net/url"
f := make(url.Values)
f.Set("name", "Jon Snow")
f.Set("email", "jon@labstack.com")
req := httptest.NewRequest(http.MethodPost, "/", strings.NewReader(f.Encode()))
req.Header.Set(echo.HeaderContentType, echo.MIMEApplicationForm)
```
### Setting Path Params
```go
c.SetParamNames("id", "email")
c.SetParamValues("1", "jon@labstack.com")
```
### Setting Query Params
```go
// import "net/url"
q := make(url.Values)
q.Set("email", "jon@labstack.com")
req := httptest.NewRequest(http.MethodGet, "/?"+q.Encode(), nil)
```
## Testing Middleware
*TBD*
For now you can look into built-in middleware [test cases](https://github.com/labstack/echo/tree/master/middleware).