From e0e10052e04ad1be2d1bf39da76dd6cb8dec5c23 Mon Sep 17 00:00:00 2001 From: Giteabot Date: Sun, 2 Aug 2026 17:17:13 -0700 Subject: [PATCH] fix: set a minio part size when the content size is unknown (#38753) (#38755) Backport #38753 by @Zettat123 ## Background `MinioStorage.Save` is called with `size = -1` on several paths, including Actions logs, Actions artifacts, repository archives, avatars and attachments. With an unknown size (-1) minio-go assumes a 5TiB object and allocates a single part-sized buffer of 528MiB per upload, regardless of the real payload size, which can exhaust the memory of small instances. Measured against a local S3 stub, five sequential uploads of a 4KiB payload grew RSS by 1041MiB before the fix and by 34MiB after it. ## Fix Pass an explicit 16MiB part size in that case, the same value minio-go uses as minimum part size (https://github.com/minio/minio-go/blob/v7.2.1/constants.go#L28). Uploads with a known size are left untouched, since minio-go already derives a part size proportional to the real object size. ## Note: One behaviour change: with an unknown size the object is now limited to 16MiB * 10000 parts = 156.25GiB (https://github.com/minio/minio-go/blob/v7.2.1/api-put-object-common.go#L112-L116). `putObjectMultipartStreamNoLength` completes the upload once it runs out of parts without checking that the reader was drained, so a stream past that limit is silently truncated rather than rejected. The previous limit was 5TiB. No payload Gitea uploads comes close to either. Parts are also uploaded serially, so a smaller part size means proportionally more round trips for large unknown-size uploads. Co-authored-by: Zettat123 Co-authored-by: silverwind --- modules/storage/minio.go | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/modules/storage/minio.go b/modules/storage/minio.go index 310dd6da5e..9bb569f2e9 100644 --- a/modules/storage/minio.go +++ b/modules/storage/minio.go @@ -26,6 +26,8 @@ import ( var _ ObjectStorage = &MinioStorage{} +const unknownSizePartSize = 1024 * 1024 * 16 // same as minio-go's minPartSize + type minioObject struct { *minio.Object } @@ -211,6 +213,10 @@ func (m *MinioStorage) Save(path string, r io.Reader, size int64) (int64, error) // * https://www.backblaze.com/b2/docs/s3_compatible_api.html // do not support "x-amz-checksum-algorithm" header, so use legacy MD5 checksum SendContentMd5: m.cfg.ChecksumAlgorithm == "md5", + + // with an unknown size (-1) minio-go assumes a 5TiB object and buffers a 528MiB part for it, even + // for a payload of a few KiB, so pin the part size there, a known size derives its own + PartSize: util.Iif[uint64](size < 0, unknownSizePartSize, 0), }, ) if err != nil {