mirror of
https://github.com/quay/quay.git
synced 2025-04-18 10:44:06 +03:00
a6713a669d
* storage: Enable multipart upload for Google Cloud Storage (PROJQUAY-6862) This PR removes the `_stream_write_internal` function override that caused excessive memory consumption and defaults to the old one which chunks uploads. Server assembly is still not suppored by GCS, so we have to assemble everything locally. However, GCS does support the copy function, so a reupload is not needed. ~~~ REPOSITORY TAG IMAGE ID CREATED SIZE registry.fedoraproject.org/fedora latest ecd9f7ee77f4 2 days ago 165 MB quay.skynet/ibazulic/big-mirror-test size138gb 8e6ba9ff13c0 3 days ago 148 GB quay.skynet/quay-mirror/big-mirror-test size138gb 8e6ba9ff13c0 3 days ago 148 GB quay.skynet/ibazulic/mfs-image-test latest ab14f2230dd9 7 days ago 5.96 GB quay.skynet/ibazulic/azure-storage-big-file-test latest ede194b926e0 7 days ago 16.1 GB quay.skynet/ibazulic/minio/minio latest 76ed5b96833a 6 weeks ago 532 B Getting image source signatures Copying blob 9d9c3d76c421 done | Copying blob fce7cf3b093c skipped: already exists Copying config 8e6ba9ff13 done | Writing manifest to image destination ~~~ For uploading extremely big layers, 5 MiB as the default chunk size is not enough. The PR also enables support for user-defined chunk sizes via `minimum_chunk_size_mb` and `maximum_chunk_size_mb` which default to 5 Mib and 100 MiB respectively. * Remove maximum_chunk_size_mb as it's not needed