1
0
mirror of https://github.com/quay/quay.git synced 2026-10-23 09:49:06 +03:00
Files
quay/go.mod
Ivan Bazulic 18d48f333e PROJQUAY-12887: fix: Decouple manifest storing in backend storage engine and make SQLite the single source of truth (#7054)
* NO-ISSUE: fix: Decouple manifest storing in backend storage engine and make SQLite the single source of truth
Previously, Distribution would write manifests to backend storage and would fall back to SQLite. This caused a split brain scenario with GC worker since GC uses data from the database as the single source of truth.

This PR does the following:
1. Adds new methods for manifest fetch that override Distribution functions and read manifests directly from the SQLite database
2. Completely removes storing of metadata in the backend storage; backend storage now only contains blob data
3. Changes the verification logic during push: no more orphaned data or placeholders (linkChild placeholder fabrication removed), manifest (be it image manifest or image index) cannot be committed unless all children and all blobs are stored in storage (follows standard Docker v2 API practice)
4. Preserves manifest children from being deleted by GC (creates temporary tags with 1 hour expiry on manifests that are pushed by digest only)
5. Removes the fallback functions that verify that manifests are stored in backend storage along with connected tests
6. Adds new tests covering new behaviour.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add queries, fix temp tag duplication issue, fix gc tests

* Correct linting errors

* Explicitly check that we do expire temporary tags, fix error messages

* Correct spelling

* Fixed final error message

* Modify schema to reflect changes, fix comments

* Upgrade sqlc to use latest 1.31.1

* Revert sqlc version change, rgenerate db files with sqlc 1.30.0 to pass CI

* Revert commit 67643d32948a74ed97c8e79076f542f79760eb7a as it breaks builds

* Set go version to 1.26 explicitly

* Wrap manifest lookup errors to properly return manifest not found to middleware

* Add lookup repo functions and test

* Add explanation about dropping support for foreign layers

* Remove references to GetManifestContent

* Reverted accidental changes to schema and seed data

* Correct linting and errors

* Add mutex to manifest/repo/blob creation, add query to scope blob validation to the proper repository

* Make sure we don't create duplicate temporary tags during pushes

* Fix assertions in registry e2e test

* fix comment to pass linting test

* Correct return error on unknown deletes

* Removed extra query clause, corrected comment, increased blob expiry to 6 hours

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-09-23 22:28:43 +02:00

93 lines
4.4 KiB
Modula-2

module github.com/quay/quay
go 1.26.0
require (
github.com/distribution/distribution/v3 v3.1.1
github.com/distribution/reference v0.6.0
github.com/go-jose/go-jose/v4 v4.1.4
github.com/google/uuid v1.6.0
github.com/jackc/pgx/v5 v5.10.0
github.com/opencontainers/go-digest v1.0.0
github.com/opencontainers/image-spec v1.1.1
github.com/prometheus/client_golang v1.23.2
github.com/prometheus/client_model v0.6.2
github.com/sirupsen/logrus v1.9.4
golang.org/x/crypto v0.49.0
gopkg.in/yaml.v3 v3.0.1
modernc.org/sqlite v1.48.1
)
require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff/v5 v5.0.3 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/docker/docker-credential-helpers v0.9.5 // indirect
github.com/docker/go-events v0.0.0-20250808211157-605354379745 // indirect
github.com/docker/go-metrics v0.0.1 // indirect
github.com/dustin/go-humanize v1.0.1 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/gorilla/handlers v1.5.2 // indirect
github.com/gorilla/mux v1.8.1 // indirect
github.com/grpc-ecosystem/grpc-gateway/v2 v2.28.0 // indirect
github.com/hashicorp/golang-lru/arc/v2 v2.0.5 // indirect
github.com/hashicorp/golang-lru/v2 v2.0.7 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20240606120523-5a60cdf6a761 // indirect
github.com/klauspost/compress v1.18.4 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/ncruces/go-strftime v1.0.0 // indirect
github.com/prometheus/common v0.67.5 // indirect
github.com/prometheus/otlptranslator v1.0.0 // indirect
github.com/prometheus/procfs v0.20.1 // indirect
github.com/redis/go-redis/extra/rediscmd/v9 v9.0.5 // indirect
github.com/redis/go-redis/extra/redisotel/v9 v9.0.5 // indirect
github.com/redis/go-redis/v9 v9.7.3 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
go.opentelemetry.io/auto/sdk v1.2.1 // indirect
go.opentelemetry.io/contrib/bridges/prometheus v0.67.0 // indirect
go.opentelemetry.io/contrib/exporters/autoexport v0.67.0 // indirect
go.opentelemetry.io/otel v1.43.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc v0.18.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp v0.19.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.42.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.43.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.42.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0 // indirect
go.opentelemetry.io/otel/exporters/prometheus v0.64.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdoutlog v0.18.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdoutmetric v1.42.0 // indirect
go.opentelemetry.io/otel/exporters/stdout/stdouttrace v1.42.0 // indirect
go.opentelemetry.io/otel/log v0.19.0 // indirect
go.opentelemetry.io/otel/metric v1.43.0 // indirect
go.opentelemetry.io/otel/sdk v1.43.0 // indirect
go.opentelemetry.io/otel/sdk/log v0.19.0 // indirect
go.opentelemetry.io/otel/sdk/metric v1.43.0 // indirect
go.opentelemetry.io/otel/trace v1.43.0 // indirect
go.opentelemetry.io/proto/otlp v1.10.0 // indirect
go.yaml.in/yaml/v2 v2.4.3 // indirect
golang.org/x/net v0.52.0 // indirect
golang.org/x/sync v0.21.0 // indirect
golang.org/x/sys v0.42.0 // indirect
golang.org/x/text v0.39.0 // indirect
google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260401024825-9d38bb4040a9 // indirect
google.golang.org/grpc v1.80.0 // indirect
google.golang.org/protobuf v1.36.11 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
modernc.org/libc v1.70.0 // indirect
modernc.org/mathutil v1.7.1 // indirect
modernc.org/memory v1.11.0 // indirect
)
require (
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/stretchr/testify v1.11.1
)