1
0
mirror of https://github.com/quay/quay.git synced 2026-10-23 09:49:06 +03:00
Files
Ivan Bazulic 18d48f333e PROJQUAY-12887: fix: Decouple manifest storing in backend storage engine and make SQLite the single source of truth (#7054)
* NO-ISSUE: fix: Decouple manifest storing in backend storage engine and make SQLite the single source of truth
Previously, Distribution would write manifests to backend storage and would fall back to SQLite. This caused a split brain scenario with GC worker since GC uses data from the database as the single source of truth.

This PR does the following:
1. Adds new methods for manifest fetch that override Distribution functions and read manifests directly from the SQLite database
2. Completely removes storing of metadata in the backend storage; backend storage now only contains blob data
3. Changes the verification logic during push: no more orphaned data or placeholders (linkChild placeholder fabrication removed), manifest (be it image manifest or image index) cannot be committed unless all children and all blobs are stored in storage (follows standard Docker v2 API practice)
4. Preserves manifest children from being deleted by GC (creates temporary tags with 1 hour expiry on manifests that are pushed by digest only)
5. Removes the fallback functions that verify that manifests are stored in backend storage along with connected tests
6. Adds new tests covering new behaviour.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* Add queries, fix temp tag duplication issue, fix gc tests

* Correct linting errors

* Explicitly check that we do expire temporary tags, fix error messages

* Correct spelling

* Fixed final error message

* Modify schema to reflect changes, fix comments

* Upgrade sqlc to use latest 1.31.1

* Revert sqlc version change, rgenerate db files with sqlc 1.30.0 to pass CI

* Revert commit 67643d32948a74ed97c8e79076f542f79760eb7a as it breaks builds

* Set go version to 1.26 explicitly

* Wrap manifest lookup errors to properly return manifest not found to middleware

* Add lookup repo functions and test

* Add explanation about dropping support for foreign layers

* Remove references to GetManifestContent

* Reverted accidental changes to schema and seed data

* Correct linting and errors

* Add mutex to manifest/repo/blob creation, add query to scope blob validation to the proper repository

* Make sure we don't create duplicate temporary tags during pushes

* Fix assertions in registry e2e test

* fix comment to pass linting test

* Correct return error on unknown deletes

* Removed extra query clause, corrected comment, increased blob expiry to 6 hours

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-09-23 22:28:43 +02:00
..