* fix(test): prevent MySQL deadlocks in parallel proxy model tests (PROJQUAY-0000)
Mark all registry proxy model test classes to run serially using
pytest-xdist group markers. These tests all use the same "quayio-cache"
organization and were causing MySQL deadlocks when run in parallel
across multiple workers with pytest -n auto.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(test): resolve Flask app naming conflict in quotaregistrysizeworker tests (PROJQUAY-0000)
Import Flask app with alias to avoid conflict with pytest 'app' fixture.
The test was using 'app.config' but 'app' resolved to a pytest fixture
definition instead of the Flask application object.
Follows the same pattern as test_securityscanningnotificationworker.py.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Brady Pratt <bpratt@redhat.com>
---------
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
* proxycache: Download blob not cached when pulling manifests with blob available locally (PROJQUAY-6708)
* Skip downloading blobs without placeholders
fixing CVE-2025-4374 by extending the create_repository method to understand if we are requesting a proxy_cache repository
added unittests for create_repository when proxy_cache.
* Revert "bug: Adding allow hidden flag while looking up for manifests (PROJQUAY-8536) (#3722)"
This reverts commit f0c153fab5.
* Revert "proxy: moving manifest check to after upstream manifest fetch (PROJQUAY-8536) (#3814)"
This reverts commit 944edd064b.
When an image is pulled by digest, a temp tag is created to prevent the manifest from being garbage collected. This is true when a manifest list is pulled by tag as well. However, if this temporary tag expires (default is 1 day for proxied organizations) and the same manifest is pulled again by digest, the system attempts to create the manifest again, leading to an integrity error because the manifest already exists in the database.
---------
Co-authored-by: shudeshp <shudeshp@redhat.com>
* registry: implements the OCI 1.1 referrers API
Migrations:
- Adds a subject column for lookup
- Adds a subject_backfilled column to track status of the backfilling
of existing manifests
- Adds a manifest_json column making use of postgres' JSONB support,
for future use.
Manifestsubjectbackfillworker: Indexes existing manifests for possible
existing subject field.
* Deprecate IGNORE_UNKNOWN_MEDIATYPES
* Cleanup
* api: add caching for get_repository method (PROJQUAY-6472)
* fixing formatting
* using pickle to serialize and deserialize objects
* adding model_cache parameter to lookup_repository func for interface and registry_proxy_model
* adding test
* formatting fix
Currently if a new manifest is created that re-uses blobs already existing in the registry but not within the namespace it's possible for that manifest to be blocked if the newly referenced blobs exceed the quota limit. The issue being that the rejection only happens after the manifest has been created and tagged. This change blocks the manifest upload before the creation of the tag and prevents that tag from appearing in the UI and being pullable.
If the manifest upload get's rejected before the tag has been created for a new manifest, a temporary tag is created outside the time machine window so it is immediately available for GC.
Garbage collect manifests no longer referenced in Quay from the
security scanner service.
Also moved quota related code from data/registry_model/ to data/model/
to avoid circular dependencies.
(PROJQUAY-5600)
Catch exception thrown when looking up tag `lifetime_start_ms` and set
`created_at` property for a manifest to None.
If `created_at` is None, don't calculate indexing SLI for manifest.
Add new parameter `allow_hidden` to `lookup_manifest_by_digest` method and set this to true on the manifest v2 endpoint.
Enables manifests to be pulled by digest, and fixes issues with recent versions of conftest being unable to push to quay.
Allows superusers to trigger a calculation of the deduplicated registry size. A superuser can go to the organization panel of the superuser page and select Calculate to queue a calculation of the registry total. The total will only be calculated when requested. Includes warning to user of increase of database load when running calculation.
Moves the resetting of child manifest temporary tags to happen on deletion instead of on push/pull. Resetting child manifest temporary tags caused issues in other portions of the code like proxy cache where temporary tags were deleted too early.
Allows for only unique blobs are counted at the namespace and repository level. Calculation includes manifest list sizes.
Add's the following internal configurations that default to true:
QUOTA_INVALIDATE_TOTALS: Invalidates calculated totals when FEATURE_QUOTA_MANAGEMENT is set to false
RESET_CHILD_MANIFEST_EXPIRATION: Resets the expiry for child manifests on push of the manifest list for immediate GC eligibility
PERMANENTLY_DELETE_TAGS: Enables features related to the permanent deletion of tags outside the configured time machine window
* API/UI: Filtering of tags API through query parameter (PROJQUAY-5362)
* Changing syntax of query param to add operation + added propagation of filtering from new UI
* added exception to return 400 on incorrect syntax
* Added tests to test filtering of /tags endpoint
* Minor fixes
The distribution spec does not require the docker-content-digest header
to be set in response to a manifest GET/HEAD request.
This changes both the proxy client and the registry proxy model to
correctly check whether a manifest is up-to-date with the upstream
registry or not when no digest header is received.
NOTE: when checking staleness against registries that do not return the
docker-content-digest header, Quay will make a GET request to the
registry and calculate the digest from the manifest itself. GET requests
usually count towards rate-limiting.
This change also sets the accept-encoding header to 'identity'. The python
requests library seems to automatically set the accept-encoding header to
'gzip'. Dockerhub ignores that header when serving blobs, but some
registries don't (namely registry.access.redhat.com). When Quay receives a
gzipped config blob (have not tested non-config blobs) for some reason it
doesn't know how to handle it. I suspect it has to do wit the fact that in
this case the content-length header will differ from the actual size of
the response body, so when Quay tries to upload the blob it cannot
correctly calculate the actual blob size, so it does a partial upload to
its object storage, which then results in a digest mismatch error
(BlobDigestMismatchException).
