1
0
mirror of https://github.com/quay/quay.git synced 2026-01-26 06:21:37 +03:00

92 Commits

Author SHA1 Message Date
jbpratt
ce559b770d feat(data): add quay.immutable manifest label support (PROJQUAY-10161) (#4926)
Add label handler for quay.immutable manifest label that automatically
marks associated tags as immutable when images are pushed with
LABEL quay.immutable=true in their Dockerfile. Only "true" value
(case-insensitive) triggers immutability; other values are ignored.

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-01-20 10:44:37 -05:00
jbpratt
2f476b2304 fix(data): set expiry on manifest list tags for existing manifests (PROJQUAY-7245) (#4798)
When a manifest list was pushed with multiple tags, only the first tag
got the expiration from child manifest labels. Subsequent tags showed
"Never" because list_manifest_labels() only checked labels directly on
the manifest list, not on child manifests.

Added _get_expiry_label_for_manifest() helper that properly queries
child manifest labels for manifest lists, matching the intersection
logic used during initial manifest creation.

Co-authored-by: Claude <noreply@anthropic.com>
2026-01-16 14:47:02 -06:00
jbpratt
a49ccd6333 feat(endpoints): add tag immutability API layer (PROJQUAY-10159) (#4839)
Expose tag immutability through the existing tag REST API endpoint.
This adds:
- immutable field to PUT /api/v1/repository/{repo}/tag/{tag}
- TagImmutable 409 exception for blocked operations
- immutable field in tag list responses
- Exception handling for DELETE and PUT on immutable tags

Write permission required to lock, admin required to unlock.

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2026-01-16 11:09:14 -05:00
jbpratt
71d219cc35 fix(test): prevent MySQL deadlocks in parallel proxy model tests (PROJQUAY-0000) (#4605)
* fix(test): prevent MySQL deadlocks in parallel proxy model tests (PROJQUAY-0000)

Mark all registry proxy model test classes to run serially using
pytest-xdist group markers. These tests all use the same "quayio-cache"
organization and were causing MySQL deadlocks when run in parallel
across multiple workers with pytest -n auto.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(test): resolve Flask app naming conflict in quotaregistrysizeworker tests (PROJQUAY-0000)

Import Flask app with alias to avoid conflict with pytest 'app' fixture.
The test was using 'app.config' but 'app' resolved to a pytest fixture
definition instead of the Flask application object.

Follows the same pattern as test_securityscanningnotificationworker.py.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Brady Pratt <bpratt@redhat.com>

---------

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-24 15:28:34 +05:30
Brandon Caton
3f317fb74b proxy: fix err on pulls from public repos if anonymous (PROJQUAY-9346) (#4229)
Check if user exists before putting on queue.
2025-08-27 14:39:03 -04:00
Kenny Lee Sin Cheong
ac562b4b75 proxycache: include hidden manifests when querying for child manifests on tag creation (PROJQUAY-8536) (#4097)
* proxycache: include hidden manifests when querying for child manifests on tag creation (PROJQUAY-8536)

* Fix mocked call
2025-07-02 13:20:22 -04:00
Brandon Caton
147e41804b deps: updating python protobuf (PROJQUAY-9081) (#4096)
updating python protobuf version
2025-07-02 11:32:11 -04:00
Kenny Lee Sin Cheong
40031c2356 proxycache: fix queueitem payload for proxycachblobworker (PROJQUAY-9018) (#4076)
* proxycache: fix queueitem payload for proxycachblobworker (PROJQUAY-9018)

* Add feature flag for proxycacheblobworker

* PROXY_CACHE_BLOB_DOWNLOAD stub
2025-06-18 12:34:43 -04:00
Kenny Lee Sin Cheong
ed70eff752 proxycache: Download blob not cached when pulling manifests with blobs available locally (PROJQUAY-6708) (#4007)
* proxycache: Download blob not cached when pulling manifests with blob available locally (PROJQUAY-6708)

* Skip downloading blobs without placeholders
2025-06-10 10:34:23 -04:00
Michaela Lang
be82aefb44 proxycache(permissions): CVE-2025-4374 (PROJQUAY-8892) (#3941)
fixing CVE-2025-4374 by extending the create_repository method to understand if we are requesting a proxy_cache repository
added unittests for create_repository when proxy_cache.
2025-05-22 09:09:02 -04:00
Brandon Caton
3f34e3a822 Reverting PROJQUAY-8536 (#3833)
* Revert "bug: Adding allow hidden flag while looking up for manifests (PROJQUAY-8536) (#3722)"

This reverts commit f0c153fab5.

* Revert "proxy: moving manifest check to after upstream manifest fetch (PROJQUAY-8536) (#3814)"

This reverts commit 944edd064b.
2025-04-29 14:31:16 -04:00
Brandon Caton
944edd064b proxy: moving manifest check to after upstream manifest fetch (PROJQUAY-8536) (#3814)
moving manifest check to after upstream manifest fetch
2025-04-24 14:22:18 -04:00
Shubhra Deshpande
f0c153fab5 bug: Adding allow hidden flag while looking up for manifests (PROJQUAY-8536) (#3722)
When an image is pulled by digest, a temp tag is created to prevent the manifest from being garbage collected. This is true when a manifest list is pulled by tag as well. However, if this temporary tag expires (default is 1 day for proxied organizations) and the same manifest is pulled again by digest, the system attempts to create the manifest again, leading to an integrity error because the manifest already exists in the database.

---------

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-03-20 11:03:11 -04:00
Shubhra Deshpande
b48e1b47ad chore: added test for proxy cache (PROJQUAY-8440) (#3610)
* adding unit test for proxy cache

---------

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-02-03 19:20:08 +00:00
Syed Ahmed
c7352f4413 storage: add debug log for time taken to upload blob (PROJQUAY-7328) (#2962)
Adds logging on uploads to object storage
during the initial upload and the final copy
2024-06-21 15:47:35 -04:00
Daniel Messer
e73d096b28 api: OCI compliant tag listing and pagination (PROJQUAY-6931) (#2884)
implement OCI compliant pagination and tag listing

Signed-off-by: dmesser <dmesser@redhat.com>
2024-06-13 11:28:02 +02:00
Kenny Lee Sin Cheong
4546163e83 registry: implements the OCI 1.1 referrers API (PROJQUAY-7280) (#2597)
* registry: implements the OCI 1.1 referrers API

Migrations:
- Adds a subject column for lookup
- Adds a subject_backfilled column to track status of the backfilling
of existing manifests
- Adds a manifest_json column making use of postgres' JSONB support,
for future use.

Manifestsubjectbackfillworker: Indexes existing manifests for possible
existing subject field.

* Deprecate IGNORE_UNKNOWN_MEDIATYPES

* Cleanup
2024-06-07 13:28:13 -04:00
Brandon Caton
d2a1fedab4 quota: refactor, tests, and optimizations (PROJQUAY-6735) (#2712)
Adds the following changes:
- Additional logging
- Moving checks before expensive DB calls
- Additional tests
2024-03-06 09:06:38 -05:00
Kenny Lee Sin Cheong
be4edd02a3 cache: add caching for manifest requests (PROJQUAY-6482) (#2522) 2024-03-04 16:56:56 -05:00
Oleg Bulatov
6d5e6293e3 chore: use oci_model directly without proxy object (#2618) 2024-01-26 15:27:27 +01:00
Oleg Bulatov
a8381d45d3 fix: pull-through should tolerate missing tag (PROJQUAY-4849) (#2598)
* fix: add a test for the bug (PROJQUAY-4849)

* fix: pull-through should tolerate missing tag (PROJQUAY-4849)
2024-01-18 15:54:49 +01:00
Sunandadadi
a7737722fc cache: adding caching on look_up repository (PROJQUAY-6472) (#2521)
* adding caching on look_up repository

* fixing query count in tests

* fixing tests

* isort fix
2024-01-16 09:58:56 -05:00
Sunandadadi
6fc77823da Revert "api: add caching for get_repository method (PROJQUAY-6472)" (#2517)
Revert "api: add caching for get_repository method (PROJQUAY-6472) (#2515)"

This reverts commit 9c89e843f3.
2023-12-01 09:07:19 -05:00
Sunandadadi
9c89e843f3 api: add caching for get_repository method (PROJQUAY-6472) (#2515)
* api: add caching for get_repository method (PROJQUAY-6472)

* fixing formatting

* using pickle to serialize and deserialize objects

* adding model_cache parameter to lookup_repository func for interface and registry_proxy_model

* adding test

* formatting fix
2023-11-30 12:46:28 -05:00
Brandon Caton
2a672551fc quota: prevent tag creation on rejected manifest (PROJQUAY-3906) (#2247)
Currently if a new manifest is created that re-uses blobs already existing in the registry but not within the namespace it's possible for that manifest to be blocked if the newly referenced blobs exceed the quota limit. The issue being that the rejection only happens after the manifest has been created and tagged. This change blocks the manifest upload before the creation of the tag and prevents that tag from appearing in the UI and being pullable.

If the manifest upload get's rejected before the tag has been created for a new manifest, a temporary tag is created outside the time machine window so it is immediately available for GC.
2023-10-03 10:30:27 -04:00
Kenny Lee Sin Cheong
5f63b3a7bb chore: drop deprecated tables and remove unused code (PROJQUAY-522) (#2089)
* chore: drop deprecated tables and remove unused code

* isort imports

* migration: check for table existence before drop
2023-08-25 12:17:24 -04:00
Kenny Lee Sin Cheong
0ea4826956 secscan: garbage collect manifests (#1663)
Garbage collect manifests no longer referenced in Quay from the
security scanner service.

Also moved quota related code from data/registry_model/ to data/model/
to avoid circular dependencies.
2023-07-06 11:59:47 -04:00
Kenny Lee Sin Cheong
d47cea46fa database: remove foreign key constraints from deprecated tables (PROJQUAY-4879) (#1348)
* database: remove foreign key constraints from deprecated tables (PROJQUAY-4879)

* database: remove deprecated model tests (PROJQUAY-4879)
2023-06-28 09:37:07 -04:00
Oleg Bulatov
ea49bb53a3 chore: Bump mypy (#1962) 2023-06-28 13:47:32 +02:00
Kenny Lee Sin Cheong
d48df9b251 chore: deprecate image table support (PROJQUAY-522) (#1841)
Remove code related to deprecated image table
2023-06-14 10:54:54 -04:00
Marcus Kok
ff68f86c90 data: Fix error getting tag while calculating scan results SLO (#1952)
(PROJQUAY-5600)

Catch exception thrown when looking up tag `lifetime_start_ms` and set
`created_at` property for a manifest to None.

If `created_at` is None, don't calculate indexing SLI for manifest.
2023-06-07 16:24:22 -04:00
Marcus Kok
b911d480cf api: allow manifests to be pulled by digest (PROJQUAY-5467) (#1877)
Add new parameter `allow_hidden` to `lookup_manifest_by_digest` method and set this to true on the manifest v2 endpoint.

Enables manifests to be pulled by digest, and fixes issues with recent versions of conftest being unable to push to quay.
2023-05-26 09:43:27 -04:00
Brandon Caton
6cf0a3531b quota: calculating registry size (PROJQUAY-5476) (#1879)
Allows superusers to trigger a calculation of the deduplicated registry size. A superuser can go to the organization panel of the superuser page and select Calculate to queue a calculation of the registry total. The total will only be calculated when requested. Includes warning to user of increase of database load when running calculation.
2023-05-24 17:10:40 -04:00
Brandon Caton
cf50901159 quota: moving resetting of child manifest temporary tags to delete endpoint (PROJQUAY-5512) (#1894)
Moves the resetting of child manifest temporary tags to happen on deletion instead of on push/pull. Resetting child manifest temporary tags caused issues in other portions of the code like proxy cache where temporary tags were deleted too early.
2023-05-23 09:05:32 -04:00
Brandon Caton
e6f2dc3354 quota: excluding robots from quota total (PROJQUAY-5469) (#1871)
Adding quotatotalworker to config services / excluding robots from quota total
2023-05-10 14:54:57 -04:00
Brandon Caton
a2c379d47c quota: Include blob deduplication in totals (PROJQUAY-3942) (#1751)
Allows for only unique blobs are counted at the namespace and repository level. Calculation includes manifest list sizes.
Add's the following internal configurations that default to true:
QUOTA_INVALIDATE_TOTALS: Invalidates calculated totals when FEATURE_QUOTA_MANAGEMENT is set to false
RESET_CHILD_MANIFEST_EXPIRATION: Resets the expiry for child manifests on push of the manifest list for immediate GC eligibility
PERMANENTLY_DELETE_TAGS: Enables features related to the permanent deletion of tags outside the configured time machine window
2023-05-01 16:40:01 -04:00
Sunandadadi
5c34296920 API/UI: Filtering of tags API through query parameter (PROJQUAY-5362) (#1839)
* API/UI: Filtering of tags API through query parameter (PROJQUAY-5362)

* Changing syntax of query param to add operation + added propagation of filtering from new UI

* added exception to return 400 on incorrect syntax

* Added tests to test filtering of /tags endpoint

* Minor fixes
2023-04-28 16:16:41 -04:00
Marcus Kok
d84b67c73c secscan: add scan metric (PROJQUAY-4488) (#1719)
re-adding metric for initial scan results duration, update bucket range
2023-02-01 11:56:53 -05:00
Marcus Kok
5e4ae6495a Revert "secscan: add metric for scan results (PROJQUAY-4488) (#1674)" (#1714)
This reverts commit 8f9d6c9447.
2023-01-19 15:53:44 -05:00
Marcus Kok
8f9d6c9447 secscan: add metric for scan results (PROJQUAY-4488) (#1674)
Add histogram metric that calculates time taken to get security scan results after pushing image to quay.
2023-01-16 10:44:49 -05:00
Harish Govindarajulu
83c866193d Fix proxy-cache overflow quota limit when pulling same image (PROJQUAY-4670) (#1597)
Signed-off-by: harishsurf <hgovinda@redhat.com>
2022-10-28 16:54:53 -04:00
Harish Govindarajulu
8a7aeec7c6 Evict LRU proxied images when quota limit is reached (PROJQUAY-3063) (#1493)
Signed-off-by: harishsurf <hgovinda@redhat.com>
2022-09-23 11:22:32 -04:00
Flavian Missi
c04d0a644d registry: correctly bump tag expiration when tag changes upstream (PROJQUAY-3976) (#1479) 2022-08-16 16:41:00 +02:00
Harish Govindarajulu
342a50eb1f Proxy cache: respect CREATE_PRIVATE_REPO_ON_PUSH flag (PROJQUAY-3743) (#1426)
Signed-off-by: harishsurf <hgovinda@redhat.com>
2022-07-26 10:01:28 -04:00
Brandon Caton
9eb4fb6aa4 Revert workqueue refactor (#1456)
Currently the prometheus and GC workers are not running correctly.
Reverting the following commits:
- 4e1a985e70
- dac183a1ef
- 68a0d9eaf0
- af1aacea08
- f334b80098
2022-07-22 13:11:39 -04:00
Kenny Lee Sin Cheong
871c43eaf3 api: handle missing tag on DELETE tag api (#1444) 2022-07-15 13:47:20 -04:00
Oleg Bulatov
4e1a985e70 Initialize model.config on its import (PROJQUAY-4006) (#1412)
* Initialize model.config on its import

* Initialze registry_model and secscan_model on import

Co-authored-by: Dave O'Connor <1656866+HammerMeetNail@users.noreply.github.com>
2022-07-12 01:15:30 +02:00
Oleg Bulatov
ed8f12ba34 Enable some flake8 checks in CI (#1389) 2022-06-16 18:22:16 +02:00
Oleg Bulatov
5eaf0584db chore: Run mypy as CI job (#1363)
* Run mypy as CI job

* Fix peewee.pyi and configure pyright
2022-06-13 11:01:17 +02:00
Flavian Missi
d029a4652d pull-thru: bump expiration of all parent manifest lists on pull (#1336)
the previous code relied on manifests always having a single manifest
list as their parents, which is false.

fixes https://issues.redhat.com/browse/PROJQUAY-3818
2022-05-25 13:32:12 +02:00