Add REST API for managing immutability policies at organization and
repository levels. Integrate policy evaluation into tag creation.
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Add database models, migration, and CRUD functions for namespace and
repository immutability policies. Policies define regex patterns that
automatically mark matching tags as immutable when created.
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
Add label handler for quay.immutable manifest label that automatically
marks associated tags as immutable when images are pushed with
LABEL quay.immutable=true in their Dockerfile. Only "true" value
(case-insensitive) triggers immutability; other values are ignored.
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
* mirror: Add FEATURE_ORG_MIRROR feature flag (PROJQUAY-1266)
Add organization-level repository mirroring feature flag to enable
the new org mirroring functionality. Feature is disabled by default.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mirror: Add GET endpoint for org mirror config (PROJQUAY-1266)
Implements the GET /v1/organization/<org>/mirror endpoint to retrieve
organization-level mirror configuration. Includes business logic layer
with get_org_mirror_config() and comprehensive unit tests.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mirror: Add POST endpoint for org mirror config (PROJQUAY-1266)
Add create endpoint for organization-level mirror configuration:
- POST /v1/organization/<orgname>/mirror creates new config
- Validates robot account ownership and credentials
- Returns 201 on success, 409 if config already exists
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mirror: Add DELETE endpoint for org mirror config (PROJQUAY-1266)
Add delete endpoint for organization-level mirror configuration:
- DELETE /v1/organization/<orgname>/mirror removes config
- Also deletes all associated discovered repositories
- Returns 204 on success, 404 if config doesn't exist
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* mirror: Add PUT endpoint for org mirror config (PROJQUAY-1266)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* fix test failure
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Eliminate N+1 lazy-load queries by accepting user IDs instead of User objects.
## Problem
When checking organization membership for permissions, the old code triggered
a lazy-load query for EACH permission just to extract the user ID:
```python
# Old caller pattern - triggers N lazy-loads!
users_filter = {perm.user for perm in repo_perms}
org_members = get_organization_member_set(org, users_filter=users_filter)
```
For an endpoint returning 50 permissions, this caused 50 extra SELECT queries.
## Solution
Accept user IDs directly via `user_ids_filter` parameter:
```python
# New pattern - no lazy-loads!
user_ids_filter = {perm.user_id for perm in repo_perms}
org_members = get_organization_member_set(org, user_ids_filter=user_ids_filter)
```
The `user_id` foreign key field is already populated on the model - accessing
it doesn't require a database query.
## Changes
- Renamed `users_filter` → `user_ids_filter` parameter
- Accept set of integer IDs instead of User objects
- Updated 6 call sites in permission_models_pre_oci.py and prototype.py
- Added comprehensive test coverage
## Performance Impact
For get_repo_permissions_by_user with 50 permissions:
- Before: 50 lazy-load queries
- After: 0 queries
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
* mirror: Add architecture filter to RepoMirrorConfig (PROJQUAY-10255)
Adds architecture_filter field to filter multi-arch images during mirroring.
Supports amd64, arm64, ppc64le, s390x, 386, and riscv64 architectures.
Empty/null value mirrors all architectures (backwards compatible).
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* shrink valid arches for repo mirroring to only the necessary ones
* run formatter on file
* fix unit test
* validate arch before setting
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
This commit implements the core enforcement layer for tag immutability,
which prevents immutable tags from being deleted, overwritten, or
permanently removed from the time machine.
Changes:
- Add ImmutableTagException class with tag_name, operation, and
repository_id fields for detailed error reporting
- Enforce immutability in delete_tag() - raises exception for immutable tags
- Enforce immutability in retarget_tag() - prevents overwriting immutable
tags, respects raise_on_error parameter
- Enforce immutability in remove_tag_from_timemachine() - blocks permanent
deletion for both alive and expired immutable tags
- Add is_tag_immutable() - returns True/False/None for tag lookup
- Add set_tag_immutable() - updates immutability with optimistic locking
The immutable column, indexes, and log entry kind were previously added
in migration 5b8dc452f5c3. This commit adds the enforcement logic and
utility functions that use those database structures.
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
* feat: Add FEATURE_SPARSE_INDEX config for sparse manifest index support
When enabled, manifests in an index that cannot be loaded will be
skipped if their architecture is not in the SPARSE_INDEX_REQUIRED_ARCHS
list. This allows for sparse manifest indexes where not all architectures
are required to be present.
New config options:
- FEATURE_SPARSE_INDEX: Enable sparse manifest index support (default: False)
- SPARSE_INDEX_REQUIRED_ARCHS: List of architectures that must be present
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* chore(pre-commit): match black version with requirements-dev
* run `make black` against repo
* ci: switch to black 24.4.2
* fix: py312
* fix: flake8 errors
* fix: flake8 conflicts
* chore: add git blame ignore revs file
When a tag is deleted and re-pushed, pull statistics now start fresh
at 0 instead of persisting from the deleted tag.
Changes:
- Clear TagPullStatistics in _delete_tag()
- Clear TagPullStatistics in remove_tag_from_timemachine()
- Add tests for tag deletion clearing pull statistics
- Add test for re-push scenario starting with fresh stats
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
* fix(oauth): prevent redirect URI validation bypass (PROJQUAY-9849)
Co-authored-by: Claude <noreply@anthropic.com>
* test(oauth): add comprehensive coverage for redirect URI validation (PROJQUAY-9849)
Co-authored-by: Claude <noreply@anthropic.com>
* fix(oauth): add percent-encoding protection and improve test coverage (PROJQUAY-9849)
Co-authored-by: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
pullstats: updated bulk upsert function to track correct pull count and timestamp in case of race condition
Co-authored-by: shudeshp <shudeshp@redhat.com>
feat: Add image pull statistics API endpoints and UI integration
- Add new API endpoints for tag and manifest pull statistics
- Integrate pull metrics into web UI with new table columns
- Add FEATURE_IMAGE_PULL_STATS feature flag and PULL_METRICS_REDIS config
- Add pullstatsredisflushworker to supervisord configuration
- Add comprehensive test coverage for pull statistics functionality
Co-authored-by: shudeshp <shudeshp@redhat.com>
Implements global read-only superuser permissions for v1 endpoints, adjusts superuser write checks, and updates app token listing and detail endpoints; includes comprehensive tests.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* mirror: Add job timeout to mirror configurations (PROJQUAY-7249)
Previous global job timeout of 5 minutes was inadequate for big images. The timeout should now be configurable in much the same way as sync is. Minimum job length is 300 seconds/5 minutes.
The PR is still work in progress.
* Fix init db, remove reference to user data in logs
* Fix tests, change repo mirror configuration
* Fix tests, make mirroring cancellable through UI
* Add cancel mirror test, change HTML document to reflect mirror timeout
* Flake8 doesn't like when '==' is used with 'None'
* Fix mirror registry tests
* Add new cypress data to fix cypress tests
* Added ability to define upload chunk size to RADOS driver, small changes to repo mirror HTML page
* Fix database migration to follow HEAD
* Upload new database data for Cypress tests
* Make skopeo_timeout_interval mandatory on API calls
---------
Co-authored-by: Ivan Bazulic <ibazulic@redhat.com>
fixing CVE-2025-4374 by extending the create_repository method to understand if we are requesting a proxy_cache repository
added unittests for create_repository when proxy_cache.
* db: use iterator chain for _get_user_repo_permissions (PROJQUAY-8839)
Unwrapping can cause increase in CPU. Use iterator chain to let
the caller unwrap
* db: optimize _get_user_repo_permissions to send to read replica (PROJQUAY-8839)
it uses a union query which doesn't invoke the replica selection
logic. Make this into 2 seperate queries
* fix unit tests
Adds an include_orgs param to the active users query used by the reconciler and sets it to true for reconciler runs
Reconciler is not including orgs as a candidate for creating corresponding RH entitlements. As a result it misses users with stripe billing that are considered orgs.
* marketplace: splittable sku for MW02702 (PROJQUAY-8151)
* Alembic migration to drop unique constraint on the orgrhsubscriptions
table
* Can split sub quantities of MW02702 across multiple orgs
* Can specify quantity for the MW02702 SKU across orgs on react UI
* Update angular UI to allow user to specify quantities for MW02702
* notifications: fetch autoprune tags with multiple policies for image expiry notification(PROJQUAY-8117)
* don't fetch notifications if tags expiry is greater than notification days + add tests
robots: Add robot federation for keyless auth (PROJQUAY-7652)
adds the ability to configure federated auth for robots by
using external OIDC providers. Each robot can be configured
to have multiple external OIDC providers as the source for
authentication.
* health: Add statement timeout to health check (PROJQUAY-7950)
Currently, our query to check team roles (part of the db health check) does not time out after a certain period but runs indefinitely. This causes worker timeouts after 30 seconds, which ends up in worker being killed by the master gunicorn process.
We limit the maximum execution time of the query to 5000 ms so that proper exception is raised if that timeout is reached.
PostgreSQL logs:
~~~
2024-09-16 09:38:56.431 EDT [115775] testuser@quayclone2 LOG: duration: 0.011 ms statement: BEGIN
2024-09-16 09:38:56.431 EDT [115775] testuser@quayclone2 LOG: duration: 0.034 ms statement: SET statement_timeout=5000
2024-09-16 09:38:56.431 EDT [115775] testuser@quayclone2 LOG: duration: 0.010 ms statement: COMMIT
2024-09-16 09:38:56.432 EDT [115775] testuser@quayclone2 LOG: duration: 0.004 ms statement: BEGIN
2024-09-16 09:38:56.432 EDT [115775] testuser@quayclone2 LOG: duration: 0.300 ms statement: SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT 1
2024-09-16 09:38:56.433 EDT [115775] testuser@quayclone2 LOG: duration: 0.010 ms statement: COMMIT
2024-09-16 09:38:56.433 EDT [115775] testuser@quayclone2 LOG: duration: 0.005 ms statement: BEGIN
2024-09-16 09:38:56.433 EDT [115775] testuser@quayclone2 LOG: duration: 0.012 ms statement: SET statement_timeout=0
2024-09-16 09:38:56.433 EDT [115775] testuser@quayclone2 LOG: duration: 0.006 ms statement: COMMIT
~~~
Quay logs:
~~~
gunicorn-web stdout | 2024-09-16 13:38:56,412 [287] [DEBUG] [peewee.pool] Created new connection 127610088683136.
gunicorn-web stdout | 2024-09-16 13:38:56,417 [287] [DEBUG] [data.model.health] Validating database connection.
gunicorn-web stdout | 2024-09-16 13:38:56,418 [287] [INFO] [data.database] Connection pooling disabled for postgresql
gunicorn-web stdout | 2024-09-16 13:38:56,431 [287] [DEBUG] [peewee] ('SET statement_timeout=%s', (5000,))
gunicorn-web stdout | 2024-09-16 13:38:56,431 [287] [DEBUG] [data.model.health] Checking for existence of team roles, timeout 5000 ms.
gunicorn-web stdout | 2024-09-16 13:38:56,432 [287] [DEBUG] [peewee] ('SELECT "t1"."id", "t1"."name" FROM "teamrole" AS "t1" LIMIT %s', [1])
gunicorn-web stdout | 2024-09-16 13:38:56,433 [287] [DEBUG] [peewee] ('SET statement_timeout=0', None)
gunicorn-web stdout | 2024-09-16 13:38:56,434 [287] [DEBUG] [app] Ending request: urn:request:d039265b-414e-4d03-b29f-3e481286bf0f (/health/instance)...
~~~
* Fix generator function
Allows users to specify a regex tag pattern when creating namespace/repository autoprune policies via the new UI. Users will have the option to prune tags that only match the tag pattern or exclude tags that match the tag pattern.
during manifest push, we generate a map of blobs which
are part of the manifest layers. This is done using a
UNION query which can overload the worker if the
number of layers is too large. Instead, run each
query individually to prevent the crash
This allows a more refined search than just the repo name. When two
organizations contain the same name repo, e.g: org1/python and
org2/python, you can now search via org1/python to get the specific
result instead of both.
* storage: Disable pushes on registry (PROJQUAY-6870)
The current read-only option for Quay is not sometimes feasible, since it requires an insert of the service key and other manual config changes. For instance, if you want to just recalculate quota on the registry, but would like to allow all registry operations (including UI) without the possibility of pushes until recalculation is done, setting the whole registry `read-only` cannot be done since it makes the database read only as well.
This PR introduces a new flag called `DISABLE_PUSHES` which allows all registry operations to continue (changing tags, repo editing, robot account creation/deletion, user creation etc.) but will disable pushes of new images to the registry (i.e. backend storage will not change). If a registry already contains the image and a new tag is simply being added, that operation should succeed.
The following message would appear in the logs:
~~~
gunicorn-registry stdout | 2024-03-13 20:19:49,414 [369] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"METHOD NOT ALLOWED","detail":{},"message":"Pushes to the registry are currently disabled. Please contact the administrator for more information."}]}\n'
gunicorn-registry stdout | 2024-03-13 20:19:49,414 [369] [INFO] [gunicorn.access] 172.17.0.1 - - [13/Mar/2024:20:19:49 +0000] "PUT /v2/ibazulic/mariadb/manifests/sha256:c4694ba424e0259694a5117bbb510d67340051f0bdb7f9fa8033941a2d66e53e HTTP/1.1" 405 169 "-" "skopeo/1.9.3"
nginx stdout | 172.17.0.1 (-) - - [13/Mar/2024:20:19:49 +0000] "PUT /v2/ibazulic/mariadb/manifests/sha256:c4694ba424e0259694a5117bbb510d67340051f0bdb7f9fa8033941a2d66e53e HTTP/1.1" 405 169 "-" "skopeo/1.9.3" (0.002 3813 0.002)
~~~
The flag defaults to `False` (pushes enabled), unless set otherwise.
* Removed constraint on storage replication when pushes are disabled
* Rebase
* Fix isort sorting
* Fix isort sorting #2
* Removed constraint on storage replication when pushes are disabled
* Rebase
* Remove constraint on storage replication worker
* Fix linting on config.py
