fixing CVE-2025-4374 by extending the create_repository method to understand if we are requesting a proxy_cache repository added unittests for create_repository when proxy_cache.
Co-authored-by: Michaela Lang <milang@redhat.com>
Allows for the full access and global readonly superuser to read all repositories from the /v2/_catalog endpoint.
---------
Co-authored-by: bcaton <bcaton@redhat.com>
registry: index manifest artifact type for filtering (PROJQUAY-7471)
Previous assumption made use of the config media type only, which is not the case
if a manifest's artifact type is explicitly set. i.e the config's media type and
artifact type are different, and the artifact type take precedence for filtering.
Co-authored-by: Kenny Lee Sin Cheong <kenny.lee28@gmail.com>
* database: adding subject_backfilled index to manifest table (PROJQUAY-7360) (#2963)
adding subject_backfilled index to manifest table
* Rebasing with main
* updating cypress data
* Rebasing with latest
* Forcing an empty commit.
* minor fixes
* Autoprune model changes
* fixing failing tests
* format fix
---------
Co-authored-by: Brandon Caton <bcaton@redhat.com>
* database: adding subject_backfilled index to manifest table (PROJQUAY-7360) (#2963)
adding subject_backfilled index to manifest table
* Rebasing with main
* updating cypress data
* fixing conflicts and rebasing with latest code
* adding tests
* Forcing an empty commit.
* Forcing an empty commit.
* skip_locked test fix
* adding tests
* minor fixes
---------
Co-authored-by: Brandon Caton <bcaton@redhat.com>
* registry: implements the OCI 1.1 referrers API
Migrations:
- Adds a subject column for lookup
- Adds a subject_backfilled column to track status of the backfilling
of existing manifests
- Adds a manifest_json column making use of postgres' JSONB support,
for future use.
Manifestsubjectbackfillworker: Indexes existing manifests for possible
existing subject field.
* Deprecate IGNORE_UNKNOWN_MEDIATYPES
* Cleanup
* marketplace: change redhatsubscriptions model to return a list of customer ids (PROJQUAY-7129)
* Use list comprehension in model and expect to return None if no rows are
returned
* fix behavior for reconciler and fix unit test for reconciler
Reducing the number of DB calls in the repo list endpoint with quota enabled by:
- Adding the id to RepositoryBaseElement when the repositories are initially fetched, removing the need to fetch the repository ID's again
- Fetching the repository sizes with a single DB call using the IN operator
* db: use read replica for selected queries (PROJQUAY-6397)
We add a new param `can_use_read_replica` to the `select`
query. This allows us to choose which queries we want to
send to the read replica. This is useful in cases where
the read replica lags behind the primary and some queries
need the latest data
* feat: Add support for auto pruning at repository level (PROJQUAY-6354)
* Add repositoryautoprunepolicy table to alembic migration script
* Add repository auto-prune policy endpoints
* Add UI for repository auto-pruning policies
* case: apply repo auto-prune policy when no namespace policy given
* case: both namespace and repo pruning policy are given
* Add tests for repository autoprune endpoint
* Add cypress test for repository auto-prune
* Add repo auto-prune policy clean-up for repository deletion
* Add repository auto pruning tables to quay db snapshot for cypress tests
* Address review comments
* Add more tests + fix CI + reformat files
* Address review comments #2
---------
Signed-off-by: harishsurf <hgovinda@redhat.com>
* Adds handling for when a subscription returned from the subscription watch api has a quantity greater than 1. Number of private repos should be correctly calculated using the quantity.
* Updates ui so that subscriptions can only be added to an org as a group, i.e. a subscription with quantity = 2 cannot be split across organizations.
* update reconciliationworker to use webCustomerId instead of
ebsAccountNumber
* fix reconciler where it was incorrectly using the ebsAccountNumber to
create subscriptions
* add job to reconciler so that it reconciles different ids between the
database and the user api
* separate skus to be used by billing and skus to be used by reconciler
Currently if a new manifest is created that re-uses blobs already existing in the registry but not within the namespace it's possible for that manifest to be blocked if the newly referenced blobs exceed the quota limit. The issue being that the rejection only happens after the manifest has been created and tagged. This change blocks the manifest upload before the creation of the tag and prevents that tag from appearing in the UI and being pullable.
If the manifest upload get's rejected before the tag has been created for a new manifest, a temporary tag is created outside the time machine window so it is immediately available for GC.
* add Postgresql client certificate authentication option
* user(robots): disallow robot login and create PROJQUAY-5968
Add a config flag `ROBOTS_DISALLOW` to turn off Robot login and creation
in Quay for all accounts.
with the Flag set existing Robots cannot login anymore
```
$ podman login -u milang+test2 -p W7B...HQA quay.example.com
Error: logging into "quay.example.com": invalid username/password
```
This behavior will be reported in the logs accordingly
```
gunicorn-registry stdout | 2023-08-24 19:08:01,907 [253] [WARNING] [auth.credentials] Failed to validate credentials for robot milang+test2: Robot account has been disabled. Please contact your administrator.
```
as well as in the UI when creating a Robot Account which will display
the `Robot account has been disabled. Please contact your
administrator.` message.
* Revert "add Postgresql client certificate authentication option"
need to branch for PR (stupid me)
This reverts commit f8f54eead2.
* fixed error message to proper english and added schema definition for the option
* add migration for orgrhskus table
* add endpoints for managing and listing skus bound to an org
* create checks in billing flow to look for org-bound skus
* refactor RH marketplace api objects to be more usable in tests
* update cypress test db data and exclude it from pre-commit hook formatting
* Pagination: Fixing paginate for /api/v1/superuser/logs API
* using offset to fetch next page items
* adding datetime parsing
* using black to format code
* removing use_offset
* Removing redundant declaration
Garbage collect manifests no longer referenced in Quay from the
security scanner service.
Also moved quota related code from data/registry_model/ to data/model/
to avoid circular dependencies.
* api: allow robot token creation with a pre-defined token (PROJQUAY-5414)
This is for usecases where we want to explicitly create a robot token
with a pre-defined token in case of migration events when we migrate
from one quay to another, we can re-use the same robot token to
avoid resetting it in all places it is used
Adds the following changes:
- Allows remove_tag_from_timemachine to expire tags even if the time machine window is set to 0, immediately marking them for deletion. This allows the quota proxy pruner to expire tags with the same method call. This wasn't required for normal push/pulls as the user would just call the DELETE /tag endpoint.
- Remove hidden = true when expiring tags. For proxy Quay will attempt to lookup the tag referenced by the manifest in order to extend it's lifetime_end_ms. Hiding this tag prevents that logic from running correctly.
Add new parameter `allow_hidden` to `lookup_manifest_by_digest` method and set this to true on the manifest v2 endpoint.
Enables manifests to be pulled by digest, and fixes issues with recent versions of conftest being unable to push to quay.
Moves the resetting of child manifest temporary tags to happen on deletion instead of on push/pull. Resetting child manifest temporary tags caused issues in other portions of the code like proxy cache where temporary tags were deleted too early.
Allows for only unique blobs are counted at the namespace and repository level. Calculation includes manifest list sizes.
Add's the following internal configurations that default to true:
QUOTA_INVALIDATE_TOTALS: Invalidates calculated totals when FEATURE_QUOTA_MANAGEMENT is set to false
RESET_CHILD_MANIFEST_EXPIRATION: Resets the expiry for child manifests on push of the manifest list for immediate GC eligibility
PERMANENTLY_DELETE_TAGS: Enables features related to the permanent deletion of tags outside the configured time machine window
* API/UI: Filtering of tags API through query parameter (PROJQUAY-5362)
* Changing syntax of query param to add operation + added propagation of filtering from new UI
* added exception to return 400 on incorrect syntax
* Added tests to test filtering of /tags endpoint
* Minor fixes
