* notifications: Support slash in repository names (PROJQUAY-7538)
Fix for PROJQUAY-7538 discussed in #3069 by only considering the first slash when separating namespace and repository.
* Test and devcontainer
* Remove devcontainer.json
* Revert irrelevant test change.
* ui: removing default ui check
* ui: add option to disable angular UI
* Creating explicit angular and react cookies with config default
* Fixing "current ui" display text to respond to light theme
* initial superuser framework
* all service key functionality except create key
* add create preshareable key
* add change logs panel and fresh login component
* messages ui and config without display
* add global display of messages
* CSRF token changes required for fresh login
* usage logs functionality first pass
* fix fetch user logs, colors, legend, chart and default route
* usage logs prevent greater than 30 days
* usage logs functionality complete
* superuser organization action menu commands
* add framework and service keys tests, fix service key date mismatch
* add all remaining superuser tests
* Re-design sidenav for superuser component
* Add missing columns and access control for organization list
Signed-off-by: harishsurf <hgovinda@redhat.com>
Adds Size and Admin columns and superuser checks + readonly support
for superuser capabilities
* Add build logs functionality for superusers
* Add missing functionality for user and org management for superuser
Adds create user modal, and other missing modals for superuser related
actions for both user and organization
* Redesign quota functionality for superuser
Only superuser should be allowed to configure quota. Adds new modal
to configure quota. Removes modifying quota from org settings tab
* Fix cypress tests + formatting + undo X-Next-CSRF-Token token change
---------
Signed-off-by: harishsurf <hgovinda@redhat.com>
Co-authored-by: harishsurf <hgovinda@redhat.com>
feat: Add image pull statistics API endpoints and UI integration
- Add new API endpoints for tag and manifest pull statistics
- Integrate pull metrics into web UI with new table columns
- Add FEATURE_IMAGE_PULL_STATS feature flag and PULL_METRICS_REDIS config
- Add pullstatsredisflushworker to supervisord configuration
- Add comprehensive test coverage for pull statistics functionality
Co-authored-by: shudeshp <shudeshp@redhat.com>
Implements global read-only superuser permissions for v1 endpoints, adjusts superuser write checks, and updates app token listing and detail endpoints; includes comprehensive tests.
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Added redis database and redis flush worker
* updated digest validation
* adding test coverage for scan_keys function
* adding test coverage
* added tests for scan function
* added coverage for flush to database function
* added coverage for gnuicorn worker initialization
* Replaced mock methods in the test with actual function calls
---------
Co-authored-by: shudeshp <shudeshp@redhat.com>
* Add ngnix routing logic to default to react UI for downstream
* Remove defaulting from env, update Makefile to build react by default for local
* Add cypress test for signin and create account workflow
* Add missing routes + fallback to backend server
* Hide UI toggle when defaulting to new UI
* Adds forgot password + recovery email, recaptcha, missing login checks
* Add external login screen + support for other login types for new UI
* Add new screen for update user after external login
* Add authorized apps section under external logins tab
* Implement updateuser react component + fix cypress test
* Fix external login OAuth flow for react
* switch logic to default to new ui
* Add DEFAULT_UI: angular to config for cypress CI
* Fix cypress tests for oauth-callback
* Rebase and fix merge conflicts
---------
Signed-off-by: harishsurf <hgovinda@redhat.com>
* fix: resolve Sentry/OpenTelemetry integration conflicts
Configure Sentry to use minimal integrations when OTEL_TRACING is enabled
to prevent instrumentation conflicts that broke exception capture
---------
Co-authored-by: shudeshp <shudeshp@redhat.com>
* mirror: Add job timeout to mirror configurations (PROJQUAY-7249)
Previous global job timeout of 5 minutes was inadequate for big images. The timeout should now be configurable in much the same way as sync is. Minimum job length is 300 seconds/5 minutes.
The PR is still work in progress.
* Fix init db, remove reference to user data in logs
* Fix tests, change repo mirror configuration
* Fix tests, make mirroring cancellable through UI
* Add cancel mirror test, change HTML document to reflect mirror timeout
* Flake8 doesn't like when '==' is used with 'None'
* Fix mirror registry tests
* Add new cypress data to fix cypress tests
* Added ability to define upload chunk size to RADOS driver, small changes to repo mirror HTML page
* Fix database migration to follow HEAD
* Upload new database data for Cypress tests
* Make skopeo_timeout_interval mandatory on API calls
---------
Co-authored-by: Ivan Bazulic <ibazulic@redhat.com>
* proxycache: Download blob not cached when pulling manifests with blob available locally (PROJQUAY-6708)
* Skip downloading blobs without placeholders
* initial checkin of schema update
* finished first iteration
* re-added the comments that got lost with json to python dict conversion
* fixed space on comments
* fixed comments
* repush for checks
* black fix
* fixed typos in schema
* initial checkin for the superuser/config endpoint to show how its intended to return data
bug: fixing NaN value error for quota displayed on member org page (PROJQUAY-6465) (#3224)
bug: fixing NaN value error for quota displayed on member org page (PROJQUAY-6465)
fixed black formatting
fixed flake and black formatting
fixed isort formatting
test need to be updated for superuser endpoints. There is no explicit superuser token test so globalreadonlysuperuser shall succeed too
fixed double json encoding
changed naming to comply with other SuperUserClasses, added SuperUserPermission check as scope only isnt sufficient
fixed another black error
fixed response for devtable check
fixed response for devtable as that is a superuser
fixed black format :/
added allow_if_global_readonly_superuser to config endpoint
repush for checks
fixed app.logger to module specific logger ; added missed SCHEMA return
added unittest for checking superuser config dump API call (no clue if the unittests build up a full setup since we mock all kind of stuff in the other calls)
removed env PWD check as it seems to be unset in the github runners
added missing unittest step
added FeatureFlag for config dump
formatting
* removed wrong commit in the branch
* changed from route decorator to in method check and changed unittests to fail as the default config is to deny the request
* added one test for security_tests
* rebumped the security tests
* ui: implement change to render modelcard stored in layers (PROJQUAY-8412)
When a manifest has certain annotations or artifactTypes, render the
applicable modelcard markdown in a new tags detail tab.
* removing untar when fetching model card
* removing extra api calls
* Add modelcar check tests
---------
Co-authored-by: bcaton <bcaton@redhat.com>
* ui: Expand support for customized footer links (PROJQUAY-5648)
Previous iteration only allowed changes to the terms of service. With this push, all footer links should be customizable through the `FOOTER_LINKS` object. Example:
~~~
FOOTER_LINKS:
TERMS_OF_SERVICE_URL: "some_url"
PRIVACY_POLICY_URL: "some_url"
SECURITY_URL: "some_url"
ABOUT_URL: "some_url"
~~~
Missing entries will not be printed out in the UI.
* Fixes to parsing of config object
* Add type annotation
* marketplace: splittable sku for MW02702 (PROJQUAY-8151)
* Alembic migration to drop unique constraint on the orgrhsubscriptions
table
* Can split sub quantities of MW02702 across multiple orgs
* Can specify quantity for the MW02702 SKU across orgs on react UI
* Update angular UI to allow user to specify quantities for MW02702
* notifications: fetch autoprune tags with multiple policies for image expiry notification(PROJQUAY-8117)
* don't fetch notifications if tags expiry is greater than notification days + add tests
robots: Add robot federation for keyless auth (PROJQUAY-7652)
adds the ability to configure federated auth for robots by
using external OIDC providers. Each robot can be configured
to have multiple external OIDC providers as the source for
authentication.
* fixing discrepancy in debug logs after changes to util.migrate.allocator
* Revert "fixing discrepancy in debug logs after changes to util.migrate.allocator"
This reverts commit b7adb7ba71.
* fixing discrepancy in logs suggested by Ivan in #3160 after verbosity of logs are altered
* fixing discrepancy in logs suggested by Ivan in #3160 after verbosity of logs are altered
Changes the timeout value for non-indexing requests to clair from 600 to
30.
Because the timeout for a vulnerability report request is so high, heavy
traffic to the security enpdoint results in database connections being
exhausted. Lowering the timeout value should allow requests to complete
and connections to the database to close.
* deps: bump PyMySQL version (PROJQUAY-7251) (#3113)
bug: bump PyMySQL version (PROJQUAY-7251)
This should resolve CVE-2024-36039.
* hide logs if debuglog is false
* test for new allocator.py code changes
* test for new allocator.py code changes
---------
Co-authored-by: Ivan Bazulic <ibazulic@redhat.com>
* storage: Disable pushes on registry (PROJQUAY-6870)
The current read-only option for Quay is not sometimes feasible, since it requires an insert of the service key and other manual config changes. For instance, if you want to just recalculate quota on the registry, but would like to allow all registry operations (including UI) without the possibility of pushes until recalculation is done, setting the whole registry `read-only` cannot be done since it makes the database read only as well.
This PR introduces a new flag called `DISABLE_PUSHES` which allows all registry operations to continue (changing tags, repo editing, robot account creation/deletion, user creation etc.) but will disable pushes of new images to the registry (i.e. backend storage will not change). If a registry already contains the image and a new tag is simply being added, that operation should succeed.
The following message would appear in the logs:
~~~
gunicorn-registry stdout | 2024-03-13 20:19:49,414 [369] [DEBUG] [endpoints.v2] sending response: b'{"errors":[{"code":"METHOD NOT ALLOWED","detail":{},"message":"Pushes to the registry are currently disabled. Please contact the administrator for more information."}]}\n'
gunicorn-registry stdout | 2024-03-13 20:19:49,414 [369] [INFO] [gunicorn.access] 172.17.0.1 - - [13/Mar/2024:20:19:49 +0000] "PUT /v2/ibazulic/mariadb/manifests/sha256:c4694ba424e0259694a5117bbb510d67340051f0bdb7f9fa8033941a2d66e53e HTTP/1.1" 405 169 "-" "skopeo/1.9.3"
nginx stdout | 172.17.0.1 (-) - - [13/Mar/2024:20:19:49 +0000] "PUT /v2/ibazulic/mariadb/manifests/sha256:c4694ba424e0259694a5117bbb510d67340051f0bdb7f9fa8033941a2d66e53e HTTP/1.1" 405 169 "-" "skopeo/1.9.3" (0.002 3813 0.002)
~~~
The flag defaults to `False` (pushes enabled), unless set otherwise.
* Removed constraint on storage replication when pushes are disabled
* Rebase
* Fix isort sorting
* Fix isort sorting #2
* Removed constraint on storage replication when pushes are disabled
* Rebase
* Remove constraint on storage replication worker
* Fix linting on config.py
Changing logger.info to logger.debug to stop too many INFO log lines from displaying in Quay logs + Adding the missing worker name that print those
The idea of the log is to tell you which block is being procesed by worker that is processing it. These logs are missing the worker names
```
2023-11-27 15:24:05,825 [80] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 210690-213109
2023-11-27 15:24:05,832 [80] [INFO] [util.migrate.allocator] No candidates, marking entire block completed 131657-134076
[sbhavsar-mac:~ sayalibhavsar]$ grep "No candidates, marking entire block completed" quay | wc -l
5300
```
For huge registries above logs could be a bit noisy. We could hide it as a debug message rather than as an info message.
