Remove the config-tool editor UI while preserving the validation CLI
functionality. The editor UI is no longer needed as configuration is now
managed through other means.
Changes:
- Delete pkg/lib/editor/ directory with all Angular UI components
- Delete editor command and related Go code
- Remove config-editor from supervisord and deployment configs
- Simplify Dockerfile without JS build stage
- Remove Cypress E2E tests from GitHub workflow
- Clean up unused dependencies
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
feat: Add image pull statistics API endpoints and UI integration
- Add new API endpoints for tag and manifest pull statistics
- Integrate pull metrics into web UI with new table columns
- Add FEATURE_IMAGE_PULL_STATS feature flag and PULL_METRICS_REDIS config
- Add pullstatsredisflushworker to supervisord configuration
- Add comprehensive test coverage for pull statistics functionality
Co-authored-by: shudeshp <shudeshp@redhat.com>
* Add ngnix routing logic to default to react UI for downstream
* Remove defaulting from env, update Makefile to build react by default for local
* Add cypress test for signin and create account workflow
* Add missing routes + fallback to backend server
* Hide UI toggle when defaulting to new UI
* Adds forgot password + recovery email, recaptcha, missing login checks
* Add external login screen + support for other login types for new UI
* Add new screen for update user after external login
* Add authorized apps section under external logins tab
* Implement updateuser react component + fix cypress test
* Fix external login OAuth flow for react
* switch logic to default to new ui
* Add DEFAULT_UI: angular to config for cypress CI
* Fix cypress tests for oauth-callback
* Rebase and fix merge conflicts
---------
Signed-off-by: harishsurf <hgovinda@redhat.com>
* proxycache: Download blob not cached when pulling manifests with blob available locally (PROJQUAY-6708)
* Skip downloading blobs without placeholders
* registry: implements the OCI 1.1 referrers API
Migrations:
- Adds a subject column for lookup
- Adds a subject_backfilled column to track status of the backfilling
of existing manifests
- Adds a manifest_json column making use of postgres' JSONB support,
for future use.
Manifestsubjectbackfillworker: Indexes existing manifests for possible
existing subject field.
* Deprecate IGNORE_UNKNOWN_MEDIATYPES
* Cleanup
Allows superusers to trigger a calculation of the deduplicated registry size. A superuser can go to the organization panel of the superuser page and select Calculate to queue a calculation of the registry total. The total will only be calculated when requested. Includes warning to user of increase of database load when running calculation.
Allows for only unique blobs are counted at the namespace and repository level. Calculation includes manifest list sizes.
Add's the following internal configurations that default to true:
QUOTA_INVALIDATE_TOTALS: Invalidates calculated totals when FEATURE_QUOTA_MANAGEMENT is set to false
RESET_CHILD_MANIFEST_EXPIRATION: Resets the expiry for child manifests on push of the manifest list for immediate GC eligibility
PERMANENTLY_DELETE_TAGS: Enables features related to the permanent deletion of tags outside the configured time machine window
Removes the obsolete `logrotate.conf` file which was used in Quay 2.9 and below when nginx logs were stored under `/var/log/nginx` inside the container instead of being redirected to `/dev/stdout`. This file is no longer needed.
Removes read support for Clair V2, along with the need to package
jwtproxy with Quay.
TODO: Drop deprecate image api + image table, remove image data model.
Support client ip from x-forwarded header. This is needed if one of
the intermediate proxy between the client and Quay does not support
proxy protocol.
If set, FEATURE_PROXY_PROTOCOL will use the L4 feature. If not, say,
when a L7 proxy is used, fallback to using x-forwarded-for, and allow
specifying trusted ip/cidr for Nginx.
* Add dev dependencies mypy and typing
* Add makefile target `types-test`, not yet included in `test` target.
* Generate stubs for imported modules to avoid mypy complaining about missing types.
* Remove generated stubs as there are way too many and they cause tons of mess in the repo. Switched to ignoring untyped modules for now, to concentrate on Quay-only type checking.
* mypy config changed to ignore missing imports
* ignore property decorator as it is not supported by mypy
* mypy annotations for many configuration variables
* re-generate mypy_stubs directory as its necessary in some classes for base classes to prevent mypy errors
* util/registry/queuefile referred to non existent definition of Empty class in multiprocessing.queues
* ignore type checking for things like monkey patching and exported/re-imported objects that
mypy does not allow.
* Adjust mypy config to warn us about unreachable return paths and useless expressions.
* Add the __annotations__ property to INTERNAL_ONLY_PROPERTIES so that it is not part of the config schema testing
* Remove redundant dependencies `typing` and `typing-extensions` which are NOOP after Python 3.5
* Remove mypy-extensions which only provides a TypedDict implementation but has not been updated since 2019.
* updated mypy to 0.910 which requires all types packages to be installed manually.
* exclude local-dev from type checking until core team can suggest an outcome for __init__.py duplicate packages
* re-add typing dependency which will be needed until Python 3.9
* ignore .mypy_cache
* add mypy stub for features module to replace inline definitions
* import annotations eager evaluation in billing.py as it was required to reference a class declared later in the module.
* remove the type definition of V1ProtocolSteps/V2ProtocolSteps to make tox happy
- Allow PYTHONUSERBASE_SITE_PACKAGE to be added as an environment variable
- Set PYTHONUSERBASE_SITE_PACKAGE to /usr/local/lib/python3.8/site-packages in downstream Dockerfile
* cicd: toggle on more OCI conformance tests (PROJQUAY-1997)
This commit also has the CI keep the report around as a workflow
artifact.
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
* external_libraries: patch out wider dependencies (PROJQUAY-1997)
This change makes it possible to run the external_libraries script
without needing to pull in the entire quay configuration framework. This
will allow for a more granular Dockerfile.
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
* cicd: enable docker buildkit backend (PROJQUAY-1997)
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
* Dockerfile: rework to be faster (PROJQUAY-1997)
This change allows for the Dockerfile to reuse more cache, and run in
parallel when using the BuildKit backend.
* init: Uses user site-package directory as Python root (PROJQUAY-1997)
Use `python3 -m site --user-site` as Python root when installing certs.
Co-authored-by: Hank Donnay <hdonnay@redhat.com>
* Use safe_load when loading the config yaml
* Setup nginx ssl termination for grpc endpoints
* Bootstrap Quay's ca cert in the build executor nodes
* Update certificate mount point in ignition config
Mount the Fedora CoreOS/RHCOS based cert directory to /certs in the
builder container, where it will be installed by the container's
entrypoint.
* Change verbs to use a DerivedStorageForManifest table instead of DerivedStorageForImage
This allows us to deprecate the DerivedStorageForImage table.
Fixes https://issues.redhat.com/browse/PROJQUAY-519
* Change uploaded blob tracking to use its own table and deprecate
RepositoryTag
* Start recording the compressed layers size and config media type on the
manifest row in the database
NOTE: This change includes a database migration which will *lock* the
manifest table
* Change tag API to return the layers size from the manifest
* Remove unused code
* Add new config_media_type field to OCI types
* Fix secscan V2 test for us no longer writing temp images
* Remove unused uploading field
* Switch registry model to use synthetic legacy images
Legacy images are now (with exception of the V2 security model) read from the *manifest* and sythensized in memory. The legacy image IDs are generated realtime based on the hashids library. This change also further deprecates a bunch of our Image APIs, reducing them to only returning the image IDs, and emptying out the remaining metadata (to avoid the requirement of us loading the information for the manifest from storage).
This has been tested with our full clients test suite with success.
* Add a backfill worker for manifest layers compressed sizes
* Change image tracks into manifest tracks now that we no longer have
manifest-less tags
* Add back in the missing method
* Add missing joins to reduce extra queries
* Remove unnecessary join when looking up legacy images
* Remove extra hidden filter on tag queries
* Further DB improvements
* Delete all Verbs, as they were deprecated
* Add back missing parameter in manifest data type
* Fix join to return None for the robot if not defined on mirror config
* switch to using secscan_v4_model for all indexing and remove most of secscan_v2_model code
* Add a missing join
* Remove files accidentally re-added due to rebase
* Add back hashids lib
* Rebase fixes
* Fix broken test
* Remove unused GPG signer now that ACI conversion is removed
* Remove duplicated repomirrorworker
* Remove unused notification code for secscan. We'll re-add it once Clair
V4 security notifications are ready to go
* Fix formatting
* Stop writing Image rows when creating manifests
* Stop writing empty layer blobs for manifests
As these blobs are shared, we don't need to write ManifestBlob rows
for them
* Remove further unused code
* Add doc comment to _build_blob_map
* Add unit test for synthetic V1 IDs
* Remove unused import
* Add an invalid value test to synthetic ID decode tests
* Add manifest backfill worker back in
Seems to have been removed at some point
* Add a test for cached active tags
* Rename test_shared to not conflict with another same-named test file
Pytest doesn't like having two test modules with the same name
* Have manifestbackfillworker also copy over the config_media_type if present
Co-authored-by: alecmerdler <alecmerdler@gmail.com>
* Convert all Python2 to Python3 syntax.
* Removes oauth2lib dependency
* Replace mockredis with fakeredis
* byte/str conversions
* Removes nonexisting __nonzero__ in Python3
* Python3 Dockerfile and related
* [PROJQUAY-98] Replace resumablehashlib with rehash
* PROJQUAY-123 - replace gpgme with python3-gpg
* [PROJQUAY-135] Fix unhashable class error
* Update external dependencies for Python 3
- Move github.com/app-registry/appr to github.com/quay/appr
- github.com/coderanger/supervisor-stdout
- github.com/DevTable/container-cloud-config
- Update to latest mockldap with changes applied from coreos/mockldap
- Update dependencies in requirements.txt and requirements-dev.txt
* Default FLOAT_REPR function to str in json encoder and removes keyword assignment
True, False, and str were not keywords in Python2...
* [PROJQUAY-165] Replace package `bencode` with `bencode.py`
- Bencode is not compatible with Python 3.x and is no longer
maintained. Bencode.py appears to be a drop-in replacement/fork
that is compatible with Python 3.
* Make sure monkey.patch is called before anything else (
* Removes anunidecode dependency and replaces it with text_unidecode
* Base64 encode/decode pickle dumps/loads when storing value in DB
Base64 encodes/decodes the serialized values when storing them in the
DB. Also make sure to return a Python3 string instead of a Bytes when
coercing for db, otherwise, Postgres' TEXT field will convert it into
a hex representation when storing the value.
* Implement __hash__ on Digest class
In Python 3, if a class defines __eq__() but not __hash__(), its
instances will not be usable as items in hashable collections (e.g sets).
* Remove basestring check
* Fix expected message in credentials tests
* Fix usage of Cryptography.Fernet for Python3 (#219)
- Specifically, this addresses the issue where Byte<->String
conversions weren't being applied correctly.
* Fix utils
- tar+stream layer format utils
- filelike util
* Fix storage tests
* Fix endpoint tests
* Fix workers tests
* Fix docker's empty layer bytes
* Fix registry tests
* Appr
* Enable CI for Python 3.6
* Skip buildman tests
Skip buildman tests while it's being rewritten to allow ci to pass.
* Install swig for CI
* Update expected exception type in redis validation test
* Fix gpg signing calls
Fix gpg calls for updated gpg wrapper, and add signing tests.
* Convert / to // for Python3 integer division
* WIP: Update buildman to use asyncio instead of trollius.
This dependency is considered deprecated/abandoned and was only
used as an implementation/backport of asyncio on Python 2.x
This is a work in progress, and is included in the PR just to get the
rest of the tests passing. The builder is actually being rewritten.
* Target Python 3.8
* Removes unused files
- Removes unused files that were added accidentally while rebasing
- Small fixes/cleanup
- TODO tasks comments
* Add TODO to verify rehash backward compat with resumablehashlib
* Revert "[PROJQUAY-135] Fix unhashable class error" and implements __hash__ instead.
This reverts commit 735e38e3c1d072bf50ea864bc7e119a55d3a8976.
Instead, defines __hash__ for encryped fields class, using the parent
field's implementation.
* Remove some unused files ad imports
Co-authored-by: Kenny Lee Sin Cheong <kenny.lee@redhat.com>
Co-authored-by: Tom McKay <thomasmckay@redhat.com>
