1
0
mirror of https://github.com/quay/quay.git synced 2026-01-26 06:21:37 +03:00
Commit Graph

469 Commits

Author SHA1 Message Date
Ryan Wallace
225f28f507 chore(deps): upgrade redis version, remove redis-py-cluster (PROJQUAY-9314) (#4703)
* chore(deps): upgrade redis version, remove redis-py-cluster

* chore: adjust deps

* test: adjust tests

* docs: add comments about backwards compatible conversions
2026-01-05 21:02:43 +00:00
Ryan Wallace
a06cc6fa43 chore: update all black versions to 24.4.2 and run make black (#4754)
* chore(pre-commit): match black version with requirements-dev

* run `make black` against repo

* ci: switch to black 24.4.2

* fix: py312

* fix: flake8 errors

* fix: flake8 conflicts

* chore: add git blame ignore revs file
2025-12-19 11:29:53 -06:00
jbpratt
963a90ecdd fix(ldap): eliminate redundant conn in user search methods (PROJQUAY-7057) (#4720)
Each LDAP user search was creating two connections: one just to verify
admin credentials worked, then another for the actual search. This
doubled the load on LDAP servers. Consolidated into a single connection
block with proper error handling for INVALID_CREDENTIALS.

Affected methods:
- _ldap_user_search(): Core search used by most LDAP operations
- at_least_one_user_exists(): User existence checks

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-15 09:56:03 -06:00
jbpratt
ced2c6ffa8 feat(endpoints,web): add audit logs for quota configuration (PROJQUAY-9859) (#4692)
Adds audit logging for all quota management operations:
- org_create_quota, org_change_quota, org_delete_quota
- org_create_quota_limit, org_change_quota_limit, org_delete_quota_limit

Backend changes:
- Add LogEntryKind types in initdb.py
- Add log_action calls in namespacequota.py endpoints
- Add Alembic migration for new log kinds
- Add unit tests for audit logging

Frontend changes:
- Add log descriptions in UseLogDescriptions.tsx
- Add Cypress e2e test for quota log display

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude <noreply@anthropic.com>
2025-12-09 09:44:10 -06:00
jbpratt
52a5a85f7f fix(data): clear pull statistics when tags are deleted (PROJQUAY-9887) (#4667)
When a tag is deleted and re-pushed, pull statistics now start fresh
at 0 instead of persisting from the deleted tag.

Changes:
- Clear TagPullStatistics in _delete_tag()
- Clear TagPullStatistics in remove_tag_from_timemachine()
- Add tests for tag deletion clearing pull statistics
- Add test for re-push scenario starting with fresh stats

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-12-03 10:22:39 -06:00
Harish Govindarajulu
a97ca5c231 fix(oauth): prevent redirect URI validation bypass (PROJQUAY-9849) (#4635)
* fix(oauth): prevent redirect URI validation bypass (PROJQUAY-9849)

Co-authored-by: Claude <noreply@anthropic.com>

* test(oauth): add comprehensive coverage for redirect URI validation (PROJQUAY-9849)

Co-authored-by: Claude <noreply@anthropic.com>

* fix(oauth): add percent-encoding protection and improve test coverage (PROJQUAY-9849)

Co-authored-by: Claude <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-11-25 13:26:38 -05:00
jbpratt
71d219cc35 fix(test): prevent MySQL deadlocks in parallel proxy model tests (PROJQUAY-0000) (#4605)
* fix(test): prevent MySQL deadlocks in parallel proxy model tests (PROJQUAY-0000)

Mark all registry proxy model test classes to run serially using
pytest-xdist group markers. These tests all use the same "quayio-cache"
organization and were causing MySQL deadlocks when run in parallel
across multiple workers with pytest -n auto.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* fix(test): resolve Flask app naming conflict in quotaregistrysizeworker tests (PROJQUAY-0000)

Import Flask app with alias to avoid conflict with pytest 'app' fixture.
The test was using 'app.config' but 'app' resolved to a pytest fixture
definition instead of the Flask application object.

Follows the same pattern as test_securityscanningnotificationworker.py.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Brady Pratt <bpratt@redhat.com>

---------

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-24 15:28:34 +05:30
jbpratt
08153b6660 chore: CI runtime improvements (#4586)
* chore: update ci to use new large ubuntu 24.04 runner

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-Authored-By: Dave O'Connor <doconnor@redhat.com>

* fix: add libfreetype6-dev for Ubuntu 24.04 compatibility

The reportlab package requires FreeType development headers to build.
On Ubuntu 24.04, this dependency is not pulled in transitively and
must be explicitly installed. This fixes the "cannot find ft2build.h"
build error.

Added libfreetype6-dev to all jobs that install system dependencies
in CI.yaml and CI-nightly.yaml workflows.

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-Authored-By: Dave O'Connor <doconnor@redhat.com>

* chore: set the TEST_DATETIME to a static value

this caused an issue in xdist when generating test names

Signed-off-by: Brady Pratt <bpratt@redhat.com>

* chore: cache pip packages in CI

Signed-off-by: Brady Pratt <bpratt@redhat.com>

* chore: run registry tests with -n auto

Signed-off-by: Brady Pratt <bpratt@redhat.com>

* chore: run psql with -n auto

Signed-off-by: Brady Pratt <bpratt@redhat.com>

* chore: add file locking to prevent parallel test db init race condition

When running pytest -n auto with multiple workers, both workers would
simultaneously execute populate_database(), causing duplicate key
violations on shared tables like imagestoragelocation:

Worker 1: Check if User "devtable" exists → No → Start populating
Worker 2: Check if User "devtable" exists → No → Start populating
Both: INSERT INTO imagestoragelocation (name) VALUES ('local_eu')
Result: IntegrityError - duplicate key violation

Solution: Wrap init_db_path fixture with FileLock to ensure only one
worker initializes the database at a time. The lock file is created
in pytest's shared temp directory, coordinating across all workers.

- First worker acquires lock and populates database
- Subsequent workers wait at lock, then see database is already
  populated (via User.get() check in populate_database())
- Works for both PostgreSQL and MySQL
- 300-second timeout prevents deadlocks

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

* chore: run mysql with -n auto

Signed-off-by: Brady Pratt <bpratt@redhat.com>

---------

Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Dave O'Connor <doconnor@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
2025-11-20 09:01:18 -05:00
Shubhra Deshpande
3d2248d723 pullstats: updated bulk upsert function to track correct pull count and timestamp in case of race condition (PROJQUAY-9684) (#4463)
pullstats: updated bulk upsert function to track correct pull count and timestamp in case of race condition

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-11-14 09:48:23 -05:00
Jordi Piriz
747d1694cd revert: tracing improving otlp handling (PROJQUAY-8902) (#4438)
Revert "tracing: improving otlp handling (PROJQUAY-8902) (#4198)"

This reverts commit 89e758846f.
2025-11-03 16:17:32 +01:00
Shubhra Deshpande
aad7ffc89f feat: Added garbage collection hook for TagPullStatistics and ManifestPullStatistics tables (PROJQUAY-7176) (#4405)
feat: Added garbage collection hook for TagPullStatistics and ManifestPullStatistics tables

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-10-28 14:35:28 -04:00
Shubhra Deshpande
240d6441ba feat: Add image pull statistics API endpoints and UI integration (PROJQUAY-7176) (#4382)
feat: Add image pull statistics API endpoints and UI integration

- Add new API endpoints for tag and manifest pull statistics
- Integrate pull metrics into web UI with new table columns
- Add FEATURE_IMAGE_PULL_STATS feature flag and PULL_METRICS_REDIS config
- Add pullstatsredisflushworker to supervisord configuration
- Add comprehensive test coverage for pull statistics functionality

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-10-27 15:19:52 -04:00
Michaela Lang
89e758846f tracing: improving otlp handling (PROJQUAY-8902) (#4198)
improve OpenTelemetry implementation
2025-10-23 12:20:34 -04:00
Dave O'Connor
d83e2c8647 feat(api v1): global readonly superuser support and app token visibility (PROJQUAY-8279) (#4276)
Implements global read-only superuser permissions for v1 endpoints, adjusts superuser write checks, and updates app token listing and detail endpoints; includes comprehensive tests.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:00:59 -04:00
Shubhra Deshpande
ba57ee67e6 db: Added TagPullStatistics and ManifestPullStatistics tables with migrations (PROJQUAY-8414) (#4318)
Added TagPullStatistics and ManifestPullStatistics tables with migrations



---------

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-10-07 10:59:37 -04:00
Shubhra Deshpande
d10032d27c bug: Add configurable timeout to Splunk HEC requests (PROJQUAY-9375) (#4248)
Add configurable timeout to Splunk HEC requests

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-09-08 14:57:09 -04:00
Brandon Caton
cc5663b5f2 usagelogs: increasing elasticsearch conn timeout (PROJQUAY-9061) (#4246) 2025-09-08 14:22:57 -04:00
Harish Govindarajulu
cc637dd40e sqlite: Add retry with exponential backoff for sqlite (PROJQUAY-8758) (#4240)
Add retry with exponential backoff for sqlite (PROJQUAY-8758)

Sqlite runs into db lock contention, when run with a
single worker count is set to 1. This adds retry logic
to resolve lock contention

Signed-off-by: harishsurf <hgovinda@redhat.com>
2025-09-08 09:22:49 -04:00
Brandon Caton
c843bf2104 quota: only create notification if one doesn't already exist (PROJQUAY-8857) (#4092)
Add check if a notification exists before creating another. Previously many notifications were being created causing tables to inflate.
2025-09-02 09:12:32 -04:00
Brandon Caton
3f317fb74b proxy: fix err on pulls from public repos if anonymous (PROJQUAY-9346) (#4229)
Check if user exists before putting on queue.
2025-08-27 14:39:03 -04:00
Harish Govindarajulu
4efbbd2115 db: enable WAL mode and other PRAGMA stmts for SQLite to avoid db locking during concurrent writes (PROJQUAY-8758) (#4193)
* ADD PRAGMA statements for SQLITE to avoid database locking

* Fix formating
2025-08-21 15:12:54 -04:00
Elliot Gustafsson
df8ced5bf4 ldap: Get federated login identifier and use that when checking user privileges in FederatedUserManager (PROJQUAY-8879) (PROJQUAY-5880) (#3978) 2025-08-08 10:37:09 -04:00
Michaela Lang
0e91a7aec0 federationuser(ldap): avoid doing LDAP lookups for Robot accounts (PROJQUAY-5137) (#2505)
BREAKING CHANGE: LDAP lookup of robot accounts in the UI for granting permission has been dropped in context of permissions granting. This impacts Users from LDAP if they have not logged in to Quay already (pre provisioning) as a Federation User will only be available after logging in or being part of a Team.
2025-07-25 09:05:22 -04:00
Kenny Lee Sin Cheong
ac562b4b75 proxycache: include hidden manifests when querying for child manifests on tag creation (PROJQUAY-8536) (#4097)
* proxycache: include hidden manifests when querying for child manifests on tag creation (PROJQUAY-8536)

* Fix mocked call
2025-07-02 13:20:22 -04:00
Brandon Caton
147e41804b deps: updating python protobuf (PROJQUAY-9081) (#4096)
updating python protobuf version
2025-07-02 11:32:11 -04:00
Jonathan King
6c358ce3d8 splunk: only raise error when ALLOW_WITHOUT_STRICT_LOGGING is unset (PROJQUAY-8595) (#4082)
Co-authored-by: Jonathan <jonathan@Jonathans-MacBook-Pro.local>
2025-06-18 19:10:54 +00:00
Kenny Lee Sin Cheong
40031c2356 proxycache: fix queueitem payload for proxycachblobworker (PROJQUAY-9018) (#4076)
* proxycache: fix queueitem payload for proxycachblobworker (PROJQUAY-9018)

* Add feature flag for proxycacheblobworker

* PROXY_CACHE_BLOB_DOWNLOAD stub
2025-06-18 12:34:43 -04:00
Ivan Bazulic
9be679eb58 mirror: Add job timeout to mirror configurations (PROJQUAY-7249) (#3723)
* mirror: Add job timeout to mirror configurations (PROJQUAY-7249)
Previous global job timeout of 5 minutes was inadequate for big images. The timeout should now be configurable in much the same way as sync is. Minimum job length is 300 seconds/5 minutes.

The PR is still work in progress.

* Fix init db, remove reference to user data in logs

* Fix tests, change repo mirror configuration

* Fix tests, make mirroring cancellable through UI

* Add cancel mirror test, change HTML document to reflect mirror timeout

* Flake8 doesn't like when '==' is used with 'None'

* Fix mirror registry tests

* Add new cypress data to fix cypress tests

* Added ability to define upload chunk size to RADOS driver, small changes to repo mirror HTML page

* Fix database migration to follow HEAD

* Upload new database data for Cypress tests

* Make skopeo_timeout_interval mandatory on API calls

---------

Co-authored-by: Ivan Bazulic <ibazulic@redhat.com>
2025-06-12 19:09:51 +02:00
Emmanuel Ferdman
bf82e26c56 fix: migrate to logging.warning (PROJQUAY-8996) (#3976)
fix: migrate to logging.warning

Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-06-10 13:50:53 -04:00
Kenny Lee Sin Cheong
ed70eff752 proxycache: Download blob not cached when pulling manifests with blobs available locally (PROJQUAY-6708) (#4007)
* proxycache: Download blob not cached when pulling manifests with blob available locally (PROJQUAY-6708)

* Skip downloading blobs without placeholders
2025-06-10 10:34:23 -04:00
Brandon Caton
5c1c21bb1b ci: fixing app type import (PROJQUAY-8991) (#4021)
ci: fixing app type import
2025-06-10 09:14:57 -04:00
Jonathan King
753977a130 fix: do not fail on splunk errors when flag set (PROJQUAY-8595) (#3757) 2025-06-10 09:14:17 -04:00
Syed Ahmed
723102e641 build: move quay to python 3.12 (PROJQUAY-8800) (#3780)
Move Quay to python version 3.12 and switch out rehash with the resumable hash library.
2025-05-29 09:35:48 -04:00
Michaela Lang
be82aefb44 proxycache(permissions): CVE-2025-4374 (PROJQUAY-8892) (#3941)
fixing CVE-2025-4374 by extending the create_repository method to understand if we are requesting a proxy_cache repository
added unittests for create_repository when proxy_cache.
2025-05-22 09:09:02 -04:00
Brandon Caton
feb2f2e459 deps: upgrading gevent (PROJQUAY-8938) (#3928)
Upgrading gevent and greenlet
2025-05-19 13:08:04 -04:00
Sunandadadi
8abbfc8ef0 migration: adding conditional check on manifestblob alter column type (#3886)
* migration: adding conditional check on manifestblob alter column type

* convert string to lower

* fix

* fixing column type fetch
2025-05-13 13:32:38 -04:00
Sunandadadi
d3bcbe0610 migration: alter id column type for table manifestblob (#3885)
* migration: alter id column type for table manifestblob

* adding raw queries
2025-05-13 10:21:20 -04:00
Brandon Caton
3f34e3a822 Reverting PROJQUAY-8536 (#3833)
* Revert "bug: Adding allow hidden flag while looking up for manifests (PROJQUAY-8536) (#3722)"

This reverts commit f0c153fab5.

* Revert "proxy: moving manifest check to after upstream manifest fetch (PROJQUAY-8536) (#3814)"

This reverts commit 944edd064b.
2025-04-29 14:31:16 -04:00
Syed Ahmed
d674ab54cb db: use iterator chain for _get_user_repo_permissions (PROJQUAY-8839) (#3822)
* db: use iterator chain for _get_user_repo_permissions (PROJQUAY-8839)

Unwrapping can cause increase in CPU. Use iterator chain to let
the caller unwrap
2025-04-29 11:44:31 -04:00
Kenny Lee Sin Cheong
454f09ed89 logging: fix unreferenced variable from logging (PROJQUAY-8136) (#3819) 2025-04-25 18:44:46 +00:00
Syed Ahmed
36a552058a db: optimize _get_user_repo_permissions to send to read replica (PROJQUAY-8839) (#3818)
* db: optimize _get_user_repo_permissions to send to read replica (PROJQUAY-8839)

it uses a union query which doesn't invoke the replica selection
logic. Make this into 2 seperate queries

* fix unit tests
2025-04-25 09:36:43 -04:00
Brandon Caton
944edd064b proxy: moving manifest check to after upstream manifest fetch (PROJQUAY-8536) (#3814)
moving manifest check to after upstream manifest fetch
2025-04-24 14:22:18 -04:00
Kenny Lee Sin Cheong
5c51ab85e1 gc: garbage collect manifests not targetted by any tags when deleting repository (PROJQUAY-8136) (#3797)
* gc: garbage collect manifests not targetted by any tags when deleting repository (PROJQUAY-8136)

* test untagged manifest removal
2025-04-22 10:42:48 -04:00
Sunandadadi
3bf6e1b16b db: moving robot search and find repo to garbage collect queries to read replica (PROJQUAY-8792) (#3795)
* db: moving robot search and find repository to garbage collect queries to read replica (PROJQUAY-8792)

* removing lookup_robot from read_replica
2025-04-17 15:13:51 +00:00
Sunandadadi
fda188e23f db: revert get_namespace_user from read replica (PROJQUAY-8792) (#3796) 2025-04-17 15:06:37 +00:00
Sunandadadi
43bc1db13e db: moving get sorted matching repos and find repos to garbage collect to read replica (PROJQUAY-8792) (#3782) 2025-04-16 13:13:06 -04:00
Marcus Kok
b629054fe5 reconciler: fix typo in exception type (PROJQUAY-0000) (#3779)
* fix typo in exception type

* update test cases
2025-04-16 09:39:29 -04:00
Sunandadadi
275b6e8540 db: moving robot search query to read replica (PROJQUAY-8792) (#3781) 2025-04-15 09:51:12 -04:00
Sunandadadi
433124d95f db: moving get user from username query to read replica (PROJQUAY-8792) (#3773) 2025-04-14 11:58:06 -04:00
Sunandadadi
58357ca84f db: moving get user repo permissions query to read replica (PROJQUAY-8792) (#3772) 2025-04-10 15:16:04 -04:00