1
0
mirror of https://github.com/quay/quay.git synced 2026-01-27 18:42:52 +03:00
Commit Graph

459 Commits

Author SHA1 Message Date
Shubhra Deshpande
aad7ffc89f feat: Added garbage collection hook for TagPullStatistics and ManifestPullStatistics tables (PROJQUAY-7176) (#4405)
feat: Added garbage collection hook for TagPullStatistics and ManifestPullStatistics tables

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-10-28 14:35:28 -04:00
Shubhra Deshpande
240d6441ba feat: Add image pull statistics API endpoints and UI integration (PROJQUAY-7176) (#4382)
feat: Add image pull statistics API endpoints and UI integration

- Add new API endpoints for tag and manifest pull statistics
- Integrate pull metrics into web UI with new table columns
- Add FEATURE_IMAGE_PULL_STATS feature flag and PULL_METRICS_REDIS config
- Add pullstatsredisflushworker to supervisord configuration
- Add comprehensive test coverage for pull statistics functionality

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-10-27 15:19:52 -04:00
Michaela Lang
89e758846f tracing: improving otlp handling (PROJQUAY-8902) (#4198)
improve OpenTelemetry implementation
2025-10-23 12:20:34 -04:00
Dave O'Connor
d83e2c8647 feat(api v1): global readonly superuser support and app token visibility (PROJQUAY-8279) (#4276)
Implements global read-only superuser permissions for v1 endpoints, adjusts superuser write checks, and updates app token listing and detail endpoints; includes comprehensive tests.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-10-21 15:00:59 -04:00
Shubhra Deshpande
ba57ee67e6 db: Added TagPullStatistics and ManifestPullStatistics tables with migrations (PROJQUAY-8414) (#4318)
Added TagPullStatistics and ManifestPullStatistics tables with migrations



---------

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-10-07 10:59:37 -04:00
Shubhra Deshpande
d10032d27c bug: Add configurable timeout to Splunk HEC requests (PROJQUAY-9375) (#4248)
Add configurable timeout to Splunk HEC requests

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-09-08 14:57:09 -04:00
Brandon Caton
cc5663b5f2 usagelogs: increasing elasticsearch conn timeout (PROJQUAY-9061) (#4246) 2025-09-08 14:22:57 -04:00
Harish Govindarajulu
cc637dd40e sqlite: Add retry with exponential backoff for sqlite (PROJQUAY-8758) (#4240)
Add retry with exponential backoff for sqlite (PROJQUAY-8758)

Sqlite runs into db lock contention, when run with a
single worker count is set to 1. This adds retry logic
to resolve lock contention

Signed-off-by: harishsurf <hgovinda@redhat.com>
2025-09-08 09:22:49 -04:00
Brandon Caton
c843bf2104 quota: only create notification if one doesn't already exist (PROJQUAY-8857) (#4092)
Add check if a notification exists before creating another. Previously many notifications were being created causing tables to inflate.
2025-09-02 09:12:32 -04:00
Brandon Caton
3f317fb74b proxy: fix err on pulls from public repos if anonymous (PROJQUAY-9346) (#4229)
Check if user exists before putting on queue.
2025-08-27 14:39:03 -04:00
Harish Govindarajulu
4efbbd2115 db: enable WAL mode and other PRAGMA stmts for SQLite to avoid db locking during concurrent writes (PROJQUAY-8758) (#4193)
* ADD PRAGMA statements for SQLITE to avoid database locking

* Fix formating
2025-08-21 15:12:54 -04:00
Elliot Gustafsson
df8ced5bf4 ldap: Get federated login identifier and use that when checking user privileges in FederatedUserManager (PROJQUAY-8879) (PROJQUAY-5880) (#3978) 2025-08-08 10:37:09 -04:00
Michaela Lang
0e91a7aec0 federationuser(ldap): avoid doing LDAP lookups for Robot accounts (PROJQUAY-5137) (#2505)
BREAKING CHANGE: LDAP lookup of robot accounts in the UI for granting permission has been dropped in context of permissions granting. This impacts Users from LDAP if they have not logged in to Quay already (pre provisioning) as a Federation User will only be available after logging in or being part of a Team.
2025-07-25 09:05:22 -04:00
Kenny Lee Sin Cheong
ac562b4b75 proxycache: include hidden manifests when querying for child manifests on tag creation (PROJQUAY-8536) (#4097)
* proxycache: include hidden manifests when querying for child manifests on tag creation (PROJQUAY-8536)

* Fix mocked call
2025-07-02 13:20:22 -04:00
Brandon Caton
147e41804b deps: updating python protobuf (PROJQUAY-9081) (#4096)
updating python protobuf version
2025-07-02 11:32:11 -04:00
Jonathan King
6c358ce3d8 splunk: only raise error when ALLOW_WITHOUT_STRICT_LOGGING is unset (PROJQUAY-8595) (#4082)
Co-authored-by: Jonathan <jonathan@Jonathans-MacBook-Pro.local>
2025-06-18 19:10:54 +00:00
Kenny Lee Sin Cheong
40031c2356 proxycache: fix queueitem payload for proxycachblobworker (PROJQUAY-9018) (#4076)
* proxycache: fix queueitem payload for proxycachblobworker (PROJQUAY-9018)

* Add feature flag for proxycacheblobworker

* PROXY_CACHE_BLOB_DOWNLOAD stub
2025-06-18 12:34:43 -04:00
Ivan Bazulic
9be679eb58 mirror: Add job timeout to mirror configurations (PROJQUAY-7249) (#3723)
* mirror: Add job timeout to mirror configurations (PROJQUAY-7249)
Previous global job timeout of 5 minutes was inadequate for big images. The timeout should now be configurable in much the same way as sync is. Minimum job length is 300 seconds/5 minutes.

The PR is still work in progress.

* Fix init db, remove reference to user data in logs

* Fix tests, change repo mirror configuration

* Fix tests, make mirroring cancellable through UI

* Add cancel mirror test, change HTML document to reflect mirror timeout

* Flake8 doesn't like when '==' is used with 'None'

* Fix mirror registry tests

* Add new cypress data to fix cypress tests

* Added ability to define upload chunk size to RADOS driver, small changes to repo mirror HTML page

* Fix database migration to follow HEAD

* Upload new database data for Cypress tests

* Make skopeo_timeout_interval mandatory on API calls

---------

Co-authored-by: Ivan Bazulic <ibazulic@redhat.com>
2025-06-12 19:09:51 +02:00
Emmanuel Ferdman
bf82e26c56 fix: migrate to logging.warning (PROJQUAY-8996) (#3976)
fix: migrate to logging.warning

Signed-off-by: Emmanuel Ferdman <emmanuelferdman@gmail.com>
2025-06-10 13:50:53 -04:00
Kenny Lee Sin Cheong
ed70eff752 proxycache: Download blob not cached when pulling manifests with blobs available locally (PROJQUAY-6708) (#4007)
* proxycache: Download blob not cached when pulling manifests with blob available locally (PROJQUAY-6708)

* Skip downloading blobs without placeholders
2025-06-10 10:34:23 -04:00
Brandon Caton
5c1c21bb1b ci: fixing app type import (PROJQUAY-8991) (#4021)
ci: fixing app type import
2025-06-10 09:14:57 -04:00
Jonathan King
753977a130 fix: do not fail on splunk errors when flag set (PROJQUAY-8595) (#3757) 2025-06-10 09:14:17 -04:00
Syed Ahmed
723102e641 build: move quay to python 3.12 (PROJQUAY-8800) (#3780)
Move Quay to python version 3.12 and switch out rehash with the resumable hash library.
2025-05-29 09:35:48 -04:00
Michaela Lang
be82aefb44 proxycache(permissions): CVE-2025-4374 (PROJQUAY-8892) (#3941)
fixing CVE-2025-4374 by extending the create_repository method to understand if we are requesting a proxy_cache repository
added unittests for create_repository when proxy_cache.
2025-05-22 09:09:02 -04:00
Brandon Caton
feb2f2e459 deps: upgrading gevent (PROJQUAY-8938) (#3928)
Upgrading gevent and greenlet
2025-05-19 13:08:04 -04:00
Sunandadadi
8abbfc8ef0 migration: adding conditional check on manifestblob alter column type (#3886)
* migration: adding conditional check on manifestblob alter column type

* convert string to lower

* fix

* fixing column type fetch
2025-05-13 13:32:38 -04:00
Sunandadadi
d3bcbe0610 migration: alter id column type for table manifestblob (#3885)
* migration: alter id column type for table manifestblob

* adding raw queries
2025-05-13 10:21:20 -04:00
Brandon Caton
3f34e3a822 Reverting PROJQUAY-8536 (#3833)
* Revert "bug: Adding allow hidden flag while looking up for manifests (PROJQUAY-8536) (#3722)"

This reverts commit f0c153fab5.

* Revert "proxy: moving manifest check to after upstream manifest fetch (PROJQUAY-8536) (#3814)"

This reverts commit 944edd064b.
2025-04-29 14:31:16 -04:00
Syed Ahmed
d674ab54cb db: use iterator chain for _get_user_repo_permissions (PROJQUAY-8839) (#3822)
* db: use iterator chain for _get_user_repo_permissions (PROJQUAY-8839)

Unwrapping can cause increase in CPU. Use iterator chain to let
the caller unwrap
2025-04-29 11:44:31 -04:00
Kenny Lee Sin Cheong
454f09ed89 logging: fix unreferenced variable from logging (PROJQUAY-8136) (#3819) 2025-04-25 18:44:46 +00:00
Syed Ahmed
36a552058a db: optimize _get_user_repo_permissions to send to read replica (PROJQUAY-8839) (#3818)
* db: optimize _get_user_repo_permissions to send to read replica (PROJQUAY-8839)

it uses a union query which doesn't invoke the replica selection
logic. Make this into 2 seperate queries

* fix unit tests
2025-04-25 09:36:43 -04:00
Brandon Caton
944edd064b proxy: moving manifest check to after upstream manifest fetch (PROJQUAY-8536) (#3814)
moving manifest check to after upstream manifest fetch
2025-04-24 14:22:18 -04:00
Kenny Lee Sin Cheong
5c51ab85e1 gc: garbage collect manifests not targetted by any tags when deleting repository (PROJQUAY-8136) (#3797)
* gc: garbage collect manifests not targetted by any tags when deleting repository (PROJQUAY-8136)

* test untagged manifest removal
2025-04-22 10:42:48 -04:00
Sunandadadi
3bf6e1b16b db: moving robot search and find repo to garbage collect queries to read replica (PROJQUAY-8792) (#3795)
* db: moving robot search and find repository to garbage collect queries to read replica (PROJQUAY-8792)

* removing lookup_robot from read_replica
2025-04-17 15:13:51 +00:00
Sunandadadi
fda188e23f db: revert get_namespace_user from read replica (PROJQUAY-8792) (#3796) 2025-04-17 15:06:37 +00:00
Sunandadadi
43bc1db13e db: moving get sorted matching repos and find repos to garbage collect to read replica (PROJQUAY-8792) (#3782) 2025-04-16 13:13:06 -04:00
Marcus Kok
b629054fe5 reconciler: fix typo in exception type (PROJQUAY-0000) (#3779)
* fix typo in exception type

* update test cases
2025-04-16 09:39:29 -04:00
Sunandadadi
275b6e8540 db: moving robot search query to read replica (PROJQUAY-8792) (#3781) 2025-04-15 09:51:12 -04:00
Sunandadadi
433124d95f db: moving get user from username query to read replica (PROJQUAY-8792) (#3773) 2025-04-14 11:58:06 -04:00
Sunandadadi
58357ca84f db: moving get user repo permissions query to read replica (PROJQUAY-8792) (#3772) 2025-04-10 15:16:04 -04:00
Marcus Kok
5e2fbd986b marketplace: free tier integration for reconciler (PROJQUAY-5698) (#3589)
free sku integration for reconciliation worker
2025-04-08 08:55:37 -04:00
Shubhra Deshpande
f0c153fab5 bug: Adding allow hidden flag while looking up for manifests (PROJQUAY-8536) (#3722)
When an image is pulled by digest, a temp tag is created to prevent the manifest from being garbage collected. This is true when a manifest list is pulled by tag as well. However, if this temporary tag expires (default is 1 day for proxied organizations) and the same manifest is pulled again by digest, the system attempts to create the manifest again, leading to an integrity error because the manifest already exists in the database.

---------

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-03-20 11:03:11 -04:00
Shubhra Deshpande
b48e1b47ad chore: added test for proxy cache (PROJQUAY-8440) (#3610)
* adding unit test for proxy cache

---------

Co-authored-by: shudeshp <shudeshp@redhat.com>
2025-02-03 19:20:08 +00:00
Kenny Lee Sin Cheong
b674c7cfc4 gc: Allow setting a deleted repo name's suffix (PROJQUAY-8408) (#3563)
Allows adding a suffix to uuid repo names to collect metrics
2025-01-13 14:21:47 -05:00
Marcus Kok
8d835a9a94 worker: include orgs in reconciler runs (PROJQUAY-8431) (#3560)
Adds an include_orgs param to the active users query used by the reconciler and sets it to true for reconciler runs

Reconciler is not including orgs as a candidate for creating corresponding RH entitlements. As a result it misses users with stripe billing that are considered orgs.
2025-01-10 13:05:22 -05:00
Sunandadadi
70a0edef5b teamsync: peewee integrity error check on teamsync (PROJQUAY-7747) (#3550)
This issue should fix PROJQUAY-7747 and PROJQUAY-8130. Adding IntegrityError to the try except block to skip adding existing user to team.
2025-01-09 15:40:01 -05:00
Marcus Kok
f69716b867 marketplace: splittable sku for MW02702 (PROJQUAY-8151) (#3389)
* marketplace: splittable sku for MW02702 (PROJQUAY-8151)
* Alembic migration to drop unique constraint on the orgrhsubscriptions
  table
* Can split sub quantities of MW02702 across multiple orgs
* Can specify quantity for the MW02702 SKU across orgs on react UI
* Update angular UI to allow user to specify quantities for MW02702
2025-01-09 09:37:09 -05:00
Kenny Lee Sin Cheong
6c10313eea gc: allow setting the repositorygc queueitems' availability (PROJQUAY-8408) (#3551) 2025-01-07 15:05:17 -05:00
Marcus Kok
de9f42c2cc chore: Revert "cve: upgrade elasticsearch and elasticsearch-dsl to 8.13.0 (PROJQUAY-6962)" (PROJQUAY-6962) (#3449)
Revert "cve: upgrade elasticsearch and elasticsearch-dsl to 8.13.0 (PROJQUAY-…"

This reverts commit fb95c90574.
2024-12-04 13:06:13 -05:00
Marcus Kok
fb95c90574 cve: upgrade elasticsearch and elasticsearch-dsl to 8.13.0 (PROJQUAY-6962) (#3418)
* upgrade elasticsearch and elasticsearch-dsl to 8.13.0
* update tests and elasticsearch logs to handle new major version
* pinning elasticsearch-transport according to check requirements.txt test
2024-12-02 09:50:42 -05:00