* chore(pre-commit): match black version with requirements-dev
* run `make black` against repo
* ci: switch to black 24.4.2
* fix: py312
* fix: flake8 errors
* fix: flake8 conflicts
* chore: add git blame ignore revs file
* fix(test): prevent MySQL deadlocks in parallel proxy model tests (PROJQUAY-0000)
Mark all registry proxy model test classes to run serially using
pytest-xdist group markers. These tests all use the same "quayio-cache"
organization and were causing MySQL deadlocks when run in parallel
across multiple workers with pytest -n auto.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
* fix(test): resolve Flask app naming conflict in quotaregistrysizeworker tests (PROJQUAY-0000)
Import Flask app with alias to avoid conflict with pytest 'app' fixture.
The test was using 'app.config' but 'app' resolved to a pytest fixture
definition instead of the Flask application object.
Follows the same pattern as test_securityscanningnotificationworker.py.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Brady Pratt <bpratt@redhat.com>
---------
Signed-off-by: Brady Pratt <bpratt@redhat.com>
Co-authored-by: Claude <noreply@anthropic.com>
* Fix race conditions in pull metrics tracking and flushing
Replace non-atomic operations with atomic Redis operations to prevent
data loss when concurrent pulls occur during flush operations.
* fixing tests
* updating tests
* added uuid to the rename factors to ensure unique key at concurrent requests
---------
Co-authored-by: shudeshp <shudeshp@redhat.com>
* Added redis database and redis flush worker
* updated digest validation
* adding test coverage for scan_keys function
* adding test coverage
* added tests for scan function
* added coverage for flush to database function
* added coverage for gnuicorn worker initialization
* Replaced mock methods in the test with actual function calls
---------
Co-authored-by: shudeshp <shudeshp@redhat.com>
* fix: resolve Sentry/OpenTelemetry integration conflicts
Configure Sentry to use minimal integrations when OTEL_TRACING is enabled
to prevent instrumentation conflicts that broke exception capture
---------
Co-authored-by: shudeshp <shudeshp@redhat.com>
* proxycache: Download blob not cached when pulling manifests with blob available locally (PROJQUAY-6708)
* Skip downloading blobs without placeholders
Adds an include_orgs param to the active users query used by the reconciler and sets it to true for reconciler runs
Reconciler is not including orgs as a candidate for creating corresponding RH entitlements. As a result it misses users with stripe billing that are considered orgs.
* notifications: fetch autoprune tags with multiple policies for image expiry notification(PROJQUAY-8117)
* don't fetch notifications if tags expiry is greater than notification days + add tests
Allows users to specify a regex tag pattern when creating namespace/repository autoprune policies via the new UI. Users will have the option to prune tags that only match the tag pattern or exclude tags that match the tag pattern.
* registry: implements the OCI 1.1 referrers API
Migrations:
- Adds a subject column for lookup
- Adds a subject_backfilled column to track status of the backfilling
of existing manifests
- Adds a manifest_json column making use of postgres' JSONB support,
for future use.
Manifestsubjectbackfillworker: Indexes existing manifests for possible
existing subject field.
* Deprecate IGNORE_UNKNOWN_MEDIATYPES
* Cleanup
* marketplace: change redhatsubscriptions model to return a list of customer ids (PROJQUAY-7129)
* Use list comprehension in model and expect to return None if no rows are
returned
* fix behavior for reconciler and fix unit test for reconciler
Changes marketplace api handler to now return a list of all web customer ids for a user.
Allows quay.io to keep track of all web customer ids instead of just personal types.
* billing: marketplace UI
adds UI in billing section for managing user and org-bound skus
add more unit tests for org binding
changed endpoint for bulk attaching skus to orgs
* update reconciliationworker to use webCustomerId instead of
ebsAccountNumber
* fix reconciler where it was incorrectly using the ebsAccountNumber to
create subscriptions
* add job to reconciler so that it reconciles different ids between the
database and the user api
* separate skus to be used by billing and skus to be used by reconciler
* chore: pass config to isort as it doesn't always detect it
* chore: mark package "test" as local, not stdlib
* chore: remove "isort: skip_file"
* chore: fix app in test_load_security_information
* chore: fix app in test_notification
* chore: fix app in test_index_report
* add migration for orgrhskus table
* add endpoints for managing and listing skus bound to an org
* create checks in billing flow to look for org-bound skus
* refactor RH marketplace api objects to be more usable in tests
* update cypress test db data and exclude it from pre-commit hook formatting
Garbage collect manifests no longer referenced in Quay from the
security scanner service.
Also moved quota related code from data/registry_model/ to data/model/
to avoid circular dependencies.
Allows for only unique blobs are counted at the namespace and repository level. Calculation includes manifest list sizes.
Add's the following internal configurations that default to true:
QUOTA_INVALIDATE_TOTALS: Invalidates calculated totals when FEATURE_QUOTA_MANAGEMENT is set to false
RESET_CHILD_MANIFEST_EXPIRATION: Resets the expiry for child manifests on push of the manifest list for immediate GC eligibility
PERMANENTLY_DELETE_TAGS: Enables features related to the permanent deletion of tags outside the configured time machine window
By default, Redlock creates a new client per instance. Using the
provided factory allows Redlock to reuse a single connection per
instance and avoid running out of connections. e.g When a worker tries
to get a lock, it should not open new connections every time.
Increase sleep duration between queue polls on
WorkerSleepException. This will give more time before retrying after
failing to acquire a lock.
Migrate from using boto2 to boto3. Changes include:
- Removes explicit bucket addressing style: Boto3 will initially try virtual-style addressing first then fallback to path-style addressing (https://github.com/boto/boto3/blob/develop/docs/source/guide/configuration.rst)
- GCS workarounds to use boto3:
- Handles CORS config
- Update signed url access key parameter name
- Uses ListBucket V1 API
- On client-side chunks join, copy using non-multipart api: Use copy_from instead of copy when joining chunks client-side. This is because copy assumes multipart upload should be used which GCS and Rados are not compatible with (S3's version. They have their own parallel upload api)
- Update RDS healthcheck to use boto3
