1
0
mirror of https://github.com/quay/quay.git synced 2026-01-29 08:42:15 +03:00
Commit Graph

234 Commits

Author SHA1 Message Date
Marcus Kok
dbfb47d6fb marketplace: change redhatsubscriptions model to return a list of customer ids (PROJQUAY-7129) (#2868)
* marketplace: change redhatsubscriptions model to return a list of customer ids (PROJQUAY-7129)

* Use list comprehension in model and expect to return None if no rows are
returned

* fix behavior for reconciler and fix unit test for reconciler
2024-05-09 11:51:06 -04:00
Marcus Kok
a69a450736 marketplace: change api call to return array of customer ids (PROJQUAY-7129) (#2867)
Changes marketplace api handler to now return a list of all web customer ids for a user.

Allows quay.io to keep track of all web customer ids instead of just personal types.
2024-05-09 09:31:02 -04:00
Brandon Caton
0de7f6f5de quota: adding VERIFY_QUOTA config option (PROJQUAY-6945) (#2798)
Adding VERIFY_QUOTA config option.
2024-03-28 09:44:53 -04:00
Marcus Kok
5f2c6ea087 marketplace: consider new skus in exclusion (PROJQUAY-6905) (#2794)
check if sku is part of reconciler skus when getting list
2024-03-26 13:43:47 -04:00
Marcus Kok
131d66d13f marketplace: add a check for masterEndSystemName when fetching subscriptions (PROJQUAY-6905) (#2775)
* add a check for masterEndSystemName when fetching subscriptions
* allow old SKUs to be used in org attachments
2024-03-25 13:47:22 -04:00
Syed Ahmed
09df48b5ee chore: fix typo (PROJQUAY-6397) (#2786)
fixing typo for DB_CONNECTION_POOLING
2024-03-22 12:11:47 -04:00
Syed Ahmed
5878631934 db: add config variable for DB pooling (PROJQUAY-6397) (#2783)
* db: add config variable for DB pooling (PROJQUAY-6397)

adding `DB_CONNECTION_POOLING` param to enable/disable
connection pooling for primary and replica DBs
2024-03-21 21:26:37 +00:00
Marcus Kok
5db4251114 marketplace: add expiration check to org subscription operations (PROJQUAY-6716) (#2696)
add expiration check to org subscription operations
2024-03-18 09:15:46 -04:00
Brandon Caton
d2a1fedab4 quota: refactor, tests, and optimizations (PROJQUAY-6735) (#2712)
Adds the following changes:
- Additional logging
- Moving checks before expensive DB calls
- Additional tests
2024-03-06 09:06:38 -05:00
Brandon Caton
4f0db4b90f quota: feature flagging quota edit/view/enforce (PROJQUAY-6734) (#2709)
feature flagging quota edit/view/enforce functionality
2024-03-05 13:10:18 -05:00
Brandon Caton
b641538bad ui: removing repo settings and build UI feature flags (PROJQUAY-6617) (#2680)
Removing repo settings and build UI feature flags since they should be enabled by default.
2024-02-19 09:19:10 -05:00
Kenny Lee Sin Cheong
e91b9e4543 registry: allow pushing manifests with artifactType and subject fields (PROJQUAY-6673) (#2659)
Allow manifests to be pushed with a artifactType and subject fields.
2024-02-12 13:32:43 -05:00
Sunandadadi
80c184f168 chore: extend market place subscription enddate (chore) (#2663)
test: extend market place subscription enddate
2024-02-08 11:09:56 -05:00
Marcus Kok
2ab7dc29f4 marketplace: add support for quantity from subscriptions api (PROJQUAY-6551) (#2633)
* Adds handling for when a subscription returned from the subscription watch api has a quantity greater than 1. Number of private repos should be correctly calculated using the quantity.

* Updates ui so that subscriptions can only be added to an org as a group, i.e. a subscription with quantity = 2 cannot be split across organizations.
2024-01-29 15:21:30 -05:00
Marcus Kok
54bc56d5be marketplace: return array of valid subscriptions when looking up subscription (PROJQUAY-6551) (#2613)
return array of valid subscriptions when looking up subscription
2024-01-17 10:07:29 -05:00
Daniel Messer
e8ff33e728 logs: add failure logging for login, push, pull and delete events (PROJQUAY-5411) (#1903)
* add login failure logging

Signed-off-by: dmesser <dmesser@redhat.com>

* move failure logging into credential validation

Signed-off-by: dmesser <dmesser@redhat.com>

* more precise tracking of affected users

Signed-off-by: dmesser <dmesser@redhat.com>

* fix indent

Signed-off-by: dmesser <dmesser@redhat.com>

* differentiate robots with wrong credentials

Signed-off-by: dmesser <dmesser@redhat.com>

* don't audit failures by default

Signed-off-by: dmesser <dmesser@redhat.com>

* discrete failure tracking for logins, push, pulls and deletes

Signed-off-by: dmesser <dmesser@redhat.com>

* refine log metadata

Signed-off-by: dmesser <dmesser@redhat.com>

* login failure log visualization

Signed-off-by: dmesser <dmesser@redhat.com>

* properly use data model

Signed-off-by: dmesser <dmesser@redhat.com>

* fix unit test bug

Signed-off-by: dmesser <dmesser@redhat.com>

* track non-existing repos differently

Signed-off-by: dmesser <dmesser@redhat.com>

* log view visualization of failed pushes and pulls

Signed-off-by: dmesser <dmesser@redhat.com>

* ensure all tests are conducted with failure logging

Signed-off-by: dmesser <dmesser@redhat.com>

* additional unicode protection

Signed-off-by: dmesser <dmesser@redhat.com>

* python black formatting

Signed-off-by: dmesser <dmesser@redhat.com>

* add cypress test data

Signed-off-by: dmesser <dmesser@redhat.com>

* add safety checks for ascii conversion attempts

Signed-off-by: dmesser <dmesser@redhat.com>

* adjusting unit test with correct error message

Signed-off-by: dmesser <dmesser@redhat.com>

* update to alembic head

Signed-off-by: dmesser <dmesser@redhat.com>

* add standard oauth token metadata in audit

Signed-off-by: dmesser <dmesser@redhat.com>

* update alembic head

Signed-off-by: dmesser <dmesser@redhat.com>

* correct field name

Signed-off-by: dmesser <dmesser@redhat.com>

* formatting

Signed-off-by: dmesser <dmesser@redhat.com>

* bump alembic head

Signed-off-by: dmesser <dmesser@redhat.com>

* refactor auth logging imports

Signed-off-by: dmesser <dmesser@redhat.com>

* bump alembic head

Signed-off-by: dmesser <dmesser@redhat.com>

* formatting

Signed-off-by: dmesser <dmesser@redhat.com>

* restore module

Signed-off-by: dmesser <dmesser@redhat.com>

* pre-commit fixes

Signed-off-by: dmesser <dmesser@redhat.com>

* adding missing default

Signed-off-by: dmesser <dmesser@redhat.com>

* bump alembic head

Signed-off-by: dmesser <dmesser@redhat.com>

* update test data

Signed-off-by: dmesser <dmesser@redhat.com>

* refactoring to save db calls

Signed-off-by: dmesser <dmesser@redhat.com>

* fix unit tests

Signed-off-by: dmesser <dmesser@redhat.com>

* handle unicode conversion errors on email look up

Signed-off-by: dmesser <dmesser@redhat.com>

* bump alembic head

Signed-off-by: dmesser <dmesser@redhat.com>

* proper debug logging and conditional db calls

Signed-off-by: dmesser <dmesser@redhat.com>

* omit wildcard import

Signed-off-by: dmesser <dmesser@redhat.com>

* re-add import

Signed-off-by: dmesser <dmesser@redhat.com>

---------

Signed-off-by: dmesser <dmesser@redhat.com>
2024-01-16 16:46:20 +01:00
Marcus Kok
26451766ad marketplace: make sure customer id from api is returned as an int (PROJQUAY-233) (#2590)
* make sure customer id from api is returned as an int
2024-01-08 15:41:34 -05:00
Marcus Kok
1c893baba5 marketplace: update reconciliationworker to use webCustomerId instead of ebsAccountNumber (PROJQUAY-233) (#2582)
* update reconciliationworker to use webCustomerId instead of
ebsAccountNumber

* fix reconciler where it was incorrectly using the ebsAccountNumber to
  create subscriptions
* add job to reconciler so that it reconciles different ids between the
  database and the user api
* separate skus to be used by billing and skus to be used by reconciler
2024-01-05 16:15:37 -05:00
Brandon Caton
ac221a601d ui: implementing build history page (PROJQUAY-6293) (#2523)
implementing build history page
2023-12-13 16:03:04 -05:00
Marcus Kok
1bf3f44820 marketplace: update user ebs number lookup to find personal account numbers (PROJQUAY-233) (#2545)
* update user ebs number lookup to find account numbers with type = person
2023-12-06 11:10:29 -05:00
Michaela Lang
0dfa72d036 user(robots): disallow robot login and create 2nd (PROJQUAY-5968) (#2483)
* Fixed typo, added ROBOTS_WHITELIST as without the ROBOTS_DISALLOW does not make sense

* added ROBOTS_WHITELIST default to config.py
2023-12-05 17:31:08 +01:00
Marcus Kok
4c2433413c marketplace: add logging for user api (PROJQUAY-233) (#2513)
* add logging for user api
2023-12-01 10:29:05 -05:00
Marcus Kok
3aa2121326 marketplace: return None if subscription api times out (PROJQUAY-5363) (#2437)
* handles `ReadTimeout` exception by returning `None` for subscription API
2023-10-27 09:27:16 -04:00
Marcus Kok
685cbef1a2 marketplace: use get method for ebsAccountNumber lookup (PROJQUAY-6219) (#2417)
use get method for ebsAccountNumber lookup
2023-10-18 14:33:51 -04:00
Syed Ahmed
27f1699b72 ui: Add delay after write operations (PROJQUAY-6146) (#2400)
This is useful for cases where reads and writes are
going to different DBs and there is a replication lag
between them
2023-10-12 12:45:16 -04:00
Brandon Caton
0e496b46a5 autoprune: add initial setup for the autoprune feature (PROJQUAY-6094) (#2277)
Adds the auto-prune worker, database models, and feature flag.
2023-10-05 13:37:36 -04:00
dependabot[bot]
d2e5a69b26 build(deps): bump pyjwt from 2.4.0 to 2.8.0 (#2166)
* build(deps): bump pyjwt from 2.4.0 to 2.8.0

Bumps [pyjwt](https://github.com/jpadilla/pyjwt) from 2.4.0 to 2.8.0.
- [Release notes](https://github.com/jpadilla/pyjwt/releases)
- [Changelog](https://github.com/jpadilla/pyjwt/blob/master/CHANGELOG.rst)
- [Commits](https://github.com/jpadilla/pyjwt/compare/2.4.0...2.8.0)

---
updated-dependencies:
- dependency-name: pyjwt
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Expect new messages

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Oleg Bulatov <oleg@bulatov.me>
2023-10-03 14:08:00 +02:00
Oleg Bulatov
84fa795ae7 chore: fix isort config and remove isort: skip_file (#2196)
* chore: pass config to isort as it doesn't always detect it

* chore: mark package "test" as local, not stdlib

* chore: remove "isort: skip_file"

* chore: fix app in test_load_security_information

* chore: fix app in test_notification

* chore: fix app in test_index_report
2023-09-21 11:46:03 -04:00
Michaela Lang
67028af9e2 user(robots): disallow robot login and create (PROJQUAY-5968) (#2155)
* add Postgresql client certificate authentication option

* user(robots): disallow robot login and create PROJQUAY-5968

Add a config flag `ROBOTS_DISALLOW` to turn off Robot login and creation
in Quay for all accounts.

with the Flag set existing Robots cannot login anymore
```
$ podman login -u milang+test2 -p W7B...HQA quay.example.com
Error: logging into "quay.example.com": invalid username/password
```

This behavior will be reported in the logs accordingly
```
gunicorn-registry stdout | 2023-08-24 19:08:01,907 [253] [WARNING] [auth.credentials] Failed to validate credentials for robot milang+test2: Robot account has been disabled. Please contact your administrator.
```

as well as in the UI when creating a Robot Account which will display
the `Robot account has been disabled. Please contact your
administrator.` message.

* Revert "add Postgresql client certificate authentication option"
  need to branch for PR (stupid me)

This reverts commit f8f54eead2.

* fixed error message to proper english and added schema definition for the option
2023-09-20 13:07:41 +02:00
Kenny Lee Sin Cheong
72f7c64ed6 chore: update werkzeug and related package versions (PROJQUAY-5098) (#1982)
* chore: update werkzeug and related package versions (PROJQUAY-5098)

Path converter related change reference: https://github.com/pallets/werkzeug/issues/2506

* Update query count
2023-09-12 11:51:09 -04:00
Brandon Caton
eede634af9 config: updating GeoLite2-Country (PROJQUAY-6023) (#2218)
updating GeoLite2-Country.mmdb
2023-09-11 11:42:09 -04:00
Marcus Kok
e44783fe19 billing: Assign SKU to org (PROJQUAY-5363) (#1989)
* add migration for orgrhskus table

* add endpoints for managing and listing skus bound to an org

* create checks in billing flow to look for org-bound skus

* refactor RH marketplace api objects to be more usable in tests

* update cypress test db data and exclude it from pre-commit hook formatting
2023-08-25 14:52:54 -04:00
Kenny Lee Sin Cheong
5f63b3a7bb chore: drop deprecated tables and remove unused code (PROJQUAY-522) (#2089)
* chore: drop deprecated tables and remove unused code

* isort imports

* migration: check for table existence before drop
2023-08-25 12:17:24 -04:00
Kenny Lee Sin Cheong
8d0ee3865e secscan: fix metrics to track all request types to secscan service (PROJQUAY-3839) (#2108) 2023-08-11 11:33:02 -04:00
Ivan Bazulic
20845a136f chore: Remove reference to Image table from the backfill replication script (#2110)
The `image` table is deprecated and also not used in the backfill replication script. This PR removes the reference left from the previous PR.
2023-08-10 12:09:10 -04:00
Jonathan King
faa0315a04 georep: Add error handling for remove location script (PROJQUAY-5878) (#2071)
- Gracefully handle errors that are thrown during remove location script
- This should handle scenarios where the location is not found or not provided, as well as other error cases
2023-08-02 12:14:26 -04:00
Kenny Lee Sin Cheong
0ea4826956 secscan: garbage collect manifests (#1663)
Garbage collect manifests no longer referenced in Quay from the
security scanner service.

Also moved quota related code from data/registry_model/ to data/model/
to avoid circular dependencies.
2023-07-06 11:59:47 -04:00
Oleg Bulatov
ea49bb53a3 chore: Bump mypy (#1962) 2023-06-28 13:47:32 +02:00
Marcus Kok
e622414280 config: Enable notifications for new indexes by default (PROJQUAY-5682) (#1993)
* config: Set feature flag default for new vulnerability notifications to True
(PROJQUAY-4659)

Change the default from `False` to `True` for
`FEATURE_SECURITY_SCANNING_NOTIFY_ON_NEW_INDEX`.

Since this flag addresses a bug, it should be enabled by default.

* add mock return values for unit tests
2023-06-27 16:01:43 +02:00
Syed Ahmed
2371c4d605 api: allow robot token creation with a pre-defined token (PROJQUAY-5414) (#1972)
* api: allow robot token creation with a pre-defined token (PROJQUAY-5414)

This is for usecases where we want to explicitly create a robot token
with a pre-defined token in case of migration events when we migrate
from one quay to another, we can re-use the same robot token to
avoid resetting it in all places it is used
2023-06-23 15:53:30 -04:00
Ivan Bazulic
d7864ed4ee ui: Add custom TOS support (PROJQUAY-5648) (#1973)
* ui: Add custom TOS support
Enable adding of a custom TOS in the Quay footer for on-premise installations via the optional `TERMS_OF_SERVICE` config flag.
If the flag is not defined, the TOS field will not be visible on Quay rendering. Also changes the behaviour of the discovery endpoint to include custom terms of service instead of pointing to Quay.io terms of service for all installations.

* Fix formatting errors

* Fix TOS page for Quay.io deployments

* Change TOS variable name to TERMS_OF_SERVICE_URL for better clarity

* Removed reference to Quay.io from the discovery endpoint description
2023-06-19 11:44:52 -04:00
Daniel Messer
e1e8dc8efb audit: ignore errors due to read-only mode in audit logs (PROJQUAY-5598) (#1928)
* ignore errors due to read-only mode in audit logs

Signed-off-by: dmesser <dmesser@redhat.com>

* remove superfluous ReadOnlyMode handling

Signed-off-by: dmesser <dmesser@redhat.com>

* add debug log messae

Signed-off-by: dmesser <dmesser@redhat.com>

---------

Signed-off-by: dmesser <dmesser@redhat.com>
2023-06-16 12:08:27 +02:00
Kenny Lee Sin Cheong
d48df9b251 chore: deprecate image table support (PROJQUAY-522) (#1841)
Remove code related to deprecated image table
2023-06-14 10:54:54 -04:00
Jonathan King
88fd1bae9a geo-rep: Initialize features from config in util script (PROJQUAY-5627) (#1966)
- Features must be initialized in order to check if storage replication is enabled, so we need to import and intialize the config from inside the util script
2023-06-12 15:39:21 -04:00
Jonathan King
2d93fc7cd0 geo-rep: Add util script to remove geo-rep location and corresponding image locations (PROJQUAY-4995) (#1892)
- This removes first the ImageStoragePlacements that are associated with the location
- Once those are cleaned up, the ImageStorageLocation itself is removed from the database
2023-06-06 17:25:56 -04:00
Marcus Kok
de8c48fa28 marketplace: fixing allowed repo count (PROJQUAY-5513) (#1891)
* Aggregate stripe plans and subscriptions.

* Validate end date for subscriptions when fetching from marketplace.

* Check subscription returned from stripe api is non-null value when
finding stripe plan for sku.
2023-05-25 10:46:58 -04:00
Brandon Caton
6cf0a3531b quota: calculating registry size (PROJQUAY-5476) (#1879)
Allows superusers to trigger a calculation of the deduplicated registry size. A superuser can go to the organization panel of the superuser page and select Calculate to queue a calculation of the registry total. The total will only be calculated when requested. Includes warning to user of increase of database load when running calculation.
2023-05-24 17:10:40 -04:00
Sunandadadi
230fd24f3d Util: Reading new UI feedback form url from config parameter (PROJQUAY-5463) (#1902)
* Util: Reading feedback form url from config parameter (PROJQUAY-5463)

* updating form link for quay plugin
2023-05-24 10:43:25 -04:00
Harish Govindarajulu
490a6b2ce8 logs: Add SSL cert support and test coverage for splunk logging (PROJQUAY-4993) (#1878)
Signed-off-by: harishsurf <hgovinda@redhat.com>
2023-05-19 16:24:13 -04:00
Daniel Messer
41cd8330d0 logs: add audit log events for login/logout to Quay (PROJQUAY-2344) (#1866)
* add auditing of login and logout events

Signed-off-by: dmesser <dmesser@redhat.com>

* python black formatting

Signed-off-by: dmesser <dmesser@redhat.com>

* correctly identify username during

Signed-off-by: dmesser <dmesser@redhat.com>

* revert change to existing code

Signed-off-by: dmesser <dmesser@redhat.com>

* remove superfluous ip tracking

Signed-off-by: dmesser <dmesser@redhat.com>

* beautify login messages

Signed-off-by: dmesser <dmesser@redhat.com>

* remove unused import

Signed-off-by: dmesser <dmesser@redhat.com>

* move alembic patch to head

Signed-off-by: dmesser <dmesser@redhat.com>

* correctly log target namespace for robots

Signed-off-by: dmesser <dmesser@redhat.com>

* python black formatting

Signed-off-by: dmesser <dmesser@redhat.com>

* login event detection refinement

Signed-off-by: dmesser <dmesser@redhat.com>

* add missing visualization

Signed-off-by: dmesser <dmesser@redhat.com>

* enrich login event data

Signed-off-by: dmesser <dmesser@redhat.com>

* make login auditing configurable

Signed-off-by: dmesser <dmesser@redhat.com>

* make update-testdata

Signed-off-by: dmesser <dmesser@redhat.com>

---------

Signed-off-by: dmesser <dmesser@redhat.com>
2023-05-17 11:22:35 +02:00