* marketplace: change redhatsubscriptions model to return a list of customer ids (PROJQUAY-7129)
* Use list comprehension in model and expect to return None if no rows are
returned
* fix behavior for reconciler and fix unit test for reconciler
Reducing the number of DB calls in the repo list endpoint with quota enabled by:
- Adding the id to RepositoryBaseElement when the repositories are initially fetched, removing the need to fetch the repository ID's again
- Fetching the repository sizes with a single DB call using the IN operator
* db: add config variable for DB pooling (PROJQUAY-6397)
adding `DB_CONNECTION_POOLING` param to enable/disable
connection pooling for primary and replica DBs
* db: use read replica for selected queries (PROJQUAY-6397)
We add a new param `can_use_read_replica` to the `select`
query. This allows us to choose which queries we want to
send to the read replica. This is useful in cases where
the read replica lags behind the primary and some queries
need the latest data
* feat: Add support for auto pruning at repository level (PROJQUAY-6354)
* Add repositoryautoprunepolicy table to alembic migration script
* Add repository auto-prune policy endpoints
* Add UI for repository auto-pruning policies
* case: apply repo auto-prune policy when no namespace policy given
* case: both namespace and repo pruning policy are given
* Add tests for repository autoprune endpoint
* Add cypress test for repository auto-prune
* Add repo auto-prune policy clean-up for repository deletion
* Add repository auto pruning tables to quay db snapshot for cypress tests
* Address review comments
* Add more tests + fix CI + reformat files
* Address review comments #2
---------
Signed-off-by: harishsurf <hgovinda@redhat.com>
* added base class for OIDC auth + UI
* adding read-only teams page + display sync config + option to remove team sync
* setting page in read only mode fix
* ui tests
* adding validation for group name input
* fixes based on review + fixing test suite
* add backend tests for externalOIDC
* minor fixes
* Adds handling for when a subscription returned from the subscription watch api has a quantity greater than 1. Number of private repos should be correctly calculated using the quantity.
* Updates ui so that subscriptions can only be added to an org as a group, i.e. a subscription with quantity = 2 cannot be split across organizations.
* billing: marketplace UI
adds UI in billing section for managing user and org-bound skus
add more unit tests for org binding
changed endpoint for bulk attaching skus to orgs
* update reconciliationworker to use webCustomerId instead of
ebsAccountNumber
* fix reconciler where it was incorrectly using the ebsAccountNumber to
create subscriptions
* add job to reconciler so that it reconciles different ids between the
database and the user api
* separate skus to be used by billing and skus to be used by reconciler
* api: add caching for get_repository method (PROJQUAY-6472)
* fixing formatting
* using pickle to serialize and deserialize objects
* adding model_cache parameter to lookup_repository func for interface and registry_proxy_model
* adding test
* formatting fix
* - added env option to select if we want to follow LDAP referrals
- added check on LDAP if the result is a list, it is an LDAP referral
* - change to config.yaml parameter option for enabling/disabling LDAP
referral lookups
- removed the per/method parameter and moved the referral into the Base
class
- verified behavior with changing the config.yaml option
`LDAP_FOLLOW_REFERRALS`
```
$ grep LDAP_FOLLOW_REFERRALS config.yaml
LDAP_FOLLOW_REFERRALS: 0
$ ./update_bundle
secret "quay-config-secret" deleted
secret/quay-config-secret created
deployment.apps/quay restarted
$ oc -n quay logs -f deploy/quay | grep referrals
gunicorn-web stdout | 2023-05-25 09:41:36,524 [74] [DEBUG] [data.users] LDAPUsers return referrals value 0
gunicorn-secscan stdout | 2023-05-25 09:41:38,093 [73] [DEBUG] [data.users] LDAPUsers return referrals value 0
gunicorn-registry stdout | 2023-05-25 09:41:38,410 [72] [DEBUG] [data.users] LDAPUsers return referrals value 0
$ sed -i -e " s#LDAP_FOLLOW_REFERRALS: 0#LDAP_FOLLOW_REFERRALS: 1#; " config.yaml
$ grep LDAP_FOLLOW_REFERRALS config.yaml
LDAP_FOLLOW_REFERRALS: 1
$ ./update_bundle
secret "quay-config-secret" deleted
secret/quay-config-secret created
deployment.apps/quay restarted
$ oc -n quay logs -f deploy/quay | grep referrals
gunicorn-secscan stdout | 2023-05-25 09:42:22,466 [72] [DEBUG] [data.users] LDAPUsers return referrals value 1
gunicorn-web stdout | 2023-05-25 09:42:22,496 [73] [DEBUG] [data.users] LDAPUsers return referrals value 1
gunicorn-registry stdout | 2023-05-25 09:42:23,658 [71] [DEBUG] [data.users] LDAPUsers return referrals value 1
```
* reformatted LDAPConnectionBulder call according to PEP
* fixed the formatting change according to pip black
* BREAKING CHANGE: LDAP lookup of robot accounts in the UI for granting
permission has been dropped. This impacts Users from LDAP
if they have not logged in already (pre provisioning).
the functions:
* _ldap_single_user_search
* _ldap_user_search
have been updated to provide `context` for dropping the LDAP query. As
the functions are used during login as well as during setting
permissions it is mandatory to be able to distinguish the context.
the functions:
* get_nonrobot_user
* find_user_by_email
from the module `data.model.user` have been utilized to identify, if the
user should be looked up in LDAP or not.
* Adding keepalive setting for LDAP connection in complex network
scenarios.
* removed PROJQUAY-5137 (part one) to simplify merging and rollback
-> Revert "BREAKING CHANGE: LDAP lookup of robot accounts in the UI for granting"
This reverts commit 8765093d73.
* fixed black
Currently if a new manifest is created that re-uses blobs already existing in the registry but not within the namespace it's possible for that manifest to be blocked if the newly referenced blobs exceed the quota limit. The issue being that the rejection only happens after the manifest has been created and tagged. This change blocks the manifest upload before the creation of the tag and prevents that tag from appearing in the UI and being pullable.
If the manifest upload get's rejected before the tag has been created for a new manifest, a temporary tag is created outside the time machine window so it is immediately available for GC.
* chore: pass config to isort as it doesn't always detect it
* chore: mark package "test" as local, not stdlib
* chore: remove "isort: skip_file"
* chore: fix app in test_load_security_information
* chore: fix app in test_notification
* chore: fix app in test_index_report
* add Postgresql client certificate authentication option
* user(robots): disallow robot login and create PROJQUAY-5968
Add a config flag `ROBOTS_DISALLOW` to turn off Robot login and creation
in Quay for all accounts.
with the Flag set existing Robots cannot login anymore
```
$ podman login -u milang+test2 -p W7B...HQA quay.example.com
Error: logging into "quay.example.com": invalid username/password
```
This behavior will be reported in the logs accordingly
```
gunicorn-registry stdout | 2023-08-24 19:08:01,907 [253] [WARNING] [auth.credentials] Failed to validate credentials for robot milang+test2: Robot account has been disabled. Please contact your administrator.
```
as well as in the UI when creating a Robot Account which will display
the `Robot account has been disabled. Please contact your
administrator.` message.
* Revert "add Postgresql client certificate authentication option"
need to branch for PR (stupid me)
This reverts commit f8f54eead2.
* fixed error message to proper english and added schema definition for the option
* add migration for orgrhskus table
* add endpoints for managing and listing skus bound to an org
* create checks in billing flow to look for org-bound skus
* refactor RH marketplace api objects to be more usable in tests
* update cypress test db data and exclude it from pre-commit hook formatting
