1
0
mirror of https://github.com/quay/quay.git synced 2026-01-26 06:21:37 +03:00
Commit Graph

274 Commits

Author SHA1 Message Date
Kenny Lee Sin Cheong
dba302b5f1 users: default to true if LDAP_RESTRICTED_USER_FILTER is not set (PROJQUAY-4776) (#1645)
When LDAP is used and FEATURE_RESTRICTED_USERS is set, if
LDAP_RESTRICTED_USER_FILTER is set, then, by default, all ldap users
are restricted.
2022-11-25 13:39:07 -05:00
Kenny Lee Sin Cheong
b128936b50 users: fix behavior when using ldap and restricted user whitelist is set (PROJQUAY-4767) (#1640) 2022-11-23 15:22:15 -05:00
Kenny Lee Sin Cheong
fe2b89d656 logs: create action logs on proxy cache config creation/deletion (PROJQUAY-4718) (#1625) 2022-11-22 12:36:57 -05:00
Kenny Lee Sin Cheong
0caa4203ec users: prevent CREATE_NAMESPACE_ON_PUSH is restricted (PROJQUAY-4702) (#1621)
Prevent creating namespaces/orgs on pushes (CREATE_NAMESPACE_ON_PUSH)
if user is restricted.

Also updates RESTRICTED_USERS_WHITELIST to defaults to all if not set,
given that FEATURE_RESTRICTED_USERS is set.
2022-11-09 18:23:01 -05:00
Kenny Lee Sin Cheong
64ec15605c superusers: grant superusers additinonal org permissions (PROJQUAY-4687) (#1613)
When FEATURE_SUPERUSERS_FULL_ACCESS is set, grant superusers org admin permissions:
- PROJQUAY-4687
- PROJQUAY-4690
- PROJQUAY-4693
- PROJQUAY-4697
- PROJQUAY-4701
2022-11-07 17:41:18 -05:00
Harish Govindarajulu
83c866193d Fix proxy-cache overflow quota limit when pulling same image (PROJQUAY-4670) (#1597)
Signed-off-by: harishsurf <hgovinda@redhat.com>
2022-10-28 16:54:53 -04:00
Kenny Lee Sin Cheong
c84067a4d6 users: add restricted users' filter (PROJQUAY-1245) (#1551)
- Similar to LDAP_SUPERUSER_FILTER, add a specific filter to define
restricted users, based on the LDAP_USER_FILTER
- restrict writes on restricted users' own namespace. Normal
permissions applies on organization membership
- add global readonly superuser GLOBAL_READONLY_SUPER_USERS (PROJQUAY-2604)
- Removes RESTRICTED_USER_INCLUDE_ROBOTS, FEATURE_RESTRICTED_READ_ONLY_USERS
2022-10-28 13:38:37 -04:00
Joseph Crosland
98801bfd3e secscan: Generate key to reduce vulnerabilities (PROJQUAY-4562) (#1547)
Currently Quay is displaying the Clair response with no
interpretation meaning when Clair reports on vulns per repo
they appear to be duplicated (RHEL based images with multiple
repos and packages existing in more than one). The correct way
to fix this is via an interpretation layer in Quay, this change
is a stop-gap to improve readability for the user.

Signed-off-by: crozzy <joseph.crosland@gmail.com>

Signed-off-by: crozzy <joseph.crosland@gmail.com>
2022-10-18 09:54:38 -07:00
Kenny Lee Sin Cheong
14e87bd41e users: fix missing references in ldap for superusers (#1542) 2022-09-26 16:15:52 -04:00
Harish Govindarajulu
8a7aeec7c6 Evict LRU proxied images when quota limit is reached (PROJQUAY-3063) (#1493)
Signed-off-by: harishsurf <hgovinda@redhat.com>
2022-09-23 11:22:32 -04:00
Kenny Lee Sin Cheong
2b3c3cc584 api: feature to limit org creation to superusers (PROJQUAY-1245) (#1516)
* api: feature to limit org creation to superusers

Introduces the following configuration flags:

FEATURE_SUPERUSERS_ORG_CREATION_ONLY:
    Limit org creation to superusers only

FEATURE_SUPERUSERS_FULL_ACCESS:
    Grant superusers read/write access to registry content in all namespaces

FEATURE_RESTRICTED_USERS:
    Users considered as restricted are not anle to create organization

RESTRICTED_USERS_WHITELIST:
    Whitelist for FEATURE_RESTRICTED_USERS

RESTRICTED_USER_INCLUDE_ROBOTS:
    Whether or not to include the user namespace's robots

RESTRICTED_USER_READ_ONLY:
    Only allow read-only operations for restricted users

* Revert superuser repositorylist endpoint
2022-09-23 09:41:23 -04:00
Sunandadadi
1e136d6dd0 User: Added function to fetch users public repositories count (PROJQUAY-0000) (#1540)
* User: Added function to fetch users public repositories count (PROJQUAY-0000)

* Fixing Formatting (Flake8)
2022-09-22 13:14:14 -04:00
Sunandadadi
adf709568e Export Compliance: Fetching quay user data from federated login username (PROJQUAY-0000) (#1530)
* Fetching quay.io user data from federated login username (PROJQUAY-0000)

* Adding tests
2022-09-20 10:39:22 -04:00
Dave O'Connor
d4c0c1e4ed Fix super (#1533) 2022-09-16 15:31:04 -04:00
Dave O'Connor
dfe4d9f98b Fix typo (#1531) 2022-09-15 15:42:12 -04:00
Ivan Bazulic
ee5ff7141f bug: Increase column size in logentry3 table (PROJQUAY-4305) (#1510)
* bug: Increase column size in logentry3 table (PROJQUAY-4305)
We increase the size of `metadata_json` column in `logentry3` table from `TEXT` to `MEDIUMTEXT` on MySQL deployments to prevent the UI from erroring out when reading and decoding the audit log. This only touches MySQL, PostgreSQL is unaffected.
2022-08-31 14:23:16 -04:00
Jonathan King
c36945b836 healthcheck: Use db_kwargs in health check (PROJQUAY-4222) (#1507)
- The database health check is currently not using the db_kwargs and not using ssl settings
- This is causing health check failures for MySQL behind SSL
2022-08-31 14:20:56 -04:00
Flavian Missi
c04d0a644d registry: correctly bump tag expiration when tag changes upstream (PROJQUAY-3976) (#1479) 2022-08-16 16:41:00 +02:00
Kenny Lee Sin Cheong
070f464b14 users: superuser group in federated identity provider (PROJQUAY-3924) (#1464) 2022-08-11 15:36:10 -04:00
Kenny Lee Sin Cheong
78896aa0ce billing: fix new private repo count (PROJQUAY-4208) (#1463) 2022-08-01 14:03:22 -04:00
Kenny Lee Sin Cheong
1777897981 billing: add new larger stripe plan (PROJQUAY-4208) (#1462) 2022-08-01 13:06:41 -04:00
Harish Govindarajulu
342a50eb1f Proxy cache: respect CREATE_PRIVATE_REPO_ON_PUSH flag (PROJQUAY-3743) (#1426)
Signed-off-by: harishsurf <hgovinda@redhat.com>
2022-07-26 10:01:28 -04:00
Brandon Caton
9eb4fb6aa4 Revert workqueue refactor (#1456)
Currently the prometheus and GC workers are not running correctly.
Reverting the following commits:
- 4e1a985e70
- dac183a1ef
- 68a0d9eaf0
- af1aacea08
- f334b80098
2022-07-22 13:11:39 -04:00
Kenny Lee Sin Cheong
871c43eaf3 api: handle missing tag on DELETE tag api (#1444) 2022-07-15 13:47:20 -04:00
Oleg Bulatov
4e1a985e70 Initialize model.config on its import (PROJQUAY-4006) (#1412)
* Initialize model.config on its import

* Initialze registry_model and secscan_model on import

Co-authored-by: Dave O'Connor <1656866+HammerMeetNail@users.noreply.github.com>
2022-07-12 01:15:30 +02:00
Kenny Lee Sin Cheong
26a334f5d8 mirroring: fix mirror claims for multiple processes (PROJQUAY-3982) (#1433)
Previous logic for claiming mirror ("locking") relied on the value
returned from updating the database row. Since this was always being
updated with a new expiration time, it would always succeed, even when
another process had already claimed the same mirror.
2022-07-11 11:15:04 -04:00
Joseph Crosland
5291daf8fe secscan: Don't delete manifest security status on error (PROJQUAY-4060) (#1434)
Currently if Clair returns errors Quay will delete the security
status for that manifest, meaning it will revert to a "Queued"
status and needs to be reindexed. This in-turn means that the
functionality is not immediately recovered when Clair becomes
healthy. It is also generally a bad idea (except in specific
use-cases to have a data modification in a read path).

Signed-off-by: crozzy <joseph.crosland@gmail.com>
2022-07-11 10:59:36 -04:00
Oleg Bulatov
366be31896 Update type hints (#1409)
* Update peewee types

Also remove tools/sharedimagestorage.py as it doesn't work anymore.

    tools/sharedimagestorage.py:3: error: "ModelSelect[ImageStorage]" has no attribute "annotate"

* Remove endpoints/api/test/test_security.py from exclude list

* Format storage/test/test_azure.py
2022-06-27 17:52:29 +02:00
Oleg Bulatov
cf52f5e371 chore: Use Python 3.9 (#1382)
* Use Python 3.9

* Adapt code for Python 3.9
2022-06-20 14:05:47 +02:00
Oleg Bulatov
68a0d9eaf0 Initialize database and log_model on their import (#1396) 2022-06-17 18:36:13 +02:00
Brandon Caton
a0df895005 quota: Fix calculating org size (PROJQUAY-3889) (#1391)
Fixing database call which will now match on manifest vs repository to correctly calculate org size.
2022-06-16 13:58:27 -04:00
Oleg Bulatov
ed8f12ba34 Enable some flake8 checks in CI (#1389) 2022-06-16 18:22:16 +02:00
Brandon Caton
f28b35cc31 quota: Fix join on caching repo size (PROJQUAY-3889) (#1378)
Method for calculating size of repo joined on repo id instead of manifest id. This causes deleted manifests to be counted. Change updates call to join on manifest id.
2022-06-14 11:15:22 -04:00
Flavian Missi
f2c4375f65 fix: enable non-admins to cache images via pull-thru (PROJQUAY-3806) (#1366)
introduces a check of the OrganizationMemberPermission for pulls
against a proxy org.

if the proxy cache feature is disabled, or the org is not a proxy org
the check is not performed and Quay will behave normally.

this check does not mean pulls will work transparently though -
non-admin users need to be added to a team in the proxy org with the
member role, and default read and write permissions need to be given to
that team so that non-admin users can pull and update the cache for
images they do not own (the user who first pulls an image ends up
owning the repository since that is when the repo gets created).
2022-06-14 07:43:05 +02:00
Oleg Bulatov
5eaf0584db chore: Run mypy as CI job (#1363)
* Run mypy as CI job

* Fix peewee.pyi and configure pyright
2022-06-13 11:01:17 +02:00
Flavian Missi
d20542aa45 fix several warnings (#1374)
mostly invalid escape sequences
2022-06-10 11:46:56 +02:00
Kenny Lee Sin Cheong
5471d3cbcb secscan: deprecate support for Clair V2 (PROJQUAY-2837) (#951)
Removes read support for Clair V2, along with the need to package
jwtproxy with Quay.

TODO: Drop deprecate image api + image table, remove image data model.
2022-05-31 10:15:54 -04:00
kwestpharedhat
1e4871eca2 Quota: Add Cache To Tag Delete (PROJQUAY-3828) (#1345)
Add quota cache to delete tag
2022-05-27 10:04:21 -05:00
Flavian Missi
d029a4652d pull-thru: bump expiration of all parent manifest lists on pull (#1336)
the previous code relied on manifests always having a single manifest
list as their parents, which is false.

fixes https://issues.redhat.com/browse/PROJQUAY-3818
2022-05-25 13:32:12 +02:00
Sunandadadi
59d586c4c6 Quota Management: Adding default quota check for image push (PROJQUAY-3789) (#1316)
* Quota Management: Adding default quota check for image push (PROJQUAY-3789)

* Removing redundant if else check

* Fixing formatting

* Setting default quota bytes to 0
2022-05-11 14:20:38 -04:00
Flavian Missi
ac6a8d2f8b PROJQUAY-3750: support registries that do not return a digest header (#1310)
The distribution spec does not require the docker-content-digest header
to be set in response to a manifest GET/HEAD request.

This changes both the proxy client and the registry proxy model to
correctly check whether a manifest is up-to-date with the upstream
registry or not when no digest header is received.

NOTE: when checking staleness against registries that do not return the
docker-content-digest header, Quay will make a GET request to the
registry and calculate the digest from the manifest itself. GET requests
usually count towards rate-limiting.

This change also sets the accept-encoding header to 'identity'. The python
requests library seems to automatically set the accept-encoding header to
'gzip'. Dockerhub ignores that header when serving blobs, but some
registries don't (namely registry.access.redhat.com). When Quay receives a
gzipped config blob (have not tested non-config blobs) for some reason it
doesn't know how to handle it. I suspect it has to do wit the fact that in
this case the content-length header will differ from the actual size of
the response body, so when Quay tries to upload the blob it cannot
correctly calculate the actual blob size, so it does a partial upload to
its object storage, which then results in a digest mismatch error
(BlobDigestMismatchException).
2022-05-10 17:56:08 +02:00
Kenny Lee Sin Cheong
9d8ff6b1c1 quota: address possible integrity exception if computing size concurrently (PROJQUAY-3752) (#1303) 2022-05-09 15:28:01 -04:00
Kenny Lee Sin Cheong
3acc55b96c secscan: split the recent manifest chunk of work into multiple batch (PROJQUAY-3501) (#1300)
Split the work of indexing the recent manifests iterator into multiple
batches. This can reduce how often duplicate work happens when
allowing multiple workers to work on the same chunk of the table.
2022-05-04 11:02:56 -04:00
Kenny Lee Sin Cheong
c2ceda5a26 chore: various small changes to fix exceptions, remove unused code (#1295) 2022-04-29 15:11:15 -04:00
Sunandadadi
1e65bff9fc Quota: Raising exception when entered quota size is too big (PROJQUAY-3702) (#1290)
* Quota: Raising exception when entered quota size is too big (PROJQUAY-3702)

* fixing formatting

* Updated error message
2022-04-27 15:28:36 -04:00
Kenny Lee Sin Cheong
3739c1fc21 notification: fix user ref when creating notification for quota (PROJQUAY-3711) (#1288) 2022-04-27 14:49:53 -04:00
Kenny Lee Sin Cheong
9ac30007f9 secscan: cleanup secscan interface (PROJQUAY-3501) (#1284) 2022-04-25 14:24:08 -04:00
Kenny Lee Sin Cheong
ed77147b21 secscan: split indexing of recent manifest into separate background operation (PROJQUAY-3501) (#1281)
Index recent manifests in a separate background process, allowing the
main process to correctly select random slabs from the entire table
set and marking them completed in the allocator (rbtree). This avoids
the worker having to start iterating from the beginning of the table
whenever it is restarted.
2022-04-25 11:21:45 -04:00
Sunandadadi
eea7389a24 Quota: Show system default on UI when quota configuration for the org is not set (PROJQUAY-3518) (#1280)
* Quota: Show system default on UI when quota configuration for the org is not set (PROJQUAY-3518)

* Fixing formatting

* Added function to reduce redundancy and shortened warning message

* added missing parameter to function call

* Fixed organization quota consumption view

* Fixing formatting

* Checking for None before calling function
2022-04-22 16:34:04 -04:00
kwestpharedhat
79a3627a64 fix default system reject (#1277) 2022-04-21 14:07:43 -05:00