1
0
mirror of https://github.com/quay/quay.git synced 2026-01-26 06:21:37 +03:00
Commit Graph

130 Commits

Author SHA1 Message Date
Syed Mushtaq Ahmed
922a82a3d9 oauth: Add the code param to the oauthrize.html template (PROJQUAY-3648) (#1362)
Fixes an issue where the code param is not passed to the app redirect
URI if the user has not authorized the app before
2022-06-07 13:35:28 -04:00
Sunandadadi
f90e5e3dce Quota: Configuring Quota for user panel(PROJQUAY-3767) (#1334)
* Quota: Configuring Quota for user panel

* Added Quota Consumed column on Super users panel

* Fixing tests

* Fixing tests

* Adding tests for user quota operations

* Reverting org api changes + new endpoint for super user get method

* Reverting changes

* Added tests

* Fetching user namespace or organization
2022-05-31 11:22:37 -04:00
kwestpharedhat
1e4871eca2 Quota: Add Cache To Tag Delete (PROJQUAY-3828) (#1345)
Add quota cache to delete tag
2022-05-27 10:04:21 -05:00
Brandon Caton
fd9a6b2e60 v2auth: Check for user before creating org (PROJQUAY-3766) (#1315)
Currently when Quay is set to automatically create organizations it will check the user table for a matching username with organization set to true. This causes conflicts when a user already exists with the same username, where the check will fail and Quay will attempt to create an organization with the same name as the user. This change checks for only a matching username.
2022-05-11 13:28:34 -04:00
Kenny Lee Sin Cheong
9d8ff6b1c1 quota: address possible integrity exception if computing size concurrently (PROJQUAY-3752) (#1303) 2022-05-09 15:28:01 -04:00
Jonathan King
ad4bb6f185 compliance: Move export screening to RHSSO class (PROJQUAY-2056) (#1302)
- Created a RHSSO class
- Put export compliance logic in class override
2022-05-04 16:30:44 -04:00
Syed Mushtaq Ahmed
6effd4cdec auth: Add state to the oauthorize redirect (PROJQUAY-3648) (#1301)
Fixes issue where state is not passed to the redirect in an Oauth App
2022-05-04 11:11:19 -04:00
Sunandadadi
a983884e0f Quota API: Add super user permissions on Organization endpoints (PROJQUAY-3742) (#1296)
* Quota API: Add super user permissions on Organization endpoints (PROJQUAY-3742)

* Removing super user permissions form userquota endpoints

* Adding super user permission checks

* Moving super user scope decorator to class level
2022-05-02 10:36:02 -04:00
Kenny Lee Sin Cheong
c2ceda5a26 chore: various small changes to fix exceptions, remove unused code (#1295) 2022-04-29 15:11:15 -04:00
Syed Mushtaq Ahmed
edb4e72166 auth: Add state to the oauthorize page (PROJQUAY-3648) (#1292)
Add the `state` parameter to the oauth permissions page
that's rendered when asking user for permission.
2022-04-28 11:58:23 -04:00
kwestpharedhat
f4093b0db5 Quota: fix caching (PROJQUAY-3660) (#1291)
* fix caching

* fix caching race condition
2022-04-28 08:47:42 -05:00
Jonathan King
3140a62e01 Revert "Revert "quayio: Add export compliance service to Red Hat SSO (PROJQUAY-2056) (#1239)" (#1273)" (#1289)
This reverts commit 8324586e4c.
2022-04-27 13:16:53 -07:00
Kenny Lee Sin Cheong
3739c1fc21 notification: fix user ref when creating notification for quota (PROJQUAY-3711) (#1288) 2022-04-27 14:49:53 -04:00
Sunandadadi
5bb2c121b1 Quota: Show a different error message if default quota is set on removing quota (PROJQUAY-3657) (#1287)
* Quota: Show a different error message if default quota is set on removing quota (PROJQUAY-3657)

* Indentation fix
2022-04-27 10:48:33 -04:00
Sunandadadi
2d63be373f Quota API: Remove trailing backslash (PROJQUAY-3625) (#1286) 2022-04-26 18:12:51 -04:00
Sunandadadi
eea7389a24 Quota: Show system default on UI when quota configuration for the org is not set (PROJQUAY-3518) (#1280)
* Quota: Show system default on UI when quota configuration for the org is not set (PROJQUAY-3518)

* Fixing formatting

* Added function to reduce redundancy and shortened warning message

* added missing parameter to function call

* Fixed organization quota consumption view

* Fixing formatting

* Checking for None before calling function
2022-04-22 16:34:04 -04:00
kwestpharedhat
a6d4332016 fix monolithic uploads (#1275) 2022-04-21 12:01:46 -05:00
Sunandadadi
2dca65f9ce API: Removing internal only decorator from exposed Super user endpoints (#1271) 2022-04-21 12:29:11 -04:00
Kenny Lee Sin Cheong
8324586e4c Revert "quayio: Add export compliance service to Red Hat SSO (PROJQUAY-2056) (#1239)" (#1273)
This reverts commit 247fec3b05.
2022-04-20 15:30:54 -04:00
Sunandadadi
cd2889439b Quota Management: Quota settings on Organization view needs to be read only (PROJQUAY-3622) (#1263)
* Quota Management: Quota settings on Organization view needs to be read only (PROJQUAY-3622)

* Adding superuser permissions check on put and delete methods

* Reverting changes

* Reverting changes
2022-04-19 18:16:11 -04:00
Jonathan King
247fec3b05 quayio: Add export compliance service to Red Hat SSO (PROJQUAY-2056) (#1239)
- Add FEATURE_EXPORT_COMPLIANCE to config schema
- Add call to export compliance service during OAuth workflow when feature is enabled
2022-04-18 12:45:29 -04:00
Sunandadadi
3d6545b9da UI: Quota UI enhancements (PROJQUAY-0000) (#1242)
* Tracking aws ip ranges.json

* Reverting change

* Added default quota limit + added error display div + fixed indentation + minor restructuring of html

* Add check for non-negative or zero quota

* Added Limit percent check. numbers between 1-100 only acceptable

* Show warning note if no quota limit is set

* Show warning if no Reject type limit is selected

* Adding Remove button to delete quota configuration

* If Reject does not exist add default reject quota limit + css

* Throw error if more than one Reject Limit Type

* Throw error on identical limits

* Fixing showing default limits

* Added Organization view settings

* Show Remove btn only if quota exists

* Fixing Remove modal pop up

* Quota Reporting if quota is not set

* Fixing Removing Quota for org

* Fixing super user and org admin permissions

* fix to checking default quota

* Added super user perms check on put, delete endpoints

* Fixing formatting

* Fixing multiple rejects checks + returing empty list instead of None

* Using super user API calls + hiding policy div till quota is set

* Added require_scpe and show_if decorators for super user permissions

* fixing configured quota check after return type changed from None to list
2022-04-14 10:33:20 -04:00
Kenny Lee Sin Cheong
896a3aab3a api: update the quota api so that it's more consistent with the other apis endpoints (PROJQUAY-2936) (#1221)
* api: update the quota api so that it's more consistent with the other apis (PROJQUAY-2936)

- Uodate the quota api to be more consistent with the rest of the
endpoints
- Handles some uncaught exceptions, such as division by zero
- Update some of the quota data models used by the api to take object
  references instead of names to make it easier to use
- Update table model naming conventions
- swagger operationid multiple nicknames
- Added more test cases for api
- Remove unused functions
- Update the UI for better UX, based on the api changes made

* quota: fix ui input form value

* quota: join quota type query

* Remove unused functions
2022-04-07 14:11:55 -04:00
Flavian Missi
514bc6f1ba registry_proxy_model: count repository size when caching images (PROJQUAY-3464) (#1215)
also check if the org is within its quota limits before caching an image
from the upstream registry.
2022-03-31 16:38:13 +02:00
Sunandadadi
7524171ac8 Proxy Cache: Interface and UI for Proxy cache Configuration (PROJQUAY-3029) (#1204) 2022-03-31 10:48:59 +02:00
Flavian Missi
5987534718 endpoints/v2: handle generic proxy related errors (#1213)
also add upstream registry to proxy cache key. if a user changes their
mind about what registry to proxy, we shouldn't try to reuse the token.
2022-03-30 15:30:05 +02:00
Brandon Caton
ef91c57c23 format: Updating black to resolve click dependency issue (PROJQUAY-3487) (#1209)
Currently the CI breaks due to a dependency of black, `click`, breaking with it's latest release with `ImportError: cannot import name '_unicodefun' from 'click'`. Since black does not pin it's version of click it pulls in the latest version containing the breaking change and fails the CI check. This updates black with the patch. [See the original issue here.](https://github.com/psf/black/issues/2964) The rest of the changes are format updates introduced with the latest version of black.
2022-03-29 15:34:57 -04:00
Flavian Missi
f248d885aa proxy: don't store entire blob in memory when caching (#1200)
also uses blob uploader to upload the blob to storage.

PROJQUAY-3459
2022-03-28 18:27:07 +02:00
Flavian Missi
fe4d66b030 feat: pull-thru proxy cache (#1053)
introduces the possibility to pull images from external registries
through Quay, storing them locally for faster subsequent pulls.

Closes PROJQUAY-3030 and PROJQUAY-3033
2022-03-23 17:20:00 +01:00
Kenny Lee Sin Cheong
02dfc63f42 api: fully deprecate image api endpoints (PROJQUAY-3418) (#1164) 2022-03-22 11:12:39 -04:00
Brandon Caton
dad7dfaf44 superuser: Fix display of build logs (PROJQUAY-3404) (#1185)
Build logs in the superuser admin panel do not show up. Errors out due to some syntax errors on the backend which this PR resolves.
2022-03-16 08:59:24 -04:00
Syed Mushtaq Ahmed
2219d5aed2 auth: Add state to the Oauth code response (PROJQUAY-3139) (#1124)
Add state param to the oauth redirect

Fixes an issue where we forward the state we recieve in
the original OAuth request
2022-03-15 11:10:06 -04:00
Sunandadadi
c82d78ae85 Fix: Adding default vault to quota parameter (PROJQUAY-0000) (#1171)
* Tracking aws ip ranges.json

* Reverting change

* Adding defaut vault to parameter

* Moving quay-show inside div

* Revert changes on this file

* Reverting changes here

* adding temporary change

* reverting change
2022-03-09 02:52:43 +05:30
kwestpharedhat
15fa20a115 QuotaManagement: Reporting (PROJQUAY-2936) (#1048)
* initial commit

* fixing some bugs

* create quota management

Fix json request json type

Creation of quota is working

All quota crud operations

crud for quota limits

repository size reporting

adding registry model

error levels

namespacequota

remove holdover from user file

finalizing refactor to namespace over organization

finalization of functionality

fixing formatting to match with black style

missed some files in formatting

fixing access to attribute

add single test to verify its working

fix some bugs and add defensive catching

bug fixes and code resiliency

Bug fixes and making quota limits detect properly where necessary

remove transitive delete and other bug fixes

fix formatting and trasnitive deletion issues

fix repositorysize does not exist error

fix not nul constraint and add security tests

fix security tests and bug

more security test fixes

reorder security tests

put docker file back and adjust security testing

security tests reduced

Missed changes for status 200

missed additional 201 responses getting 200

security bypass for now

Another tweak to security testing

forgot 1 endpoint

bug fix for parsing dictionary

remove unnecessary check at blob head

add initdb for quota

Incorrect syntax repair

mysql only supports decimal

adding quota specific notifications

optimization

add permission checks

adjust security and add configuration parameter

fix security test for new security levels

Fix logic errors and improve caching

fix logic issue and error reporting

adjust things according to PR comments

fix refactor left overs

miscapitilazation

missed refactor location

refactor code to remove quota limit groupings

fix refactor errors

remove transitive deletion

fix transitive deletes

Transitive deletion work

Transitive deletion work

refactor registry model and remove it

place api behind feature flag

patch feature enabledment for tests

patch feature enabledment for tests

testing to see if the config is the problem

remove patch

fix new org bug

fixing notifications

mismatched parameters

fix org not exists

fixed paramter mismatch

fix nonetype access

fix nonetype access

new tables created user deletion issues

new tables created user deletion issues

parameter mismatch

fix transitive delete

fix model access error

record does not exist missing catch

fix quota deletion to always delete limits

quotalimits deletion on quota deletion

mistake

fix quota limits deletion

patch tests and disable feature

typo

switch to toggle feature

add feature patch to top of file

change testconfigpy

* change permissions

* adjust permissions

* change config access

* fix formatting

* gether feature information differently

* duplicate function name

* fix config name

* type conversion

* config adjustments

* incorrect keyword

* Update security api tests

* duplicate naming

* fix config schema

* revert files and fix error

* QuotaManagement: UI (PROJQUAY-2936) (#1)

* [WIP]: Quota Reporting on Quay UI

* Integrating quota reporting UI with backend

* Humanizing bytes on UI

* Quota Reporting UI on repo table view

* Taking pull and updating code

* Adding quota management view

* Added support for CRUD operations for org quota

* create quota management

Fix json request json type

Creation of quota is working

All quota crud operations

crud for quota limits

repository size reporting

adding registry model

error levels

namespacequota

remove holdover from user file

finalizing refactor to namespace over organization

finalization of functionality

fixing formatting to match with black style

missed some files in formatting

fixing access to attribute

add single test to verify its working

fix some bugs and add defensive catching

bug fixes and code resiliency

Bug fixes and making quota limits detect properly where necessary

remove transitive delete and other bug fixes

fix formatting and trasnitive deletion issues

fix repositorysize does not exist error

fix not nul constraint and add security tests

fix security tests and bug

more security test fixes

reorder security tests

put docker file back and adjust security testing

security tests reduced

Missed changes for status 200

missed additional 201 responses getting 200

security bypass for now

Another tweak to security testing

forgot 1 endpoint

bug fix for parsing dictionary

remove unnecessary check at blob head

add initdb for quota

Incorrect syntax repair

mysql only supports decimal

adding quota specific notifications

optimization

add permission checks

adjust security and add configuration parameter

fix security test for new security levels

Fix logic errors and improve caching

fix logic issue and error reporting

adjust things according to PR comments

fix refactor left overs

miscapitilazation

missed refactor location

refactor code to remove quota limit groupings

fix refactor errors

remove transitive deletion

fix transitive deletes

Transitive deletion work

Transitive deletion work

refactor registry model and remove it

place api behind feature flag

patch feature enabledment for tests

patch feature enabledment for tests

testing to see if the config is the problem

remove patch

fix new org bug

fixing notifications

mismatched parameters

fix org not exists

fixed paramter mismatch

fix nonetype access

fix nonetype access

new tables created user deletion issues

new tables created user deletion issues

parameter mismatch

fix transitive delete

fix model access error

record does not exist missing catch

fix quota deletion to always delete limits

quotalimits deletion on quota deletion

mistake

fix quota limits deletion

patch tests and disable feature

typo

switch to toggle feature

add feature patch to top of file

change testconfigpy

* Removing quota and state conf from repo-list and user-view

* Removing quota and state conf form app list page

* Removing quota conf from repo-list.html

* minor fixes

* Added Quota Repoting and configuring quota from UI

* Making quota configuration component reusable + added support to read bytes via KB, MB, etc + Added reporting for total org consumption + Added org consumption for super user panel + Added quota configurable support on super user panel

* Adding older quota management component

* Removing not reusable quota management component

* Adding % consumption for repo quotas

* Adding % consumption for organization level quota

* Adding check to verify request.args

* Removing todo

* Adding default 0 to quota

* Formatting with black

* Fixing params for tests

* Formatting test file

Co-authored-by: Keith Westphal <kwestpha@redhat.com>

* remove migration

* add migration back

* repair formatting

* QuotaManagement: Moving the logic for bytes conversion to human friendly units to the frontend  (PROJQUAY-2936) (#3)

* Moving the logic for bytes conversion to human friendly units to the frontend

* Reading updates from quota_limit_id

* Formatting using black

* remote unused function

* Adding quota configuring on super user panel (#4)

* Converting quota bytes to human friendly format (#5)

* PR refactors

* invalid reference

* bad return value

* fix bad reference

* bad reference

* fix tests

* Quota Config: UI improvements (#6)

* Quota UI Improvements

* Rendering table for quota limit config

* Removing proxy cache files

* Disabling quota config for org view

* Removing redundant get

* Fixing PR requests

* repair formatting

Co-authored-by: Sunandadadi <Sunandadadi@users.noreply.github.com>
2022-03-08 12:02:03 -06:00
Brandon Caton
0d3ecb132e imagemirror: Add unsigned registries mirror option (PROJQUAY-3106) (#1085)
Currently when attempting to mirror a registry containing unsigned images the mirror will fail due to not finding the source signature. This is caused by the updated version of Skopeo blocking unsigned images by default. This allows users to specify the ability to pull unsigned images per-repository. The Skopeo version is also now pinned.
2022-02-04 13:10:19 -05:00
Hank Donnay
861c247faf trigger_analyzer: fix confusing print (PROJQUAY-1995) (#1073)
Signed-off-by: Hank Donnay <hdonnay@redhat.com>
2022-01-27 16:43:32 -06:00
Syed Mushtaq Ahmed
0033f9b851 auth: Fix oauth code flow (PROJQUAY-781) (#1044)
OAuth Authorization Code Flow is broken in quay. Code
Flow is more secure than implicit flow and is used
by server side applications to get the access token
2021-12-23 14:17:36 -05:00
Kenny Lee Sin Cheong
2ffc12b3eb chore: cleanup remaining artifacts remaining related to aci signing (PROJQUAY-2792) (#968)
Removes deprecated artifacts from code and files for ACI.
2021-11-19 13:49:28 -05:00
mosen
fca67e7729 feat: mypy type annotations (PROJQUAY-740) (#455)
* Add dev dependencies mypy and typing

* Add makefile target `types-test`, not yet included in `test` target.

* Generate stubs for imported modules to avoid mypy complaining about missing types.

* Remove generated stubs as there are way too many and they cause tons of mess in the repo. Switched to ignoring untyped modules for now, to concentrate on Quay-only type checking.

* mypy config changed to ignore missing imports

* ignore property decorator as it is not supported by mypy

* mypy annotations for many configuration variables

* re-generate mypy_stubs directory as its necessary in some classes for base classes to prevent mypy errors

* util/registry/queuefile referred to non existent definition of Empty class in multiprocessing.queues

* ignore type checking for things like monkey patching and exported/re-imported objects that 
mypy does not allow.

* Adjust mypy config to warn us about unreachable return paths and useless expressions.

* Add the __annotations__ property to INTERNAL_ONLY_PROPERTIES so that it is not part of the config schema testing

* Remove redundant dependencies `typing` and `typing-extensions` which are NOOP after Python 3.5

* Remove mypy-extensions which only provides a TypedDict implementation but has not been updated since 2019.

* updated mypy to 0.910 which requires all types packages to be installed manually.

* exclude local-dev from type checking until core team can suggest an outcome for __init__.py duplicate packages

* re-add typing dependency which will be needed until Python 3.9

* ignore .mypy_cache

* add mypy stub for features module to replace inline definitions

* import annotations eager evaluation in billing.py as it was required to reference a class declared later in the module.

* remove the type definition of V1ProtocolSteps/V2ProtocolSteps to make tox happy
2021-10-25 09:56:26 +02:00
Jonathan King
a29f3e0eea defaults: Update defaults in config and schema (PROJQUAY-2425) (#923)
- Update defaults in config.pu
2021-10-12 14:47:40 -07:00
Kenny Lee Sin Cheong
32322bc1f6 registry: update blob mount behaviour when from parameter is missing (PROJQUAY-2570) (#899)
Update the behaviour when attempting to mount existing blob whenever
the `from` parameter is missing. Currently, Quay throws an Bad Request
error whenever the `from parameter is missing. However, according to
the specs (Docker Registry and OCI dist-spec), we should be falling
back to the default upload behaviour.

References:
  - https://docs.docker.com/registry/spec/api/#pushing-an-image
  -
  https://github.com/opencontainers/distribution-spec/blob/main/spec.md#mounting-a-blob-from-another-repository
  - https://github.com/opencontainers/distribution-spec/pull/275/files
2021-09-23 10:03:18 -04:00
Kenny Lee Sin Cheong
b0adc9667c registry: add support for extended repository names (PROJQUAY-1535) (#814)
Allows forward slashes to be used in repository names according to
https://docs.docker.com/docker-hub/repos/.

NOTE: This change simply allows the use of "/" in repository
names needed for certain Openshift use cases. This does not implement
any new permission model for nested paths. i.e A repository with a
nested path is treated as a single repository under a _single_
namespace.
2021-07-28 12:27:33 -04:00
Kenny Lee Sin Cheong
64bc11fe46 oci: allow oci artifact registration (PROJQUAY-1032) (#803)
Allow registration of custom oci artifact types in Quay's configuration.
2021-07-26 09:52:33 -04:00
thomasmckay
6470248be1 api: /v1/user/initialize to create first user (PROJQUAY-1926) (#771)
Add an unauthenticated API endpoint to create the initial user in the database. Usage is primarily intended for deployment automation.
2021-07-23 12:03:47 -04:00
Kenny Lee Sin Cheong
a839a78eb5 chore: allows Quay to run for account recoveries (PROJQUAY-970) (#793)
Adds ACCOUNT_RECOVERY_MODE to allow Quay to run with some core
features disabled. When this is set, the instance should only be used
in order by existing users who hasn't linked their account to an
external login service, after database authentication has been
disabled.
2021-07-07 12:45:24 -04:00
thomasmckay
6ba0e88128 organization: config to allow organization creation on push (PROJQUAY-928) (#799)
Config CREATE_ORGANIZATION_ON_PUSH that will create an organization if it doesn't yet exist when image is pushed. Defaults to False.
2021-06-07 08:27:55 -04:00
thomasmckay
a1b7e4b519 repository: config to allow public repo create (PROJQUAY-1929) (#772)
New config CREATE_PRIVATE_REPO_ON_PUSH to control global default for new repositories created via push. Defaults to True, meaning the repo will be created with "private" visibility. If set to False, new repos created by push will be created with "public" visibility.
2021-05-26 09:44:16 -04:00
Syed Mushtaq Ahmed
df4ad94527 modelcache: Make ModelCache TTL configurable (PROJQUAY-1878) (#765)
Adds a configuration option to modify the cache expiry timeout
for ModelCache objects

Co-authored-by: Syed <syed@apache.org>
2021-04-28 10:46:32 -04:00
Kenny Lee Sin Cheong
311241af09 secscan: allow basic auth on the secscan api endpoint when anonymous api resource is set. (#608)
Allow the auth flow to use basic auth on the secscan api resource when
FEATURE_ANONYMOUS_ACCESS (e.g CSO uses pull secrets to fetch secscan
resources)
2021-03-22 11:38:41 -04:00
Kenny Lee Sin Cheong
7d9a49da10 Replace jwkest with authlib (#685)
* Replace jwkest with authlib and PyCrypto with cryptography

Remove pycryptodome dependencies.
Remove post-fork random seed init - python-cryptography's pseudo RNG should be fork safe:
- https://cryptography.io/en/latest/security.html?highlight=fork
- https://cryptography.io/en/latest/hazmat/backends/openssl.html?highlight=fork#os-random-engine

* deps: Pin cryptography to 3.3.1

Latest available version available on RHEL.
As of 3.4, cryptography builds on rust, which is not readily available
on RHEL.
2021-03-15 15:25:23 -04:00