* marketplace: splittable sku for MW02702 (PROJQUAY-8151)
* Alembic migration to drop unique constraint on the orgrhsubscriptions
table
* Can split sub quantities of MW02702 across multiple orgs
* Can specify quantity for the MW02702 SKU across orgs on react UI
* Update angular UI to allow user to specify quantities for MW02702
* feat: Add support for auto pruning at repository level (PROJQUAY-6354)
* Add repositoryautoprunepolicy table to alembic migration script
* Add repository auto-prune policy endpoints
* Add UI for repository auto-pruning policies
* case: apply repo auto-prune policy when no namespace policy given
* case: both namespace and repo pruning policy are given
* Add tests for repository autoprune endpoint
* Add cypress test for repository auto-prune
* Add repo auto-prune policy clean-up for repository deletion
* Add repository auto pruning tables to quay db snapshot for cypress tests
* Address review comments
* Add more tests + fix CI + reformat files
* Address review comments #2
---------
Signed-off-by: harishsurf <hgovinda@redhat.com>
* Adds handling for when a subscription returned from the subscription watch api has a quantity greater than 1. Number of private repos should be correctly calculated using the quantity.
* Updates ui so that subscriptions can only be added to an org as a group, i.e. a subscription with quantity = 2 cannot be split across organizations.
* billing: marketplace UI
adds UI in billing section for managing user and org-bound skus
add more unit tests for org binding
changed endpoint for bulk attaching skus to orgs
* update reconciliationworker to use webCustomerId instead of
ebsAccountNumber
* fix reconciler where it was incorrectly using the ebsAccountNumber to
create subscriptions
* add job to reconciler so that it reconciles different ids between the
database and the user api
* separate skus to be used by billing and skus to be used by reconciler
* add migration for orgrhskus table
* add endpoints for managing and listing skus bound to an org
* create checks in billing flow to look for org-bound skus
* refactor RH marketplace api objects to be more usable in tests
* update cypress test db data and exclude it from pre-commit hook formatting
Removes read support for Clair V2, along with the need to package
jwtproxy with Quay.
TODO: Drop deprecate image api + image table, remove image data model.
* api: update the quota api so that it's more consistent with the other apis (PROJQUAY-2936)
- Uodate the quota api to be more consistent with the rest of the
endpoints
- Handles some uncaught exceptions, such as division by zero
- Update some of the quota data models used by the api to take object
references instead of names to make it easier to use
- Update table model naming conventions
- swagger operationid multiple nicknames
- Added more test cases for api
- Remove unused functions
- Update the UI for better UX, based on the api changes made
* quota: fix ui input form value
* quota: join quota type query
* Remove unused functions
* Change verbs to use a DerivedStorageForManifest table instead of DerivedStorageForImage
This allows us to deprecate the DerivedStorageForImage table.
Fixes https://issues.redhat.com/browse/PROJQUAY-519
* Change uploaded blob tracking to use its own table and deprecate
RepositoryTag
* Start recording the compressed layers size and config media type on the
manifest row in the database
NOTE: This change includes a database migration which will *lock* the
manifest table
* Change tag API to return the layers size from the manifest
* Remove unused code
* Add new config_media_type field to OCI types
* Fix secscan V2 test for us no longer writing temp images
* Remove unused uploading field
* Switch registry model to use synthetic legacy images
Legacy images are now (with exception of the V2 security model) read from the *manifest* and sythensized in memory. The legacy image IDs are generated realtime based on the hashids library. This change also further deprecates a bunch of our Image APIs, reducing them to only returning the image IDs, and emptying out the remaining metadata (to avoid the requirement of us loading the information for the manifest from storage).
This has been tested with our full clients test suite with success.
* Add a backfill worker for manifest layers compressed sizes
* Change image tracks into manifest tracks now that we no longer have
manifest-less tags
* Add back in the missing method
* Add missing joins to reduce extra queries
* Remove unnecessary join when looking up legacy images
* Remove extra hidden filter on tag queries
* Further DB improvements
* Delete all Verbs, as they were deprecated
* Add back missing parameter in manifest data type
* Fix join to return None for the robot if not defined on mirror config
* switch to using secscan_v4_model for all indexing and remove most of secscan_v2_model code
* Add a missing join
* Remove files accidentally re-added due to rebase
* Add back hashids lib
* Rebase fixes
* Fix broken test
* Remove unused GPG signer now that ACI conversion is removed
* Remove duplicated repomirrorworker
* Remove unused notification code for secscan. We'll re-add it once Clair
V4 security notifications are ready to go
* Fix formatting
* Stop writing Image rows when creating manifests
* Stop writing empty layer blobs for manifests
As these blobs are shared, we don't need to write ManifestBlob rows
for them
* Remove further unused code
* Add doc comment to _build_blob_map
* Add unit test for synthetic V1 IDs
* Remove unused import
* Add an invalid value test to synthetic ID decode tests
* Add manifest backfill worker back in
Seems to have been removed at some point
* Add a test for cached active tags
* Rename test_shared to not conflict with another same-named test file
Pytest doesn't like having two test modules with the same name
* Have manifestbackfillworker also copy over the config_media_type if present
Co-authored-by: alecmerdler <alecmerdler@gmail.com>
* Convert all Python2 to Python3 syntax.
* Removes oauth2lib dependency
* Replace mockredis with fakeredis
* byte/str conversions
* Removes nonexisting __nonzero__ in Python3
* Python3 Dockerfile and related
* [PROJQUAY-98] Replace resumablehashlib with rehash
* PROJQUAY-123 - replace gpgme with python3-gpg
* [PROJQUAY-135] Fix unhashable class error
* Update external dependencies for Python 3
- Move github.com/app-registry/appr to github.com/quay/appr
- github.com/coderanger/supervisor-stdout
- github.com/DevTable/container-cloud-config
- Update to latest mockldap with changes applied from coreos/mockldap
- Update dependencies in requirements.txt and requirements-dev.txt
* Default FLOAT_REPR function to str in json encoder and removes keyword assignment
True, False, and str were not keywords in Python2...
* [PROJQUAY-165] Replace package `bencode` with `bencode.py`
- Bencode is not compatible with Python 3.x and is no longer
maintained. Bencode.py appears to be a drop-in replacement/fork
that is compatible with Python 3.
* Make sure monkey.patch is called before anything else (
* Removes anunidecode dependency and replaces it with text_unidecode
* Base64 encode/decode pickle dumps/loads when storing value in DB
Base64 encodes/decodes the serialized values when storing them in the
DB. Also make sure to return a Python3 string instead of a Bytes when
coercing for db, otherwise, Postgres' TEXT field will convert it into
a hex representation when storing the value.
* Implement __hash__ on Digest class
In Python 3, if a class defines __eq__() but not __hash__(), its
instances will not be usable as items in hashable collections (e.g sets).
* Remove basestring check
* Fix expected message in credentials tests
* Fix usage of Cryptography.Fernet for Python3 (#219)
- Specifically, this addresses the issue where Byte<->String
conversions weren't being applied correctly.
* Fix utils
- tar+stream layer format utils
- filelike util
* Fix storage tests
* Fix endpoint tests
* Fix workers tests
* Fix docker's empty layer bytes
* Fix registry tests
* Appr
* Enable CI for Python 3.6
* Skip buildman tests
Skip buildman tests while it's being rewritten to allow ci to pass.
* Install swig for CI
* Update expected exception type in redis validation test
* Fix gpg signing calls
Fix gpg calls for updated gpg wrapper, and add signing tests.
* Convert / to // for Python3 integer division
* WIP: Update buildman to use asyncio instead of trollius.
This dependency is considered deprecated/abandoned and was only
used as an implementation/backport of asyncio on Python 2.x
This is a work in progress, and is included in the PR just to get the
rest of the tests passing. The builder is actually being rewritten.
* Target Python 3.8
* Removes unused files
- Removes unused files that were added accidentally while rebasing
- Small fixes/cleanup
- TODO tasks comments
* Add TODO to verify rehash backward compat with resumablehashlib
* Revert "[PROJQUAY-135] Fix unhashable class error" and implements __hash__ instead.
This reverts commit 735e38e3c1d072bf50ea864bc7e119a55d3a8976.
Instead, defines __hash__ for encryped fields class, using the parent
field's implementation.
* Remove some unused files ad imports
Co-authored-by: Kenny Lee Sin Cheong <kenny.lee@redhat.com>
Co-authored-by: Tom McKay <thomasmckay@redhat.com>
* Optimize repository lookup queries to meet the expected maximums
We were accidentally looking up more data that strictly allowed
Adds some additional assertions and testing as well
Fixes https://issues.redhat.com/browse/PROJQUAY-439
* Change loading of repositories in the repo view to be paginated
We drop the "card" view and switch to a table-only view, but still
load the full set of repositories
A followup change will begin to change the UI to only load additional
repos when requested
* Implement OCI manifest and index support
* Remove unnecessary data model check in registry protocol fixtures
* Implement OCI testing
* Add migration for adding OCI content types
* Remove unused supports_schema2
* Add OCI_NAMESPACE_WHITELIST and reformat with black
* Catch errors in legacy image population and raise appropriately
* Add support for registration of additional artifact types
This change adds the infrastructure to support artifacts in OCI
manifests, but does not yet register any types
* Add a feature flag for enabling experimental Helm support via OCI
See: https://helm.sh/docs/topics/registries/
