1
0
mirror of https://github.com/quay/quay.git synced 2026-01-27 18:42:52 +03:00
Commit Graph

35 Commits

Author SHA1 Message Date
Joseph Schorr
59bc23313c Add additional detection for transitive deletes and fix those found (#281)
We were not testing for transitive deletes when performing *User*
deletion, which led us to have a few
2020-03-25 15:58:12 -04:00
Joseph Schorr
dce80ab26d Fix support for empty manifests in V2_2 (#278)
Fixes https://issues.redhat.com/browse/PROJQUAY-510
2020-03-24 11:26:08 -04:00
Joseph Schorr
49862a9eda Move shared image modules into a shared package in prep for adding OCI (#271) 2020-03-20 12:33:43 -04:00
Joseph Schorr
8701577cff Have the RepositoryActionCount worker cleanup old rows in RAC (#274)
We don't make use of any action counts older than a year, so this
change will have the worker remove old rows, one month (roughly) at
a time
2020-03-18 17:00:14 -04:00
Alec Merdler
7b827d953c add new 'ManifestSecurityStatus' database model for Clair v4 (#216) 2020-03-17 15:31:15 -07:00
Joseph Schorr
0844256a71 Major post-OCI cleanup (#225)
* Remove V3_UPGRADE_MODE

* Remove tag backfill worker and all callers to tag backfill and upgrades

* Change initdb to create all test data via the manifest builder, rather than manually via legacy images

* Convert various code paths to use the registry_model where we previously did not do so

* Convert the GC test suite to build via the manifest builder

* Delete an old, unused tool

* Delete the Pre OCI registry model

* Add additional error handling to the manifest creation code path

* Add additional error handling to the OCI tag creation code path

* Change how we respond to invalid manifest content types to better handle unknowns

* Change legacy secscan test suite to use the registry model

* Change the repo build badge to use the registry model (also fixes a bug)

* Delete now-unused data model code

* Remove old model adjustment code from OCI model

* Mark older data models as deprecated which will prevent new rows from being inserted

* Remove references to old registry test suite from various testing files

* Remove tag backfill worker (again; got re-added during rebase)

* Move all deprecated model checks into a central function

* Make data_migration more Pythonic

* Small requested fixes to Tag module styling

* Have tag backfill worker fail to migrate if there are TagManifest's

Since this backfill should only be called in future releases for empty models, this should catch someone attempting to upgrade from a too-old version

* Remove labelbackfillworker as it is no longer needed

* Remove unused invalid import

* Reimplement the tag test for the remaining method used
2020-03-17 13:27:29 -04:00
Joseph Schorr
7b8d96a250 Ensure that temporary tags cannot be written for repositories marked (#258)
for deletion

The purge code assumes that once it has deleted tags, no new ones will
be added, so we need to enforce that state
2020-03-12 15:41:54 -04:00
Joseph Schorr
04cf7cc2c3 Catch infinite loops in GC purge code and raise an exception instead (#257)
If we have a bug that causes the GC's assumptions to be wrong (such
as an Image added after a purge began), then the purge loop can enter a
state where it will never halt. This change detects that condition and
raises an exception so we can properly investigate the bug.

Fixes https://issues.redhat.com/browse/PROJQUAY-452
2020-03-10 17:30:12 -04:00
Joseph Schorr
5b50258897 Change the lookup of active orgs to skip the unnecessary heavy filter (#249) 2020-03-04 16:50:17 -05:00
Joseph Schorr
424c1a19d7 Have the BlobUpload cleanup worker run on a single instance only (#239)
Instead of running simultaneously across multiple nodes, we change the
worker to run only from a single instance. This is better for the DB and
the previous behavior was not necessary given the size of the BlobUpload
table.

Fixes https://issues.redhat.com/browse/PROJQUAY-365
2020-02-27 13:16:40 -05:00
Joseph Schorr
1b023b8c2f Fix performance of the global prom stats worker queries (#238)
- Change the counts to use the MySQL information schema for estimates
- Change the lookup of active users to skip the unnecessary heavy filter
2020-02-27 13:16:35 -05:00
Joseph Schorr
ca2c2c67ea Add pre-deletion of RepositoryAuthorizedEmail rows 2020-02-14 16:26:50 -05:00
Joseph Schorr
11e4d96130 Handle integrity errors being raised in v2 auth
Instead of raising an unhandled exception, we now check for the repository during the create process. If it was created since we last checked, we simply use the reference. Otherwise, we return an error to the user.

Fixes https://issues.redhat.com/projects/PROJQUAY/issues/PROJQUAY-289
2020-02-13 13:48:05 -05:00
Alec Merdler
c688f3a05e Merge pull request #180 from alecmerdler/fix-docstrings
Fix All the Docstrings
2020-02-06 21:06:43 -08:00
alecmerdler
3dd5f045ff fix all the docstrings 2020-02-05 19:55:07 -08:00
Joseph Schorr
7753b5be54 Pre-flight the cleanup of all big tables referencing a purged repo
This should prevent the final deletion transaction from timing out
2020-02-04 13:23:11 -05:00
Joseph Schorr
2d15282129 Merge pull request #212 from josephschorr/extra-storage-verification
Add extra verification on storage GC to ensure we don't end up with dangling ImageStorage rows
2020-01-31 16:43:52 -05:00
Joseph Schorr
e71ce8c811 Add extra verification on storage GC to ensure we don't end up with dangling ImageStorage rows
Also adds an optimization to avoid GCing specialized blobs
2020-01-31 14:49:26 -05:00
Joseph Schorr
21b149325a Additional fixes for config load failure in manifest creation
- Add additional retries and slightly increase the delay time
- Add a proper exception returned on failure
- Add a test to verify that we raise the proper exception
2020-01-31 11:46:48 -05:00
Joseph Schorr
36729c7672 Add a retry and proper error handling to the retriever
Some storage engines are eventually consistent, so we add a small number of retries, and we return `None` properly if the file could not be loaded
2020-01-30 16:06:48 -05:00
Joseph Schorr
59deab00ec Catch IntegrityError's in a better way and log
Currently, if we see an IntegrityError, we assume its because the manifest already exists. However, there appears to be a corner case where this is not the case. This change allows us to catch that additional exception, log the original issue, and return a better error to the user

Fixes https://issues.redhat.com/browse/PROJQUAY-219
2020-01-30 15:36:05 -05:00
Joseph Schorr
6db1910a97 Change repository deletion to occur in the background
Fixes https://issues.redhat.com/browse/PROJQUAY-124
2020-01-27 14:16:01 -05:00
Kurtis Mullins
1b2d29d1e8 Revert "Revert "Remove the active migration for encrypted tokens now that it is complete"" 2020-01-27 11:22:22 -05:00
Joseph Schorr
e129fbac32 Merge pull request #168 from josephschorr/joseph.schorr/PROJQUAY-178/label-fixes
Fix label handling for empty labels
2020-01-22 11:20:49 -05:00
Kurtis Mullins
90d0dbf3fb Revert "Remove the active migration for encrypted tokens now that it is complete" 2020-01-21 11:53:54 -05:00
Joseph Schorr
883a415dc4 Remove the active migration for encrypted tokens now that it is complete
Fixes https://issues.redhat.com/browse/PROJQUAY-185
2020-01-20 14:51:51 -05:00
Joseph Schorr
06e1d12332 Fix bitbucket trigger creation around access tokens
Fixes https://issues.redhat.com/browse/PROJQUAY-186
2020-01-20 10:42:04 -05:00
Joseph Schorr
d97a1f6d90 Skip empty labels in label creation
This can happen if a Dockerfile contains a LABEL statement without a key value. In this case, we simply ignore the label and don't add it to our table, as it has no defined key.
2020-01-16 16:33:57 -05:00
Joseph Schorr
345bc6a155 Add missing descriptions to create manifest exceptions around labels
This ensures the error is raised to the user
2020-01-16 16:09:05 -05:00
Joseph Schorr
52594e0a17 Small fixes for the encrypted token migration
Fixes the migration ID in one of the phases, and adds an extra assertion and checks in the robot migration

Fixes https://issues.redhat.com/browse/PROJQUAY-168
2020-01-15 13:30:33 -05:00
Joseph Schorr
db2eefcb7e Fix the assertion in lookup of logs by namespace
Adds additional testing and assertions as well

Fixes https://issues.redhat.com/browse/QUAY-2239
2020-01-05 18:15:02 +02:00
Joseph Schorr
6affc2f1f4 Fix out of bounds on GC worker when there are no namespaces defined
Fixes https://issues.jboss.org/projects/PROJQUAY/issues/PROJQUAY-37
2019-12-02 18:00:37 -05:00
Tom McKay
e6ae94db8b 'make black' results 2019-12-02 12:23:08 -05:00
Tom McKay
36d0aa4fbb PROJQUAY-12 - remove mirror/rules API 2019-11-20 09:30:52 -05:00
Jimmy Zelinskie
9c0dd3b722 initial import for Open Source 🎉 2019-11-12 11:09:47 -05:00