Having only one worker index recent manifest doesn't keep up with the
rate new manifests being pushed, given the time it takes for an index
request to complete. Adding the option to bypass the global lock
allows for more workers, but also increase the chance of duplicate work.
Index recent manifests in a separate background process, allowing the
main process to correctly select random slabs from the entire table
set and marking them completed in the allocator (rbtree). This avoids
the worker having to start iterating from the beginning of the table
whenever it is restarted.
Add a global lock on security worker iterations, based on the value of
the current pagination token. This is to avoid multilple worker
processes possibly doing work on the same set of manifests.
Adds ACCOUNT_RECOVERY_MODE to allow Quay to run with some core
features disabled. When this is set, the instance should only be used
in order by existing users who hasn't linked their account to an
external login service, after database authentication has been
disabled.
* local-dev: implement local development environment
this commit copies the files in /init into /local-dev, edits those files
to support hot-reload features for local development, and introduces
docker-compose/makefile targets in order to support local dev.
Signed-off-by: ldelossa <ldelossa@redhat.com>
* local-dev: hop quay workers to gunicorn
this commit adds a uwsgi worker application delegate and a factory
function allowing each worker to be ran by gunicorn.
each worker now supports hot-reload and will reload itself when it's
code is updated.
this changes only affects the local dev env.
Signed-off-by: ldelossa <ldelossa@redhat.com>
* local-dev: add docs
Signed-off-by: ldelossa <ldelossa@redhat.com>
This change replaces the metricqueue library with a native Prometheus
client implementation with the intention to aggregated results with the
Prometheus PushGateway.
This change also adds instrumentation for greenlet context switches.
