* handle cache value size limit in memcached
* only print cache value for redis in debug logs on redis errors
---------
Signed-off-by: dmesser <dmesser@redhat.com>
* api: add caching for get_repository method (PROJQUAY-6472)
* fixing formatting
* using pickle to serialize and deserialize objects
* adding model_cache parameter to lookup_repository func for interface and registry_proxy_model
* adding test
* formatting fix
* Add dev dependencies mypy and typing
* Add makefile target `types-test`, not yet included in `test` target.
* Generate stubs for imported modules to avoid mypy complaining about missing types.
* Remove generated stubs as there are way too many and they cause tons of mess in the repo. Switched to ignoring untyped modules for now, to concentrate on Quay-only type checking.
* mypy config changed to ignore missing imports
* ignore property decorator as it is not supported by mypy
* mypy annotations for many configuration variables
* re-generate mypy_stubs directory as its necessary in some classes for base classes to prevent mypy errors
* util/registry/queuefile referred to non existent definition of Empty class in multiprocessing.queues
* ignore type checking for things like monkey patching and exported/re-imported objects that
mypy does not allow.
* Adjust mypy config to warn us about unreachable return paths and useless expressions.
* Add the __annotations__ property to INTERNAL_ONLY_PROPERTIES so that it is not part of the config schema testing
* Remove redundant dependencies `typing` and `typing-extensions` which are NOOP after Python 3.5
* Remove mypy-extensions which only provides a TypedDict implementation but has not been updated since 2019.
* updated mypy to 0.910 which requires all types packages to be installed manually.
* exclude local-dev from type checking until core team can suggest an outcome for __init__.py duplicate packages
* re-add typing dependency which will be needed until Python 3.9
* ignore .mypy_cache
* add mypy stub for features module to replace inline definitions
* import annotations eager evaluation in billing.py as it was required to reference a class declared later in the module.
* remove the type definition of V1ProtocolSteps/V2ProtocolSteps to make tox happy
Remove GlobalLock from Redis model cache implementation in
favor of 'nx=True' when setting the key.
Signed-off-by: Alec Merdler <alecmerdler@gmail.com>
Adds implementation of DataModelCache interface backed by Redis.
All containers in a Quay cluster deployment will share a single
model cache, rather than each container using its own cache.
Signed-off-by: Alec Merdler <alecmerdler@gmail.com>
The PooledCache implementation incorrectly returns `None` instead of
the default if an exception occurs, but `None` is a valid cached value,
so we instead catch the exceptions ourselves now and skip the caching
code if we encounter an exception
pymemcache is apparently not thread safe, so our reuse of the client
was causing the occasional hang on read. This change has us open a new
connection per request, and then close it once complete. We also make
sure to close the DB connection before making the memcache connection,
in case the memcache connection takes a bit of time
We should investigate switching to the PooledClient library for reusing
connections (safely) once we verify this change works
