1
0
mirror of https://github.com/regclient/regclient.git synced 2025-04-18 22:44:00 +03:00

106 Commits

Author SHA1 Message Date
Brandon Mitchell
95c9152941
Chore: Modernize Go to the 1.22 specs
- Use "any" instead of an empty interface.
- Use range over an integer for for loops.
- Remove shadow variables in loops now that Go no longer reuses the variable.
- Use "slices.Contains", "slices.Delete", "slices.Equal", "slices.Index", "slices.SortFunc".
- Use "cmp.Or", "min", and "max".
- Use "fmt.Appendf" instead of "Sprintf" for generating a byte slice.
- Use "errors.Join" or "fmt.Errorf" with multiple "%w" for multiple errors.

Additionally, use modern regclient features:

- Use "ref.SetTag", "ref.SetDigest", and "ref.AddDigest".
- Call "regclient.ManifestGet" using "WithManifestDesc" instead of setting the digest on the reference.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2025-02-18 14:32:06 -05:00
Brandon Mitchell
7a2a05efb5
Feat: Support incomplete subject descriptor
While the descriptor requires a size and media type, the referrer API does not.
This makes regclient more tolerant of non-conformant implementations.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-12-10 15:23:21 -05:00
Brandon Mitchell
763599514d
Feat: Include source in referrers response
This simplifies handling external sources.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-12-03 17:12:10 -05:00
Brandon Mitchell
a8a9640fec
Feat: Support referrers in an external repository
This allows organizations to maintain a repository of referrers separate from the upstream images.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-11-29 15:42:44 -05:00
Brandon Mitchell
289533b02c
Feat: Move logrus calls into files excluded by wasm
This permits builds of wasm binaries.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-11-12 10:18:47 -05:00
Brandon Mitchell
1eb1ea4b34
Feat: Refactor logging to use log/slog
This updates the regclient Go library.
Existing users of logrus will continue to work using a logrus handler to slog.
Updates to the various commands will be made in a future commit.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-11-10 17:14:57 -05:00
Brandon Mitchell
17434c3c7b
Feat: Add default host config
In regctl, this exposes a flag to set the default credential helper.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-09-06 17:00:46 -04:00
Brandon Mitchell
05adb86db3
Chore: Move throttle from config to reghttp
Breaking: config.Host.Throttle() has been removed. Use scheme.Throttler instead.

Ideally the config should only have static config values.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-27 16:11:42 -04:00
Brandon Mitchell
89be757fd0
Feat: Consolidate warnings
When a registry returns a warning header on every request, clients should try to consolidate these.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-22 16:52:40 -04:00
Brandon Mitchell
10a645bdb3
Feat: Support auth on redirect
A registry server that is just a redirect to a real server that requires auth will now work.
Previously, auth was only sent to the first server, now it is only sent to the server that performed the auth.
The auth package itself was also refactored a bit, removing unneeded interfaces and reducing exported fields.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-19 10:36:46 -04:00
Brandon Mitchell
b752cdf9c7
Configure priority queue algorithm and reorder image copy steps
- Priority queue algorithm prefers a small entry (non-blob API), and 50/50 split of largest and oldest queued entries.
- Image reorder starts the blob copies sooner to avoid blocking on the tag listing for referrers or digest tags.
- pqueue.AcquireMulti releases queues in reverse order to minimize risk of an acquire blocked by a soon to release queue.
- Include type and size in the request for priority queue.
- Use expected request size to validate response.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-13 15:31:29 -04:00
Brandon Mitchell
54bcd7bd24
Refactor: Remove reghttp.Resp interface
Replace it with a pointer to a struct with private values.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-13 15:31:29 -04:00
Brandon Mitchell
0ff350a85d
Refactor: Remove multiple API support
Breaking: This removes an undocumented API for deleting images from Hub.
Users that depend on that functionality should consider a Docker Hub alternative like hub-tool.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-13 15:31:29 -04:00
Brandon Mitchell
ca18029b93
Feat: Add a priority queue for network requests
Breaking: Update scheme to use pqueue instead of throttle.

This is the first step to redesign the network requests for more efficiency.

Library users are unlikely to encounter issues with the scheme change since
both throttle and pqueue are internal packages, so it shouldn't be possible
to call their methods directly or specify their types in variables.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-13 15:31:29 -04:00
Brandon Mitchell
1b7d3124ab
Fix: Lookup referrers when registry does not give digest with head
Some registries do not include the docker digest headers.
This should support referrers listing by tag on those registries.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-08 11:30:20 -04:00
Brandon Mitchell
7d17cff26c
Fix: Validate the digest of the ref when provided
This prefers the user provided digest for the manifest over the registry header when it is available.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-08-02 11:44:08 -04:00
Brandon Mitchell
6c91f44ac4
Support pushing tagged manifests with different digest algorithms
These parameters are currently experimental and not standardized by OCI yet.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-07-12 10:54:51 -04:00
Brandon Mitchell
e805426203
Support changing digest algorithm
This adds validation and the ability to modify the digest.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-07-07 11:44:40 -04:00
Brandon Mitchell
16a9d83434
Remove rwfs internal package
This was used for test implementations that were better handled with t.TempDir or olareg backed by memory.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-06-26 17:21:24 -04:00
Brandon Mitchell
5e05f50a3c
Refactor pulling a platform specific manifest
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-06-23 10:23:53 -04:00
Brandon Mitchell
58c1cfb5d5
Use the full descriptor on a blob get response
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-04-24 11:00:14 -04:00
Brandon Mitchell
cf6c678cf7
Chore: Cleanup unused params
This is now flagged by gopls, but not yet in staticcheck.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-03-13 16:02:30 -04:00
Brandon Mitchell
eea06e2a5c
Refactoring the type package
I feel like I need to explain, this is all to move the descriptor package.
The platform package could not use the predefined errors in types because of a circular dependency from descriptor.
The most appropriate way to reorg this is to move descriptor out of the type package since it was more complex than a self contained type.
When doing that, type aliases were needed to avoid breaking changes to existing users.
Those aliases themselves caused circular dependency loops because of the media types and errors, so those were also pulled out to separate packages.
All of the old values were aliased and deprecated, and to fix the linter, those deprecations were fixed by updating the imports... everywhere.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-03-04 15:43:18 -05:00
Brandon Mitchell
033e940357
Fix wrapped errors
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-02-27 16:26:17 -05:00
Brandon Mitchell
24092924db
Remote wraperr package
This can be done with a "%.0w" format on Errorf.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-02-27 15:51:35 -05:00
Brandon Mitchell
cc0ae63f8f
Replace t.Error and return with t.Fatal
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-02-27 15:33:52 -05:00
Brandon Mitchell
caa949c329
Replace uuid with random id generator
This removes an unneeded dependency.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-02-26 09:12:15 -05:00
Brandon Mitchell
e6bc1fb637
Add regctl blob delete and ocidir blob delete API
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-02-22 18:54:19 -05:00
Brandon Mitchell
3a101001a9
Cancel failed blob uploads to a registry
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-02-21 14:52:31 -05:00
Brandon Mitchell
16211860a8
Preserve the descriptor on chunked blob put
This also adds special handling for the empty blob, and validates the descriptor.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-01-03 15:51:36 -05:00
Brandon Mitchell
38b4b10ac9
Handle unretryable failures in http requests
The blob put method depends on the input being an io.Seeker for retries.
If that is not the case, abort and return the previous error.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2024-01-03 14:11:34 -05:00
Brandon Mitchell
f5f7e77201
Add blob tests
Test for pushing blob without a descriptor, and zero length blob.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-11-22 15:59:54 -05:00
Brandon Mitchell
3bc242a860
Add tests for ocidir throttle race
The lock on the throttle was also moved to avoid future foot-guns.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-11-15 20:22:09 -05:00
Petu Eusebiu
8f20d256cf fix(race): fixed throttle locks in OCIDir and Host config
Signed-off-by: Petu Eusebiu <peusebiu@cisco.com>
2023-11-15 18:34:15 +02:00
Brandon Mitchell
0a514c3e2d
Updating Ping method
- Adds a struct to the return so headers can be inspected.
- Puts Ping into the scheme interface.
- Adds an ocidir implementation to verify directory is accessible.
- Fixes some http/auth handling.
- Warns on `regctl registry config` and `login` of Ping failure.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-11-08 19:41:19 -05:00
Nikolay Edigaryev
2881dcfb0d
RegClient: implement Ping() method
Signed-off-by: Nikolay Edigaryev <edigaryev@gmail.com>
2023-11-08 19:41:14 -05:00
Brandon Mitchell
68a6c083d9
Refactor ref package
- Add NewHost to parse hostname or ocidir path.
- Add IsSet method to verify reference is fully defined.
- Add SetDigest and SetTag methods to create a new Ref.
- Update other packages to use new methods.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-11-06 19:05:29 -05:00
Brandon Mitchell
d99294d76e
Fix crash on RepoList from invalid hostname
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-10-16 21:33:38 -04:00
Brandon Mitchell
0ad815ee02
Update tests to use t.Parallel where possible
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-10-02 20:09:11 -04:00
Brandon Mitchell
b2a7336453
Reorder Go imports to move local packages last
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-09-30 22:21:10 -04:00
Brandon Mitchell
6d9822af08
Cleanup links in "deprecated" comments
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-09-30 12:40:03 -04:00
Brandon Mitchell
934e52602a
Test for implementation of interfaces
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-09-26 16:51:27 -04:00
Brandon Mitchell
b933caa59d
Add gosec security linter
Most of this is error handling. Many of the file IO warnings are
intentionally ignored since these tools are designed to run locally.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-09-25 21:44:24 -04:00
Brandon Mitchell
b38b60aa24
Add ability to filter artifacts in an index
- Move descriptor filter/sort into descriptor funcs
- Reuse those methods for referrers get/list
- Use those methods in artifact get when an index is seen

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-09-17 20:00:03 -04:00
Brandon Mitchell
3996b335be
Add the ability to access the OCI index in tag ls
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-08-20 15:59:16 -04:00
Brandon Mitchell
c09c324f4a
Avoid and clean stale content in ocidir
- manifest delete should not fail when referrer file is missing
- artifact put of referrer should not add a manifest reference in ocidir
- reproducible image creation scripts should prune stale referrers

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-08-13 21:10:36 -04:00
Brandon Mitchell
ad0b9650c7
Add manifest and blob get/put limits
- This adds a 4MB manifest put limit (per the OCI spec).
- It also adds an 8MB manifest get limit to prevent DOS attacks.
- Blob reads also stop when the expected length is reached.
- HTTP retry backoff checks were added to read/close methods.

Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-08-12 17:25:29 -04:00
Brandon Mitchell
44ea53b40f
Add an in memory cache of manifests and referrers
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-08-05 14:52:59 -04:00
Brandon Mitchell
6859db04e2
Cleanup artifact annotations
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-07-29 21:47:01 -04:00
Brandon Mitchell
3ad77dbbcc
Reorder manifest accept headers
Signed-off-by: Brandon Mitchell <git@bmitch.net>
2023-07-12 20:02:40 -04:00