These experimental packages are now available in the Go standard
library since Go 1.21:
1. golang.org/x/exp/slices -> slices [1]
2. golang.org/x/exp/maps -> maps [2]
[1]: https://go.dev/doc/go1.21#slices
[2]: https://go.dev/doc/go1.21#maps
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
Add API for adding arbitrary layers at commit-time via CommitOptions,
and via methods of the Builder type.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Make setting the Parent field in the config blob of a docker format
image optional (yes, we're bringing it back!), since it no longer
appears to be set by newer versions of docker build.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Use slices.Clone() and maps.Clone() instead of our own non-generic
functions. We have to be more careful in a couple of places where we
set items in maps which aren't unconditionally initialized.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a --sbom flag to `buildah build` and `buildah commit` which will
scan the rootfs and specified context directories to build SPDX or
CycloneDX SBOMs and lists of package URLs.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a flag to `buildah commit` which allows adding arbitrary files to
the image while we're committing it. When not squashing, they'll take
the form of a second new layer.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add an OverrideChanges and an OverrideConfig field to CommitOptions,
both of which can be used to make last-minute edits to the configuration
of an image that we're committing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a --cw option to `buildah build` and `buildah commit`, which takes a
comma-separated list of arguments and produces an image laid out for use
as a confidential workload:
type: sev or snp
attestation_url: location of a key broker server
cpus: expected number of virtual CPUs to run with
memory: expected megabytes of memory to run with
workload_id: a distinguishing identifier for the key broker server
ignore_attestation_errors: ignore errors registering the workload
passphrase: for encrypting the disk image
slop: extra space to allocate for the disk image
At least one of attestation_url and passphrase must be specified in
order for the encrypted disk image to be decryptable at run-time. Other
arguments can be omitted. ignore_attestation_errors is intentionally
undocumented, as it's mainly used to permit some amount of testing on
systems which don't have the required hardware.
Add an `mkcw` top-level command, for converting directly from an image
to a confidential workload.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
In golang 1.19, `io/ioutil` is fully deprecated preventing Buildah from
compiling. Replace all calls with equivalent calls from the `os`
package.
Signed-off-by: Chris Evich <cevich@redhat.com>
Podman adds an Error: to every error message. So starting an error
message with "error" ends up being reported to the user as
Error: error ...
This patch removes the stutter.
Also ioutil.ReadFile errors report the Path, so wrapping the err message
with the path causes a stutter.
Signed-off-by: Daniel J Walsh dwalsh@redhat.com
[NO NEW TESTS NEEDED]
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
PR https://github.com/containers/storage/pull/1153 added a dedicated API
to remove names assigned image so use `RemoveNames` instead of racy
`SetNames`.
How to verify
```console
printf 'from quay.io/jitesoft/alpine:latest\nrun for i in $(seq 0 10000); do touch /$i; done\n' >Containerfile && for i in `seq 1 25`; do ./buildah build --squash --iidfile id.$i --timestamp 0 . & done; wait; ls -al
```
* Refer to newly added integration test.
Closes: https://github.com/containers/podman/issues/15162
Signed-off-by: Aditya R <arajan@redhat.com>
We now use the golang error wrapping format specifier `%w` instead of
the deprecated github.com/pkg/errors package.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
As per `OCI` `image-spec` setting history is optional so buildah must
support these use-cases via `--omit-history=true` some of the use-cases
are discussed below.
* There are use-cases when image-spec contributes to the image size by
adding build history to image-spec in certain use-cases this image
history is not required by the end-users but there is no way to remove
it. See: https://github.com/containers/buildah/issues/3513
* Certain build tools don't honor setting build history in image spec
however buildah does not allow processing images without build history
since setting build history is still optional as per `OCI` image-spec
( https://github.com/opencontainers/image-spec/blob/main/config.md#properties)
so buildah must support such use-cases.
Closes: https://github.com/containers/buildah/issues/4025
Closes: https://github.com/containers/buildah/issues/3513
Signed-off-by: Aditya R <arajan@redhat.com>
Add a `--env` option to `buildah build` that functions similarly to the
`buildah config --env` option, to complement `buildah build`'s
`--unsetenv` option.
Document that `buildah config`'s `--env` function fetches the current
value for a variable when the name is supplied, but no `=` or value
follows it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Allows end-users to export final build content or rootfs to external formats.
By default, a local container image is created from the build result. The --output (or -o) flag allows you to override this behavior, and a specify a custom exporter. For example, custom exporters allow you to export the build artifacts as files on the local filesystem instead of a Container image, which can be useful for generating local binaries, code generation etc.
The value for --output is a CSV-formatted string defining the exporter type and options. Currently, local and tar exporters are supported. The local exporter writes the resulting build files to a directory on the client side. The tar exporter is similar but writes the files as a single tarball (.tar).
```console
buildah build --output type=local,dest=dir .
buildah build --output type=tar,dest=rootfs.tar .
buildah build -o dir .
```
Reference: https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs
Signed-off-by: Aditya R <arajan@redhat.com>
Move multiple-platform build juggling logic from the CLI wrapper
directly into the imagebuildah package, to make using it easier for
packages that consume us as a library.
This requires reading Dockerfiles into byte slices so that we can
re-parse them for each per-platform build, rather than parsing them
directly, as we used to, since building modifies the parsed tree.
When building for multiple platforms, prefix progress log messages with
the platform description.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a pkg/parse.PlatformsFromOptions() which understands a "variant"
value as an optional third value in an OS/ARCH[/VARIANT] argument value,
which accepts a comma-separated list of them, and which returns a list
of platforms.
Teach "from" and "pull" about the --platform option and add integration
tests for them, warning if --platform was given multiple values.
Add a define.BuildOptions.JobSemaphore which an imagebuildah executor
will use in preference to one that it might allocate for itself.
In main(), allocate a JobSemaphore if the number of jobs is not 0 (which
we treat as "unlimited", and continue to allow executors to do).
In addManifest(), take a lock on the manifest list's image ID so that we
don't overwrite changes that another thread might be making while we're
attempting to make changes to it. In main(), create an empty list if
the list doesn't already exist before we start down this path, so that
we don't get two threads trying to create that manifest list at the same
time later on. Two processes could still try to create the same list
twice, but it's an incremental improvement.
Finally, if we've been given multiple platforms to build for, run their
builds concurrently and gather up their results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Instead of extracting layer content to a temporary file when we're
committing to transports that aren't containers-storage, record the ID
of a layer and the uncompressed size it has recorded for its contents.
When later asked for a blob, if we cached a layer ID and size, generate
the layer diff on the fly, otherwise check for a file named after the
digest of the requested blob in our cache directory location (usually
used for new layers that we're adding) and the supplemental location
(which can be supplied by a caller).
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Update containers common to the latest HEAD. Some bug fixes in libimage
forced us to have a clearer separation between ordinary images and
manifest lists. Hence, when looking up manifest lists without recursing
into any of their instances, we need to use `LookupManifestList()`.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Restore the push and pull API that commit dcd2a92e5669 removed.
These changes would break vendoring into openshift/builder due
to build errors.
For the same reason, restore `util.FindImage` and `util.AddImageNames`
but deprecate the `findRegistry` argument.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Move all code related handling container image over to the new
`libimage` package in containers/common. The new package is an
attempt to consolidate the code across the containers tools under the
github.com/containers umbrella.
The new `libimage` packages provides functionality to perform all kinds
of operations for managing images such as local lookups, pushing,
pulling, listing, removing, etc.
The following packages have been moved over the containers/common:
`manifests` -> `common/image/manifests`
`pkg/manifests` -> `common/pkg/manifests`
`pkg/supplemented` -> `common/pkg/supplemented`
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Buildah currently handles multiple tags when building, but does not
report it to the user by default. This reports the tags back to the
user.
Removed some unused code from commit.go, that would blow up if a user
specified AdditionalTags to the commit command, even though this is not
exposed to the user currently. In a previous try to fix this, the
removed code was causing breakage, and I did not see a real purpose in
the code.
Fixes: https://github.com/containers/buildah/issues/3084
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Prefix the image ID with the hash prefix when using `--iidfile` to be
compatible with Docker. The absence of the hash can cause
docker-compose to error out.
Reported-in: github.com/containers/podman/issues/9260
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Buildah bud --manifest XYZ was not working.
The manifest was never created. This PR Finishes
the plumbing and allows users to create a manifest
while building an image in one single command.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Golang built in functions like os.Create and others print the name of
the file system object when they fail. Wrapping them a second time
with the file system object, makes the error message look like crap
when reported to the user.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
If the registry is set to insecure allowd using BUILD_REGISTRY_SOURCES, hardcode to skip the tls verify to avoid the errors.
Returns error if set insecureRegistries but force to use tls-verify.
Signed-off-by: Qi Wang <qiwan@redhat.com>
2573: Heed our retry delay option values when retrying commit/pull/push r=TomSweeneyRedHat a=nalind
#### What type of PR is this?
/kind bug
#### What this PR does / why we need it:
Pass in our own API values for retry delays to common's retry package when we use it to handle retrying image pull/commit/push operations.
#### How to verify it
Attempt to pull an image from a registry which can't be reached, which is considered a transient error that we'll retry on:
`buildah pull --debug 127.0.0.0/bogus`
Our CLI's default is to retry with a fixed delay of 2 seconds after each failed attempt, while the retry package's default behavior is to use progressively longer delays after each attempt, so if each attempt retries after 2 seconds, the change is having the desired effect.
#### Which issue(s) this PR fixes:
None
#### Special notes for your reviewer:
This undoes a behavior change we'd otherwise have made compared to 1.15.x.
#### Does this PR introduce a user-facing change?
```
None
```
Co-authored-by: Nalin Dahyabhai <nalin@redhat.com>
Pass our own API values for retry delays to common's retry package when
we use it to handle retrying image pull/commit/push operations.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We recieved feedback on the --omit-timestamp that
users would rather specify the timestamp seconds
rather then just use EPOCH.
This PR removes --omit-timestamp from buildah bud
since this has never been released.
We also hide --omit-timestamp from buildah commit
and allow users to continue to use it, but it conflicts
with --timestamp.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
use of github.com/openshift/api/config/v1 is costly in size, something around 10MB. removal of this dependancy and using a locally defined struct took the buildah executable size from around 38MB to around 27MB. This also hits libpod and allows it also shed that size as well.
Signed-off-by: Brent Baude <bbaude@redhat.com>
If PullOptions/BuilderOptions/CommitOptions/PushOptions includes a
MaxRetries value other than 0, retry operations except for (currently)
connection-refused, authentication, and no-such-repository/no-such-tag
errors, at a default-but-configurable interval of 5 seconds.
Set the default for `buildah pull/from/commit/push` to 3 retries at 2
second intervals.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When building images we now allow the architecture and operating system
to be overwritten via the new `--os` and `--arch` flags. This allows the
use case of packing pre-built or cross-compiled binaries into container
images with specifying the correct environment. The `--platform` flag
now takes the `os/arch` format into account as well.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #2097
Approved by: rhatdan
Add the --sign-by option to `buildah build-using-dockerfile`,
`buildah commit`, `buildah push`, and `buildah manifest push`. Add the
`--remove-signatures` option to `buildah pull`, `buildah push`, and
`buildah manifest push`. We just pass them to the image library, which
does all of the heavy lifting.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #2085
Approved by: rhatdan
The storage library can return its errors wrapped in additional context,
so to do a direct comparison we need to dig down to the root cause.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If $BUILD_REGISTRY_SOURCES is set, expect it to be a valid
github.com/openshift/api/config/v1.Image, and parse its
AllowedRegistries and BlockedRegistries lists when, pulling, committing,
or pushing images.
Override the local signature policy when committing or pushing an image
to ensure that local storage is always allowed.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1787
Approved by: rhatdan
