Allows end-users to export final build content or rootfs to external formats.
By default, a local container image is created from the build result. The --output (or -o) flag allows you to override this behavior, and a specify a custom exporter. For example, custom exporters allow you to export the build artifacts as files on the local filesystem instead of a Container image, which can be useful for generating local binaries, code generation etc.
The value for --output is a CSV-formatted string defining the exporter type and options. Currently, local and tar exporters are supported. The local exporter writes the resulting build files to a directory on the client side. The tar exporter is similar but writes the files as a single tarball (.tar).
```console
buildah build --output type=local,dest=dir .
buildah build --output type=tar,dest=rootfs.tar .
buildah build -o dir .
```
Reference: https://docs.docker.com/engine/reference/commandline/build/#custom-build-outputs
Signed-off-by: Aditya R <arajan@redhat.com>
Move multiple-platform build juggling logic from the CLI wrapper
directly into the imagebuildah package, to make using it easier for
packages that consume us as a library.
This requires reading Dockerfiles into byte slices so that we can
re-parse them for each per-platform build, rather than parsing them
directly, as we used to, since building modifies the parsed tree.
When building for multiple platforms, prefix progress log messages with
the platform description.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Add a pkg/parse.PlatformsFromOptions() which understands a "variant"
value as an optional third value in an OS/ARCH[/VARIANT] argument value,
which accepts a comma-separated list of them, and which returns a list
of platforms.
Teach "from" and "pull" about the --platform option and add integration
tests for them, warning if --platform was given multiple values.
Add a define.BuildOptions.JobSemaphore which an imagebuildah executor
will use in preference to one that it might allocate for itself.
In main(), allocate a JobSemaphore if the number of jobs is not 0 (which
we treat as "unlimited", and continue to allow executors to do).
In addManifest(), take a lock on the manifest list's image ID so that we
don't overwrite changes that another thread might be making while we're
attempting to make changes to it. In main(), create an empty list if
the list doesn't already exist before we start down this path, so that
we don't get two threads trying to create that manifest list at the same
time later on. Two processes could still try to create the same list
twice, but it's an incremental improvement.
Finally, if we've been given multiple platforms to build for, run their
builds concurrently and gather up their results.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Instead of extracting layer content to a temporary file when we're
committing to transports that aren't containers-storage, record the ID
of a layer and the uncompressed size it has recorded for its contents.
When later asked for a blob, if we cached a layer ID and size, generate
the layer diff on the fly, otherwise check for a file named after the
digest of the requested blob in our cache directory location (usually
used for new layers that we're adding) and the supplemental location
(which can be supplied by a caller).
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Update containers common to the latest HEAD. Some bug fixes in libimage
forced us to have a clearer separation between ordinary images and
manifest lists. Hence, when looking up manifest lists without recursing
into any of their instances, we need to use `LookupManifestList()`.
[NO NEW TESTS NEEDED]
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Restore the push and pull API that commit dcd2a92e5669 removed.
These changes would break vendoring into openshift/builder due
to build errors.
For the same reason, restore `util.FindImage` and `util.AddImageNames`
but deprecate the `findRegistry` argument.
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Move all code related handling container image over to the new
`libimage` package in containers/common. The new package is an
attempt to consolidate the code across the containers tools under the
github.com/containers umbrella.
The new `libimage` packages provides functionality to perform all kinds
of operations for managing images such as local lookups, pushing,
pulling, listing, removing, etc.
The following packages have been moved over the containers/common:
`manifests` -> `common/image/manifests`
`pkg/manifests` -> `common/pkg/manifests`
`pkg/supplemented` -> `common/pkg/supplemented`
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Buildah currently handles multiple tags when building, but does not
report it to the user by default. This reports the tags back to the
user.
Removed some unused code from commit.go, that would blow up if a user
specified AdditionalTags to the commit command, even though this is not
exposed to the user currently. In a previous try to fix this, the
removed code was causing breakage, and I did not see a real purpose in
the code.
Fixes: https://github.com/containers/buildah/issues/3084
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Prefix the image ID with the hash prefix when using `--iidfile` to be
compatible with Docker. The absence of the hash can cause
docker-compose to error out.
Reported-in: github.com/containers/podman/issues/9260
Signed-off-by: Valentin Rothberg <rothberg@redhat.com>
Buildah bud --manifest XYZ was not working.
The manifest was never created. This PR Finishes
the plumbing and allows users to create a manifest
while building an image in one single command.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Golang built in functions like os.Create and others print the name of
the file system object when they fail. Wrapping them a second time
with the file system object, makes the error message look like crap
when reported to the user.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
If the registry is set to insecure allowd using BUILD_REGISTRY_SOURCES, hardcode to skip the tls verify to avoid the errors.
Returns error if set insecureRegistries but force to use tls-verify.
Signed-off-by: Qi Wang <qiwan@redhat.com>
2573: Heed our retry delay option values when retrying commit/pull/push r=TomSweeneyRedHat a=nalind
#### What type of PR is this?
/kind bug
#### What this PR does / why we need it:
Pass in our own API values for retry delays to common's retry package when we use it to handle retrying image pull/commit/push operations.
#### How to verify it
Attempt to pull an image from a registry which can't be reached, which is considered a transient error that we'll retry on:
`buildah pull --debug 127.0.0.0/bogus`
Our CLI's default is to retry with a fixed delay of 2 seconds after each failed attempt, while the retry package's default behavior is to use progressively longer delays after each attempt, so if each attempt retries after 2 seconds, the change is having the desired effect.
#### Which issue(s) this PR fixes:
None
#### Special notes for your reviewer:
This undoes a behavior change we'd otherwise have made compared to 1.15.x.
#### Does this PR introduce a user-facing change?
```
None
```
Co-authored-by: Nalin Dahyabhai <nalin@redhat.com>
Pass our own API values for retry delays to common's retry package when
we use it to handle retrying image pull/commit/push operations.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
We recieved feedback on the --omit-timestamp that
users would rather specify the timestamp seconds
rather then just use EPOCH.
This PR removes --omit-timestamp from buildah bud
since this has never been released.
We also hide --omit-timestamp from buildah commit
and allow users to continue to use it, but it conflicts
with --timestamp.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
use of github.com/openshift/api/config/v1 is costly in size, something around 10MB. removal of this dependancy and using a locally defined struct took the buildah executable size from around 38MB to around 27MB. This also hits libpod and allows it also shed that size as well.
Signed-off-by: Brent Baude <bbaude@redhat.com>
If PullOptions/BuilderOptions/CommitOptions/PushOptions includes a
MaxRetries value other than 0, retry operations except for (currently)
connection-refused, authentication, and no-such-repository/no-such-tag
errors, at a default-but-configurable interval of 5 seconds.
Set the default for `buildah pull/from/commit/push` to 3 retries at 2
second intervals.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When building images we now allow the architecture and operating system
to be overwritten via the new `--os` and `--arch` flags. This allows the
use case of packing pre-built or cross-compiled binaries into container
images with specifying the correct environment. The `--platform` flag
now takes the `os/arch` format into account as well.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #2097
Approved by: rhatdan
Add the --sign-by option to `buildah build-using-dockerfile`,
`buildah commit`, `buildah push`, and `buildah manifest push`. Add the
`--remove-signatures` option to `buildah pull`, `buildah push`, and
`buildah manifest push`. We just pass them to the image library, which
does all of the heavy lifting.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #2085
Approved by: rhatdan
The storage library can return its errors wrapped in additional context,
so to do a direct comparison we need to dig down to the root cause.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
If $BUILD_REGISTRY_SOURCES is set, expect it to be a valid
github.com/openshift/api/config/v1.Image, and parse its
AllowedRegistries and BlockedRegistries lists when, pulling, committing,
or pushing images.
Override the local signature policy when committing or pushing an image
to ensure that local storage is always allowed.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1787
Approved by: rhatdan
This commit enabled the `unparam` linter and applies all reported issues.
Signed-off-by: Sascha Grunert <sgrunert@suse.com>
Closes: #1719
Approved by: rhatdan
Avoid calling fmt.Printf() to print things in library logic, which can't
be controlled or suppressed by callers. Prefer returning values and
printing them in our CLI wrapper, as callers would.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1596
Approved by: rhatdan
Ignore the global signature policy, which is used to control whether or
not we can trust an image, when we're attempting to commit an image from
a container, or pushing an image.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1579
Approved by: rhatdan
When building an image with multiple layers, go back to committing
images for instructions for which we previously wouldn't bother
committing an image, but create them without adding a new layer.
This violates some assumptions that we currently make elsewhere, as it's
possible for an image that's derived from a base image to add no layers
relative to the base image, when previously it was always the case that
we'd add at least one whenever we committed it.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1539
Approved by: rhatdan
We don't read the contents of the OnBuild field in CommitOptions, so
drop it from the structure definition.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1539
Approved by: rhatdan
Instead of passing in an increasing number of the fields of
CommitOptions to makeImageRef(), just pass the whole thing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1539
Approved by: rhatdan
Always set a parent ID when we go to commit an image, whether it's as
part of build-using-dockerfile or our "commit" CLI. Coerce the parent
image's ID directly into the value that we use instead of digesting it
again.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1538
Approved by: vrothberg
Make the name of the image to create an optional parameter. If none is
specified, use a temporary mostly-random name that can't be interpreted
as an ID, so that the image copying logic will compute the correct ID to
assign to the new image, and remove the temporary name before returning.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1486
Approved by: rhatdan
The Builder object already keeps a pointer to the Store that holds its
container, so a second value that could be different is just confusing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1444
Approved by: TomSweeneyRedHat
should omit type ... from declaration of var ...; it will be
inferred from the right-hand side
Signed-off-by: Douglas Schilling Landgraf <dougsland@redhat.com>
Closes: #1426
Approved by: nalind
Miloslav had some good comments on a previous commit.
https://github.com/containers/buildah/pull/1411
These changes address his issues by removing them.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1412
Approved by: mtrmac
Currently rootless podman attempts to write to /var/lib/containers/cache
and fails. This causes us to repeatedly push images that have already been
pushed. This cache directory should be relative to the location of containers/storage
and not always stored in the same directory.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
Closes: #1411
Approved by: TomSweeneyRedHat
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Add Quiet to the PullOptions and PushOptions structures.
If set to true output will be in 'quiet' mode. This
will primarily be used by callers such as OpenShift.
Signed-off-by: TomSweeneyRedHat <tsweeney@redhat.com>
Closes: #1302
Approved by: rhatdan
By default, a field called "createdAt" is written into the image manifest.
However, even if my image build process is perfectly deterministic, the image
sha256 hash will be different every time, even if my sources are exactly the
same. In many cases it is desirable that the same input results in the exactly
same output.
This commit introduces the flag --omit-timestamp to the commit command. If set to
true, the timestamp is set to epoch 0, instead of the current timestamp (which
causes different results every time commit is invoked).
Signed-off-by: Johannes Brüderl <johannes.bruederl@gmail.com>
Closes: #1294
Approved by: nalind
Add API hooks for designating locations to be used as blob caches when
pulling and pushing images. When we commit read-only copies of
container layers for use in images, if we're using blob caching, store a
copy of the layer in the blob cache directory so that it can be found.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1149
Approved by: rhatdan
Add a --disable-compression flag to the 'build-using-dockerfile'
command, and set OCIAcceptUncompressedLayers and DirForceCompress in the
SystemContext during Commit() and Push() based on whether or not the
flag is set.
For other transports, we still end up going with the receiver's
preference, so update the man page to mention that this controls a
default.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1149
Approved by: rhatdan
Copying an image now returns the manifest of the written image. Return
the digest of that manifest, and a canonical reference, from our Commit
and Push APIs.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1097
Approved by: rhatdan
Make sure that when attempting to diagnose an error, if we encounter an
error during the diagnostic attempt, we return the original error rather
than the error encountered in trying to diagnose it. Log that one.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Closes: #1072
Approved by: rhatdan
