* Eliminate all uses of `std.debug.print` in make() functions, instead
properly using the step failure reporting mechanism.
* Introduce the concept of skipped build steps. These do not cause the
build to fail, and they do allow their dependants to run.
* RunStep gains a new flag, `skip_foreign_checks` which causes the
RunStep to be skipped if stdio mode is `check` and the binary cannot
be executed due to it being a foreign executable.
- RunStep is improved to automatically use known interpreters to
execute binaries if possible (integrating with flags such as
-fqemu and -fwasmtime). It only does this after attempting a native
execution and receiving a "exec file format" error.
- Update RunStep to use an ArrayList for the checks rather than this
ad-hoc reallocation/copying mechanism.
- `expectStdOutEqual` now also implicitly adds an exit_code==0 check
if there is not already an expected termination. This matches
previously expected behavior from older API and can be overridden by
directly setting the checks array.
* Add `dest_sub_path` to `InstallArtifactStep` which allows choosing an
arbitrary subdirectory relative to the prefix, as well as overriding
the basename.
- Delete the custom InstallWithRename step that I found deep in the
test/ directory.
* WriteFileStep will now update its step display name after the first
file is added.
* Add missing stdout checks to various standalone test case build
scripts.
Rework std.Build.Step to have an `owner: *Build` field. This
simplified the implementation of installation steps, as well as provided
some much-needed common API for the new parallelized build system.
--verbose is now defined very concretely: it prints to stderr just
before spawning a child process.
Child process execution is updated to conform to the new
parallel-friendly make() function semantics.
DRY up the failWithCacheError handling code. It now integrates properly
with the step graph instead of incorrectly dumping to stderr and calling
process exit.
In the main CLI, fix `zig fmt` crash when there are no errors and stdin
is used.
Deleted steps:
* EmulatableRunStep - this entire thing can be removed in favor of a
flag added to std.Build.RunStep called `skip_foreign_checks`.
* LogStep - this doesn't really fit with a multi-threaded build runner
and is effectively superseded by the new build summary output.
build runner:
* add -fsummary and -fno-summary to override the default behavior,
which is to print a summary if any of the build steps fail.
* print the dep prefix when emitting error messages for steps.
std.Build.FmtStep:
* This step now supports exclude paths as well as a check flag.
* The check flag decides between two modes, modify mode, and check
mode. These can be used to update source files in place, or to fail
the build, respectively.
Zig's own build.zig:
* The `test-fmt` step will do all the `zig fmt` checking that we expect
to be done. Since the `test` step depends on this one, we can simply
remove the explicit call to `zig fmt` in the CI.
* The new `fmt` step will actually perform `zig fmt` and update source
files in place.
std.Build.RunStep:
* expose max_stdio_size is a field (previously an unchangeable
hard-coded value).
* rework the API. Instead of configuring each stream independently,
there is a `stdio` field where you can choose between
`infer_from_args`, `inherit`, or `check`. These determine whether the
RunStep is considered to have side-effects or not. The previous
field, `condition` is gone.
* when stdio mode is set to `check` there is a slice of any number of
checks to make, which include things like exit code, stderr matching,
or stdout matching.
* remove the ill-defined `print` field.
* when adding an output arg, it takes the opportunity to give itself a
better name.
* The flag `skip_foreign_checks` is added. If this is true, a RunStep
which is configured to check the output of the executed binary will
not fail the build if the binary cannot be executed due to being for
a foreign binary to the host system which is running the build graph.
Command-line arguments such as -fqemu and -fwasmtime may affect
whether a binary is detected as foreign, as well as system
configuration such as Rosetta (macOS) and binfmt_misc (Linux).
- This makes EmulatableRunStep no longer needed.
* Fix the child process handling to properly integrate with the new
bulid API and to avoid deadlocks in stdout/stderr streams by polling
if necessary.
std.Build.RemoveDirStep now uses the open build_root directory handle
instead of an absolute path.
The compiler now provides a server protocol for an interactive session
with another process. The build runner uses this protocol to communicate
compilation errors semantically from zig compiler subprocesses to the
build runner.
The protocol is exposed via stdin/stdout, or on a network socket,
depending on whether the CLI flag `--listen=-` or e.g.
`--listen=127.0.0.1:1337` is used.
Additionally:
* add the zig version string to the build runner cache prefix
* remove --prominent-compile-errors CLI flag because it no longer does
anything. Compilation errors are now unconditionally displayed at the
bottom of the build summary output when using the terminal-based
build runner.
* Remove the color field from std.Build. The build steps are no longer
supposed to interact with stderr directly. Instead they communicate
semantically back to the build runner, which has its own logic about
TTY configuration.
* Use the cleanExit() pattern in the build runner.
* Build steps can now use error.MakeFailed when they have already
properly reported an error, or they can fail with any other error
code in which case the build runner will create a simple message
based on this error code.
* Step.init() now takes an options struct
* Step.init() now captures a small stack trace and stores it in the
Step so that it can be accessed when printing user-friendly debugging
information, including the lines of code that created the step in
question.
Today I found out that posix_spawn is trash. It's actually implemented
on top of fork/exec inside of libc (or libSystem in the case of macOS).
So, anything posix_spawn can do, we can do better. In particular, what
we can do better is handle spawning of child processes that are
potentially foreign binaries. If you try to spawn a wasm binary, for
example, posix spawn does the following:
* Goes ahead and creates a child process.
* The child process writes "foo.wasm: foo.wasm: cannot execute binary file"
to stderr (yes, it prints the filename twice).
* The child process then exits with code 126.
This behavior is indistinguishable from the binary being successfully
spawned, and then printing to stderr, and exiting with a failure -
something that is an extremely common occurrence.
Meanwhile, using the lower level fork/exec will simply return ENOEXEC
code from the execve syscall (which is mapped to zig error.InvalidExe).
The posix_spawn behavior means the zig build runner can't tell the
difference between a failure to run a foreign binary, and a binary that
did run, but failed in some other fashion. This is unacceptable, because
attempting to excecve is the proper way to support things like Rosetta.
And make it not do any installation, only objcopying. We already have
install steps for doing installation.
This commit also makes ObjCopyStep properly integrate with caching.
This reverts commit 772a0eb68a, reversing
changes made to 0bb178bbb2.
This needs a rebase against master branch - it has build-breaking merge
conflicts. I also added a "changes requested" review on the original
pull request.
This makes it so that when there is a tree of std.Build objects, only
one zig-cache is used (the top-level application) instead of polluting
package directories with zig-cache folders.
* Use std.Build.Cache.Directory instead of a string for storing the
cache roots and build roots.
* Set up a std.Build.Cache in build_runner.zig and use it in
std.Build.RunStep for avoiding redundant work.
Deprecate CompileStep.run. The problem with this function is that it
does the RunStep with the same build.zig context as the CompileStep, but
this is not desirable when running an executable that is provided by a
dependency package. Instead, users should use `b.addRunArtifact`.
This has the additional benefit of conforming to the existing naming
conventions.
Additionally, support enum literals in config header options values.
Breaking API change to std.Build.addConfigHeader. It now uses an options
struct.
Introduce std.Build.CompileStep.installConfigHeader which also accepts
an options struct. This is used to add a generated config file into the
set of installed header files for a particular compilation artifact.
std.Build.ConfigHeaderStep now additionally supports a "blank" style
where a header is generated from scratch. It no longer exposes
`output_dir`. Instead it exposes a FileSource via `output_file`.
It now additionally accepts an `include_path` option which affects the
include path of CompileStep when using the `#include` directive, as well
as affecting the default installation subdirectory for header
installation purposes.
The hash used for the directory to store the generated config file now
includes the contents of the generated file. This fixes possible race
conditions when generating multiple header files simultaneously.
The values hash table is now an array hash map, to preserve order for
the "blank" use case.
I also took the opportunity to remove output_dir from TranslateCStep and
WriteFileStep. This is technically a breaking change, but it was always
naughty to access these fields.
Add ability to generate a c header file from scratch, and then both
compile with it and install it if needed.
Example:
```zig
const avconfig_h = b.addConfigHeader(.{ .path = "libavutil/avconfig.h" }, .generated, .{
.AV_HAVE_BIGENDIAN = 0, // TODO: detect based on target
.AV_HAVE_FAST_UNALIGNED = 1, // TODO: detect based on target
});
lib.addConfigHeader(avconfig_h);
lib.installConfigHeader(avconfig_h);
```
New API introduced: std.Build.addModule
This function exposes a zig module with the given name, which can be
used by packages that depend on this one via std.Build.Dependency.module.
std.Build.Pkg and related functionality is deleted. Every use case has a
straightforward upgrade path using the new Module struct.
std.Build.OptionsStep.getPackage is replaced by
std.Build.OptionsStep.createModule.
std.Build.CompileStep.addPackagePath is replaced by
std.Build.CompileStep.addAnonymousModule.
This partially addresses #14307 by renaming some of the instances of
"package" to "module".
Closes#14278
Usage of `catch unreachable` in build scripts is completely harmless
because build scripts are always run in Debug mode, however, it sets a
poor example for beginners to learn from.
