This is fairly straightforward; the actual compiler changes are limited
to the CLI, since `Compilation` already supports this combination.
A new `std.Build` API is introduced to allow representing this. By
passing the `emit_object` option to `std.Build.addTest`, you get a
`Step.Compile` which emits an object file; you can then use that as you
would any other object, such as either installing it for external use,
or linking it into another step.
A standalone test is added to cover the build system API. It builds a
test into an object, and links it into a final executable, which it then
runs.
Using this build system mechanism prevents the build system from
noticing that you're running a `zig test`, so the build runner and test
runner do not communicate over stdio. However, that's okay, because the
real-world use cases for this feature don't want to do that anyway!
Resolves: #23374
This change fixes false-positive cache hits for run steps that get run
with different sets of environment variables due the the environment map
being excluded from the cache hash.
This reverts commit dea72d15da, reversing
changes made to ab381933c8.
The changeset does not work as advertised and does not have sufficient
test coverage.
Reopens#22822
This can also be extended to ELF later as it means roughly the same thing there.
This addresses the main issue in #21721 but as I don't have a macOS machine to
do further testing on, I can't confirm whether zig cc is able to pass the entire
cgo test suite after this commit. It can, however, cross-compile a basic program
that uses cgo to x86_64-macos-none which previously failed due to lack of -x
support. Unlike previously, the resulting symbol table does not contain local
symbols (such as C static functions).
I believe this satisfies the related donor bounty: https://ziglang.org/news/second-donor-bounty
Functions like isMinGW() and isGnuLibC() have a good reason to exist: They look
at multiple components of the target. But functions like isWasm(), isDarwin(),
isGnu(), etc only exist to save 4-8 characters. I don't think this is a good
enough reason to keep them, especially given that:
* It's not immediately obvious to a reader whether target.isDarwin() means the
same thing as target.os.tag.isDarwin() precisely because isMinGW() and similar
functions *do* look at multiple components.
* It's not clear where we would draw the line. The logical conclusion before
this commit would be to also wrap Arch.isX86(), Os.Tag.isSolarish(),
Abi.isOpenHarmony(), etc... this obviously quickly gets out of hand.
* It's nice to just have a single correct way of doing something.
`std.Build.Step.Run` makes the very reasonable assumption that
`error.InvalidExe` will be reported on `spawn` if it will happen.
However, this property does not currently hold on POSIX targets. This
is, through a slightly convoluted series of events, partially
responsible for the sporadic `BrokenPipe` errors we've been seeing more
and more in CI runs.
Making `spawn` wait for the child to exec in the POSIX path introduces
a block of up to 400us. So, instead of doing that, we add a new API for
this particular case: `waitForSpawn`. This function is a nop on Windows,
but on POSIX it blocks until the child successfully (or otherwise) calls
`execvpe`, and reports the error if necessary. `std.Build.Step.Run`
calls this function, so that it can get `error.InvalidExe` when it wants
it.
I'm not convinced that this API is optimal. However, I think this entire
API needs to be either heavily refactored or straight-up redesigned
(related: #22504), so I'm not too worried about hitting the perfect API:
I'd rather just fix this bug for now, and figure out the long-term goal
a bit later.
The previous logic here was trying to assume that custom test runners
never used `std.zig.Server` to communicate with the build runner;
however, it was flawed, because modifying the `test_runner` field on
`Step.Compile` would not update this flag. That might have been
intentional (allowing a way for the user to specify a custom test runner
which *does* use the compiler server protocol), but if so, it was a
flawed API, since it was too easy to update one field without updating
the other.
Instead, bundle these two pieces of state into a new type
`std.Build.Step.Compile.TestRunner`. When passing a custom test runner,
you are now *provided* to specify whether it is a "simple" runner, or
whether it uses the compiler server protocol.
This is a breaking change, but is unlikely to affect many people, since
custom test runners are seldom used in the wild.
This was done by regex substitution with `sed`. I then manually went
over the entire diff and fixed any incorrect changes.
This diff also changes a lot of `callconv(.C)` to `callconv(.c)`, since
my regex happened to also trigger here. I opted to leave these changes
in, since they *are* a correct migration, even if they're not the one I
was trying to do!
The goals of this branch are to:
* compile faster when using the wasm linker and backend
* enable saving compiler state by directly copying in-memory linker
state to disk.
* more efficient compiler memory utilization
* introduce integer type safety to wasm linker code
* generate better WebAssembly code
* fully participate in incremental compilation
* do as much work as possible outside of flush(), while continuing to do
linker garbage collection.
* avoid unnecessary heap allocations
* avoid unnecessary indirect function calls
In order to accomplish this goals, this removes the ZigObject
abstraction, as well as Symbol and Atom. These abstractions resulted
in overly generic code, doing unnecessary work, and needless
complications that simply go away by creating a better in-memory data
model and emitting more things lazily.
For example, this makes wasm codegen emit MIR which is then lowered to
wasm code during linking, with optimal function indexes etc, or
relocations are emitted if outputting an object. Previously, this would
always emit relocations, which are fully unnecessary when emitting an
executable, and required all function calls to use the maximum size LEB
encoding.
This branch introduces the concept of the "prelink" phase which occurs
after all object files have been parsed, but before any Zcu updates are
sent to the linker. This allows the linker to fully parse all objects
into a compact memory model, which is guaranteed to be complete when Zcu
code is generated.
This commit is not a complete implementation of all these goals; it is
not even passing semantic analysis.
This commit amends `std.Build.ExecutableOptions` etc to have a new
field, `root_module`, which allows artifacts to be created whose root
module is an existing `*Module` rather than a freshly constructed one.
This API can be far more versatile, allowing construction of complex
module graphs before creating any compile steps, and therefore also
allowing easy reuse of modules.
The fields which correspond to module options, such as
`root_source_file`, are all considered deprecated. They may not be
populated at the same time as the `root_module` field. In the next
release cycle, these deprecated fields will be removed, and the
`root_module` field made non-optional.
At the expense of a slight special case in the build runner, we can make
the handling of dependencies between modules a little shorter and much
easier to follow.
When module and step graphs are being constructed during the "configure"
phase, we do not set up step dependencies triggered by modules. Instead,
after the configure phase, the build runner traverses the whole
step/module graph, starting from the root top-level steps, and
configures all step dependencies implied by modules. The "make" phase
then proceeds as normal. Also, the old `Module.dependencyIterator` logic
is replaced by two separate iterables. `Module.getGraph` takes the root
module of a compilation, and returns all modules in its graph; while
`Step.Compile.getCompileDependencies` takes a `*Step.Compile` and
returns all `*Step.Compile` it depends on, recursively, possibly
excluding dynamic libraries. The old `Module.dependencyIterator`
combined these two functions into one unintuitive iterator; they are now
separated, which in particular helps readability at the usage sites
which only need one or the other.
This commit changes the `root_module` field of `std.Build.Step.Compile`
to be a `*Module` rather than a `Module`. This is a breaking change, but
an incredibly minor one (the full potential extent of the breakage can
be seen in the modified standalone test).
This change will be necessary for an upcoming improvement, so it was
convenient to make it here.
We have deduced that it seems the sporadic BrokenPipe failures happening
on the CI runners (e.g.
https://github.com/ziglang/zig/actions/runs/12035916948/job/33555963190)
are likely caused by the test runner's stdin pipe abnormally closing,
likely due to the process crashing. Here, we introduce error handling
for this case, so that if these writes fail, the step is marked as
failed correctly, and we still collect the child's stderr to report.
This won't fix the CI issues, but it should promote them to proper error
messages including child stderr, which -- at least in theory -- should
allow us to ultimately track down where the errors come from.
Note that this change is desirable regardless of bugs in the test runner
or similar, since the child process could terminate abnormally for any
number of reasons (e.g. a crashing test), and such cases should be
correctly reported by the build runner.
When using Build.Step.Run.captureStdOut with a program that prints more
than 10 megabytes of output, the build process hangs.
This is because evalGeneric returns an error without reading child's
stdin till the end, so we subsequently get stuck in `try child.wait()`.
To fix this, make sure to kill the child in case of an error!
Output before this change:
λ ./zig/zig build -Dmultiversion=0.15.6 -Dconfig-release=0.15.7 -Dconfig-release-client-min=0.15.6
[3/8] steps
└─ run gh
^C
λ # an hour of debugging
Output after this change:
λ ./zig/zig build -Dmultiversion=0.15.6 -Dconfig-release=0.15.7 -Dconfig-release-client-min=0.15.6
install
└─ install generated to ../tigerbeetle
└─ run build_mutliversion (tigerbeetle)
└─ run unzip
└─ run gh failure
error: unable to spawn gh: StdoutStreamTooLong
Build Summary: 3/8 steps succeeded; 1 failed (disable with --summary none)
install transitive failure
└─ install generated to ../tigerbeetle transitive failure
└─ run build_mutliversion (tigerbeetle) transitive failure
└─ run unzip transitive failure
└─ run gh failure
error: the following build command failed with exit code 1:
/home/matklad/p/tb/work/.zig-cache/o/c0e3f5e66ff441cd16f9a1a7e1401494/build /home/matklad/p/tb/work/zig/zig /home/matklad/p/tb/work /home/matklad/p/tb/work/.zig-cache /home/matklad/.cache/zig --seed 0xc1d4efc8 -Zaecc61299ff08765 -Dmultiversion=0.15.6 -Dconfig-release=0.15.7 -Dconfig-release-client-min=0.15.6
