Commit graph

622 commits

Author SHA1 Message Date
Andrew Kelley
6c41d43519
Merge pull request #15641 from jacobly0/cache-race
Cache: fix race condition
2023-05-11 12:51:56 -07:00
Andrew Kelley
7f7bd206dc
Merge pull request #15519 from dweiller/issue-15482
Optimize lowering of `s[start..][0..len]`
2023-05-11 08:59:44 -07:00
Jacob Young
6e292f66db Cache: fix unnecessary cache misses
With the old logic, it was possible for a bunch of processes to queue up
to update a cache entry, and then each to do so one at a time.  Now, it
rechecks whether there still a cache miss or another process has
completed the work in the interim.
2023-05-11 00:55:40 -04:00
Jacob Young
fae6290387 Cache: fix race condition
When checking a cache entry with no input files for a hit, if
`createFile` returned `error.WouldBlock` we would forget about the fact
that the file has been created, and all future checks will assume that a
cache hit has happened, even though one never has or does, leading to
rare `FileNotFound` errors trying the access the protected files.

This fix works by writing an extra byte to the manifest file to
distinguish hits and misses when there no input files to write.
2023-05-11 00:55:40 -04:00
Ali Chraghi
05ceac52c9 std.Build: support #cmakedefine01 pattern 2023-05-10 16:10:03 +03:00
0x5a4
11fabc4cbe fix 'zig build test' crashing with no tests
fixes #15553
2023-05-10 00:43:45 -07:00
dweiller
bd3360e03d convert s[start..start+len] to s[start..][0..len] 2023-05-07 15:55:21 +10:00
Veikka Tuominen
3f3b1a6808 std.Build: use Step.* instead of *Step
Follow up to 13eb7251d3
2023-05-03 20:55:29 -07:00
Nicolas Sterchele
13eb7251d3 build: rename std.Build.*Step to std.Build.Step.*
Follow-up actions from #14647

Fixes #14947
2023-05-03 08:39:24 +03:00
Linus Groh
94e30a756e std: fix a bunch of typos
The majority of these are in comments, some in doc comments which might
affect the generated documentation, and a few in parameter names -
nothing that should be breaking, however.
2023-04-30 18:16:04 -07:00
Andrew Kelley
e3bb06a984 std.Build.RunStep: show test name on process termination
This is a small change to help when reading failure logs which makes the
"exited with code 1" and similar message include the test name.

Further enhancements could do the following:
 * even if one unit test crashes the process, the parent process
   continues running the other unit tests
 * ability to test for expected panics (#1356)
 * timeouts on individual tests
2023-04-28 13:29:39 -07:00
Andrew Kelley
6261c13731 update codebase to use @memset and @memcpy 2023-04-28 13:24:43 -07:00
zooster
bc8e1e1de4
Improvements to docs and text
* docs(std.math): elaborate on difference between absCast and absInt

* docs(std.rand.Random.weightedIndex): elaborate on likelihood

I think this makes it easier to understand.

* langref: add small reminder

* docs(std.fs.path.extension): brevity

* docs(std.bit_set.StaticBitSet): mention the specific types

* std.debug.TTY: explain what purpose this struct serves

This should also make it clearer that this struct is not supposed to provide unrelated terminal manipulation functionality such as setting the cursor position or something because terminals are complicated and we should keep this struct simple and focused on debugging.

* langref(package listing): brevity

* langref: explain what exactly `threadlocal` causes to happen

* std.array_list: link between swapRemove and orderedRemove

Maybe this can serve as a TLDR and make it easier to decide.

* PrefetchOptions.locality: clarify docs that this is a range

This confused me previously and I thought I can only use either 0 or 3.

* fix typos and more

* std.builtin.CallingConvention: document some CCs

* langref: explain possibly cryptic names

I think it helps knowing what exactly these acronyms (@clz and @ctz) and
abbreviations (@popCount) mean.

* variadic function error: add missing preposition

* std.fmt.format docs: nicely hyphenate

* help menu: say what to optimize for

I think this is slightly more specific than just calling it
"optimizations". These are speed optimizations. I used the word
"performance" here.
2023-04-23 21:06:21 +03:00
Manlio Perillo
4374ce51b0 std.Build.RunStep: fix captureStdOut function return type
The return type of the captureStdOut function is incorrect.
Replace *std.Build.GeneratedFile with std.Build.FileSource.
2023-04-21 15:24:23 -07:00
Andrew Kelley
ceff278202 fixes to the previous commit
* CompileStep: Avoid calling producesPdbFile() to determine whether the
   option should be respected. If the user asks for it, put it on the
   command line and let the Zig CLI deal with it appropriately.
 * Make the namespace of `std.dwarf.Format.dwarf32` no longer have a
   redundant "dwarf" in it.
 * Add `zig cc` integration for `-gdwarf32` and `-gdwarf64`.
 * Toss in a bonus bug fix for `-gdwarf-2`, `-gdwarf-3`, etc.
 * Avoid using default init values for struct fields unnecessarily.
 * Add missing cache hash addition for the new option.
2023-04-20 15:17:07 -07:00
David Gonzalez Martin
d026202a26 Expose an option for producing 64-bit DWARF format
This commit enables producing 64-bit DWARF format for Zig executables
that are produced through the LLVM backend. This is achieved by exposing
both command-line flags and CompileStep flags. The production of the
64-bit format only affects binaries that use the DWARF format and it is
disabled on MacOS due to it being problematic. This commit, despite
generating the interface for the Zig user to be able to tell the compile
which format is wanted, is just implemented for the LLVM backend, so
clang and the self-hosted backends will need this to be implemented in a
future commit.

This is an effort to work around #7962, since the emission of the 64-bit
format automatically produces 64-bit relocations. Further investigation
will be needed to make DWARF 32-bit format to emit bigger relocations
when needed and not make the linker angry.
2023-04-20 14:46:53 -07:00
Andrew Kelley
34286530b7 Cache: fix multi-process race condition on macOS
This fixes `.INVAL => unreachable` being triggered by the cache system
on macOS when multiple processes race to create the same compilation.
The problem is that when two processes race to create a file, it
sometimes returns ENOENT even though that error code is nonsensical for
this situation.

Commit 2b0929929d purportedly solved this,
but it did not open the file with write permissions, leading to the
EINVAL panic later on. This commit remedies the situation by introducing
a loop and simply retrying when the ENOENT occurs.
2023-04-18 17:53:47 -07:00
Andrew Kelley
4a233d1871 CI: more C backend test coverage
The CI now runs C backend tests in addition to compiling them. It uses
-std=c99 -pedantic -Werror in order to catch non-conformant C code.

This necessitated disabling a test case that caused a C compile error,
in addition to disabling a handful of warnings that are already being
triggered by Zig's C backend output for the behavior tests.

The upshot is that I was able to, very cleanly, integrate the C backend
tests into the build system, so that it communicates via the test runner
protocol along with all the other behavior tests.
2023-04-15 10:33:08 -07:00
Andrew Kelley
adc9b77d5f std.Build: add some more init options to CompileStep 2023-04-15 10:33:08 -07:00
Andrew Kelley
e0890734f0 std.Build.CompileStep: fix installLibraryHeaders regression
reverts regression introduced in
d2ad3f5074.

The commit correctly removed dest_builder from InstallArtifactStep, but
the change to installLibraryHeaders was incorrect since it affected
different steps than that one.
2023-04-14 11:59:27 -07:00
mlugg
ccf670c2b0 Zir: implement explicit block_comptime instruction
Resolves: #7056
2023-04-12 12:06:19 -04:00
Andrew Kelley
3c3cee2cfa fix build logic due to state mutations and break the API accordingly
* remove setName, setFilter, and setTestRunner. Please set these
   options directly when creating the CompileStep.
 * removed unused field
 * remove computeOutFileNames and inline the logic, making clear the
   goal of avoiding state mutations after the build step is created.
2023-04-11 08:42:14 -07:00
Andrew Kelley
5a8b1bde5b std.Build.CompileStep: remove output_dir
Build scripts must instead use the FileSource abstraction rather than
telling the compiler directly where to output files.

closes #14951
2023-04-11 08:41:51 -07:00
Andrew Kelley
c96cb98ad1 CLI: remove --enable-cache option
This use case is now handled instead by the --listen option.

closes #15025
closes #15072
2023-04-11 08:41:24 -07:00
Andrew Kelley
7221e9560e std.Build.RunStep: don't close stdin
Instead, ignore stdin. Otherwise this interfers with a sub-process which
expects to have a stdin file descriptor open.
2023-04-10 18:35:14 -07:00
Andrew Kelley
60eabc0eca std.Build.CompileStep: remove run() and install()
These functions are problematic in light of dependencies because they
run and install, respectively, for the *owner* package rather than for
the *user* package. By removing these functions, the build script is
forced to provide the *Build object to associate the new step with,
making everything less surprising.

Unfortunately, this is a widely breaking change.

see #15079
2023-04-10 18:35:14 -07:00
Andrew Kelley
38698f4f6a std.Build.CompileStep: delete install_step field
This was used to ensure that an artifact would only be installed once,
but this is not only unnecessary, but actively harmful, in the face of
dependencies.

see #15079
2023-04-10 18:35:14 -07:00
Andrew Kelley
d2ad3f5074 std.Build.CompileStep: remove redundant dest_builder field
The destination *Build object is already known to be step.owner, while
the source object is artifact.step.owner.
2023-04-10 18:35:14 -07:00
Jakub Konka
a88c0b4d08 link: handle -u flag in all linkers
Also clean up parsing of linker args - reuse `ArgsIterator`.

In MachO, ensure we add every symbol marked with `-u` as undefined
before proceeding with symbol resolution. Additionally, ensure those
symbols are never garbage collected.

MachO entry_in_dylib test: pass `-u _my_main` when linking executable
so that it is not incorrectly garbage collected by the linker.
2023-04-01 14:22:44 +02:00
kcbanner
4569a28ea3 build: fixes from review 2023-03-25 16:33:25 -04:00
kcbanner
113f80bcf7 coff: change dynamicbase to default to true (to match lld), change it to pass the negation to lld, and add --no-dynamicbase
build: expose linker_dynamicbase on CompileStep and map it to emit --no-dynamicbase
2023-03-25 16:26:48 -04:00
Jakub Konka
145f93ba96 build: allow for deferred FileSource matching in CheckObjectStep
Re-enable all of functionality of MachO dylib test.
2023-03-23 23:46:53 +01:00
Jakub Konka
5d2892740a build: when parsing rpaths, do not expand special runtime paths on Darwin
Special runtime paths on Darwin are: `@executable_path` and `@loader_path`.
2023-03-23 22:35:27 +01:00
Andrew Kelley
626a75bbc2 std.Build.RunStep: fix control flow with qemu+glibc logic 2023-03-20 17:14:48 -04:00
Andrew Kelley
8f481dfc3c fix std.Build.OptionsStep
* use the same hash function as the rest of the steps
 * fix race condition due to a macOS oddity.
 * fix race condition due to file truncation (rename into place instead)
 * integrate with marking Step.result_cached. check if the file already
   exists with fs.access before doing anything else.
 * use a directory so that the file basename can be "options.zig"
   instead of a hash digest.
 * better error reporting in case of file system failures.
2023-03-19 00:39:29 -04:00
Nicolas Sterchele
49d37e2d17 build-step: remove latest LogStep ref
LogStep was removed during the build parallel enhancement made in this
commit 58edefc6d1
2023-03-18 15:08:53 -04:00
Jacob Young
cfcd6698cd main: add debug option to dump unoptimized llvm ir 2023-03-17 01:57:14 -04:00
Andrew Kelley
4aa5895d32 std.Build: fix invalid assumption about fifos
Previously this code asserted that a fifo's readable length was greater
than or equal to the length of its readable slice, which was an invalid
assertion.

This code avoids making that assumption.
2023-03-15 10:48:15 -07:00
Andrew Kelley
717e2c8718 std.Build.Cache: make unit tests not depend on cwd
This makes them more resilient to being run multiple times by multiple
different processes at the same time.
2023-03-15 10:48:15 -07:00
Andrew Kelley
37a7d2c78d std.Build.RunStep: fix handling spawn failure
The error was caught and created a Step failure rather than bubbling up
so that the interpreter logic could handle it. Fixes hundreds of test
failures on Windows.
2023-03-15 10:48:15 -07:00
Andrew Kelley
11de55d0dd std.Build.Cache: handle ENOENT on createFile race
Companion commit to 628fec41593a2d2eca8b504e4fe90de9823aeded
2023-03-15 10:48:15 -07:00
Andrew Kelley
3e328c89b7 std.Build.CompileStep: remove setNamePrefix and add setName 2023-03-15 10:48:15 -07:00
Andrew Kelley
2b0929929d std.Build.Cache: handle ENOENT on createFile race
There are no dir components, so you would think that this was
unreachable, however we have observed on macOS two processes racing
to do openat() with O_CREAT manifest in ENOENT.
2023-03-15 10:48:15 -07:00
Andrew Kelley
0f88ad8c72 std.Build.CompileStep: proper step dependency on headers
Rather than calling make() from within make().
2023-03-15 10:48:15 -07:00
Andrew Kelley
a26a2e1a17 build runner: fix compilation errors on windows 2023-03-15 10:48:15 -07:00
Andrew Kelley
a0dd2919eb std.build.RunStep: clean up some leftover mess
* Remove some functions that are no longer needed since
   EmulateableRunStep is gone.
 * Add removeEnvironmentVariable function.
 * Support printing environment variables in --verbose mode.
2023-03-15 10:48:14 -07:00
Andrew Kelley
ede5dcffea make the build runner and test runner talk to each other
std.Build.addTest creates a CompileStep as before, however, this kind of
step no longer actually runs the unit tests. Instead it only compiles
it, and one must additionally create a RunStep from the CompileStep in
order to actually run the tests.

RunStep gains integration with the default test runner, which now
supports the standard --listen=- argument in order to communicate over
stdin and stdout. It also reports test statistics; how many passed,
failed, and leaked, as well as directly associating the relevant stderr
with the particular test name that failed.

This separation of CompileStep and RunStep means that
`CompileStep.Kind.test_exe` is no longer needed, and therefore has been
removed in this commit.

 * build runner: show unit test statistics in build summary
 * added Step.writeManifest since many steps want to treat it as a
   warning and emit the same message if it fails.
 * RunStep: fixed error message that prints the failed command printing
   the original argv and not the adjusted argv in case an interpreter
   was used.
 * RunStep: fixed not passing the command line arguments to the
   interpreter.
 * move src/Server.zig to std.zig.Server so that the default test runner
   can use it.
 * the simpler test runner function which is used by work-in-progress
   backends now no longer prints to stderr, which is necessary in order
   for the build runner to not print the stderr as a warning message.
2023-03-15 10:48:14 -07:00
Andrew Kelley
f829f848dd std.Build.InstallFileStep: add missing step dependencies
in the creation function, which had to change from init() to create().
2023-03-15 10:48:14 -07:00
Andrew Kelley
f31aeb0010 std.Build.WriteFileStep: add missing step dependencies 2023-03-15 10:48:14 -07:00
Andrew Kelley
a333bb91ff zig objcopy: support the compiler protocol
This commit extracts out server code into src/Server.zig and uses it
both in the main CLI as well as `zig objcopy`.

std.Build.ObjCopyStep now adds `--listen=-` to the CLI for `zig objcopy`
and observes the protocol for progress and other kinds of integrations.

This fixes the last two test failures of this branch when I run
`zig build test` locally.
2023-03-15 10:48:14 -07:00