Commit graph

368 commits

Author SHA1 Message Date
Alex Rønne Petersen
c76a98f28a
std.Target: Rename is_lib{c,cxx}_lib_name() to isLib{C,Cxx}LibName(). 2024-10-16 22:24:46 +02:00
Andrew Kelley
01aab9f6b3 std.Build.Step.CheckObject: display source file path
and upgrade to getPath3
2024-10-10 14:21:52 -07:00
Andrew Kelley
22661f3d67 link tests: add a way to check prefix and use it 2024-10-08 21:57:08 -07:00
Patrick Wickenhaeuser
e1d54b6d1a 19009: zig objcopy: integrate --add-section, --set-section-alignment and --set-section-flags into std.Build.Step.ObjCopy 2024-10-04 15:49:50 +02:00
dave caruso
085cc54aad replace TranslateC.addIncludeDir with variants with LazyPath/library names 2024-09-25 21:50:55 -07:00
Linus Groh
8588964972 Replace deprecated default initializations with decl literals 2024-09-12 16:01:23 +01:00
Andrew Kelley
1bec824cad LLVM: disable inline 8-bit counters when using trace pc guard 2024-08-28 18:07:13 -07:00
mlugg
0fe3fd01dd
std: update std.builtin.Type fields to follow naming conventions
The compiler actually doesn't need any functional changes for this: Sema
does reification based on the tag indices of `std.builtin.Type` already!
So, no zig1.wasm update is necessary.

This change is necessary to disallow name clashes between fields and
decls on a type, which is a prerequisite of #9938.
2024-08-28 08:39:59 +01:00
mlugg
a0b03d7fff
std.Build.Step.TranslateC: propagate target, optimize, link_libc to created module 2024-08-21 19:13:26 +01:00
Robin Voetter
43f73af359
fix various issues related to Path handling in the compiler and std
A compilation build step for which the binary is not required could not
be compiled previously. There were 2 issues that caused this:

- The compiler communicated only the results of the emitted binary and
  did not properly communicate the result if the binary was not emitted.

  This is fixed by communicating the final hash of the artifact path (the
  hash of the corresponding /o/<hash> directory) and communicating this
  instead of the entire path. This changes the zig build --listen protocol
  to communicate hashes instead of paths, and emit_bin_path is accordingly
  renamed to emit_digest.

- There was an error related to the default llvm object path when
  CacheUse.Whole was selected. I'm not really sure why this didn't manifest
  when the binary is also emitted.

  This was fixed by improving the path handling related to flush() and
  emitLlvmObject().

In general, this commit also improves some of the path handling throughout
the compiler and standard library.
2024-08-19 19:09:11 +02:00
Andrew Kelley
e64a00950e fuzzer web ui: introduce entry points
so you can have somewhere to start browsing
2024-08-07 00:48:32 -07:00
Andrew Kelley
517cfb0dd1 fuzzing: progress towards web UI
* libfuzzer: close file after mmap
* fuzzer/main.js: connect with EventSource and debug dump the messages.
  currently this prints how many fuzzer runs have been attempted to
  console.log.
* extract some `std.debug.Info` logic into `std.debug.Coverage`.
  Prepares for consolidation across multiple different executables which
  share source files, and makes it possible to send all the
  PC/SourceLocation mapping data with 4 memcpy'd arrays.
* std.Build.Fuzz:
  - spawn a thread to watch the message queue and signal event
    subscribers.
  - track coverage map data
  - respond to /events URL with EventSource messages on a timer
2024-08-07 00:48:32 -07:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Carter Snook
219acaa1d6 std.fs.Dir: Refactor atomicSymLink from std.fs
Deprecates std.fs.atomicSymLink and removes the allocator requirement
from the new std.fs.Dir.atomicSymLink. Replaces the two usages of this
within std.

I did not include the TODOs from the original code that were based
off of `switch (err) { ..., else => return err }` not having correct
inference that cases handled in `...` are impossible in the error
union return type because these are not specified in many places but
I can add them back if wanted.

Thank you @squeek502 for help with fixing buffer overflows!
2024-07-28 20:08:52 -07:00
Andrew Kelley
a3c74aca99 add --debug-rt CLI arg to the compiler + bonus edits
The flag makes compiler_rt and libfuzzer be in debug mode.

Also:
* fuzzer: override debug logs and disable debug logs for frequently
  called functions
* std.Build.Fuzz: fix bug of rerunning the old unit test binary
* report errors from rebuilding the unit tests better
* link.Elf: additionally add tsan lib and fuzzer lib to the hash
2024-07-25 18:52:21 -07:00
Andrew Kelley
bce3b1efb0 build runner sends a start_fuzzing message to test runner 2024-07-25 18:52:21 -07:00
Andrew Kelley
711ed56ce3 build runner: extract logic to std.Build.Fuzz 2024-07-25 18:52:21 -07:00
Andrew Kelley
047640383e add --fuzz CLI argument to zig build
This flag makes the build runner rebuild unit tests after the pipeline
finishes, if it finds any unit tests.

I did not make this integrate with file system watching yet.

The test runner is updated to detect which tests are fuzz tests.

Run step is updated to track which test indexes are fuzz tests.
2024-07-25 18:52:20 -07:00
Andrew Kelley
26d2a7960e default "trace pc guard" coverage off
* Add -f(no-)sanitize-coverage-trace-pc-guard CLI flag which defaults to
  off. This value lowers to TracePCGuard = true (LLVM backend) and -Xclang
  -fsanitize-coverage-trace-pc-guard. These settings are not
  automatically included with -ffuzz.
* Add `Build.Step.Compile` flag for sanitize_coverage_trace_pc_guard
  with appropriate documentation.
* Add `zig cc` integration for the respective flags.
* Avoid crashing in ELF linker code when -ffuzz -femit-llvm-ir used
  together.
2024-07-23 17:30:53 -07:00
Alex Rønne Petersen
be1e1fa180 std.Build.Step.Run: Fix invocation syntax for Wasmtime 14+.
https://github.com/bytecodealliance/wasmtime/issues/7384
2024-07-23 12:39:40 -07:00
Andrew Kelley
33c7984183 add std.testing.random_seed
closes #17609
2024-07-23 11:43:12 -07:00
Eric Joldasov
d263f1ec0e zig build: respect PKG_CONFIG environment variable
`PKG_CONFIG` environment variable is used to override path to
pkg-config executable, for example when it's name is prepended by
target triple for cross-compilation purposes:

```
PKG_CONFIG=/usr/bin/aarch64-unknown-linux-gnu-pkgconf zig build
```

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2024-07-21 02:32:33 -07:00
Andrew Kelley
eb4028bf30 add std.fmt.hex
converts an unsigned integer into an array
2024-07-20 01:06:29 -07:00
Andrew Kelley
716b128a24 frontend: add -fincremental, -fno-incremental flag
Remove --debug-incremental

This flag is also added to the build system. Importantly, this tells
Compile step whether or not to keep the compiler running between
rebuilds. It defaults off because it is currently crashing
zirUpdateRefs.
2024-07-14 21:18:09 -07:00
Andrew Kelley
abf8955951 make zig compiler processes live across rebuilds
Changes the `make` function signature to take an options struct, which
additionally includes `watch: bool`. I intentionally am not exposing
this information to configure phase logic.

Also adds global zig cache to the compiler cache prefixes.

Closes #20600
2024-07-14 19:51:16 -07:00
kcbanner
373e53d7c5 Compile: Pass the default --zig-lib-dir along to child processes
main: print the self_exe_path when `findZigLibDirFromSelfExe` fails in all cases
2024-07-13 19:04:55 -04:00
Andrew Kelley
1d20ff11d7
Merge pull request #20580 from ziglang/watch
introduce file system watching features to the zig build system
2024-07-12 16:56:17 -07:00
Andrew Kelley
4f9a8b6843 update build system unit test
need to add another field to initialize now
2024-07-12 11:00:52 -07:00
xtex
0d79aa0176
std.Build.Step.Run: support prefixed artifact args
Just like how addPrefixedFileArg and addPrefixedDirectoryArg works, we can make it for artifacts.

Signed-off-by: Bingwu Zhang <xtexchooser@duck.com>
2024-07-12 11:06:06 +02:00
Andrew Kelley
818f9cb5a0 std.Build.Step.ObjCopy: remove random bytes from cache hash
The cache hash already has the zig version in there, so it's not really
needed.
2024-07-12 00:14:08 -07:00
Andrew Kelley
2e42969786 std.Build.Step.Run: integrate with --watch 2024-07-12 00:14:08 -07:00
Andrew Kelley
6fcb1897d2 std.Build.Step.WriteFile: remove random bytes from cache hash
The cache hash already has the zig version in there, so it's not really
needed.
2024-07-12 00:14:08 -07:00
Andrew Kelley
a966eee090 std.Build.Step.WriteFile: fix handling of directories
and add file system watching integration.

`addDirectoryWatchInput` now returns a `bool` which helps remind the
caller to
1. call addDirectoryWatchInputFromPath on any derived paths
2. but only if the dependency is not already captured by a step
   dependency edge.

The make function now recursively walks all directories and adds the
found files to the cache hash rather than incorrectly only adding the
directory name to the cache hash.

closes #20571
2024-07-12 00:14:08 -07:00
Andrew Kelley
5c3fae3a32 td.Build.Step.InstallDir: leave hint for wrong cached status
Since I spent a couple minutes debugging this, hopefully this saves
someone some future trouble doing the same.
2024-07-12 00:14:08 -07:00
Andrew Kelley
7bccef3e4e std.Build.Watch: introduce special file "." to watch entire dir
And use it to implement InstallDir Step watch integration.

I'm not seeing any events triggered when I run `mkdir` in the watched
directory, however, and I have not yet figured out why.
2024-07-12 00:14:08 -07:00
Andrew Kelley
d1c14f2f52 std.Build.Step.WriteFile: extract UpdateSourceFiles
This has been planned for quite some time; this commit finally does it.

Also implements file system watching integration in the make()
implementation for UpdateSourceFiles and fixes the reporting of step
caching for both.

WriteFile does not yet have file system watching integration.
2024-07-12 00:14:08 -07:00
Andrew Kelley
0cc492a272 make more build steps integrate with the watch system 2024-07-12 00:14:08 -07:00
Andrew Kelley
bbd90a562e build runner: implement --watch (work-in-progress)
I'm still learning how the fanotify API works but I think after playing
with it in this commit, I finally know how to implement it, at least on
Linux. This commit does not accomplish the goal but I want to take the
code in a different direction and still be able to reference this point
in time by viewing a source control diff.

I think the move is going to be saving the file_handle for the parent
directory, which combined with the dirent names is how we can correlate
the events back to the Step instances that have registered file system
inputs. I predict this to be similar to implementations on other
operating systems.
2024-07-12 00:14:07 -07:00
Andrew Kelley
26d506c0f8 std.Build: remove the "push installed file" mechanism
Tracked by #14943
2024-07-12 00:14:07 -07:00
Andrew Kelley
6e025fc2e2 build system: add --watch flag and report source file in InstallFile
This direction is not quite right because it mutates shared state in a
threaded context, so the next commit will need to fix this.
2024-07-12 00:14:07 -07:00
Krzysztof Wolicki
7205756a69 Step.TranslateC: fix defineCMacro 2024-07-07 12:28:28 +02:00
Jonathan Marler
d9f1a952b8 build: fix WriteFile and addCSourceFiles not adding LazyPath deps
Adds a missing call to addLazyPathDependenciesOnly in
std.Build.Module.addCSourceFiles.  Also fixes an issue in
std.Build.Step.WriteFile where it wasn't updating all the GeneratedFile
instances for every directory.  To fix the second issue, I removed
all the GeneratedFile instances and now all files/directories reference
the steps main GeneratedFile via sub paths.
2024-07-04 13:34:17 -04:00
Sashko
604a332e21 Extract getZigArgs function in std.Build.Step.Compile 2024-07-01 16:36:13 -07:00
Michael Bradshaw
642093e04b Rename *[UI]LEB128 functions to *[UI]leb128 2024-06-23 04:30:12 +01:00
Andrew Kelley
fda2458f6a
Merge pull request #20321 from ziglang/build-system-fmt
enhance `std.Build.Step.Fmt` and use it more
2024-06-17 01:04:57 -04:00
Alex Rønne Petersen
6220c754cf std.Build: Add Step.Fail and addFail() function.
Closes #15373.
2024-06-16 23:13:12 -04:00
Andrew Kelley
eb8201afde std.Build.Step.Fmt: display non-conforming files
When in --check mode, and files are found to not conform, emit them
explicitly as step errors. Previously this stdout data was being
ignored.
2024-06-16 19:30:14 -07:00
Krzysztof Wolicki
6327a3994a
std.Build: remove deprecated b.host in favor of b.graph.host (#20237) 2024-06-08 12:22:38 -07:00
MrDmitry
84d1580873 Report error on missing values for addConfigHeader 2024-06-08 15:18:56 -04:00
wooster0
aef66eebe3 objcopy build step: don't accept multiple sections
The actual `zig objcopy` does not accept keeping multiple sections.
If you pass multiple `-j .section` arguments to `zig objcopy`, it will
only respect the last one passed.

Originally I changed `zig objcopy` to accept multiple sections
and then concatenate them instead of returning after outputting the
first section (see emitElf) but I realized concatenating probably doesn't make sense.
2024-06-05 19:39:43 -04:00