Commit graph

922 commits

Author SHA1 Message Date
Andrew Kelley
ede5dcffea make the build runner and test runner talk to each other
std.Build.addTest creates a CompileStep as before, however, this kind of
step no longer actually runs the unit tests. Instead it only compiles
it, and one must additionally create a RunStep from the CompileStep in
order to actually run the tests.

RunStep gains integration with the default test runner, which now
supports the standard --listen=- argument in order to communicate over
stdin and stdout. It also reports test statistics; how many passed,
failed, and leaked, as well as directly associating the relevant stderr
with the particular test name that failed.

This separation of CompileStep and RunStep means that
`CompileStep.Kind.test_exe` is no longer needed, and therefore has been
removed in this commit.

 * build runner: show unit test statistics in build summary
 * added Step.writeManifest since many steps want to treat it as a
   warning and emit the same message if it fails.
 * RunStep: fixed error message that prints the failed command printing
   the original argv and not the adjusted argv in case an interpreter
   was used.
 * RunStep: fixed not passing the command line arguments to the
   interpreter.
 * move src/Server.zig to std.zig.Server so that the default test runner
   can use it.
 * the simpler test runner function which is used by work-in-progress
   backends now no longer prints to stderr, which is necessary in order
   for the build runner to not print the stderr as a warning message.
2023-03-15 10:48:14 -07:00
Andrew Kelley
a333bb91ff zig objcopy: support the compiler protocol
This commit extracts out server code into src/Server.zig and uses it
both in the main CLI as well as `zig objcopy`.

std.Build.ObjCopyStep now adds `--listen=-` to the CLI for `zig objcopy`
and observes the protocol for progress and other kinds of integrations.

This fixes the last two test failures of this branch when I run
`zig build test` locally.
2023-03-15 10:48:14 -07:00
Andrew Kelley
5b90fa05a4 extract ThreadPool and WaitGroup from compiler to std lib 2023-03-15 10:48:12 -07:00
Jakub Konka
e9fc0aba4c x86_64: add missing source files to CMakeLists.txt 2023-03-12 22:08:29 +01:00
Andrew Kelley
3169f0529b eliminate posix_spawn from the standard library
Today I found out that posix_spawn is trash. It's actually implemented
on top of fork/exec inside of libc (or libSystem in the case of macOS).

So, anything posix_spawn can do, we can do better. In particular, what
we can do better is handle spawning of child processes that are
potentially foreign binaries. If you try to spawn a wasm binary, for
example, posix spawn does the following:

 * Goes ahead and creates a child process.
 * The child process writes "foo.wasm: foo.wasm: cannot execute binary file"
   to stderr (yes, it prints the filename twice).
 * The child process then exits with code 126.

This behavior is indistinguishable from the binary being successfully
spawned, and then printing to stderr, and exiting with a failure -
something that is an extremely common occurrence.

Meanwhile, using the lower level fork/exec will simply return ENOEXEC
code from the execve syscall (which is mapped to zig error.InvalidExe).

The posix_spawn behavior means the zig build runner can't tell the
difference between a failure to run a foreign binary, and a binary that
did run, but failed in some other fashion. This is unacceptable, because
attempting to excecve is the proper way to support things like Rosetta.
2023-03-10 15:41:07 -05:00
Isaac Freund
3e99afdbfe build: add -Dpie option
It is becoming increasingly common for distributions to want to enable
PIE for all binaries and zig currently does not provide any way to do
so aside from patching the build.zig.
2023-03-08 17:47:36 -05:00
Andrew Kelley
d399f8a489 Merge remote-tracking branch 'origin/master' into llvm16 2023-02-27 16:10:48 -07:00
matu3ba
97b9facb98
compiler_rt: declutter int.zig, add mulXi3 tests (#14623)
- Combine mulXi3 routines for follow-up cleanup.
- DRY up Dwords and Twords
- rename both to HalveInt and use instance
  * Justification: Not all processors have word size 32 bit.
* remove test file from CMakeLists
* DRY things.
2023-02-24 13:27:02 -05:00
Andrew Kelley
c9e02d3e69
Merge pull request #14691 from jacobly0/ctype 2023-02-22 11:06:13 -05:00
Jacob Young
3eed197c95 CBE: use stdint.h types instead of zig_ prefixes
This requires manual defines before C99 which may not have stdint.h.

Also have update-zig1 leave a copy of lib/zig.h in stage1/zig.h, which
allows lib/zig.h to be updated without needing to update zig1.wasm.
Note that since the object already existed with the exact same contents,
this completely avoids repo bloat due to zig.h changes.
2023-02-20 23:59:48 -05:00
Jacob Young
d8fada6b63 CBE: add CType interning 2023-02-20 23:48:36 -05:00
mlugg
09a84c8384
Update std.Build to new module CLI, update zig1 and CMakeLists
Resolves: #14667
2023-02-21 01:59:37 +00:00
Andrew Kelley
efdc94c107 Merge remote-tracking branch 'origin/master' into llvm16 2023-02-18 09:33:27 -07:00
Andrew Kelley
9cb52ca6ce move the cache system from compiler to std lib 2023-02-13 06:42:25 -07:00
Andrew Kelley
fab9b7110e Merge remote-tracking branch 'origin/master' into llvm16 2023-02-03 12:49:40 -07:00
Andrew Kelley
873bb29c98 introduce ZON: Zig Object Notation
* std.zig.parse is moved to std.zig.Ast.parse
 * the new function has an additional parameter that requires passing
   Mode.zig or Mode.zon
 * moved parser.zig code to Parse.zig
 * added parseZon function next to parseRoot function
2023-02-03 00:06:11 -07:00
Andrew Kelley
97b1a9bb6b cmake: no -Werror on the C++ code we copy from llvm 2023-01-26 20:13:27 -05:00
Andrew Kelley
5994a5d18d update clang and ar to llvm 16 2023-01-25 20:56:23 -07:00
Andrew Kelley
9c3bd437e6 update cmake files to LLVM 16 2023-01-25 20:26:14 -07:00
Jakub Konka
835a60a34f zld: parse, synthesise and emit unwind records 2023-01-20 18:43:16 +01:00
Jakub Konka
069fe0d63f update CMakeLists.txt with new entries 2023-01-19 00:35:33 +01:00
kcbanner
3cacbea95b build: simplify llvm-config provided system library parsing
- Revert the addition of CLANG_SYSTEM_LIBARIES and LLVM_SYSTEM_LIBRARIES
- Change addCMakeLibraryList to parse non-absolute path .lib dependencies as system libraries
2023-01-04 21:45:06 -05:00
kcbanner
f5135f8e3b cmake: fix handling of transitive zstd dependency
- Remove ZIG_ENABLE_ZSTD in favour of allowing ZIG_STATIC_ZSTD to be toggled off explicitly when ZIG_STATIC is on
- Remove ZIG_ENABLE_LIBCPP (now unused)
2023-01-04 21:45:06 -05:00
kcbanner
88e442253b cmake: add /FORCE:MULTIPLE to handle fabsl being defined in compiler_rt as well as libcmpt 2023-01-04 21:45:06 -05:00
kcbanner
bd252c58bc cmake: /O0 -> /Od 2023-01-04 21:45:06 -05:00
kcbanner
701996083a cmake: /std:c99 isn't a valid cl option 2023-01-04 21:45:06 -05:00
kcbanner
b42442f5b4 windows: fixes to support using zig cc/c++ with CMake on Windows
Using zig cc with CMake on Windows was failing during compiler
detection. -nostdinc was causing the crt not to be linked, and Coff/lld.zig
assumed that wWinMainCRTStartup would be present in this case.

-nostdlib did not prevent the default behaviour of linking libc++ when
zig c++ was used. This caused libc++ to be built when CMake ran
ABI detection using zig c++, which fails as libcxxabi cannot compile
under MSVC.

- Change the behaviour of COFF -nostdinc to set /entry to the function that the
default CRT method for the specified subsystem would have called.
- Fix -ENTRY being passed twice if it was specified explicitly and -nostdlib was present.
- Add support for /pdb, /version, /implib, and /subsystem as linker args (passed by CMake)
- Remove -Ddisable-zstd, no longer needed
- Add -Ddisable-libcpp for use when bootstrapping on msvc
2023-01-04 21:45:06 -05:00
kcbanner
0471eea0e2 build: first pass on geting stage3 building under x64_64-windows-msvc 2023-01-04 21:45:05 -05:00
kcbanner
c3945d9ede cmake: output binaries to the build directory (ie. instead of Release/Debug subfolders) 2023-01-04 21:45:05 -05:00
Michael Dusan
0cb8ed6b85
cmake: ignore case for TARGET ARCH/OS mapping
- first convert to lower-case then perform mappings
- effect: freebsd, netbsd and openbsd convert arch `amd64` to `x86_64`
- effect: convert `arm64` to `aarch64`
2023-01-02 19:18:32 -05:00
Kim SHrier
5bde627f9d map lower case amd64 to x86_64 2022-12-24 02:23:31 -05:00
Frank Denis
8d66aacb64
compiler-rt: implement __udivei4 and __umodei4 (#14023)
Allows dividing and formatting arbitrary-large unsigned integers.
2022-12-22 16:29:19 -05:00
Jakub Konka
585c21e54d macho: move parallel file hashing back to CodeSignature
I need to think some more how to calculate UUID in parallel, if
it is even possible, to preserve UUID's determinism.
2022-12-16 18:05:58 +01:00
Jakub Konka
660270b7a9 macho: calculate UUID excluding stabs and part of contributing strtab 2022-12-16 18:05:58 +01:00
Jakub Konka
79457fc76a macho: generalize parallel hasher; impl parallel MD5-like hash
By pulling out the parallel hashing setup from `CodeSignature.zig`,
we can now reuse it different places across MachO linker (for now;
I can totally see its usefulness beyond MachO, eg. in COFF or ELF too).
The parallel hasher is generic over actual hasher such as Sha256 or MD5.
The implementation is kept as it was.

For UUID calculation, depending on the linking mode:
* incremental - since it only supports debug mode, we don't bother with MD5
  hashing of the contents, and populate it with random data but only once
  per a sequence of in-place binary patches
* traditional - in debug, we use random string (for speed); in release,
  we calculate the hash, however we use LLVM/LLD's trick in that we
  calculate a series of MD5 hashes in parallel and then one an MD5 of MD5
  final hash to generate digest.
2022-12-16 00:01:04 +01:00
Jakub Konka
ec40c6b28f macho: calculate UUID as a streaming MD5 hash of the file contents 2022-12-16 00:01:04 +01:00
Jakub Konka
db2052bc35 macho: dedup LC emitting logic
Fix path written to `LC_ID_DYLIB` to include the current CWD (if any).
2022-12-16 00:01:04 +01:00
Andrew Kelley
c079d5b15d CMake: fix aarch64-windows target detection 2022-12-10 16:28:49 -07:00
Andrew Kelley
cacb962ba7 CMake: add workaround for CMAKE_AR not supporting spaces
This allows using `zig ar` for `CMAKE_AR`. Unfortunately, it requires a
patch to CMakeLists.txt and cannot be done merely with flags to the
cmake line.
2022-12-10 16:28:49 -07:00
Andrew Kelley
c51288f1f6 remove the zstd mechanism from the build process 2022-12-09 20:59:13 -07:00
h57624paen
b483c796c6 fix building wasm2c output with msvc
cmake tweaks to allow zig1 to compile successfully with msvc
2022-12-09 22:35:05 -05:00
Jacob Young
a0a2e75773 cmake: disable stack protector for zig2 to avoid link errors on windows
On windows we get:

    lld-link: error: undefined symbol: __stack_chk_fail
    >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(main)
    >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(main_main)
    >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(log_scoped_28_default_29_err__anon_2764)
    >>> referenced 36192 more times

    lld-link: error: undefined symbol: __stack_chk_guard
    >>> referenced by CMakeFiles/zig2.dir/zig2.c.obj:(.refptr.__stack_chk_guard)
    >>> referenced by CMakeFiles/zig2.dir/compiler_rt.c.obj
2022-12-06 12:27:28 -07:00
Jacob Young
98338358ed cmake: fix bootstrap dependencies 2022-12-06 12:27:28 -07:00
Andrew Kelley
eef4348953 CMake: remove --color on for compiler_rt too
Missing change from bcd4ea9b28a7560a1c1ef28c6c7258c6126c80e5
2022-12-06 12:27:28 -07:00
Andrew Kelley
aaf95ce849 CMake: adjust apple clang compiler check
Before it was checking if the compiler was AppleClang, however, this did
not handle the case when using a compiled-from-source Clang on a macOS
computer, in which case the linker is still Apple ld64, and it is in
fact the linker that needs to have the different flag to communicate
stack size.
2022-12-06 12:15:05 -07:00
Andrew Kelley
ec1334d2d4 CMake: remove --color on
This flag makes CI servers print garbage to the terminal. I started
implementing detection in wasi.c, but it would have destroyed the
beautiful code that only touches the C stdlib, does not do any
conditional compilation based on the operating system, or rely on any
POSIX functions.

So instead, let's just do without this flag to retain simplicity in this
step of the build process.
2022-12-06 12:15:05 -07:00
Jacob Young
9f4ef4de23 wasm2c: remove unnecessary brackets to reduce max bracket depth
This avoids the need to pass `-fbracket-depth=512` to clang.
2022-12-06 12:15:04 -07:00
Jacob Young
fdb98c5ce1 cmake: fix stack size linker argument on mac 2022-12-06 12:15:04 -07:00
Jacob Young
687ea31ef9 cmake: fix host target detection 2022-12-06 12:15:04 -07:00
Andrew Kelley
54b960aa4d CMake: add the bracket-depth workaround for zig1 2022-12-06 12:15:04 -07:00