Commit graph

320 commits

Author SHA1 Message Date
Veikka Tuominen
2b67f56c35 std.fs: split Dir into IterableDir
Also adds safety check for attempting to iterate directory not opened with `iterate = true`.
2022-07-15 13:04:21 +03:00
Luuk de Gram
8033767082
wasm-linker: Implement linker tests (#12006)
* test/link: initial wasm support

This adds basic parsing and dumping of wasm section so they
can be tested using the new linker-test infrastructure.

* test/link: all wasm sections parsing and dumping

We now parse and dump all sections for the wasm binary format.
Currently, this only dumps the name of a custom section.
Later this should also dump symbol table, name, linking metadata and relocations.
All of those live within the custom sections.

* Add wasm linker test

This also fixes a parser mistake in reading the flags.

* test/link: implement linker tests wasm & fixes

Adds several test cases to test the wasm self-hosted linker.
This also introduces fixes that were caught during the implementation
of those tests.

* test-runner: obey omit_stage2 for standalone

When a standalone test requires stage2, but stage2 is omit
from the compiler, such test case will not be included as part
of the test suite that is being ran. This is to support CI's
where we omit stage2 to lower the memory usage.
2022-07-12 14:36:33 +02:00
Mikko Kaihlavirta
6f55b294f6 use std.log for logging 2022-07-11 11:18:52 +03:00
Jakub Konka
20bd722464 build: handle weakly imported libs and frameworks 2022-06-28 09:19:01 +02:00
Jakub Konka
0dd28920da macho: implement and handle -needed-* and -needed_* family of flags
MachO linker now handles `-needed-l<name>`, `-needed_library=<name>`
and `-needed_framework=<name>`. While on macOS `-l` is equivalent
to `-needed-l`, and `-framework` to `-needed_framework`, it can be
used to the same effect as on Linux if combined with `-dead_strip_dylibs`.

This commit also adds handling for `-needed_library` which is macOS
specific flag only (in addition to `-needed-l`).

Finally, in order to leverage new linker testing harness, this commit
added ability to specify lowering to those flags via `build.zig`:
`linkSystemLibraryNeeded` (and related), and `linkFrameworkNeeded`.
2022-06-27 19:53:38 +02:00
Jakub Konka
efc5c97bff macho: implement -dead_strip_dylibs linker flag 2022-06-27 19:53:38 +02:00
Jakub Konka
589bf67635 macho: implement -headerpad_max_install_names 2022-06-25 18:04:40 +02:00
Jakub Konka
8c1feef4cd macho: implement -headerpad_size option
Includes both traditiona and incremental codepaths with one caveat that
in incremental case, the requested size cannot be smaller than the
default padding size due to prealloc required due to incremental nature
of linking.

Also parse `-headerpad_max_install_names`, however, not actionable just yet -
missing implementation.
2022-06-25 17:59:08 +02:00
Jakub Konka
0df7ed79d3 macho: implement -search_dylibs_first linker option 2022-06-24 20:25:16 +02:00
Jakub Konka
23a63f4ce4 link-tests: rename CheckMachOStep to CheckObjectStep and accept obj format 2022-06-22 10:27:51 +02:00
Jakub Konka
5fbdfb3f34 link-tests: add CheckMachOStep
CheckMachOStep specialises CheckFileStep into directed (surgical)
MachO file fuzzy searches. This will be the building block for
comprehensive MachO linker tests.
2022-06-21 15:44:24 +02:00
Jakub Konka
2d09540a63 link-tests: test pagezero_size option for macho 2022-06-20 18:25:20 +02:00
Isaac Freund
d8cae4d197
std.build: Expose -z norelro and -z lazy 2022-06-08 15:13:04 +02:00
Andrew Kelley
59219e7e91 stage2: add support for -fbuild-id,-fno-build-id
closes #3047
2022-05-31 13:36:33 -07:00
Hanna
4e918873e7
Rename std.build.Pkg.path to std.build.Pkg.source (#11557) 2022-05-26 19:32:28 -04:00
Jonathan Marler
9e89000ffc Update usages of process.getEnvMap and change BufMap -> EnvMap where applicable
# Conflicts:
#	lib/std/build/RunStep.zig
2022-05-11 18:40:53 -06:00
Evan Haas
94b9bcd034 stdlib: escape backslashes and double quotes in Builder response file
Fixes #11595
2022-05-09 18:42:42 +03:00
Andrew Kelley
e8c85450fe
Merge pull request #11592 from ziglang/stage3-macos-linker
Sema: solve a false positive "depends on itself"
2022-05-07 04:38:19 -04:00
Andrew Kelley
ec95e00e28 flatten lib/std/special and improve "pkg inside another" logic
stage2: change logic for detecting whether the main package is inside
the std package. Previously it relied on realpath() which is not portable.
This uses resolve() which is how imports already work.

 * stage2: fix cleanup bug when creating Module
 * flatten lib/std/special/* to lib/*
   - this was motivated by making main_pkg_is_inside_std false for
     compiler_rt & friends.
 * rename "mini libc" to "universal libc"
2022-05-06 22:41:00 -07:00
Andrew Kelley
496eb69273 CI: add non-LLVM backends to the test matrix
We can't yet run the behavior tests with stage3, but at least we can run
them with stage2, and we can use the proper test matrix.

This commit also adds use_llvm and ofmt to the zig build system.
2022-05-05 22:29:25 -07:00
Jimmi Holst Christensen
a0a2ce92ca std: Do not allocate the result for ChildProcess.init
Instead, just return ChildProcess directly. This structure does not
require a stable address, so we can put it on the stack just fine. If
someone wants it on the heap they should do.

  const proc = try allocator.create(ChildProcess);
  proc.* = ChildProcess.init(args, allocator);
2022-04-29 22:50:34 -04:00
Andrew Kelley
c5e847744c Revert "Merge pull request #11214 from iddev5/ay-build-runner"
This reverts commit 75c9936737, reversing
changes made to 7f13f5cd5f.

I don't think `runZigBuild` belongs in std.testing. We already have
`test/standalone/*` for this.

Additionally test names should explain what they are testing rather than
referencing GitHub issue numbers.
2022-04-28 14:12:32 -07:00
Andrew Kelley
9181c98225 std.build: fix dll-export-fns API
We have a pattern using `?bool` for the -f/-fno- style flags.

Fixup for 8e3add8736.
2022-04-28 11:13:33 -07:00
Veikka Tuominen
75c9936737
Merge pull request #11214 from iddev5/ay-build-runner
std: explicitly handle error.UnexpectedExitCode in build_runner
2022-04-28 18:38:44 +03:00
Erik Hugne
8e3add8736
std.build: make no_dll_export_fns accessible in build step 2022-04-28 18:34:15 +03:00
Andrew Kelley
0c5ad335d2 build system: add -fstage1/-fno-stage1 to zig build
So that people can start experimenting with compiling their projects
with the self-hosted compiler.

I expect this commit to be reverted after #89 is closed.
2022-04-19 14:40:27 -07:00
iddev5
d9f9948b65 std.build: add test for issue 10381 2022-04-15 17:44:34 +05:30
Isaac Freund
38d6e1d8a8 std.build: Fix transitive linkSystemLibraryName() dependencies
Currently transitive system library dependencies are always linked using
linkSystemLibrary() and therefore pkg-config even if they were
originally specified with linkSystemLibraryName() instead. This causes
problems in practice for projects needing total control over exactly
what library is linked, such as the mach game engine.

This is fixed by keeping track of whether libraries are to be linked
with pkg-config or not and holding off on actually running pkg-config
until after transitive dependency resolution in LibExeObjStep.make().

This also fixes a separate issue with the pkg-config handling that could
cause partial application of pkg-config flags if the first part of the
pkg-config output parses correctly but there is an error later on. This
error isn't always fatal as we fall back to a plain -lfoo in the case of
linkSystemLibrary().
2022-04-12 06:12:44 -04:00
Jakub Konka
0376fd09bc macho: extend CodeSignature to accept entitlements
With this change, we can now bake in entitlements into the binary.
Additionally, I see this as the first step towards full code signature
support which includes baking in Apple issued certificates for
redistribution, etc.
2022-03-22 07:06:39 +01:00
Andrew Kelley
0fb005d1d0 Sema: dummy implementation of @errorReturnTrace
Also update std/build.zig to use stage2 function pointer semantics.

This gets us a little bit closer to `zig build` working, although it is
now hitting a new crash in the compiler.
2022-03-21 19:08:30 -07:00
Jakub Konka
dd55b72949 std: introduce posix_spawn as an alt to fork-exec
Currently, the new API will only be available on macOS with
the intention of adding more POSIX systems to it incrementally
(such as Linux, etc.).

Changes:
* add `posix_spawn` wrappers in a separate container in
  `os/posix_spawn.zig`
* rewrite `ChildProcess.spawnPosix` using `posix_spawn` targeting macOS
  as `ChildProcess.spawnMacos`
* introduce a `posix_spawn` specific `std.c.waitpid` wrapper which
  does return an error in case the child process failed to exec - this
  is required for any process that was spawned using `posix_spawn`
  mechanism as, by definition, the errors returned by `posix_spawn`
  routine cover only the `fork`-equivalent; `pre-exec()` and `exec()`
  steps are covered by a catch-all error `ECHILD` returned by `waitpid`
  on unsuccessful execution, e.g., no such file error, etc.
2022-03-16 19:40:44 +01:00
Stephen Gutekanst
cfb4f48941 Revert "Revert "Merge pull request #10950 from hexops/sg/responsefiles""
This reverts commit 5ab5e2e673.
2022-03-05 16:04:21 -07:00
Andrew Kelley
5ab5e2e673 Revert "Merge pull request #10950 from hexops/sg/responsefiles"
This reverts commit 136a43934b, reversing
changes made to 9dd839b7ed.

This broke the behavior of `zig run`.
2022-02-24 12:11:11 -07:00
Jakub Konka
136a43934b
Merge pull request #10950 from hexops/sg/responsefiles
Do not fail to build if 'zig build-lib' etc. arguments exceed OS limits
2022-02-23 22:54:25 +01:00
xReveres
b2805666a7 stage1-wasm: implement shared memory 2022-02-23 08:57:20 +01:00
Stephen Gutekanst
efd473bbfc std: Builder: use response files for zig test invocations too
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
2022-02-21 13:06:23 -07:00
Stephen Gutekanst
9c1c4747f4 std: Builder: account for null term considering response files
Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
2022-02-21 12:58:13 -07:00
Stephen Gutekanst
2bd10f4db9 std: have Builder use response files if zig build args exceed OS limits
In Mach engine we're seeing command line arguments to `zig build-lib`
exceed the 32 KiB limit that Windows imposes, due to the number of
sources and compiler flags we must pass in order to build gpu-dawn.

This change fixes the issue by having `Builder` check if the arguments
to a `zig build-*` command are >30 KiB and, if so, writes the arguments
to a file `zig-cache/args/<SHA2 of args>`. Then the command invocation
merely becomes `zig build-lib @<that file>`.

Fixes #10693

Fixes hexops/mach#167

Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
2022-02-20 14:44:39 -07:00
Andrew Kelley
5aa35f62c3 Revert "reduce build error noise"
This reverts commit baead472d7.

Let's go through the proposal process on this one. I want to push back
on this. My position is that, at the very least, a full trace of command
lines of sub-processes should be printed on failure, with the exception
of opt-in flags such as `--prominent-compile-errors`.
2022-02-17 14:30:12 -07:00
Jonathan Marler
baead472d7 reduce build error noise
Address https://github.com/ziglang/zig/issues/3477
This provides a mechanism for builds to fully report an error to the user and prevent zig from piling on extra noise.
2022-02-17 14:48:22 +02:00
Cody Tapscott
5065830aa0 Avoid depending on child process execution when not supported by host OS
In accordance with the requesting issue (#10750):
- `zig test` skips any tests that it cannot spawn, returning success
- `zig run` and `zig build` exit with failure, reporting the command the cannot be run
- `zig clang`, `zig ar`, etc. already punt directly to the appropriate clang/lld main(), even before this change
- Native `libc` Detection is not supported

Additionally, `exec()` and related Builder functions error at run-time, reporting the command that cannot be run
2022-02-06 22:21:46 -07:00
Johannes Löthberg
6f87f49f3d CLI: remove remainders of --verbose-ast and --verbose-tokenize
These options were removed in 5e63baae8 (CLI: remove --verbose-ast and
--verbose-tokenize, 2021-06-09) but some remainders were left in.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
2022-02-06 01:57:04 -05:00
praschke
f2a82bafae std: allow tests to use cache and setOutputDir 2022-02-05 16:33:57 +02:00
rohlem
fbc06f9c91 std.build.TranslateCStep: add C macro support
The string construction code is moved out of std.build.LibExeObjStep
into std.build.constructCMacroArg, to allow reusing it elsewhere.
2022-02-05 03:17:07 -05:00
Jonathan Marler
3f341bdc26
Normalize some build function names
An attempt to normalize some of the function names in build.zig.  Normalize add*Dir to add*Path.  Also use "Library" instead of the "Lib" abbreviation.

The PR does not remove the old names, only adds the new normalized ones to faciliate a transition period.
2022-01-24 20:15:32 +02:00
Andrew Kelley
fd6d1fe015 stage2: improvements to entry point handling
* rename `entry` to `entry_symbol_name` for the zig build API
 * integrate with `zig cc` command line options
 * integrate with COFF linking with LLD
 * integrate with self-hosted ELF linker
 * don't put it in the hash for MachO since it is ignored
2022-01-19 11:41:08 -07:00
Kenta Iwasaki
5ae3e4e9bd lld: allow for entrypoint symbol name to be set
This commit enables for the entrypoint symbol to be set when linking ELF
or WebAssembly modules with lld using the Zig compiler.
2022-01-19 11:22:10 -07:00
Jimmi Holst Christensen
349a7cc272 Make prominent_compile_errors also work for test steps 2022-01-12 00:13:30 -05:00
Luuk de Gram
247b638ccf
Pass --allow-unknown-exports to wasmtime
Also skip exporting non-function symbols when we're building a WASI command using the stage2 llvm backend.
2022-01-07 18:35:08 +01:00
Luuk de Gram
4cb2f11693 wasm-linker: Implement the --export-table and --import-table flags.
This implements the flags for both the linker frontend as well as the self-hosted linker.

Closes #5790
2021-12-21 12:38:50 -08:00