Commit graph

418 commits

Author SHA1 Message Date
Andrew Kelley
9501ea688f stage2: --autofix proof-of-concept
* Introduce the `--autofix` CLI flag when building an executable,
   object, or library.
 * Refactor std.zig.render to use a struct parameter to make it easier
   to add/remove fields from the struct.
 * Introduce a "Fixups" concept to std.zig.render which can perform
   edits while rendering code, while leaving the Ast read-only.
 * Add a fixup for adding a discard after a variable declaration.
 * Update the Module code to check for fixable errors after AstGen
   lowering.

Improvements that need to be made before this can be merged:

 * Introduce an error for "pointless discard" and a fixup for it so that
   --autofix can undo the effects of itself when a variable becomes
   used.
 * Support local variables as well as local constants.
 * Support captures in addition to local variables and constants.
 * Integrate properly with incremental compilation.
 * Integrate with the Zig build system.
 * Distinguish between AstGen errors that can be autofixed and those
   that cannot.
 * Remove std.debug.print statements.
 * Only perform fixups when all errors are autofixable errors. However,
   suppress all autofixable errors when reporting errors with --autofix.

The purpose of this feature is to satisfy two use cases that
traditionally have been at odds:

 * Some people want the guarantee that all Zig code they read has the
   property that there are no unused locals.
 * Some people find no value from such errors and want to not have to
   deal with them.

The problem with an `--allow-unused` flag is that people from the second
group will use it, and then some projects will fail to compile without
the flag enabled.

I like to think of Zig as having "inline warnings". The warnings are
there, inside your source code, next to the relevant lines, ready to be
noticed by diffs, code reviews, and when refactoring.

`--autofix` is a way for Zig to automatically insert inline warnings for
those who wish to iterate quickly on a messy codebase.
2022-09-09 18:49:05 -07:00
Andrew Kelley
3ee01c14ee std.zig.system.NativeTargetInfo: detection ignores self exe
Before, native glibc and dynamic linker detection attempted to use the
executable's own binary if it was dynamically linked to answer both the
C ABI question and the dynamic linker question. However, this could be
problematic on a system that uses a RUNPATH for the compiler binary,
locking it to an older glibc version, while system binaries such as
/usr/bin/env use a newer glibc version. The problem is that libc.so.6
glibc version will match that of the system while the dynamic linker
will match that of the compiler binary. Executables with these versions
mismatching will fail to run.

Therefore, this commit changes the logic to be the same regardless of
whether the compiler binary is dynamically or statically linked. It
inspects `/usr/bin/env` as an ELF file to find the answer to these
questions, or if there is a shebang line, then it chases the referenced
file recursively. If that does not provide the answer, then the function
falls back to defaults.

This commit also solves a TODO to remove an Allocator parameter to the
detect() function.
2022-09-08 20:52:49 -07:00
Motiejus Jakštys
a833bdcd7e [ld] add --print-* for diagnostics
This adds the following for passthrough to lld:
- `--print-gc-sections`
- `--print-icf-sections`
- `--print-map`

I am not adding these to the cache manifest, since it does not change
the produced artifacts.

Tested with an example from #11398: it successfully prints the resulting
map and the GC'd sections.
2022-09-08 22:30:32 -04:00
riChar
f7784a081f
stage2: fix panic when the dependency is missing 2022-09-02 12:53:48 +03:00
Andrew Kelley
6f9b7c8cee make 'zig build' respect ZIG_LIB_DIR
follow-up to 56bdd0127f
2022-08-29 23:35:07 -07:00
Meghan
56bdd0127f implement cache environment variables for zig build 2022-08-29 22:26:41 -04:00
Andrew Kelley
507aae4a1a make self-hosted the default compiler
stage1 is available behind the -fstage1 flag.

closes #89
2022-08-19 16:45:15 -07:00
Andrew Kelley
c0b7f20893 stage2: implement stack protectors
This is one of the final remaining TODOs for the LLVM backend.
2022-08-19 03:41:13 -07:00
Andrew Kelley
b975f7a56f std.Target gains ObjectFormat field 2022-08-18 18:58:28 -07:00
Andrew Kelley
a82d7c2063 std: add missing error to windows.WriteFile
I encountered this error today when testing the self-hosted compiler on
Windows.
2022-08-11 06:22:05 -04:00
Andrew Kelley
e8d02c5b90 add ZIG_GLOBAL_CACHE_DIR env var 2022-08-10 16:43:25 -07:00
Jakub Konka
a0ec07fdec cc: add support for -M flag 2022-08-08 14:26:10 +02:00
Andrew Kelley
3cf8f283d3
Merge pull request #12085 from topolarity/dyn-link-libcpp
Dynamically link `libc++` if integrating with system LLVM
2022-07-29 14:18:32 -07:00
Cody Tapscott
b0525344a2 Add check to verify libc++ is shared by LLVM/Clang
This check is needed because if static/dynamic linking is mixed incorrectly,
it's possible for Clang and LLVM to end up with duplicate "copies" of libc++.

This is not benign: Static variables are not shared, so equality comparisons
that depend on pointers to static variables will fail. One such failure is
std::generic_category(), which causes POSIX error codes to compare as unequal
when passed between LLVM and Clang.

I believe this is the cause of https://github.com/ziglang/zig/issues/11168

In order to avoid affecting build times when Zig is repeatedly invoked,
we only enable this check for "zig env" and "zig version"
2022-07-28 22:21:24 -07:00
Veikka Tuominen
ece1d1daf4 CLI: add error for duplicate package 2022-07-28 15:20:01 -07:00
r00ster91
baafb8a491 std.fmt: add more invalid format string errors 2022-07-27 18:07:53 +03:00
InKryption
a0d3a87ce1 std.fmt: require specifier for unwrapping ?T and E!T 2022-07-26 11:25:49 -07:00
Andrew Kelley
934573fc5d Revert "std.fmt: require specifier for unwrapping ?T and E!T."
This reverts commit 7cbd586ace.

This is causing a fail to build from source:

```
./lib/std/fmt.zig:492:17: error: cannot format optional without a specifier (i.e. {?} or {any})
                @compileError("cannot format optional without a specifier (i.e. {?} or {any})");
                ^
./src/link/MachO/Atom.zig:544:26: note: called from here
                log.debug("  RELA({s}) @ {x} => %{d} in object({d})", .{
                         ^
```

I looked at the code to fix it but none of those args are optionals.
2022-07-24 11:50:10 -07:00
InKryption
7cbd586ace
std.fmt: require specifier for unwrapping ?T and E!T.
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-07-24 12:01:56 +03:00
Jakub Konka
7ec9a4f382 cli: support --gc-sections and --no-gc-sections for Zig sources 2022-07-22 16:58:20 +02:00
Jakub Konka
44ee42c6bc cli: parse -dead_strip MachO linker flag 2022-07-22 16:58:20 +02:00
Veikka Tuominen
b2486fbc5e
Merge pull request #12121 from Vexu/span
Stage2 point to error location using spans
2022-07-16 12:22:53 +03:00
Veikka Tuominen
da94227f78
Merge pull request #12060 from Vexu/IterableDir
std.fs: split `Dir` into `IterableDir`
2022-07-16 12:18:21 +03:00
Luuk de Gram
6ba2fb3db7 zig cc: support -Wl,--export=<symbol> 2022-07-15 16:59:23 +02:00
Veikka Tuominen
1463144fc8 Compilation: point caret in error message at the main token 2022-07-15 15:11:43 +03:00
Veikka Tuominen
2b67f56c35 std.fs: split Dir into IterableDir
Also adds safety check for attempting to iterate directory not opened with `iterate = true`.
2022-07-15 13:04:21 +03:00
Veikka Tuominen
b5a838247b stage2: point to error location using spans 2022-07-14 22:18:58 +03:00
Veikka Tuominen
2a3f3766a4 stage2: handle parser notes in a more general way 2022-07-12 12:50:59 +03:00
Veikka Tuominen
5d22204d2d parser: add helpful error for C style container declarations
```zig
// a.zig
struct Foo {
    a: u32,
};
```

before:
```
a.zig:1:1: error: expected test, comptime, var decl, or container field, found 'struct'
struct Foo {
^
```
after:
```
a.zig:1:8: error: 'struct Foo' is invalid
struct Foo {
       ^
a.zig:1:8: note: to declare a container do 'const Foo = struct'
struct Foo {
       ^
```
2022-07-12 12:50:59 +03:00
Andrew Kelley
2b99182e25 stage2: cleanups to --compress-debug-sections
* make the setting in the linker backend be non-optional; by this time
   all defaults are supposed to be resolved.
 * integrate with `zig cc`
 * change the CLI parsing to match C compiler parsing, allowing
   `--compress-debug-sections` alone to choose a default encoding of
   zlib.
2022-07-11 14:18:56 -07:00
Motiejus Jakštys
1f410b500c ELF: understand -Wl,--compress-debug-sections
This argument is both a compiler and a linker flag. The linker flag was
not understood; now it is. Go likes to use it as a linker flag.

Tested with sqlite3. The size difference is significant, and I confirmed
gdb understands both binaries.

zlib: 3.66MB
------------

    CC="zig cc"
    CFLAGS="-Wl,--compress-debug-sections=zlib -O2"
    ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      39.1%  1.43Mi  88.4%  1.43Mi    .text
      19.6%   734Ki   0.0%       0    .debug_info
      16.4%   613Ki   0.0%       0    .debug_loc
      13.1%   492Ki   0.0%       0    .debug_line
       4.2%   157Ki   9.5%   157Ki    .rodata
       2.3%  87.6Ki   0.0%       0    .debug_ranges
       1.5%  56.2Ki   0.0%       0    .symtab
       1.1%  40.2Ki   0.0%       0    .strtab
       1.0%  38.2Ki   0.0%       0    .debug_str
       0.7%  26.2Ki   0.0%       0    .debug_frame
       0.4%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.1%  3.65Ki   0.2%  3.26Ki    [16 Others]
       0.1%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.1%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.17Ki   0.0%       0    .debug_abbrev
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  3.66Mi 100.0%  1.62Mi    TOTAL

none: 8.56MB
------------

    CC="zig cc" CFLAGS="-O2" ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      41.1%  3.52Mi   0.0%       0    .debug_loc
      18.5%  1.59Mi   0.0%       0    .debug_info
      16.7%  1.43Mi  88.4%  1.43Mi    .text
      11.8%  1.01Mi   0.0%       0    .debug_line
       5.9%   515Ki   0.0%       0    .debug_ranges
       1.8%   157Ki   9.5%   157Ki    .rodata
       1.3%   118Ki   0.0%       0    .debug_frame
       1.3%   110Ki   0.0%       0    .debug_str
       0.6%  56.2Ki   0.0%       0    .symtab
       0.5%  40.2Ki   0.0%       0    .strtab
       0.2%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.0%  3.64Ki   0.2%  3.26Ki    [16 Others]
       0.0%  2.98Ki   0.0%       0    .debug_abbrev
       0.0%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.0%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  8.56Mi 100.0%  1.62Mi    TOTAL
2022-07-11 13:55:29 -07:00
Andrew Kelley
2ee864ca5e CLI: add support for -fno-builtin 2022-07-05 15:21:20 -07:00
Andrew Kelley
3ab5912eee LLD: the driver functions now return "false" to mean "error" 2022-07-01 22:13:30 -07:00
Andrew Kelley
1fa4a58d2c stage1: update to LLVM 14 API 2022-07-01 19:19:13 -07:00
Jakub Konka
7cc4176448 clang: add Zig equivalent for -headerpad_max_install_names cli flag 2022-06-29 17:21:32 +02:00
Jakub Konka
c2c1998269 clang: update cmdline options to include weak libs and frameworks
Clang accepts `-weak-lx`, `-weak_library x` and `-weak_framework x`.
2022-06-28 21:16:23 +02:00
Jakub Konka
f353b59efa macho: discriminate between normal and weak dylibs
Parse `-weak-lx` and `-weak_framework x` in the CLI.
2022-06-28 09:18:54 +02:00
Jakub Konka
1188415f4c cli: typo --needed_library should be -needed_library on macos 2022-06-28 07:30:13 +02:00
Jakub Konka
0dd28920da macho: implement and handle -needed-* and -needed_* family of flags
MachO linker now handles `-needed-l<name>`, `-needed_library=<name>`
and `-needed_framework=<name>`. While on macOS `-l` is equivalent
to `-needed-l`, and `-framework` to `-needed_framework`, it can be
used to the same effect as on Linux if combined with `-dead_strip_dylibs`.

This commit also adds handling for `-needed_library` which is macOS
specific flag only (in addition to `-needed-l`).

Finally, in order to leverage new linker testing harness, this commit
added ability to specify lowering to those flags via `build.zig`:
`linkSystemLibraryNeeded` (and related), and `linkFrameworkNeeded`.
2022-06-27 19:53:38 +02:00
Jakub Konka
efc5c97bff macho: implement -dead_strip_dylibs linker flag 2022-06-27 19:53:38 +02:00
Jakub Konka
589bf67635 macho: implement -headerpad_max_install_names 2022-06-25 18:04:40 +02:00
Jakub Konka
8c1feef4cd macho: implement -headerpad_size option
Includes both traditiona and incremental codepaths with one caveat that
in incremental case, the requested size cannot be smaller than the
default padding size due to prealloc required due to incremental nature
of linking.

Also parse `-headerpad_max_install_names`, however, not actionable just yet -
missing implementation.
2022-06-25 17:59:08 +02:00
Jakub Konka
eadac47631 cli: let the linker resolve libs into dynamic and static
Unlike targeting ELF-based OSes such as Linux, resolving system libs
on Darwin should follow one of two strategies: `-search_paths_first`
or `-search_dylibs_first` and hence we defer always forcing linking
a static library to the linker.
2022-06-24 22:02:19 +02:00
Jakub Konka
0df7ed79d3 macho: implement -search_dylibs_first linker option 2022-06-24 20:25:16 +02:00
Motiejus Jakštys
d589047e80 zld: ignore -search_paths_first
Ignore MachO-specific flag -search_paths_first, since it is the default
in zld and ld64.

Also see Jakub's comment[1]:

    Changing topic slightly, @motiejus dunno if you noticed, with this
    change building arm64 Zig binary fails on macos - it complains about
    unknown -search_paths_first flag. This one is an easy fix: we should
    ignore it since this is the default behaviour in both ld64 and zld.

[1]: https://github.com/ziglang/zig/pull/11906#issuecomment-1163545849
2022-06-24 19:03:15 +02:00
Jakub Konka
8752db3285 macho: -pagezero_size is always in hex
This matches the behavior of other linkers out there including
`ld64` and `lld`.
2022-06-20 14:26:59 +02:00
Jakub Konka
ea9b7a0626 macho: round down pagezero size to page size
If page aligned requested pagezero size is 0, skip generating
__PAGEZERO segment.

Add misc improvements to the pipeline, and correctly transfer the
requested __PAGEZERO size to the linker.
2022-06-20 13:39:33 +02:00
Motiejus Jakštys
98138ba78c [MachO] add -pagezero_size
Pass `-pagezero_size` to the MachO linker. This is the final
"unsupported linker arg" that I could chase that CGo uses. After this
and #11874 we may be able to fail on an "unsupported linker arg" instead
of emiting a warning.

Test case:

    zig=/code/zig/build/zig
    CGO_ENABLED=1 GOOS=darwin GOARCH=amd64 CC="$zig cc -target x86_64-macos" CXX="$zig c++ -target x86_64-macos" go build -a -ldflags "-s -w" cgo.go

I compiled a trivial CGo program and executed it on an amd64 Darwin
host.

To be honest, I am not entirely sure what this is doing. This feels
right after reading what this argument does in LLVM sources, but I am by
no means qualified to make MachO pull requests. Will take feedback.
2022-06-20 13:39:33 +02:00
Motiejus Jakštys
d506275a06 [elf] understand -no-pie
This passes -Wl,-no-pie linker arg. Golang uses that. From the `ld(1)`
man page:

   Create a position dependent executable.  This is the default.

Not adding to the help text, because this is the default.
2022-06-16 15:27:30 -04:00
Isaac Freund
5816d3eaec linker: remove -z noexecstack option
Note that the current documentation for the `-z noexecstack` is
incorrect. This indicates that an object *does not* require an
executable stack.

This is actually the default of LLD, and there has never been a way to
override this default by passing `-z execstack` to LLD.

This commit removes the redundant `-z noexecstack` option from
zig build-exe/build-lib/build-obj and ignores the option if passed
to zig cc for compatibility.

As far as I can tell, there is no reason for code to require an
executable stack. This option only exists because the stack was
originally executable by default and some programs came to depend
on that behavior. Instead, mprotect(2) may be used to make memory
pages executable.
2022-06-09 18:51:43 -04:00