Commit graph

28475 commits

Author SHA1 Message Date
Luuk de Gram
0a030d6598
wasm: Use File.Index for symbol locations
Rather than using the optional, we now directly use `File.Index` which
can already represent an unknown file due to its `.null` value. This
means we do not pay for the memory cost.

This type of index is now used for:
- SymbolLoc
- Key of the functions map
- InitFunc

Now we can simply pass things like atom.file, object.file, loc.file etc
whenever we need to access its representing object file which makes it
a lot easier.
2024-02-29 15:23:03 +01:00
Luuk de Gram
94f3a18c88
wasm: Add File abstraction 2024-02-29 15:23:03 +01:00
Luuk de Gram
cbc8d33062
wasm: fix symbol resolution and atom processing 2024-02-29 15:23:03 +01:00
Luuk de Gram
143e9599d6
wasm: use File abstraction instead of object
When merging sections we now make use of the `File` abstraction so all
objects such as globals, functions, imports, etc are also merged from
the `ZigObject` module. This allows us to use a singular way to perform
each link action without having to check the kind of the file.
The logic is mostly handled in the abstract file module, unless its
complexity warrants the handling within the corresponding module itself.
2024-02-29 15:23:03 +01:00
Luuk de Gram
12505c6d3d
wasm: store File.Index on the Atom
Also, consolidate the creation of Atoms so they all use `createAtom`.
2024-02-29 15:23:03 +01:00
Luuk de Gram
f6896ef218
wasm: create linking objects in correct module
CodeGen will create linking objects such as symbols, function types, etc
in ZigObject, rather than in the linker driver where the final result
will be stored. They will end up in the linker driver module during
the `flush` phase instead.

This must mean we must call functions such as `addOrGetFuncType` in the
correct namespace or else it will be created in the incorrect list and
therefore return incorrect indexes.
2024-02-29 15:23:03 +01:00
Luuk de Gram
9b3c8fd3a8
wasm: initialize a ZigObject when required
When we have a ZigCompileUnit and don't use LLVM, we initialize the
ZigObject which will encapsulate the Zig Module as an object file in-
memory. During initialization we also create symbols which the object
will need such as the stack pointer.
2024-02-29 15:23:02 +01:00
Luuk de Gram
e54177e852
wasm: move incremental Dwarf info into ZigObject 2024-02-29 15:23:02 +01:00
Luuk de Gram
ba0e84a411
wasm: move Zig module-linkage to ZigObject
Rather than specializing the linker-driver to be able to handle objects
generated by a ZCU, we store all data in-memory in ZigObject. ZigObject
acts more like a regular object file which will allow us to treat it
as us. This will make linking much more simple, but will also reduce
the complexity of incremental-linking as we can simply update ZigObject
and relink it.
2024-02-29 15:22:58 +01:00
Andrew Kelley
5c0766b6c8
Merge pull request #19127 from ziglang/export-symbol-names-cache-hash
add missing export symbol names to whole mode wasm cache hash
2024-02-29 02:09:17 -08:00
Andrew Kelley
3e292e5fc1
Merge pull request #19128 from ziglang/tar
std.tar: avoid dependency on file system
2024-02-29 00:30:34 -08:00
Andrew Kelley
83d2eef3e0 build.zig: fix typo in extension exclusion list
we've been shipping these test files for a while now. oops.
2024-02-28 22:37:11 -07:00
Andrew Kelley
f7d095d372 use hash.addListOfBytes where applicable 2024-02-28 20:38:24 -07:00
Andrew Kelley
30688f33a4 add missing export symbol names to whole mode wasm cache hash
Fixes false positive cache hit.
2024-02-28 20:38:24 -07:00
Andrew Kelley
beca85e644 std.tar.iterator: make the buffers configurable
Provides more API flexibility and correctness, while still preserving
the handy high level `pipeToFileSystem` API.
2024-02-28 20:33:29 -07:00
Andrew Kelley
b3ad45f267 std.tar: avoid dependency on file system
In the iterator function which is the low-level API, don't depend on
`std.fs.MAX_PATH_BYTES` because this is not defined on all operating
systems, such as freestanding.

However in such environments it still makes sense to be able to extract
from a tar file.

An even more flexible solution would be to accept the buffers as
arguments to iterator() which I think is a good idea, but for now let's
just set the same upper limmit across all operating systems.
2024-02-28 20:14:21 -07:00
Andrew Kelley
f5aad47287
Merge pull request #19122 from ziglang/lazy-aro
make aro-based translate-c lazily built from source
2024-02-28 18:21:30 -08:00
Jacob Young
69df00f657
Merge pull request #19115 from antlilja/llvm-misc
LLVM: Miscellaneous improvements to Builder
2024-02-29 01:03:58 +01:00
Andrew Kelley
49437d34e6 zig fmt: off in the generated files 2024-02-28 13:21:05 -07:00
Andrew Kelley
240d0b68f6 make aro-based translate-c lazily built from source
Part of #19063.

Primarily, this moves Aro from deps/ to lib/compiler/ so that it can be
lazily compiled from source. src/aro_translate_c.zig is moved to
lib/compiler/aro_translate_c.zig and some of Zig CLI logic moved to a
main() function there.

aro_translate_c.zig becomes the "common" import for clang-based
translate-c.

Not all of the compiler was able to be detangled from Aro, however, so
it still, for now, remains being compiled with the main compiler
sources due to the clang-based translate-c depending on it. Once
aro-based translate-c achieves feature parity with the clang-based
translate-c implementation, the clang-based one can be removed from Zig.

Aro made it unnecessarily difficult to depend on with these .def files
and all these Zig module requirements. I looked at the .def files and
made these observations:

- The canonical source is llvm .def files.
- Therefore there is an update process to sync with llvm that involves
  regenerating the .def files in Aro.
- Therefore you might as well just regenerate the .zig files directly
  and check those into Aro.
- Also with a small amount of tinkering, the file size on disk of these
  generated .zig files can be made many times smaller, without
  compromising type safety in the usage of the data.

This would make things much easier on Zig as downstream project,
particularly we could remove those pesky stubs when bootstrapping.

I have gone ahead with these changes since they unblock me and I will
have a chat with Vexu to see what he thinks.
2024-02-28 13:21:05 -07:00
Andrew Kelley
9410b11ca6
Merge pull request #19114 from ziglang/lazy-resinator
move `zig libc` command to be lazily built
2024-02-28 11:34:45 -08:00
Andrew Kelley
a1b083b666
Merge pull request #19120 from jacobly0/os-cleanup
posix: fix socket fd leak
2024-02-28 05:51:15 -08:00
antlilja
5c2e463ecd LLVM: Don't create a debug lexical block when inlining 2024-02-28 14:46:43 +01:00
antlilja
40f99862e1 Builder: Implement StrtabString and use it for Global names 2024-02-28 14:46:43 +01:00
antlilja
826c6c0ec6 LLVM: Implement more efficient blob writing 2024-02-28 14:46:43 +01:00
Andrew Kelley
dff45c266e wasi.c: report no environment variables available
I moved part of the compiler that checks environment variables to the
standard library, so it doesn't have access to `build_options.only_c`
anymore, which means some environment variable checks are leaking into
the zig1.wasm build of zig. This logic still makes it return "no
environment variables found" though.
2024-02-27 22:55:00 -07:00
Andrew Kelley
420b7713d4 CLI: strip lazy built commands by default
Saves a lot of time since we use -OReleaseFast.
Disabled when ZIG_DEBUG_CMD=1 is passed.
2024-02-27 22:55:00 -07:00
Andrew Kelley
a4380a30f5 move zig libc command to be lazily built
part of #19063

This is a prerequisite for doing the same for Resinator.
2024-02-27 22:55:00 -07:00
IntegratedQuantum
6e078883ee Expand the memcpy fast path in flate.CircularBuffer.writeMatch to allow for overlapping regions. 2024-02-27 21:26:26 -08:00
Jacob Young
f446d8e8f9 posix: @as and other general cleanup 2024-02-28 04:30:49 +01:00
Igor Anić
62ce753814 compress: activate tests in wasm32
They were disabled because insufficient stack size.
That is
[changed](d51aa9748f) now.
2024-02-27 19:19:59 -08:00
Jacob Young
17d0bb5bea posix: fix socket fd leaks 2024-02-28 04:09:14 +01:00
Ryan Liptak
ae7f3fc360 Eliminate error.InvalidHandle from OpenError and RealPathError
InvalidHandle in OpenError is no longer a possible error on any platform. In the past it was able to be returned in `openOptionsFromFlagsWasi`, but the implementation was changed in 7680c5330c to make it no longer possible.

InvalidHandle in RealPathError was a holdover from before d5312d53a0, which made realpath a compile error on WASI. However, InvalidHandle was also a possible error in the FreeBSD fallback implementation added in 537624734c. This commit changes the FreeBSD fallback implementation to return FileNotFound instead of InvalidHandle which matches how EBADF is handled in all the other `realpath` implementations (including the FreeBSD non-fallback implementation).

Closes #19084
2024-02-27 15:59:50 -08:00
Travis Staloch
30bf8d7147 testing.expectFmt() - reuse expectEqualStrings() 2024-02-27 14:40:28 -08:00
dweiller
bd0dbb0a13 std.compress.zstd: enable tests for wasm32
The increase in stack size for wasm32 targets in commit d51aa9748f
allows the streaming decompressor to be tested on wasm32-wasi.
2024-02-27 11:37:48 -08:00
Andrew Kelley
6f7354a041
Merge pull request #19102 from ziglang/decouple-zir
JIT `zig fmt` and `zig reduce`
2024-02-27 11:03:08 -08:00
Luuk de Gram
27f589dea1
Merge pull request #18538 from Pangoraw/wasm_vector_abi
wasm: allow non-int vectors
2024-02-27 18:42:12 +01:00
Andrew Kelley
085bde6889
Merge pull request #19087 from squeek502/redundant-test-naming
Remove redundant test name prefixes now that test names are fully qualified
2024-02-26 23:43:02 -08:00
Andrew Kelley
97f2a8b5cb introduce ZIG_DEBUG_CMD to choose debug mode
for lazily built commands such as `zig fmt` and `zig reduce`. Useful if
you want to test a patch to them.
2024-02-26 23:48:56 -07:00
Andrew Kelley
1a01151a4e back out the build_runner.zig moving change
I'd like to move this file but to do so requires a zig1.wasm update, so
I'll choose a more opportune moment to make this change.
2024-02-26 23:44:01 -07:00
Andrew Kelley
dfe430e9f4 move lazily compiled source files to lib/compiler/ 2024-02-26 23:43:56 -07:00
Andrew Kelley
0157e1196c compiler: JIT zig reduce
See #19063
2024-02-26 23:43:42 -07:00
Andrew Kelley
ba575595bb std.zig: don't try to unit test a separate exe
This results in "file exists in multiple modules" errors.

A future commit should move these subcommands to outside std/.
2024-02-26 22:53:17 -07:00
Andrew Kelley
d661f0f35b compiler: JIT zig fmt
See #19063
2024-02-26 22:26:19 -07:00
Andrew Kelley
b116063e02 move AstGen to std.zig.AstGen
Part of an effort to ship more of the compiler in source form.
2024-02-26 21:51:19 -07:00
Andrew Kelley
a2e87aba66 rearrange std.zig
This frees up std.zig.fmt to be used for the implementation of `zig
fmt`.
2024-02-26 21:35:33 -07:00
Andrew Kelley
7b37bc771b move Zir to std.zig.Zir
Part of an effort to ship more of the compiler in source form.
2024-02-26 21:35:30 -07:00
Andrew Kelley
5f3b21a5b6 Zir: decouple from InternPool
Note that the correctness of these enum tag values is still protected
by the comptime logic at the top of Zcu (currently src/Module.zig).
2024-02-26 21:35:30 -07:00
Andrew Kelley
f7143e18e3 move Zcu.LazySrcLoc to std.zig.LazySrcLoc
Part of an effort to ship more of the compiler in source form.
2024-02-26 21:35:30 -07:00
Jacob Young
4e2570baaf http: fix fetching a github release
* Support different keep alive defaults with different http versions.
 * Fix incorrect usage of `copyBackwards`, which copies in a backwards
   direction allowing data to be moved forward in a buffer, not
   backwards in a buffer.
2024-02-26 20:11:43 -08:00