Commit graph

10891 commits

Author SHA1 Message Date
Andrew Kelley
66954e8330 std.debug.FixedBufferReader is fine
it does not need to be deprecated
2024-08-07 00:48:32 -07:00
Andrew Kelley
de47acd732 code coverage dumping tool basic implementation
* std.debug.Dwarf: add `sortCompileUnits` along with a field to track
  the state for the purpose of assertions and correct API usage.
  This makes batch lookups faster.
  - in the future, findCompileUnit should be enhanced to rely on sorted
    compile units as well.
* implement `std.debug.Dwarf.resolveSourceLocations` as well as
  `std.debug.Info.resolveSourceLocations`. It's still pretty slow, since
  it calls getLineNumberInfo for each array element, repeating a lot of
  work unnecessarily.
* integrate these APIs with `std.Progress` to understand what is taking
  so long.

The output I'm seeing from this tool shows a lot of missing source
locations. In particular, the main area of interest is missing for my
tokenizer fuzzing example.
2024-08-07 00:48:32 -07:00
Andrew Kelley
2e12b45d8b introduce tool for dumping coverage file
with debug info resolved.

begin efforts of providing `std.debug.Info`, a cross-platform
abstraction for loading debug information into an in-memory format that
supports queries such as "what is the source location of this virtual
memory address?"

Unlike `std.debug.SelfInfo`, this API does not assume the debug
information in question happens to match the host CPU architecture, OS,
or other target properties.
2024-08-07 00:48:32 -07:00
Andrew Kelley
107b272766 fuzzer: share zig to html rendering with autodocs 2024-08-07 00:48:32 -07:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Andrew Kelley
ffc050e055 fuzzer: log errors and move deduplicated runs to shared mem 2024-08-07 00:48:32 -07:00
Andrew Kelley
97643c1ecc fuzzer: track code coverage from all runs
When a unique run is encountered, track it in a bit set memory-mapped
into the fuzz directory so it can be observed by other processes, even
while the fuzzer is running.
2024-08-07 00:48:32 -07:00
Andrew Kelley
c8a226a429
Merge pull request #20960 from alexrp/size-align-fixes
`std.Target`: A bunch of C type size/alignment fixes
2024-08-06 14:55:15 -07:00
Alex Rønne Petersen
ef502daafe
std.Target: Handle mesa3d in c_type_bit_size(). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
23b5a6c71e
std.Target: Treat spirv as identical to spirv64 for ABI size/alignment purposes.
This is arbitrary since spirv (as opposed to spirv32/spirv64) refers to the
version with logical memory layout, i.e. no 'real' pointers. This change at
least matches what clang does.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
231f322a65
std.Target: Fix C long double size for ve (16, not 8). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
7b47ebe576
std.Target: Fix C long double size for sparc32 (8, not 16). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
eef499812c
std.Target: Fix C long double size for amdhsa, amdpal, and mesa3d (8, not 16). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
a1d3e56763
std.Target: Fix C long double size for opencl (8, not 16).
This value was correct for the old SPIR, but not for SPIR-V.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
606d011acf
std.Target: Fix C long long size for opencl (8, not 16).
This value was correct for the old SPIR, but not for SPIR-V.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
5dd66cd964
std.Target: Fix C type alignment calculation for ve. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
e678862520
std.Target: Fix C type alignment calculation for sparc64. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
98153c8d81
std.Target: Fix C type alignment calculation for loongarch64. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
29321ca4a2
std.Target: Fix C type alignment calculation for spirv32. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
e67388c2e5
std.Target: Fix C type alignment calculation for dxil. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
db8f00e277
std.Target: Fix ptrBitWidth_cpu_abi() for sparc32.
CPU feature set has nothing to do with ABI choice; the pointer bit width is
determined only by looking at the choice of sparc vs sparc64.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
ab154be7f0
std.Target: Fix ptrBitWidth_cpu_abi() for dxil (64-bit, not 32-bit).
The DXIL documentation claims 32-bit pointers:

https://github.com/microsoft/DirectXShaderCompiler/blob/main/docs/DXIL.rst#memory-accesses

Despite this, Clang considers pointers 64-bit when targeting it.
2024-08-06 20:17:03 +02:00
Andrew Kelley
1b88c6a8a2 glibc: update abilists file
updated for glibc 2.40, and adds arc, csky, and m68k
2024-08-05 09:53:37 -07:00
Andrew Kelley
fab5df4028
Merge pull request #20934 from alexrp/sparc32-leon3
`std.Target`: Bump baseline CPU for sparc32 to v9; add `sparcv9-linux-gnu` to `process_headers.zig`
2024-08-05 09:52:52 -07:00
Alex Rønne Petersen
388248ad33 std.zig.target: Set minimum glibc version for csky to 2.29.
https://lists.gnu.org/archive/html/info-gnu/2019-01/msg00018.html
2024-08-05 09:49:47 -07:00
Alex Rønne Petersen
27775f1a9e
std.Target: Use v9 as the baseline CPU for sparc32.
It is impossible to even build projects like glibc when targeting a generic
SPARC v8 CPU; LEON3 is effectively considered the baseline for `sparc-linux-gnu`
now, particularly due to it supporting a CASA instruction similar to the one in
SPARC v9. However, it's slightly incompatible with SPARC v9 due to having a
different ASI tag, so resulting binaries would not be portable to regular SPARC
CPUs. So, as the least bad option, make v9 the baseline for sparc32.
2024-08-05 12:20:41 +02:00
Fri3dNstuff
a655c15c40
std.sort: Remove key argument from binary-search-like functions (#20927)
closes #20110
2024-08-04 22:02:15 +00:00
Linus Groh
a18293214d std.Target: Specify dynamic linker and C type sizes for serenity 2024-08-04 12:03:43 -07:00
Alex Rønne Petersen
cb1fffb29e
std.os.windows.tls: Set AddressOfCallBacks to &__xl_a + 1.
`__xl_a` is just a global variable containing a null function pointer. There's
nothing magical about it or its name at all.

The section names used on `__xl_a` and `__xl_b` (`.CRT$XLA` and `.CRT$XLZ`) are
the real magic here. The compiler emits TLS variables into `.CRT$XL<x>`
sections, where `x` is an uppercase letter between A and Z (exclusive). The
linker then sorts those sections alphabetically (due to the `$`), and the result
is a neat array of TLS initialization callbacks between `__xl_a` and `__xl_z`.

That array is null-terminated, though! Normally, `__xl_z` serves as the null
terminator; however, by pointing `AddressesOfCallBacks` to `__xl_a`, which just
contains a null function pointer, we've effectively made it so that the PE
loader will just immediately stop invoking TLS callbacks. Fix that by pointing
to the first actual TLS callback instead (or `__xl_z` if there are none).
2024-08-03 20:55:00 +02:00
Alex Rønne Petersen
1d8fca0060
std.os.windows.tls: Only define _tls_array when targeting MSVC.
LLVM does not use it when emitting code for the MinGW ABI.
2024-08-03 20:48:48 +02:00
Alex Rønne Petersen
0f1db90198
std.os.windows.tls: Slightly improve type safety. 2024-08-03 20:48:48 +02:00
Alex Rønne Petersen
c2fcdc21c2
std.os.windows.tls: Change type of _tls_start/_tls_end to *anyopaque.
If they're typed as `u8`, they can be aligned to anything. We want at least
pointer size alignment.
2024-08-03 20:47:08 +02:00
Alex Rønne Petersen
8056a85151
std: Move start_windows_tls.zig to os/windows/tls.zig.
Just to be consistent with Linux.
2024-08-03 20:35:08 +02:00
Alex Rønne Petersen
0ad97b4122
std.os.linux: Add VDSO definition for riscv32/riscv64. 2024-08-03 18:49:12 +02:00
Alex Rønne Petersen
64e119124f
std.os.linux: Fix CGT_SYM for mips/mips64. 2024-08-03 18:48:53 +02:00
Alex Rønne Petersen
d71076c982
std.os.linux: Replace @hasDecl() with != void check for VDSO.
If there is a VDSO, it will have clock_gettime(). The main thing we're concerned
about is architectures that don't have a VDSO at all, of which there are a few.
2024-08-03 18:46:04 +02:00
Andrew Kelley
6d606cc38b reintroduce std.Dwarf.abi.supportsUnwinding
There are two concepts here: one for whether dwarf supports unwinding on
that target, and another for whether the Zig standard library
implements it yet.
2024-08-02 14:15:03 -07:00
Andrew Kelley
975c185b92 fix compilation on powerpc GNU systems
...which have a ucontext_t but not a PC register. The current stack
unwinding implementation does not yet support this architecture.

Also fix name of `std.debug.SelfInfo.openSelf` to remove redundancy.

Also removed this hook into root providing an "openSelfDebugInfo"
function. Sorry, this debugging code is not of sufficient quality to
offer a plugin API right now.
2024-08-02 14:14:59 -07:00
Tristan Ross
3ebb880663
glibc: Replace ilp32/lp64 headers with ilp32d/lp64d.
The former are soft float; the latter are hard float. We primarily care about
hard float here.

Signed-off-by: Alex Rønne Petersen <alex@alexrp.com>
2024-08-02 09:54:08 +02:00
Alex Rønne Petersen
635a3d87de
glibc: Change riscv32-linux-gnuilp32 target triple to riscv32-linux-gnu.
This target triple was weird on multiple levels:

* The `ilp32` ABI is the soft float ABI. This is not the main ABI we want to
  support on RISC-V; rather, we want `ilp32d`.
* `gnuilp32` is a bespoke tag that was introduced in Zig. The rest of the world
  just uses `gnu` for RISC-V target triples.
* `gnu_ilp32` is already the name of an ILP32 ABI used on AArch64. `gnuilp32` is
  too easy to confuse with this.
* We don't use this convention for `riscv64-linux-gnu`.
* Supporting all RISC-V ABIs with this convention will result in combinatorial
  explosion; see #20690.
2024-08-02 09:54:08 +02:00
Alex Rønne Petersen
3c2733de6e
std.Target: Use a saner default dynamic linker path for riscv.
The vast majority of Linux systems will be running these ABIs.
2024-08-02 09:54:08 +02:00
Alex Rønne Petersen
948bc91d12
std.Target.VersionRange: Make default() respect the minimum glibc version. 2024-08-02 09:54:08 +02:00
Andrew Kelley
48d584e3a3 std.debug: reorg and clarify API goals
After this commit:

`std.debug.SelfInfo` is a cross-platform abstraction for the current
executable's own debug information, with a goal of minimal code bloat
and compilation speed penalty.

`std.debug.Dwarf` does not assume the current executable is itself the
thing being debugged, however, it does assume the debug info has the
same CPU architecture and OS as the current executable. It is planned to
remove this limitation.
2024-08-01 23:11:59 -07:00
Andrew Kelley
290966c249 std.debug: rename Info to SelfInfo
This code has the hard-coded goal of supporting the executable's own
debug information and makes design choices along that goal, such as
memory-mapping the inputs, using dl_iterate_phdr, and doing conditional
compilation on the host target.

A more general-purpose implementation of debug information may be able
to share code with this, but there are some fundamental
incompatibilities. For example, the "SelfInfo" implementation wants to
avoid bloating the binary with PDB on POSIX systems, and likewise DWARF
on Windows systems, while a general-purpose implementation needs to
support both PDB and DWARF from the same binary. It might, for example,
inspect the debug information from a cross-compiled binary.

`SourceLocation` now lives at `std.debug.SourceLocation` and is
documented.

Deprecate `std.debug.runtime_safety` because it returns the optimization
mode of the standard library, when the caller probably wants to use the
optimization mode of their own module.

`std.pdb.Pdb` is moved to `std.debug.Pdb`, mirroring the recent
extraction of `std.debug.Dwarf` from `std.dwarf`.

I have no idea why we have both Module (with a Windows-specific
definition) and WindowsModule. I left some passive aggressive doc
comments to express my frustration.
2024-08-01 22:11:35 -07:00
Andrew Kelley
ab0253f662 std.debug.Info: rename ModuleDebugInfo to Module 2024-08-01 22:11:29 -07:00
Andrew Kelley
1ba6b56c81 std.debug.Info: extract to separate file 2024-08-01 22:11:23 -07:00
Andrew Kelley
e5b46eab3b std: dwarf namespace reorg
std.debug.Dwarf is the parsing/decoding logic. std.dwarf remains the
unopinionated types and bits alone.

If you look at this diff you can see a lot less redundancy in
namespaces.
2024-08-01 13:56:12 -07:00
Andrew Kelley
377274ee9a std.debug.DebugInfo: rename to std.debug.Info
avoiding redundancy in the fully qualified namespace
2024-08-01 13:47:42 -07:00
Andrew Kelley
12d0c9a2fc add std.fs.Dir.Handle
mirrors std.fs.File.Handle
2024-08-01 13:47:23 -07:00
Andrew Kelley
5c6f5e6cf2 test runner: avoid spawning progress thread when instrumented
because it causes unwanted concurrent accesses to pc tracking
2024-08-01 13:47:09 -07:00