Commit graph

9661 commits

Author SHA1 Message Date
Andrew Kelley
0e99f517f2
Merge pull request #20958 from ziglang/fuzz
introduce a fuzz testing web interface
2024-08-07 11:55:30 -07:00
Ryan Liptak
e912411fbe Run update_cpu_features against LLVM 18.1.8
- RISCV unaligned-scalar-mem was added in LLVM 18.1.6
- Fixes arm v9_5a confusion
2024-08-07 09:03:05 -07:00
Josh Wolfe
a805454dea fix partial strings getting dropped when multi-byte codepoints span input buffers 2024-08-07 06:29:34 -04:00
Josh Wolfe
8bec737ca2 fix tests actually checking string contents 2024-08-07 06:29:34 -04:00
Alex Rønne Petersen
f9f8942008 std.os.linux: Move clone() here and stop exporting it. 2024-08-07 01:19:51 -07:00
Andrew Kelley
75f78bfb77
Merge pull request #20922 from alexrp/vdso
`std.os.linux`: Fix VDSO for mips, add VDSO for riscv
2024-08-07 01:18:35 -07:00
Andrew Kelley
cd5f673cae
Merge pull request #20909 from alexrp/glibc-riscv
Support building glibc for riscv32/riscv64
2024-08-07 01:14:54 -07:00
Andrew Kelley
8184912a98
Merge pull request #20925 from alexrp/windows-tls
`std`: Some Windows TLS cleanup and fixes
2024-08-07 01:12:11 -07:00
Alex Rønne Petersen
c0681d6b6e start: Add sparc32 support. 2024-08-07 01:09:18 -07:00
Alex Rønne Petersen
8cbf091a9a std.os.linux.start_pie: Add sparc/sparc64 support. 2024-08-07 01:09:06 -07:00
Andrew Kelley
b071b10ce8
Merge pull request #20894 from alexrp/target-cleanup-4
`std.Target`: Minor rework to some `isArch()` functions, fix some related issues throughout `std`
2024-08-07 01:08:44 -07:00
Jakub Dóka
a6486492be
std.crypto.ecdsa: use separate function for null seed (#20953)
Due to the `std.crypto.ecdsa.KeyPair.create` taking and optional of seed, even if the seed is generated, cross-compiling to the environments without standard random source (eg. wasm) (`std.crypto.random.bytes`) will fail to compile.

This commit changes the API of the problematic function and moves the random seed generation to a new utility function.
2024-08-07 01:06:15 -07:00
Alex Rønne Petersen
ebd0c6ffd0 std.zig.target: Add arc-linux-gnu.
https://lists.gnu.org/archive/html/info-gnu/2020-08/msg00002.html
2024-08-07 01:03:22 -07:00
Alex Rønne Petersen
00097c3bb8 std.Target: Some corrections and additions to charSignedness().
Sourced from GCC backends and some manual testing with clang.
2024-08-07 00:56:02 -07:00
Andrew Kelley
40edd11516 std.debug: fix compile errors on windows and macos 2024-08-07 00:48:32 -07:00
Andrew Kelley
8dae629c4f update branch for latest std.sort changes 2024-08-07 00:48:32 -07:00
Andrew Kelley
bfc2ee03e5 fuzzer web ui: resolve cwd in sources.tar
because the wasm code needs to string match against debug information
2024-08-07 00:48:32 -07:00
Andrew Kelley
e64a00950e fuzzer web ui: introduce entry points
so you can have somewhere to start browsing
2024-08-07 00:48:32 -07:00
Andrew Kelley
dec7e45f7c fuzzer web UI: receive coverage information
* libfuzzer: track unique runs instead of deduplicated runs
  - easier for consumers to notice when to recheck the covered bits.
* move common definitions to `std.Build.Fuzz.abi`.

build runner sends all the information needed to fuzzer web interface
client needed in order to display inline coverage information along with
source code.
2024-08-07 00:48:32 -07:00
Andrew Kelley
22925636f7 std.debug.Coverage: use extern structs
helps the serialization use case
2024-08-07 00:48:32 -07:00
Andrew Kelley
b9fd0eeca6 add std.http.WebSocket 2024-08-07 00:48:32 -07:00
Andrew Kelley
d36c182748 std.posix: add some more void bits
prevents unnecessary compilation errors on wasm32-freestanding
2024-08-07 00:48:32 -07:00
Andrew Kelley
517cfb0dd1 fuzzing: progress towards web UI
* libfuzzer: close file after mmap
* fuzzer/main.js: connect with EventSource and debug dump the messages.
  currently this prints how many fuzzer runs have been attempted to
  console.log.
* extract some `std.debug.Info` logic into `std.debug.Coverage`.
  Prepares for consolidation across multiple different executables which
  share source files, and makes it possible to send all the
  PC/SourceLocation mapping data with 4 memcpy'd arrays.
* std.Build.Fuzz:
  - spawn a thread to watch the message queue and signal event
    subscribers.
  - track coverage map data
  - respond to /events URL with EventSource messages on a timer
2024-08-07 00:48:32 -07:00
Andrew Kelley
53aa9d75a9 std.debug.Info.resolveSourceLocations: O(N) implementation 2024-08-07 00:48:32 -07:00
Andrew Kelley
c2ab4614b6 std.Debug.Info: remove std.Progress integration
it's too fast to need it now
2024-08-07 00:48:32 -07:00
Andrew Kelley
1792258dc8 std.debug.Dwarf: precompute .debug_line table
yields a 60x speedup for resolveSourceLocations in debug builds
2024-08-07 00:48:32 -07:00
Andrew Kelley
66954e8330 std.debug.FixedBufferReader is fine
it does not need to be deprecated
2024-08-07 00:48:32 -07:00
Andrew Kelley
de47acd732 code coverage dumping tool basic implementation
* std.debug.Dwarf: add `sortCompileUnits` along with a field to track
  the state for the purpose of assertions and correct API usage.
  This makes batch lookups faster.
  - in the future, findCompileUnit should be enhanced to rely on sorted
    compile units as well.
* implement `std.debug.Dwarf.resolveSourceLocations` as well as
  `std.debug.Info.resolveSourceLocations`. It's still pretty slow, since
  it calls getLineNumberInfo for each array element, repeating a lot of
  work unnecessarily.
* integrate these APIs with `std.Progress` to understand what is taking
  so long.

The output I'm seeing from this tool shows a lot of missing source
locations. In particular, the main area of interest is missing for my
tokenizer fuzzing example.
2024-08-07 00:48:32 -07:00
Andrew Kelley
2e12b45d8b introduce tool for dumping coverage file
with debug info resolved.

begin efforts of providing `std.debug.Info`, a cross-platform
abstraction for loading debug information into an in-memory format that
supports queries such as "what is the source location of this virtual
memory address?"

Unlike `std.debug.SelfInfo`, this API does not assume the debug
information in question happens to match the host CPU architecture, OS,
or other target properties.
2024-08-07 00:48:32 -07:00
Andrew Kelley
107b272766 fuzzer: share zig to html rendering with autodocs 2024-08-07 00:48:32 -07:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Alex Rønne Petersen
ef502daafe
std.Target: Handle mesa3d in c_type_bit_size(). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
23b5a6c71e
std.Target: Treat spirv as identical to spirv64 for ABI size/alignment purposes.
This is arbitrary since spirv (as opposed to spirv32/spirv64) refers to the
version with logical memory layout, i.e. no 'real' pointers. This change at
least matches what clang does.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
231f322a65
std.Target: Fix C long double size for ve (16, not 8). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
7b47ebe576
std.Target: Fix C long double size for sparc32 (8, not 16). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
eef499812c
std.Target: Fix C long double size for amdhsa, amdpal, and mesa3d (8, not 16). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
a1d3e56763
std.Target: Fix C long double size for opencl (8, not 16).
This value was correct for the old SPIR, but not for SPIR-V.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
606d011acf
std.Target: Fix C long long size for opencl (8, not 16).
This value was correct for the old SPIR, but not for SPIR-V.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
5dd66cd964
std.Target: Fix C type alignment calculation for ve. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
e678862520
std.Target: Fix C type alignment calculation for sparc64. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
98153c8d81
std.Target: Fix C type alignment calculation for loongarch64. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
29321ca4a2
std.Target: Fix C type alignment calculation for spirv32. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
e67388c2e5
std.Target: Fix C type alignment calculation for dxil. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
db8f00e277
std.Target: Fix ptrBitWidth_cpu_abi() for sparc32.
CPU feature set has nothing to do with ABI choice; the pointer bit width is
determined only by looking at the choice of sparc vs sparc64.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
ab154be7f0
std.Target: Fix ptrBitWidth_cpu_abi() for dxil (64-bit, not 32-bit).
The DXIL documentation claims 32-bit pointers:

https://github.com/microsoft/DirectXShaderCompiler/blob/main/docs/DXIL.rst#memory-accesses

Despite this, Clang considers pointers 64-bit when targeting it.
2024-08-06 20:17:03 +02:00
Andrew Kelley
fab5df4028
Merge pull request #20934 from alexrp/sparc32-leon3
`std.Target`: Bump baseline CPU for sparc32 to v9; add `sparcv9-linux-gnu` to `process_headers.zig`
2024-08-05 09:52:52 -07:00
Alex Rønne Petersen
388248ad33 std.zig.target: Set minimum glibc version for csky to 2.29.
https://lists.gnu.org/archive/html/info-gnu/2019-01/msg00018.html
2024-08-05 09:49:47 -07:00
Alex Rønne Petersen
27775f1a9e
std.Target: Use v9 as the baseline CPU for sparc32.
It is impossible to even build projects like glibc when targeting a generic
SPARC v8 CPU; LEON3 is effectively considered the baseline for `sparc-linux-gnu`
now, particularly due to it supporting a CASA instruction similar to the one in
SPARC v9. However, it's slightly incompatible with SPARC v9 due to having a
different ASI tag, so resulting binaries would not be portable to regular SPARC
CPUs. So, as the least bad option, make v9 the baseline for sparc32.
2024-08-05 12:20:41 +02:00
Fri3dNstuff
a655c15c40
std.sort: Remove key argument from binary-search-like functions (#20927)
closes #20110
2024-08-04 22:02:15 +00:00
Linus Groh
a18293214d std.Target: Specify dynamic linker and C type sizes for serenity 2024-08-04 12:03:43 -07:00