mirrors/zig
1
0
Fork 0
mirror of https://codeberg.org/ziglang/zig.git synced 2025-12-09 23:29:03 +00:00
Code Issues Projects Releases Packages Wiki Activity Actions 1
zig/lib/std
History
Andrew Kelley 377e8579f9 std.zig.tokenizer: simplify 
I pointed a fuzzer at the tokenizer and it crashed immediately. Upon
inspection, I was dissatisfied with the implementation. This commit
removes several mechanisms:
* Removes the "invalid byte" compile error note.
* Dramatically simplifies tokenizer recovery by making recovery always
  occur at newlines, and never otherwise.
* Removes UTF-8 validation.
* Moves some character validation logic to `std.zig.parseCharLiteral`.

Removing UTF-8 validation is a regression of #663, however, the existing
implementation was already buggy. When adding this functionality back,
it must be fuzz-tested while checking the property that it matches an
independent Unicode validation implementation on the same file. While
we're at it, fuzzing should check the other properties of that proposal,
such as no ASCII control characters existing inside the source code.

Other changes included in this commit:

* Deprecate `std.unicode.utf8Decode` and its WTF-8 counterpart. This
  function has an awkward API that is too easy to misuse.
* Make `utf8Decode2` and friends use arrays as parameters, eliminating a
  runtime assertion in favor of using the type system.

After this commit, the crash found by fuzzing, which was
"\x07\xd5\x80\xc3=o\xda|a\xfc{\x9a\xec\x91\xdf\x0f\\\x1a^\xbe;\x8c\xbf\xee\xea"
no longer causes a crash. However, I did not feel the need to add this
test case because the simplified logic eradicates most crashes of this
nature.
2024-07-31 16:57:42 -07:00
..
Build Merge pull request #20667 from jayrod246/windows-watch 2024-07-28 20:35:18 -07:00
c std.Target: Remove kfreebsd OS specifier. 2024-07-20 05:08:14 +02:00
compress std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
crypto Fix compilation issues in crypto.bccrypt and poly1305 (#20756) 2024-07-23 19:45:24 +00:00
dwarf dwarf: use StackIterator.MemoryAccessor to check memory accesses instead of isValidMemory 2024-07-10 10:46:35 -04:00
fmt std.fmt.formatFloat: skip f80 round-trip tests on x86_64 windows 2024-07-14 11:19:34 +12:00
fs std.fs: Rework to always use statx() instead of fstat()/fstatat() on Linux. 2024-07-29 09:50:41 +02:00
hash std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
heap riscv: workarounds for riscv threading 2024-07-26 14:53:50 -07:00
http std.posix.iovec: use .base and .len instead of .iov_base and .iov_len 2024-04-28 00:20:30 -07:00
io implement std.testing.fuzzInput 2024-07-25 18:52:20 -07:00
json Add std.json.ParseOptions.parse_numbers to preserve float precision (#20744) 2024-07-25 20:55:06 -04:00
math add std.testing.random_seed 2024-07-23 11:43:12 -07:00
mem
meta std: promote tests to doctests 2024-03-21 14:11:46 -07:00
net std.net, std.http: simplify 2024-02-23 02:37:11 -07:00
os Merge pull request #20389 from alexrp/riscv32 2024-07-29 16:13:35 -07:00
posix std: Stop supporting Linux/glibc versions older than declared in std.Target. 2024-07-29 09:50:41 +02:00
process Allow setting PGID in std.process.Child.spawn 2024-07-22 11:50:00 +09:00
Random add std.testing.random_seed 2024-07-23 11:43:12 -07:00
sort std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
tar std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
Target std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
testing
Thread std.c reorganization 2024-07-19 00:30:32 -07:00
time std: promote tests to doctests 2024-03-21 14:11:46 -07:00
tz
unicode
valgrind std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
zig std.zig.tokenizer: simplify 2024-07-31 16:57:42 -07:00
zip std: Convert deprecated aliases to compile errors and fix usages 2024-06-13 10:18:59 -04:00
array_hash_map.zig docs: ArrayHashMap: warn against the shrink footgun 2024-07-26 15:24:25 -07:00
array_list.zig std.ArrayList.unusedCapacitySlice: Return unaligned slice (#20490) 2024-07-14 00:56:29 +00:00
ascii.zig Rename isASCII to isAscii 2024-07-02 16:31:15 +02:00
atomic.zig std.atomic: Implement specialized spinLoopHint() for more architectures. 2024-07-29 09:41:47 +02:00
base64.zig improve documentation in std 2024-03-10 18:13:30 -07:00
bit_set.zig std: promote tests to doctests 2024-03-21 14:11:46 -07:00
BitStack.zig std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
bounded_array.zig Implement addManyAsSlice for BoundedArray 2024-05-21 13:46:05 +03:00
buf_map.zig
buf_set.zig Change many test blocks to doctests/decltests 2024-02-26 15:18:31 -08:00
Build.zig Add libcpp option to test options 2024-07-28 20:40:53 -07:00
builtin.zig riscv: implement ptr_slice_ptr_ptr 2024-07-26 04:05:40 -07:00
c.zig chore: correct non-standard comments. 2024-07-28 21:34:14 -07:00
coff.zig std.coff: Add missing entries to MachineType. 2024-07-24 15:02:55 -07:00
compress.zig improve documentation in std 2024-03-10 18:13:30 -07:00
crypto.zig Merge pull request #19239 from jedisct1/ml-kem 2024-03-11 18:48:08 -07:00
debug.zig std: Stop supporting Linux/glibc versions older than declared in std.Target. 2024-07-29 09:50:41 +02:00
dwarf.zig dwarf: use StackIterator.MemoryAccessor to check memory accesses instead of isValidMemory 2024-07-10 10:46:35 -04:00
dynamic_library.zig std.c reorganization 2024-07-19 00:30:32 -07:00
elf.zig std.elf: Add Elf32_Relr and Elf64_Relr definitions. 2024-07-26 00:40:41 +02:00
enums.zig EnumMap: fix init 2024-05-26 16:13:54 -04:00
fifo.zig std: promote tests to doctests 2024-03-21 14:11:46 -07:00
fmt.zig fix std.fmt.hex 2024-07-20 01:06:29 -07:00
fs.zig std.fs.Dir: Refactor atomicSymLink from std.fs 2024-07-28 20:08:52 -07:00
gpu.zig std: add gpu namespace 2024-02-05 11:55:14 +03:30
hash.zig
hash_map.zig add std.testing.random_seed 2024-07-23 11:43:12 -07:00
heap.zig std.c reorganization 2024-07-19 00:30:32 -07:00
http.zig std: avoid references that trigger compile errors 2024-07-04 21:01:42 +01:00
io.zig Better implementation of GetLastError. (#20623) 2024-07-15 10:49:51 -07:00
json.zig Delete compile errors for deprecated decls 2024-05-03 13:27:30 -07:00
leb128.zig Rename *[UI]LEB128 functions to *[UI]leb128 2024-06-23 04:30:12 +01:00
linked_list.zig
log.zig update the codebase for the new std.Progress API 2024-05-27 20:56:48 -07:00
macho.zig The Mach-O header flags gained two new members at some point, and these are missing in the Zig std library. This PR adds these. 2024-04-18 20:08:03 -07:00
math.zig aarch64: reenable tests that are no longer regressed 2024-07-25 20:23:23 +03:00
mem.zig std.mem.readVarInt: fix for 8-bits and below (Reloaded) (#20513) 2024-07-28 20:26:36 -07:00
meta.zig chore: correct non-standard comments. 2024-07-28 21:34:14 -07:00
multi_array_list.zig InternPool: implement and use thread-safe list for strings 2024-07-07 22:59:52 -04:00
net.zig std.c reorganization 2024-07-19 00:30:32 -07:00
once.zig std: improve std.once tests 2024-04-15 15:24:30 -07:00
os.zig std.c reorganization 2024-07-19 00:30:32 -07:00
packed_int_array.zig chore: correct non-standard comments. 2024-07-28 21:34:14 -07:00
pdb.zig extract std.posix from std.os 2024-03-19 11:45:09 -07:00
posix.zig std: Stop supporting Linux/glibc versions older than declared in std.Target. 2024-07-29 09:50:41 +02:00
priority_dequeue.zig add std.testing.random_seed 2024-07-23 11:43:12 -07:00
priority_queue.zig std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
process.zig move std.zig.fatal to std.process.fatal 2024-07-20 01:06:28 -07:00
Progress.zig std.posix: Consider invalid signal numbers to sigaction() to be programmer error. 2024-07-21 22:38:09 +02:00
Random.zig std: Convert deprecated aliases to compile errors and fix usages 2024-06-13 10:18:59 -04:00
RingBuffer.zig std.RingBuffer: use sliceAt/sliceFirst in read*AssumeLength 2024-03-10 18:17:23 +11:00
segmented_list.zig Remove redundant test name prefixes now that test names are fully qualified 2024-02-26 15:18:31 -08:00
SemanticVersion.zig Change many test blocks to doctests/decltests 2024-02-26 15:18:31 -08:00
simd.zig aarch64: reenable tests that are no longer regressed 2024-07-25 20:23:23 +03:00
sort.zig add std.testing.random_seed 2024-07-23 11:43:12 -07:00
start.zig Merge pull request #20852 from ziglang/init-array-start-code 2024-07-29 18:40:27 -07:00
start_windows_tls.zig
static_string_map.zig std.static_string_map: Cast length of input for runtime initialization 2024-06-20 23:22:39 +00:00
std.zig std: Convert deprecated aliases to compile errors and fix usages 2024-06-13 10:18:59 -04:00
tar.zig std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
Target.zig std: fix long double size for loongarch. 2024-07-30 15:21:14 -07:00
testing.zig implement std.testing.fuzzInput 2024-07-25 18:52:20 -07:00
Thread.zig std.Thread: Add some syscall comments to LinuxThreadImpl.ThreadCompletion.freeAndExit(). 2024-07-29 09:50:40 +02:00
time.zig std.c reorganization 2024-07-19 00:30:32 -07:00
treap.zig Condense and extend std.Treap's traversal functionalities. (#20002) 2024-07-28 19:47:55 -07:00
tz.zig
unicode.zig std.zig.tokenizer: simplify 2024-07-31 16:57:42 -07:00
Uri.zig std.http.Client: always omit port when it matches default 2024-04-12 22:37:07 -07:00
valgrind.zig Rename nonSIMDCall* to nonSimdCall* 2024-07-03 19:00:17 +01:00
wasm.zig Remove redundant test name prefixes now that test names are fully qualified 2024-02-26 15:18:31 -08:00
zig.zig move std.zig.fatal to std.process.fatal 2024-07-20 01:06:28 -07:00
zip.zig std.zip: Fix typo in doc comments 2024-06-17 16:12:19 -04:00