Commit graph

65 commits

Author SHA1 Message Date
dweiller
ac1b957e79 std.compress.zstd: remove allocation from DecompressStream 2024-02-23 02:37:11 -07:00
dweiller
63fa151f1c std.compress.zstandard: fix buffer sizes
This change corrects the size of various internal buffers used. The
previous behavior did not cause validity problems but wasted space.
2024-02-23 02:37:11 -07:00
mlugg
51595d6b75
lib: correct unnecessary uses of 'var' 2023-11-19 09:55:07 +00:00
dweiller
138a35df8f zstandard: fix division by zero when using RingBuffer
This change fixes some division-by-zero bugs introduced by the optimized
ring buffer read/write functions in d8c067966.

There are edge cases where decompression can use a length zero ring
buffer as the size of the ring buffer used is exactly the the window
size specified by a Zstandard frame, and this can be zero. Switching
away from loops to mem copies means that we need to ensure ring buffers
do not have length zero ring when attempting to read/write from them.
2023-11-10 15:18:16 -05:00
dweiller
f6de3ec963 zstandard: fix incorrect RLE decompression into ring buffer
This reverts a change introduced in #17400 causing a bug when
decompressing an RLE block into a ring buffer.

RLE blocks contain only a single byte of data to copy into the output,
so attempting to copy a slice causes buffer overruns and incorrect
decompression.
2023-11-03 23:03:43 -04:00
Andrew Kelley
3fc6fc6812 std.builtin.Endian: make the tags lower case
Let's take this breaking change opportunity to fix the style of this
enum.
2023-10-31 21:37:35 -04:00
Jacob Young
d890e81761 mem: fix ub in writeInt
Use inline to vastly simplify the exposed API.  This allows a
comptime-known endian parameter to be propogated, making extra functions
for a specific endianness completely unnecessary.
2023-10-31 21:37:35 -04:00
joadnacer
a8eb70c9d7 compress: Improve RingBuffer usage 2023-10-21 19:20:12 +01:00
mlugg
f26dda2117 all: migrate code to new cast builtin syntax
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:

* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
2023-06-24 16:56:39 -07:00
Eric Joldasov
50339f595a all: zig fmt and rename "@XToY" to "@YFromX"
Signed-off-by: Eric Joldasov <bratishkaerik@getgoogleoff.me>
2023-06-19 12:34:42 -07:00
Ali Chraghi
3db3cf7790 std.sort: add pdqsort and heapsort 2023-05-23 17:55:59 -07:00
Andrew Kelley
6261c13731 update codebase to use @memset and @memcpy 2023-04-28 13:24:43 -07:00
zooster
bc8e1e1de4
Improvements to docs and text
* docs(std.math): elaborate on difference between absCast and absInt

* docs(std.rand.Random.weightedIndex): elaborate on likelihood

I think this makes it easier to understand.

* langref: add small reminder

* docs(std.fs.path.extension): brevity

* docs(std.bit_set.StaticBitSet): mention the specific types

* std.debug.TTY: explain what purpose this struct serves

This should also make it clearer that this struct is not supposed to provide unrelated terminal manipulation functionality such as setting the cursor position or something because terminals are complicated and we should keep this struct simple and focused on debugging.

* langref(package listing): brevity

* langref: explain what exactly `threadlocal` causes to happen

* std.array_list: link between swapRemove and orderedRemove

Maybe this can serve as a TLDR and make it easier to decide.

* PrefetchOptions.locality: clarify docs that this is a range

This confused me previously and I thought I can only use either 0 or 3.

* fix typos and more

* std.builtin.CallingConvention: document some CCs

* langref: explain possibly cryptic names

I think it helps knowing what exactly these acronyms (@clz and @ctz) and
abbreviations (@popCount) mean.

* variadic function error: add missing preposition

* std.fmt.format docs: nicely hyphenate

* help menu: say what to optimize for

I think this is slightly more specific than just calling it
"optimizations". These are speed optimizations. I used the word
"performance" here.
2023-04-23 21:06:21 +03:00
dweiller
32cf1d7cbf std.compress.zstandard: fix error sets for streaming API 2023-02-21 22:06:25 +11:00
dweiller
c7c35bf9e6 std.RingBuffer: add (non-concurrent) RingBuffer implementation 2023-02-21 12:58:34 +11:00
dweiller
a74f800dd7 std.compress.zstandard: update for multi-for-loop change 2023-02-20 16:58:48 +11:00
dweiller
1a862175d5 std.compress.zstandard: fix zstandardStream content size validation 2023-02-20 09:09:06 +11:00
dweiller
12aa478db0 std.compress.zstandard: also check block size when sequence count is 0 2023-02-20 09:09:06 +11:00
dweiller
a53cf299a6 std.compress.zstandard: add error condition to ring buffer decoding
Previously `executeSequenceRingBuffer()` would not verify the offset
against the number of bytes already decoded, so it would happily copy
garbage bytes rather than return an error before the window was filled.

To fix this a new `written_count` is added to the decode state that
tracks the total number of bytes decoded.
2023-02-20 09:09:06 +11:00
dweiller
5a31fc2014 std.compress.zstandard: fix erroneous literal stream empty checks 2023-02-20 09:09:06 +11:00
dweiller
8fd41314bd std.compress.zstandard: remove unneeded branch 2023-02-20 09:09:06 +11:00
dweiller
373d8ef26e std.compress.zstandard: check FSE bitstreams are fully consumed 2023-02-20 09:09:06 +11:00
dweiller
1530e73648 std.compress.zstandard: bytes read assert to error in decodeBlockReader 2023-02-20 09:09:06 +11:00
dweiller
ee5af3c74c std.compress.zstandard: cleanup high-level api docs and error sets 2023-02-20 09:09:06 +11:00
dweiller
31cc4605ab std.compress.zstandard: fix errors and crashes in ZstandardStream 2023-02-20 09:09:06 +11:00
dweiller
55e6e9409c std.compress.zstandard: fix content size check 2023-02-20 09:09:06 +11:00
dweiller
6d48b055af std.compress.zstandard: add decodeFrameHeader
Also do some other minor API cleanup
2023-02-20 09:09:06 +11:00
dweiller
3975a9d7ca std.compress.zstandard: error when FSE bitstream is no fully consumed 2023-02-20 09:09:06 +11:00
dweiller
77ca1f7859 std.compress.zstandard: remove UnusedBitSet error 2023-02-20 09:09:06 +11:00
dweiller
d9a90e1818 std.compress.zstandard: fix decodeAlloc() and remove decodeFrameAlloc() 2023-02-20 09:09:06 +11:00
dweiller
2134769436 std.compress.zstandard: validate skippable frame size 2023-02-20 09:09:06 +11:00
dweiller
98bbd959b0 std.compress.zstandard: improve block size validation 2023-02-20 09:09:06 +11:00
dweiller
ece52e0771 std.compress.zstandard: verify content size and fix crash 2023-02-20 09:09:06 +11:00
dweiller
06ab5a2cd2 std.compress.zstandard: add multi-frame decoding functions 2023-02-20 09:09:06 +11:00
dweiller
a625df4636 std.compress.zstandard: fix fse decoding crash 2023-02-20 09:09:06 +11:00
dweiller
1c509f483a std.compress.zstandard: fix crashes 2023-02-20 09:09:06 +11:00
dweiller
596a97fb55 std.compress.zstandard: fix crashes 2023-02-20 09:09:06 +11:00
dweiller
a651704876 std.compress.zstandard: free allocated result on error 2023-02-20 09:09:06 +11:00
dweiller
3f1c4306ca std.compress.zstandard: fix capitalisation of Zstandard 2023-02-20 09:09:06 +11:00
dweiller
ddeabc9aa7 std.compress.zstandard: add decodeFrameAlloc() 2023-02-20 09:09:06 +11:00
dweiller
89f9c5cb37 std.compress.zstandard: improve doc comments 2023-02-20 09:09:06 +11:00
dweiller
7e2755646f std.compress.zstandard: split decompressor into multiple files 2023-02-20 09:09:06 +11:00
dweiller
6e3e72884b std.compress.zstandard: fix crashes 2023-02-20 09:09:06 +11:00
dweiller
5723291444 std.compress.zstandard: add decodeBlockReader 2023-02-20 09:09:06 +11:00
dweiller
947ad3e268 std.compress.zstandard: add FrameContext and add literals into DecodeState 2023-02-20 09:09:06 +11:00
dweiller
2d35c16ee7 std.compress.zstandard: add init/deinit for ring buffer, fix len() 2023-02-20 09:09:06 +11:00
dweiller
e92575d3d4 std.compress.zstandard: verify checksum in decodeFrameAlloc() 2023-02-20 09:09:06 +11:00
dweiller
3bfba36548 std.compress.zstandard: clean up error sets and line lengths 2023-02-20 09:09:06 +11:00
dweiller
3c06e2e7d0 std.compress.zstandard: add doc comments for RingBuffer 2023-02-20 09:09:06 +11:00
dweiller
1e5b8be509 std.compress.zstandard: add window size limit param 2023-02-20 09:09:06 +11:00