Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:
* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
* std.crypto.chacha: support larger vectors on AVX2 and AVX512 targets
Ryzen 7 7700, ChaCha20/8 stream, long outputs:
Generic: 3268 MiB/s
AVX2 : 6023 MiB/s
AVX512 : 8086 MiB/s
Bump the rand.chacha buffer a tiny bit to take advantage of this.
More than 8 blocks doesn't seem to make any measurable difference.
ChaChaPoly also gets a small performance boost from this, albeit
Poly1305 remains the bottleneck.
Generic: 707 MiB/s
AVX2 : 981 MiB/s
AVX512 : 1202 MiB/s
aarch64 appears to generally benefit from 4-way vectorization.
Verified on Apple Silicon, but also on a Cortex A72.
These are great permutations, and there's nothing wrong with them
from a practical security perspective.
However, both were competing in the NIST lightweight crypto
competition.
Gimli didn't pass the 3rd selection round, and is not much used
in the wild besides Zig and libhydrogen. It will never be
standardized and is unlikely to get more traction in the future.
Xoodyak, that Xoodoo is the permutation of, was a finalist.
It has a lot of advantages and *might* be standardized without NIST.
But this is too early to tell, and too risky to commit to it
in a standard library.
For lightweight crypto, Ascon is the one that we know NIST will
standardize and that we can safely rely on from a usage perspective.
Switch to a traditional ChaCha-based CSPRNG, with an Ascon-based one
as an option for constrained systems.
Add a RNG benchmark by the way.
Gimli and Xoodoo served us well. Their code will be maintained,
but outside the standard library.
Ascon has been selected as new standard for lightweight cryptography
in the NIST Lightweight Cryptography competition.
Ascon won over Gimli and Xoodoo.
The permutation is unlikely to change. However, NIST may tweak
the constructions (XOF, hash, authenticated encryption) before
standardizing them. For that reason, implementations of those
are better maintained outside the standard library for now.
In fact, we already had an Ascon implementation in Zig:
`std.crypto.aead.isap` is based on it. While the implementation was
here, there was no public API to access it directly.
So:
- The Ascon permutation is now available as `std.crypto.core.Ascon`,
with everything needed to use it in AEADs and other Ascon-based
constructions
- The ISAP implementation now uses std.crypto.core.Ascon instead of
keeping a private copy
- The default CSPRNG replaces Xoodoo with Ascon. And instead of an
ad-hoc construction, it's using the XOFa mode of the NIST submission.
The PCG32 fill function seems to have been copy-pasted from code using u64, so requesting `n` bytes where `(n & 7) > 4` bytes would cause the last few bytes to be all 0.
Gimli was a game changer. A permutation that is large enough to be
used in sponge-like constructions, yet small enough to be compact
to implement and fast on a wide range of platforms.
And Gimli being part of the Zig standard library was awesome.
But since then, Gimli entered the NIST Lightweight Cryptography
Competition, competing againt other candidates sharing a similar set
of properties.
Unfortunately, Gimli didn't pass the 3rd round.
There are no practical attacks against Gimli when used correctly, but
NIST's decision means that Gimli is unlikely to ever get any traction.
So, maybe the time has come to move Gimli from the standard library
to another repository.
We shouldn't do it without providing an alternative, though.
And the best candidate for this is probably Xoodoo.
Xoodoo is the core function of Xoodyak, one of the finalists of the
NIST LWC competition, and the most direct competitor to Gimli. It is
also a 384-bit permutation, so it can easily be used everywhere Gimli
was used with no parameter changes.
It is the building block of Xoodyak (for actual encryption and hashing)
as well as Charm, that some Zig applications are already using.
Like Gimli that it was heavily inspired from, it is compact and
suitable for constrained environments.
This change adds the Xoodoo permutation to std.crypto.core.
The set of public functions includes everything required to later
implement existing Xoodoo-based constructions.
In order to prepare for the Gimli deprecation, the default
CSPRNG was changed to a Xoodoo-based that works exactly the same way.
- Test: Fix bucket counting. Previously, the first hit was not counted.
This off-by-one error slightly increased the mean of `*_total_variance`,
which decreased the acceptance rate for a particular random seed
from 95% to 92.6%. (Irrelevant for test failure because the seed is fixed.)
- Improve comments
* unify the logic for exporting math functions from compiler-rt,
with the appropriate suffixes and prefixes.
- add all missing f128 and f80 exports. Functions with missing
implementations call other functions and have TODO comments.
- also add f16 functions
* move math functions from freestanding libc to compiler-rt (#7265)
* enable all the f128 and f80 code in the stage2 compiler and behavior
tests (#11161).
* update std lib to use builtins rather than `std.math`.
As suggested by @leecannon, this provides more flexibility to the
`Random` interface. For exmaple, this allows for an implementation to
provide multiple different fill functions.
These changes have been made to resolve issue #10037. The `Random`
interface was implemented in such a way that causes significant slowdown
when calling the `fill` function of the rng used.
The `Random` interface is no longer stored in a field of the rng, and is
instead returned by the child function `random()` of the rng. This
avoids the performance issues caused by the interface.
We already have a LICENSE file that covers the Zig Standard Library. We
no longer need to remind everyone that the license is MIT in every single
file.
Previously this was introduced to clarify the situation for a fork of
Zig that made Zig's LICENSE file harder to find, and replaced it with
their own license that required annual payments to their company.
However that fork now appears to be dead. So there is no need to
reinforce the copyright notice in every single file.
When filling the last (len % 4) bytes of a buffer, the random number n was only being shifted right by 4 bits for each byte instead of 8. A random u16, for example, would always have its middle two nybbles be equal when generated this way. For comparison, Isaac64.zig, Sfc64.zig, and Xoroshiro128.zig all correctly shift right by 8 bits for each of the last bytes in their nearly identical fill functions.
std.crypto.random
* cross platform, even freestanding
* can't fail. on initialization for some systems requires calling
os.getrandom(), in which case there are rare but theoretically
possible errors. The code panics in these cases, however the
application may choose to override the default seed function and then
handle the failure another way.
* thread-safe
* supports the full Random interface
* cryptographically secure
* no syscall required to initialize on Linux (AT_RANDOM)
* calls arc4random on systems that support it
`std.crypto.randomBytes` is removed in favor of `std.crypto.random.bytes`.
I moved some of the Random implementations into their own files in the
interest of organization.
stage2 no longer requires passing a RNG; instead it uses this API.
Closes#6704
Ziggurat rng was using deprecated `random.scalar(u64)` which was causing compile errors on calls to public facing stdlib APIs (randExp) on 0.6+, this fixed those errors.
