The TurboSHAKE paper just got published:
https://eprint.iacr.org/2023/342.pdf
and unlike the previous K12 paper, suggests 0x1F instead of 0x01
as the default value for "D".
* Fix SHA3 with streaming
Leftover bytes should be added to the buffer, not to the state.
(or, always to the state; we can and probably should eventually get
rid of the buffer)
Fixes#14851
* Add a test for SHA-3 with streaming
Make the Keccak permutation public, as it's useful for more than
SHA-3 (kMAC, SHAKE, TurboSHAKE, TupleHash, etc).
Our Keccak implementation was accepting f as a comptime parameter,
but always used 64-bit words and 200 byte states, so it actually
didn't work with anything besides f=1600.
That has been fixed. The ability to use reduced-round versions
was also added in order to support M14 and K12.
The state was constantly converted back and forth between bytes
and words, even though only a part of the state is actually used
for absorbing and squeezing bytes. It was changed to something
similar to the other permutations we have, so we can avoid extra
copies, and eventually add vectorized implementations.
In addition, the SHAKE extendable output function (XOF) was
added (SHAKE128, SHAKE256). It is required by newer schemes,
such as the Kyber post-quantum key exchange mechanism, whose
implementation is currently blocked by SHAKE missing from our
standard library.
Breaking change: `Keccak_256` and `Keccak_512` were renamed to
`Keccak256` and `Keccak512` for consistency with all other
hash functions.
Ascon has been selected as new standard for lightweight cryptography
in the NIST Lightweight Cryptography competition.
Ascon won over Gimli and Xoodoo.
The permutation is unlikely to change. However, NIST may tweak
the constructions (XOF, hash, authenticated encryption) before
standardizing them. For that reason, implementations of those
are better maintained outside the standard library for now.
In fact, we already had an Ascon implementation in Zig:
`std.crypto.aead.isap` is based on it. While the implementation was
here, there was no public API to access it directly.
So:
- The Ascon permutation is now available as `std.crypto.core.Ascon`,
with everything needed to use it in AEADs and other Ascon-based
constructions
- The ISAP implementation now uses std.crypto.core.Ascon instead of
keeping a private copy
- The default CSPRNG replaces Xoodoo with Ascon. And instead of an
ad-hoc construction, it's using the XOFa mode of the NIST submission.
Although RFC 8446 states:
> Each party MUST send a "close_notify" alert before closing its write
> side of the connection
In practice many servers do not do this. Also in practice, truncation
attacks are thwarted at the application layer by comparing the amount of
bytes received with the amount expected via the HTTP headers.
Previously, the code only checked Common Name, leading to unable to
validate valid certificates which relied on the subject_alt_name
extension for host name verification.
This commit also adds rsa_pss_rsae_* back to the signature algorithms
list in the ClientHello.
This commit adds `writeEnd` and `writeAllEnd` in order to send data and
also notify the server that there will be no more data written.
Unfortunately, it seems most TLS implementations in the wild get this
wrong and immediately close the socket when they see a close_notify,
rather than only ending the data stream on the application layer.
This commit introduces tls.Decoder and then uses it in tls.Client. The
purpose is to make it difficult to introduce vulnerabilities in the
parsing code. With this abstraction in place, bugs in the TLS
implementation will trip checks in the decoder, regardless of the actual
length of packets sent by the other party, so that we can have
confidence when using ReleaseFast builds.
The code we are borrowing from https://github.com/shiguredo/tls13-zig
requires an Allocator for doing RSA certificate verification. As a
stopgap measure, this commit uses a FixedBufferAllocator to avoid heap
allocation for these functions.
Thank you to @naoki9911 for providing this great resource which has been
extremely helpful for me when working on this standard library TLS
implementation. Until Zig has std.crypto.rsa officially, we will borrow
this implementation of RSA. 🙏
Here's what I landed on for the TLS client. It's 16896 bytes
(max_ciphertext_record_len is 16640). I believe this is the theoretical
minimum size, give or take a few bytes.
These constraints are satisfied:
* a call to the readvAdvanced() function makes at most one call to the
underlying readv function
* iovecs are provided by the API, and used by the implementation for
underlying readv() calls to the socket
* the theoretical minimum number of memcpy() calls are issued in all
circumstances
* decryption is only performed once for any given TLS record
* large read buffers are fully exploited
This is accomplished by using the partial read buffer to storing both
cleartext and ciphertext.
The read function has been renamed to readAdvanced since it has slightly
different semantics than typical read functions, specifically regarding
the end-of-file. A higher level read function is implemented on top.
Now, API users may pass small buffers to the read function and
everything will work fine. This is done by re-decrypting the same
ciphertext record with each call to read() until the record is finished
being transmitted.
If the buffer supplied to read() is large enough, then any given
ciphertext record will only be decrypted once, since it decrypts
directly to the read() buffer and therefore does not need any memcpy. On
the other hand, if the buffer supplied to read() is small, then the
ciphertext is decrypted into a stack buffer, a subset is copied to the
read() buffer, and then the entire ciphertext record is saved for the
next call to read().
