They way OpenSSH does key derivation to protect keys using a password
is not the standard PBKDF2, but something funky, picking key material
non-linearly.
* std.crypto.aes: introduce AES block vectors
Modern Intel CPUs with the VAES extension can handle more than a
single AES block per instruction.
So can some ARM and RISC-V CPUs. Software implementations with
bitslicing can also greatly benefit from this.
Implement low-level operations on AES block vectors, and the
parallel AEGIS variants on top of them.
AMD Zen4:
aegis-128x4: 73225 MiB/s
aegis-128x2: 51571 MiB/s
aegis-128l: 25806 MiB/s
aegis-256x4: 46742 MiB/s
aegis-256x2: 30227 MiB/s
aegis-256: 8436 MiB/s
aes128-gcm: 5926 MiB/s
aes256-gcm: 5085 MiB/s
AES-GCM, and anything based on AES-CTR are also going to benefit
from this later.
* Make AEGIS-MAC twice a fast
* crypto.keccak.State: don't unconditionally permute after a squeeze()
Now, squeeze() behaves like absorb()
Namely,
squeeze(x[0..t]);
squeeze(x[t..n)); with t <= n
becomes equivalent to squeeze(x[0..n]).
* keccak: in debug mode, track transitions to prevent insecure ones.
Fixes#22019
Our key pair creation API was ugly and inconsistent between ecdsa
keys and other keys.
The same `generate()` function can now be used to generate key pairs,
and that function cannot fail.
For deterministic keys, a `generateDeterministic()` function is
available for all key types.
Fix comments and compilation of the benchmark by the way.
Fixes#21002
By default, programs built in debug mode that open a https connection
will append secrets to the file specified in the SSLKEYLOGFILE
environment variable to allow protocol debugging by external programs.
This was preventing TLSv1.2 from working in some cases, because servers
are allowed to send multiple handshake messages in the first handshake
record, whereas this inital loop was assuming that it only contained a
server hello.
This is mostly nfc cleanup as I was bisecting the client hello to find
the problematic part, and the only bug fix ended up being
key_share.x25519_kp.public_key ++
key_share.ml_kem768_kp.public_key.toBytes()
to
key_share.ml_kem768_kp.public_key.toBytes() ++
key_share.x25519_kp.public_key)
and the same swap in `KeyShare.exchange` as per some random blog that
says "a hybrid keyshare, constructed by concatenating the public KEM key
with the public X25519 key". I also note that based on the same blog
post, there was a draft version of this method that indeed had these
values swapped, and that used to be supported by this code, but it was
not properly fixed up when this code was updated from the draft spec.
Closes#21747
Note that the removed `error.TlsIllegalParameter` case is still caught
below when it is compared to a fixed-length string, but after checking
the proper protocol version requirement first.
This commit reworks how anonymous struct literals and tuples work.
Previously, an untyped anonymous struct literal
(e.g. `const x = .{ .a = 123 }`) was given an "anonymous struct type",
which is a special kind of struct which coerces using structural
equivalence. This mechanism was a holdover from before we used
RLS / result types as the primary mechanism of type inference. This
commit changes the language so that the type assigned here is a "normal"
struct type. It uses a form of equivalence based on the AST node and the
type's structure, much like a reified (`@Type`) type.
Additionally, tuples have been simplified. The distinction between
"simple" and "complex" tuple types is eliminated. All tuples, even those
explicitly declared using `struct { ... }` syntax, use structural
equivalence, and do not undergo staged type resolution. Tuples are very
restricted: they cannot have non-`auto` layouts, cannot have aligned
fields, and cannot have default values with the exception of `comptime`
fields. Tuples currently do not have optimized layout, but this can be
changed in the future.
This change simplifies the language, and fixes some problematic
coercions through pointers which led to unintuitive behavior.
Resolves: #16865
The old `CallingConvention` type is replaced with the new
`NewCallingConvention`. References to `NewCallingConvention` in the
compiler are updated accordingly. In addition, a few parts of the
standard library are updated to use the new type correctly.
Most of these changes seem like improvements. The PDB thing had a TODO
saying it used to crash; I anticipate it works now, we'll see what CI
does.
The `std.os.uefi` field renames are a notable breaking change.
The compiler actually doesn't need any functional changes for this: Sema
does reification based on the tag indices of `std.builtin.Type` already!
So, no zig1.wasm update is necessary.
This change is necessary to disallow name clashes between fields and
decls on a type, which is a prerequisite of #9938.
In a `memoized_call`, store how many backwards braches the call
performs. Add this to `sema.branch_count` when using a memoized call. If
this exceeds the quota, perform a non-memoized call to get a correct
"exceeded X backwards branches" error.
Also, do not memoize calls which do `@setEvalBranchQuota` or similar, as
this affects global state which must apply to the caller.
Change some eval branch quotas so that the compiler itself still builds correctly.
This commit manually changes a file in Aro which is automatically
generated. The sources which generate the file are not in this repo.
Upstream Aro should make the suitable changes on their end before the
next sync of Aro sources into the Zig repo.
std.crypto has quite a few instances of breaking naming conventions.
This is the beginning of an effort to address that.
Deprecates `std.crypto.utils`.
Due to the `std.crypto.ecdsa.KeyPair.create` taking and optional of seed, even if the seed is generated, cross-compiling to the environments without standard random source (eg. wasm) (`std.crypto.random.bytes`) will fail to compile.
This commit changes the API of the problematic function and moves the random seed generation to a new utility function.
On decryption tls client should remove zero byte padding after the
content type field. This padding is rarely used, the only site (from the
list of top domains) that I found using it is `tutanota.com`.
From [RFC](https://datatracker.ietf.org/doc/html/rfc8446#section-5.4):
> All encrypted TLS records can be padded.
> Padding is a string of zero-valued bytes appended to the ContentType
field before encryption.
> the receiving implementation scans the field from the end toward the
beginning until it finds a non-zero octet. This non-zero octet is the
content type of the message.
Currently we can't connect to that site:
```
$ zig run main.zig -- tutanota.com
error: TlsInitializationFailed
/usr/local/zig/zig-linux-x86_64-0.14.0-dev.208+854e86c56/lib/std/crypto/tls/Client.zig:476:45: 0x121fbed in init__anon_10331 (http_get_std)
if (inner_ct != .handshake) return error.TlsUnexpectedMessage;
^
/usr/local/zig/zig-linux-x86_64-0.14.0-dev.208+854e86c56/lib/std/http/Client.zig:1357:99: 0x1161f0b in connectTcp (http_get_std)
conn.data.tls_client.* = std.crypto.tls.Client.init(stream, client.ca_bundle, host) catch return error.TlsInitializationFailed;
^
/usr/local/zig/zig-linux-x86_64-0.14.0-dev.208+854e86c56/lib/std/http/Client.zig:1492:14: 0x11271e1 in connect (http_get_std)
} orelse return client.connectTcp(host, port, protocol);
^
/usr/local/zig/zig-linux-x86_64-0.14.0-dev.208+854e86c56/lib/std/http/Client.zig:1640:9: 0x111a24e in open (http_get_std)
try client.connect(valid_uri.host.?.raw, uriPort(valid_uri, protocol), protocol);
^
/home/ianic/Code/tls.zig/example/http_get_std.zig:28:19: 0x1118f8c in main (http_get_std)
var req = try client.open(.GET, uri, .{ .server_header_buffer = &server_header_buffer });
^
```
using this example:
```zig
const std = @import("std");
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
const allocator = gpa.allocator();
const args = try std.process.argsAlloc(allocator);
defer std.process.argsFree(allocator, args);
if (args.len > 1) {
const domain = args[1];
var client: std.http.Client = .{ .allocator = allocator };
defer client.deinit();
// Add https:// prefix if needed
const url = brk: {
const scheme = "https://";
if (domain.len >= scheme.len and std.mem.eql(u8, domain[0..scheme.len], scheme))
break :brk domain;
var url_buf: [128]u8 = undefined;
break :brk try std.fmt.bufPrint(&url_buf, "https://{s}", .{domain});
};
const uri = try std.Uri.parse(url);
var server_header_buffer: [16 * 1024]u8 = undefined;
var req = try client.open(.GET, uri, .{ .server_header_buffer = &server_header_buffer });
defer req.deinit();
try req.send();
try req.wait();
}
}
```
It is now composed of these main sections:
* Declarations that are shared among all operating systems.
* Declarations that have the same name, but different type signatures
depending on the operating system. Often multiple operating systems
share the same type signatures however.
* Declarations that are specific to a single operating system.
- These are imported one per line so you can see where they come from,
protected by a comptime block to prevent accessing the wrong one.
Closes#19352 by changing the convention to making types `void` and
functions `{}`, so that it becomes possible to update `@hasDecl` sites
to use `@TypeOf(f) != void` or `T != void`. Happily, this ended up
removing some duplicate logic and update some bitrotted feature
detection checks.
A handful of types have been modified to gain namespacing and type
safety. This is a breaking change.
Oh, and the last usage of `usingnamespace` site is eliminated.
Instead of calling the dynamically loaded kernel32.GetLastError, we can extract it from the TEB.
As shown by [Wine](34b1606019/include/winternl.h (L439)), the last error lives at offset 0x34 of the TEB in 32-bit Windows and at offset 0x68 in 64-bit Windows.
When calculating how much ciphertext from the stream can fit into
user and internal buffers we should also take into account ciphertext
data which are already in internal buffer.
Fixes: 15226
Tested with
[this](https://github.com/ziglang/zig/issues/15226#issuecomment-2218809140).
Using client with different read buffers until I, hopefully, understood
what is happening.
Not relevant to this fix, but this
[part](95d9292a7a/lib/std/crypto/tls/Client.zig (L988-L991))
is still mystery to me. Why we don't use free_size in buf_cap
calculation. Seems like rudiment from previous implementation without iovec.
This makes comparing host name with dns name from certificate case
insensitive.
I found a few domains (from the
[cloudflare](https://radar.cloudflare.com/domains) list of top domains)
for which tls.Client fails to connect. Error is:
```zig
error: TlsInitializationFailed
Code/zig/lib/std/crypto/Certificate.zig:336:9: 0x1177b1f in verifyHostName (http_get_std)
return error.CertificateHostMismatch;
Code/zig/lib/std/crypto/tls23/handshake_client.zig:461:25: 0x11752bd in parseServerCertificate (http_get_std)
try subject.verifyHostName(opt.host);
```
In its certificate this domains have host names which are not strictly
lower case. This is what checkHostName is comparing:
|host_name | dns_name |
|------------------------------------------------|
|ey.com | EY.COM |
|truist.com | Truist.com |
|wscampanhas.bradesco | WSCAMPANHAS.BRADESCO |
|dell.com | Dell.com |
From
[RFC2818](https://datatracker.ietf.org/doc/html/rfc2818#section-2.4):
> Matching is performed using the matching rules specified by
[RFC2459].
From [RFC2459](https://datatracker.ietf.org/doc/html/rfc2459#section-4.2.1.7):
> When comparing URIs, conforming implementations
> MUST compare the scheme and host without regard to case, but assume
> the remainder of the scheme-specific-part is case sensitive.
Testing with:
```
const std = @import("std");
pub fn main() !void {
var gpa = std.heap.GeneralPurposeAllocator(.{}){};
const allocator = gpa.allocator();
const args = try std.process.argsAlloc(allocator);
defer std.process.argsFree(allocator, args);
if (args.len > 1) {
const domain = args[1];
var client: std.http.Client = .{ .allocator = allocator };
defer client.deinit();
// Add https:// prefix if needed
const url = brk: {
const scheme = "https://";
if (domain.len >= scheme.len and std.mem.eql(u8, domain[0..scheme.len], scheme))
break :brk domain;
var url_buf: [128]u8 = undefined;
break :brk try std.fmt.bufPrint(&url_buf, "https://{s}", .{domain});
};
const uri = try std.Uri.parse(url);
var server_header_buffer: [16 * 1024]u8 = undefined;
var req = try client.open(.GET, uri, .{ .server_header_buffer = &server_header_buffer });
defer req.deinit();
try req.send();
try req.wait();
}
}
```
`$ zig run example/main.zig -- truist.com `
Most of the functions related to points on the Edwards25519 curve
check that input points are not in a small-order subgroup.
They don't check that points are on the prime-order subgroup,
because this is expensive, and not always necessary.
However, applications may require such a check in order to
ensure that a public key is valid, and that a secret key counterpart
exists.
Many functions in the public API of libsodium related to arithmetic
over Edwards25519 also do that check unconditionally. This is
expensive, but a good way to catch bugs in protocols and
implementations.
So, add a `rejectUnexpectedSubgroup()` function to achieve this.
The documentation on the edwards25519->curve25519 conversion
function was also updated, in order to explain how to match
libsodium's behavior if necessary.
We use an addition chain to multiply the point by the order of
the prime group.
An alternative we may implement later is Pornin's point halving
technique: https://eprint.iacr.org/2022/1164.pdf
