Commit graph

1395 commits

Author SHA1 Message Date
Igor Anić
be4a77d9aa io_uring: fix ci failing test
Getting this error in ci:
run test std-arm-linux-none-generic-Debug: error: 'test.accept/connect/send_zc/recv' failed: /home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:60:23: 0x70b06b in init_params (test)
            .NOSYS => return error.SystemOutdated,
                      ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:27:16: 0x70b6b7 in init (test)
        return try IO_Uring.init_params(entries, &params);
               ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:3807:16: 0x72405b in test.accept/connect/send_zc/recv (test)
    var ring = try IO_Uring.init(16, 0);

https://github.com/ziglang/zig/actions/runs/6909813408/job/18801841015?pr=18025
2023-11-18 05:40:24 +01:00
Igor Anić
7eee0d3353 io_uring: try to pass windows/macos tests 2023-11-17 23:39:29 +01:00
Igor Anić
ccf5a6cc5c io_uring: make Linux version check runtime instead od comptime
Reverting previous change.
I'm building test bin and then running it in virtual machines with different
kernels. So Linux kernel checks has to be runtime instead of comptime.
2023-11-17 22:17:00 +01:00
Igor Anić
a7001b86f1 io_uring: include review comments
Thanks @rootbeer for review.
This adds description to send_zc behavior.
Cleans up tests.
2023-11-17 19:58:23 +01:00
Igor Anić
257b8131ec io_uring: use kernel version test in send_zc test 2023-11-17 17:21:54 +01:00
Igor Anić
87dd8d5669 io_uring: simplify tests by checking kernel version
So far we relied on getting EINVAL in CQE for operations that kernel don't
support. The problem with that approach is that there are many other reasons
(like wrong params) to get EINVAL. The other problem is when we have an
operation that existed before and gets new behavior via different attributes,
like accept and accept_direct. Older kernels can fall back to non direct
operation although we set attributes for direct operation. Operation completes
successfully in both cases but with different results.

This commit introduces kernel version check at the start of the test. Making
body of the test free of checking for various kernel version differences.

Feature availability references:
  * https://manpages.debian.org/unstable/liburing-dev/io_uring_enter.2.en.html
  * https://kernel.dk/axboe-kr2022.pdf
  * 5acf7969bc/lib/std/os/linux.zig (L3727)
  * 5acf7969bc/lib/std/os/linux.zig (L3993)
2023-11-17 16:31:57 +01:00
Igor Anić
256384a2ec io_uring: add direct operations
Add operation on direct file descriptors. Also referred to as fixed or
registered files.

References:
https://kernel.dk/axboe-kr2022.pdf
https://lwn.net/Articles/863071/

Added functions:

IO_Uring
  accept_direct
  accept_multishot_direct
  openat_direct
  close_direct
  socket
  socket_direct
  socket_direct_alloc

Raw prepare operations:
  io_uring_prep_accept_direct
  io_uring_prep_multishot_accept_direct
  io_uring_prep_openat_direct
  io_uring_prep_close_direct
  io_uring_prep_socket
  io_uring_prep_socket_direct
  io_uring_prep_socket_direct_alloc

Tested on this kernels:

5.4.0-164-generic
2559 passed; 70 skipped; 0 failed.

5.8.0-63-generic
2573 passed; 56 skipped; 0 failed.

5.11.0-49-generic
2576 passed; 53 skipped; 0 failed.

5.13.0-52-generic
2576 passed; 53 skipped; 0 failed.

5.15.0-87-generic
2579 passed; 50 skipped; 0 failed.

5.19.0-46-geneic
2584 passed; 45 skipped; 0 failed.

6.2.0-35-generic.log
2585 passed; 44 skipped; 0 failed.

6.5.0-9-generic
2585 passed; 44 skipped; 0 failed.
2023-11-17 13:48:17 +01:00
David
941090d94f
Move duplicate field detection for struct init expressions into AstGen
Partially addresses #17916.
2023-11-16 14:38:16 +00:00
Igor Anić
087ee497d4 io_uring: add zero-copy send operation
`send_zc` tries to avoid making intermediate copies of data. Zerocopy execution
is not guaranteed and may fall back to copying.

The flags field of the first struct io_uring_cqe may likely contain
IORING_CQE_F_MORE , which means that there will be a second completion event /
notification for the request, with the user_data field set to the same value.
The user must not modify the data buffer until the notification is posted. The
first cqe follows the usual rules and so its res field will contain the number
of bytes sent or a negative error code. The notification's res field will be set
to zero and the flags field will contain IORING_CQE_F_NOTIF. The two step model
is needed because the kernel may hold on to buffers for a long time, e.g.
waiting for a TCP ACK, and having a separate cqe for request completions allows
userspace to push more data without extra delays. Note, notifications are only
responsible for controlling the lifetime of the buffers, and as such don't mean
anything about whether the data has atually been sent out or received by the
other end. Even errored requests may generate a notification, and the user must
check for IORING_CQE_F_MORE rather than relying on the result.

Available since kernel 6.0.

References:
https://man7.org/linux/man-pages/man3/io_uring_prep_send_zc.3.html
https://man7.org/linux/man-pages/man2/io_uring_enter.2.html
2023-11-10 16:44:18 +01:00
Bogdan Romanyuk
ee8da40769 replace deprecated std.math.absCast 2023-11-09 17:39:27 +02:00
Igor Anić
715e5f757f fix io_uring tests on kernel 5.4
Unsupported tests are now skipped on kernel 5.4:
```
uname -a
Linux d20 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:31:37 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
zig test lib/std/std.zig --zig-lib-dir lib --main-mod-path lib/std 2>&1 | cat
709/2616 test.nop... OK
710/2616 test.readv... OK
711/2616 test.writev/fsync/readv... OK
712/2616 test.write/read... SKIP
713/2616 test.splice/read... SKIP
714/2616 test.write_fixed/read_fixed... OK
715/2616 test.openat... SKIP
716/2616 test.close... SKIP
717/2616 test.accept/connect/send/recv... SKIP
718/2616 test.sendmsg/recvmsg... OK
719/2616 test.timeout (after a relative time)... SKIP
720/2616 test.timeout (after a number of completions)... OK
721/2616 test.timeout_remove... SKIP
722/2616 test.accept/connect/recv/link_timeout... SKIP
723/2616 test.fallocate... SKIP
724/2616 test.statx... SKIP
725/2616 test.accept/connect/recv/cancel... SKIP
726/2616 test.register_files_update... SKIP
727/2616 test.shutdown... SKIP
728/2616 test.renameat... SKIP
729/2616 test.unlinkat... SKIP
730/2616 test.mkdirat... SKIP
731/2616 test.symlinkat... SKIP
732/2616 test.linkat... SKIP
733/2616 test.provide_buffers: read... SKIP
734/2616 test.remove_buffers... SKIP
735/2616 test.provide_buffers: accept/connect/send/recv... SKIP
736/2616 test.accept multishot... SKIP
```
2023-11-08 18:52:03 +01:00
Igor Anić
7a1dbbd4f9 fix io_uring timeout_remove test on kernel 5.4
There is no grantee that `copy_cqes` will return exactly wait_nr number of cqes.
If there are ready cqes it can return > 0 but < wait_nr number of cqes.
2023-11-08 18:25:29 +01:00
Igor Anić
bec36aa7c0 io_uring: add multishot accept operation
Server networking application typically accept multiple connections. Multishot
accept simplifies handling these situations. Applications submits once and
receives CQE whenever a new connection request comes in.

Multishot is active until it is canceled or experience error. While active, and
further notification are expected CQE completion will have IORING_CQE_F_MORE set
in the flags. If this flag isn't set, the application must re-arm this request
by submitting a new one.

Reference: [io_uring and networking in 2023](https://github.com/axboe/liburing/wiki/io_uring-and-networking-in-2023#multi-shot)
2023-11-06 00:12:39 +02:00
Jacob Young
509be7cf1f x86_64: fix std test failures 2023-11-03 23:18:21 -04:00
Ryan Liptak
ae6df9e967 start.zig: Replace kernel32 usage with ntdll
Co-authored-by: e4m2 <git@e4m2.com>
2023-11-03 19:25:46 +02:00
XXIV
c2cf47ab28
std.os.windows: remove unnecessary @ptrCast (#17816) 2023-11-01 18:45:51 -04:00
Kai Jellinghaus
7ee7c90274 Fix regressing #17290 from #17734 2023-11-01 18:30:56 +02:00
Andrew Kelley
3fc6fc6812 std.builtin.Endian: make the tags lower case
Let's take this breaking change opportunity to fix the style of this
enum.
2023-10-31 21:37:35 -04:00
Jacob Young
d890e81761 mem: fix ub in writeInt
Use inline to vastly simplify the exposed API.  This allows a
comptime-known endian parameter to be propogated, making extra functions
for a specific endianness completely unnecessary.
2023-10-31 21:37:35 -04:00
Kamil T
37295696ec std.os.windows additions and fixes 2023-10-29 11:01:54 +02:00
Andrew Kelley
e7ead9c5ea std: delete psapi, gdi32, and winmm
The only remaining Windows APIs now are the ones that the standard
library itself depends on for its cross-platform abstractions.

closes #4426
2023-10-29 00:58:00 -04:00
Andrew Kelley
d817a3c517 delete std.os.windows.user32
This is progress towards #4426
Closes #17417
2023-10-28 13:16:47 -04:00
Jacob Young
3b0dce8ebd
Merge pull request #17716 from jacobly0/x86_64
x86_64: pass more tests
2023-10-27 08:29:24 -04:00
Jacob Young
98cd378208 x86_64: fix behavior of getValue
Old behavior renamed to `getValueIfFree`.
2023-10-26 21:45:58 -04:00
Jacob Young
5e83441096 x86_64: implement @divFloor and @mod for i128 2023-10-26 21:45:57 -04:00
joadnacer
28b848e3f0 std.io_uring: Improve splice implementation 2023-10-27 00:31:55 +01:00
joadnacer
0005e7e70b std.linux: Update io_uring structs and consts for kernel 6.3.8 2023-10-27 00:31:52 +01:00
Jacob Young
b55377a5ab x86_64: pass more tests
* 128-bit integer multiplication with overflow
 * more instruction encodings used by std inline asm
 * implement the `try_ptr` air instruction
 * follow correct stack frame abi
 * enable full panic handler
 * enable stack traces
2023-10-25 04:28:30 -04:00
Jacob Young
8f69e977f1 x86_64: implement 128-bit builtins
* `@clz`
 * `@ctz`
 * `@popCount`
 * `@byteSwap`
 * `@bitReverse`
 * various encodings used by std
2023-10-23 22:42:18 -04:00
Jacob Young
27fe945a00 Revert "Revert "Merge pull request #17637 from jacobly0/x86_64-test-std""
This reverts commit 6f0198cadb.
2023-10-22 15:46:43 -04:00
Andrew Kelley
6f0198cadb Revert "Merge pull request #17637 from jacobly0/x86_64-test-std"
This reverts commit 0c99ba1eab, reversing
changes made to 5f92b070bf.

This caused a CI failure when it landed in master branch due to a
128-bit `@byteSwap` in std.mem.
2023-10-22 12:16:35 -07:00
Jacob Young
32e85d44eb x86_64: disable failing tests, enable test-std testing 2023-10-21 10:55:41 -04:00
Jacob Young
2e6e39a700 x86_64: fix bugs and disable erroring tests 2023-10-21 10:55:41 -04:00
joadnacer
89aad1b89c linux.io_uring: Implement splice functions 2023-10-21 05:02:17 +03:00
Prokop Randacek
fee09009a8 gettimeofday takes nullable pointers 2023-10-20 00:15:34 -04:00
Ben Crist
6d47198303 return error.AlreadyTerminated from std.ChildProcess.kill when necessary 2023-10-17 16:56:15 -07:00
kcbanner
68ed787751 windows: fix ReadLink not using a synchronous file handle 2023-10-17 14:33:55 -04:00
Andrew Kelley
877393d17a std.fs: fix relative symbolic links on Windows
closes #17564
2023-10-17 06:27:07 -04:00
Ratakor
8ce33795e9 Add pause() to linux.zig 2023-10-06 11:49:31 -07:00
Ratakor
cef90eab57 Add filled_sigset to os.linux
filled_sigset is equivalent to sigfillset() as empty_sigset is
equivalent to sigemptyset().
2023-10-06 11:49:31 -07:00
Jacob Young
2a5335d7b6 x86_64: implement C abi for 128-bit integers 2023-10-04 14:42:35 -04:00
nikneym
9f0d2f9417 linux: add fanotify API 2023-10-04 13:48:22 +03:00
Kai Jellinghaus
11489bb04f
Update IORING_OP to reflect upstream (#17388)
Reference [upstream io_uring.h](cbf3a2cb15/include/uapi/linux/io_uring.h (L234))
2023-10-04 09:18:14 +00:00
Stephen Gregoratto
285970982a Add illumos OS tag
- Adds `illumos` to the `Target.Os.Tag` enum. A new function,
  `isSolarish` has been added that returns true if the tag is either
  Solaris or Illumos. This matches the naming convention found in Rust's
  `libc` crate[1].
- Add the tag wherever `.solaris` is being checked against.
- Check for the C pre-processor macro `__illumos__` in CMake to set the
  proper target tuple. Illumos distros patch their compilers to have
  this in the "built-in" set (verified with `echo | cc -dM -E -`).

  Alternatively you could check the output of `uname -o`.

Right now, both Solaris and Illumos import from `c/solaris.zig`. In the
future it may be worth putting the shared ABI bits in a base file, and
mixing that in with specific `c/solaris.zig`/`c/illumos.zig` files.

[1]: 6e02a329a2/src/unix/solarish
2023-10-02 15:31:49 -06:00
Stephen Gregoratto
bc0bf4e97a Linux: Add cachestat wrapper.
Can be tested using this program I whipped up:
https://gist.github.com/The-King-of-Toasters/aee448f5975c50f735fd1946794574f7
2023-09-28 18:58:05 +10:00
Stephen Gregoratto
5f456b2b97 Update Linux syscalls for kernel 5.5
The latest addition is `cachestat`, which provides more detailed
information for paged files.
2023-09-28 18:58:05 +10:00
Kai Jellinghaus
d1e39b6914 Add new fields to io_sqring_offsets & io_cqring_offsets
`user_addr`s were introduced in `03d89a2` ([github link](03d89a2de2) which was shipped in v6.5
`flags` was introduced even earlier
2023-09-26 18:16:36 -07:00
Aven Bross
6c54ed73f7 Added basic support for wasm32-emscripten target 2023-09-22 12:49:03 -07:00
nikneym
ada02ac6fe wasi: update sock_recv and sock_send according to preview1 2023-09-18 00:27:27 -07:00
bfredl
1a0e6bcdb1 bpf: remove unhelpful "comptime" and fix union order
Insn.st() can be used with dynamic size just like Insn.stx(), which is
relevant in a code generation context.

using ImmOrReg caused an error as its fields were ordered differently than
Source.
2023-09-15 01:05:52 -07:00