Usage of FILE_RENAME_IGNORE_READONLY_ATTRIBUTE or
FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE for posix semantics require
win10_rs5 instead of win10_rs1 necessary for posix semantics. Keep it as simple
as possible, since it is reasonable to expect users being able to update
win10_rs5 or use non-posix semantics instead.
Closes#17049.
This matches how other filesystem functions were made to handle BAD_NETWORK_PATH/BAD_NETWORK_NAME in https://github.com/ziglang/zig/pull/16568. ReadLink was the odd one out, but that is no longer the case.
This fixes a few things:
- Previously, CreateSymbolicLink would always create a relative link if a `dir` was provided, but the relative-ness of a link should be determined by the target path, not the null-ness of the `dir`.
- Special handling is now done to symlink to 'rooted' paths correctly (they are treated as a relative link, which is different than how the xToPrefixedFileW functions treat them)
- ReadLink now correctly supports UNC paths via a new `ntToWin32Namespace` function which intends to be an analog of `RtlNtPathNameToDosPathName` (RtlNtPathNameToDosPathName is not used because it seems to heap allocate as it takes an RTL_UNICODE_STRING_BUFFER)
Previously, a relative path like `..` would:
- Attempt to be normalized (i.e. remove . and .. without any path resolution), but would error with TooManyParentDirs
- This would make wToPrefixedFileW run it through `RtlGetFullPathName_U` to do the necessary path resolution, but `RtlGetFullPathName_U` always resolves relative paths relative to the CWD
Instead, when TooManyParentDirs occurs, we now look up the path of the passed in `dir` (if it's non-null) and append the relative path to it before giving it to `RtlGetFullPathName_U`. If `dir` is null, then we just give it RtlGetFullPathName_U directly and let it resolve it relative to the CWD.
Closes#16779
The C++ version of this code used this logic, and it turns out it is able to find some setups that the current registry/Vs7 methods cannot.
For example, if only the "Build Tools for Visual Studio" are installed but not Visual Studio itself, then only the ISetupEnumInstances method seems to find it.
Follow up to #15657, fixes a regression caused by moving from the C++ version to the Zig version
Before this commit, there were three issues with the makePath implementation:
1. The component iteration did not 'collapse' consecutive path separators; instead, it would treat `a/b//c` as `a/b//c`, `a/b/`, `a/b`, and `a`.
2. Trailing path separators led to redundant `makeDir` calls, e.g. with the path `a/b/` (if `a` doesn't exist), it would try to create `a/b/`, then try `a/b`, then try `a`, then try `a/b`, and finally try `a/b/` again.
3. The iteration did not treat the root of a path specially, so on Windows it could attempt to make a directory with a path like `X:` for something like `X:\a\b\c` if the `X:\` drive doesn't exist. This didn't lead to any problems that I could find, but there's no reason to try to make a drive letter as a directory (or any other root path).
This commit fixes all three issues by introducing a ComponentIterator that is root-aware and handles both sequential path separators and trailing path separators and uses it in `Dir.makePath`. This reduces the number of `makeDir` calls for paths where (1) the root of the path doesn't exist, (2) there are consecutive path separators, or (3) there are trailing path separators
As an example, here are the makeDir calls that would have been made before this commit when calling `makePath` for a relative path like `a/b//c//` (where the full path needs to be created):
a/b//c//
a/b//c/
a/b//c
a/b/
a/b
a
a/b
a/b/
a/b//c
a/b//c/
a/b//c//
And after this commit:
a/b//c
a/b
a
a/b
a/b//c
When calling NtCreateFile with a UNC path, if either `\\server` or `\\server\share` are not found, then the statuses `BAD_NETWORK_PATH` or `BAD_NETWORK_NAME` are returned (respectively).
These statuses are not translated into `error.FileNotFound` because they convey more information than the typical FileNotFound error. For example, if you were trying to call `Dir.makePath` with an absolute UNC path like `\\MyServer\MyShare\a\b\c\d`, then knowing that `\\MyServer\MyShare` was not found allows for returning after trying to create the first directory instead of then trying to create `a\b\c`, `a\b`, etc. when it's already known that they will all fail in the same way.
This fixes a regression caused by https://github.com/ziglang/zig/pull/13993
As an optimization, the first call to `NtQueryDirectoryFile` would only ask for a single result and assume that if the result returned did not match the app_name exactly, then the unappended app_name did not exist. However, this relied on the assumption that the unappended app_name would always be returned first, but that only seems to be the case on NTFS. On FAT filesystems, the order of returned files can be different, which meant that it could assume the unappended file doesn't exist when it actually does.
This commit fixes that by fully iterating the wildcard matches via `NtQueryDirectoryFile` and taking note of any unappended/PATHEXT-appended filenames it finds. In practice, this strategy does not introduce a speed regression compared to the previous (buggy) implementation.
Benchmark 1 (10 runs): winpathbench-master.exe
measurement mean ± σ min … max outliers delta
wall_time 508ms ± 4.08ms 502ms … 517ms 1 (10%) 0%
peak_rss 3.62MB ± 2.76KB 3.62MB … 3.63MB 0 ( 0%) 0%
Benchmark 2 (10 runs): winpathbench-fat32-fix.exe
measurement mean ± σ min … max outliers delta
wall_time 500ms ± 21.4ms 480ms … 535ms 0 ( 0%) - 1.5% ± 2.8%
peak_rss 3.62MB ± 2.76KB 3.62MB … 3.63MB 0 ( 0%) - 0.0% ± 0.1%
---
Partially addresses #16374 (it fixes `zig build` on FAT32 when no `zig-cache` is present)
Using FileDispositionInformationEx (and therefore flags like FILE_DISPOSITION_POSIX_SEMANTICS and FILE_DISPOSITION_IGNORE_READONLY_ATTRIBUTE) is only supported on NTFS, so the comptime Windows version range check is not enough to determine whether or not the NtSetInformationFile will succeed.
This commit makes DeleteFile always try using FileDispositionInformationEx first, but if INVALID_PARAMETER is received (which is the status that occurs when the filesystem doesn't support FileDispositionInformationEx), then it will fallback and try calling NtSetInformationFile with FileDispositionInformation.
This keeps NTFS as fast as it was before, since it will do at most 1 NtSetInformationFile call, but on non-NTFS filesystems (e.g. FAT32), DeleteFile may need to do 2 NtSetInformationFile calls.
Closes#16497
I had accidentally regressed support for -gdwarf in 461fb499f3 when I changed the logic to
use the already-mapped exe/dll image instead of loading it from disk. The string table is mapped as all zeroes by the loader,
so if a section header's name is longer than 8 bytes (like the ones generated by -gdwarf), then the name can't be read.
Now, if any section headers require the string table, the file is mapped from disk.
windows: Add NtCreateSection/NtMapViewOfSection/NtUnmapViewOfSection
This allows doing Windows-style case insensitive comparisons from any target, but means that it will need to include its own copy of the uppercase data table (5,088 bytes) to do so.
When targeting Windows, the ntdll functions are used instead to avoid including a redundant copy of the uppercase data in the resulting binary.
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:
* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
There are many different types of Windows paths, and there are a few different possible namespaces on top of that. Before this commit, NT namespaced paths were somewhat supported, and for Win32 paths (those without a namespace prefix), only relative and drive absolute paths were supported. After this commit, all of the following are supported:
- Device namespaced paths (`\\.\`)
- Verbatim paths (`\\?\`)
- NT-namespaced paths (`\??\`)
- Relative paths (`foo`)
- Drive-absolute paths (`C:\foo`)
- Drive-relative paths (`C:foo`)
- Rooted paths (`\foo`)
- UNC absolute paths (`\\server\share\foo`)
- Root local device paths (`\\.` or `\\?` exactly)
Plus:
- Any of the path types and namespace types can be mixed and matched together as appropriate.
- All of the `std.os.windows.*ToPrefixedFileW` functions will accept any path type, prefixed or not, and do the appropriate thing to convert them to an NT-prefixed path if necessary.
This is achieved by making the `std.os.windows.*ToPrefixedFileW` functions behave like `ntdll.RtlDosPathNameToNtPathName_U`, but with a few differences:
- Does not allocate on the heap (this is why we can't use `ntdll.RtlDosPathNameToNtPathName_U` directly, it does internal heap allocation).
- Relative paths are kept as relative unless they contain too many .. components, in which case they are treated as 'drive relative' and resolved against the CWD (this is how it behaved before this commit as well).
- Special case device names like COM1, NUL, etc are not handled specially (TODO)
- `.` and space are not stripped from the end of relative paths (potential TODO)
Most of the non-trivial conversion of non-relative paths is done via `ntdll.RtlGetFullPathName_U`, which AFAIK is used internally by `ntdll.RtlDosPathNameToNtPathName_U`.
Some relevant reading on Windows paths:
- https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html
- https://chrisdenton.github.io/omnipath/Overview.htmlCloses#8205
Might close (untested) #12729
Note:
- This removes checking for illegal characters in `std.os.windows.sliceToPrefixedFileW`, since the previous solution (iterate the whole string and error if any illegal characters were found) was naive and won't work for all path types. This is further complicated by things like file streams (where `:` is used as a delimiter, e.g. `file.ext:stream_name:$DATA`) and things in the device namespace (where a path like `\\.\GLOBALROOT\??\UNC\localhost\C$\foo` is valid despite the `?`s in the path and is effectively equivalent to `C:\foo`). Truly validating paths is complicated and would need to be tailored to each path type. The illegal character checking being removed may open up users to more instances of hitting `OBJECT_NAME_INVALID => unreachable` when using `fs` APIs.
+ This is related to https://github.com/ziglang/zig/issues/15607
FILE_DISPOSITION_ON_CLOSE is used to set/clear the FILE_DELETE_ON_CLOSE,
but we do not use that anymore and FILE_DISPOSITION_POSIX_SEMANTICS
already implies unmapping of the handle and removal fo it on close.
Justification: When a file is deleted on Windows, it may not be
immediately removed from the directory. This can cause problems
with future scans of that directory, which will see the partially
deleted file. Under some workloads and system configurations,
Windows files may appear to be deleted immediately.
This is the PR with requested fixup. Thanks to @SpexGuy for the
original PR.
The majority of these are in comments, some in doc comments which might
affect the generated documentation, and a few in parameter names -
nothing that should be breaking, however.
Now they use slices or array pointers with any element type instead of
requiring byte pointers.
This is a breaking enhancement to the language.
The safety check for overlapping pointers will be implemented in a
future commit.
closes#14040
DELETE_PENDING can happen when the file is yet to be closed for deletion
or if it never get closed. In that case, DeleteFile should assume the
file deletion is succeeding (no CloseHandle is required as it's a "failure"). In case of `DELETE_PENDING` failure, the file
may still exist. In which case if it's part of `deleteTree`, it will
eventually fail on `error.DirNotEmpty`.
Using `FILE_DELETE_ON_CLOSE` can silently succeed without reporting any error
on non-empty directory. This commit adds usage of NtSetInformationFile
which will report `DIRECTORY_NOT_EMPTY`.
`GetPhysicallyInstalledSystemMemory` uses SMBios to grab the physical
memory size which can lead to unecessary allocation and inacurate
representation of the total memory. Using `System_Basic_Information`
help to retrieve the physical memory which is not reserved for the
kernel/tables. This aligns better with the linux side as `/proc/meminfo`
does the same thing.
`GetProcessMemoryInfo` is implemented using `NtQueryInformationProcess`
with `ProcessVmCounters` to obtain `VM_COUNTERS`. The structs, enum
definitions are found in `winternl.h` or `ntddk.h` in the latest WDK.
This should give the same results as using `K32GetProcessMemoryInfo`
* Fix GetFileInformationByHandle compile error
The wrapper function was mistakenly referencing ntdll.zig when the actual function is declared in kernel32.zig.
* delete GetFileInformationByHandle since it's not used by the stdlib
This function is unused, and the current implementation contains a few footguns:
- The current wrapper treats all possible errors as unexpected, even likely ones like BUFFER_OVERFLOW (which is returned if the size of the out_buffer is too small to contain all the variable-length members of the requested info, which the user may not actually care about)
- Each caller may need to handle errors differently, different errors might be possible depending on the FILE_INFORMATION_CLASS, etc, and making a wrapper that handles all of those different use-cases nicely seems like it'd be more trouble than it's worth (FILE_INFORMATION_CLASS has 76 different possible values)
If a wrapper for NtQueryInformationFile is wanted, then it should probably have wrapper functions per-use-case, like how QueryObjectName wraps NtQueryObject for the `ObjectNameInformation` class
This error means that there *was* a file in this location on the file
system, but it was deleted. However, the OS is not finished with the
deletion operation, and so this CreateFile call has failed. There is not
really a sane way to handle this other than retrying the creation after
the OS finishes the deletion.
- Fixes the first few code units of the name being omitted (it was using `@sizeOf(FILE_NAME_INFO)` as the start of the name bytes, but that includes the length of the dummy [1]u16 field and padding; instead the start should be the offset of the dummy [1]u16 field)
- Replaces kernel32.GetFileInformationByHandleEx call with ntdll.NtQueryInformationFile
+ Contributes towards #1840
- Checks that the handle is a named pipe first before querying and checking the name, which is a much faster call than NtQueryInformationFile (this was about a 10x speedup in my probably-not-so-good/take-it-with-a-grain-of-salt benchmarking)
