The motivating problem here was a memory leak in the hash maps of
Module.Namespace.
The commit deletes more of the legacy incremental compilation
implementation. It had things like use of orderedRemove and trying to do
too much OOP-style creation and deletion of objects.
Instead, this commit iterates over all the namespaces on Module deinit
and calls deinit on the hash map fields. This logic is much simpler to
reason about.
Similarly, change global inline assembly to an array hash map since
iterating over the values is a primary use of it, and clean up the
remaining values on Module deinit, solving another memory leak.
After this there are no more memory leaks remaining when using the
x86 backend in a libc-less compiler.
This incremental compilation logic will need to be reworked so that it
does not depend on buried pointers - that is, long-lived pointers that
are owned by non-top-level objects such as Decl.
In the meantime, this fixes memory leaks since the memory management of
these dependencies has bitrotted.
- Add resolveUnionAlignment, to resolve a union's alignment only, without triggering layout resolution.
- Update resolveUnionLayout to cache size, alignment, and padding. abiSizeAdvanced and abiAlignmentAdvanced
now use this information instead of computing it each time.
This commit starts by making Zir.Inst.Index a nonexhaustive enum rather
than a u32 alias for type safety purposes, and the rest of the changes
are needed to get everything compiling again.
This field had the wrong type. It's not a `Zir.Inst.Index`, it's
actually a `Zir.OptionalExtraIndex`. Also, the former is currently
aliased to `u32` while the latter is a nonexhaustive enum that gives us
more type checking.
This commit is preparation for making this field non-optional. Now it
can be changed to `Zir.ExtraIndex` and then the compiler will point out
all the places that the non-optional assumption is being violated.
The main motivating change here is to prevent the creation of a fake
Decl object by the frontend in order to `@export()` a value.
Instead, `link.updateDeclExports` is renamed to `link.updateExports` and
accepts a tagged union which can be either a Decl.Index or a
InternPool.Index.
* 128-bit integer multiplication with overflow
* more instruction encodings used by std inline asm
* implement the `try_ptr` air instruction
* follow correct stack frame abi
* enable full panic handler
* enable stack traces
The main goal of this commit is to remove the `runtime_value` field from
`InternPool.Key` (and its associated representation), but there are a
few dominos. Specifically, this mostly eliminates the "maybe runtime"
concept from value resolution in Sema: so some resolution functions like
`resolveMaybeUndefValAllowVariablesMaybeRuntime` are gone. This required
a small change to struct/union/array initializers, to no longer
use `runtime_value` if a field was a `variable` - I'm not convinced this
case was even reachable, as `variable` should only ever exist as the
trivial value of a global runtime `var` decl.
Now, the only case in which a `Sema.resolveMaybeUndefVal`-esque function
can return the `variable` key is `resolveMaybeUndefValAllowVariables`,
which is directly called from `Sema.resolveInstValueAllowVariables`
(previously `Sema.resolveInstValue`), which is only used for resolving
the value of a Decl from `Module.semaDecl`.
While changing these functions, I also slightly reordered and
restructured some of them, and updated their doc comments.
This feature was made to work with the legacy incremental compilation
mechanism which is being reworked.
This commit regresses the ability to update files used with `@embedFile`
while the compiler is running.
In exchange, we get these benefits:
* The embedded file contents are copied directly into InternPool rather
than there being an extra allocation and memcpy.
* The EmbedFile struct, which represents a long-lived object, is made
more serialization friendly.
* Eliminate the creation and use of a Decl as an anonymous decl.
When implementing the new incremental compilation mechanism,
functionality will need to be added back for handling `@embedFile`.
* add Module instances for each package's build.zig and attach it to the
dependencies.zig module with the hash digest hex string as the name.
* fix incorrectly skipping the wrong packages for creating
dependencies.zig
* a couple more renaming of "package" to "module"
Finish the work started in 4c4fb839972f66f55aa44fc0aca5f80b0608c731.
Now the compiler compiles again.
Wire up dependency tree fetching code in the CLI for `zig build`.
Everything is hooked up except for `createDependenciesModule` is not yet
implemented.
* start renaming "package" to "module" (see #14307)
- build system gains `main_mod_path` and `main_pkg_path` is still
there but it is deprecated.
* eliminate the object-oriented memory management style of what was
previously `*Package`. Now it is `*Package.Module` and all pointers
point to externally managed memory.
* fixes to get the new Fetch.zig code working. The previous commit was
work-in-progress. There are still two commented out code paths, the
one that leads to `Compilation.create` and the one for `zig build`
that fetches the entire dependency tree and creates the required
modules for the build runner.
Instead of explicitly creating a `Module.Decl` object for each anonymous
declaration, each `InternPool.Index` value is implicitly understood to
be an anonymous declaration when encountered by backend codegen.
The memory management strategy for these anonymous decls then becomes to
garbage collect them along with standard InternPool garbage.
In the interest of a smooth transition, this commit only implements this
new scheme for string literals and leaves all the previous mechanisms in
place.
My previous change for reading / writing to unions at comptime did not handle
union field read/writes correctly in all cases. Previously, if a field was
written to a union, it would overwrite the entire value. This is problematic
when a field of a larger size is subsequently read, because the value would not
be long enough, causing a panic.
Additionally, the writing behaviour itself was incorrect. Writing to a field of
a packed or extern union should only overwrite the bits corresponding to that
field, allowing for memory reintepretation via field writes / reads.
I addressed these problems as follows:
Add the concept of a "backing type" for extern / packed unions
(`Type.unionBackingType`). For extern unions, this is a `u8` array, for packed
unions it's an integer matching the `bitSize` of the union. Whenever union
memory is read at comptime, it's read as this type.
When union memory is written at comptime, the tag may still be known. If so, the
memory is written using the tagged type. If the tag is unknown (because this
union had previously been read from memory), it's simply written back out as the
backing type.
I added `write_packed` to the `reinterpret` field of
`ComptimePtrMutationKit`. This causes writes of the operand to be packed - which
is necessary when writing to a field of a packed union. Without this, writing a
value to a `u1` field would overwrite the entire byte it occupied.
The final case to address was reading a different (potentially larger) field
from a union when it was written with a known tag. To handle this, a new kind of
bitcast was introduced (`bitCastUnionFieldVal`) which supports reading a larger
field by using a backing buffer that has the unwritten bits set to
undefined. The reason to support this (vs always just writing the union as it's
backing type), is that no reads to larger fields ever occur at comptime, it
would be strictly worse to have spent time writing the full backing type.
Instead of linear search every time a packed struct field's bit or byte
offset is wanted, they are computed once during resolution of the packed
struct's backing int type, and stored in InternPool for O(1) lookup.
Closes#17178
When the tag is not known, it's set to `.none`. In this case, the value is either an
array of bytes (for extern unions) or an integer (for packed unions).
Previously it would canonicalize or not depending on some volatile
internal state of the compiler, now it forces resolution of the element
type to determine the alignment if it needs to.
Structs were previously using `SegmentedList` to be given indexes, but
were not actually backed by the InternPool arrays.
After this, the only remaining uses of `SegmentedList` in the compiler
are `Module.Decl` and `Module.Namespace`. Once those last two are
migrated to become backed by InternPool arrays as well, we can introduce
state serialization via writing these arrays to disk all at once.
Unfortunately there are a lot of source code locations that touch the
struct type API, so this commit is still work-in-progress. Once I get it
compiling and passing the test suite, I can provide some interesting
data points such as how it affected the InternPool memory size and
performance comparison against master branch.
I also couldn't resist migrating over a bunch of alignment API over to
use the log2 Alignment type rather than a mismash of u32 and u64 byte
units with 0 meaning something implicitly different and special at every
location. Turns out you can do all the math you need directly on the
log2 representation of alignments.
* Use 32-bit integers instead of pointers for compactness and
serialization friendliness.
* Use a separate hash map for runtime and comptime capture scopes,
avoiding the 1-bit union tag.
* Use a compact array representation instead of a tree of hash maps.
* Eliminate the only instance of ref-counting in the compiler, instead
relying on garbage collection (not implemented yet but is the plan for
almost all long-lived objects related to incremental compilation).
Because a code modification may need to access capture scope data, this
makes capture scope data long-lived state. My goal is to get incremental
compilation state serialization down to a single pwritev syscall, by
unifying the on-disk representation with the in-memory representation.
This commit eliminates the last remaining pointer field of
`Module.Decl`.
This makes the call sites easier to read, reduces the number of `catch`
expressions required, and prepares for comptime reasons to appear
earlier in the list of notes.
There are a couple concepts here worth understanding:
Key.UnionType - This type is available *before* resolving the union's
fields. The enum tag type, number of fields, and field names, field
types, and field alignments are not available with this.
InternPool.UnionType - This one can be obtained from the above type with
`InternPool.loadUnionType` which asserts that the union's enum tag type
has been resolved. This one has all the information available.
Additionally:
* ZIR: Turn an unused bit into `any_aligned_fields` flag to help
semantic analysis know whether a union has explicit alignment on any
fields (usually not).
* Sema: delete `resolveTypeRequiresComptime` which had the same type
signature and near-duplicate logic to `typeRequiresComptime`.
- Make opaque types not report comptime-only (this was inconsistent
between the two implementations of this function).
* Implement accepted proposal #12556 which is a breaking change.
The key changes in this commit are:
```diff
- names: []const NullTerminatedString,
+ names: NullTerminatedString.Slice,
- values: []const Index,
+ values: Index.Slice,
```
Which eliminates the slices from `InternPool.Key.EnumType` and replaces
them with structs that contain `start` and `len` indexes. This makes the
lifetime of `EnumType` change from expiring with updates to InternPool,
to expiring when the InternPool is garbage-collected, which is currently
never.
This is gearing up for a larger change I started working on locally
which moves union types into InternPool.
As a bonus, I fixed some unnecessary instances of `@as`.
Some builtin types have a special InternPool index (e.g.
`.type_info_type`) so that AstGen can refer to them before semantic
analysis. Unfortunately, this previously led to a second index existing
to refer to the type once it was resolved, complicating Sema by having
the concept of an "unresolved" type index.
This change makes Sema modify these InternPool indices in-place to
contain the expanded representation when resolved. The analysis of the
corresponding decls is caught in `Module.semaDecl`, and a field is set
on Sema telling it which index to place struct/union/enum types at. This
system could break if `std.builtin` contained complex decls which
evaluate multiple struct types, but this will be caught by the
assertions in `InternPool.resolveBuiltinType`.
The AstGen result types which were disabled in 6917a8c have been
re-enabled.
Resolves: #16603
After ff37ccd, interned values are trivial to convert to Air refs, using
`Air.internedToRef`. This made functions like `Sema.addConstant` effectively
redundant. This commit removes `Sema.addConstant` and `Sema.addType`, replacing
them with direct usages of `Air.internedToRef`.
Additionally, a new helper `Module.undefValue` is added, and the following
functions are moved into Module:
* `Sema.addConstUndef` -> `Module.undefRef`
* `Sema.addUnsignedInt` -> `Module.intRef` (now also works for signed types)
The general pattern here is that any `Module.xyzValue` helper may also have a
corresponding `Module.xyzRef` helper, which just wraps the call in
`Air.internedToRef`.
AstGen provides all function call arguments with a result location,
referenced through the call instruction index. The idea is that this
should be the parameter type, but for `anytype` parameters, we use
generic poison, which is required to be handled correctly.
Previously, generic instantiations and inline calls worked by evaluating
all args in advance, before resolving generic parameter types. This
means any generic parameter (not just `anytype` ones) had generic poison
result types. This caused missing result locations in some cases.
Additionally, the generic instantiation logic caused `zirParam` to
analyze the argument types a second time before coercion. This meant
that for nominal types (struct/enum/etc), a *new* type was created,
distinct to the result type which was previously forwarded to the
argument expression.
This commit fixes both of these issues. Generic parameter type
resolution is now interleaved with argument analysis, so that we don't
have unnecessary generic poison types, and generic instantiation logic
now handles parameters itself rather than falling through to the
standard zirParam logic, so avoids duplicating the types.
Resolves: #16566Resolves: #16258Resolves: #16753
* Disable runtime calls, since it is not possible to know the proper
stack adjustment to follow the callee abi.
* Disable runtime returns, since it is not possible to know where the
return address is stored in general.
* Allow implicit returns regardless of the return type, which allows
naked functions with a non-void return type to be written.
