Commit Graph

112 Commits

Author SHA1 Message Date
Zalathar
6af9fef085 coverage: Emit the filenames section before encoding per-function mappings
Most coverage metadata is encoded into two sections in the final executable.
The `__llvm_covmap` section mostly just contains a list of filenames, while the
`__llvm_covfun` section contains encoded coverage maps for each instrumented
function.

The catch is that each per-function record also needs to contain a hash of the
filenames list that it refers to. Historically this was handled by assembling
most of the per-function data into a temporary list, then assembling the
filenames buffer, then using the filenames hash to emit the per-function data,
and then finally emitting the filenames table itself.

However, now that we build the filenames table up-front (via a separate
traversal of the per-function data), we can hash and emit that part first, and
then emit each of the per-function records immediately after building. This
removes the awkwardness of having to temporarily store nearly-complete
per-function records.
2023-10-22 23:17:15 +11:00
Zalathar
de4cfbca2e coverage: Encode function mappings without re-sorting them
The main change here is that `VirtualFileMapping` now uses an internal hashmap
to de-duplicate incoming global file IDs. That removes the need for
`encode_mappings_for_function` to re-sort its mappings by filename in order to
de-duplicate them.

(We still de-duplicate runs of identical filenames to save work, but this is
not load-bearing for correctness, so a sort is not necessary.)
2023-10-22 20:37:39 +11:00
Zalathar
88159cafa7 coverage: Encapsulate local-to-global file mappings 2023-10-22 20:37:39 +11:00
Zalathar
e985ae5a45 coverage: Build the global file table ahead of time 2023-10-22 20:37:37 +11:00
Zalathar
86b55cccff coverage: Fetch expressions and mappings separately
The combined `get_expressions_and_counter_regions` method was an artifact of
having to prepare the expressions and mappings at the same time, to avoid
ownership/lifetime problems with temporary data used by both.

Now that we have an explicit transition from `FunctionCoverageCollector` to the
final `FunctionCoverage`, we can prepare any shared data during that step and
store it in the final struct.
2023-10-22 20:11:48 +11:00
Zalathar
371883a05a coverage: Split FunctionCoverage into distinct collector/finished phases
This gives us a clearly-defined place to run code after the instance's MIR has
been traversed by codegen, but before we emit its `__llvm_covfun` record.
2023-10-22 20:11:45 +11:00
Zalathar
6f1ca8d9eb coverage: Change query codegened_and_inlined_items to a plain function
This query has a name that sounds general-purpose, but in fact it has
coverage-specific semantics, and (fortunately) is only used by coverage code.

Because it is only ever called once (from one designated CGU), it doesn't need
to be a query, and we can change it to a regular function instead.
2023-10-21 12:20:05 +11:00
Zalathar
cdeeffde64 coverage: Move unused-function helpers closer to where they are used 2023-10-21 12:20:05 +11:00
Zalathar
e964ea5bf5 coverage: Emit mappings for unused functions without generating stubs 2023-10-21 12:20:05 +11:00
Oli Scherer
60956837cf s/Generator/Coroutine/ 2023-10-20 21:10:38 +00:00
bors
94c4e5c411 Auto merge of #115214 - Urgau:rfc-3127-trim-paths, r=compiler-errors
Implement rustc part of RFC 3127 trim-paths

This PR implements (or at least tries to) [RFC 3127 trim-paths](https://github.com/rust-lang/rust/issues/111540), the rustc part. That is `-Zremap-path-scope` with all of it's components/scopes.

`@rustbot` label: +F-trim-paths
2023-10-19 19:09:29 +00:00
Zalathar
33da0978ac coverage: Explicitly note that counter/expression IDs are function-local 2023-10-18 23:44:36 +11:00
Zalathar
13b2d604ec coverage: Store expression data in function coverage info
Even though expression details are now stored in the info structure, we still
need to inject `ExpressionUsed` statements into MIR, because if one is missing
during codegen then we know that it was optimized out and we can remap all of
its associated code regions to zero.
2023-10-18 23:44:34 +11:00
Zalathar
6da319f635 coverage: Store all of a function's mappings in function coverage info
Previously, mappings were attached to individual coverage statements in MIR.
That necessitated special handling in MIR optimizations to avoid deleting those
statements, since otherwise codegen would be unable to reassemble the original
list of mappings.

With this change, a function's list of mappings is now attached to its MIR
body, and survives intact even if individual statements are deleted by
optimizations.
2023-10-18 23:42:39 +11:00
Zalathar
4099ab1997 coverage: Make expression simplification non-destructive
Instead of modifying the accumulated expressions in-place, we now build a set
of expressions that are known to be zero, and then consult that set on the fly
when converting the expression data for FFI.

This will be necessary when moving mappings and expression data into function
coverage info, which can't be mutated during codegen.
2023-10-18 22:29:31 +11:00
Zalathar
8efdd4cca6 coverage: Collect a function's coverage mappings into a single list
This is an intermediate step towards being able to store all of a function's
mappings in function coverage info.
2023-10-18 21:24:01 +11:00
Zalathar
79f935b96c coverage: Rename Operand to CovTerm
Later patches in this PR will use `CovTerm` to represent things that are not
expression operands.
2023-10-18 21:23:58 +11:00
Zalathar
a18c5f3b75 coverage: Store the number of counters/expressions in function coverage info
Coverage codegen can now allocate arrays based on the number of
counters/expressions originally used by the instrumentor.

The existing query that inspects coverage statements is still used for
determining the number of counters passed to `llvm.instrprof.increment`. If
some high-numbered counters were removed by MIR optimizations, the instrumented
binary can potentially use less memory and disk space at runtime.
2023-10-18 21:22:40 +11:00
Zalathar
c479bc7f3b coverage: Attach an optional FunctionCoverageInfo to mir::Body
This allows coverage information to be attached to the function as a whole when
appropriate, instead of being smuggled through coverage statements in the
function's basic blocks.

As an example, this patch moves the `function_source_hash` value out of
individual `CoverageKind::Counter` statements and into the per-function info.

When synthesizing unused functions for coverage purposes, the absence of this
info is taken to indicate that a function was not eligible for coverage and
should not be synthesized.
2023-10-18 21:20:29 +11:00
Urgau
eccc9e6628 [RFC 3127 - Trim Paths]: Condition remapped filepath on remap scopes 2023-10-17 10:11:30 +02:00
Zalathar
ee9d00f6b8 coverage: Let each coverage statement hold a vector of code regions
This makes it possible for a `StatementKind::Coverage` to hold more than one
code region, but that capability is not yet used.
2023-10-03 13:03:39 +11:00
Zalathar
59a11af1f9 coverage: Mappings for unused functions can all be zero
There is no need to include a dummy counter reference in the coverage mappings
for an unused function.
2023-10-03 13:03:39 +11:00
Zalathar
041a232b62 coverage: Don't bother renumbering expressions on the Rust side
The LLVM API that we use to encode coverage mappings already has its own code
for removing unused coverage expressions and renumbering the rest.

This lets us get rid of our own complex renumbering code, making it easier to
change our coverage code in other ways.
2023-09-21 18:32:49 +10:00
Zalathar
527c629cd9 coverage: Explicitly simplify coverage expressions in codegen
After coverage instrumentation and MIR transformations, we can sometimes end up
with coverage expressions that always have a value of zero. Any expression
operand that refers to an always-zero expression can be replaced with a literal
`Operand::Zero`, making the emitted coverage mapping data smaller and simpler.

This simplification step is mostly redundant with the simplifications performed
inline in `expressions_with_regions`, except that it does a slightly more
thorough job in some cases (because it checks for always-zero expressions
*after* other simplifications).

However, adding this simplification step will then let us greatly simplify that
code, without affecting the quality of the emitted coverage maps.
2023-09-21 18:32:49 +10:00
Zalathar
659575aade coverage: Make the zero counter a constant 2023-09-21 18:32:49 +10:00
bors
c728bf3963 Auto merge of #114656 - bossmc:rework-no-coverage-attr, r=oli-obk
Rework `no_coverage` to `coverage(off)`

As discussed at the tail of https://github.com/rust-lang/rust/issues/84605 this replaces the `no_coverage` attribute with a `coverage` attribute that takes sub-parameters (currently `off` and `on`) to control the coverage instrumentation.

Allows future-proofing for things like `coverage(off, reason="Tested live", issue="#12345")` or similar.
2023-09-14 01:05:18 +00:00
Zalathar
6e968b1e45 coverage: Simplify grouping of mappings by file
This removes an ad-hoc implementation of `group_by`.
2023-09-11 14:29:10 +10:00
Zalathar
1f56fa9657 coverage: Push down the call to get_expressions_and_counter_regions
These expressions and counter regions are only needed by the function that
encodes a function's coverage mappings payload.
2023-09-11 14:29:10 +10:00
Zalathar
99da8a83c2 coverage: Push down creation of the mappings payload buffer
Instead of writing coverage mappings into a supplied `&RustString`, this
function can just create the buffer itself and return the resulting vector of
bytes.
2023-09-11 14:29:10 +10:00
Zalathar
fbbb543ced coverage: Reserve capacity for all of a function's mapping regions
We already know in advance how many entries will be pushed onto this vector.
2023-09-11 14:29:10 +10:00
Zalathar
4f88aa0fbd coverage: Use a stable sort when grouping mapped regions by file
If two or more mappings cover exactly the same region, their relative order
will now be preserved from `get_expressions_and_counter_regions`, rather than
being disturbed by implementation details of an unstable sort.

The current order is: counter mappings, expression mappings, zero mappings.

(LLVM will also perform its own stable sort on these mappings, but that sort
only compares file ID, start location, and `RegionKind`.)
2023-09-11 14:29:09 +10:00
Zalathar
525ac15b66 coverage: Convert CoverageMapGenerator to GlobalFileTable
This struct was only being used to hold the global file table, and one of its
methods didn't even use the table. Changing its methods to ordinary functions
makes it easier to see where the table is mutated.
2023-09-11 14:29:09 +10:00
Andy Caldwell
8e03371fc3
Rework no_coverage to coverage(off) 2023-09-08 12:46:06 +01:00
Arpad Borsos
2ceea9ae9d
Inline functions called from add_coverage
This removes quite a bit of indirection and duplicated code related to getting the `FunctionCoverage`.
2023-08-22 10:59:19 +02:00
Zalathar
4b154bc8e2 coverage: Don't convert symbol names to CString for FFI 2023-08-04 10:48:05 +10:00
Zalathar
e184118683 coverage: Don't convert filenames to CString for FFI 2023-08-04 10:48:05 +10:00
Zalathar
d6ed6e3904 coverage: Consolidate FFI types into one module
Coverage FFI types were historically split across two modules, because some of
them were needed by code in `rustc_codegen_ssa`.

Now that all of the coverage codegen code has been moved into
`rustc_codegen_llvm` (#113355), it's possible to move all of the FFI types into
a single module, making it easier to see all of them at once.
2023-08-02 15:26:47 +10:00
Zalathar
3920e07f0b Make coverage counter IDs count up from 0, not 1
Operand types are now tracked explicitly, so there is no need to reserve ID 0
for the special always-zero counter.

As part of the renumbering, this change fixes an off-by-one error in the way
counters were counted by the `coverageinfo` query. As a result, functions
should now have exactly the number of counters they actually need, instead of
always having an extra counter that is never used.
2023-08-01 11:29:55 +10:00
Zalathar
f103db894f Make coverage expression IDs count up from 0, not down from u32::MAX
Operand types are now tracked explicitly, so there is no need for expression
IDs to avoid counter IDs by descending from `u32::MAX`. Instead they can just
count up from 0, and can be used directly as indices when necessary.
2023-08-01 11:29:55 +10:00
Zalathar
1a014d42f4 Replace ExpressionOperandId with enum Operand
Because the three kinds of operand are now distinguished explicitly, we no
longer need fiddly code to disambiguate counter IDs and expression IDs based on
the total number of counters/expressions in a function.

This does increase the size of operands from 4 bytes to 8 bytes, but that
shouldn't be a big deal since they are mostly stored inside boxed structures,
and the current coverage code is not particularly size-optimized anyway.
2023-08-01 11:29:55 +10:00
Zalathar
01f3cc1272 coverage: Obtain the __llvm_covfun section name outside a per-function loop
This section name is always constant for a given target, but obtaining it from
LLVM requires a few intermediate allocations. There's no need to do so
repeatedly from inside a per-function loop.
2023-07-24 21:58:00 +10:00
bors
ffb9b61294 Auto merge of #113430 - Zalathar:hash, r=b-naber
Remove `LLVMRustCoverageHashCString`

Coverage has two FFI functions for computing the hash of a byte string. One takes a ptr/len pair (`LLVMRustCoverageHashByteArray`), and the other takes a NUL-terminated C string (`LLVMRustCoverageHashCString`).

But on closer inspection, the C string version is unnecessary. The calling-side code converts a Rust `&str` into a `CString`, and the C++ code then immediately turns it back into a ptr/len string before actually hashing it. So we can just call the ptr/len version directly instead.

---

This PR also fixes a bug in the C++ declaration of `LLVMRustCoverageHashByteArray`. It should be `size_t`, since that's what is declared and passed on the Rust side, and it's what `StrRef`'s constructor expects to receive on the callee side.
2023-07-16 01:56:23 +00:00
Mahdi Dibaiee
e55583c4b8 refactor(rustc_middle): Substs -> GenericArg 2023-07-14 13:27:35 +01:00
Zalathar
352d031599 Remove LLVMRustCoverageHashCString
Coverage has two FFI functions for computing the hash of a byte string. One
takes a ptr/len pair, and the other takes a NUL-terminated C string.

But on closer inspection, the C string version is unnecessary. The calling-side
code converts a Rust `&str` into a C string, and the C++ code then immediately
turns it back into a ptr/len string before actually hashing it.
2023-07-13 11:31:15 +10:00
Zalathar
7a5ad35da4 Pass a byte slice to coverageinfo::hash_bytes instead of an owned vector
The function body immediately treats it as a slice anyway, so this just makes
it possible to call the hash function with arbitrary read-only byte slices.
2023-07-13 11:28:50 +10:00
Zalathar
29c53d8748 Don't clone symbol names for coverage hashing
A symbol already contains a `&str`, and in this context there's no need to make
an owned copy, so we can just use the original string reference.
2023-07-13 11:16:27 +10:00
Boxy
12138b8e5e Move TyCtxt::mk_x to Ty::new_x where applicable 2023-07-05 20:27:07 +01:00
Zalathar
cb570d6bc1 Move coverageinfo::ffi and coverageinfo::map out of SSA 2023-07-05 20:40:40 +10:00
Zalathar
9c430d38cf Remove trait CoverageInfoMethods, since non-LLVM backends don't need it
These methods are only ever called from within `rustc_codegen_llvm`, so they
can just be declared there as well.
2023-07-05 20:40:40 +10:00
Zalathar
4169d0f756 Narrow trait CoverageInfoBuilderMethods down to just one method
This effectively inlines most of `FunctionCx::codegen_coverage` into the LLVM
implementation of `CoverageInfoBuilderMethods`.
2023-07-05 20:40:39 +10:00