Instead of trying to "fix" the original driver invocation by appending
arguments to it, split it into multiple commands, and for each -cc1
command use a CompilerInvocation to give precise control over the
invocation.
This change should make it easier to (in the future) canonicalize the
command-line (e.g. to improve hits in something like ccache), apply
optimizations, or start supporting multi-arch builds, which would
require different modules for each arch.
In the long run it may make sense to treat the TU commands as a
dependency graph, each with their own dependencies on modules or earlier
TU commands, but for now they are simply a list that is executed in
order, and the dependencies are simply duplicated. Since we currently
only support single-arch builds, there is no parallelism available in
the execution.
Differential Revision: https://reviews.llvm.org/D132405
This patch introduces new option `-eager-load-pcm` to `clang-scan-deps`, which controls whether the resulting command-lines will load PCM files eagerly (at the start of compilation) or lazily (when handling import directive). This patch also switches the default from eager to lazy.
To reduce the potential for churn in LIT tests in the future, this patch also removes redundant checks of command-line arguments and introduces new test `modules-dep-args.c` as a substitute.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D132066
Since D129389 (and downstream PR https://github.com/apple/llvm-project/pull/4965), the dependency scanner is responsible for generating full command-lines, including the modules paths. This patch removes the flag that was making this an opt-in behavior in clang-scan-deps.
Reviewed By: benlangmuir
Differential Revision: https://reviews.llvm.org/D131420
This is first of a series of patches for making the special lexing for dependency scanning a first-class feature of the `Preprocessor` and `Lexer`.
This patch only includes NFC renaming changes to make reviewing of the functionality changing parts easier.
Differential Revision: https://reviews.llvm.org/D125484
In D92191, a bunch of test cases were added to check `clang-scan-deps` works in `clang-cl` mode as well.
We don't need to duplicate all test cases, though. Testing the few special cases we have in `clang-scan-deps` for `clang-cl` should be good enough:
1. Deducing output path (and therefore target name in our make output).
2. Ignoring `-Xclang` arguments in step 1.
3. Deducing resource directory by invoking the compiler executuable.
This test de-duplicates the extra clang-cl test cases.
Reviewed By: dexonsmith, saudi
Differential Revision: https://reviews.llvm.org/D121812
Instead of outputting the test directory into the JSON result file, parsing it with `FileCheck` and then potentially stripping it, simply use `FileCheck`'s `-D` option.
Note that we use `%/t` instead of `%t` in order to normalize to forward slashes on Windows, which matches what we do with `sed 's:\\\\\?:/:g'`.
Differential Revision: https://reviews.llvm.org/D121516
The `clang-scan-deps` tool currently generates `-fmodule-file=` command-line arguments for the whole transitive closure of modular dependencies. This is not necessary, we only need to provide the direct dependencies on the command line. Information about transitive dependencies is stored within the `.pcm` files of direct dependencies. This makes the command lines shorter, but should be a NFC otherwise (unless there are bugs in the loading mechanism for explicit modules).
Depends on D120465.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D118915
Since D113473, we don't report any module map files via `-fmodule-map-file=` in explicit builds. The ultimate goal here is to make sure Clang doesn't open/read/parse/evaluate unnecessary module maps.
However, implicit module maps still end up reading all reachable module maps. This patch disables implicit module maps in explicit builds.
Unfortunately, we still need to report some module map files that aren't encoded in PCM files of dependencies: module maps that are necessary to correctly evaluate includes in modules marked as `[no_undeclared_includes]`.
Depends on D120464.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D120465
The dependency scanner already generates canonical -cc1 command lines that can be used to compile discovered modular dependencies.
For translation unit command lines, the scanner only generates additional driver arguments the build system is expected to append to the original command line.
While this works most of the time, there are situations where that's not the case. For example with `-Wunused-command-line-argument`, Clang will complain about the `-fmodules-cache-path=` argument that's not being used in explicit modular builds. Combine that with `-Werror` and the build outright fails.
To prevent such failures, this patch changes the dependency scanner to return the full driver command line to compile the original translation unit. This gives us more opportunities to massage the arguments into something reasonable.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D118986
During explicit modules build, when all modules are provided via `-fmodule-file=<path>` and implicit modules and implicit module maps are disabled (`-fno-implicit-modules`, `-fno-implicit-module-maps`), we don't need to load the original module map files at all. This patch stops emitting the `-fmodule-map-file=` arguments we don't need, saving some compilation time due to avoiding parsing such module maps and making the command line shorter.
Reviewed By: bnbarham
Differential Revision: https://reviews.llvm.org/D113473
The `sed` command ensures Windows-specific path separators (single and double backslashes) are replaced by forward slashes in the output file. FileCheck can continue using forward slashes in paths this way.
The `clang-scan-deps` tests for the full output format were written under the assumption that most TUs/modules have the same context hash. This is no longer true, since we're changing the original compilation options. This patch updates the tests, which no longer conflate multiple context hashes into a single FileCheck variable.
Dependency scanning currently performs an implicit build. When testing that Clang can build modules with the command-lines generated by `clang-scan-deps`, the actual compilation would overwrite artifacts created during the scan, which makes debugging harder than it should be and can lead to errors in multi-step builds.
To prevent this, this patch adds new flag to `clang-scan-deps` that allows developers to customize the directory to use when generating module map paths, instead of always using the module cache. Moreover, the explicit context hash in now part of the PCM path, which will be useful in D102488, where the context hash can change due to command-line pruning.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D103516
This reapplies commit 95033eb3 that reverted commit 1d9e8e13.
The tests were failing on Windows due to spaces and backslashes in paths not being handled carefully.
This patch adds support for inferred modules to the dependency scanner.
Effectively a cherry-pick of https://github.com/apple/llvm-project/pull/699 authored by @Bigcheese with libclang and other changes omitted.
Contains following changes:
1. [Clang][ScanDeps] Ignore __inferred_module.map dependency.
* This shows up with inferred modules, but it doesn't exist on disk, so don't report it as a dependency.
2. [Clang][ScanDeps] Use the module map a module was inferred from for inferred modules.
Also includes a smoke test that uses clang-scan-deps output to perform an explicit build. There's no intention to duplicate whatever `test/Modules` contains, just to verify the produced command-line does "work" (with very loose definition of work).
Split from D100934.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D102495
Add option to `clang-scan-deps` to enable/disable generation of command-line arguments with absolute paths. This is essentially a revert of D100533, but with improved naming and added test.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D101051
This reverts commit 199c397482.
This time, clang-scan-deps's search for output argument in clang-cl command line will now ignore arguments preceded by "-Xclang".
That way, it won't detect a /o argument in "-Xclang -ivfsoverlay -Xclang /opt/subpath"
Initial patch description:
clang-scan-deps contains some command line parsing and modifications.
This patch adds support for clang-cl command options.
Differential Revision: https://reviews.llvm.org/D92191
This patch uses the new `CompilerInvocation::generateCC1CommandLine` to generate the full canonical command line for modular dependencies, instead of only appending additional arguments.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D100534
This patch removes the `-full-command-line` option from `clang-scan-deps`. It's only used with `-format=experimental-full`, where omitting the command lines doesn't make much sense. There are no tests without `-full-command-line`.
Depends on D100531.
Reviewed By: Bigcheese
Differential Revision: https://reviews.llvm.org/D100533
This patch simplifies (and renames) the `appendCommonModuleArguments` function.
It no longer tries to construct the command line for explicitly building modules. Instead, it only performs the DFS traversal of modular dependencies and queries the callbacks to collect paths to `.pcm` and `.modulemap` files.
This makes it more flexible and usable in two contexts:
* Generating additional command line arguments for the main TU in modular build. The `std::vector<std::string>` output parameters can be used to manually generate appropriate command line flags.
* Generate full command line for a module. The output parameters can be the corresponding parts of `CompilerInvocation`. (In a follow-up patch.)
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D100531
clang-scan-deps contains some command line parsing and modifications.
This patch adds support for clang-cl command options.
Differential Revision: https://reviews.llvm.org/D92191
clang-scan-deps contains some command line parsing and modifications.
This patch adds support for clang-cl command options.
Differential Revision: https://reviews.llvm.org/D92191
Differential Revision: https://reviews.llvm.org/D70268
This is a recommit of f978ea4983 with a fix for the PowerPC failure.
The issue was that:
* `CompilerInstance::ExecuteAction` calls
`getTarget().adjust(getLangOpts());`.
* `PPCTargetInfo::adjust` changes `LangOptions::HasAltivec`.
* This happens after the first few calls to `getModuleHash`.
There’s even a FIXME saying:
```
// FIXME: We shouldn't need to do this, the target should be immutable once
// created. This complexity should be lifted elsewhere.
```
This only showed up on PowerPC because it's one of the few targets that
almost always changes a hashed langopt.
I looked into addressing the fixme, but that would be a much larger
change, and it's not the only thing that happens in `ExecuteAction` that
can change the module context hash. Instead I changed the code to not
call `getModuleHash` until after it has been modified in `ExecuteAction`.
This is a recommit of d8a4ef0e68 with the nondeterminism fixed.
This adds experimental support for extracting a Clang module dependency graph
from a compilation database. The output format is experimental and will change.
It is currently a concatenation of JSON outputs for each compilation. Future
patches will change this to deduplicate modules between compilations.
Differential Revision: https://reviews.llvm.org/D69420
This adds experimental support for extracting a Clang module dependency graph
from a compilation database. The output format is experimental and will change.
It is currently a concatenation of JSON outputs for each compilation. Future
patches will change this to deduplicate modules between compilations.
Differential Revision: https://reviews.llvm.org/D69420
Ben Langmuir