Recently OpenMP has transitioned to using the "new" driver which
primarily merges the device and host linking phases into a single
wrapper that handles both at the same time. This replaced a few tools
that were only used for OpenMP offloading, such as the
`clang-offload-wrapper` and `clang-nvlink-wrapper`. The new driver
carries some marked benefits compared to the old driver that is now
being deprecated. Things like device-side LTO, static library
support, and more compatible tooling. As such, we should be able to
completely deprecate the old driver, at least for OpenMP. The old driver
support will still exist for CUDA and HIP, although both of these can
currently be compiled on Linux with `--offload-new-driver` to use the new
method.
Note that this does not deprecate the `clang-offload-bundler`, although
it is unused by OpenMP now, it is still used by the HIP toolchain both
as their device binary format and object format.
When I proposed deprecating this code I heard some vendors voice
concernes about needing to update their code in their fork. They should
be able to just revert this commit if it lands.
Reviewed By: jdoerfert, MaskRay, ye-luo
Differential Revision: https://reviews.llvm.org/D130020
Use this instead of `*_LIBDIR_SUFFIX`, from which it is computed.
This gets us ready for D130586, in which `*_LIBDIR_SUFFIX` is
deprecated.
Differential Revision: https://reviews.llvm.org/D132300
We held off on this before as `LLVM_LIBDIR_SUFFIX` conflicted with it.
Now we return this.
`LLVM_LIBDIR_SUFFIX` is kept as a deprecated way to set
`CMAKE_INSTALL_LIBDIR`. The other `*_LIBDIR_SUFFIX` are just removed
entirely.
I imagine this is too potentially-breaking to make LLVM 15. That's fine.
I have a more minimal version of this in the disto (NixOS) patches for
LLVM 15 (like previous versions). This more expansive version I will
test harder after the release is cut.
Reviewed By: sebastian-ne, ldionne, #libc, #libc_abi
Differential Revision: https://reviews.llvm.org/D130586
When performing device only compilation, there was an issue where
`cubin` outputs were being renamed to `cubin` despite the user's name.
This is required in a normal compilation flow as the Nvidia tools only
understand specific filenames instead of checking magic bytes for some
unknown reason. We do not want to perform this transformation when the
user is performing device only compilation.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D131278
We currently call the `addNVPTXFeatures` function in two places, inside
of the CUDA Toolchain and inside of Clang in the standard entry point.
We normally add features to the job in Clang, so the call inside of the
CUDA toolchain is redundant and results in `+ptx` features being added.
Since we remove this call, we no longer will have a cached CUDA
installation so we will usually create it twice.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D128752
Previously, when using the new driver we created a fatbinary with the
PTX and Cubin output. This was mainly done in an attempt to create some
backwards compatibility with the existing CUDA support that embeds the
fatbinary in each TU. This will most likely be more work than necessary
to actually implement. The linker wrapper cannot do anything with these
embedded PTX files because we do not know how to link them, and if we
did want to include multiple files it should go through the
`clang-offload-packager` instead. Also this didn't repsect the setting
that disables embedding PTX (although it wasn't used anyway).
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D128441
Summary:
Normally we parse through the CUDA installation to disover the needed
features. However, we may want to build libraries on targets that do not
currently have CUDA installed but still need to know which features to
make use of when creating the PTX or bitcode. This flag is a simple way
to specify this so we can compile certain codes withotu a valid CUDA
installation.
Ideally this could be done via an -Xarch or simimlar flag but currently
they cannot handle this. We would need to support using an -Xarch flag
that takes multiple arguments that then pass them to the -Xclang
functionality.
This patch adds support for OpenMP to use the `--offload-arch` and
`--no-offload-arch` options. Traditionally, OpenMP has only supported
compiling for a single architecture via the `-Xopenmp-target` option.
Now we can pass in a bound architecture and use that if given, otherwise
we default to the value of the `-march` option as before.
Note that this only applies the basic support, the OpenMP target runtime
does not yet know how to choose between multiple architectures.
Additionally other parts of the offloading toolchain (e.g. LTO) require
the `-march` option, these should be worked out later.
Reviewed By: tra
Differential Revision: https://reviews.llvm.org/D124721
The root cause of this is, in `NVPTX::Assembler::ConstructJob`, the output file name might not match the `Output`'s file name passed into the function because `CudaToolChain::getInputFilename` is a specialized version. That means the real output file is not added to the temp files list, which will be all removed in the d'tor of `Compilation`. In order to "fix" it, in the function `NVPTX::OpenMPLinker::ConstructJob`, before calling `clang-nvlink-wrapper`, the function calls `getToolChain().getInputFilename(II)` to get the right output file name for each input, and add it to temp file, and then they can be removed w/o any issue. However, this whole logic doesn't work when using the new OpenMP driver because `NVPTX::OpenMPLinker::ConstructJob` is not called at all, which causing the issue that the cubin file generated in each single unit compilation is out of track.
In this patch, we add the real output file into temp files if its name doesn't match `Output`. We add it when the file is an output instead of doing it when it is an input, like what we did in `NVPTX::OpenMPLinker::ConstructJob`, which makes more sense.
Reviewed By: jhuber6
Differential Revision: https://reviews.llvm.org/D124253
Summary:
There was a naming conflict in the getNVPTXTargetFeatures function that
prevented some compilers from correctly disambiguating between the
enumeration and variable of the same name. Rename the variable to avoid
this.
The NVPTX toolchain uses target features to determine the PTX version to
use. However this isn't exposed externally like most other toolchain
specific target features are. Add this functionaliy in preparation for
using it in for OpenMP offloading.
Reviewed By: jdoerfert, tra
Differential Revision: https://reviews.llvm.org/D122089
Now that the old device runtime has been deleted there is only a single
target that differs by the triple and the architecture. Simplify the
scheme for identifying the library but directly using the triple.
Reviewed By: JonChesterfield
Differential Revision: https://reviews.llvm.org/D119638
Summary:
The new runtime was deleted. AMD's old runtime used the triple name
`amdgcn` while the new runtime used `amdgpu`. This was not updated when
the old runtime was removed causing the library to not be found on
AMDGPU.
This patch completely removes the old OpenMP device runtime. Previously,
the old runtime had the prefix `libomptarget-new-` and the old runtime
was simply called `libomptarget-`. This patch makes the formerly new
runtime the only runtime available. The entire project has been deleted,
and all references to the `libomptarget-new` runtime has been replaced
with `libomptarget-`.
Reviewed By: JonChesterfield
Differential Revision: https://reviews.llvm.org/D118934
Summary:
This patch adds support for linking the OpenMP device bitcode library
late when doing LTO. This simply passes it in as an additional device
file when doing the final device linking phase with LTO. This has the
advantage that we don't link it multiple times, and the device
references do not get inlined and prevent us from doing needed OpenMP
optimizations when we have visiblity of the whole module.
Fix some failings where the implicit conversion of an Error to an
Expected triggered the deleted copy constructor.
Depends on D116675
Differential revision: https://reviews.llvm.org/D117048
This patch changes the `-fopenmp-target-new-runtime` option which controls if
the new or old device runtime is used to be true by default. Disabling this to
use the old runtime now requires using `-fno-openmp-target-new-runtime`.
Reviewed By: JonChesterfield, tianshilei1992, gregrodgers, ronlieb
Differential Revision: https://reviews.llvm.org/D114890
Added support of a "--nvlink-path" option in clang-nvlink-wrapper which
takes the path of nvlink binary.
Static Device Library support for OpenMP (D105191) now searches for
nvlink binary and passes its location via this option. In absence
of this option, nvlink binary is searched in locations in PATH.
Differential Revision: https://reviews.llvm.org/D111488
An archive containing device code object files can be passed to
clang command line for linking. For each given offload target
it creates a device specific archives which is either passed to llvm-link
if the target is amdgpu, or to clang-nvlink-wrapper if the target is
nvptx. -L/-l flags are used to specify these fat archives on the command
line. E.g.
clang++ -fopenmp -fopenmp-targets=nvptx64 main.cpp -L. -lmylib
It currently doesn't support linking an archive directly, like:
clang++ -fopenmp -fopenmp-targets=nvptx64 main.cpp libmylib.a
Linking with x86 offload also does not work.
Reviewed By: ye-luo
Differential Revision: https://reviews.llvm.org/D105191
An archive containing device code object files can be passed to
clang command line for linking. For each given offload target
it creates a device specific archives which is either passed to llvm-link
if the target is amdgpu, or to clang-nvlink-wrapper if the target is
nvptx. -L/-l flags are used to specify these fat archives on the command
line. E.g.
clang++ -fopenmp -fopenmp-targets=nvptx64 main.cpp -L. -lmylib
It currently doesn't support linking an archive directly, like:
clang++ -fopenmp -fopenmp-targets=nvptx64 main.cpp libmylib.a
Linking with x86 offload also does not work.
Reviewed By: ye-luo
Differential Revision: https://reviews.llvm.org/D105191
This allows clang to work on Linux distributions like Debian where
<CUDA-PATH>/include may be a symlink to /usr/include. We only need
`cuda_wrappers` to be present before the standard C++ library headers.
The CUDA SDK headers themselves do not need to be found that early.
This addresses https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=995122
mentioned in post-commit comments on D108247
Differential Revision: https://reviews.llvm.org/D110596
Always use cuda.h to detect CUDA version. It's a more universal approach
compared to version.txt which is no longer present in recent CUDA versions.
Split the 'unknown CUDA version' warning in two:
* when detected CUDA version is partially supported by clang. It's expected to
work in general, at the feature parity with the latest supported CUDA
version. and may be missing support for the new features/instructions/GPU
variants. Clang will issue a warning.
* when detected version is new. Recent CUDA versions have been working with
clang reasonably well, and will likely to work similarly to the partially
supported ones above. Or it may not work at all. Clang will issue a warning and
proceed as if the latest known CUDA version was detected.
Differential Revision: https://reviews.llvm.org/D108247
Attempt to enable MemCpyOpt unconditionally in D104801 uncovered the fact that
there are users that do not expect LLVM to materialize `memset` intrinsic.
While other passes can do that, too, MemCpyOpt triggers it more frequently and
breaks sanitizers and some downstream users.
For now introduce a flag to force-enable the flag and opt-in only CUDA
compilation with NVPTX back-end.
Differential Revision: https://reviews.llvm.org/D106401
Clang diagnostics should not start with a capital letter or use
trailing punctuation (https://clang.llvm.org/docs/InternalsManual.html#the-format-string),
but quite a few driver diagnostics were not following this advice. This
corrects the grammar and punctuation to improve consistency, but does
not change the circumstances under which the diagnostics are produced.
Moving `InputInfo.h` from `lib/Driver/` into `include/Driver` to be able to expose it in an API consumed from outside of `clangDriver`.
Reviewed By: dexonsmith
Differential Revision: https://reviews.llvm.org/D106787
This patch adds a driver flag `-fopenmp-target-new-runtime` to optionally enable the new device runtime
bitcode library. This allows users to enable the new experimental runtime
before it becomes the default in the future.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D106793
https://lists.llvm.org/pipermail/openmp-dev/2021-March/003940.html reports
test failure in `openmp-offload-gpu.c`. The failure is, when using `-S` in the
clang driver, it still reports bitcode library doesn't exist. However, it is not
exposed in my local run and Phabiractor test. The reason it escaped from Phabricator
test is, the test machine doesn't have CUDA, so `LibDeviceFile` is empty. In this
case, the check of `OPT_S` will be hit, and we get "expected" result. However, if
the test machine has CUDA, `LibDeviceFile` will not be empty, then the check will
not be done, and it just proceeds, trying to add the bitcode library. The reason
it escaped from my local run is, I didn't build ALL targets, so this case was
marked UNSUPPORTED.
Reviewed By: kkwli0
Differential Revision: https://reviews.llvm.org/D98902
In D97003, CUDA 9.2 is the minimum requirement for OpenMP offloading on
NVPTX target. We don't need to have macros in source code to select right functions
based on CUDA version. we don't need to compile multiple bitcode libraries of
different CUDA versions for each SM. We don't need to worry about future
compatibility with newer CUDA version.
`-target-feature +ptx61` is used in this patch, which corresponds to the highest
PTX version that CUDA 9.2 can support.
Reviewed By: jdoerfert
Differential Revision: https://reviews.llvm.org/D97198
In current implementation of `deviceRTLs`, we're using some functions
that are CUDA version dependent (if CUDA_VERSION < 9, it is one; otheriwse, it
is another one). As a result, we have to compile one bitcode library for each
CUDA version supported. A worse problem is forward compatibility. If a new CUDA
version is released, we have to update CMake file as well.
CUDA 9.2 has been released for three years. Instead of using various weird tricks
to make `deviceRTLs` work with different CUDA versions and still have forward
compatibility, we can simply drop support for CUDA 9.1 or lower version. It has at
least two benifits:
- We don't need to generate bitcode libraries for each CUDA version;
- Clang driver doesn't need to search for the bitcode lib based on CUDA version.
We can claim that starting from LLVM 12, OpenMP offloading on NVPTX target requires
CUDA 9.2+.
Reviewed By: jdoerfert, JonChesterfield
Differential Revision: https://reviews.llvm.org/D97003
This patch uses the existing logic of CUDA for searching libomptarget
and extracts it to a common method.
Reviewed By: JonChesterfield, tianshilei1992
Differential Revision: https://reviews.llvm.org/D96248
The patch only plumbs through the option necessary for targeting sm_86 GPUs w/o
adding any new functionality.
Differential Revision: https://reviews.llvm.org/D95974
Currently -fgpu-rdc is not passed to host clang -cc1.
This causes issue because -fgpu-rdc affects shadow
variable linkage in host compilation.
Reviewed by: Artem Belevich
Differential Revision: https://reviews.llvm.org/D96105
From this patch (plus some landed patches), `deviceRTLs` is taken as a regular OpenMP program with just `declare target` regions. In this way, ideally, `deviceRTLs` can be written in OpenMP directly. No CUDA, no HIP anymore. (Well, AMD is still working on getting it work. For now AMDGCN still uses original way to compile) However, some target specific functions are still required, but they're no longer written in target specific language. For example, CUDA parts have all refined by replacing CUDA intrinsic and builtins with LLVM/Clang/NVVM intrinsics.
Here're a list of changes in this patch.
1. For NVPTX, `DEVICE` is defined empty in order to make the common parts still work with AMDGCN. Later once AMDGCN is also available, we will completely remove `DEVICE` or probably some other macros.
2. Shared variable is implemented with OpenMP allocator, which is defined in `allocator.h`. Again, this feature is not available on AMDGCN, so two macros are redefined properly.
3. CUDA header `cuda.h` is dropped in the source code. In order to deal with code difference in various CUDA versions, we build one bitcode library for each supported CUDA version. For each CUDA version, the highest PTX version it supports will be used, just as what we currently use for CUDA compilation.
4. Correspondingly, compiler driver is also updated to support CUDA version encoded in the name of bitcode library. Now the bitcode library for NVPTX is named as `libomptarget-nvptx-cuda_[cuda_version]-sm_[sm_number].bc`, such as `libomptarget-nvptx-cuda_80-sm_20.bc`.
With this change, there are also multiple features to be expected in the near future:
1. CUDA will be completely dropped when compiling OpenMP. By the time, we also build bitcode libraries for all supported SM, multiplied by all supported CUDA version.
2. Atomic operations used in `deviceRTLs` can be replaced by `omp atomic` if OpenMP 5.1 feature is fully supported. For now, the IR generated is totally wrong.
3. Target specific parts will be wrapped into `declare variant` with `isa` selector if it can work properly. No target specific macro is needed anymore.
4. (Maybe more...)
Reviewed By: JonChesterfield
Differential Revision: https://reviews.llvm.org/D94745
D94700 removed the static library so we no longer need to pass
`-llibomptarget-nvptx` to `nvlink`. Since the bitcode library is the only device
runtime for now, instead of emitting a warning when it is not found, an error
should be raised. We also set a new option `libomptarget-nvptx-bc-path` to let
user choose which bitcode library is being used.
Reviewed By: JonChesterfield
Differential Revision: https://reviews.llvm.org/D95161
Summary:
Optimized debugging is not supported by ptxas. Debugging information is degraded to line information only if optimizations are enabled, but debugging information would be added back in by the driver if remarks were enabled. This solves https://bugs.llvm.org/show_bug.cgi?id=48153.
Reviewers: jdoerfert tra jholewinski serge-sans-paille
Differential Revision: https://reviews.llvm.org/D94123
If CUDA version can not be determined based on version.txt file, attempt to find
CUDA_VERSION macro in cuda.h.
This is a follow-up to D89752,
Differntial Revision: https://reviews.llvm.org/D89832
CUDA-11.1 does not carry version.txt which causes clang to assume that it's
CUDA-7.0, which used to be the only CUDA version w/o version.txt.
In order to tell CUDA-7.0 apart from the new versions, clang now probes for the
presence of libdevice.10.bc which is not present in the old CUDA versions.
This should keep Clang working for CUDA-11.1.
PR47332: https://bugs.llvm.org/show_bug.cgi?id=47332
Differential Revision: https://reviews.llvm.org/D89752
Object of class `Command` contains various properties of a command to
execute, but output file was missed from them. This change adds this
property. It is required for reporting consumed time and memory implemented
in D78903 and may be used in other cases too.
Differential Revision: https://reviews.llvm.org/D78902
In CUDA/HIP a function may become implicit host device function by
pragma or constexpr. A host device function is checked in both
host and device compilation. However it may be emitted only
on host or device side, therefore the diagnostics should be
deferred until it is known to be emitted.
Currently clang is only able to defer certain diagnostics. This causes
false alarms and limits the usefulness of host device functions.
This patch lets clang defer all overloading resolution diagnostics for host device functions.
An option -fgpu-defer-diag is added to control this behavior. By default
it is off.
It is NFC for other languages.
Differential Revision: https://reviews.llvm.org/D84364
Joseph Huber