Commit Graph

119 Commits

Author SHA1 Message Date
Joe Loser 1d650d4f1d [MLIR] Fix typo in `add_mlir_library` docs. NFC.
`s/librar/library` in `llvm_add_library`.

Differential Revision: https://reviews.llvm.org/D139265
2022-12-03 19:28:29 -07:00
Thomas Preud'homme ecfa2d3d99 Add version to all LLVM cmake package
Add a version to non-LLVM cmake package so that users needing an exact
version match can use the version parameter to find_package. Also adjust
the find_package(LLVM) to use an exact version match as well.

Reviewed By: arsenm, stellaraccident, mceier

Differential Revision: https://reviews.llvm.org/D138274
2022-11-25 21:57:58 +00:00
Thomas Preud'homme b62c8d396f Revert: Add version to all LLVM cmake package
Summary: This reverts commit ad485b71b5.

Reviewers:

Subscribers:
2022-11-25 10:54:58 +00:00
Thomas Preud'homme ad485b71b5 Add version to all LLVM cmake package
Add a version to non-LLVM cmake package so that users needing an exact
version match can use the version parameter to find_package. Also adjust
the find_package(LLVM) to use an exact version match as well.

Reviewed By: arsenm, stellaraccident

Differential Revision: https://reviews.llvm.org/D138274
2022-11-20 21:09:50 +00:00
John Ericson 3a1c81e327 [CMake] Avoid `LLVM_BINARY_DIR` when other more specific variable are better-suited, part 2
A simple sed doing these substitutions:

- `${LLVM_BINARY_DIR}/lib${LLVM_LIBDIR_SUFFIX}\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`

where `\>` means "word boundary".

The only manual modifications were reverting changes in

- `runtimes/CMakeLists.txt`

because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.

There are some `${LLVM_BINARY_DIR}/lib` without the `${LLVM_LIBDIR_SUFFIX}`, but these refer to the lib subdirectory of the source (`llvm/lib`). That `lib` is automatically appended to make the local `CMAKE_CURRENT_BINARY_DIR` value by `add_subdirectory`; since the directory name in the source tree is fixed without any suffix, the corresponding `CMAKE_CURRENT_BINARY_DIR` will also be. We therefore do not replace it but leave it as-is.

This picks up where D133828 left off, getting the occurrences with*out* `CMAKE_CFG_INTDIR`. But this is difficult to do correctly and so not done in the (retroactively) previous diff.

This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.

Reviewed By: sebastian-ne

Differential Revision: https://reviews.llvm.org/D132316
2022-09-14 15:48:58 -04:00
Mehdi Amini 95201fcca0 Revert "[mlir][cmake] Don't add dependencies on mlir-(generic-)headers"
This reverts commit 7691b69d5b.

Bots are broken because we're missing CMake dependencies all around now.
2022-09-03 01:45:18 +00:00
Jeff Niu 7691b69d5b [mlir][cmake] Don't add dependencies on mlir-(generic-)headers
Every dialect was dependent on `mlir-headers`, which was causing the
build of any single MLIR dialect to pull in a bunch of extra
dependencies that aren't needed. Now, MLIR dialects will need to
explicitly depend on `MLIR*IncGen` targets to pull in any needed
headers.

This does not impact the actual `mlir-header` target.

Consider the "simple" Arithmetic dialect. Before:

```
% ninja MLIRArithmeticDialect
[151/812] Building CXX object lib/TableGen/CMakeFiles/LLVMTableGen.dir/JSONBackend.cpp.o
```

After:

```
% ninja MLIRArithmeticDialect
[207/374] Building CXX object tools/mlir/lib/TableGen/CMakeFiles/MLIRTableGen.dir/GenInfo.cpp.o
```

(Both clean builds)

Reviewed By: rriddle, jpienaar

Differential Revision: https://reviews.llvm.org/D133132
2022-09-02 12:13:49 -07:00
John Ericson 34fe6ddce1 Revert "[CMake] Avoid `LLVM_BINARY_DIR` when other more specific variable are better-suited"
This reverts commit ad8c34bc30.
2022-08-25 11:13:46 -04:00
John Ericson ad8c34bc30 [CMake] Avoid `LLVM_BINARY_DIR` when other more specific variable are better-suited
A simple sed doing these substitutions:

- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?lib(${LLVM_LIBDIR_SUFFIX})?\>` -> `${LLVM_LIBRARY_DIR}`
- `${LLVM_BINARY_DIR}/(\$\{CMAKE_CFG_INTDIR}/)?bin\>` -> `${LLVM_TOOLS_BINARY_DIR}`

where `\>` means "word boundary".

The only manual modifications were reverting changes in

- `compiler-rt/cmake/Modules/CompilerRTUtils.cmake
- `runtimes/CMakeLists.txt`

because these were "entry points" where we wanted to tread carefully not not introduce a "loop" which would end with an undefined variable being expanded to nothing.

This hopefully increases readability overall, and also decreases the usages of `LLVM_LIBDIR_SUFFIX`, preparing us for D130586.

Reviewed By: sebastian-ne

Differential Revision: https://reviews.llvm.org/D132316
2022-08-24 10:14:05 -04:00
John Ericson e941b031d3 Revert "[cmake] Use `CMAKE_INSTALL_LIBDIR` too"
This reverts commit f7a33090a9.

Unfortunately this causes a number of failures that didn't show up in my
local build.
2022-08-18 22:46:32 -04:00
John Ericson f7a33090a9 [cmake] Use `CMAKE_INSTALL_LIBDIR` too
We held off on this before as `LLVM_LIBDIR_SUFFIX` conflicted with it.
Now we return this.

`LLVM_LIBDIR_SUFFIX` is kept as a deprecated way to set
`CMAKE_INSTALL_LIBDIR`. The other `*_LIBDIR_SUFFIX` are just removed
entirely.

I imagine this is too potentially-breaking to make LLVM 15. That's fine.
I have a more minimal version of this in the disto (NixOS) patches for
LLVM 15 (like previous versions). This more expansive version I will
test harder after the release is cut.

Reviewed By: sebastian-ne, ldionne, #libc, #libc_abi

Differential Revision: https://reviews.llvm.org/D130586
2022-08-18 15:33:35 -04:00
Stella Laurenzo f55a6fde1d [mlir] Make sure that aggregate shared libraries define all of their symbols.
We were hitting issues on Linux where this was only being caught at runtime, and different linkers (BFD vs LLD) are differently strict in such situations. Such libraries will also fail to build properly on Windows (but test coverage of that is limited, so it is better to enforce globally).

Differential Revision: https://reviews.llvm.org/D131911
2022-08-15 12:02:24 -07:00
Sebastian Neubauer ae222dae56 [CMake] Fix add_subdirectory llvm builds
Fixes a regression from D117973, that used CMAKE_BINARY_DIR instead of
LLVM_BINARY_DIR in some places.

Differential Revision: https://reviews.llvm.org/D130555
2022-07-26 11:24:23 +02:00
John Ericson ac0d1d5c7b [cmake] Support custom package install paths
Firstly, we we make an additional GNUInstallDirs-style variable. With
NixOS, for example, this is crucial as we want those to go in
`${dev}/lib/cmake` not `${out}/lib/cmake` as that would a cmake subdir
of the "regular" libdir, which is installed even when no one needs to do
any development.

Secondly, we make *Config.cmake robust to absolute package install
paths. We for NixOS will in fact be passing them absolute paths to make
the `${dev}` vs `${out}` distinction mentioned above, and the
GNUInstallDirs-style variables are suposed to support absolute paths in
general so it's good practice besides the NixOS use-case.

Thirdly, we make `${project}_INSTALL_PACKAGE_DIR` CACHE PATHs like other
install dirs are.

Reviewed By: sebastian-ne

Differential Revision: https://reviews.llvm.org/D117973
2022-07-25 21:02:53 +00:00
Mike Urbach e0af5032f1 [mlir] Update Python CMake version requirement.
The minimum required version is now 3.19 due to the usage of some
more recent features. Update the version check and error message
accordingly. Also remove some logic that behaved differently before
3.18, since we can assume we are now on version 3.19+.

Reviewed By: stella.stamenova

Differential Revision: https://reviews.llvm.org/D130171
2022-07-25 11:29:33 -06:00
Sebastian Neubauer efe1527e28 [CMake] Copy folder without permissions
Copying the folder keeps the original permissions by default. This
creates problems when the source folder is read-only, e.g. in a
packaging environment.
Then, the copied folder in the build directory is read-only as well.
Later on, other files are copied into that directory (in the build
tree), failing when the directory is read-only.

Fix that problem by copying the folder without keeping the original
permissions.

Follow-up to D130254.

Differential Revision: https://reviews.llvm.org/D130338
2022-07-25 10:47:04 +02:00
John Ericson 07b749800c [cmake] Don't export `LLVM_TOOLS_INSTALL_DIR` anymore
First of all, `LLVM_TOOLS_INSTALL_DIR` put there breaks our NixOS
builds, because `LLVM_TOOLS_INSTALL_DIR` defined the same as
`CMAKE_INSTALL_BINDIR` becomes an *absolute* path, and then when
downstream projects try to install there too this breaks because our
builds always install to fresh directories for isolation's sake.

Second of all, note that `LLVM_TOOLS_INSTALL_DIR` stands out against the
other specially crafted `LLVM_CONFIG_*` variables substituted in
`llvm/cmake/modules/LLVMConfig.cmake.in`.

@beanz added it in d0e1c2a550 to fix a
dangling reference in `AddLLVM`, but I am suspicious of how this
variable doesn't follow the pattern.

Those other ones are carefully made to be build-time vs install-time
variables depending on which `LLVMConfig.cmake` is being generated, are
carefully made relative as appropriate, etc. etc. For my NixOS use-case
they are also fine because they are never used as downstream install
variables, only for reading not writing.

To avoid the problems I face, and restore symmetry, I deleted the
exported and arranged to have many `${project}_TOOLS_INSTALL_DIR`s.
`AddLLVM` now instead expects each project to define its own, and they
do so based on `CMAKE_INSTALL_BINDIR`. `LLVMConfig` still exports
`LLVM_TOOLS_BINARY_DIR` which is the location for the tools defined in
the usual way, matching the other remaining exported variables.

For the `AddLLVM` changes, I tried to copy the existing pattern of
internal vs non-internal or for LLVM vs for downstream function/macro
names, but it would good to confirm I did that correctly.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D117977
2022-07-21 19:04:00 +00:00
Stella Laurenzo 2aa6d56dce Restore Python install behavior from before D128230.
In D128230, we accidentally moved the install for Python sources outside of the loop, having one install() per group of files. While it would be nice if we could do this, it means that we flatten the relative directory tree and every source ends up in the root. The right way to do this is to use FILE_SETS, which preserve the relative directory tree, but they are not available until CMake 3.23.

Differential Revision: https://reviews.llvm.org/D129434
2022-07-09 19:22:51 -07:00
Stella Laurenzo aa78c5298e Fix MLIR Python CMake bug causing duplicate sources target.
The refactor in https://reviews.llvm.org/D128230 introduced a new target and the name is not scoped properly, leading to name collisions on larger projects. It is done properly on the target just below, so applying the same pattern here fixes the issue.
2022-07-04 07:07:53 -07:00
Stella Stamenova ac521d9ecd [mlir] Leverage CMake interface libraries for mlir python
This is already partially the case, but we can rely more heavily on interface libraries and how they are imported/exported in other to simplify the implementation of the mlir python functions in Cmake.

This change also makes a couple of other changes:
1) Add a new CMake function which handles "pure" sources. This was done inline previously
2) Moves the headers associated with CAPI libraries to the libraries themselves. These were previously managed in a separate source target. They can now be added directly to the CAPI libraries using DECLARED_HEADERS.
3) Cleanup some dependencies that showed up as an issue during the refactor

This is a big CMake change that should produce no impact on the build of mlir and on the produced *build tree*. However, this change fixes an issue with the *install tree* of mlir which was previously unusable for projects like torch-mlir because both the "pure" and "extension" targets were pointing to either the build or source trees.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D128230
2022-06-28 10:42:58 -07:00
Jacques Pienaar 983cb6c92f [mlir][pdll] Add new tablegen helper NFC
Command line option injected by tablegen rule cannot be respected by
PDLL here, so add new helper function that is copy of original without
any additional flags injected. This avoids compilation failure when
compiler warnings are disabled.

Kept it as a mechanical copy.

Fixes #55716
2022-06-23 05:31:32 -07:00
John Ericson 0bb317b7bf Revert "[cmake] Don't export `LLVM_TOOLS_INSTALL_DIR` anymore"
This reverts commit d5daa5c5b0.
2022-06-10 19:26:12 +00:00
John Ericson d5daa5c5b0 [cmake] Don't export `LLVM_TOOLS_INSTALL_DIR` anymore
First of all, `LLVM_TOOLS_INSTALL_DIR` put there breaks our NixOS
builds, because `LLVM_TOOLS_INSTALL_DIR` defined the same as
`CMAKE_INSTALL_BINDIR` becomes an *absolute* path, and then when
downstream projects try to install there too this breaks because our
builds always install to fresh directories for isolation's sake.

Second of all, note that `LLVM_TOOLS_INSTALL_DIR` stands out against the
other specially crafted `LLVM_CONFIG_*` variables substituted in
`llvm/cmake/modules/LLVMConfig.cmake.in`.

@beanz added it in d0e1c2a550 to fix a
dangling reference in `AddLLVM`, but I am suspicious of how this
variable doesn't follow the pattern.

Those other ones are carefully made to be build-time vs install-time
variables depending on which `LLVMConfig.cmake` is being generated, are
carefully made relative as appropriate, etc. etc. For my NixOS use-case
they are also fine because they are never used as downstream install
variables, only for reading not writing.

To avoid the problems I face, and restore symmetry, I deleted the
exported and arranged to have many `${project}_TOOLS_INSTALL_DIR`s.
`AddLLVM` now instead expects each project to define its own, and they
do so based on `CMAKE_INSTALL_BINDIR`. `LLVMConfig` still exports
`LLVM_TOOLS_BINARY_DIR` which is the location for the tools defined in
the usual way, matching the other remaining exported variables.

For the `AddLLVM` changes, I tried to copy the existing pattern of
internal vs non-internal or for LLVM vs for downstream function/macro
names, but it would good to confirm I did that correctly.

Reviewed By: nikic

Differential Revision: https://reviews.llvm.org/D117977
2022-06-10 14:35:18 +00:00
Alex Zinenko 3f71765a71 [mlir] provide Python bindings for the Transform dialect
Python bindings for extensions of the Transform dialect are defined in separate
Python source files that can be imported on-demand, i.e., that are not imported
with the "main" transform dialect. This requires a minor addition to the
ODS-based bindings generator. This approach is consistent with the current
model for downstream projects that are expected to bundle MLIR Python bindings:
such projects can include their custom extensions into the bundle similarly to
how they include their dialects.

Reviewed By: nicolasvasilache

Differential Revision: https://reviews.llvm.org/D126208
2022-05-30 17:37:52 +02:00
River Riddle dc9fb65c4f [mlir][Tablegen-LSP] Add support for a compilation database
This provides a format for externally specifying the include directories
for a source file. The format of the tablegen database is exactly the
same as that for PDLL, namely it includes the absolute source file name and
the set of include directories. The database format is shared to simplify
the infra, and also because the format itself is general enough to share. Even
if we desire to expand in the future to contain the actual compilation command,
nothing there is specific enough that we would need two different formats.

As with PDLL, support for generating the database is added to our mlir_tablegen
cmake command.

Differential Revision: https://reviews.llvm.org/D125441
2022-05-27 02:39:49 -07:00
Stella Stamenova 784a5bccfd [mlir] Fix python bindings build on Windows in Debug
Currently, building mlir with the python bindings enabled on Windows in Debug is broken because pybind11, python and cmake don't like to play together. This change normalizes how the three interact, so that the builds can now run and succeed.

The main issue is that python and cmake both make assumptions about which libraries are needed in a Windows build based on the flavor.
- cmake assumes that a debug (or a debug-like) flavor of the build will always require pythonX_d.lib and provides no option/hint to tell it to use a different library. cmake does find both the debug and release versions, but then uses the debug library.
- python (specifically pyconfig.h and by extension python.h) hardcodes the dependency on pythonX_d.lib or pythonX.lib depending on whether `_DEBUG` is defined. This is NOT transparent - it does not show up anywhere in the build logs until the link step fails with `pythonX_d.lib is missing` (or `pythonX.lib is missing`)
- pybind11 tries to "fix" this by implementing a workaround - unless Py_DEBUG is defined, `_DEBUG` is explicitly undefined right before including python headers. This also requires some windows headers to be included differently, so while clever, this is a non-trivial workaround.

mlir itself includes the pybind11 headers (which contain the workaround) AS WELL AS python.h, essentially always requiring both pythonX.lib and pythonX_d.lib for linking. cmake explicitly only adds one or the other, so the build fails.

This change does a couple of things:
- In the cmake files, explicitly add the release version of the python library on Windows builds regardless of flavor. Since Py_DEBUG is not defined, pybind11 will always require release and it will be satisfied
- To satisfy python as well, this change removes any explicit inclusions of Python.h on Windows instead relying on the fact that pybind11 headers will bring in what is needed

There are a few additional things that we could do but I rejected as unnecessary at this time:
- define Py_DEBUG based on the CMAKE_BUILD_TYPE - this will *mostly* work, we'd have to think through multiconfig generators like VS, but it's possible. There doesn't seem to be a need to link against debug python at the moment, so I chose not to overcomplicate the build and always default to release
- similar to above, but define Py_DEBUG based on the CMAKE_BUILD_TYPE *as well as* the presence of the debug python library (`Python3_LIBRARY_DEBUG`). Similar to above, this seems unnecessary right now. I think it's slightly better than above because most people don't actually have the debug version of python installed, so this would prevent breaks in that case.
- similar to the two above, but add a cmake variable to control the logic
- implement the pybind11 workaround directly in mlir (specifically in Interop.h) so that Python.h can still be included directly. This seems prone to error and a pain to maintain in lock step with pybind11
- reorganize how the pybind11 headers are included and place at least one of them in Interop.h directly, so that the header has all of its dependencies included as was the original intention. I decided against this because it really doesn't need pybind11 logic and it's always included after pybind11 is, so we don't necessarily need the python includes

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D125284
2022-05-09 19:46:47 -07:00
Stella Stamenova 057863a9bc [mlir] Fix build & test of mlir python bindings on Windows
There are a couple of issues with the python bindings on Windows:
- `create_symlink` requires special permissions on Windows - using `copy_if_different` instead allows the build to complete and then be usable
- the path to the `python_executable` is likely to contain spaces if python is installed in Program Files. llvm's python substitution adds extra quotes in order to account for this case, but mlir's own python substitution does not
- the location of the shared libraries is different on windows
- if the type is not specified for numpy arrays, they appear to be treated as strings

I've implemented the smallest possible changes for each of these in the patch, but I would actually prefer a slightly more comprehensive fix for the python_executable and the shared libraries.

For the python substitution, I think it makes sense to leverage the existing %python instead of adding %PYTHON and instead add a new variable for the case when preloading is needed. This would also make it clearer which tests are which and should be skipped on platforms where the preloading won't work.

For the shared libraries, I think it would make sense to pass the correct path and extension (possibly even the names) to the python script since these are known by lit and don't have to be hardcoded in the test at all.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D125122
2022-05-09 11:10:20 -07:00
River Riddle fb5a59f6e1 [mlir][PDLL] Add initial support for a PDLL compilation database
The compilation database acts in a similar way to the compilation database
(compile_commands.json) used by clang-tidy, i.e. it provides additional
information about the compilation of project files to help the language
server. The main piece of information provided by the PDLL compilation
database in this commit is the set of include directories used when processing
the input .pdll file. This allows for the server to properly process .pdll files
that use includes anchored by the include directories set up in the build system.

The structure of the textual form of a compilation database is a yaml file
containing documents of the following form:

```
--- !FileInfo:
  filepath: <string> - Absolute file path of the file.
  includes: <string> - Semi-colon delimited list of include directories.
```

This commit also adds support to cmake for automatically generating
a `pdll_compile_commands.yml` file at the top-level of the build
directory.

Differential Revision: https://reviews.llvm.org/D124076
2022-04-26 18:33:17 -07:00
River Riddle 597fde54a8 [mlir][PDLL] Add support for generating PDL patterns from PDLL at build time
This essentially sets up mlir-pdll to function in a similar manner to mlir-tblgen. Aside
from the boilerplate of configuring CMake and setting up a basic initial test, two new
options are added to mlir-pdll to mirror options provided by tblgen:

* -d
 This option generates a dependency file (i.e. a set of build time dependencies) while
 processing the input file.

* --write-if-changed
 This option only writes to the output file if the data would have changed, which for
 the build system prevents unnecesarry rebuilds if the file was touched but not actually
 changed.

Differential Revision: https://reviews.llvm.org/D124075
2022-04-26 18:33:16 -07:00
Mehdi Amini 03078ec20b Remove include_directories() from cmake `mlir_tablegen()` function (NFC)
This is present since the beginning, but does not seem needed by any
in-tree target right now. This seems like the kind of thing to populate
by the caller if needed.

Differential Revision: https://reviews.llvm.org/D121565
2022-03-14 04:36:24 +00:00
John Demme cb318526b0 [MLIR] [CMake/Python] Fix ignored Python3_EXECUTABLE
On Windows (at least), cmake ignores Python3_EXECUTABLE unless the
'Interpreter' component is being found. If the user is specifying a
different version than the latest installed (say, 3.8 vs 3.9) with the
Python3_EXECUTABLE, cmake was using a combination of the newest version
and the desired version. Mitigated by adding 'Interpreter' in the first
invocation like the second one.
2022-03-10 07:43:43 -08:00
rkayaith d75ac58117 [mlir] Add option to control python search priming
This adds an option to configure the CMake python search priming
behaviour that was introduced in D118148. In some environments the
priming would cause the "real" search to fail. The default behaviour is
unchanged, i.e. the search will be primed.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D120765
2022-03-01 14:52:56 -08:00
Stella Laurenzo b4626f22de [mlir] Do not use an empty source file when building aggregate libraries.
See discussion: https://discourse.llvm.org/t/check-mlir-times-examples-standalone-testing-time/6073/7

It turns out that it has been legal since CMake 3.11 to omit sources at library creation time if they are added later via target_sources, as is done here. This side-steps the issue of having a file that changes and invalidates the build of leaves.

Differential Revision: https://reviews.llvm.org/D119069
2022-02-05 23:06:28 -08:00
Stephen Neuendorffer 84fe34a0b7 [MLIR] Workaround for python detection problems.
Although cmake should be platform-independent, we've observed
that some aspects of Python detection don't work on all platforms,
even with recent versions of cmake.  This appears to be due to bugs
in the python detection logic, especially when the NumPy component
is required and not located within the python installation.
As a workaround, this patch first searches for "Development" before
searching for "Development.Module", which seems to workaround the
issue.

Differential Revision: https://reviews.llvm.org/D118148
2022-01-27 13:32:29 -08:00
Stella Stamenova 438f805145 [mlir] Configure mlir so that it can be directly included in a parent cmakelists.txt
The goal is to allow a project to directly include the `CMakeLists.txt` of mlir via something such as `add_directory`. This currently doesn't work because the exported targets then end up being imported in the same top-level project and `cmake` is not happy. LLVM works around this by guarding the exports based on the existence of the `LLVMSupport` library. This change does the same using `MLIRSupport`. In our experience, no targets need to be added to the export in the case.

If there's a preferred way of doing this or a better lib to use as the main target to check for, please let me know.

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D117599
2022-01-19 12:02:22 -08:00
John Ericson 5ad96992ef [mlir][cmake] Use `GNUInstallDirs` to support custom installation dirs
I am breaking apart D99484 so the cause of build failures is easier to
understand.

Differential Revision: https://reviews.llvm.org/D117540
2022-01-19 17:11:04 +00:00
Alex Zinenko 89a92fb3ba [mlir] Rework subclass construction in PybindAdaptors.h
The constructor function was being defined without indicating its "__init__"
name, which made it interpret it as a regular fuction rather than a
constructor. When overload resolution failed, Pybind would attempt to print the
arguments actually passed to the function, including "self", which is not
initialized since the constructor couldn't be called. This would result in
"__repr__" being called with "self" referencing an uninitialized MLIR C API
object, which in turn would cause undefined behavior when attempting to print
in C++. Even if the correct name is provided, the mechanism used by
PybindAdaptors.h to bind constructors directly as "__init__" functions taking
"self" is deprecated by Pybind. The new mechanism does not seem to have access
to a fully-constructed "self" object (i.e., the constructor in C++ takes a
`pybind11::detail::value_and_holder` that cannot be forwarded back to Python).

Instead, redefine "__new__" to perform the required checks (there are no
additional initialization needed for attributes and types as they are all
wrappers around a C++ pointer). "__new__" can call its equivalent on a
superclass without needing "self".

Bump pybind11 dependency to 3.8.0, which is the first version that allows one
to redefine "__new__".

Reviewed By: stellaraccident

Differential Revision: https://reviews.llvm.org/D117646
2022-01-19 18:09:05 +01:00
John Ericson da77db58d7 Revert "[cmake] Use `GNUInstallDirs` to support custom installation dirs."
https://lab.llvm.org/buildbot/#/builders/46/builds/21146 Still have
this odd error, not sure how to reproduce, so I will just try breaking
up my patch.

This reverts commit 4a678f8072.
2022-01-16 05:48:30 +00:00
John Ericson 4a678f8072 [cmake] Use `GNUInstallDirs` to support custom installation dirs.
This is the original patch in my GNUInstallDirs series, now last to merge as the final piece!

It arose as a new draft of D28234. I initially did the unorthodox thing of pushing to that when I wasn't the original author, but since I ended up

 - Using `GNUInstallDirs`, rather than mimicking it, as the original author was hesitant to do but others requested.

 - Converting all the packages, not just LLVM, effecting many more projects than LLVM itself.

I figured it was time to make a new revision.

I have used this patch series (and many back-ports) as the basis of https://github.com/NixOS/nixpkgs/pull/111487 for my distro (NixOS), which was merged last spring (2021). It looked like people were generally on board in D28234, but I make note of this here in case extra motivation is useful.

---

As pointed out in the original issue, a central tension is that LLVM already has some partial support for these sorts of things. Variables like `COMPILER_RT_INSTALL_PATH` have already been dealt with. Variables like `LLVM_LIBDIR_SUFFIX` however, will require further work, so that we may use `CMAKE_INSTALL_LIBDIR`.

These remaining items will be addressed in further patches. What is here is now rote and so we should get it out of the way before dealing more intricately with the remainder.

Reviewed By: #libunwind, #libc, #libc_abi, compnerd

Differential Revision: https://reviews.llvm.org/D99484
2022-01-16 05:33:07 +00:00
John Ericson 6e52bfe09d Revert "[cmake] Use `GNUInstallDirs` to support custom installation dirs."
Sorry for the disruption, I will try again later.

This reverts commit efeb501970.
2022-01-15 07:35:02 +00:00
John Ericson efeb501970 [cmake] Use `GNUInstallDirs` to support custom installation dirs.
This is the original patch in my GNUInstallDirs series, now last to merge as the final piece!

It arose as a new draft of D28234. I initially did the unorthodox thing of pushing to that when I wasn't the original author, but since I ended up

 - Using `GNUInstallDirs`, rather than mimicking it, as the original author was hesitant to do but others requested.

 - Converting all the packages, not just LLVM, effecting many more projects than LLVM itself.

I figured it was time to make a new revision.

I have used this patch series (and many back-ports) as the basis of https://github.com/NixOS/nixpkgs/pull/111487 for my distro (NixOS), which was merged last spring (2021). It looked like people were generally on board in D28234, but I make note of this here in case extra motivation is useful.

---

As pointed out in the original issue, a central tension is that LLVM already has some partial support for these sorts of things. Variables like `COMPILER_RT_INSTALL_PATH` have already been dealt with. Variables like `LLVM_LIBDIR_SUFFIX` however, will require further work, so that we may use `CMAKE_INSTALL_LIBDIR`.

These remaining items will be addressed in further patches. What is here is now rote and so we should get it out of the way before dealing more intricately with the remainder.

Reviewed By: #libunwind, #libc, #libc_abi, compnerd

Differential Revision: https://reviews.llvm.org/D99484
2022-01-15 01:08:35 +00:00
Stephen Neuendorffer 3523876873 Fix exported MLIR_TABLEGEN_EXE
LLVM_OPTIMIZED_TABLEGEN results in MLIR_TABLEGEN_EXE pointing to
an absolute path in the build directory.  This doesn't work
when exporting to an install directory.  This patch fixes the exported
information for an install directory to refer to the installed
mlir-tblgen.  (Note that this is probably a debug version if
LLVM_OPTIMIZED_TABLEGEN is set)
2022-01-09 23:27:08 -08:00
John Ericson 44e3365775 [CMake] Factor out config prefix finding logic
See the docs in the new function for details.

 I think I found every instance of this copy pasted code. Polly could
 also use it, but currently does something different, so I will save the
 behavior change for a future revision.

We get the shared, non-installed CMake modules following the pattern
established in D116472.

It might be good to have LLD and Flang also use this, but that would be
a functional change and so I leave it as future work.

Reviewed By: beanz, lebedev.ri

Differential Revision: https://reviews.llvm.org/D116521
2022-01-07 20:16:18 +00:00
Mogball ca6bd9cd43 [mlir][ods] AttrOrTypeGen uses Class
AttrOrType def generator uses `Class` code gen helper,
instead of naked raw_ostream.

Depends on D113714 and D114807

Reviewed By: rriddle

Differential Revision: https://reviews.llvm.org/D113715
2021-12-01 16:53:23 +00:00
Stella Laurenzo 132bc6e2d4 Re-apply "[mlir] Allow out-of-tree python building from installed MLIR."
Re-applies D111513:
* Adds a full-fledged Python example dialect and tests to the Standalone example (need to do a bit of tweaking in the top level CMake and lit tests to adapt better to if not building with Python enabled).
* Rips out remnants of custom extension building in favor of pybind11_add_module which does the right thing.
* Makes python and extension sources installable (outputs to src/python/${name} in the install tree): Both Python and C++ extension sources get installed as downstreams need all of this in order to build a derived version of the API.
* Exports sources targets (with our properties that make everything work) by converting them to INTERFACE libraries (which have export support), as recommended for the forseeable future by CMake devs. Renames custom properties to start with lower-case letter, as also recommended/required (groan).
* Adds a ROOT_DIR argument to declare_mlir_python_extension since now all C++ sources for an extension must be under the same directory (to line up at install time).
* Downstreams will need to adapt by:

  * Remove absolute paths from any SOURCES for declare_mlir_python_extension (I believe all downstreams are just using ${CMAKE_CURRENT_SOURCE_DIR} here, which can just be ommitted). May need to set ROOT_DIR if not relative to the current source directory.
  * To allow further downstreams to install/build, will need to make sure that all C++ extension headers are also listed under SOURCES for declare_mlir_python_extension.

This reverts commit 1a6c26d1f5.

Reviewed By: stephenneuendorffer

Differential Revision: https://reviews.llvm.org/D113732
2021-11-14 20:31:34 -08:00
Mehdi Amini 1a6c26d1f5 Revert "[mlir] Allow out-of-tree python building from installed MLIR."
This reverts commit c7be8b7539.

Build is broken (multiple buildbots)
2021-11-12 02:30:53 +00:00
Stella Laurenzo c7be8b7539 [mlir] Allow out-of-tree python building from installed MLIR.
* Depends on D111504, which provides the boilerplate for building aggregate shared libraries from installed MLIR.
* Adds a full-fledged Python example dialect and tests to the Standalone example (need to do a bit of tweaking in the top level CMake and lit tests to adapt better to if not building with Python enabled).
* Rips out remnants of custom extension building in favor of `pybind11_add_module` which does the right thing.
* Makes python and extension sources installable (outputs to src/python/${name} in the install tree): Both Python and C++ extension sources get installed as downstreams need all of this in order to build a derived version of the API.
* Exports sources targets (with our properties that make everything work) by converting them to INTERFACE libraries (which have export support), as recommended for the forseeable future by CMake devs. Renames custom properties to start with lower-case letter, as also recommended/required (groan).
* Adds a ROOT_DIR argument to `declare_mlir_python_extension` since now all C++ sources for an extension must be under the same directory (to line up at install time).
* Need to validate against a downstream or two and adjust, prior to submitting.

Downstreams will need to adapt by:

* Remove absolute paths from any SOURCES for `declare_mlir_python_extension` (I believe all downstreams are just using `${CMAKE_CURRENT_SOURCE_DIR}` here, which can just be ommitted). May need to set `ROOT_DIR` if not relative to the current source directory.
* To allow further downstreams to install/build, will need to make sure that all C++ extension headers are also listed under SOURCES for `declare_mlir_python_extension`.

Reviewed By: stephenneuendorffer, mikeurbach

Differential Revision: https://reviews.llvm.org/D111513
2021-11-11 18:04:31 -08:00
Stella Laurenzo a897590f11 Add MLIR_INSTALL_AGGREGATE_OBJECTS and default it to ON.
* Package maintainers can opt to disable installation of these objects.
* Per discussion on https://reviews.llvm.org/D111504

Differential Revision: https://reviews.llvm.org/D112090
2021-10-19 16:14:04 -07:00
Stella Laurenzo 8e54f24f89 Disable add_mlir_aggregate() debug file generation.
* Leaves it as a commented out area with a note on how to debug.
2021-10-14 14:32:27 -07:00
Stella Laurenzo fe6d9937b3 [mlir] Ability to build CAPI dylibs from out of tree projects against installed LLVM.
* Incorporates a reworked version of D106419 (which I have closed but has comments on it).
* Extends the standalone example to include a minimal CAPI (for registering its dialect) and a test which, from out of tree, creates an aggregate dylib and links a little sample program against it. This will likely only work today in *static* MLIR builds (until the TypeID fiasco is finally put to bed). It should work on all platforms, though (including Windows - albeit I haven't tried this exact incarnation there).
* This is the biggest pre-requisite to being able to build out of tree MLIR Python-based projects from an installed MLIR/LLVM.
* I am rather nauseated by the CMake shenanigans I had to endure to get this working. The primary complexity, above and beyond the previous patch is because (with no reason given), it is impossible to export target properties that contain generator expressions... because, of course it isn't. In this case, the primary reason we use generator expressions on the individual embedded libraries is to support arbitrary ordering. Since that need doesn't apply to out of tree (which import everything via FindPackage at the outset), we fall back to a more imperative way of doing the same thing if we detect that the target was imported. Gross, but I don't expect it to need a lot of maintenance.
* There should be a relatively straight-forward path from here to rebase libMLIR.so on top of this facility and also make it include the CAPI.

Differential Revision: https://reviews.llvm.org/D111504
2021-10-13 18:45:55 -07:00