Commit Graph

700 Commits

Author SHA1 Message Date
Kazu Hirata c68af42fa8 [lld] Use std::nullopt instead of None (NFC)
This patch mechanically replaces None with std::nullopt where the
compiler would warn if None were deprecated.  The intent is to reduce
the amount of manual work required in migrating from Optional to
std::optional.

This is part of an effort to migrate from llvm::Optional to
std::optional:

https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-12-02 23:12:36 -08:00
Guillaume Chatelet cfe77f23d6 [lld][Alignment][NFC] Use Align instead of log2 of alignment in Wasm Sections
I intend to slowly upgrade all alignments to the Align type in lld as well.
Some places talk about alignment in Bytes while other specify them as Log2(Bytes).
Let's make sure all of this is coherent.

Differential Revision: https://reviews.llvm.org/D139181
2022-12-02 12:47:41 +00:00
Guillaume Chatelet 173f62d98f [Alignment][NFC] Use Align in StringTableBuilder ctor 2022-12-02 12:43:01 +00:00
Zequan Wu 84be92d26f Reland "[LTO][COFF] Use bitcode file names in lto native object file names."
This reverts commit 3410808294 with fixes.
2022-11-22 13:46:42 -08:00
Roman Lebedev 3410808294
Revert "Reland "[LTO][COFF] Use bitcode file names in lto native object file names.""
Breaks build of LLVMgold here:
```
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1108:19: error: no matching function for call to 'localCache'
    Cache = check(localCache("ThinLTO", "Thin", options::cache_dir, AddBuffer));
                  ^~~~~~~~~~
/repositories/llvm-project/llvm/include/llvm/Support/Caching.h:72:21: note: candidate function not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1102:20)' to 'llvm::AddBufferFn' (aka 'function<void (unsigned int, const llvm::Twine &, std::unique_ptr<MemoryBuffer>)>') for 4th argument
Expected<FileCache> localCache(
                    ^
/repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1110:18: error: no viable conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'llvm::AddStreamFn' (aka 'function<Expected<std::unique_ptr<CachedFileStream>> (unsigned int, const llvm::Twine &)>')
  check(Lto->run(AddStream, Cache));
                 ^~~~~~~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:375:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::nullptr_t' for 1st argument
      function(nullptr_t) noexcept
      ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:386:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'const std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &' for 1st argument
      function(const function& __x)
      ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:404:7: note: candidate constructor not viable: no known conversion from '(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20)' to 'std::function<llvm::Expected<std::unique_ptr<llvm::CachedFileStream>> (unsigned int, const llvm::Twine &)> &&' for 1st argument
      function(function&& __x) noexcept
      ^
/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../include/c++/12/bits/std_function.h:435:2: note: candidate template ignored: requirement '_Callable<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20), std::__invoke_result<(lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &, unsigned int, const llvm::Twine &>>::value' was not satisfied [with _Functor = (lambda at /repositories/llvm-project/llvm/tools/gold/gold-plugin.cpp:1094:20) &]
        function(_Functor&& __f)
        ^
/repositories/llvm-project/llvm/include/llvm/LTO/LTO.h:278:25: note: passing argument to parameter 'AddStream' here
  Error run(AddStreamFn AddStream, FileCache Cache = nullptr);
                        ^

```

This reverts commit 387620aa8c.
2022-11-23 00:40:12 +03:00
Zequan Wu 387620aa8c Reland "[LTO][COFF] Use bitcode file names in lto native object file names."
This reverts commit eef5405f74.
2022-11-22 11:26:18 -08:00
Zequan Wu eef5405f74 Revert "[LTO][COFF] Use bitcode file names in lto native object file names."
This reverts commit 531ed6d5aa.
2022-11-22 10:55:05 -08:00
Zequan Wu 531ed6d5aa [LTO][COFF] Use bitcode file names in lto native object file names.
Currently the lto native object files have names like main.exe.lto.1.obj. In
PDB, those names are used as names for each compiland. Microsoft’s tool
SizeBench uses those names to present to users the size of each object files.
So, names like main.exe.lto.1.obj is not user friendly.

This patch makes the lto native object file names more readable by using
the bitcode file names as part of the file names. For example, if the input
bitcode file has path like "path/to/foo.obj", its corresponding lto native
object file path would be "path/to/main.exe.lto.foo.obj". Since the lto native
object file name only bothers PDB, this patch only changes the lld-linker's
behavior.

Reviewed By: tejohnson, MaskRay, #lld-macho

Differential Revision: https://reviews.llvm.org/D137217
2022-11-22 10:19:58 -08:00
Ying Yi ca875cb4fe [ThinLTO] a ThinLTO warning is added if cache_size_bytes or cache_size_files is too small for the current link job. The warning recommends the user to consider adjusting --thinlto-cache-policy.
A specific case for ThinLTO cache pruning is that the current build is huge, and the cache wasn't big enough to hold the intermediate object files of that build. So in doing that build, a file would be cached, and later in that same build it would be evicted. This was significantly decreasing the effectiveness of the cache. By giving this warning, the user could identify the required cache size/files and improve ThinLTO link speed.

Differential Revision: https://reviews.llvm.org/D135590
2022-11-14 12:27:29 +00:00
Fangrui Song 640d9b3296 [lld] Fix duplicate word typos. NFC
Based on lld/ part of D137338 but reflowed comments.
2022-11-08 17:28:04 -08:00
Sam Clegg 3c28a6d2cb [lld][WebAssembly] Add support for LLD_REPRODUCE
This is feature of the ELF and COFF linker that we were missing until
now.

Differential Revision: https://reviews.llvm.org/D137377
2022-11-03 16:55:37 -07:00
Florian Mayer 2390bb2347 [NFC] use has_value instead of hasValue 2022-10-31 16:43:14 -07:00
Dan Gohman d4c8a0edca [wasm-ld] Allow importing/exporting the output module's memory with arbitrary names
This adds an `--export-memory` option to wasm-ld which allows passing
a name to give to the exported memory, and extends `--import-memory` to
allow passing a <module>,<name> pair specifying where the memory should
be imported from.

This is based on https://reviews.llvm.org/D131376, with the main
difference being that it only supports exporting memory by one name.

Differential Revision: https://reviews.llvm.org/D135898
2022-10-31 13:59:46 -07:00
Sam Clegg 6912ed7b8f [lld][WebAssembly] Don't allow `--global-base` to be specified in -share/-pie or --relocatable modes
Add some checks around this combination of flags

Also, honor `--global-base` when specified in `--stack-first` mode
rather than ignoring it.  But error out if the specified base preseeds
the end of the stack.

Differential Revision: https://reviews.llvm.org/D136117
2022-10-18 17:19:26 -07:00
Dan Gohman 4b24e9be17 [wasm-ld] Define a `__heap_end` symbol marking the end of allocated memory.
Define a `__heap_end` symbol that marks the end of the memory region
that starts at `__heap_base`. This will allow malloc implementations to
know how much memory they can use at `__heap_base` even if someone has
done a `memory.grow` before they can initialize their state.

Differential Revision: https://reviews.llvm.org/D136110
2022-10-17 16:39:06 -07:00
Jez Ng 32647c8f53 [lld][nfc] Remove lld::demangle() (partial revert of D116279)
{D116279}, in addition to adding support for other demanglers, also
factored out some of the demangling logic. However, I don't think the
abstraction really carries its weight -- after {D135942}, only the ELF
and WASM backends call it with anything other than a non-constant
`shouldDemangle` argument. The COFF and Mach-O backends were already
doing the should-demangle check before calling `demangle()`.

Reviewed By: MaskRay, #lld-macho

Differential Revision: https://reviews.llvm.org/D135943
2022-10-14 15:28:47 -04:00
Sam Clegg 1532be98f9 [lld][WebAssembly] Add symbols marking start/end of stack region
Currently emscripten is make assumptions about that memory layout,
assuming the stack is between `__data_end` and `__heap_base`:

af961ad5c4/system/lib/compiler-rt/stack_limits.S (L42-L61)

With this change we can be more precise:

https://github.com/emscripten-core/emscripten/pull/18057

Differential Revision: https://reviews.llvm.org/D135910
2022-10-13 14:52:17 -07:00
Sam Clegg c07e838130 [lld][WebAssembly] Add `--extra-features` flag to add addional features
This flag acts just like the existing `--features` flag but instead
of replacing the set of inferred features it adds to it.

This is useful for example if you want to `--export` a mutable global
but none of the input of object were built with mutable global support.
In that case you can do `--extra-features=mutable-globals` to avoid the
linker error that would otherwise be generated in this case:

wasm-ld: error: mutable global exported but 'mutable-globals' feature not present in inputs: `__stack_pointer`. Use --no-check-features to suppress.

Differential Revision: https://reviews.llvm.org/D135831
2022-10-13 09:25:02 -07:00
Sam Clegg 0a9756fc15 [lld][WebAssemlby] Improve support for -L / -l and add testing
- Add support -Bdynamic/-Bstatic and their aliases
- Add support for `--library` and `--library-path` long form args
- Add test based on test/ELF/libsearch.s
- In `-Bdynamic` mode search for `.so` files in preference to `.a`.
- Unlike ELF continue to default to static mode until `-pie` or
  `-shared` are used.

Differential Revision: https://reviews.llvm.org/D135087
2022-10-03 16:53:30 -07:00
Kazu Hirata f46245c3e9 [lld] Don't include SetVector.h (NFC) 2022-09-17 13:36:15 -07:00
Nico Weber cd7ffa2e52 lld: Include name of output file in "failed to write output" diag
Differential Revision: https://reviews.llvm.org/D133110
2022-09-14 14:57:47 -04:00
Kazu Hirata 3850edd9e0 Use llvm::count_if (NFC) 2022-09-03 11:17:35 -07:00
Sam Clegg 113b568829 [lld][WebAssembly] Rename SymbolTable::getSymbols to match ELF backend. NFC
The ELF backend originally used `getSymbols()` but went though a
sequence of changes that resulted in this method being called
`symbols()`.

d8f8abbd4a replaced `getSymbols()` with
`forEachSymbol`.

a2fc964417 replaced `forEachSymbol` with
`llvm::iterator_range`.

e9262edf0d replaced `llvm::iterator_range`
with `symbols()`.

Differential Revision: https://reviews.llvm.org/D131284
2022-08-31 14:33:45 -07:00
Kazu Hirata f5a68feab3 Use llvm::none_of (NFC) 2022-08-14 16:25:39 -07:00
Fangrui Song 4b2b68d5ab [lld] Change vector to SmallVector. NFC
My lld executable is 1.6KiB smaller and some functions are now more efficient.
2022-07-30 18:11:21 -07:00
Sam Clegg b0f18af30b [lld][WebAssemby] Demote LazySymbols back to undefined symbols if they are not loaded
A LazySymbol is one that lives in `.a` archive and gets pulled in by a
strong reference.  However, weak references to such symbols do not
result in them be loaded from the archive.  In this case we want to
treat such symbols at undefined rather then lazy, once symbols
resolution is complete.

This fixes a crash bug in the linker when weakly referenced symbol that
lives in an archive file is live at the end of the link.  In the case of
dynamic linking this is expected to turn into an import with (in the
case of a function symbol) a function index.

Differential Revision: https://reviews.llvm.org/D130736
2022-07-29 13:53:54 -07:00
Kazu Hirata 70257fab68 Use any_of (NFC) 2022-07-22 01:05:17 -07:00
Kazu Hirata 5cff5142a8 Use value instead of getValue (NFC) 2022-07-15 20:03:13 -07:00
Kazu Hirata e5f568a49f Use has_value instead of hasValue (NFC) 2022-07-13 01:58:03 -07:00
Sam Clegg 53217ecb88 [lld][WebAssembly] Don't apply data relocations at static constructor time
Instead, export `__wasm_apply_data_relocs` and `__wasm_call_ctors`
separately.

This is required since user code in a shared library (such as static
constructors) should not be run until relocations have been applied to
all loaded libraries.

See: https://github.com/emscripten-core/emscripten/issues/17295

Differential Revision: https://reviews.llvm.org/D128515
2022-06-27 15:50:02 -07:00
Kazu Hirata 586fb81eee [lld] Don't use Optional::hasValue (NFC)
This patch replaces x.hasValue() with x where x is contextually
convertible to bool.
2022-06-26 19:37:14 -07:00
Kazu Hirata 3b7c3a654c Revert "Don't use Optional::hasValue (NFC)"
This reverts commit aa8feeefd3.
2022-06-25 11:56:50 -07:00
Kazu Hirata aa8feeefd3 Don't use Optional::hasValue (NFC) 2022-06-25 11:55:57 -07:00
Kazu Hirata 064a08cd95 Don't use Optional::hasValue (NFC) 2022-06-20 20:05:16 -07:00
Kazu Hirata 5413bf1bac Don't use Optional::hasValue (NFC) 2022-06-20 11:33:56 -07:00
Nico Weber 8c589939f5 fix comment typos to cycle bots 2022-06-19 18:34:12 -04:00
Nico Weber 7effcbda49 Rename parallelForEachN to just parallelFor
Patch created by running:

  rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: https://reviews.llvm.org/D128140
2022-06-19 17:49:00 -04:00
Kazu Hirata 757d9d22cd [lld] Use value_or instead of getValueOr (NFC) 2022-06-19 00:29:41 -07:00
Sam Clegg 457f38a7b0 [lld][WebAssembly] Revert moving of data relocations to start function
Back in https://reviews.llvm.org/D117412 we moved the application of
data reloctions to the wasm start function.

However, because the dynamic linker doesn't know the final addresses
at module instantiation time, this proved to be too early and the
relocations could be applied with the wrong values.

Fixes: https://github.com/emscripten-core/emscripten/issues/17150

Differential Revision: https://reviews.llvm.org/D127333
2022-06-09 17:49:35 -07:00
Fangrui Song 025b309631 Revert D126950 "[lld][WebAssembly] Retain data segments referenced via __start/__stop"
This reverts commit dcf3368e33.

It breaks -DLLVM_ENABLE_ASSERTIONS=on builds. In addition, the description is
incorrect about ld.lld behavior. For wasm, there should be justification to add
the new mode.
2022-06-03 22:18:06 -07:00
Yuta Saito dcf3368e33 [lld][WebAssembly] Retain data segments referenced via __start/__stop
As well as ELF linker does, retain all data segments named X referenced
through `__start_X` or `__stop_X`.

For example, `FOO_MD` should not be stripped in the below case, but it's currently mis-stripped

```llvm
@FOO_MD  = global [4 x i8] c"bar\00", section "foo_md", align 1
@__start_foo_md = external constant i8*
@__stop_foo_md = external constant i8*
@llvm.used = appending global [1 x i8*] [i8* bitcast (i32 ()* @foo_md_size to i8*)], section "llvm.metadata"

define i32 @foo_md_size()  {
entry:
  ret i32 sub (
    i32 ptrtoint (i8** @__stop_foo_md to i32),
    i32 ptrtoint (i8** @__start_foo_md to i32)
  )
}
```

This fixes https://github.com/llvm/llvm-project/issues/55839

Reviewed By: sbc100

Differential Revision: https://reviews.llvm.org/D126950
2022-06-04 02:28:31 +00:00
Sam Clegg 87099a0438 [lld][WebAssembly] Remove unnecessary accessor methods. NFC
This is less code, and matches more closely the ELF linker.

Differential Revision: https://reviews.llvm.org/D126979
2022-06-03 11:43:44 -07:00
Sam Clegg 0e8f4ce31d [lld][WebAssembly] Fix crash on undefined+weak function syms in LTO objects
Symbols from LTO objects don't contain Wasm signatures, but we need a
signature when we create undefined/stub functions for missing weakly
undefined symbols.

Luckily, after LTO, we know that symbols that are not referenced by a
regular object file must not be needed in the final output so there
is no need to generate undefined/stub function for them.

Differential Revision: https://reviews.llvm.org/D126554
2022-05-27 11:41:34 -07:00
Derek Schuff a205f2904d [WebAssembly] Consolidate sectionTypeToString in BinaryFormat [NFC]
Currently there are 2 duplicate implementation, and I want to add
a use in a 3rd place. Combine them in lib/BinaryFormat so they can
be shared.

Also update toString for symbol and reloc types to use StringRef

Differential Revision: https://reviews.llvm.org/D126553
2022-05-27 09:26:36 -07:00
Sam Clegg 87628f5804 [lld][WebAssembly] Require double dash for modern linker flags
This matches the behaviour of the ELF backend (in fact this change
is mostly just copying directly from ELF/Options.td).

Differential Revision: https://reviews.llvm.org/D126500
2022-05-26 14:42:52 -07:00
Sam Clegg 198815e18d [lld][WebAssembly] Avoid importing/exporting hidden symbols in shared libraries
We have some special handling for weakly defined symbols where we both
import and export them, but this is not needed for hidden symbols which
should never be imported or exported.

See https://github.com/emscripten-core/emscripten/pull/16972

This should also help with:
https://github.com/emscripten-core/emscripten/issues/15487

Differential Revision: https://reviews.llvm.org/D126491
2022-05-26 13:43:25 -07:00
Sam Clegg 966427b847 [lld][WebAssemlby] Check for command line flags with missing arguments
I'm really not sure how this was overlooked when we first ported lld
to Wasm.  The upstream code in the ELF backend has these two lines but
for some reason they never make it into the Wasm version.

Differential Revision: https://reviews.llvm.org/D126497
2022-05-26 13:35:27 -07:00
Sam Clegg 74f9841977 [lld][WebAssembly] Allow use of statically allocated TLS region.
It turns out we were already allocating static address space for TLS
data along with the non-TLS static data, but this space was going
unused/ignored.

With this change, we include the TLS segment in `__wasm_init_memory`
(which does the work of loading the passive segments into memory when a
module is first loaded).  We also set the `__tls_base` global to point
to the start of this segment.

This means that the runtime can use this static copy of the TLS data for
the first/primary thread if it chooses, rather than doing a runtime
allocation prior to calling `__wasm_init_tls`.

Practically speaking, this will allow emscripten to avoid dynamic
allocation of TLS region on the main thread.

Differential Revision: https://reviews.llvm.org/D126107
2022-05-23 17:27:17 -07:00
Nikita Popov b8f50abd04 [lld] Remove support for legacy pass manager
This removes options for performing LTO with the legacy pass
manager in LLD. Options that explicitly enable the new pass manager
are retained as no-ops.

Differential Revision: https://reviews.llvm.org/D123219
2022-04-07 10:17:31 +02:00
Nikita Popov ed4e6e0398 [cmake] Remove LLVM_ENABLE_NEW_PASS_MANAGER cmake option
Or rather, error out if it is set to something other than ON. This
removes the ability to enable the legacy pass manager by default,
but does not remove the ability to explicitly enable it through
various flags like -flegacy-pass-manager or -enable-new-pm=0.

I checked, and our test suite definitely doesn't pass with
LLVM_ENABLE_NEW_PASS_MANAGER=OFF anymore.

Differential Revision: https://reviews.llvm.org/D123126
2022-04-06 09:52:21 +02:00