Commit Graph

741 Commits

Author SHA1 Message Date
Fangrui Song b067aa56ce [ELF] Produce DT_RISCV_VARIANT_CC
https://github.com/riscv/riscv-elf-psabi-doc/pull/190 introduced STO_RISCV_VARIANT_CC.
The linker should:

* Copy the STO_RISCV_VARIANT_CC bit to .symtab/.dynsym: already fulfilled after
  82ed93ea05
* Produce DT_RISCV_VARIANT_CC if at least one R_RISCV_JUMP_SLOT relocation
  references a symbol with the STO_RISCV_VARIANT_CC bit. Done by this patch.

Reviewed By: kito-cheng

Differential Revision: https://reviews.llvm.org/D107951
2022-12-04 21:08:37 -08:00
Guillaume Chatelet 173f62d98f [Alignment][NFC] Use Align in StringTableBuilder ctor 2022-12-02 12:43:01 +00:00
Guillaume Chatelet 08e2a76381 [lld][NFC] rename ELF alignment into addralign 2022-12-01 16:20:12 +00:00
Fangrui Song 4191fda69c [ELF] Change most llvm::Optional to std::optional
https://discourse.llvm.org/t/deprecating-llvm-optional-x-hasvalue-getvalue-getvalueor/63716
2022-11-26 19:19:15 -08:00
Fangrui Song c3c9e45312 [ELF] Add InputSectionBase::{addRelocs,relocs} and GotSection::addConstant to add/access relocations
to prepare for changing `relocations` from a SmallVector to a pointer.

Also change the `isec` parameter in `addAddendOnlyRelocIfNonPreemptible` to `GotSection &`.
2022-11-21 04:12:03 +00:00
Fangrui Song 2bf5d86422 [ELF] Change rawData to content() and data() to contentMaybeDecompress()
Clarify data() which may trigger decompression and make it feasible to refactor
the member variable rawData.
2022-11-20 22:43:22 +00:00
Andreas Hollandt a86cfceb44 [ELF] Add gdb index time trace
Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D135659
2022-11-06 15:13:32 -08:00
Fangrui Song cb30072a72 [ELF] Fix duplicate work typo. NFC 2022-11-03 23:10:19 -07:00
Fangrui Song 685b212553 [ELF] Make relocateAlloc target specific. NFC
The target-specific code (AArch64, PPC64) does not fit into the generic code and
adds virtual function overhead. Move relocateAlloc into ELF/Arch/ instead. This
removes many virtual functions (relaxTls*). In addition, this helps get rid of
getRelocTargetVA dispatch and many RelExpr members in the future.
2022-10-17 11:01:11 -07:00
Fangrui Song 2b153088be [ELF] Set DF_STATIC_TLS for AArch64/PPC32/PPC64 2022-10-16 12:08:08 -07:00
Fangrui Song 14f996dca8 [ELF] Move inputSections/ehInputSections into Ctx. NFC 2022-10-16 00:49:48 -07:00
Fangrui Song 9c626d4a0d [ELF] Remove symtab indirection. NFC
Add LLVM_LIBRARY_VISIBILITY to remove unneeded GOT and unique_ptr indirection.
2022-10-01 14:46:49 -07:00
Fangrui Song 34fa860048 [ELF] Remove ctx indirection. NFC
Add LLVM_LIBRARY_VISIBILITY to remove unneeded GOT and unique_ptr
indirection. We can move other global variables into ctx without
indirection concern. In the long term we may consider passing Ctx
as a parameter to various functions and eliminate global state as
much as possible and then remove `Ctx::reset`.
2022-10-01 12:06:33 -07:00
Fangrui Song e3ecc6a912 [ELF] Make symAux[0] a sentinel
And default auxIdx to 0.
2022-09-29 00:50:19 -07:00
Fangrui Song 7dac9f4e48 [ELF] Make --pack-dyn-relocs=android compatible with parallel relocation scanning 2022-09-28 06:58:58 +00:00
Fangrui Song 12607f57da [ELF] Cache compute_thread_count. NFC 2022-09-12 19:09:08 -07:00
Fangrui Song e6aebff674 [ELF] Parallelize relocation scanning
* Change `Symbol::flags` to a `std::atomic<uint16_t>`
* Add `llvm::parallel::threadIndex` as a thread-local non-negative integer
* Add `relocsVec` to part.relaDyn and part.relrDyn so that relative relocations can be added without a mutex
* Arbitrarily change -z nocombreloc to move relative relocations to the end. Disable parallelism for deterministic output.

MIPS and PPC64 use global states for relocation scanning. Keep serial scanning.

Speed-up with mimalloc and --threads=8 on an Intel Skylake machine:

* clang (Release): 1.27x as fast
* clang (Debug): 1.06x as fast
* chrome (default): 1.05x as fast
* scylladb (default): 1.04x as fast

Speed-up with glibc malloc and --threads=16 on a ThunderX2 (AArch64):

* clang (Release): 1.31x as fast
* scylladb (default): 1.06x as fast

Reviewed By: andrewng

Differential Revision: https://reviews.llvm.org/D133003
2022-09-12 12:56:35 -07:00
Fangrui Song bd16ffb389 [ELF] Merge Symbol::needs* into uint16_t flags. NFC
Split off from D133003 ([ELF] Parallelize relocation scanning) to make its diff smaller.
2022-09-09 14:37:18 -07:00
Joseph Huber c1d19a8489 [ELF] Provide the GNU hash function in libObject
GNU uses a different hashing function compared to the sys-V standard
function already provided in libObject. This is already used internally
in LLD for generating synthetic sections. This patch simply extracts
this definition and makes it availible to other users of `libObject`.
This is done in preparation for supporting symbol name lookups via the
GNU hash table.

Reviewed By: MaskRay, jhenderson

Differential Revision: https://reviews.llvm.org/D132696
2022-09-05 11:04:57 -05:00
Fangrui Song 82ed93ea05 [ELF] Use stOther to track visibility
This simplifies SymbolTableSection<ELFT>::writeTo. Add dsoProtected to be used
in canDefineSymbolInExecutable and get the side benefit that the protected DSO
preemption diagnostic is clearer.
2022-09-04 17:27:35 -07:00
Fangrui Song 5e643cd7b7 [ELF] --gdb-index: error if constant pool size exceeds UINT32_MAX
If so, the last symbol's name_offset likely exceeds 0xffffffff and is not
supported by the format
(https://sourceware.org/gdb/onlinedocs/gdb/Index-Section-Format.html#Index-Section-Format).
I have seen an internal oversized executable with such a corrupted .gdb_index
2022-08-31 21:10:01 -07:00
Fangrui Song 7a20d6abe4 [ELF] Correctly compute .gdb_index size when symbol's name offset overflows
if `nameOff` overflows, `size` may be underestimated.
In writeTo, `memcpy(buf + sym.nameOff, sym.name.data(), sym.name.size());` may
cause an out-of-bounds write, leading to a SIGSEGV.
2022-08-31 21:04:26 -07:00
Kazu Hirata 8e494b85a5 Use llvm::drop_begin (NFC) 2022-08-20 21:18:30 -07:00
Fangrui Song e3fcf2e06f [ELF] Simplify llvm::enumerate with structured binding. NFC 2022-08-09 21:52:08 -07:00
Alex Brachet dbd04b853b [ELF] Support --package-metadata
This was recently introduced in GNU linkers and it makes sense for
ld.lld to have the same support. This implementation omits checking if
the input string is valid json to reduce size bloat.

Differential Revision: https://reviews.llvm.org/D131439
2022-08-08 21:31:58 +00:00
Fangrui Song ec04e45c03 [lld] LLVM_FALLTHROUGH => [[fallthrough]]. NFC
With C++17 there is no Clang pedantic warning or MSVC C5051.
2022-08-07 00:02:35 +00:00
Fangrui Song 28d05d6723 [ELF][PPC64] Fix potentially corrupted section content with empty .got
D91426 makes .got possibly empty while needed. If .got and .data have the same
address, and .got's content is written after .data, the first word of .data will
be corrupted.

The bug is not testable without D131247.
2022-08-05 15:22:57 -07:00
Fangrui Song b2559f2f5c [ELF] .eh_frame: remove config->wordsize padding
Linux Standard Base Core Specification says that CIE/FDE is padded to an
addressing unit size boundary, but in practice GNU assembler/LLVM integrated
assembler pad FDE/CIE to 4 and the last FDE to 8 on 64-bit systems.

In addition, GNU ld doesn't pad to 8, so let's drop excess padding, too.
If the assembler provides aligned pieces, the output will be aligned.

Noticed .eh_frame size reduction for 3 executables: 0.3% (chrome), 4.7% (clang),
7.6% (an internal program).
2022-07-31 23:35:44 -07:00
Fangrui Song 81ed005c4c [ELF] Remove EhFrameSection::addSection. NFC 2022-07-31 19:55:05 -07:00
Fangrui Song 3e9adff456 [ELF] Split EhInputSection::pieces into cies and fdes
This simplifies code, removes a read32 (for id==0 check), and makes it feasible
to combine some operations in EhInputSection::split and EhFrameSection::addRecords.

Mostly NFC, but fixes "Relocation not in any piece" assertion failure in an
erroneous case when a relocation offset precedes all CIE/FDE pices.
2022-07-31 16:16:10 -07:00
Fangrui Song c09d323599 [ELF] Move EhInputSection out of inputSections. NFC
inputSections temporarily contains EhInputSection objects mainly for
combineEhSections. Place EhInputSection objects into a new vector
ehInputSections instead of inputSections.
2022-07-31 11:58:08 -07:00
Fangrui Song 2e2d5304f0 [ELF] Move combineEhSections from Writer to SyntheticSections. NFC
This not only places the function in the right place, but also allows inlining addSection.
2022-07-29 00:47:30 -07:00
Fangrui Song 85cfd91723 [ELF] Optimize some non-constant alignTo with alignToPowerOf2. NFC
My x86-64 lld executable is 2KiB smaller. .eh_frame writing gets faster as there
were lots of divisions.
2022-07-24 11:20:49 -07:00
Kazu Hirata 1cc7f5bede Use static_assert instead of assert (NFC)
Identified with misc-static-assert.
2022-07-23 09:22:27 -07:00
Dmitri Gribenko aba43035bd Use llvm::sort instead of std::sort where possible
llvm::sort is beneficial even when we use the iterator-based overload,
since it can optionally shuffle the elements (to detect
non-determinism). However llvm::sort is not usable everywhere, for
example, in compiler-rt.

Reviewed By: nhaehnle

Differential Revision: https://reviews.llvm.org/D130406
2022-07-23 15:19:05 +02:00
Fangrui Song 9a572164d5 [ELF] Move InputFiles global variables (memoryBuffers, objectFiles, etc) into Ctx. NFC 2022-06-29 18:53:38 -07:00
Nico Weber 7effcbda49 Rename parallelForEachN to just parallelFor
Patch created by running:

  rg -l parallelForEachN | xargs sed -i '' -c 's/parallelForEachN/parallelFor/'

No behavior change.

Differential Revision: https://reviews.llvm.org/D128140
2022-06-19 17:49:00 -04:00
Martin Storsjö 9617ffce0d [LLD] [ELF] Add parentheses to silence a GCC warning. NFC.
This silences the following warning:

../tools/lld/ELF/SyntheticSections.cpp:1596:48: warning: suggest parentheses around ‘&&’ within ‘||’ [-Wparentheses]
 1596 |   assert((index != 0 || type != target->gotRel && type != target->pltRel ||

Differential Revision: https://reviews.llvm.org/D127395
2022-06-09 22:26:37 +03:00
Fangrui Song f4a3569d0a [ELF] Fix spurious GOT/PLT assertion failure when .dynsym is discarded
Linux kernel arch/arm64/kernel/vmlinux.lds.S discards .dynsym . D123985 triggers
a spurious assertion failure. Detect the case with
`!mainPart->dynSymTab->getParent()`.
2022-04-20 22:49:49 -07:00
Shoaib Meenai 610a0e8b53 [ELF] Assert on invalid GOT or PLT relocations
Because of https://llvm.org/PR50675, we can end up producing a PLT
relocation referencing a symbol that's dropped from the dynamic symbol
table, which in turn causes a crash at runtime. We ran into this again
recently, resulting in crashes for our users. A subsequent diff will fix
that issue, but add an assert to catch it if it happens again.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D123985
2022-04-20 10:46:04 -07:00
Matt Arsenault 63fe6d7eae lld/AMDGPU: Fix asserts if no object files are involved in link
Fixes issue 47690. The reproduction steps produced a shared object
from clang directly, and then fed the shared object back into
lld. With no regular object files, this assert was hit. I'm not sure
if we need to or should be looking for equivalent fields in shared
objects.
2022-04-08 14:18:52 -04:00
Mitch Phillips 786c89fed3 [ELF][MTE] Add --android-memtag-* options to synthesize ELF notes
This ELF note is aarch64 and Android-specific. It specifies to the
dynamic loader that specific work should be scheduled to enable MTE
protection of stack and heap regions.

Current synthesis of the ".note.android.memtag" ELF note is done in the
Android build system. We'd like to move that to the compiler. This patch
adds the --memtag-stack, --memtag-heap, and --memtag-mode={async, sync,
none} flags to the linker, which synthesises the note for us.

Future changes will add -fsanitize=memtag* flags to clang which will
pass these through to lld.

Depends on D119381.

Differential Revision: https://reviews.llvm.org/D119384
2022-04-04 11:17:36 -07:00
Fangrui Song bdb98bd979 [ELF] Use endianness-aware read32 to avoid dispatch. NFC 2022-03-15 23:51:11 -07:00
Fangrui Song 385573e07b [ELF] Inline ARMExidxSyntheticSection::classof. NFC
To optimize the only call site `dyn_cast<ARMExidxSyntheticSection>(first)` and
decrease code size.
2022-03-15 23:41:30 -07:00
Fangrui Song 4a8de2832a [ELF] Add -z pack-relative-relocs
GNU ld 2.38 added -z pack-relative-relocs which is similar to
--pack-dyn-relocs=relr but synthesizes the `GLIBC_ABI_DT_RELR` version
dependency if a shared object named `libc.so.*` has a `GLIBC_2.*` version
dependency.

This is used to implement the (as some glibc folks call) version lockout
mechanism. Add this option, because glibc does not want to support
--pack-dyn-relocs=relr which does not add `GLIBC_ABI_DT_RELR`.
See https://maskray.me/blog/2021-10-31-relative-relocations-and-relr for
detail.

Close https://github.com/llvm/llvm-project/issues/53775

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D120701
2022-03-10 19:54:21 -08:00
Fangrui Song 6c814931bc [ELF] Don't use multiple inheritance for OutputSection. NFC
Add an OutputDesc class inheriting from SectionCommand. An OutputDesc wraps an
OutputSection. This change allows InputSection::getParent to be inlined.

Differential Revision: https://reviews.llvm.org/D120650
2022-03-08 11:23:42 -08:00
Joao Moreira 9d7001eba9 [ELF][X86] Don't create IBT .plt if there is no PLT entry
https://github.com/ClangBuiltLinux/linux/issues/1606
When GNU_PROPERTY_X86_FEATURE_1_IBT is enabled, ld.lld will create .plt output
section even if there is no PLT entry. Fix this by implementing
IBTPltSection::isNeeded instead of using the default code path (which always
returns true).

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D120600
2022-02-26 03:55:40 +00:00
Fangrui Song ae1ba6194f [ELF] Replace uncompressed InputSectionBase::data() with rawData. NFC
In many call sites we know uncompression cannot happen (non-SHF_ALLOC, or the
data (even if compressed) must have been uncompressed by a previous pass).
Prefer rawData in these cases. data() increases code size and prevents
optimization on rawData.
2022-02-21 00:39:26 -08:00
Fangrui Song c12d49c4e2 [ELF] Remove .strtab deduplication
D118577: the 0.1~1.1% .strtab size reduction does not justify the 3~6%
link time increase. Just remove it even for -O2. release/14.x
has D118577 and the release note mentioned that this may be removed.

Fix https://github.com/ClangBuiltLinux/linux/issues/1578
caused by D118577 (empty string not in stringMap).
2022-02-18 14:54:10 -08:00
Fangrui Song ce45c95694 [ELF] Remove obscure -dp and GNU ld incompatible --[no-]define-common, ignore -d/-dc
https://maskray.me/blog/2022-02-06-all-about-common-symbols#no-define-common

In GNU ld, -dc only affects -r links and causes COMMON symbols to be allocated.
--no-define-common is defined to make COMMON symbols undefined for -shared.
AIUI --no-define-common is a workaround around glibc 2.1 time and not really useful.

gold confuses --define-common with -d/FORCE_COMMON_ALLOCATION and implements
--define-common with -d semantics. Its --no-define-common is incompatible with
GNU ld.

In ld.lld, b2a23cf3c0 fixed the default -r
behavior for COMMON symbols but ported the incompatible gold
--[no-]define-common. To the best of my knowledge, no project uses -dp
--[no-]define-common. So just remove these options.

-d/-dc are used by the following projects:

* grub grub-core/genmod.sh.in uses -Wl,-r,-d (https://lists.gnu.org/archive/html/grub-devel/2022-02/msg00088.html)
* FreeBSD crunchgen uses -Wl,-dc (https://reviews.freebsd.org/D34215)

A no-op implementation works for them. Only when a program inspects relocatable
output by itself and does not recognize COMMON symbols, there may be a problem.
This is an extremely unlikely case.

Reviewed By: peter.smith

Differential Revision: https://reviews.llvm.org/D119108
2022-02-09 10:35:53 -08:00