Commit Graph

549 Commits

Author SHA1 Message Date
Zarko Todorovski 7f7dac7126 [NFC][llvm] Inclusive language: reword uses of sanity test and check
Part of continuing work to use more inclusive language. Reworded uses
of sanity check and sanity test in llvm/test/
2021-11-25 07:21:42 -05:00
Keith Smiley 68311f21eb [llvm-objcopy][MachO] Add llvm-strip support for newer load commands
Previously llvm-strip would fail because of unknown commands.

Fixes https://bugs.llvm.org/show_bug.cgi?id=50044

Differential Revision: https://reviews.llvm.org/D113734
2021-11-17 10:36:35 -08:00
Leonard Chan b75cc51df7 Limit test to x86 for now. 2021-11-16 14:46:02 -08:00
Leonard Chan 25bcd94234 [llvm-objcopy] Add --update-section
This is another attempt at D59351 which attempted to add --update-section, but
with some heuristics for adjusting segment/section offsets/sizes in the event
the data copied into the section is larger than the original size of the section.
We are opting to not support this case. GNU's objcopy was able to do this because
the linker and objcopy are tightly coupled enough that segment reformatting was
simpler. This is not the case with llvm-objcopy and lld where they like to be separated.

This will attempt to copy data into the section without changing any other
properties of the parent segment (if the section is part of one).

Differential Revision: https://reviews.llvm.org/D112116
2021-11-16 14:10:40 -08:00
Keith Smiley 47bb456b2f [llvm-obcopy][MachO] Add error for MH_PRELOAD
Previously this would crash. Fixes https://bugs.llvm.org/show_bug.cgi?id=51877

Differential Revision: https://reviews.llvm.org/D113819
2021-11-12 19:18:34 -08:00
Nico Weber 3c0cf7e1a9 Unbreak code_signature_lc.test on macOS after 911be05743 2021-10-26 21:05:48 -04:00
Daniel Rodríguez Troitiño 911be05743 [test][objcopy] Replace GNU sed extension with BSD compatible syntax.
GNU sed offers the `,+4d` to delete the line a next four lines, but BSD
sed doesn't seem to support it (at least in macOS 10.15, but seems to do
in my 11.6 version).

Replace the usage of the extension with the equivalent syntax that works
both in BSD and GNU sed. I don't have a macOS 10.15 to check, but this
works in both my macOS 11.6 and Linux machines.

Differential Revision: https://reviews.llvm.org/D112583
2021-10-26 17:35:56 -07:00
Nuri Amari a299b24712 Regenerate LC_CODE_SIGNATURE during llvm-objcopy operations
**Context:**

This is a second attempt at introducing signature regeneration to llvm-objcopy. In this diff: https://reviews.llvm.org/D109840, a script was introduced to test
the validity of a code signature. In this diff: https://reviews.llvm.org/D109803 (now reverted), an effort was made to extract the signature generation behavior out of LLD into a common location for use in llvm-objcopy. In this diff: https://reviews.llvm.org/D109972 it was decided that there was no appropriate common location and that a small amount of duplication to bring signature generation to llvm-objcopy would be better. This diff introduces this duplication.

**Summary**

Prior to this change, if a LC_CODE_SIGNATURE load command
was included in the binary passed to llvm-objcopy, the command and
associated section were simply copied and included verbatim in the
new binary. If rest of the binary was modified at all, this results
in an invalid Mach-O file. This change regenerates the signature
rather than copying it.

The code_signature_lc.test test was modified to include the yaml
representation of a small signed MachO executable in order to
effectively test the signature generation.

Reviewed By: alexander-shaposhnikov, #lld-macho

Differential Revision: https://reviews.llvm.org/D111164
2021-10-26 14:51:13 -07:00
Igor Kudrin 7b424b9333 [llvm-objcopy] Rename relocation sections together with their targets.
As for now, llvm-objcopy renames only sections that are specified
explicitly in --rename-section, while GNU objcopy keeps names of
relocation sections in sync with their targets. For example:

> readelf -S test.o
...
  [ 1] .foo      PROGBITS
  [ 2] .rela.foo RELA

> objcopy --rename-section .foo=.bar test.o gnu.o
> readelf -S gnu.o
...
  [ 1] .bar      PROGBITS
  [ 2] .rela.bar RELA

> llvm-objcopy --rename-section .foo=.bar test.o llvm.o
> readelf -S llvm.o
...
  [ 1] .bar      PROGBITS
  [ 2] .rela.foo RELA

This patch makes llvm-objcopy to match the behavior of GNU objcopy better.

Differential Revision: https://reviews.llvm.org/D110352
2021-09-29 16:36:37 +07:00
Sam Clegg e4b2f3054a [WebAssembly][libObject] Avoid re-use of Section object during parsing
The re-use of this struct across iterations of the loop was causing
fields (specifically Name) to be incorrectly shared between multiple
sections.

Differential Revision: https://reviews.llvm.org/D108984
2021-09-10 09:30:50 -04:00
Alfonso Sánchez-Beato b25ab4f313 [llvm-objcopy][COFF] Fix test for debug dir presence
If the number of directories was 6 (equal to the DEBUG_DIRECTORY
index), patchDebugDirectory() was run even though the debug directory
is actually the 7th entry. Use <= in the comparison to fix that.

This fixes https://llvm.org/PR51243

Differential Revision: https://reviews.llvm.org/D106940

Reviewed by: jhenderson
2021-09-10 09:57:18 +01:00
gbreynoo e28cd75a50 [OptTable] Reapply Improve error message output for grouped short options
This reapplies 71d7fed3bc which was
reverted by 3e2bd82f02. This change
includes the fix for breaking the sanitizer bots.

As seen in https://bugs.llvm.org/show_bug.cgi?id=48880 the current
implementation for parsing grouped short options can return unclear
error messages. This change fixes the example given in the ticket in
which a flag is incorrectly given an argument. Also when parsing a
group we now keep reading past the first incorrect option and output
errors for all incorrect options in the group.

Differential Revision: https://reviews.llvm.org/D108770
2021-09-03 11:13:52 +01:00
Kevin Athey 3e2bd82f02 Revert "[OptTable] Improve error message output for grouped short options"
This reverts commit 71d7fed3bc.

Reason: broke sanitizer bots
more info: https://reviews.llvm.org/D108770
2021-08-31 14:06:11 -07:00
gbreynoo 71d7fed3bc [OptTable] Improve error message output for grouped short options
As seen in https://bugs.llvm.org/show_bug.cgi?id=48880 the current
implementation for parsing grouped short options can return unclear
error messages. This change fixes the example given in the ticket in
which a flag is incorrectly given an argument. Also when parsing a
group we now keep reading past the first incorrect option and output
errors for all incorrect options in the group.

Differential Revision: https://reviews.llvm.org/D108770
2021-08-31 16:41:08 +01:00
Alfonso Sánchez-Beato cdd407286a [llvm-objcopy] [COFF] Consider section flags when adding section
The --set-section-flags option was being ignored when adding a new
section. Take it into account if present.

Fixes https://llvm.org/PR51244

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D106942
2021-08-25 23:11:41 +03:00
Igor Kudrin 68616584c3 [llvm-objcopy][ELF] Avoid reordering section headers
As for now, llvm-objcopy sorts section headers according to the offsets
of the sections in the input file. That can corrupt section references
in the dynamic symbol table because it is a loadable section and as such
is not updated by the tool. Even though the section references are not
required for loading the binary correctly, they are still handy for a
user who analyzes the file.

While the patch removes global reordering of section headers, it layouts
the sections in the same way as before, i.e. according to their original
offsets. All that helps the output file to resemble the input better.

Note that the patch removes sorting SHT_GROUP sections to the start of
the list, which was introduced in D62620 in order to ensure that they
come before the group members, along with the corresponding test. The
original issue was caused by the sorting of section headers, so dropping
the sorting also resolves the issue.

Differential Revision: https://reviews.llvm.org/D107653
2021-08-12 17:12:09 +07:00
Pirama Arumuga Nainar 16ebb7ab5c [llvm-objcopy] [COFF] Do not patch debug entries if PointerToRawData is zero
Fix an edge case missed by https://reviews.llvm.org/D78921.  For e.g.,
the Repro debug entry (generated with the /Brepro linker flag) does not
have a debug-directory payload.  Do not attempt to patch Debug entries
without a payload.

Differential Revision: https://reviews.llvm.org/D107324
2021-08-06 09:23:25 -07:00
Daniel Rodríguez Troitiño d6704e5ed9 [llvm-objcopy][MachO] Ignore all LC_SUB_* commands.
The LC_SUB_FRAMEWORK, LC_SUB_UMBRELLA, LC_SUB_CLIENT, and LC_SUB_LIBRARY
are used to indicate related libraries, binaries or framework names.
Their only payload is the string with the name of the object. Adding
those commands to the list of ignored/skipped load commands will avoid
an error that stop the process of copying/stripping and will copy their
contents verbatim.

Additionally, in order to have a test for this case, `yaml2obj` now
allows those four commands to contain a `Content`.

Differential Revision: https://reviews.llvm.org/D106412
2021-07-28 17:35:26 -07:00
Fangrui Song 792c206e2b [llvm-objcopy] Drop GRP_COMDAT if the group signature is localized
See [GRP_COMDAT group with STB_LOCAL signature](https://groups.google.com/g/generic-abi/c/2X6mR-s2zoc)
objcopy PR: https://sourceware.org/bugzilla/show_bug.cgi?id=27931

GRP_COMDAT deduplication is purely based on the signature symbol name in
ld.lld/GNU ld/gold. The local/global status is not part of the equation.

If the signature symbol is localized by --localize-hidden or
--keep-global-symbol, the intention is likely to make the group fully
localized. Drop GRP_COMDAT to suppress deduplication.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D106782
2021-07-26 09:05:18 -07:00
Fangrui Song c0da287c30 [yaml2obj][MachO] Rename PayloadString to Content
The new name is conciser and matches yaml2obj ELF & DWARF.

Reviewed By: #lld-macho, thakis

Differential Revision: https://reviews.llvm.org/D106759
2021-07-26 09:04:51 -07:00
Daniel Rodríguez Troitiño 3c4023b225 [test] Use double pound to denote comments.
Use double pound at the start of the line to differentiate comments from
statements for Lit or FileCheck.

I will also use this small commit to check my commit access.

Differential Revision: https://reviews.llvm.org/D106103
2021-07-15 17:39:34 -07:00
Fangrui Song aa3df8ddcd [test] Avoid llvm-readelf/llvm-readobj one-dash long options and deprecated aliases (e.g. --file-headers) 2021-07-15 10:26:21 -07:00
Derek Schuff 7cb25f5387 [llvm-strip][WebAssembly] Support strip flags
Summary:
Add support for the basic section stripping (and keeping) flags for wasm:
strip with no flags, --strip-all, --strip-debug,
--only-section, --keep-section, and --only-keep-debug.

Factor section removal into a function and use a predicate chain like
the ELF implementation.

Reviewers: jhenderson, sbc100

Differential Revision: https://reviews.llvm.org/D73820
2021-07-14 14:17:02 -07:00
Daniel Rodríguez Troitiño b77533fb70 [llvm-strip] Support grouped options in llvm-strip
GNU and Apple `strip` implementations seems to support grouped options.
Enable the support for grouped options introduced in
https://reviews.llvm.org/D83639 for `llvm-strip` invocations.

Includes test that checks that both the grouped and non grouped
invocations produces the same result.

Reviewed By: alexander-shaposhnikov, MaskRay

Differential Revision: https://reviews.llvm.org/D105249
2021-07-01 13:36:45 -07:00
Fangrui Song 814dffa4b7 [llvm-objcopy][MachO] Support LC_LINKER_OPTIMIZATION_HINT load command
The load command is currently specific to arm64 and holds information
for instruction rewriting, e.g.  converting a GOT load to an ADR to
compute a local address.
(On ELF the information is usually conveyed by relocations, e.g.
R_X86_64_REX_GOTPCRELX, R_PPC64_TOC16_HA)

Reviewed By: alexander-shaposhnikov

Differential Revision: https://reviews.llvm.org/D104968
2021-06-29 18:47:55 -07:00
Fangrui Song 69937a8080 [llvm-objcopy][MachO] Support ARM64_RELOC_ADDEND
An ARM64_RELOC_ADDEND relocation reuses the symbol field for the addend value.
We should pass through such relocations.

Reviewed By: alexander-shaposhnikov

Differential Revision: https://reviews.llvm.org/D104967
2021-06-29 11:23:30 -07:00
Fangrui Song d619cf5ac5 [llvm-objcopy][MachO] Copy LC_LINKER_OPTIMIZATION_HINT
This fixes `error: unsupported load command (cmd=0x2e)`
2021-06-16 12:09:50 -07:00
Fangrui Song 1de18ad8d7 [llvm-objcopy] Make ihex writer similar to binary writer
There is no need to differentiate whether `UseSegments` is true or
false. Unifying the cases makes the behavior closer to BinaryWriter.

This improves compatibility with objcopy because SHF_ALLOC sections not in
a PT_LOAD will not be skipped. Such cases are usually erroneous input, though.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D104186
2021-06-16 10:08:20 -07:00
Ian McIntyre 5899278758 [llvm-objcopy] Exclude empty sections in IHexWriter output
IHexWriter was evaluating a section's physical address when deciding if
that section should be written to an output. This approach does not
account for a zero-sized section that has the same physical address as a
sized section. The behavior varies from GNU objcopy, and may result in a
HEX file that does not include all program sections.

The IHexWriter now excludes zero-sized sections when deciding what
should be written to the output. This affects the contents of the
writer's `Sections` collection; we will not try to insert multiple
sections that could have the same physical address. The behavior seems
consistent with GNU objcopy, which always excludes empty sections,
no matter the address.

The new test case evaluates the IHexWriter behavior when provided a
variety of empty sections that overlap or append a filled section. See
the input file's comments for more information. Given that test input,
and the change to the IHexWriter, GNU objcopy and llvm-objcopy produce
the same output.

Reviewed By: jhenderson, MaskRay, evgeny777

Differential Revision: https://reviews.llvm.org/D101332
2021-06-12 12:23:07 -07:00
Alexander Shaposhnikov 0276cc742b [llvm-objcopy][MachO] Do not strip symbols with the flag REFERENCED_DYNAMICALLY set
Do not strip symbols having the flag REFERENCED_DYNAMICALLY set.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D104092
2021-06-11 16:34:59 -07:00
Sergey Dmitriev 1fb5278882 [llvm-strip] Add support for '--' for delimiting options from input files
This will allow to use llvm-strip with file names that begin with dashes.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D102825
2021-05-20 03:33:51 -07:00
Sergey Dmitriev 0b12963b74 [llvm-objcopy] Update LIT test to resolve bot failure [NFC]
Reviewed By: hubert.reinterpretcast

Differential Revision: https://reviews.llvm.org/D102823
2021-05-19 19:56:35 -07:00
Sergey Dmitriev f24f140290 [llvm-objcopy] Add support for '--' for delimiting options from input/output files
This will allow to use llvm-objcopy with file names that begin with dashes.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D102665
2021-05-19 01:56:46 -07:00
Alex Orlov d8e65585f7 Fixed llvm-objcopy to add correct symbol table for ELF with program headers.
This fixes the following bugs:
https://bugs.llvm.org/show_bug.cgi?id=43935

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D102258
2021-05-12 12:39:30 +04:00
Fangrui Song b3336bfa2e [llvm-objcopy][ELF] --only-keep-debug: set offset/size of segments with no sections to zero
PR50160: we currently ignore non-PT_PHDR segments with no sections, not
accounting for its p_offset and p_filesz: this can cause an out-of-bounds write
in `writeSegmentData` if the p_offset+p_filesz is larger than the total file
size.

This can be fixed by setting p_offset=p_filesz=0. The logic nicely unifies with
the logic added in D90897.

Reviewed By: jhenderson, rupprecht

Differential Revision: https://reviews.llvm.org/D101560
2021-05-05 10:26:57 -07:00
Fangrui Song 96f3a63076 [llvm-objcopy] --dump-section: error if '=' is missing or filename is empty
Fix PR45416: the diagnostic when '=' is missing is misleading.
`FileOutputBuffer::create` returns successfully when the filename is empty
(the temporary file is `.tmp%%%%%%%`), but `FileOutputBuffer::commit` will error when
renaming `.tmp%%%%%%%` to the empty name).

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D101697
2021-05-04 17:30:57 -07:00
Jez Ng c00fc180ec [llvm-readobj] Recognize N_THUMB_DEF as a symbol flag
The right symbol flag mask is ~0x7, not ~0xf.

Also emit string names for the other flags (we were missing some).

Reviewed By: #lld-macho, gkm

Differential Revision: https://reviews.llvm.org/D101548
2021-04-30 17:39:56 -04:00
Alexander Shaposhnikov 86f291ebb2 [llvm-objcopy][MachO] Add support for LC_THREAD/LC_UNIXTHREAD
Add support for LC_THREAD/LC_UNIXTHREAD
(these load commands can be copied over without any modifications).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D101384
2021-04-28 16:29:33 -07:00
Alexander Shaposhnikov 412437aec0 Revert "[llvm-objcopy][MachO] Add support for LC_THREAD/LC_UNIXTHREAD"
This reverts commit 4dfddf715b
since it breaks some build bots (e.g. clang-ppc64be-linux)
2021-04-27 16:19:59 -07:00
Alexander Shaposhnikov 4dfddf715b [llvm-objcopy][MachO] Add support for LC_THREAD/LC_UNIXTHREAD
Add support for LC_THREAD/LC_UNIXTHREAD
(these load commands can be copied over without any modifications).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D101384
2021-04-27 15:54:51 -07:00
Alexander Shaposhnikov ede56e5127 [llvm-objcopy][MachO] Add support for --keep-undefined
This diff introduces --keep-undefined in llvm-objcopy/llvm-strip for Mach-O
which makes the tools preserve undefined symbols.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D97040
2021-03-08 18:57:25 -08:00
James Henderson 076698154a [llvm-objcopy] Fix crash for binary input files with non-ascii names
The code was using the standard isalnum function which doesn't handle
values outside the non-ascii range. Switching to using llvm::isAlnum
instead ensures we don't provoke undefined behaviour, which can in some
cases result in crashes.

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D97663
2021-03-05 08:57:40 +00:00
James Henderson 47c343d768 [llvm-objcopy][test] Fix test that could have passed spuriously
The test was showing that when --strip-unneeded is specified for an
executable, all the symbols are stripped. However, the set of symbols
used in the test would be stripped by --strip-unneeded for an ET_REL
object too. Fix this by adding additional symbols that aren't normally
stripped by --strip-unneeded.

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D97664
2021-03-05 08:57:39 +00:00
James Henderson 1562e4552c [llvm-objcopy][llvm-strip][test] Improve testing
This patch adds a number of new test cases that cover various
llvm-objcopy and llvm-strip features that had missing test coverage of
various descriptions:
* --add-section - checked the shdr properties, not just the content.
* Dedicated test case for --add-symbol when there are many sections.
* Show that --change-start accepts negative values without overflow.
  This was previously present but got lost between review versions.
* --dump-section - show that multiple sections can be dumped
  simultaneously to different files, and that an error is reported when
  a section cannot be found.
* --globalize-symbol(s) - show that symbols that are not mentioned are
  not globalized, if they would otherwise be, and that missing symbols
  from the list do not cause problems.
* --keep-global-symbol - show that the --regex option can be used in
  conjunction with this option.
* --keep-symbol - show that the --regex option can be used in
  conjunction with this option.
* --localize-symbol(s) - show that symbols that are not mentioned are
  not localized, if they would otherwise be, and that missing symbols
  from the list do not cause problems.
* --prefix-alloc-sections - show the behaviour of an empty string
  argument and multiple arguments.
* --prefix-symbols - show the behaviour of an empty string argument and
  multiple arguments. Also show the option applies to undefined symbols.
* --redefine-symbol - show that symbols with no name can be renamed,
  that it is not an error if a symbol is not specified, and that the
  option doesn't chain (i.e. --redefine-sym a=b --redefine-sym b=c does
  not redefine a as c).
* --rename-section - show that all section flags are preserved if none
  are specified. Also show that the option does not chain.
* --set-section-alignment - show that only specified sections have
  their alignments changed.
* --set-section-flags - show which section flags are preserved when this
  option is used. Also show that unspecified sections are not affected.
* --preserve-dates - show that -p is an alias of --preserve-dates.
* --strip-symbol - show that --regex works with this option for
  llvm-objcopy as well as llvm-strip.
* --strip-unneeded-symbol(s) - show more clearly that needed symbols are
  not stripped even if requested by this option.
* --allow-broken-links - show the sh_link of a symbol table is set to 0
  when its string table has been removed when this option is specified.
* --weaken-symbol(s) - show that symbols that are not mentioned are not
  weakened, if they would otherwise be, and that missing symbols from
  the list do not cause problems.
* --wildcard - show the wildcard behaviour for several options that were
  previously unchecked.

Reviewed by: alexshap

Differential Revision: https://reviews.llvm.org/D97666
2021-03-04 11:32:27 +00:00
James Henderson 8bb74d16ef [llvm-objcopy/strip] Fix off-by-one error in SYMTAB_SHNDX need check
The check for whether an extended symbol index table was required
dropped the first SHN_LORESERVE sections from the sections array before
checking whether the remaining sections had symbols. Unfortunately, the
null section header is not present in this list, so the check was
skipping the first section that might be important. If that section
contained a symbol, and no subsequent ones did, the .symtab_shndx
section would not be emitted, leading to a corrupt object.

Also consolidate and expand test coverage in the area to cover this bug
and other aspects of the SYMTAB_SHNDX section.

Reviewed by: alexshap, MaskRay

Differential Revision: https://reviews.llvm.org/D97661
2021-03-04 10:23:45 +00:00
James Henderson 49c91a64fd [llvm-objcopy][test] Improve many-sections object and test case
Additionally do some test tidy-ups and improve coverage of symbol
section indexes where the logical section index >= SHN_LORESERVE.

The symbol and section names in the many-section input object were
mostly shared. This patch changes them to be distinct, enabling
different operations such as --add-symbol, to be more targeted, when
using the object. It also makes the test less confusing and removes some
oddness in the symbol table order, presumably caused by the duplicate
names.

The input object was built from assembly that was of the form:
.section s1
sym1:
.section s2
sym2:
...
with a total of 65536 such occurrences. llvm-objcopy was then used to
remove the empty .text section automatically generated by MC, and
incidentally to move .strtab to the end of the object. This ensured that
the section/symbol indexes matched their name (i.e. section index 1 was
s1, section index 2 was s2 etc, and sym1 was in s1, sym2 in s2 etc).

Reviewed by: MaskRay

Differential Revision: https://reviews.llvm.org/D97660
2021-03-04 09:42:43 +00:00
Fangrui Song 17b4e695ce [llvm-objcopy] If input=output, preserve umask bits, otherwise drop S_ISUID/S_ISGID bits
This makes the behavior similar to cp

```
chmod u+s,g+s,o+x a
sudo llvm-strip a -o b
// With this patch, b drops set-user-ID and set-group-ID bits.
// sudo cp a b => b does not have set-user-ID or set-group-ID bits.
```

This also changes the behavior for the following case:

```
chmod u+s,g+s,o+x a
llvm-strip a
// a preserves set-user-ID and set-group-ID bits.
// This matches binutils<2.36 and probably >=2.37.  2.36 and 2.36.1 have some compatibility issues.
```

Differential Revision: https://reviews.llvm.org/D97253
2021-02-24 11:10:09 -08:00
Fangrui Song c465429f28 [llvm-objcopy] Delete --build-id-link-{dir,input,output}
The few options are niche. They solved a problem which was traditionally solved
with more shell commands (`llvm-readelf -n` fetches the Build ID. Then
`ln` is used to hard link the file to a directory derived from the Build ID.)

Due to limitation, they are no longer used by Fuchsia and they don't appear to
be used elsewhere (checked with Google Search and Debian Code Search). So delete
them without a transition period.

Announcement: https://lists.llvm.org/pipermail/llvm-dev/2021-February/148446.html

Differential Revision: https://reviews.llvm.org/D96310
2021-02-15 11:17:32 -08:00
Fangrui Song 3e837e1735 [llvm-objcopy][test] Stablize build-id-link-dir.test 2021-02-08 17:22:22 -08:00
Patrick Oppenlander 93345e825a [llvm-objcopy] -O binary: consider SHT_NOBITS sections to be empty
This is consistent with BFD objcopy.

Previously llvm objcopy would allocate space for SHT_NOBITS sections
often resulting in enormous binary files.

New test case (binary-paddr.test %t6).

Reviewed By: jhenderson, MaskRay

Differential Revision: https://reviews.llvm.org/D95569
2021-02-01 15:01:25 -08:00
Abhina Sreeskantharajan 42a21778f6 [test] Use host platform specific error message substitution in lit tests
On z/OS, the following error message is not matched correctly in lit tests.

```
EDC5129I No such file or directory.
```

This patch uses a lit config substitution to check for platform specific error messages.

Reviewed By: muiez, jhenderson

Differential Revision: https://reviews.llvm.org/D95246
2021-01-29 07:16:30 -05:00
Georgii Rymar d5e48f1347 [yaml2obj][obj2yaml] - Improve how we set/dump the sh_entsize field.
We already set the `sh_entsize` field in a single place
for all non-implicit sections.

This patch reorders the logic slightly and with it
we finally have the only one place where the `sh_entsize` is set.

obj2yaml will not dump the `EntSize` key for `SHT_DYNSYM/SHT_SYMTAB` sections anymore,
when the value of `sh_entsize` is equal to `sizeof(Elf_Sym)`

Note that this also seems revealed an issue in llvm-objcopy:
Previously yaml2obj set the `sh_entsize` for the `.symtab` section to 0x18,
now we it sets it for `SHT_SYMTAB` sections, i.e. by type.
But the `llvm-objcopy/ELF/only-keep-debug.test` has a `.symtab` section of type `SHT_STRTAB`,
and now yaml2obj sets the `sh_entsize` to 0 for it.
I had to update the corresponding check lines for `ES`, but the behavior of
`llvm-objcopy` should be fixed instead I think.
I've added a TODO and a comment.

Differential revision: https://reviews.llvm.org/D95364
2021-01-26 13:33:02 +03:00
Abhina Sreeskantharajan 978444d531 Revert "[SystemZ][z/OS] Fix No such file or directory expression error"
This reverts commit 06f8a49693.
2021-01-25 08:29:38 -05:00
Abhina Sreeskantharajan 84851a274e Revert "[SystemZ][z/OS] Fix No such file or directory expression error matching in lit tests - continued"
This reverts commit 520b5ecf85.
2021-01-25 08:29:38 -05:00
Abhina Sreeskantharajan 520b5ecf85 [SystemZ][z/OS] Fix No such file or directory expression error matching in lit tests - continued
This is a continuation of https://reviews.llvm.org/D94239. I missed some other spellings of the same error.

Reviewed By: muiez

Differential Revision: https://reviews.llvm.org/D95246
2021-01-22 13:54:25 -05:00
Abhina Sreeskantharajan 689aaba7ac [SystemZ][z/OS] Fix No such file or directory expression error matching in lit tests
On z/OS, the following error message is not matched correctly in lit tests. This patch updates the CHECK expression to match successfully.
```
EDC5129I No such file or directory.
```

Reviewed By: muiez

Differential Revision: https://reviews.llvm.org/D94239
2021-01-18 07:14:37 -05:00
Brandon Bergren 696bd3073f [PowerPC] Support powerpcle target in LLVMObject [2/5]
Add object file handling for powerpcle-*-*.

Adjust tests.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D93916
2021-01-02 12:17:39 -06:00
Georgii Rymar 98a4289810 [llvm-readobj] - For SHT_REL relocations, don't display an addend.
This is https://bugs.llvm.org/show_bug.cgi?id=44257.

In LLVM style we always print `0` as addend when dumping
SHT_REL relocations. It is confusing, this patch stops
printing it as the first comment on the bug page suggests.

Differential revision: https://reviews.llvm.org/D93033
2020-12-14 12:03:00 +03:00
Fangrui Song 20de182246 [llvm-objcopy] --only-keep-debug: place zero-size segment according to its parent segment
Alternative to D74755. sectionWithinSegment() treats an empty section as having
a size of 1. Due to the rule, an empty .tdata will not be attributed to an
empty PT_TLS. (The empty p_align=64 PT_TLS is for Android Bionic's TCB
compatibility (ELF-TLS). See https://reviews.llvm.org/D62055#1507426)

Currently --only-keep-debug will not layout a segment with no section
(layoutSegmentsForOnlyKeepDebug()), thus p_offset of PT_TLS can go past the end
of the file. The strange p_offset can trigger validation errors for subsequent
tools, e.g. llvm-objcopy errors when reading back the separate debug file
(readProgramHeaders()).

This patch places such an empty segment according to its parent segment.  This
special cases works for the empty PT_TLS used in Android. For a non-empty
segment, it should have at least one non-empty section and will be handled by
the normal code. Note, p_memsz PT_LOAD is rejected by both Linux and FreeBSD.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D90897
2020-11-11 09:21:10 -08:00
Georgii Rymar a7a447be0f [yaml2obj] - ProgramHeaders: introduce FirstSec/LastSec instead of Sections list.
Imagine we have a YAML declaration of few sections: `foo1`, `<unnamed 2>`, `foo3`, `foo4`.

To put them into segment we can do (1*):

```
Sections:
 - Section: foo1
 - Section: foo4
```

or we can use (2*):

```
Sections:
 - Section: foo1
 - Section: foo3
 - Section: foo4
```

or (3*) :

```
Sections:
 - Section: foo1
## "(index 2)" here is a name that we automatically created for a unnamed section.
 - Section: (index 2)
 - Section: foo3
 - Section: foo4
```

It looks really confusing that we don't have to list all of sections.

At first I've tried to make this rule stricter and report an error when there is a gap
(i.e. when a section is included into segment, but not listed explicitly).
This did not work perfect, because such approach conflicts with unnamed sections/fills (see (3*)).

This patch drops "Sections" key and introduces 2 keys instead: `FirstSec` and `LastSec`.
Both are optional.

Differential revision: https://reviews.llvm.org/D90458
2020-11-09 13:00:50 +03:00
Cornelius Aschermann 4d54c6fc5a [llvm-objcopy][MachO] Fix adding multiple sections
This diff fixes missing fields initialization (Size, VMSize).
Previously this resulted in broken binaries when multiple sections
were added in one tool's invocatation.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90690
2020-11-07 18:16:06 -08:00
Alexander Shaposhnikov f147f59cd3 [llvm-objcopy][MachO] Skip sections with zero offset
Some binaries can contain regular sections with zero offset and zero size.
This diff makes llvm-objcopy's handling of such sections consistent with
cctools's strip (which doesn't modify them),
previously the tool would allocate file space for them.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90796
2020-11-06 13:29:43 -08:00
Fangrui Song ee142c4988 [llvm-objcopy] Make --set-section-flags work with --add-section
This matches behavior GNU objcopy and can simplify clang-offload-bundler
(which currently works around the issue by invoking llvm-objcopy twice).

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D90438
2020-11-04 09:39:14 -08:00
Keith Smiley 51f8d46491 [llvm-install-name-tool] Quote passed rpath args in error messages
This diff refactors error reporting to make it more clear
what arguments were passed to llvm-install-name-tool.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D90080
2020-10-28 15:22:33 -07:00
Keith Smiley 77cbf25953 [llvm-install-name-tool] Add -prepend_rpath option
This diff adds the option -prepend_rpath which inserts an rpath as
the first rpath in the binary.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D89605
2020-10-23 15:01:03 -07:00
Alexander Shaposhnikov 27e11d7120 [MC] Adjust StringTableBuilder for linked Mach-O binaries
LD64 emits string tables which start with a space and a zero byte.
This diff adjusts StringTableBuilder for linked Mach-O binaries to match LD64's behavior.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D89561
2020-10-22 19:19:41 -07:00
Alexander Shaposhnikov 861429160c [llvm-objcopy][MachO] Fix the calculation of the output size
Virtual sections do not contribute to the final output size.
This diff fixes the corresponding calculations in the method MachOWriter::totalSize.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D89661
2020-10-22 13:58:11 -07:00
Tobias Hieta 61133e0b11 [llvm-install-name-tool] Add -delete_all_rpaths option
This diff adds an option to remove all rpaths from a Mach-O binary.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D88674
2020-10-13 00:45:57 -07:00
Jez Ng 84cc39c329 [llvm-readobj] Don't print out section names for STABS symbols
This diff is similar to what D71394 did for `llvm-objdump` -- it avoids
trying to look up a section name for STABS symbols, since some STABS
symbol types (like `N_OSO`) use the `n_sect` field to store other data
instead of a section index.

Differential Revision: https://reviews.llvm.org/D88468
2020-10-12 18:55:40 -07:00
Alexander Shaposhnikov 315970de1d [llvm-objcopy][MachO] Add support for universal binaries
This diff adds support for universal binaries to llvm-objcopy.
This is a recommit of 32c8435ef7 with the asan issue fixed.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D88400
2020-10-06 04:01:40 -07:00
Dmitri Gribenko 80f66ac0d5 Revert "[llvm-objcopy][MachO] Add support for universal binaries"
This reverts commit 32c8435ef7. It fails
ASan, details in https://reviews.llvm.org/D88400.
2020-10-06 11:29:24 +02:00
Alexander Shaposhnikov 32c8435ef7 [llvm-objcopy][MachO] Add support for universal binaries
This diff adds support for universal binaries to llvm-objcopy.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D88400
2020-10-05 10:44:28 -07:00
Alexander Shaposhnikov e60a760b7d [llvm-objcopy][MachO] Fix --add-section
This diff fixes --add-section functionality and simplifies the tests organization.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87497
2020-09-24 01:51:10 -07:00
Georgii Rymar d4035af253 [llvm-readelf/obj] - Print section symbol names properly when dumping relocations.
Currently `--relocations` ignores section symbol names and always prints
section names for them. This is inconsistent with GNU readelf and with `--symbols`.

We have a code in `getFullSymbolName` (which is used for `--symbols`) which can be
reused for `getRelocationTarget` (used for `--relocations`).
With that the issue described is fixed and code becomes a bit shorter.
Also with this change we start to print more relocations (in situations when we just
showed warnings instead before) and also start to report more diagnostic warnings
(see reloc-zero-name-or-value.test).

Differential revision: https://reviews.llvm.org/D87613
2020-09-23 13:25:39 +03:00
Alexander Shaposhnikov 5495b69164 [llvm-objcopy][MachO] Add llvm-bitcode-strip driver
This diff adds llvm-bitcode-strip driver to llvm-objcopy.
In the future this will enable us to build a replacement for the tool bitcode_strip.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87212
2020-09-18 18:13:05 -07:00
Xing GUO 2d35092cd2 [DWARFYAML] Make the include_directories, file_names and opcodes fields of the line table optional.
This patch makes the include_directories, file_names and opcodes fields
of the line table optional. This helps us simplify some tests.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D87878
2020-09-18 20:21:11 +08:00
Alexander Shaposhnikov b897729a39 [llvm-install-name-tool] Add -V flag
This diff adds -V alias for --version to make llvm-install-name-tool
consistent with other tools (llvm-objcopy, llvm-strip, etc).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87264
2020-09-09 17:11:08 -07:00
Alexander Shaposhnikov ce49b7d9ca [llvm-install-name-tool] Add a test with multiple input files
This diff adds a test which checks the error-message when multiple input files
are passed to llvm-install-name-tool.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87268
2020-09-08 11:31:28 -07:00
Alexander Shaposhnikov 10af5bad44 [llvm-objcopy] Consolidate and unify version tests
In this diff the tests which verify version printing functionality are refactored.
Since they are not specific to a particular format we move them into tool-version.test
and slightly unify (similarly to tool-name.test and tool-help-message.test).

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D87211
2020-09-07 18:44:32 -07:00
Xing GUO 1d01fc100b [Test] Simplify DWARF test cases. NFC.
The Length, AbbrOffset and Values fields of the debug_info section are
optional. This patch helps remove them and simplify test cases.

Reviewed By: MaskRay

Differential Revision: https://reviews.llvm.org/D86857
2020-08-31 14:03:48 +08:00
Xing GUO 290e399f96 [DWARFYAML] Add support for emitting multiple abbrev tables.
This patch adds support for emitting multiple abbrev tables. Currently,
compilation units will always reference the first abbrev table.

Reviewed By: jhenderson, labath

Differential Revision: https://reviews.llvm.org/D86194
2020-08-21 10:12:08 +08:00
Alexandre Ganea a680ea2c72 Fix "last accessed time" test failing on Windows
Before this patch, the tests in llvm/test/tools/llvm-objcopy/ELF/strip-preserve-atime.test used to fail on my machine, because the "last accessed time" is disabled in the OS by default since Windows XP. One needs to explicitly enable it for the feature to work. Otherwise the last access time is the last write time. Please see: https://superuser.com/questions/251263/the-last-access-date-is-not-changed-even-after-reading-the-file-on-windows-7

    Differential Revision: https://reviews.llvm.org/D85669
2020-08-10 16:03:14 -04:00
Fangrui Song dd405f1a53 Revert D83834 "Add test utility 'extract'"
This reverts commit d054c7ee2e.

There are discussions about the utility name, its functionality and user interface.
Revert before we reach consensus.
2020-07-28 13:26:33 -07:00
Fangrui Song d054c7ee2e Add test utility 'extract'
See https://lists.llvm.org/pipermail/llvm-dev/2020-July/143373.html
"[llvm-dev] Multiple documents in one test file" for some discussions.

`extract part filename` splits the input file into multiple parts separated by
regex `^(.|//)--- ` and extract the specified part to stdout or the
output file (if specified).

Use case A (organizing input of different formats (e.g. linker
script+assembly) in one file).

```
// RUN: extract lds %s -o %t.lds
// RUN: extract asm %s -o %t.s
// RUN: llvm-mc %t.s -o %t.o
// RUN: ld.lld -T %t.lds %t.o -o %t
This is sometimes better than the %S/Inputs/ approach because the user
can see the auxiliary files immediately and don't have to open another file.
```

Use case B (for utilities which don't have built-in input splitting
feature):

```
// RUN: extract case1 %s | llc | FileCheck %s --check-prefix=CASE1
// RUN: extract case2 %s | llc | FileCheck %s --check-prefix=CASE2
Combing tests prudently can improve readability.
This is sometimes better than having multiple test files.
```

Since this is a new utility, there is no git history concerns for
UpperCase variable names. I use lowerCase variable names like mlir/lld.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D83834
2020-07-23 19:15:35 -07:00
Sameer Arora 3b5db7fc69 [llvm-install-name-tool] Merge install-name options
This diff merges all options for llvm-install-name-tool under a single
function processLoadCommands. Also adds another test case for -add_rpath
option.

Test plan: make check-all

Reviewed by: jhenderson, alexshap, smeenai, Ktwu

Differential Revision: https://reviews.llvm.org/D82812
2020-07-06 20:32:32 -07:00
Eric Christopher 4029f8ede4 Temporarily Revert "[llvm-install-name-tool] Merge install-name options" as it breaks the objcopy build.
This reverts commit c143900a08.
2020-07-06 15:40:14 -07:00
Sameer Arora c143900a08 [llvm-install-name-tool] Merge install-name options
This diff merges all options for llvm-install-name-tool under a single
function processLoadCommands. Also adds another test case for -add_rpath
option.

Test plan: make check-all

Reviewed by: jhenderson, alexshap, smeenai, Ktwu

Differential Revision: https://reviews.llvm.org/D82812
2020-07-06 15:15:20 -07:00
Georgy Komarov afd81a637d
[llvm-objcopy] Fix crash when removing symbol table at same time as adding a symbol
This patch resolves crash that occurs when user wanted to remove all
symbols and add a brand new one using:

```
llvm-objcopy -R .symtab --add-symbol foo=1234 in.o out.o
```

Before these changes the symbol table internally being null when adding
new symbols. For now we will regenerate symtab in this case.

This fixes: https://bugs.llvm.org/show_bug.cgi?id=43930

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82935
2020-07-05 05:14:00 +03:00
Shoaib Meenai 5f56da3763 [llvm-install-name-tool] Tighten some path checks
Just having --implicit-check-not=/usr breaks when the LLVM checkout path
contains '/usr', since llvm-objdump prints out the path to the input
file in the first line. Tighten the checks by adding the 'name' prefix
that's used when printing load command payloads. An alternative would be
to redirect the input file into llvm-objdump, in which case it prints
out 'a.out' as the file name, but I'm not sure how reliable that
behavior is.
2020-06-30 17:20:00 -07:00
Sameer Arora 2bdcd8b14f [llvm-install-name-tool] Add -change option
Implement `-change` option for install-name-tool. The behavior exactly
matches that of cctools. Depends on D82410.

Reviewed By: jhenderson, smeenai

Differential Revision: https://reviews.llvm.org/D82613
2020-06-30 11:28:53 -07:00
Sameer Arora ca518c4918 [llvm-install-name-tool] Add -id option
Implement `-id` option for install-name-tool. Differences from cctool's
behavior:
 - Does **NOT** throw an error if multiple -id options are specified.
    Instead, picks the last one.
 - Throws an error in case empty id is specified.

Reviewed By: jhenderson, smeenai

Differential Revision: https://reviews.llvm.org/D82410
2020-06-30 11:28:53 -07:00
Xing GUO fe08ab542b [DWARFYAML][debug_info] Replace 'InitialLength' with 'Format' and 'Length'.
'InitialLength' is replaced with 'Format' (DWARF32 by default) and 'Length' in this patch.
Besides, test cases for DWARFv4 and DWARFv5, DWARF32 and DWARF64 is
added.

Reviewed By: jhenderson

Differential Revision: https://reviews.llvm.org/D82622
2020-06-30 16:28:39 +08:00
Alexander Shaposhnikov 2c663aa539 [llvm-objcopy][MachO] Fix segment size alignment
This diff addresses the old TODO in MachOObjcopy.cpp and
correctly sets the page size used for alignment of segments.
In particular, now llvm-objcopy's output is consistent
with the input (the alignment of vmsize doesn't change).

Test plan:

1. make check-all
2. verify that a binary copied via llvm-objcopy now correctly works on iOS.

Differential revision: https://reviews.llvm.org/D82503
2020-06-29 23:55:21 -07:00
Georgy Komarov e503851d80 [llvm-objcopy] Emit error if removing symtab referenced by group section
SHT_GROUP sections contain a reference to a symbol indicating their
"signature" symbol. The symbol table containing this symbol is referred
to by the group section's sh_link field. If llvm-objcopy is instructed
to remove the symbol table, it will emit an error.

This fixes https://bugs.llvm.org/show_bug.cgi?id=46153.

Reviewed By: jhenderson, Higuoxing

Differential Revision: https://reviews.llvm.org/D82274
2020-06-29 10:42:03 +01:00
Alexander Shaposhnikov b925ca37a8 [llvm-install-name-tool] Add support for -rpath option
This diff implements -rpath option for llvm-install-name-tool
which replaces the rpath value in the specified Mach-O binary.

Patch by Sameer Arora!

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D82051
2020-06-26 17:23:03 -07:00
Sid Manning e5911de377 [Hexagon][llvm-objcopy] Add missing check for SHN_HEXAGON_SCOMMON_1
Differential Revision: https://reviews.llvm.org/D82484
2020-06-24 19:56:01 -05:00
Alexander Shaposhnikov 395920a614 [llvm-objcopy] Update help message tests
This diff merges help message tests for llvm-objcopy, llvm-strip and
llvm-install-name-tool.

Patch by Sameer Arora!

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D82012
2020-06-24 12:40:31 -07:00
Alexander Shaposhnikov ca133cdecf [llvm-objcopy] Update help messages
This diff updates the help messages for llvm-objcopy, llvm-strip and
llvm-install-name-tool.

Patch by Sameer Arora!

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D81907
2020-06-24 11:19:56 -07:00
Alexander Shaposhnikov d332ec9209 [llvm-install-name-tool] Implement delete_rpath option
This diff adds support for deleting an rpath from a Mach-O binary.

Patch by Sameer Arora!

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D81527
2020-06-22 16:49:49 -07:00
Alexander Shaposhnikov 913bc312b5 [llvm-objcopy][MachO] Add support for LC_CODE_SIGNATURE
This diff adds support for copying binaries
containing a LC_CODE_SIGNATURE load command.

Test plan: make check-all

Differential revision: https://reviews.llvm.org/D81768
2020-06-15 18:55:59 -07:00