When a '>>' token is split into two '>' tokens (in C++11 onwards), or (as an
extension) when we do the same for other tokens starting with a '>', we can't
just use a location pointing to the first '>' as the location of the split
token, because that would result in our miscomputing the length and spelling
for the token. As a consequence, for example, a refactoring replacing 'A<X>'
with something else would sometimes replace one character too many, and
similarly diagnostics highlighting a template-id source range would highlight
one character too many.
Fix this by creating an expansion range covering the first character of the
'>>' token, whose spelling is '>'. For this to work, we generalize the
expansion range of a macro FileID to be either a token range (the common case)
or a character range (used in this new case).
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@331155 91177308-0d34-0410-b5e6-96231b3b80d8
Fix makes the loop in LexAngledStringLiteral more like the loops in
LexStringLiteral, LexCharConstant. When we skip a character after
backslash, we need to check if we reached the end of the file instead of
reading the next character unconditionally.
Discovered by OSS-Fuzz:
https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=3832
rdar://problem/35572754
Reviewers: arphaman, kcc, rsmith, dexonsmith
Reviewed By: rsmith, dexonsmith
Subscribers: cfe-commits, rsmith, dexonsmith
Differential Revision: https://reviews.llvm.org/D41423
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@322390 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: This patch implements 4.3 of http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4220.pdf. If a raw string contains a newline character, replace each newline character with the \n escape code. Without this patch, included test case (macro_raw_string.cpp) results compilation failure.
Reviewers: rsmith, doug.gregor, jkorous-apple
Reviewed By: jkorous-apple
Subscribers: jkorous-apple, vsapsai, cfe-commits
Differential Revision: https://reviews.llvm.org/D39279
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@319904 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Lexer::GetBeginningOfToken produced invalid location when
backtracking across escaped new lines.
This fixes PR26228
Reviewers: akyrtzi, alexfh, rsmith, doug.gregor
Reviewed By: alexfh
Subscribers: alexfh, cfe-commits
Patch by Paweł Żukowski!
Differential Revision: https://reviews.llvm.org/D30748
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@310576 91177308-0d34-0410-b5e6-96231b3b80d8
- Extracted the reading of the tokens out into a separate function.
- Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).
This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.
I will also directly update some extra clang tooling that is broken by the change from Argument to Parameter.
Hopefully the bots will stay appeased.
Thanks!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@308190 91177308-0d34-0410-b5e6-96231b3b80d8
- Extracted the reading of the tokens out into a separate function.
- Replace 'Argument' with 'Parameter' when referring to the identifiers of the macro definition (as opposed to the supplied arguments - MacroArgs - during the macro invocation).
This is in preparation for submitting patches for review to implement __VA_OPT__ which will otherwise just keep lengthening the HandleDefineDirective function and making it less comprehensible.
Thanks!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@308157 91177308-0d34-0410-b5e6-96231b3b80d8
A new LexerTest unittest introduced a memory leak. This patch
uses a unique_ptr with a custom deleter to ensure it is properly
deleted.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@305491 91177308-0d34-0410-b5e6-96231b3b80d8
Werror was catching a signed/unsigned compare in
an assert, correct the signed 'expected' value to be
unsigned.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@305435 91177308-0d34-0410-b5e6-96231b3b80d8
correct getNumArguments
StringifiedArguments is allocated (resized) based on the size the
getNumArguments function. However, this function ACTUALLY currently
returns the amount of total UnexpArgTokens which is minimum the same as
the new implementation of getNumMacroArguments, since empty/omitted arguments
result in 1 UnexpArgToken, and included ones at minimum include 2
(1 for the arg itself, 1 for eof).
This patch renames the otherwise unused getNumArguments to be more clear
that it is the number of arguments that the Macro expects, and thus the maximum
number that can be stringified. This patch also replaces the explicit memset
(which results in value instantiation of the new tokens, PLUS clearing the
memory) with brace initialization.
Differential Revision: https://reviews.llvm.org/D32046
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@305425 91177308-0d34-0410-b5e6-96231b3b80d8
as part of a compilation.
This is intended for two purposes:
1) Writing self-contained test cases for modules: we can now write a single
source file test that builds some number of module files on the side and
imports them.
2) Debugging / test case reduction. A single-source testcase is much more
amenable to reduction, compared to a VFS tarball or .pcm files.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@305101 91177308-0d34-0410-b5e6-96231b3b80d8
This reverts commit r298185, effectively reapplying r298165, after fixing the
new unit tests (PR32338). The memory buffer generator doesn't null-terminate
the MemoryBuffer it creates; this version of the commit informs getMemBuffer
about that to avoid the assert.
Original commit message follows:
----
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@298278 91177308-0d34-0410-b5e6-96231b3b80d8
Clang's internal build system for implicit modules uses lock files to
ensure that after a process writes a PCM it will read the same one back
in (without contention from other -cc1 commands). Since PCMs are read
from disk repeatedly while invalidating, building, and importing, the
lock is not released quickly. Furthermore, the LockFileManager is not
robust in every environment. Other -cc1 commands can stall until
timeout (after about eight minutes).
This commit changes the lock file from being necessary for correctness
to a (possibly dubious) performance hack. The remaining benefit is to
reduce duplicate work in competing -cc1 commands which depend on the
same module. Follow-up commits will change the internal build system to
continue after a timeout, and reduce the timeout. Perhaps we should
reconsider blocking at all.
This also fixes a use-after-free, when one part of a compilation
validates a PCM and starts using it, and another tries to swap out the
PCM for something new.
The PCMCache is a new type called MemoryBufferCache, which saves memory
buffers based on their filename. Its ownership is shared by the
CompilerInstance and ModuleManager.
- The ModuleManager stores PCMs there that it loads from disk, never
touching the disk if the cache is hot.
- When modules fail to validate, they're removed from the cache.
- When a CompilerInstance is spawned to build a new module, each
already-loaded PCM is assumed to be valid, and is frozen to avoid
the use-after-free.
- Any newly-built module is written directly to the cache to avoid the
round-trip to the filesystem, making lock files unnecessary for
correctness.
Original patch by Manman Ren; most testcases by Adrian Prantl!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@298165 91177308-0d34-0410-b5e6-96231b3b80d8
The lexer sets the end location of macro arguments incorrectly *if*,
while merging consecutive args to fit into a single SLocEntry, it finds
args which come from different macro files.
Fix the issue by using separate SLocEntries in this situation.
This fixes a code coverage crasher (rdar://problem/26181005). Because
the lexer reported end locations for certain macro args incorrectly, we
would generate bogus coverage mappings with negative line offsets.
Reviewed-by: akyrtzi
Differential Revision: http://reviews.llvm.org/D20401
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@270160 91177308-0d34-0410-b5e6-96231b3b80d8
Use it to calculate UserLabelPrefix, instead of specifying it (often
incorrectly).
Note that the *actual* user label prefix has always come from the
DataLayout, and is handled within LLVM. The main thing clang's
TargetInfo::UserLabelPrefix did was to set the #define value. Having
these be different from each-other is just silly.
Differential Revision: http://reviews.llvm.org/D17183
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@262737 91177308-0d34-0410-b5e6-96231b3b80d8
It has no place there; it's not a property of the Module, and it makes
restoring the visibility set when we leave a submodule more difficult.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@236300 91177308-0d34-0410-b5e6-96231b3b80d8
This corrects long-standing misuses of LLVM's internal config.h.
In most cases the public llvm-config.h header was intended and we can now
remove the old hacks thanks to LLVM r210144.
The config.h header is private, won't be installed and should no longer be
included by clang or other modules.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@210145 91177308-0d34-0410-b5e6-96231b3b80d8
Eliminate createMainFileID() / createMainFileIDForMemBuffer() utility
functions. These didn't add much convenience and conflated two distinct
operations.
This change makes things easier to follow by providing a consistent interface
and getting rid of a bunch of cast-to-voids.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@209266 91177308-0d34-0410-b5e6-96231b3b80d8
The Preprocessor::Initialize() function already offers a clear interface to
achieve this, further reducing the confusing number of states a newly
constructed preprocessor can have.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@207825 91177308-0d34-0410-b5e6-96231b3b80d8
This allows using virtual file mappings on the original SourceManager to
map in virtual module.map files. Without this patch, the ModuleMap
search will find a module.map file (as the FileEntry exists in the
FileManager), but will be unable to get the content from the
SourceManager (as ModuleMap previously created its own SourceManager).
Two problems needed to be fixed which this patch exposed:
1. Storing the inferred module map
When writing out a module, the ASTWriter stores the names of the files
in the main source manager; when loading the AST again, the ASTReader
errs out if such a file is found missing, unless it is overridden.
Previously CompilerInstance's compileModule method would store the
inferred module map to a temporary file; the problem with this approach
is that now that the module map is handled by the main source manager,
the ASTWriter stores the name of the temporary module map as source to
the compilation; later, when the module is loaded, the temporary file
has already been deleted, which leads to a compilation error. This patch
changes the inferred module map to instead inject a virtual file into
the source manager. This both saves some disk IO, and works with how the
ASTWriter/ASTReader handle overridden source files.
2. Changing test input in test/Modules/Inputs/*
Now that the module map file is handled by the main source manager, the
VerifyDiagnosticConsumer will not ignore diagnostics created while
parsing the module map file. The module test test/Modules/renamed.m uses
-I test/Modules/Inputs and triggers recursive loading of all module maps
in test/Modules/Inputs, some of which had conflicting names, thus
leading errors while parsing the module maps. Those diagnostics already
occur on trunk, but before this patch they would not break the test, as
they were ignored by the VerifyDiagnosticConsumer. This patch thus
changes the module maps that have been recently introduced which broke
the invariant of compatible modules maps in test/Modules/Inputs.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@193314 91177308-0d34-0410-b5e6-96231b3b80d8
the new Objective-C NSArray/NSDictionary/NSNumber literal syntax.
This introduces a new library, libEdit, which provides a new way to support
migration of code that improves on the original ARC migrator. We now believe
that most of its functionality can be refactored into the existing libraries,
and thus this new library may shortly disappear.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@152141 91177308-0d34-0410-b5e6-96231b3b80d8
The class name is long enough without the llvm:: added.
Also bring in RefCountedBase and RefCountedBaseVPTR.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@150958 91177308-0d34-0410-b5e6-96231b3b80d8
Richard Smith