Summary: This patch implements 4.3 of http://open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4220.pdf. If a raw string contains a newline character, replace each newline character with the \n escape code. Without this patch, included test case (macro_raw_string.cpp) results compilation failure.
Reviewers: rsmith, doug.gregor, jkorous-apple
Reviewed By: jkorous-apple
Subscribers: jkorous-apple, vsapsai, cfe-commits
Differential Revision: https://reviews.llvm.org/D39279
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@319904 91177308-0d34-0410-b5e6-96231b3b80d8
when needed
This commit implements the semicolon insertion logic into the extract
refactoring. The following rules are used:
- extracting expression: add terminating ';' to the extracted function.
- extracting statements that don't require terminating ';' (e.g. switch): add
terminating ';' to the callee.
- extracting statements with ';': move (if possible) the original ';' from the
callee and add terminating ';'.
- otherwise, add ';' to both places.
Differential Revision: https://reviews.llvm.org/D39441
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@317343 91177308-0d34-0410-b5e6-96231b3b80d8
This patch fixes broken preamble-skipping when the preamble region includes a byte order mark (BOM). Previously, parsing would fail if preamble PCH generation was enabled and a BOM was present.
This also fixes preamble invalidation when a BOM appears or disappears. This may seem to be an obscure edge case, but it happens regularly with IDEs that pass buffer overrides that never (or always) have a BOM, yet the underlying file from the initial parse that generated a PCH might (or might not) have a BOM.
I've included a test case for these scenarios.
Differential Revision: https://reviews.llvm.org/D37491
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@313796 91177308-0d34-0410-b5e6-96231b3b80d8
Discovered due to a goofy git setup, the test system-headerline-directive.c
(and a few others) failed because the token-consumption will consume only the
'\r' in CRLF, making the preprocessor's printed value give the wrong line number
when returning from an include. For example:
(line 1):#include <noline.h>\r\n
The "file exit" code causes the printer to try to print the 'returned to the
main file' line. It looks up what the current line number is. However, since the
current 'token' is the '\n' (since only the \r was consumed), it will give the
line number as '1", not '2'. This results in a few failed tests, but more
importantly, results in error messages being incorrect when compiling a
previously preprocessed file.
Differential Revision: https://reviews.llvm.org/D37079
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@311683 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Lexer::GetBeginningOfToken produced invalid location when
backtracking across escaped new lines.
This fixes PR26228
Reviewers: akyrtzi, alexfh, rsmith, doug.gregor
Reviewed By: alexfh
Subscribers: alexfh, cfe-commits
Patch by Paweł Żukowski!
Differential Revision: https://reviews.llvm.org/D30748
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@310576 91177308-0d34-0410-b5e6-96231b3b80d8
for preprocessing
r300667 added support for editor placeholder to Clang. That commit didn’t take
into account that users who use Clang for preprocessing only (-E) will get the
"editor placeholder in source file" error when preprocessing their source
(PR33394). This commit ensures that Clang doesn't lex editor placeholders when
running a preprocessor only action.
rdar://32718000
Differential Revision: https://reviews.llvm.org/D34256
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@305576 91177308-0d34-0410-b5e6-96231b3b80d8
Previously, a preamble only included #if blocks (and friends like
ifdef) if there was a corresponding #endif before any declaration or
definition. The problem is that any header file that uses include guards
will not have a preamble generated, which can make code-completion very
slow.
To prevent errors about unbalanced preprocessor conditionals in the
preamble, and unbalanced preprocessor conditionals after a preamble
containing unfinished conditionals, the conditional stack is stored
in the pch file.
This fixes PR26045.
Differential Revision: http://reviews.llvm.org/D15994
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@304207 91177308-0d34-0410-b5e6-96231b3b80d8
retrieving the identifer info for an Objective-C keyword
This commit fixes an assertion that's triggered in getIdentifier when the token
is an annotation token.
rdar://32225463
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@303246 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds a fix-it for the -Wunguarded-availability warning. This fix-it
is similar to the Swift one: it suggests that you wrap the statement in an
`if (@available)` check. The produced fixits are indented (just like the Swift
ones) to make them look nice in Xcode's fix-it preview.
rdar://31680358
Differential Revision: https://reviews.llvm.org/D32424
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@302253 91177308-0d34-0410-b5e6-96231b3b80d8
This commit teaches Clang to recognize editor placeholders that are produced
when an IDE like Xcode inserts a code-completion result that includes a
placeholder. Now when the lexer sees a placeholder token, it emits an
'editor placeholder in source file' error and creates an identifier token
that represents the placeholder. The parser/sema can now recognize the
placeholders and can suppress the diagnostics related to the placeholders. This
ensures that live issues in an IDE like Xcode won't get spurious diagnostics
related to placeholders.
This commit also adds a new compiler option named '-fallow-editor-placeholders'
that silences the 'editor placeholder in source file' error. This is useful
for an IDE like Xcode as we don't want to display those errors in live issues.
rdar://31581400
Differential Revision: https://reviews.llvm.org/D32081
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@300667 91177308-0d34-0410-b5e6-96231b3b80d8
Previously, if an escaped newline was followed by a newline or a nul, we'd lex
the escaped newline as a bogus space character. This led to a bunch of
different broken corner cases:
For the pattern "\\\n\0#", we would then have a (horizontal) space whose
spelling ends in a newline, and would decide that the '#' is at the start of a
line, and incorrectly start preprocessing a directive in the middle of a
logical source line. If we were already in the middle of a directive, this
would result in our attempting to process multiple directives at the same time!
This resulted in crashes, asserts, and hangs on invalid input, as discovered by
fuzz-testing.
For the pattern "\\\n" at EOF (with an implicit following nul byte), we would
produce a bogus trailing space character with spelling "\\\n". This was mostly
harmless, but would lead to clang-format getting confused and misformatting in
rare cases. We now produce a trailing EOF token with spelling "\\\n",
consistent with our handling for other similar cases -- an escaped newline is
always part of the token containing the next character, if any.
For the pattern "\\\n\n", this was somewhat more benign, but would produce an
extraneous whitespace token to clients who care about preserving whitespace.
However, it turns out that our lexing for line comments was relying on this bug
due to an off-by-one error in its computation of the end of the comment, on the
slow path where the comment might contain escaped newlines.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@300515 91177308-0d34-0410-b5e6-96231b3b80d8
Summary: When using the C preprocessor with assembly files, either with a
capital `S` file extension, or with `-xassembler-with-cpp`, the Unicode escape
sequence `\u` is ignored. The `\u` pattern can be used for expanding a macro
argument that starts with `u`.
Author: Salman Arif <salman.arif@arm.com>
Reviewers: rengolin, olista01
Reviewed By: olista01
Subscribers: cfe-commits
Differential Revision: https://reviews.llvm.org/D31765
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@299754 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
This lets people link against LLVM and their own version of the UTF
library.
I determined this only affects llvm, clang, lld, and lldb by running
$ git grep -wl 'UTF[0-9]\+\|\bConvertUTF\bisLegalUTF\|getNumBytesFor' | cut -f 1 -d '/' | sort | uniq
clang
lld
lldb
llvm
Tested with
ninja lldb
ninja check-clang check-llvm check-lld
(ninja check-lldb doesn't complete for me with or without this patch.)
Reviewers: rnk
Subscribers: klimek, beanz, mgorny, llvm-commits
Differential Revision: https://reviews.llvm.org/D24996
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@282822 91177308-0d34-0410-b5e6-96231b3b80d8
This can happen as we look for '<<<<' while scanning tokens but then expect
'<<<<\n' to tell apart perforce from diff3 conflict markers. Just harden
the pointer arithmetic.
Found by libfuzzer + asan!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@265125 91177308-0d34-0410-b5e6-96231b3b80d8
This patch adds the reserved operator ^^ when compiling for OpenCL (spec v1.1 s6.3.g),
which results in a more meaningful error message.
Patch by Neil Hickey!
Review: http://reviews.llvm.org/D13280
M test/SemaOpenCL/unsupported.cl
M include/clang/Basic/TokenKinds.def
M include/clang/Basic/DiagnosticParseKinds.td
M lib/Basic/OperatorPrecedence.cpp
M lib/Lex/Lexer.cpp
M lib/Parse/ParseExpr.cpp
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@259651 91177308-0d34-0410-b5e6-96231b3b80d8
Move the function to get a macro name from DiagnosticRenderer.cpp to Lexer.cpp
so that other files can use it. Lexer now has two functions to get the
immediate macro name, the newly added one is better for diagnostic purposes.
Make -Wnull-conversion use this function for better NULL macro detection.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@258778 91177308-0d34-0410-b5e6-96231b3b80d8
Summary:
Fix PR22407, where the Lexer overflows the buffer when parsing
#include<\
(end of file after slash)
Test Plan:
Added a test that will trigger in asan build.
This case is also covered by the clang-fuzzer bot.
Reviewers: rnk
Reviewed By: rnk
Subscribers: cfe-commits
Differential Revision: http://reviews.llvm.org/D9489
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@236466 91177308-0d34-0410-b5e6-96231b3b80d8
We would check if the terminator marker is on a newline. However, the
logic would end up out-of-bounds if the terminator marker immediately
follows the start marker.
This fixes PR21820.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@224210 91177308-0d34-0410-b5e6-96231b3b80d8
Changes diagnostic options, language standard options, diagnostic identifiers, diagnostic wording to use c++14 instead of c++1y. It also modifies related test cases to use the updated diagnostic wording.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@215982 91177308-0d34-0410-b5e6-96231b3b80d8
This code doesn't care where the data it is processing comes from, so a
StringRef is probably the most natural interface.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@215448 91177308-0d34-0410-b5e6-96231b3b80d8
(dropping const from the reference as MemoryBuffer is immutable already,
so const is just redundant - and while I'd personally put const
everywhere, that's not the LLVM Way (see llvm::Type for another example
of an immutable type where "const" is omitted for brevity))
Changing the pointer argument to a reference parameter makes call sites
identical between callers with unique_ptrs or raw pointers, minimizing
the churn in a pending unique_ptr migrations.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@215391 91177308-0d34-0410-b5e6-96231b3b80d8
The compilation pipeline doesn't actually need to know about the high-level
concept of diagnostic mappings, and hiding the final computed level presents
several simplifications and other potential benefits.
The only exceptions are opportunistic checks to see whether expensive code
paths can be avoided for diagnostics that are guaranteed to be ignored at a
certain SourceLocation.
This commit formalizes that invariant by introducing and using
DiagnosticsEngine::isIgnored() in place of individual level checks throughout
lex, parse and sema.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@211005 91177308-0d34-0410-b5e6-96231b3b80d8
Extend the SSE2 comment lexing to AVX2. Only 16byte align when not on AVX2.
This provides some 3% speedup when preprocessing gcc.c as a single file.
The patch is wrong, it always uses SSE2, and when I fix that there's no speedup
at all. I am not sure where the 3% came from previously.
--Thi lie, and those below, will be ignored--
M Lex/Lexer.cpp
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@205548 91177308-0d34-0410-b5e6-96231b3b80d8
Taewook Oh