reparsing an ASTUnit. When saving a preamble, create a buffer larger
than the actual file we're working with but fill everything from the
end of the preamble to the end of the file with spaces (so the lexer
will quickly skip them). When we load the file, create a buffer of the
same size, filling it with the file and then spaces. Then, instruct
the lexer to start lexing after the preamble, therefore continuing the
parse from the spot where the preamble left off.
It's now possible to perform a simple preamble build + parse (+
reparse) with ASTUnit. However, one has to disable a bunch of checking
in the PCH reader to do so. That part isn't committed; it will likely
be handled with some other kind of flag (e.g., -fno-validate-pch).
As part of this, fix some issues with null termination of the memory
buffers created for the preamble; we were trying to explicitly
NULL-terminate them, even though they were also getting implicitly
NULL terminated, leading to excess warnings about NULL characters in
source files.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@109445 91177308-0d34-0410-b5e6-96231b3b80d8
code into a MoveUpIncludeHierarchy helper, and use the helper to
fix a case involving macros which regressed from my recent patch.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@103288 91177308-0d34-0410-b5e6-96231b3b80d8
to be algorithmically faster and avoid an std::map. This routine
basically boils down to finding the nearest common ancestor in a
tree, and we (implicitly) have information about nesting depth,
use it!
This wraps up rdar://7948633 - SourceManager::isBeforeInTranslationUnit has poor performance
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@103239 91177308-0d34-0410-b5e6-96231b3b80d8
method to be correct. Right now it correctly computes the cache, then
goes ahead and computes the result the hard way, then asserts that they
match. Next I'll actually turn it on.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@103231 91177308-0d34-0410-b5e6-96231b3b80d8
method will sometimes return different results for the same input SourceLocations. I haven't
unraveled this method completely yet, so this truly is a workaround until a better fix comes
along.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@103143 91177308-0d34-0410-b5e6-96231b3b80d8
into ContentCache::getBuffer. This allows it to produce
diagnostics on the broken #include line instead of without a
location.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@101939 91177308-0d34-0410-b5e6-96231b3b80d8
about it instead of producing tons of garbage from the lexer.
It would be even better for sourcemgr to dynamically transcode (e.g.
from UTF16 -> UTF8).
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@101924 91177308-0d34-0410-b5e6-96231b3b80d8
precompiled headers and/or when reading the contents of the file into
memory. These checks seem to be causing spurious regression-test
failures on Windows.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@100866 91177308-0d34-0410-b5e6-96231b3b80d8
Diagnostic subsystem, which is used in the rare case where we find a
serious problem (i.e., an inconsistency in the file system) while
we're busy formatting another diagnostic. In this case, the delayed
diagnostic will be emitted after we're done with the other
diagnostic. This is only to be used for fatal conditions detected at
very inconvenient times, where we can neither stop the current
diagnostic in flight nor can we suppress the second error.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@99175 91177308-0d34-0410-b5e6-96231b3b80d8
entry in a precompiled header, so that we can detect modified files
even when we miss in the stat cache.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@99149 91177308-0d34-0410-b5e6-96231b3b80d8
deserialization of precompiled headers, where the deserialization of
the source location entry for a buffer (e.g., macro instantiation
scratch space) would overwrite a one-element FileID cache in the
source manager. When tickled at the wrong time, we would return the
wrong decomposed source location and eventually cause c-index-test to
crash.
Found by dumb luck. It's amazing this hasn't shown up before.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98940 91177308-0d34-0410-b5e6-96231b3b80d8
buffer was invalid when it was created, and use that bit to always set
the "Invalid" flag according to whether the buffer is invalid. This
ensures that all accesses to an invalid buffer are marked invalid,
improving recovery.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98690 91177308-0d34-0410-b5e6-96231b3b80d8
SourceManager's getBuffer() and, therefore, could fail, along with
Preprocessor::getSpelling(). Use the Invalid parameters in the literal
parsers (string, floating point, integral, character) to make them
robust against errors that stem from, e.g., PCH files that are not
consistent with the underlying file system.
I still need to audit every use caller to all of these routines, to
determine which ones need specific handling of error conditions.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98608 91177308-0d34-0410-b5e6-96231b3b80d8
SourceManager's getBuffer() (and similar) operations. This abstract
can be used to force callers to cope with errors in getBuffer(), such
as missing files and changed files. Fix a bunch of callers to use the
new interface.
Add some very basic checks for file consistency (file size,
modification time) into ContentCache::getBuffer(), although these
checks don't help much until we've updated the main callers (e.g.,
SourceManager::getSpelling()).
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@98585 91177308-0d34-0410-b5e6-96231b3b80d8
end-of-line source location when given a column number beyond the
length of the line, or an end-of-file source location when given a
line number beyond the length of the file. Previously, we would return
an invalid location.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@97299 91177308-0d34-0410-b5e6-96231b3b80d8
we'd add an offset from the spelling location space to the
instantiation location, which doesn't make sense and would
lead up to the text diagnostics crashing when presented with
non-sensical locations.
This fixes rdar://7597492, a crash on 255.vortex.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@96004 91177308-0d34-0410-b5e6-96231b3b80d8
files.
- The issue is that PCH uses a stat cache, which may reference files which have
been deleted or moved. In such cases ContentCache::getBuffer was returning 0
but most clients are incapable of dealing with this (i.e., they don't).
For the time being, resolve this issue by just making up some invalid file
contents and. Eventually we should detect that we are in an inconsistent
situation and error out with a nice message that the PCH is out of date.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90699 91177308-0d34-0410-b5e6-96231b3b80d8
files with the contents of an arbitrary memory buffer. Use this new
functionality to drastically clean up the way in which we handle file
truncation for code-completion: all of the truncation/completion logic
is now encapsulated in the preprocessor where it belongs
(<rdar://problem/7434737>).
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90300 91177308-0d34-0410-b5e6-96231b3b80d8
The later assumption is patently false, but this was already broken -- this situation is conceptually impossible, my feeling is we should fix SourceManager and friends to make it impossible in practice as well. However, we need to fix PR5662 and perhaps some other things involving memory buffers first. In the short term I'm pretty sure this is reliable.
Chris, Argiris, is this going to break anything that wasn't already broken?
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90280 91177308-0d34-0410-b5e6-96231b3b80d8
in diagnostics when we fail to open a file. This allows us to
report things like:
$ clang test.c -I.
test.c:2:10: fatal error: error opening file './foo.h': Permission denied
#include "foo.h"
^
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@90276 91177308-0d34-0410-b5e6-96231b3b80d8
-code-completion-at=filename:line:column
which performs code completion at the specified location by truncating
the file at that position and enabling code completion. This approach
makes it possible to run multiple tests from a single test file, and
gives a more natural command-line interface.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@82571 91177308-0d34-0410-b5e6-96231b3b80d8
file. In particular, only eagerly load source location entries for
files and for the predefines buffer. Other buffers and
macro-instantiation source location entries are loaded lazily.
With the Cocoa-prefixed "Hello, World", we only load 815/26555 source
location entities. This halves the amount of user time we spend in
this "Hello, World" program with -fsyntax-only (down to .007s).
This optimization is part 1 of 2 for the source manager. This
eliminates most of the user time in loading a PCH file. We still spend
too much time initialize File structures (especially in the calls to
stat), so we need to either make the loading of source location
entries for files lazy or import the stat cache from the PTH
implementation.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@70196 91177308-0d34-0410-b5e6-96231b3b80d8
Now instead of just tracking the expansion history, also track the full
range of the macro that got replaced. For object-like macros, this doesn't
change anything. For _Pragma and function-like macros, this means we track
the locations of the ')'.
This is required for PR3579 because apparently GCC uses the line of the ')'
of a function-like macro as the location to expand __LINE__ to.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@64601 91177308-0d34-0410-b5e6-96231b3b80d8
line markers, including maintenance of the virtual include stack.
For something like this:
# 42 "bar.c" 1
# 142 "bar2.c" 1
#warning zappa
# 92 "bar.c" 2
#warning gonzo
# 102 "foo.c" 2
#warning bonkta
we now produce these three warnings:
#1:
In file included from foo.c:3:
In file included from bar.c:42:
bar2.c:143:2: warning: #warning zappa
#warning zappa
^
#2:
In file included from foo.c:3:
bar.c:92:2: warning: #warning gonzo
#warning gonzo
^
#3:
foo.c:102:2: warning: #warning bonkta
#warning bonkta
^
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63722 91177308-0d34-0410-b5e6-96231b3b80d8
location below it report as coming from the #line location. For example,
with:
#line 92 "foo.h"
#warning blarg!
#warning blarg!
we now emit:
foo.h:92:2: warning: #warning blarg!
#warning blarg!
^
foo.h:92:2: warning: #warning blarg!
#warning blarg!
^
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63709 91177308-0d34-0410-b5e6-96231b3b80d8
getColumnNumber. This fixes a FIXME in
SourceManager::getPresumedLoc because we now just decompose
the sloc once.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63701 91177308-0d34-0410-b5e6-96231b3b80d8
makes it clear to clients that they have to pick an instantiation
or spelling location before calling it and allows optimization based
on that.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63698 91177308-0d34-0410-b5e6-96231b3b80d8
ContentCache objects to using a densemap and list, and allocating
the ContentCache objects from a bump pointer. This does not speed
up or slow down things substantially, but gives us control over
their alignment.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63628 91177308-0d34-0410-b5e6-96231b3b80d8
as reported to the user and as manipulated by #line. This is what __FILE__,
__INCLUDE_LEVEL__, diagnostics and other things should follow (but not
dependency generation!).
This patch also includes several cleanups along the way:
- SourceLocation now has a dump method, and several other places
that did similar things now use it.
- I cleaned up some code in AnalysisConsumer, but it should probably be
simplified further now that NamedDecl is better.
- TextDiagnosticPrinter is now simplified and cleaned up a bit.
This patch is a prerequisite for #line, but does not actually provide
any #line functionality.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63098 91177308-0d34-0410-b5e6-96231b3b80d8
address space we used up. Some interesting data:
For c99-intconst-1.c:
6912762 SLocEntry's allocated, 25592386B of Sloc address space used.
For cocoa.h:
26469 SLocEntry's allocated, 10278752B of Sloc address space used.
For carbon.h:
27364 SLocEntry's allocated, 12398141B of Sloc address space used.
Clearly 2G of sloc address space should be enough for anyone?!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63093 91177308-0d34-0410-b5e6-96231b3b80d8
source locations, allow creation of them. We can now say that
a token was instantiated here, then here, then here.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63034 91177308-0d34-0410-b5e6-96231b3b80d8
which I think is rdar://6527005, and make getDecomposedSpellingLocSlowCase
handle nested spelling locations.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@63030 91177308-0d34-0410-b5e6-96231b3b80d8
ground work for implementing #line, and fixes the "out of macro ID's"
problem.
There is nothing particularly tricky about the code, other than the
very performance sensitive SourceManager::getFileID() method.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62978 91177308-0d34-0410-b5e6-96231b3b80d8
the chunk ID not the file ID. This exposes problems in
TextDiagnosticPrinter where it should have been using the canonical
file ID but wasn't. Fix these along the way.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62427 91177308-0d34-0410-b5e6-96231b3b80d8
"FileID" a concept that is now enforced by the compiler's type checker
instead of yet-another-random-unsigned floating around.
This is an important distinction from the "FileID" currently tracked by
SourceLocation. *That* FileID may refer to the start of a file or to a
chunk within it. The new FileID *only* refers to the file (and its
#include stack and eventually #line data), it cannot refer to a chunk.
FileID is a completely opaque datatype to all clients, only SourceManager
is allowed to poke and prod it.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62407 91177308-0d34-0410-b5e6-96231b3b80d8
the "physical" location of tokens, refer to the "spelling" location.
This is more concrete and useful, tokens aren't really physical objects!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@62309 91177308-0d34-0410-b5e6-96231b3b80d8
- Big Idea:
Source files are now mmaped when ContentCache::getBuffer() is first called.
While this doesn't change the functionality when lexing regular source files,
it can result in source files not being paged in when using PTH.
- Performance change:
- No observable difference (-fsyntax-only/-Eonly) on Cocoa.h when doing
regular source lexing.
- No observable time difference (-fsyntax-only/-Eonly) on Cocoa.h when using
PTH. We do observe, however, a reduction of 279K in memory mapped source
code (3% reduction). The majority of pages from Cocoa.h (and friends) are
still being pulled in, however, because any literal will cause
Preprocessor::getSpelling() to be called (causing the source for the file to
get pulled in). The next possible optimization is to cache literal strings
in the PTH file to avoid the need for the original header sources entirely.
- Right now there is a preprocessor directive to toggle between "lazy" and
"eager" creation of MemBuffers. This is not permanent, and is there in the
short term to just test additional optimizations.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61827 91177308-0d34-0410-b5e6-96231b3b80d8
- 'Buffer' is now private and must be accessed via 'getBuffer()'.
This paves the way for lazily mapping in source files on demand.
- Added 'getSize()' (which gets the size of the content without
necessarily accessing the MemBuffer) and 'getSizeBytesMapped()'.
- Modifed SourceManager to use these new methods. This reduces the
number of places that actually access the MemBuffer object for a file
to those that actually look at the character data.
These changes result in no performance change for -fsyntax-only on Cocoa.h.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@61782 91177308-0d34-0410-b5e6-96231b3b80d8
to whether the fileid is a 'extern c system header' in addition to whether it
is a system header, most of this is spreading plumbing around. Once we have that,
PPLexerChange bases its "file enter/exit" notifications to PPCallbacks to
base the system header state on FileIDInfo instead of HeaderSearch. Finally,
in Preprocessor::HandleIncludeDirective, mirror logic in GCC: the system headerness
of a file being entered can be set due to the #includer or the #includee.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@56688 91177308-0d34-0410-b5e6-96231b3b80d8
* Move FormatError() from TextDiagnostic up to DiagClient, remove now
empty class TextDiagnostic
* Make DiagClient optional for Diagnostic
This fixes the following problems:
* -html-diags (and probably others) does now output the same set of
warnings as console clang does
* nothing crashes if one forgets to call setHeaderSearch() on
TextDiagnostic
* some code duplication is removed
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@54620 91177308-0d34-0410-b5e6-96231b3b80d8
The new MemoryBuffer doesn't "leak" file descriptors and handles the
small file case efficiently.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@49032 91177308-0d34-0410-b5e6-96231b3b80d8
This is a temporary solution to avoid running out of file descriptors (which defaults to 256).
Need to benchmark to understand the speed benefit. If the benefit is small, the simple solution is to avoid memory mapping files. If the benefit is significant, more thought is necessary.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48991 91177308-0d34-0410-b5e6-96231b3b80d8
lib dir and move all the libraries into it. This follows the main
llvm tree, and allows the libraries to be built in parallel. The
top level now enforces that all the libs are built before Driver,
but we don't care what order the libs are built in. This speeds
up parallel builds, particularly incremental ones.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@48402 91177308-0d34-0410-b5e6-96231b3b80d8
Douglas Gregor