George Karpenkov
ef9eb3e6cd
[analyzer] [tests] Add an option for showing statistics after running tests.
...
Do not show statistics by default.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@338323 91177308-0d34-0410-b5e6-96231b3b80d8
2018-07-30 23:01:47 +00:00
George Karpenkov
cfa1c8b413
[analyzer] [tests] Style fixes for testing harness.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@338322 91177308-0d34-0410-b5e6-96231b3b80d8
2018-07-30 23:01:20 +00:00
Mikhail R. Gadelha
3d25876adf
Fix a (possible) division by zero check in the CmpRuns script
...
I missed updating the check in r333375
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@333529 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-30 11:17:55 +00:00
Mikhail R. Gadelha
cea74394ed
Introduces --stats-only option to only show changes in statistics.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@333375 91177308-0d34-0410-b5e6-96231b3b80d8
2018-05-28 15:40:39 +00:00
George Karpenkov
a701b12944
[analyzer] [tests] Update CmpRuns to write to stdout correctly in multithreaded environment
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@325070 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-13 23:36:01 +00:00
George Karpenkov
9f84a49dbd
[analyzer] [tests] Fixing an error after non-atomic cherry-pick
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@324762 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-09 18:48:31 +00:00
George Karpenkov
3cc1cf93fc
[analyzer] [tests] [NFC] Remove a fragile tightly-coupled component emulating parser output
...
...when we can just use the real parser instead.
Differential Revision: https://reviews.llvm.org/D43098
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@324759 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-09 18:39:47 +00:00
George Karpenkov
c58ab38fd0
[analyzer] [tests] Show function name in CmpRuns output
...
Combined with enabled flag for stable filenames, this greatly simplifies
finding the offending report.
Differential Revision: https://reviews.llvm.org/D42831
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@324362 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-06 17:22:09 +00:00
George Karpenkov
7a1006f6f3
[analyzer] [tests] Fix crash in SATestBuild.py
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@324114 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-02 18:27:14 +00:00
George Karpenkov
de212366a3
[analyzer] [tests] [NFC] Remove dead code from CmpRuns
...
Indeed, "CHANGE" is not a thing yet, and we should probably not carry
around dead code which does not do anything apart from confusing the
reader.
Differential Revision: https://reviews.llvm.org/D42819
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@324027 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-01 22:40:01 +00:00
George Karpenkov
ef79198e68
[analyzer] [tests] Add an option to show the histogram of path differences between the analyzer runs
...
Differential Revision: https://reviews.llvm.org/D42778
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@324021 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-01 22:25:18 +00:00
George Karpenkov
ea016449cc
[analyzer] [tests] Show the number of removed/added bug reports
...
Differential Revision: https://reviews.llvm.org/D42718
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@323941 91177308-0d34-0410-b5e6-96231b3b80d8
2018-02-01 02:38:42 +00:00
George Karpenkov
86395f102e
[Analyzer] Make testing scripts flake8 compliant
...
Differential Review: https://reviews.llvm.org/D38213
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@314692 91177308-0d34-0410-b5e6-96231b3b80d8
2017-10-02 17:59:12 +00:00
George Karpenkov
d578df7f78
[Analyzer] Refactor analyzer testing scripts.
...
- Exporting needed function for future reuse.
- Idiomatic python: using with `file as f` instead of `try/finally`.
- Fixing some indentation issues.
- No need to reinvent python `multiprocessing.getCPUCount()`
- Removing a function parameter which is always the same under all invocations.
- Adding some docstrings.
Differential Revision: https://reviews.llvm.org/D38156
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@313949 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-22 01:41:16 +00:00
George Karpenkov
b937f758ff
[Analyzer] Remove dead code from CmpRuns.py.
...
Differential Revision: https://reviews.llvm.org/D38003
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@313923 91177308-0d34-0410-b5e6-96231b3b80d8
2017-09-21 21:47:13 +00:00
Gabor Horvath
61a690a04f
[analyzer] Bug identification
...
This patch adds hashes to the plist and html output to be able to identfy bugs
for suppressing false positives or diff results against a baseline. This hash
aims to be resilient for code evolution and is usable to identify bugs in two
different snapshots of the same software. One missing piece however is a
permanent unique identifier of the checker that produces the warning. Once that
issue is resolved, the hashes generated are going to change. Until that point
this feature is marked experimental, but it is suitable for early adoption.
Differential Revision: http://reviews.llvm.org/D10305
Original patch by: Bence Babati!
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@251011 91177308-0d34-0410-b5e6-96231b3b80d8
2015-10-22 11:53:04 +00:00
Ted Kremenek
99bb39afb4
[analyzer] Apply whitespace cleanups by Honggyu Kim.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@246978 91177308-0d34-0410-b5e6-96231b3b80d8
2015-09-08 03:50:52 +00:00
Gabor Horvath
7d70c91050
[Static Analyzer] Make the paths relative from the project root when generating reference results in the test suite.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@241706 91177308-0d34-0410-b5e6-96231b3b80d8
2015-07-08 18:39:31 +00:00
Gabor Horvath
939c4bfc23
[Static Analyzer] Minor improvements to SATest.
...
Differential Revision: http://reviews.llvm.org/D10812
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@241073 91177308-0d34-0410-b5e6-96231b3b80d8
2015-06-30 15:31:17 +00:00
Jordan Rose
e30024c96f
[analyzer] CmpRuns.py: Accept single files as input.
...
This allows us to compare two direct invocations of the analyzer on a
single source file without having to wrap the output plists in their
own directories.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@177804 91177308-0d34-0410-b5e6-96231b3b80d8
2013-03-23 01:21:26 +00:00
Anna Zaks
f0024960d5
[analyzer] CmpRuns: move the clang_version info into the run (out of
...
report)
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@168105 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-15 22:42:44 +00:00
Anna Zaks
25691f67b2
[analyzer] Ensure that CmpRuns recursively walks the output directory.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@167981 91177308-0d34-0410-b5e6-96231b3b80d8
2012-11-14 21:32:16 +00:00
Anna Zaks
20d928164a
[analyzer] CmpRuns: cleanup APIs, allow processing of plists with no
...
clang_version, fix the compare routine.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@166129 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-17 21:09:26 +00:00
Anna Zaks
68aa3a9389
[analyzer] Expose clang_version in CmpRuns
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@166048 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-16 19:36:39 +00:00
Anna Zaks
81765577db
[analyzer] CmpRuns - cleaned up the API, added relative path getter
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@165995 91177308-0d34-0410-b5e6-96231b3b80d8
2012-10-15 22:48:21 +00:00
Anna Zaks
44cceb96ea
[analyzer] Ted fixed the plist-html issue, so the assert can go away.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162598 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-24 21:07:49 +00:00
Anna Zaks
b81a9adc24
[analyzer] Unbreak the static analyzer buildbot.
...
We no longer associate the correct HTML file with plist diagnostic.
Fall out from r162028.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@162071 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-17 01:09:13 +00:00
Anna Zaks
2a84b8bd10
[analyzer] CmpRuns should include file name in the issue identifier.
...
This prevents us from treating the issues from different files with the
same function names and same offsets as the same.
The issue identifier now includes the file name. Also added a way to
strip off the root directories form the source file names.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@161150 91177308-0d34-0410-b5e6-96231b3b80d8
2012-08-02 00:41:40 +00:00
Jordan Rose
ac57d47546
Un-break standalone invocations of CmpRuns.py.
...
Fallout from CmpRuns.py API changes in r160314.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@160800 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-26 20:03:51 +00:00
Anna Zaks
7acc407240
[analyzer] Make CmpRuns external-user friendly.
...
CmpRuns can be used for static analyzer bug report comparison. However,
we want to make sure external users do not rely on the way bugs are
represented (plist files). Make sure that we have a user
friendly/documented API for CmpRuns script.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@160314 91177308-0d34-0410-b5e6-96231b3b80d8
2012-07-16 20:21:42 +00:00
Anna Zaks
19b17cb57a
[analyzer] Use "issue hash" in CmpRuns; followup on r158180
...
(For the future: It would be more efficient to produce a hash key with
the embedded function info inside the compiler.)
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@158187 91177308-0d34-0410-b5e6-96231b3b80d8
2012-06-08 01:50:49 +00:00
Anna Zaks
2c3038edc2
[analyzer] Add the HTML file to the SATest diagnostic diff.
...
(Uses the functionality which has been in CmpRuns long before.)
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148868 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-24 21:57:35 +00:00
Anna Zaks
ab2d46eb04
[analyzer] Testing: make diagnostic diffs more informative (add
...
diagnostic message).
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@148867 91177308-0d34-0410-b5e6-96231b3b80d8
2012-01-24 21:57:32 +00:00
Anna Zaks
a7a2564ff5
[analyzer] Testing: Display the number of reported differences in the main log.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@144113 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-08 19:56:31 +00:00
Anna Zaks
e1e73670b3
[analyzer] Relax the cmp criteria for analyzer reports.
...
Until we find out a way to easily find out what changed by looking at the logs.
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@143789 91177308-0d34-0410-b5e6-96231b3b80d8
2011-11-05 05:20:56 +00:00
Anna Zaks
f95a2012ab
[analyzer] CmpRuns can now optionally delete empty reports.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139550 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-12 22:40:36 +00:00
Anna Zaks
544055fa46
[analyzer] CmpRuns.cmpScanBuildResults() should be easy to call from other modules.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139543 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-12 21:32:41 +00:00
Anna Zaks
22d70e0ca3
Rename CmpRuns into CmpRuns.py so that it could be used as a module.
...
git-svn-id: https://llvm.org/svn/llvm-project/cfe/trunk@139540 91177308-0d34-0410-b5e6-96231b3b80d8
2011-09-12 21:07:18 +00:00