When the request is asked which tokens are have in an intersection, the previous solution was searching for first Index (linearly) and then filtered everything that was coming after that index using `intersect function`
The updated solution will search for the first token index using`binary search`
# Speedup
While this is only one function was updated, the next options were considered:
- [**lin+filter**] old solution
- [**lin+prefix**] old solution with `prefix:wihle` instead of filtering
- [**bin+filter**] binary search with filter
- [**bin+prefix**] binary search with `prefix:wihle` instead of filtering
The speedup highly depends on the file sizes. The bigger/longer files the bigger win is
# Benchmark
## Kickstarter
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|0.494|0.243|0.390|\***0.117\***| ~4x |
## Swift
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|1.739|0.740|1.273|\***0.103**\*| ~16x |
## WordPress
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|1.270|0.526|0.918|0.148| ~8x |
# Testing code
This code was tested with these parts of code (in Release build)
```
fileprivate var counter = 0
fileprivate var times: [String: Double] = [:]
fileprivate let timesQueue = DispatchQueue.init(label: "benchmarks")
fileprivate func timeLog<T>(_ name: String, block: () -> T) -> T {
let start = DispatchTime.now()
let result = block()
let end = DispatchTime.now()
timesQueue.async {
let oldValue = times[name, default:0.0]
let diff = TimeInterval(end.uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000
let newValue = oldValue + diff
times[name] = newValue
counter += 1
if counter % 1000 * times.count == 0 {
print("!!!!: \(times)")
}
}
return result
}
internal func tokens(inByteRange byteRange: NSRange) -> [SyntaxToken] {
let new = timeLog("new") { tokensFast(inByteRange: byteRange) }
let new2 = timeLog("old") { tokensOld(inByteRange: byteRange) }
return arc4random() % 2 == 1 ? new : new2
}
```
* Use binary search when searching for lines indexes
* Update Changelog
* Remove unused old method of searching line by offset
* Update line by offset call with already implemented function
* Add Mark rule for triple slash Mark comments
* Move changelog entry to appropriate section
* Re-add Package.resolved
* Fix trailing comma
* Rebuild Rules.md using Xcode 10.3
* Combine two regexes into one
* Require Swift 5.0 to build
* Update CI
* Stop testing with Swift 4.x & Xcode 10.0/10.1
* Use official Swift docker image instead of norionomura's
* Use Xcode 10.3 as latest stable version
* Update READMEs
* Fixup xcodeproj
* Fixup CI Swift container image tag
* Fixup changelog
As was discussed #2737, I utilized the `indexsource` request to look up the operators and fetch the cursorInfo per operator. In the implementation I refrained from using `usr` to look up the operator because SourceKit [doesn't support](5add168042/tools/SourceKit/lib/SwiftLang/SwiftSourceDocInfo.cpp (L1799)) fetching cursorInfo by `usr` when it comes from C (but it can still be fetched by offset).
I also made [an alternative implementation](https://github.com/realm/SwiftLint/compare/master...biboran:fix-unused-imports-for-operators-alternative) which uses the `indexsource` request instead of the `editor.open`. It seems to work with the unit tests but I am not 100% sure it doesn't introduce a regression since there is no oss-check for analyzer rules.
I also updated [the example](https://github.com/biboran/synthax-bug-example) to better reflect the original issue in case you want to test the changes manually
This PR adds a new `unused_declaration` analyzer rule to lint for unused declarations.
By default, detects unused `fileprivate`, `private` and `internal` declarations.
Configure the rule with `include_public_and_open: true` to also detect unused `public` and `open` declarations.
Completely remove the `unused_private_declaration` rule.
This is built on the work enabling collecting rule infrastructure in https://github.com/realm/SwiftLint/pull/2714.
All CollectingRules implement AnyCollectingRule, which is used to check
whether a linter will perform any collections and only print the
"Collecting" log message if so.
In order for rules to collect arbitrary information about all files
being linted, a shared RuleStorage instance is defined in each command
and passed into the linter.
Linting now requires two "passes": once to call collect and populate the
storage (rules that are non-collecting do nothing here), and again to
call validate. The old Linter factory now creates a Prelinter, which can
collect for a file and produce a Linter that orchestrates all the
traditional validation/collection logic.
This design enforces that a file is only validated once it has been
collected; in turn, the file-visiting loop ensures that all files are
collected before the first is validated, so that the storage is fully
populated.
Use storage-backed correct method
Crash if storage for a rule is accessed prematurely
Key FileInfo by File
Rename Prelinter to Linter and Linter to CollectedLinter
Clean up rule protocols such that rule-facing storage methods are actually called
Make RuleStorage a reference type to solve mutating data races
* Migrate LinterCache to use Codable models
improving performance and type safety
* Fix Linux
* Avoid creating a Decoder if it won't be used
For example if the file doesn't exist or can't be read.
* Use corelibs plist coder if available
It's available in the Swift 5.1 branch: https://github.com/apple/swift-corelibs-foundation/pull/1849
* Remove unused error case
* Split cache into one file per configuration
* Only write cache files with changes
* Avoid converting from Data -> String -> Data when saving cache files
* Move empty check to for where
* Split cache changelog entry
* Reword changelog entry to better reflect impact
Prevents extensions being flagged as in the wrong order when a file only contains extensions and nothing else (enums, classes, structs). In this case it's not intuitive that the longest extension be considered the "main" one so the file can be considered as having no "main" type.
* Add nestedTypeSeparator to FileNameConfiguration
* Add tests for nested_type_separator configurability of file_name rule
* Add changelog entry
* Fix changelog position after rebase
* Update project to compile with Xcode 10.2
* Fix explicit_acl with Swift 5
* Update macOS VM image
* Fix ConfigurationTests on Swift 5
* Update Podspec
* Workaround SR-10486 to fix validation tests
* Fix testDetectSwiftVersion on Swift 5
* Update specs repo when validating CocoaPods spec
* Fix redundant_set_access_control tests
* Manually update Package.resolved
* Use .upToNextMinor
* [Azure Pipelines] Add `sw_vers`
* Relax expecting conditions for crashing output in `testSimulateHomebrewTest()`
Because TSAN makes `swiflint` to produce additional output on crashing.
* Add CHANGELOG entry
* Ignore extensions in explicit ACL rules
* Work around SR-10486 for Sourcery
* [reduce_boolean] Implement rule
* [reduce_boolean] Improve reason text
* [reduce_boolean] Fix new violations
* [reduce_boolean] Add CHANGELOG.md entry
* [reduce_boolean] Shorten rule name
* [reduce_boolean] Use regular map instead of compactMap
* [reduce_boolean] Match only possible syntax kinds
Paul Taykalo