When the request is asked which tokens are have in an intersection, the previous solution was searching for first Index (linearly) and then filtered everything that was coming after that index using `intersect function`
The updated solution will search for the first token index using`binary search`
# Speedup
While this is only one function was updated, the next options were considered:
- [**lin+filter**] old solution
- [**lin+prefix**] old solution with `prefix:wihle` instead of filtering
- [**bin+filter**] binary search with filter
- [**bin+prefix**] binary search with `prefix:wihle` instead of filtering
The speedup highly depends on the file sizes. The bigger/longer files the bigger win is
# Benchmark
## Kickstarter
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|0.494|0.243|0.390|\***0.117\***| ~4x |
## Swift
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|1.739|0.740|1.273|\***0.103**\*| ~16x |
## WordPress
|lin+filter|lin+prefix|bin+filter|bin+prefix|speedup|
|-|-|-|-|-|
|1.270|0.526|0.918|0.148| ~8x |
# Testing code
This code was tested with these parts of code (in Release build)
```
fileprivate var counter = 0
fileprivate var times: [String: Double] = [:]
fileprivate let timesQueue = DispatchQueue.init(label: "benchmarks")
fileprivate func timeLog<T>(_ name: String, block: () -> T) -> T {
let start = DispatchTime.now()
let result = block()
let end = DispatchTime.now()
timesQueue.async {
let oldValue = times[name, default:0.0]
let diff = TimeInterval(end.uptimeNanoseconds - start.uptimeNanoseconds) / 1_000_000_000
let newValue = oldValue + diff
times[name] = newValue
counter += 1
if counter % 1000 * times.count == 0 {
print("!!!!: \(times)")
}
}
return result
}
internal func tokens(inByteRange byteRange: NSRange) -> [SyntaxToken] {
let new = timeLog("new") { tokensFast(inByteRange: byteRange) }
let new2 = timeLog("old") { tokensOld(inByteRange: byteRange) }
return arc4random() % 2 == 1 ? new : new2
}
```
JP Simard