As we discussed here https://github.com/realm/SwiftLint/pull/3325 sometimes current excluding algorithm maybe slower than excluding paths by absolute prefix. So I added option for such cases.
Based on what I've checked it works faster for next scenarios:
- the number of input files is relatively small (e.g. when using `use-script-input-files`) and excluded directories contain relatively big number of lintable files
- the number of excluded directories relatively small (e.g. Pods + ThirdParty) and globs not used
Performance has gotten pretty bad for complex SwiftLint configurations like the one used for Lyft's iOS code base involving lots of files in the directories being linted, large configuration files and many nested configuration files.
Two main areas were particularly ripe for improvement were:
1. Collecting which files to lint
2. Lint cache lookups
### Collecting which files to lint
Improve this by:
* using an NSOrderedSet to remove excluded paths instead of `Array.filter`
* parallelizing calls to `filesToLint` for all paths to lint and exclude
* using `FileManager.subpaths(atPath:)` instead of `enumerator(atPath:)`
|Change|Before|After|Speed up|
|-|-|-|-|
|NSOrderedSet|2.438s|0.917s|2.659x|
|Parallel Flat Map|2.438s|2.248s|1.085x|
|Subpaths|0.939s|0.867s|1.083x|
|**Total**|**2.438s**|**0.720s**|**3.386x**|
### Lint cache lookups
By using an MD5 hash of the Configuration description from CryptoSwift as the cache key instead of instead the full description, we can drastically speed up cache lookups for projects with complex SwiftLint configurations. I think the dictionary lookup for very large string keys doesn't perform very well.
---
* Speed up Configuration.lintablePaths
* Improve cache lookup performance by up to 10x
By using an MD5 hash of the Configuration description from CryptoSwift
as the cache key instead of instead the full description.
* Add changelog entries
* Swift 4.0 & Linux compatibility
* os(Darwin) isn't a thing
* Allow warnings in pod lib lint
SwiftLint supports building with Swift 4.0 to 4.2.
There is no version of CryptoSwift to support both Swift 4.0 and
Swift 4.2.
So allow warnings for now. We'll make one more Swift 4.0 compatible
release, then we'll bump the build requirements to Swift 4.2 and
remove the `--allow-warnings` flag.
The MIT license doesn't require that all files be prepended with this
licensing or copyright information. Realm confirmed that they're ok with this
change. This will enable some companies to contribute to SwiftLint and the
date & authorship information will remain accessible via git source control.
also slightly speed up writing to the cache.
For example, on the Lyft codebase with 1,500 Swift files:
```bash
$ time swiftlint lint --quiet
swiftlint --quiet 3.53s user 0.27s system 388% cpu 0.979 total
$ rm -rf ~/Library/Caches/SwiftLint && time swiftlint lint --quiet
swiftlint --quiet 35.20s user 1.22s system 371% cpu 9.806 total
$ time swiftlint lint --quiet
swiftlint lint --quiet 0.90s user 0.13s system 218% cpu 0.472 total
$ rm -rf ~/Library/Caches/SwiftLint && time swiftlint lint --quiet
swiftlint lint --quiet 31.78s user 1.18s system 360% cpu 9.146 total
```
Martin Redington