Motivation:
A small number of socket options have values that do not fit into a C
int type. Our current ChannelOption based approach for setting these
simply does not work, and cannot be extended to support the truly arbitrary
types that the setsockopt/getsockopt functions allow here.
This makes it impossible to use some socket options, which is hardly a
good place to be.
There were a number of ways we could have addressed this: we could have
special-cased all socket options with non-integer types in ChannelOption,
but I believe that would be too manual, and risk limiting users that need
to set other socket options. We could also have added a ChannelOption
that simply allows users to pass a buffer to write into or read from,
but that is a very un-Swift-like API that feels pretty gross to hold.
Ultimately, the nicest seemed to be a new protocol users could check
for, and that would provide APIs that let users hold the correct concrete
type. As with setsockopt/getsockopt, while this API is typed it is
not type-safe: ultimately, the struct we have here is treated just as a
buffer by setsockopt/getsockopt. We do not attempt to prevent users from
shooting themselves in the foot here.
This PR does not include an example use case in any server, as I will
provide such an example in a subsequent multicast PR.
Modifications:
- Added a SocketOptionChannel protocol.
- Conformed BaseSocketChannel to SocketOptionChannel.
- Wrote some tests for this.
Result:
Users can set and get sockopts with data that is larger than a C int.
Motivation:
Quite embarrasingly, we previously would only store one `ChannelOption`
per `ChannelOption` type. Most channel option types are distinct and
that's probably why it took so long to find this issue. Thanks
@pushkarnk for reporting. Unfortunately though, the most important
`ChannelOption` is `.socket` which crucially also holds a level and a
name. That means if you set two `ChannelOptions.socket` options with
distinct name/level, one would still override the other.
Example:
.serverChannelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEPORT), value: 1)
.serverChannelOption(ChannelOptions.socket(SocketOptionLevel(SOL_SOCKET), SO_REUSEADDR), value: 1)
would only actually set the latter.
Modifications:
- made all common `ChannelOption` types equatable (for 2.0 this will
be a protocol requirement)
- deprecated non-Equatable `ChannelOption` types
- zero out buffer before calling getsockopt as Linux doesn't do that
Result:
you can now set two distinct `ChannelOptions` for one type
match all values in HTTPHeaders.isKeepAlive(...)
HTTPHeaders.isKeepAlive(...) does only match the first value.
### Motivation:
Keep-alive and Close may be something on comma separated arrays
### Modifications:
- Made an extension function to `ByteBuffer` that could separate the strings at low level, without Swift string API, so that it could split the values
- Another extension functions that compares the comma separated array with a given array to tell the caller which exists and which no
- Added unit tests for them
- Used them in the HTTPHeader
### Result:
- Now, if a request is sent using Keep-alive or Close where it was in an array, it will be handled
- fixes#410
Motivation:
Previously we were not running the (child/server)channelInitializers on the
event loop associated to the `Channel` we're initialising. Almost all
operations in there are pipeline modifications which are thread safe so
it presumably wasn't a massive correctness issue. However it's very
expensive to hop threads that often and it is also very unexpected. This
addresses this issue.
Modifications:
made all (child/server)channelInitializers run on the event loop that is
associated to their `Channel`
Result:
- more correctness
- less unexpected behaviour
Motivation:
In certain cases it's useful to quiesce a channel instead of just
closing them immediately for example when receiving a signal.
This lays the groundwork by introducing the
`ChannelShouldQuiesceUserEvent` user event that when received can be
interpreted by a ChannelHandler in a protocol & application specific
way. Some protocols support tear down and that would be a good place to
initiate the tear down.
Modifications:
- introduce `ChannelShouldQuiesceUserEvent`
- handle `ChannelShouldQuiesceUserEvent` in the `AcceptHandler` with
closing the server socket
- handle `ChannelShouldQuiesceUserEvent` in the
`HTTPServerPipelineHandler` by only handling a already in-flight
request and then no longer accepting input
- added `CircularBuffer.removeAll` (& tests)
- added tests for `nextPowerOf2()`
Result:
- handlers can now support quiescing
Motivation:
When implementing a custom ChannelCore, you will probably need access
to the data inside a NIOAny. It should be possible to unwrap that.
Modifications:
Added an extension to ChannelCore to allow unwrapping a NIOAny.
Added @_versioned to almost all of NIOAny.
Result:
ChannelCore is implementable outside NIO
Motivation:
HTTP message framing has a number of edge cases that NIO currently does
not tolerate. We should decide what our position is on each of these edge
cases and handle it appropriately.
Modifications:
Provide an extensive test suite that codifies our expected handling of
these edge cases. Fix divergences from this behaviour.
Result:
Better tolerance for the weird corners of HTTP.
Motivation:
Currently the HTTP decoders can throw errors, but they will be ignored
and lead to a simple EOF. That's not ideal: in most cases we should make
a best-effort attempt to send a 4XX error code before we shut the client
down.
Modifications:
Provided a new ChannelHandler that generates 400 errors when the HTTP
decoder fails.
Added a flag to automatically add that handler to the channel pipeline.
Added the handler to the HTTP sample server.
Enabled integration test 12.
Result:
Easier error handling for HTTP servers.
Motivation:
Our HTTP code handles only HTTP/1.X. There is no reason to support
HTTP/0.9, and we cannot safely handle a major protocol higher than 1 in
this code, so we should simply treat requests/responses claiming to be
of those protocols as errors.
Modifications:
HTTPDecoder now checks the major version is equal to 1 before it
continues with parsing. If it hits an error, that error will be propagated
out to the user.
Result:
Better resilience against bad HTTP messages.
Motivation:
We only use our GetaddrinfoResolver for SocketChannels (TCP stream channels), so we should set some hints for getaddrinfo.
Modifications:
Set proto and socktype hints.
Result:
More correct usage of getaddrinfo. Fixes https://github.com/apple/swift-nio/issues/201.
Motivation:
We triggered extra allocations for the generics on PriorityQueue/Heap.
They unfortunately can't be solved using `@_specialize` as the element
type is `ScheduledTask`.
Fortunately we never exposed those types externally so we can just
inline without breaking the public API.
Modifications:
Moved everything from the `NIOPriorityQueue` module into the `NIO`
module.
Result:
Less allocations, more happiness.
Motivation:
It's possible someone deregistered a Selectable while Selector.whenReady(...) is processed and an even for the now deregistered Selectable is ready. In this case we crashed (due a force-unwrap) on linux.
Modifications:
- Skip the processing of the Selectable if deregistered (by replacing force-unwrap with an if let).
- Add unit test.
Result:
No more crashes caused by this on Linux
Motivation:
We not always ensure correct ordering which made it quite hard to reason about things. We should always first notify the promise before we call the fire* method that belongs to the event. Beside this we sometimes fired events or notified promised before correctly update the active state / addresses of a Channel which could result in unexpected results when query the Channel during execution of callbacks and handlers.
Modifications:
- Ensure we always notify promise first
- Always correctly update channel state before notify promise
- Add test to verify notification order.
Result:
Correct ordering of events which makes things easier to reason about and which follows what netty is doing.
Motivation:
Often accept errors are recoverable over time as for example you may just run out of file descriptors. Because of this it may be useful to just "delay" accepts a bit to recover.
Modifications:
Add ChannelHandler which will backoff accepts.
Result:
Be able to recover from accept errors gracefully.
Motivation:
Websockets is a major protocol in use on the web today, and is
particularly valuable in applications that use asynchronous I/O
as it allows servers to keep connections open for long periods of
time for fully-duplex communication.
Users of NIO should be able to build websocket clients and servers
without too much difficulty.
Modifications:
Provided a WebsocketFrameEncoder and Decoder that can serialize and
deserialize Websocket frames.
Result:
Easier use of websockets.
Motivation:
HTTP pipelining can be tricky to handle properly on the server side.
In particular, it's very easy to write out of order or inconsistently
mutate state. Users often need help to handle this appropriately.
Modifications:
Added a HTTPServerPipelineHandler that only lets one request through
at a time.
Result:
Better servers that are more able to handle HTTP pipelining
Motivation:
Right now the test gen script will rewrite every file. Not ideal!
Modifications:
Added the license header, and made the follow-on comment match that
style.
Result:
Test files will look the same when running this script as they do
now.
* Expose BlockingIOThreadPool
Motivation:
Sometimes we need to execute some blocking IO. For this we should expose the BlockingIOThreadPool that can be used.
Modifications:
- Factor out BlockingIOThreadPool
- Added tests
- Correctly start threadpool before execute NonBlockingIO tests.
Result:
Possible to do blocking IO.
* Corrys comment
* Correctly start pool before using it
- This fixes Bootstrap's ChannelOptionStore.applyAll to return a future rather
than synchronously iterating through all the options.
- This is particular important because when a server accepts a child, if the
child channel is on a different event loop then it is possible the
synchronous calls may deadlock (if the child's eventloop happens to be
scheduled with a similar accept sequence).
- I did not tackle also making getOption() async, which means the Channel API
is asymmetric at the moment. That should probably be addressed, potentialy
with synchronous wrappers for API compatibility.
- Fixes: <rdar://problem/37191923> [Omega] Worker tasks fail to close subtasks (many connections in CLOSE_WAIT state)
motivation: generated linux tests dont have consitent sorting in different OSs
changes:
* change test generator to sort tests and imports in memory before writing to file
* update LinuxMain to the sorted version
* HTTP/1.1 Chunked Encoding
* fix MarkedCircularBuffer: if not marked do not try to move mark when removing first
* fix HTTPRequestDecoder: dispatch callouts after parsing is complete
* fix HTTPTest: feed pipeline the inbound data correctly
Cory Benfield