closes#14204
In order to add tests for this I need to implement an HTTP server in the
standard library (#910) so that's probably the next thing I'll do.
* std.http.Status.Class: add a "nonstandard" enum tag. Instead of
having `class` return an optional value, it can potentially return
nonstandard.
* extract out std.http.Client.Connection from std.http.Client.Request
- this code abstracts over plain/TLS only
- this is the type that will potentially be stored in a client's LRU
connection map
* introduce two-staged HTTP header parsing
- API users can rely on a heap-allocated buffer with a maximum limit,
which defaults to 16 KB, or they can provide a static buffer that
is borrowed by the Request instance.
- The entire HTTP header is buffered because there are strings in
there and they must be accessed later, such as with the case of
HTTP redirects.
- When buffering the HTTP header, the parser only looks for the
\r\n\r\n pattern. Further validation is done later.
- After the full HTTP header is buffered, it is parsed into
components such as Content-Length and Location.
* HTTP redirects are handled, with a maximum redirect count option that
defaults to 3.
- Connection: close is always used for now; implementing keep-alive
connections and an LRU connection pool in std.http.Client is a task
for another day.
see #2007
Although RFC 8446 states:
> Each party MUST send a "close_notify" alert before closing its write
> side of the connection
In practice many servers do not do this. Also in practice, truncation
attacks are thwarted at the application layer by comparing the amount of
bytes received with the amount expected via the HTTP headers.
It appears to be implemented incorrectly in the wild and causes the read
connection to be closed even though that is a direct violation of RFC
8446 Section 6.1.
The writeEnd function variants are still there, ready to be used.
This commit adds `writeEnd` and `writeAllEnd` in order to send data and
also notify the server that there will be no more data written.
Unfortunately, it seems most TLS implementations in the wild get this
wrong and immediately close the socket when they see a close_notify,
rather than only ending the data stream on the application layer.
This is a streaming HTTP header parser. All it currently does is detect
the end of headers. This will be a non-allocating parser where one can
bring supply their own buffer if they want to handle custom headers.
This commit also improves std.http.Client to not return the HTTP headers
with the read functions.
The read function has been renamed to readAdvanced since it has slightly
different semantics than typical read functions, specifically regarding
the end-of-file. A higher level read function is implemented on top.
Now, API users may pass small buffers to the read function and
everything will work fine. This is done by re-decrypting the same
ciphertext record with each call to read() until the record is finished
being transmitted.
If the buffer supplied to read() is large enough, then any given
ciphertext record will only be decrypted once, since it decrypts
directly to the read() buffer and therefore does not need any memcpy. On
the other hand, if the buffer supplied to read() is small, then the
ciphertext is decrypted into a stack buffer, a subset is copied to the
read() buffer, and then the entire ciphertext record is saved for the
next call to read().
