RFC: The HTTP Wrapper Type

Status: RFC

Applies to: client

For a summarized list of proposed changes, see the Changes Checklist section.

This RFC defines the API of our wrapper types around http::Request and http::Response. For more information about why we are wrapping these types, see RFC 0036: The HTTP Dependency.

Terminology

Extensions / "Request Extensions": The http crate Request/Response types include a typed property bag to store additional metadata along with the request.

The user experience if this RFC is implemented

In the current version of the SDK, external customers and internal code interacts directly with the http crate. Once this RFC is implemented, interactions at the public API level will occur with our own http types instead.

Our types aim to be nearly drop-in-compatible for types in the http crate, however:

We will not expose existing HTTP types in public APIs in ways that are ossified.
When possible, we aim to simplify the APIs to make them easier to use.
We will add SDK specific helper functionality when appropriate, e.g. first-level support for applying an endpoint to a request.

How to actually implement this RFC

We will need to add two types, HttpRequest and HttpResponse.

To string or not to String

Our header library restricts header names and values to Strings (UTF-8).

Although the http library is very precise in its representation—it allows for HeaderValues that are both a super and subset of String—a superset because headers support arbitrary binary data but a subset because headers cannot contain control characters like \n.

Although technically allowed, headers containing arbitrary binary data are not widely supported. Generally, Smithy protocols will use base-64 encoding when storing binary data in headers.

Finally, it's nicer for users if they can stay in "string land". Because of this, HttpRequest and Response expose header names and values as strings. Internally, the current design uses HeaderName and HeaderValue, however, there is a gate on construction that enforces that values are valid UTF-8.

This is a one way door because .as_str() would panic in the future if we allow non-string values into headers.

Where should these types live?

These types will be used by all orchestrator functionality, so they will be housed in aws-smithy-runtime-api

What's in and what's out?

At the onset, these types focus on supporting the most ossified usages: &mut modification of HTTP types. They do not support construction of HTTP types, other than impl From<http::Request> and From<http::Response>. We will also make it possible to use http::HeaderName / http::HeaderValue in a zero-cost way.

The `AsHeaderComponent` trait

All header insertion methods accept impl AsHeaderComponent. This allows us to provide a nice user experience while taking advantage of zero-cost usage of 'static str. We will seal this trait to prevent external usage. We will have separate implementation for:

&'static str
String
http02x::HeaderName

Additional Functionality

Our wrapper type will add the following additional functionality:

Support for self.try_clone()
Support for &mut self.apply_endpoint(...)

Handling failure

There is no stdlib type that cleanly defines what may be placed into headers—String is too broad (even if we restrict to ASCII). This RFC proposes moving fallibility to the APIs:

impl HeadersMut<'_> {
    pub fn try_insert(
        &mut self,
        key: impl AsHeaderComponent,
        value: impl AsHeaderComponent,
    ) -> Result<Option<String>, BoxError> {
        // ...
    }
}

This allows us to offer user-friendly types while still avoiding runtime panics. We also offer insert and append which panic on invalid values.

Request Extensions

There is ongoing work which MAY restrict HTTP extensions to clone types. We will preempt that by:

Preventing Extensions from being present when initially constructing our HTTP request wrapper.
Forbidding non-clone extensions from being inserted into the wrapped request.

This also enables supporting request extensions for different downstream providers by allowing cloning into different extension types.

Proposed Implementation

Proposed Implementation of `request`

{{#include ../../../rust-runtime/aws-smithy-runtime-api/src/client/http/request.rs}}

Future Work

Currently, the only way to construct Request is from a compatible type (e.g. http02x::Request)

Changes checklist

Implement initial implementation and test it against the SDK as written
Add test suite of HTTP wrapper
External design review
Update the SigV4 crate to remove http API dependency
Update the SDK to use the new type (breaking change)

Smithy Rust