RFC: Supporting multiple HTTP versions for SDKs that use Event Stream
Status: Accepted
For a summarized list of proposed changes, see the Changes Checklist section.
Most AWS SDK operations use HTTP/1.1, but bi-directional streaming operations that use the Event Stream message framing format need to use HTTP/2 (h2).
Smithy models can also customize which HTTP versions are used in each individual protocol trait.
For example,
@restJson1
has attributes http
and eventStreamHttp
to list out the versions that should be used in a priority order.
There are two problems in play that this doc attempts to solve:
- Connector Creation: Customers need to be able to create connectors with the HTTP settings they desire, and these custom connectors must align with what the Smithy model requires.
- Connector Selection: The generated code must be able to select the connector that best matches the requirements from the Smithy model.
Terminology
Today, there are three layers of Client
that are easy to confuse, so to make the following easier to follow,
the following terms will be used:
- Connector: An implementor of Tower's
Service
trait that converts a request into a response. This is typically a thin wrapper around a Hyper client. - Smithy Client: A
aws_smithy_client::Client<C, M, R>
struct that is responsible for gluing together the connector, middleware, and retry policy. This isn't intended to be used directly. - Fluent Client: A code generated
Client<C, M, R>
that has methods for each service operation on it. A fluent builder is generated alongside it to make construction easier. - AWS Client: A specialized Fluent Client that uses a
DynConnector
,DefaultMiddleware
, andStandard
retry policy.
All of these are just called Client
in code today. This is something that could be clarified in a separate refactor.
How Clients Work Today
Fluent clients currently keep a handle to a single Smithy client, which is a wrapper
around the underlying connector. When constructing operation builders, this handle is Arc
cloned and
given to the new builder instances so that their send()
calls can initiate a request.
The generated fluent client code ends up looking like this:
struct Handle<C, M, R> {
client: aws_smithy_client::Client<C, M, R>,
conf: crate::Config,
}
pub struct Client<C, M, R = Standard> {
handle: Arc<Handle<C, M, R>>,
}
Functions are generated per operation on the fluent client to gain access to the individual operation builders. For example:
pub fn assume_role(&self) -> fluent_builders::AssumeRole<C, M, R> {
fluent_builders::AssumeRole::new(self.handle.clone())
}
The fluent operation builders ultimately implement send()
, which chooses the one and only Smithy client out
of the handle to make the request with:
pub struct AssumeRole<C, M, R> {
handle: std::sync::Arc<super::Handle<C, M, R>>,
inner: crate::input::assume_role_input::Builder,
}
impl<C, M, R> AssumeRole<C, M, R> where ...{
pub async fn send(self) -> Result<AssumeRoleOutput, SdkError<AssumeRoleError>> where ... {
// Setup code omitted ...
// Make the actual request
self.handle.client.call(op).await
}
}
Smithy clients are constructed from a connector, as shown:
let connector = Builder::new()
.https()
.middleware(...)
.build();
let client = Client::with_config(connector, Config::builder().build());
The https()
method on the Builder constructs the actual Hyper client, and is driven off Cargo features to
select the correct TLS implementation. For example:
#![allow(unused)] fn main() { #[cfg(feature = "rustls")] pub fn https() -> Https { let https = hyper_rustls::HttpsConnector::with_native_roots(); let client = hyper::Client::builder().build::<_, SdkBody>(https); // HyperAdapter is a Tower `Service` request -> response connector that just calls the Hyper client crate::hyper_impls::HyperAdapter::from(client) } }
Solving the Connector Creation Problem
Customers need to be able to provide HTTP settings, such as timeouts, for all connectors that the clients use.
These should come out of the SharedConfig
when it is used. Connector creation also needs to be customizable
so that alternate HTTP implementations can be used, or so that a fake implementation can be used for tests.
To accomplish this, SharedConfig
will have a make_connector
member. A customer would configure
it as such:
let config = some_shared_config_loader()
.with_http_settings(my_http_settings)
.with_make_connector(|reqs: &MakeConnectorRequirements| {
Some(MyCustomConnector::new(reqs))
})
.await;
The passed in MakeConnectorRequirements
will hold the customer-provided HttpSettings
as well
as any Smithy-modeled requirements, which will just be HttpVersion
for now. The MakeConnectorRequirements
struct will be marked non_exhaustive
so that new requirements can be added to it as the SDK evolves.
A default make_connector
implementation would be provided that creates a Hyper connector based on the
Cargo feature flags. This might look something like this:
#![allow(unused)] fn main() { #[cfg(feature = "rustls")] pub fn default_connector(reqs: &HttpRequirements) -> HyperAdapter { let https = hyper_rustls::HttpsConnector::with_native_roots(); let mut builder = hyper::Client::builder(); builder = configure_settings(builder, &reqs.http_settings); if let Http2 = &reqs.http_version { builder = builder.http2_only(true); } HyperAdapter::from(builder.build::<_, SdkBody>(https)) } }
For any given service, make_connector
could be called multiple times to create connectors
for all required HTTP versions and settings.
Note: the make_connector
returns an Option
since an HTTP version may not be required, but rather, preferred
according to a Smithy model. For operations that list out ["h2", "HTTP/1.1"]
as the desired versions,
a customer could choose to provide only an HTTP 1 connector, and the operation should still succeed.
Solving the Connector Selection Problem
Each service operation needs to be able to select a connector that meets its requirements best from the customer provided connectors. Initially, the only selection criteria will be the HTTP version, but later when per-operation HTTP settings are implemented, the connector will also need to be keyed off of those settings. Since connector creation is not a cheap process, connectors will need to be cached after they are created.
This caching is currently handled by the Handle
in the fluent client, which holds on to the
Smithy client. This cache needs to be adjusted to:
- Support multiple connectors, keyed off of the customer provided
HttpSettings
, and also off of the Smithy modeled requirements. - Be lazy initialized. Services that have a mix of Event Stream and non-streaming operations shouldn't create an HTTP/2 client if the customer doesn't intend to use the Event Stream operations that require it.
To accomplish this, the Handle
will hold a cache that is optimized for many reads and few writes:
#[derive(Debug, Hash, Eq, PartialEq)]
struct ConnectorKey {
http_settings: HttpSettings,
http_version: HttpVersion,
}
struct Handle<C, M, R> {
clients: RwLock<HashMap<HttpRequirements<'static>, aws_smithy_client::Client<C, M, R>>>,
conf: crate::Config,
}
pub struct Client<C, M, R = Standard> {
handle: Arc<Handle<C, M, R>>,
}
With how the generics are organized, the connector type will have to be the same between HTTP implementations,
but this should be fine since it is generally a thin wrapper around a separate HTTP implementor.
For cases where it is not, the custom connector type can host its own dyn Trait
solution.
The HttpRequirements
struct will hold HttpSettings
as copy-on-write so that it can be used
for cache lookup without having to clone HttpSettings
:
struct HttpRequirements<'a> {
http_settings: Cow<'a, HttpSettings>,
http_version: HttpVersion,
}
impl<'a> HttpRequirements<'a> {
// Needed for converting a borrowed HttpRequirements into an owned cache key for cache population
pub fn into_owned(self) -> HttpRequirements<'static> {
Self {
http_settings: Cow::Owned(self.http_settings.into_owned()),
http_version: self.http_version,
}
}
}
With the cache established, each operation needs to be aware of its requirements. The code generator will be
updated to store a prioritized list of HttpVersion
in the property bag in an input's make_operation()
method.
This prioritized list will come from the Smithy protocol trait's http
or eventStreamHttp
attribute, depending
on the operation. The fluent client will then pull this list out of the property bag so that it can determine which
connector to use. This indirection is necessary so that an operation still holds all information
needed to make a service call from the Smithy client directly.
Note: This may be extended in the future to be more than just HttpVersion
, for example, when per-operation
HTTP setting overrides are implemented. This doc is not attempting to solve that problem.
In the fluent client, this will look as follows:
impl<C, M, R> AssumeRole<C, M, R> where ... {
pub async fn send(self) -> Result<AssumeRoleOutput, SdkError<AssumeRoleError>> where ... {
let input = self.create_input()?;
let op = input.make_operation(&self.handle.conf)?;
// Grab the `make_connector` implementation
let make_connector = self.config.make_connector();
// Acquire the prioritized HttpVersion list
let http_versions = op.properties().get::<HttpVersionList>();
// Make the actual request (using default HttpSettings until modifying those is implemented)
let client = self.handle
.get_or_create_client(make_connector, &default_http_settings(), &http_versions)
.await?;
client.call(op).await
}
}
If an operation requires a specific protocol version, and if the make_connection
implementation can't
provide that it, then the get_or_create_client()
function will return SdkError::ConstructionFailure
indicating the error.
Changes Checklist
-
Create
HttpVersion
inaws-smithy-http
withHttp1_1
andHttp2
-
Refactor existing
https()
connector creation functions to takeHttpVersion
-
Add
make_connector
toSharedConfig
, and wire up thehttps()
functions as a default -
Create
HttpRequirements
inaws-smithy-http
-
Implement the connector cache on
Handle
- Implement function to calculate a minimum required set of HTTP versions from a Smithy model in the code generator
-
Update the
make_operation
code gen to put anHttpVersionList
into the operation property bag -
Update the fluent client
send()
function code gen grab the HTTP version list and acquire the correct connector with it -
Add required defaulting for models that don't set the optional
http
andeventStreamHttp
protocol trait attributes