RFC: API for Presigned URLs
Status: Implemented
For a summarized list of proposed changes, see the Changes Checklist section.
Several AWS services allow for presigned requests in URL form, which is described well by S3's documentation on authenticating requests using query parameters.
This doc establishes the customer-facing API for creating these presigned URLs and how they will be implemented in a generic fashion in the SDK codegen.
Terminology
To differentiate between the clients that are present in the generated SDK today, the following terms will be used throughout this doc:
- Smithy Client: A
aws_smithy_client::Client<C, M, R>
struct that is responsible for gluing together the connector, middleware, and retry policy. This is not generated and lives in theaws-smithy-client
crate. - Fluent Client: A code-generated
Client<C, M, R>
that has methods for each service operation on it. A fluent builder is generated alongside it to make construction easier.
Presigned URL config
Today, presigned URLs take an expiration time that's not part of the service API. The SDK will make this configurable as a separate struct so that there's no chance of name collisions, and so that additional fields can be added in the future. Fields added later will require defaulting for backwards compatibility.
Customers should also be able to set a start time on the presigned URL's expiration so that they can
generate URLs that become active in the future. An optional start_time
option will be available and
default to SystemTime::now()
.
Construction PresigningConfig
can be done with a builder, but a PresigningConfig::expires_in
convenience function will be provided to bypass the builder for the most frequent use-case.
#[non_exhaustive]
#[derive(Debug, Clone)]
pub struct PresigningConfig {
start_time: SystemTime,
expires_in: Duration,
}
#[non_exhaustive]
#[derive(Debug)]
pub struct Builder {
start_time: Option<SystemTime>,
expires_in: Option<Duration>,
}
impl Builder {
pub fn start_time(self, start_time: SystemTime) -> Self { ... }
pub fn set_start_time(&mut self, start_time: Option<SystemTime>) { ... }
pub fn expires_in(self, expires_in: Duration) -> Self { ... }
pub fn set_expires_in(&mut self, expires_in: Option<Duration>) { ... }
// Validates `expires_in` is no greater than one week
pub fn build(self) -> Result<PresigningConfig, Error> { ... }
}
impl PresigningConfig {
pub fn expires_in(expires_in: Duration) -> PresigningConfig {
Self::builder().expires(expires).build().unwrap()
}
pub fn builder() -> Builder { ... }
}
Construction of PresigningConfig
will validate that expires_in
is no greater than one week, as this
is the longest supported expiration time for SigV4. This validation will result in a panic.
It's not inconceivable that PresigningConfig
will need additional service-specific parameters as customizations,
so it will be code generated with each service rather than living a shared location.
Fluent Presigned URL API
The generated fluent builders for operations that support presigning will have a presigned()
method
in addition to send()
that will return a presigned URL rather than sending the request. For S3's GetObject,
the usage of this will look as follows:
let config = aws_config::load_config_from_environment().await;
let client = s3::Client::new(&config);
let presigning_config = PresigningConfig::expires_in(Duration::from_secs(86400));
let presigned: PresignedRequest = client.get_object()
.bucket("example-bucket")
.key("example-object")
.presigned(presigning_config)
.await?;
This API requires a client, and for use-cases where no actual service calls need to be made, customers should be able to create presigned URLs without the overhead of an HTTP client. Once the HTTP Versions RFC is implemented, the underlying HTTP client won't be created until the first service call, so there will be no HTTP client overhead to this approach.
In a step away from the general pattern of keeping fluent client capabilities in line with Smithy client capabilities, creating presigned URLs directly from the Smithy client will not be supported. This is for two reasons:
- The Smithy client is not code generated, so adding a method to do presigning would apply to all operations, but not all operations can be presigned.
- Presigned URLs are not currently a Smithy concept (although this may change soon).
The result of calling presigned()
is a PresignedRequest
, which is a wrapper with delegating functions
around http::Request<()>
so that the request method and additional signing headers are also made available.
This is necessary since there are some presignable POST operations that require the signature to be in the
headers rather than the query.
Note: Presigning needs to be async
because the underlying credentials provider used to sign the
request may need to make service calls to acquire the credentials.
Input Presigned URL API
Even though generating a presigned URL through the fluent client doesn't necessitate an HTTP client, it will be clearer that this is the case by allowing the creation of presigned URLs directly from an input. This would look as follows:
let config = aws_config::load_config_from_environment().await;
let presigning_config = PresigningConfig::expires_in(Duration::from_secs(86400));
let presigned: PresignedRequest = GetObjectInput::builder()
.bucket("example-bucket")
.key("example-bucket")
.presigned(&config, presigning_config)
.await?;
Creating the URL through the input will exercise the same code path as creating it through the client, but it will be more apparent that the overhead of a client isn't present.
Behind the scenes
From an SDK's perspective, the following are required to make a presigned URL:
- Valid request input
- Endpoint
- Credentials to sign with
- Signing implementation
The AWS middleware provides everything except the request, and the request is provided as part
of the fluent builder API. The generated code needs to be able to run the middleware to fully populate
a request property bag, but not actually dispatch it. The expires_in
value from the presigning config
needs to be piped all the way through to the signer. Additionally, the SigV4 signing needs to adjusted
to do query param signing, which is slightly different than its header signing.
Today, request dispatch looks as follows:
- The customer creates a new fluent builder by calling
client.operation_name()
, fills in inputs, and then callssend()
. send()
:- Builds the final input struct, and then calls its
make_operation()
method with the stored config to create a SmithyOperation
. - Calls the underlying Smithy client with the operation.
- Builds the final input struct, and then calls its
- The Smithy client constructs a Tower Service with AWS middleware and a dispatcher at the bottom, and then executes it.
- The middleware acquire and add required signing parameters (region, credentials, endpoint, etc) to the request property bag.
- The SigV4 signing middleware signs the request by adding HTTP headers to it.
- The dispatcher makes the actual HTTP request and returns the response all the way back up the Tower.
Presigning will take advantage of a lot of these same steps, but will cut out the Operation
and
replace the dispatcher with a presigned URL generator:
- The customer creates a new fluent builder by calling
client.operation_name()
, fills in inputs, and then callspresigned()
. presigned()
:- Builds the final input struct, calls the
make_operation()
method with the stored config, and then extracts the request from the operation (discarding the rest). - Mutates the
OperationSigningConfig
in the property bag to:- Change the
signature_type
toHttpRequestQueryParams
so that the signer runs the correct signing logic. - Set
expires_in
to the value given by the customer in the presigning config.
- Change the
- Constructs a Tower Service with
AwsMiddleware
layered in, and aPresignedUrlGeneratorLayer
at the bottom. - Calls the Tower Service and returns its result
- Builds the final input struct, calls the
- The
AwsMiddleware
will sign the request. - The
PresignedUrlGeneratorLayer
directly returns the request since all of the work is done by the middleware.
It should be noted that the presigned()
function above is on the generated input struct, so implementing this for
the input API is identical to implementing it for the fluent client.
All the code for the new make_request()
is already in the existing make_operation()
and will just need to be split out.
Modeling Presigning
AWS models don't currently have any information about which operations can be presigned.
To work around this, the Rust SDK will create a synthetic trait to model presigning with, and
apply this trait to known presigned operations via customization. The code generator will
look for this synthetic trait when creating the fluent builders and inputs to know if a
presigned()
method should be added.
Avoiding name collision
If a presignable operation input has a member named presigned
, then there will be a name collision with
the function to generate a presigned URL. To mitigate this, RustReservedWords
will be updated
to rename the presigned
member to presigned_value
similar to how send
is renamed.
Changes Checklist
-
Update
aws-sigv4
to support query param signing -
Create
PresignedOperationSyntheticTrait
- Customize models for known presigned operations
-
Create
PresigningConfig
and its builder -
Implement
PresignedUrlGeneratorLayer
-
Create new AWS codegen decorator to:
-
Add new
presigned()
method to input code generator -
Add new
presigned()
method to fluent client generator
-
Add new
-
Update
RustReservedWords
to reservepresigned()
- Add integration test to S3
- Add integration test to Polly
-
Add examples for using presigning for:
- S3 GetObject and PutObject
- Polly SynthesizeSpeech