RFC: Independent Crate Versioning
Status: RFC
During its alpha and dev preview releases, the AWS SDK for Rust adopted a short-term solution for versioning and publishing to crates.io. This doc proposes a long-term versioning strategy that will carry the SDK from dev preview into general availability.
This strategy will be implemented in two phases:
- Dev Preview: The SDK will break with its current version strategy
of maintaining consistent
major.minor
version numbers. - Stability and 1.x: This phase begins when the SDK becomes generally available. The major version will be bumped to 1, and backwards breaking changes will no longer be allowed without a major version bump to all crates in the SDK.
Terminology
- AWS SDK Crate: A crate that provides a client for calling a given AWS service, such as
aws-sdk-s3
for calling S3. - AWS Runtime Crate: Any runtime crate that the AWS SDK generated code relies on, such as
aws-types
. - Smithy Runtime Crate: Any runtime crate that the smithy-rs generated code relies on, such as
smithy-types
.
Requirements
Versioning
Cargo uses semver for versioning, with a major.minor.patch-pre
format:
major
: Incompatible API changesminor
: Added functionality in backwards compatible mannerpatch
: Backwards compatible bug fixespre
: Pre-release version tag (omitted for normal releases)
In the new versioning strategy, the minor
version number will no longer be coordinated across
all SDK and Smithy runtime crates.
During phases 1 and 2, the major
version will always be 0, and the following scheme will be used:
minor
:- New features
- Breaking changes
- Dependency updates for dependencies that are part of the public API
- Model updates with API changes
- For code-generated crates: when a newer version of smithy-rs is used to generate the crate
patch
:- Bug fixes that do not break backwards compatibility
- Model updates that only have documentation changes
During phase 3:
major
: Breaking changesminor
:- Changes that aren't breaking
- Dependency updates for dependencies that are part of the public API
- Model updates with API changes
- For code-generated crates: when a newer version of smithy-rs is used to generate the crate
patch
:- Bug fixes that do not break backwards compatibility
- Model updates that only have documentation changes
During phase 3, bumps to the major
version must be coordinated across all SDK and runtime crates.
Release Identification
Since there will no longer be one SDK "version", release tags will be dates in YYYY-MM-DD
format
rather than version numbers. Additionally, the SDK's user agent string will need to include a separate
service version number (this requirement has already been implemented).
Yanking
It must be possible to yank an entire release with a single action. The publisher tool must be updated to understand which crate versions were released with a given release tag, and be able to yank all the crates published from that tag.
Phase 1: Dev Preview
Phase 1 will address the following challenges introduced by uncoordinating the major.minor
versions:
- Tracking of versions associated with a release tag
- Creation of version bump process for code generated crates
- Enforcement of version bump process in runtime crates
- Yanking of versions associated with a release tag
Version Tracking
A new manifest file will be introduced in the root of aws-sdk-rust named versions.toml
that describes
all versioning information for any given commit in the repository. In the main branch, the versions.toml
in tagged commits will become the source of truth for which crate versions belong to that release, as well
as additional metadata that's required for maintaining version process in the future.
The special 0.0.0-smithy-rs-head
version that is used prior to Phase 1 for maintaining the runtime crate
versions will no longer be used (as detailed in Versioning for Runtime Crates).
This format will look as follows:
smithy_rs_version = "<release-tag|commit-hash>"
[aws-smithy-types]
version = "0.50.1"
[aws-config]
version = "0.40.0"
[aws-sdk-s3]
version = "0.89.0"
model_hash = "<hash>"
# ...
The auto-sync tool is responsible for maintaining this file. When it generates a new SDK, it will take the version numbers from runtime crates directly, and it will use the rules from the next section to determine the version numbers for the generated crates.
Versioning for Code Generated (SDK Service) Crates
Code generated crates will have their minor
version bumped when the version of smithy-rs used to generate
them changes, or when model updates with API changes are made. Three pieces of information are required to
handle this process: the previously released version number, the smithy-rs version used to generate the code,
and the level of model updates being applied. For this last one, if there are multiple model updates that
affect only documentation, but then one model update that affects an API, then as a whole they will be
considered as affecting an API and require a minor
version bump.
The previously released version number will be retrieved from crates.io using its API. The smithy-rs version
used during code generation will become a build artifact that is saved to versions.toml
in aws-sdk-rust.
During phase 1, the tooling required to know if a model is a documentation-only change will not be available,
so all model changes will result in a minor
version bump during this phase.
Overall, determining a generated crate's version number looks as follows:
flowchart TD start[Generate crate version] --> smithyrschanged{A. smithy-rs changed?} smithyrschanged -- Yes --> minor1[Minor version bump] smithyrschanged -- No --> modelchanged{B. model changed?} modelchanged -- Yes --> minor2[Minor version bump] modelchanged -- No --> keep[Keep current version]
- A: smithy-rs changed?: Compare the
smithy_rs_version
in the previousversions.toml
with the nextversions.toml
file, and if the values are different, consider smithy-rs to have changed. - B: model changed?: Similarly, compare the
model_hash
for the crate inversions.toml
.
Versioning for Runtime Crates
The old scheme of all runtime crates in smithy-rs having a fake 0.0.0-smithy-rs-head
version number with
a build step to replace those with a consistent major.minor
will be removed. These runtime crates will begin
having their actual next version number in the Cargo.toml file in smithy-rs.
This introduces a new problem where a developer can forget to bump a runtime crate version, so a method of
process enforcement needs to be introduced. This will be done through CI when merging into smithy-rs/main
and repeated when merging into aws-sdk-rust/main
.
The following checks need to be run for runtime crates:
flowchart TD A[Check runtime crate] --> B{A. Crate has changed?} B -- Yes --> C{B. Minor bumped?} B -- No --> H{C. Version changed?} C -- Yes --> K[Pass] C -- No --> E{D. Patch bumped?} E -- Yes --> F{E. Semverver passes?} E -- No --> L[Fail] F -- Yes --> D[Pass] F -- No --> G[Fail] H -- Yes --> I[Fail] H -- No --> J[Pass]
- A: Crate has changed? The crate's source files and manifest will be hashed for the previous version and the next version. If these hashes match, then the crate is considered unchanged.
- B: Minor bumped? The previous version is compared against the next version to see if the minor version number was bumped.
- C: Version changed? The previous version is compared against the next version to see if it changed.
- D: Patch bumped? The previous version is compared against the next version to see if the patch version number was bumped.
- E: Semverver passes? Runs rust-semverver against the old and new versions of the crate.
- If semverver fails to run (for example, if it needs to be updated to the latest nightly to succeed), then fail CI saying that either semverver needs maintenance, or that a minor version bump is required.
- If semverver results in errors, fail CI indicating a minor version bump is required.
- If semverver passes, then pass CI.
When running semverver, the path dependencies of the crate under examination should be updated to be crates.io references if there were no changes in those crates since the last public to crates.io. Otherwise, the types referenced from those crates in the public API will always result in breaking changes since, as far as the Rust compiler is concerned, they are different types originating from separate path-dependency crates.
For CI, the aws-sdk-rust/main
branch's versions.toml
file is the source of truth for the previous release's
crate versions and source code.
Yanking
The publisher tool will be updated to read the versions.toml
to yank all versions published in a release.
This process will look as follows:
- Take a path to a local clone of the aws-sdk-rust repository
- Confirm the working tree is currently unmodified and on a release tag.
- Read
versions.toml
and print out summary of crates to yank - Confirm with user before proceeding
- Yank crates
Changes Checklist
-
Update
rust-semverver
to a newer nightly that can compileaws-smithy-client
-
Establish initial
versions.toml
inaws-sdk-rust/main
- Set version numbers in runtime crates in smithy-rs
-
Update the auto-sync tool to generate
versions.toml
-
Create CI tool to check runtime crate version
-
Integrate with
smithy-rs/main
CI -
Integrate with
aws-sdk-rust/main
CI
-
Integrate with
-
Update CI to verify no older runtime crates are used. For example, if
aws-smithy-client
is bumped to0.50.0
, then verify no crates (generated or runtime) depend on0.49.0
or lower.
Estimate: 2-4 dev weeks
Phase 2: Stability and 1.x
When stabilizing to 1.x, the version process will stay the same, but the minor version bumps caused by version bumping runtime crates, updating models, or changing the code generator will be candidate for automatic upgrade per semver. At that point, no further API breaking changes can be made without a major version bump.