Skip to content
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Commit b1f5bef

Browse files
sebstosebsto
committedDec 25, 2024·
Plugin v2 proposal - initial commit
1 parent 38cc01a commit b1f5bef

File tree

1 file changed

+443
-0
lines changed

1 file changed

+443
-0
lines changed
 

‎Plugins/Documentation.docc/Proposals/0001-v2-plugins.md

Lines changed: 443 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,443 @@
1+
# v2 Plugin Proposal for swift-aws-lambda-runtime
2+
3+
`swift-aws-lambda-runtime` is an important library for the Swift on Server ecosystem. The initial versions of the library focused on the API, enabling developers to write Lambda functions in the Swift programming language. The library provided developers with basic support for building and packaging their functions.
4+
5+
We believe it is time to consider the end-to-end developer experience, from project scaffolding to deployment, taking into account the needs of Swift developers new to AWS and Lambda.
6+
7+
This document describes the proposal for the v2 plugins for `swift-aws-lambda-runtime`. The plugins will focus on project scaffolding, building, archiving, and deployment of Lambda functions.
8+
9+
## Overview
10+
11+
Versions:
12+
13+
* v1 (2024-12-25): Initial version
14+
15+
## Motivation
16+
17+
The current version of `swift-aws-lambda-runtime` provides a solid foundation for Swift developers to write Lambda functions. However, the developer experience can be improved. For example, the current version does not provide any support for project scaffolding or deployment of Lambda functions.
18+
19+
This creates a high barrier to entry for Swift developers new to AWS and Lambda, as well as for AWS professionals learning Swift. We propose to lower this barrier by providing a set of plugins that will assist developers in creating, building, packaging, and deploying Lambda functions.
20+
21+
As a source of inspiration, we looked to the Rust community, which created Cargo-Lambda ([https://www.cargo-lambda.info/guide/what-is-cargo-lambda.html](https://www.cargo-lambda.info/guide/what-is-cargo-lambda.html)). Cargo-Lambda helps developers deploy Rust Lambda functions. We aim to provide a similar experience for Swift developers.
22+
23+
### Current Limitations
24+
25+
The current version of `swift-aws-lambda-runtime` does not provide any support for project scaffolding or deployment of Lambda functions. This makes it difficult for Swift developers new to AWS and Lambda, or AWS Professionals new to Swift, to get started.
26+
27+
The main limitations of the current version of the `archive` plugin are as follows:
28+
29+
* It only handles cross-compilation using Docker.
30+
* It only supports archiving of the Lambda function as a ZIP file.
31+
32+
### New Plugins
33+
34+
To address the limitations of the `archive` plugin, we propose creating three new plugins:
35+
36+
* `lambda-init`: This plugin will assist developers in creating a new Lambda project from scratch by scaffolding the project structure and its dependencies.
37+
38+
* `lambda-build`: This plugin will help developers build and package their Lambda function (similar to the current `archive` plugin). This plugin will allow for multiple cross-compilation options. We will retain the current Docker-based cross-compilation but will also provide a way to cross-compile without Docker, such as using the [Swift Static Linux SDK](https://www.swift.org/documentation/articles/static-linux-getting-started.html) (with musl) or a (non-existent at the time of this writing) custom Swift SDK for Amazon Linux (built with the [Custom SDK Generator](https://github.com/swiftlang/swift-sdk-generator)). The plugin will also provide an option to package the binary as a ZIP file or as a Docker image.
39+
40+
* `lambda-deploy`: This plugin will assist developers in deploying their Lambda function to AWS. This plugin will handle the deployment of the Lambda function, including the creation of the IAM role, the creation of the Lambda function, and the optional configuration of a Lambda function URL.
41+
42+
We may consider additional plugins in a future release. For example, we could consider a plugin to help developers invoke their Lambda function (`lambda-invoke`) or to monitor CloudWatch logs (`lambda-logs`).
43+
44+
## Detailed Solution
45+
46+
### Dependencies
47+
48+
One of the design objective of the Swift AWS Lambda Runtime is to minimize its dependencies on other libraries. We will strive to keep the dependencies of the plugins to a minimum.
49+
50+
### Create a New Project (lambda-init)
51+
52+
The `lambda-init` plugin will assist developers in creating a new Lambda project from scratch. The plugin will scaffold the project code. It will create a ready-to-deploy `main.swift` file containing a simple Lambda function. The plugin will allow users to choose from a selection of basic templates, such as a simple "Hello World" Lambda function or a function invoked by a URL.
53+
54+
The plugin cannot be invoked without the required dependencies installed in `Package.swift`. The process of creating a new project will consist of three steps and four commands, all executable from the command line:
55+
56+
```bash
57+
# Step 1: Create a new Swift executable package
58+
swift package init --type executable --name MyLambda
59+
60+
# Step 2: Add the Swift AWS Lambda Runtime dependency
61+
swift package add-dependency https://github.com/swift-server/swift-aws-lambda-runtime.git --branch main
62+
swift package add-target-dependency AWSLambdaRuntime MyLambda --package swift-aws-lambda-runtime
63+
64+
# Step 3: Call the lambda-init plugin
65+
swift package lambda-init
66+
```
67+
68+
The plugin will offer the following options:
69+
70+
```text
71+
OVERVIEW: A SwiftPM plugin to scaffold a Hello World Lambda function.
72+
73+
By default, it creates a Lambda function that receives a JSON document and responds with another JSON document.
74+
75+
USAGE: swift package lambda-init
76+
[--help] [--verbose]
77+
[--with-url]
78+
[--allow-writing-to-package-directory]
79+
80+
OPTIONS:
81+
--with-url Create a Lambda function exposed by a URL
82+
--allow-writing-to-package-directory Don't ask for permission to write files.
83+
--verbose Produce verbose output for debugging.
84+
--help Show help information.
85+
```
86+
87+
The initial implementation will use hardcoded templates. In a future release, we might consider fetching the templates from a GitHub repository and allowing developers to create custom templates.
88+
89+
### Build and Package (lambda-build)
90+
91+
The `lambda-build` plugin will assist developers in building and packaging their Lambda function. It will allow for multiple cross-compilation options. We will retain the current Docker-based cross-compilation but also provide a way to cross-compile without Docker, such as using the Swift Static Linux SDK (with musl) or a custom Swift SDK for Amazon Linux.
92+
93+
We also propose to automatically strip the binary of debug symbols to reduce the size of the ZIP file. Our tests showed that this can reduce the size by up to 50%. An option to disable stripping will be provided.
94+
95+
The `lambda-build` plugin is similar to the existing `archive` plugin. We propose to keep the same interface to facilitate migration of existing projects. If technically feasible, we will also consider keeping the `archive` plugin as an alias to the `lambda-build` plugin.
96+
97+
The plugin interface is based on the existing `archive` plugin, with the addition of the `--no-strip` and `--cross-compile` options:
98+
99+
```text
100+
OVERVIEW: A SwiftPM plugin to build and package your Lambda function.
101+
102+
REQUIREMENTS: To use this plugin, Docker must be installed and running.
103+
104+
USAGE: swift package --allow-network-connections docker archive
105+
[--help] [--verbose]
106+
[--output-directory <path>]
107+
[--products <list of products>]
108+
[--configuration debug | release]
109+
[--swift-version <version>]
110+
[--base-docker-image <docker_image_name>]
111+
[--disable-docker-image-update]
112+
113+
114+
OPTIONS:
115+
--output-directory <path> The path of the binary package.
116+
(default: .build/plugins/AWSLambdaPackager/outputs/...)
117+
--products <list> The list of executable targets to build.
118+
(default: taken from Package.swift)
119+
--configuration <name> The build configuration (debug or release)
120+
(default: release)
121+
--swift-version <version> The Swift version to use for building.
122+
(default: latest)
123+
This parameter cannot be used with --base-docker-image.
124+
--base-docker-image <name> The name of the base Docker image to use for the build.
125+
(default: swift-<version>:amazonlinux2)
126+
This parameter cannot be used with --swift-version.
127+
This parameter cannot be used with a value other than Docker provided to --cross-compile.
128+
--disable-docker-image
129+
```
130+
131+
### Deploy (lambda-deploy)
132+
133+
The `lambda-deploy` plugin will assist developers in deploying their Lambda function to AWS. It will handle the deployment process, including creating the IAM role, the Lambda function itself, and optionally configuring a Lambda function URL.
134+
135+
The plugin will not require any additional dependencies. It will interact with the AWS REST API directly, without using the AWS SDK fro Swift or Soto. Users will need to provide AWS access key and secret access key credentials. The plugin will attempt to locate these credentials in standard locations, such as environment variables or the `~/.aws/credentials` file.
136+
137+
The plugin supports deployment through either the REST and Base64 payload or by uploading the code to a temporary S3 bucket. Refer to [the `Function Code` section](https://docs.aws.amazon.com/lambda/latest/api/API_FunctionCode.html) of the [CreateFunction](https://docs.aws.amazon.com/lambda/latest/dg/API_CreateFunction.html) API for more details.
138+
139+
The plugin can deploy to multiple regions. Users can specify the desired region as a command-line argument.
140+
141+
In addition to deploying the Lambda function, the plugin can also create an IAM role for it. Users can specify the IAM role name as a command-line argument. If no role name is provided, the plugin will create a new IAM role with the necessary permissions for the Lambda function.
142+
143+
The plugin allows developers to update the code for an existing Lambda function. The update command remains the same as for initial deployment. The plugin will detect whether the function already exists and update the code accordingly.
144+
145+
Finally, the plugin can help developers delete a Lambda function and its associated IAM role.
146+
147+
An initial version of this plugin might look like this:
148+
149+
```text
150+
"""
151+
OVERVIEW: A SwiftPM plugin to deploy a Lambda function.
152+
153+
USAGE: swift package lambda-deploy
154+
[--with-url]
155+
[--help] [--verbose]
156+
157+
OPTIONS:
158+
--region The AWS region to deploy the Lambda function to.
159+
(default is us-east-1)
160+
--iam-role The name of the IAM role to use for the Lambda function.
161+
when none is provided, a new role will be created.
162+
--with-url Add an URL to access the Lambda function
163+
--delete Delete the Lambda function and its associated IAM role
164+
--verbose Produce verbose output for debugging.
165+
--help Show help information.
166+
"""
167+
```
168+
169+
### Dependencies
170+
171+
Minimizing dependencies is a key priority for the plugins. We aim to avoid including unnecessary dependencies, such as the AWS SDK for Swift or Soto, for the `lambda-deploy` plugin.
172+
173+
Four essential dependencies have been identified for the plugins:
174+
175+
* an argument parser
176+
* an HTTP client
177+
* a library to sign AWS requests
178+
* a library to calculate HMAC-SHA256 (used in the AWS signing process)
179+
180+
These functionalities can be incorporated by vending source code from other projects. We will consider the following options:
181+
182+
**Argument Parser:**
183+
184+
* We will leverage the `ArgumentExtractor` from the `swift-package-manager` project ([https://github.com/swiftlang/swift-package-manager/blob/main/Sources/PackagePlugin/ArgumentExtractor.swift](https://github.com/swiftlang/swift-package-manager/blob/main/Sources/PackagePlugin/ArgumentExtractor.swift)). This is a simple argument parser used by the Swift Package Manager. The relevant files will be copied into the plugin.
185+
186+
**HTTP Client:**
187+
188+
* We will utilize the `URLSession` provided by `FoundationNetworking`. No additional dependencies will be introduced for the HTTP client.
189+
190+
**AWS Request Signing:**
191+
192+
* To interact with the AWS REST API, requests must be signed. We will include the `AWSRequestSigner` from [the `aws-signer-v4` project](https://github.com/adam-fowler/aws-signer-v4). This is a simple library that signs requests using AWS Signature Version 4. The relevant files will be copied into the plugin.
193+
194+
**HMAC-SHA256 Implementation:**
195+
196+
* The `AWSRequestSigner` has a dependency on the `swift-crypto` library. We will consider two options:
197+
* Include the HMAC-SHA256 implementation from the popular `CryptoSwift` library ([https://github.com/krzyzanowskim/CryptoSwift](https://github.com/krzyzanowskim/CryptoSwift)), which provides a wide range of cryptographic functions. The relevant files will be copied into the plugin.
198+
* Develop a clean implementation of the HMAC-SHA256 algorithm. This is a relatively simple algorithm used for request signing.
199+
200+
The dependencies will be vendored within the plugin and will not be listed as dependencies in the `Package.swift` file.
201+
202+
The following files will be copied into the plugin, without modifications from their original projects:
203+
204+
```text
205+
Sources/AWSLambdaPluginHelper/Vendored
206+
├── crypto
207+
│ ├── Array+Extensions.swift
208+
│ ├── Authenticator.swift
209+
│ ├── BatchedCollections.swift
210+
│ ├── Bit.swift
211+
│ ├── Collections+Extensions.swift
212+
│ ├── Digest.swift
213+
│ ├── DigestType.swift
214+
│ ├── Generics.swift
215+
│ ├── HMAC.swift
216+
│ ├── Int+Extension.swift
217+
│ ├── NoPadding.swift
218+
│ ├── Padding.swift
219+
│ ├── SHA1.swift
220+
│ ├── SHA2.swift
221+
│ ├── SHA3.swift
222+
│ ├── UInt16+Extension.swift
223+
│ ├── UInt32+Extension.swift
224+
│ ├── UInt64+Extension.swift
225+
│ ├── UInt8+Extension.swift
226+
│ ├── Updatable.swift
227+
│ ├── Utils.swift
228+
│ └── ZeroPadding.swift
229+
├── signer
230+
│ ├── AWSCredentials.swift
231+
│ └── AWSSigner.swift
232+
└── spm
233+
└── ArgumentExtractor.swift
234+
```
235+
236+
### Implementation
237+
238+
SwiftPM plugin can not share code within sources files or using a shared Library target. The recommended way to share code between plugins is to create an executable target to implement the plugin functionalities and to implement the plugin as a thin wrapper that invokes the executable target.
239+
240+
We propose to add an executable target and three plugins to the `Package.swift` file of the `swift-aws-lambda-runtime` package.
241+
242+
```swift
243+
let package = Package(
244+
name: "swift-aws-lambda-runtime",
245+
platforms: [.macOS(.v15)],
246+
products: [
247+
248+
//
249+
// The runtime library targets
250+
//
251+
252+
// this library exports `AWSLambdaRuntimeCore` and adds Foundation convenience methods
253+
.library(name: "AWSLambdaRuntime", targets: ["AWSLambdaRuntime"]),
254+
255+
// this has all the main functionality for lambda and it does not link Foundation
256+
.library(name: "AWSLambdaRuntimeCore", targets: ["AWSLambdaRuntimeCore"]),
257+
258+
//
259+
// The plugins
260+
// 'lambda-init' creates a new Lambda function
261+
// 'lambda-build' packages the Lambda function
262+
// 'lambda-deploy' deploys the Lambda function
263+
//
264+
// Plugins requires Linux or at least macOS v15
265+
//
266+
267+
// plugin to create a new Lambda function, based on a template
268+
.plugin(name: "AWSLambdaInitializer", targets: ["AWSLambdaInitializer"]),
269+
270+
// plugin to package the lambda, creating an archive that can be uploaded to AWS
271+
.plugin(name: "AWSLambdaBuilder", targets: ["AWSLambdaBuilder"]),
272+
273+
// plugin to deploy a Lambda function
274+
.plugin(name: "AWSLambdaDeployer", targets: ["AWSLambdaDeployer"]),
275+
276+
//
277+
// Testing targets
278+
//
279+
// ... unchanged ...
280+
],
281+
dependencies: [ // unchanged
282+
.package(url: "https://github.com/apple/swift-nio.git", from: "2.76.0"),
283+
.package(url: "https://github.com/apple/swift-log.git", from: "1.5.4"),
284+
],
285+
targets: [
286+
287+
// library target, unchanged
288+
// ....
289+
290+
//
291+
// The plugins targets
292+
//
293+
.plugin(
294+
name: "AWSLambdaInitializer",
295+
capability: .command(
296+
intent: .custom(
297+
verb: "lambda-init",
298+
description:
299+
"Create a new Lambda function in the current project directory."
300+
),
301+
permissions: [
302+
.writeToPackageDirectory(reason: "Create a file with an HelloWorld Lambda function.")
303+
]
304+
),
305+
dependencies: [
306+
.target(name: "AWSLambdaPluginHelper")
307+
]
308+
),
309+
// keep this one (with "archive") to not break workflows
310+
// This will be deprecated at some point in the future
311+
// .plugin(
312+
// name: "AWSLambdaPackager",
313+
// capability: .command(
314+
// intent: .custom(
315+
// verb: "archive",
316+
// description:
317+
// "Archive the Lambda binary and prepare it for uploading to AWS. Requires docker on macOS or non Amazonlinux 2 distributions."
318+
// ),
319+
// permissions: [
320+
// .allowNetworkConnections(
321+
// scope: .docker,
322+
// reason: "This plugin uses Docker to create the AWS Lambda ZIP package."
323+
// )
324+
// ]
325+
// ),
326+
// path: "Plugins/AWSLambdaBuilder" // same sources as the new "lambda-build" plugin
327+
// ),
328+
.plugin(
329+
name: "AWSLambdaBuilder",
330+
capability: .command(
331+
intent: .custom(
332+
verb: "lambda-build",
333+
description:
334+
"Archive the Lambda binary and prepare it for uploading to AWS. Requires docker on macOS or non Amazonlinux 2 distributions."
335+
),
336+
permissions: [
337+
.allowNetworkConnections(
338+
scope: .docker,
339+
reason: "This plugin uses Docker to create the AWS Lambda ZIP package."
340+
)
341+
]
342+
),
343+
dependencies: [
344+
.target(name: "AWSLambdaPluginHelper")
345+
]
346+
),
347+
.plugin(
348+
name: "AWSLambdaDeployer",
349+
capability: .command(
350+
intent: .custom(
351+
verb: "lambda-deploy",
352+
description:
353+
"Deploy the Lambda function. You must have an AWS account and know an access key and secret access key."
354+
),
355+
permissions: [
356+
.allowNetworkConnections(
357+
scope: .all(ports: [443]),
358+
reason: "This plugin uses the AWS Lambda API to deploy the function."
359+
)
360+
]
361+
),
362+
dependencies: [
363+
.target(name: "AWSLambdaPluginHelper")
364+
]
365+
),
366+
367+
/// The executable target that implements the three plugins functionality
368+
.executableTarget(
369+
name: "AWSLambdaPluginHelper",
370+
dependencies: [
371+
.product(name: "NIOHTTP1", package: "swift-nio"),
372+
.product(name: "NIOCore", package: "swift-nio"),
373+
],
374+
swiftSettings: [.swiftLanguageMode(.v6)]
375+
),
376+
377+
// remaining targets, unchanged
378+
]
379+
)
380+
381+
```
382+
383+
A plugin would be a thin wrapper around the executable target. For example:
384+
385+
```swift
386+
struct AWSLambdaInitializer: CommandPlugin {
387+
388+
func performCommand(context: PackagePlugin.PluginContext, arguments: [String]) async throws {
389+
let tool = try context.tool(named: "AWSLambdaPluginHelper")
390+
391+
let args = ["init", "--dest-dir", context.package.directoryURL.path()] + arguments
392+
393+
// Invoke the plugin helper on the target directory, passing a configuration
394+
// file from the package directory.
395+
let process = try Process.run(tool.url, arguments: args)
396+
process.waitUntilExit()
397+
398+
// Check whether the subprocess invocation was successful.
399+
if !(process.terminationReason == .exit && process.terminationStatus == 0) {
400+
let problem = "\(process.terminationReason):\(process.terminationStatus)"
401+
Diagnostics.error("AWSLambdaPluginHelper invocation failed: \(problem)")
402+
}
403+
}
404+
}
405+
```
406+
407+
And the executable target would dispatch the invocation to a struct implementing the actual functionality of the plugin:
408+
409+
```swift
410+
public static func main() async throws {
411+
let args = CommandLine.arguments
412+
let helper = AWSLambdaPluginHelper()
413+
let command = try helper.command(from: args)
414+
switch command {
415+
case .`init`:
416+
try await Initializer().initialize(arguments: args)
417+
case .build:
418+
try await Builder().build(arguments: args)
419+
case .deploy:
420+
try await Deployer().deploy(arguments: args)
421+
}
422+
}
423+
```
424+
425+
## Considered Alternatives
426+
427+
In addition to the proposed solution, we evaluated the following alternatives:
428+
429+
1. **VSCode Extension for Project Scaffolding:**
430+
431+
We considered using a VSCode extension, such as the `vscode-aws-lambda-swift-sam` extension ([https://github.com/swift-server-community/vscode-aws-lambda-swift-sam](https://github.com/swift-server-community/vscode-aws-lambda-swift-sam)), to scaffold new Lambda projects.
432+
433+
This extension creates a new Lambda project from scratch, including the project structure and dependencies. It provides a ready-to-deploy `main.swift` file with a simple Lambda function and allows users to choose from basic templates, such as "Hello World" or an OpenAPI-based function. However, the extension relies on the AWS CLI and SAM CLI for deployment. It is only available in the Visual Studio Code Marketplace.
434+
435+
While the extension offers a user-friendly graphical interface, it does not align well with our goals of simplicity for first-time users and minimal dependencies. Users would need to install VSCode, the extension itself, the AWS CLI, and the SAM CLI before getting started.
436+
437+
2. **Deployment DSL with AWS SAM:**
438+
439+
We also considered using a domain-specific language (DSL) to describe deployments, such as the `swift-aws-lambda-sam-dsl` project ([https://github.com/swift-server-community/swift-aws-lambda-sam-dsl](https://github.com/swift-server-community/swift-aws-lambda-sam-dsl)), and leveraging AWS SAM for the actual deployment.
440+
441+
This plugin allows developers to describe their deployment using Swift code, and the plugin automatically generates the corresponding SAM template. However, the plugin depends on the SAM CLI for deployment. Additionally, new developers would need to learn a new DSL for deployment configuration.
442+
443+
We believe the `lambda-deploy` plugin is a preferable alternative because it interacts directly with the AWS REST API and avoids introducing additional dependencies for the user.

0 commit comments

Comments
 (0)
Please sign in to comment.