Skip to content

docs: HelloWorldStreamFunction in examples fails with sam #1532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 67 commits into from
Dec 5, 2023
Merged
Show file tree
Hide file tree
Changes from 50 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
e66095e
Setting up Kotlin environment. Converting test to Kotlin.
Oct 2, 2023
ea0b496
Deploying via SAM successfully.
Oct 2, 2023
615a095
Added Kotlin example.
Oct 3, 2023
5199105
Removing unused Gradle build file.
Oct 3, 2023
37020f3
Merge branch 'main' into main
scottgerring Oct 3, 2023
6386358
Adding SAM template so can be used as an existing project and Java ta…
Oct 4, 2023
19679da
Adding SAM template so can be used as an existing project
Oct 4, 2023
17c3a5e
Updating guidance to use SAM for build and deploy
Oct 4, 2023
1312c02
Restructuring separate Java and Kotlin examples.
Oct 4, 2023
18c7f48
Updating core examples readme to represent new structure for Java and…
Oct 4, 2023
d968f3c
Refactoring application code for efficiency, updating build to cover …
Oct 4, 2023
5ce71f7
Updating to fix trailing \n
Oct 4, 2023
35fbbac
Updating guidance to be more specific for examples
Oct 4, 2023
6ffa295
Merge branch 'main' into main
scottgerring Oct 5, 2023
d277c5c
Adopting new mechanism for specifying jvm target.
Oct 6, 2023
6c4646e
accommodating new project structure
Oct 6, 2023
6eecb90
Fixing link typo after refactoring
Oct 6, 2023
66a3dc2
Merge branch 'main' into main
scottgerring Oct 8, 2023
1431899
Setting up Kotlin environment. Converting test to Kotlin.
Oct 2, 2023
3396766
Deploying via SAM successfully.
Oct 2, 2023
baa0e3f
Added Kotlin example.
Oct 3, 2023
16380a8
Removing unused Gradle build file.
Oct 3, 2023
3f3f4e8
Adding SAM template so can be used as an existing project and Java ta…
Oct 4, 2023
9d70b57
Adding SAM template so can be used as an existing project
Oct 4, 2023
bcff275
Updating guidance to use SAM for build and deploy
Oct 4, 2023
84935c9
Restructuring separate Java and Kotlin examples.
Oct 4, 2023
bf6da0d
Updating core examples readme to represent new structure for Java and…
Oct 4, 2023
f67f59a
Refactoring application code for efficiency, updating build to cover …
Oct 4, 2023
f7288d5
Updating to fix trailing \n
Oct 4, 2023
f03b01b
Updating guidance to be more specific for examples
Oct 4, 2023
e523554
Adopting new mechanism for specifying jvm target.
Oct 6, 2023
a71e263
accommodating new project structure
Oct 6, 2023
51fc239
Fixing link typo after refactoring
Oct 6, 2023
6cef711
Flattening structure back to original to make merging easier for v2
Oct 13, 2023
2444ba4
Merge remote-tracking branch 'origin/main'
Oct 13, 2023
c24ee6d
Adding build for Kotlin Gradle
Oct 13, 2023
de19b56
Adding build for Kotlin Gradle - Restructuring Java examples to v1 ap…
Oct 13, 2023
6e23b26
Correcting paths
Oct 13, 2023
480dd8b
Adding SNAPSHOT support and local capability for Maven. Testing using…
Oct 13, 2023
7a8dc16
Reviewed and updated against PR comments.
Oct 20, 2023
de08cbc
Merge branch 'main' into main
scottgerring Oct 23, 2023
bf843a7
Merging with Terraform additions from main branch
Oct 23, 2023
9a154dd
Un-commenting examples
Oct 23, 2023
e3c1cda
Un-commenting examples
Oct 31, 2023
e8ddb45
Adding validation step for IaC SAM
Oct 31, 2023
ec17480
Adding Terraform for Java projects IaC validator and linter
Nov 1, 2023
7d11fac
Adding additional projects for SAM validation and matrix approach
Nov 1, 2023
e932d10
Refactoring stream function to process input logging example with a L…
Nov 17, 2023
ab31b09
Demonstrating Java streaming response
Nov 23, 2023
d4bed89
Refactoring stream function to process input logging example to return
Nov 28, 2023
4322ce9
Update CONTRIBUTING.md
jeromevdl Nov 8, 2023
494d73f
fix: get trace id from system property when env var is not set (#1503)
mriccia Nov 8, 2023
e6e2eb0
fix #1500 (#1506)
jeromevdl Nov 8, 2023
7c51a38
feat: Add support for POWERTOOLS_LOGGER_LOG_EVENT (#1510)
AlexeySoshin Nov 14, 2023
f814d4d
chore: Addition of Warn Message If Invalid Annotation Key While Traci…
jdoherty Nov 14, 2023
9a3a788
feat: ALC (#1514)
jeromevdl Nov 16, 2023
7fc6ff2
chore:Prep release 1.18.0 (#1515)
jeromevdl Nov 16, 2023
6635b50
chore: update version to next snapshot: 1-19.0-SNAPSHOT (#1516)
jeromevdl Nov 16, 2023
9d8b614
Add some more margin to the test pause (#1518)
scottgerring Nov 17, 2023
223d196
test: e2e tests with java 21 (#1517)
jeromevdl Nov 17, 2023
eeaec59
update doc for ALC (#1520)
jeromevdl Nov 20, 2023
150b28f
chore: Testing java21 aspectj pre-release (#1519)
scottgerring Nov 20, 2023
2411a07
chore: Remove build cruft
scottgerring Nov 22, 2023
772b30e
Adding context for using RequestStreamHandler
Nov 29, 2023
c69e3dc
replacing current main version of pr_build
Nov 29, 2023
2259c66
removing pr_lint
Nov 29, 2023
080edc0
Update examples/powertools-examples-core/sam/src/main/java/helloworld…
jasoniharris Dec 4, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 0 additions & 21 deletions .github/workflows/pr_build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,33 +79,12 @@ jobs:
if: ${{ matrix.java == '8' }} # Gradle example can only be built on Java 8
working-directory: examples/powertools-examples-core/kotlin
run: ./gradlew build
- name: Setup Terraform
if: ${{ matrix.java == '11' }}
uses: hashicorp/setup-terraform@633666f66e0061ca3b725c73b2ec20cd13a8fdd1 #v2.0.3
- name: Setup AWS credentials
if: ${{ matrix.java == '11' }}
uses: aws-actions/configure-aws-credentials@5fd3084fc36e372ff1fff382a39b10d03659f355 # v2.2.0
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN_TO_ASSUME }}
aws-region: ${{ env.AWS_REGION }}
- name: Terraform validate
working-directory: examples/powertools-examples-core/terraform
if: ${{ matrix.java == '11' }}
run: |
terraform -version
terraform init -backend=false
terraform validate
terraform plan
- name: Setup Terraform lint
if: ${{ matrix.java == '11' }}
uses: terraform-linters/setup-tflint@a5a1af8c6551fb10c53f1cd4ba62359f1973746f # v3.1.1
- name: Terraform lint
working-directory: examples/powertools-examples-core/terraform
if: ${{ matrix.java == '11' }}
run: |
tflint --version
tflint --init
tflint -f compact
- name: Upload coverage to Codecov
uses: codecov/codecov-action@d9f34f8cd5cb3b3eb79b3e4b5dae3a16df499a70 # v3.1.1
if: ${{ matrix.java == '11' }} # publish results once
Expand Down
45 changes: 45 additions & 0 deletions .github/workflows/pr_iac_lint_.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Validate IaC

on:
pull_request:
branches:
- main
- v2
paths:
- 'examples/**'
jobs:
linter:
runs-on: ubuntu-latest
strategy:
matrix:
project: ["sam", "gradle", "kotlin"]
steps:
- uses: actions/checkout@c85c95e3d7251135ab7dc9ce3241c5835cc595a9 # v3.5.3
- name: Setup java JDK
uses: actions/setup-java@5ffc13f4174014e2d4d4572b3d74c3fa61aeb2c2 # v3.11.0
with:
distribution: 'corretto'
java-version: 11
- name: Run SAM validator to check syntax of IaC templates - Java
working-directory: examples/powertools-examples-core/${{ matrix.project }}
run: |
sam build
sam validate --lint
- name: Setup Terraform
uses: hashicorp/setup-terraform@633666f66e0061ca3b725c73b2ec20cd13a8fdd1 #v2.0.3
- name: Run Terraform validator to check syntax of IaC templates and produce a plan of changes
working-directory: examples/powertools-examples-core/terraform
run: |
mvn install
terraform -version
terraform init -backend=false
terraform validate
terraform plan
- name: Setup Terraform lint
uses: terraform-linters/setup-tflint@a5a1af8c6551fb10c53f1cd4ba62359f1973746f # v3.1.1
- name: Run Terraform lint to check for best practices, errors, deprecated syntax etc.
working-directory: examples/powertools-examples-core/terraform
run: |
tflint --version
tflint --init
tflint -f compact
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,40 @@
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestStreamHandler;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Map;
import java.nio.charset.StandardCharsets;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import software.amazon.lambda.powertools.logging.Logging;
import software.amazon.lambda.powertools.metrics.Metrics;

import java.io.InputStreamReader;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.OutputStreamWriter;
import java.io.PrintWriter;

public class AppStream implements RequestStreamHandler {
private static final ObjectMapper mapper = new ObjectMapper();
private final static Logger log = LogManager.getLogger(AppStream.class);

@Override
@Logging(logEvent = true)
@Metrics(namespace = "ServerlessAirline", service = "payment", captureColdStart = true)
public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException {
Map map = mapper.readValue(input, Map.class);
public void handleRequest(InputStream input, OutputStream output, Context context) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be helpful to add some info here about why you would use RequestStreamHandler - e.g. to make it clear that it has nothing to do with HTTP streaming.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note added to explain why RequestStreamHandler is preferred here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's not quite right - I think the use case is more typically "we need to control [de]serialization ourselves".

I find "rather than a simple HTTP request-response model" a little confusing, because you still have exactly the same surrounding HTTP interaction as you would with a 'normal' RequestHandler, you're just choosing to read-and-write the Body using java streams rather than letting Lambda handle the serialization for you.

@jeromevdl I don't know if you have some super clear wording handy? The core Lambda documentation isn't that prescriptive.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree wording is not clear or even wrong. RequestStreamHandler is not linked to Lambda streaming which was announced recently. It is used when you want to use your own serialization: https://docs.aws.amazon.com/lambda/latest/dg/java-handler.html#java-handler-interfaces. I agree the name of the interface is not super well chosen but it has nothing to do with streams...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to rephrase the wording to better reflect what this handler actually does

try (BufferedReader reader = new BufferedReader(new InputStreamReader(input, StandardCharsets.UTF_8));
PrintWriter writer = new PrintWriter(new BufferedWriter(new OutputStreamWriter(output, StandardCharsets.UTF_8)))) {

System.out.println(map.size());
log.info("Received: " + mapper.writerWithDefaultPrettyPrinter().writeValueAsString(mapper.readTree(reader)));

writer.write("{\"body\": \"" + System.currentTimeMillis() + "\"} ");
} catch (IOException e) {
log.error("Something has gone wrong: ", e);
}
}
}