Skip to content

Update s3 java example #391

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Sep 12, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sample-apps/s3-java/5-invoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ if [ ! -f event.json ]; then

fi
while true; do
aws lambda invoke --function-name $FUNCTION --payload file://event.json out.json
aws lambda invoke --function-name $FUNCTION --payload file://event.json out.json --cli-binary-format raw-in-base64-out
cat out.json
echo ""
sleep 2
Expand Down
15 changes: 12 additions & 3 deletions sample-apps/s3-java/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

The project source includes function code and supporting resources:

- `src/main` - A Java function.
- `src/main` - A Java Lambda function that scales down an image stored in S3.
- `src/test` - A unit test and helper classes.
- `template.yml` - An AWS CloudFormation template that creates an application.
- `build.gradle` - A Gradle build file.
Expand Down Expand Up @@ -63,10 +63,14 @@ You can also build the application with Maven. To use maven, add `mvn` to the co
...

# Test
To upload an image file to the application bucket and trigger the function, run `4-upload.sh`.
This Lambda function takes an image that's currently stored in S3, and scales it down into
a thumbnail-sized image. To upload an image file to the application bucket, run `4-upload.sh`.

s3-java$ ./4-upload.sh

In your `s3-java-bucket-<random_uuid>` bucket that was created in step 3, you should now see a
key `inbound/sample-s3-java.png` file, which represents the original image.

To invoke the function directly, run `5-invoke.sh`.

s3-java$ ./5-invoke.sh
Expand All @@ -75,7 +79,12 @@ To invoke the function directly, run `5-invoke.sh`.
"ExecutedVersion": "$LATEST"
}

Let the script invoke the function a few times and then press `CRTL+C` to exit.
Let the script invoke the function a few times and then press `CRTL+C` to exit. Note that you
may see function timeouts in the first few iterations due to cold starts; after a while, they
should begin to succeed.

If you look at the `s3-java-bucket-<random_uuid>` bucket in your account, you should now see a
key `resized-inbound/sample-s3-java.png` file, which represents the new, shrunken image.

The application uses AWS X-Ray to trace requests. Open the [X-Ray console](https://console.aws.amazon.com/xray/home#/service-map) to view the service map.

Expand Down
15 changes: 8 additions & 7 deletions sample-apps/s3-java/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,19 @@ repositories {
}

dependencies {
implementation platform('com.amazonaws:aws-xray-recorder-sdk-bom:2.4.0')
implementation platform('software.amazon.awssdk:bom:2.15.0')
implementation platform('com.amazonaws:aws-xray-recorder-sdk-bom:2.11.0')
implementation 'software.amazon.awssdk:s3'
implementation 'com.amazonaws:aws-lambda-java-core:1.2.1'
implementation 'com.amazonaws:aws-lambda-java-events:2.2.9'
implementation 'com.amazonaws:aws-java-sdk-s3:1.11.578'
implementation 'com.amazonaws:aws-lambda-java-events:3.11.0'
implementation 'org.apache.logging.log4j:log4j-api:[2.17.1,)'
implementation 'org.apache.logging.log4j:log4j-core:[2.17.1,)'
implementation 'org.apache.logging.log4j:log4j-slf4j18-impl:[2.17.1,)'
runtimeOnly 'com.amazonaws:aws-lambda-java-log4j2:1.5.1'
implementation 'com.amazonaws:aws-xray-recorder-sdk-core'
implementation 'com.amazonaws:aws-xray-recorder-sdk-aws-sdk'
implementation 'com.amazonaws:aws-xray-recorder-sdk-aws-sdk-instrumentor'
implementation 'com.google.code.gson:gson:2.8.6'
implementation 'org.apache.logging.log4j:log4j-api:[2.17.1,)'
implementation 'org.apache.logging.log4j:log4j-core:[2.17.1,)'
implementation 'org.apache.logging.log4j:log4j-slf4j18-impl:[2.17.1,)'
runtimeOnly 'com.amazonaws:aws-lambda-java-log4j2:1.5.0'
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.6.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.6.0'
}
Expand Down
53 changes: 32 additions & 21 deletions sample-apps/s3-java/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -11,26 +11,45 @@
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>bom</artifactId>
<version>2.16.1</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-xray-recorder-sdk-bom</artifactId>
<version>2.11.0</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3</artifactId>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-core</artifactId>
<version>1.2.0</version>
<version>1.2.1</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-events</artifactId>
<version>2.2.7</version>
<version>3.11.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-lambda-java-log4j2</artifactId>
<version>1.5.0</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
<version>1.5.1</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
Expand All @@ -47,30 +66,22 @@
<artifactId>log4j-slf4j18-impl</artifactId>
<version>[2.17.1,)</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-java-sdk-s3</artifactId>
<version>1.11.578</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-xray-recorder-sdk-core</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-xray-recorder-sdk-aws-sdk-core</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-xray-recorder-sdk-aws-sdk</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>com.amazonaws</groupId>
<artifactId>aws-xray-recorder-sdk-aws-sdk-instrumentor</artifactId>
<version>2.4.0</version>
</dependency>
<dependency>
<groupId>com.google.code.gson</groupId>
<artifactId>gson</artifactId>
<version>2.8.6</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
Expand Down
158 changes: 97 additions & 61 deletions sample-apps/s3-java/src/main/java/example/Handler.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,21 +8,24 @@
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.util.HashMap;
import java.util.Map;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import javax.imageio.ImageIO;

import com.amazonaws.AmazonServiceException;
import software.amazon.awssdk.awscore.exception.AwsServiceException;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.S3Object;
import software.amazon.awssdk.services.s3.S3Client;

import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.AmazonS3;
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;
import com.amazonaws.services.s3.model.GetObjectRequest;
import com.amazonaws.services.s3.model.ObjectMetadata;
import com.amazonaws.services.s3.model.S3Object;
import com.amazonaws.services.s3.AmazonS3ClientBuilder;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.S3EventNotificationRecord;

import com.google.gson.Gson;
import com.google.gson.GsonBuilder;
Expand All @@ -34,12 +37,12 @@
public class Handler implements RequestHandler<S3Event, String> {
Gson gson = new GsonBuilder().setPrettyPrinting().create();
private static final Logger logger = LoggerFactory.getLogger(Handler.class);
private static final float MAX_WIDTH = 100;
private static final float MAX_HEIGHT = 100;
private final String JPG_TYPE = (String) "jpg";
private final String JPG_MIME = (String) "image/jpeg";
private final String PNG_TYPE = (String) "png";
private final String PNG_MIME = (String) "image/png";
private static final float MAX_DIMENSION = 100;
private final String REGEX = ".*\\.([^\\.]*)";
private final String JPG_TYPE = "jpg";
private final String JPG_MIME = "image/jpeg";
private final String PNG_TYPE = "png";
private final String PNG_MIME = "image/png";
@Override
public String handleRequest(S3Event s3event, Context context) {
try {
Expand All @@ -55,7 +58,7 @@ public String handleRequest(S3Event s3event, Context context) {
String dstKey = "resized-" + srcKey;

// Infer the image type.
Matcher matcher = Pattern.compile(".*\\.([^\\.]*)").matcher(srcKey);
Matcher matcher = Pattern.compile(REGEX).matcher(srcKey);
if (!matcher.matches()) {
logger.info("Unable to infer image type for key " + srcKey);
return "";
Expand All @@ -67,63 +70,96 @@ public String handleRequest(S3Event s3event, Context context) {
}

// Download the image from S3 into a stream
AmazonS3 s3Client = AmazonS3ClientBuilder.defaultClient();
S3Object s3Object = s3Client.getObject(new GetObjectRequest(
srcBucket, srcKey));
InputStream objectData = s3Object.getObjectContent();

// Read the source image
BufferedImage srcImage = ImageIO.read(objectData);
int srcHeight = srcImage.getHeight();
int srcWidth = srcImage.getWidth();
// Infer the scaling factor to avoid stretching the image
// unnaturally
float scalingFactor = Math.min(MAX_WIDTH / srcWidth, MAX_HEIGHT
/ srcHeight);
int width = (int) (scalingFactor * srcWidth);
int height = (int) (scalingFactor * srcHeight);

BufferedImage resizedImage = new BufferedImage(width, height,
BufferedImage.TYPE_INT_RGB);
Graphics2D g = resizedImage.createGraphics();
// Fill with white before applying semi-transparent (alpha) images
g.setPaint(Color.white);
g.fillRect(0, 0, width, height);
// Simple bilinear resize
g.setRenderingHint(RenderingHints.KEY_INTERPOLATION,
RenderingHints.VALUE_INTERPOLATION_BILINEAR);
g.drawImage(srcImage, 0, 0, width, height, null);
g.dispose();
S3Client s3Client = S3Client.builder().build();
InputStream s3Object = getObject(s3Client, srcBucket, srcKey);

// Read the source image and resize it
BufferedImage srcImage = ImageIO.read(s3Object);
BufferedImage newImage = resizeImage(srcImage);

// Re-encode image to target format
ByteArrayOutputStream os = new ByteArrayOutputStream();
ImageIO.write(resizedImage, imageType, os);
InputStream is = new ByteArrayInputStream(os.toByteArray());
// Set Content-Length and Content-Type
ObjectMetadata meta = new ObjectMetadata();
meta.setContentLength(os.size());
ByteArrayOutputStream outputStream = new ByteArrayOutputStream();
ImageIO.write(newImage, imageType, outputStream);

// Upload new image to S3
putObject(s3Client, outputStream, dstBucket, dstKey, imageType);

logger.info("Successfully resized " + srcBucket + "/"
+ srcKey + " and uploaded to " + dstBucket + "/" + dstKey);
return "Ok";
} catch (IOException e) {
throw new RuntimeException(e);
}
}

private InputStream getObject(S3Client s3Client, String bucket, String key) {
GetObjectRequest getObjectRequest = GetObjectRequest.builder()
.bucket(bucket)
.key(key)
.build();
return s3Client.getObject(getObjectRequest);
}

private void putObject(S3Client s3Client, ByteArrayOutputStream outputStream,
String bucket, String key, String imageType) {
Map<String, String> metadata = new HashMap<>();
metadata.put("Content-Length", Integer.toString(outputStream.size()));
if (JPG_TYPE.equals(imageType)) {
meta.setContentType(JPG_MIME);
}
if (PNG_TYPE.equals(imageType)) {
meta.setContentType(PNG_MIME);
metadata.put("Content-Type", JPG_MIME);
} else if (PNG_TYPE.equals(imageType)) {
metadata.put("Content-Type", PNG_MIME);
}

PutObjectRequest putObjectRequest = PutObjectRequest.builder()
.bucket(bucket)
.key(key)
.metadata(metadata)
.build();

// Uploading to S3 destination bucket
logger.info("Writing to: " + dstBucket + "/" + dstKey);
logger.info("Writing to: " + bucket + "/" + key);
try {
s3Client.putObject(dstBucket, dstKey, is, meta);
s3Client.putObject(putObjectRequest,
RequestBody.fromBytes(outputStream.toByteArray()));
}
catch(AmazonServiceException e)
catch(AwsServiceException e)
{
logger.error(e.getErrorMessage());
logger.error(e.awsErrorDetails().errorMessage());
System.exit(1);
}
logger.info("Successfully resized " + srcBucket + "/"
+ srcKey + " and uploaded to " + dstBucket + "/" + dstKey);
return "Ok";
} catch (IOException e) {
throw new RuntimeException(e);
}
}

/**
* Resizes (shrinks) an image into a small, thumbnail-sized image.
*
* The new image is scaled down proportionally based on the source
* image. The scaling factor is determined based on the value of
* MAX_DIMENSION. The resulting new image has max(height, width)
* = MAX_DIMENSION.
*
* @param srcImage BufferedImage to resize.
* @return New BufferedImage that is scaled down to thumbnail size.
*/
private BufferedImage resizeImage(BufferedImage srcImage) {
int srcHeight = srcImage.getHeight();
int srcWidth = srcImage.getWidth();
// Infer scaling factor to avoid stretching image unnaturally
float scalingFactor = Math.min(
MAX_DIMENSION / srcWidth, MAX_DIMENSION / srcHeight);
int width = (int) (scalingFactor * srcWidth);
int height = (int) (scalingFactor * srcHeight);

BufferedImage resizedImage = new BufferedImage(width, height,
BufferedImage.TYPE_INT_RGB);
Graphics2D graphics = resizedImage.createGraphics();
// Fill with white before applying semi-transparent (alpha) images
graphics.setPaint(Color.white);
graphics.fillRect(0, 0, width, height);
// Simple bilinear resize
graphics.setRenderingHint(RenderingHints.KEY_INTERPOLATION,
RenderingHints.VALUE_INTERPOLATION_BILINEAR);
graphics.drawImage(srcImage, 0, 0, width, height, null);
graphics.dispose();
return resizedImage;
}
}
18 changes: 8 additions & 10 deletions sample-apps/s3-java/src/test/java/example/InvokeTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,14 @@
import com.amazonaws.services.lambda.runtime.LambdaLogger;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import com.amazonaws.services.lambda.runtime.events.S3Event;
import com.amazonaws.services.s3.event.S3EventNotification;
import com.amazonaws.services.s3.event.S3EventNotification.S3EventNotificationRecord;
import com.amazonaws.services.s3.event.S3EventNotification.RequestParametersEntity;
import com.amazonaws.services.s3.event.S3EventNotification.ResponseElementsEntity;
import com.amazonaws.services.s3.event.S3EventNotification.S3Entity;
import com.amazonaws.services.s3.event.S3EventNotification.UserIdentityEntity;
import com.amazonaws.services.s3.event.S3EventNotification.GlacierEventDataEntity;
import com.amazonaws.services.s3.event.S3EventNotification.S3BucketEntity;
import com.amazonaws.services.s3.event.S3EventNotification.S3ObjectEntity;
import com.amazonaws.services.s3.event.S3EventNotification.UserIdentityEntity;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.RequestParametersEntity;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.ResponseElementsEntity;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.S3BucketEntity;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.S3Entity;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.S3EventNotificationRecord;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.S3ObjectEntity;
import com.amazonaws.services.lambda.runtime.events.models.s3.S3EventNotification.UserIdentityEntity;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
Expand Down