S3 batch

[msailes]

Issue #, if available:

#177

Description of changes:

Event and response objects for S3 Batch operations.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[david-kingsbury-bah]

After making the one field name change (result VS results), I was able to successfully run an S3 Batch Operations job that invokes a Lambda written using these models.

[david-kingsbury-bah]

Suggested change

	private List<Result> result;
	private List<Result> results;

[msailes]

fixed.

[david-kingsbury-bah]

Since we have the ResultCode enum now, I'm thinking S3BatchOpResponse.treatMissingKeysAs should be of type ResultCode.

private ResultCode treatMissingKeysAs;

The following are ideas that might go too far for this library; let me know what you think.

We could default treatMissingKeysAs to PermanentFailure, as this is likely the value most people will use.

private ResultCode treatMissingKeysAs = ResultCode.PermanentFailure;

We could have a special constructor to seed a response instance. These must be set for the response to be considered valid. If one does not want to use this, they can use the other constructors.

    public S3BatchResponse(final S3BatchEvent request) {
        this.invocationSchemaVersion = request.getInvocationSchemaVersion();
        this.invocationId = request.getInvocationId();
        this.results = Collections.singletonList(
                Result.builder()
                        .withTaskId(request.getTasks().get(0).getTaskId())
                        .build()
        );
    }

[msailes]

Hi David,

Great feedback! What about something like this in the S3BatchResponse.java class.

public static S3BatchResponseBuilder fromS3BatchEvent(S3BatchEvent s3BatchEvent) {
        return S3BatchResponse.builder()
                .withInvocationId(s3BatchEvent.getInvocationId())
                .withInvocationSchemaVersion(s3BatchEvent.getInvocationSchemaVersion());
}

I think I'd be nervous adding any default behavior which might have a negative impact. I don't know this area of the platform that well, so I would definitely appreciate any feedback.

Thanks,

Mark

[david-kingsbury-bah]

The fromS3BatchEvent method looks great!

I think the question now is should fromS3BatchEvent go as far as my example and create a single Result with the Task ID set.

• S3 Batch Ops expects the response result Task ID to match what was passed in through the request (S3BatchEvent). I suspect this allows S3 Batch Ops to do many concurrent invocations and still be able to gather and sort results.
• S3 Batch Ops only sends 1 task, and expects only 1 result. Here is a snippet from their docs

  # S3 batch operations currently only passes a single task at a time in the array of tasks.
  task = event['tasks'][0]
  

I'm on the fence. The API allows for multiple tasks and results, however it is certainly 1 task per event today.

[msailes]

In my opinion no.

If all that was required was the task id it would make sense. But the user will still have to add their own values for resultCode and resultString. So I don't think there is much to be gained.

[msailes]

I've also tested this now.

[carlzogh]

LGTM!

[carlzogh]

Thanks @msailes and @david-kingsbury-bah! :)