Copy-free signing of Netty FullHttpRequest for lambda

[fabienrenaud]

I work on a gateway-type application built in Netty which proxies http requests to aws lambda. The gateway app signs the inbound http request using the awssdk 2.x before sending to lambda with a Netty http client.

Here is the code snippet:

//
// 1. inbound FullHttpRequest. content is a Netty ByteBuf
//
ByteBuf content = request.content();


//
// 2. Convert Netty ByteBuf to a type InvokeRequest accepts
//

byte[] arr;
if (buffer.hasArray()) {
  arr = buffer.array();
} else {
  arr = new byte[buffer.readableBytes()];
  buffer.getBytes(buffer.readerIndex(), arr);
}
SdkBytes sdkBytes = SdkBytes.fromByteArray(arr); // always copy :(
buffer.release(); // fromByteArray always copy => release ByteBuf


//
// 3. sign request with AWS SDK v2
//

InvokeRequest invokeRequest =
    InvokeRequest.builder()
        .functionName(functionName)
        .invocationType(InvocationType.REQUEST_RESPONSE)
        .qualifier(qualifier)
        .payload(sdkBytes)
        .build();

AwsJsonProtocolFactory protocolFactory =
    AwsJsonProtocolFactory.builder()
        .clientConfiguration(
            SdkClientConfiguration.builder()
                .option(SdkClientOption.ENDPOINT, endpointUri)
                .build())
        .build();

InvokeRequestMarshaller invokeRequestMarshaller = new InvokeRequestMarshaller(protocolFactory);
SdkHttpFullRequest marshalledRequest = invokeRequestMarshaller.marshall(invokeRequest);

// looks thread-safe. TODO: get Amazon to officially confirm & document
Aws4Signer aws4Signer = Aws4Signer.create();
SdkHttpFullRequest signedRequest =
    aws4Signer.sign(
        marshalledRequest,
        Aws4SignerParams.builder()
            .awsCredentials(CREDENTIALS)
            .signingName("lambda")
            .signingRegion(AWS_REGION)
            .timeOffset(0)
            .build());


//
// 3. convert AWS signed request back into a Netty FullHttpRequest
// Another piece of work here that requires copying the signed request InputStream into a Netty ByteBuf
//

FullHttpRequest req = toNettyRequest(signedRequest);
/* 
 * private FullHttpRequest toNettyRequest(SdkHttpFullRequest awsReq) throws IOException {
 *    ByteBuf content;
 *    Optional<ContentStreamProvider> contentStreamProvider = awsReq.contentStreamProvider();
 *    // FIXME: copy InputStream directly into pooled ByteBuf
 *    if (contentStreamProvider.isPresent()) {
 *      byte[] contentBytes = IoUtils.toByteArray(contentStreamProvider.get().newStream());
 *      content = Unpooled.wrappedBuffer(contentBytes);
 *    } else {
 *      content = Unpooled.buffer(0);
 *    }
 *    ... etc.
 */

// 4. send signed FullHttpRequest to lambda with Netty client
myNettyClient.send(req);

With the current AWS SDK API, this implementation requires doing several copies of the request content ByteBuf/payload:

1. First to turn the Netty ByteBuf into a byte array
2. SdkBytes copies the byte array again
3. SdkBytes parent class (BytesWrapper) exposes methods that suggest more copies may happen depending on implementation of AWS SDK classes... (perhaps ContentStreamProvider is a copy of SdkBytes... I haven't checked...)
4. ContentStreamProvider's InputStream is copied in chunks to a byte[] in IoUtils.toByteArray.
5. The ByteArrayOutputStream (in IoUtils.toByteArray) gets resized (which causes copies) as more space is needed. InputStream.available() method is not used due to lack of clarity from the Java+AWS API (will it always return the full size of the underlying stream in this case?)

Request: I'd like this entire flow to be copy free and be able to reuse my allocated pooled Netty ByteBufs.

If InvokeRequest could accept an InputStream instead of a SdkBytes for its payload method, this would already be a great help as I would be able to wrap the Netty ByteBuf in a ByteBufInputStream. Underlying implementation of the request marshaller and signer would also need to be copy-free.

Alternatively, if I could sign the Netty request content without needing to create a SdkHttpFullRequest and just get the additional headers I need to add to the Netty request, this would make this entire flow simpler and more efficient.

[fabienrenaud]

Work around is to custom build the SdkHttpFullRequest:

    ByteBuf content = ...;

    String uri;
    String path = "/2015-03-31/functions/" + functionName + "/invocations";
    var awsRequestBuilder =
        SdkHttpFullRequest.builder()
            .protocol(protocol)
            .host(host)
            .method(SdkHttpMethod.POST)
            .encodedPath(path)
            .putHeader("Content-Type", "application/x-amz-json-null")
            .putHeader("X-Amz-Invocation-Type", "RequestResponse")
            .contentStreamProvider(() -> new ByteBufInputStream(content.duplicate())); // bytebuf reused here
    if (qualifier == null) {
      uri = path;
    } else {
      uri = path + "?Qualifier=" + qualifier;
      awsRequestBuilder.putRawQueryParameter("Qualifier", qualifier);
    }
    var awsRequest = awsRequestBuilder.build();

    Region awsRegion = Region.of(region);
    // looks thread-safe. todo: get Amazon to officially confirm & document
    Aws4Signer aws4Signer = Aws4Signer.create();
    SdkHttpFullRequest signedRequest =
        aws4Signer.sign(
            awsRequest,
            Aws4SignerParams.builder()
                .awsCredentials(credentials)
                .signingName("lambda")
                .signingRegion(awsRegion)
                .timeOffset(0)
                .build());

    FullHttpRequest req = new DefaultFullHttpRequest(HTTP_1_1, POST, uri, content);
    HttpHeaders headers = req.headers();
    signedRequest.headers().forEach(headers::set);

Would still be nice to have a more flexible abstraction than SdkBytes to allow reusing ByteBuffer, ByteBuf, etc.