docs(readme): highlight high level concepts (#2205)

alexforsyth · trivikr · web-flow · commit 39011c8266ee · 2021-04-15T14:04:13.000-04:00
* fix: add table of contents to top-level readme

* chore: add high level concept details

* chore: high level concepts go at the bottom

* Update README.md

Co-authored-by: Trivikram Kamat &lt;16024985+trivikr@users.noreply.github.com&gt;

* Update README.md

Co-authored-by: Trivikram Kamat &lt;16024985+trivikr@users.noreply.github.com&gt;

Co-authored-by: Trivikram Kamat &lt;16024985+trivikr@users.noreply.github.com&gt;
diff --git a/README.md b/README.md
@@ -26,7 +26,24 @@ and provides step-by-step migration instructions to v3.
 To test your universal JavaScript code in Node.js, browser and react-native environments,
 visit our [code samples repo](https://github.com/aws-samples/aws-sdk-js-tests).
 
-## Getting started
+# Table of Contents
+
+1. [Getting Started](#getting-started)
+1. [New Features](#new-features)
+   1. [Modularized packages](#modularized-packages)
+   1. [API consistency changes](#api-changes)
+      1. [Configuration](#configuration)
+      1. [Middleware Stack](#Middleware)
+   1. [How to upgrade](#other-changes)
+1. [High Level Concepts in V3](#high-level-concepts)
+   1. [Generated Packages](#generated-code)
+   1. [Paginators](#paginators)
+   1. [Abort Controller](#abort-controller)
+   1. [Middleware Stack](#middleware-stack)
+1. [Install from Source](#install-from-Source)
+1. [Giving feedback and contributing](#giving-feedback-and-contributing)
+
+## Getting Started
 
 Let’s walk through setting up a project that depends on DynamoDB from the SDK and makes a simple service call. The following steps use yarn as an example. These steps assume you have Node.js and yarn already installed.
 
@@ -215,3 +232,235 @@ track the discussion daily, so feel free to open a GitHub issue if your question
 You can open pull requests for fixes or additions to the new AWS SDK for JavaScript v3.
 All pull requests must be submitted under the Apache 2.0 license and will be reviewed by an SDK team member prior to merging.
 Accompanying unit tests are appreciated. See [Contributing](CONTRIBUTING.md) for more information.
+
+## High Level Concepts
+
+This is an introduction to some of the high level concepts behind AWS SDK for JavaScript (v3)
+which are shared between services and might make your life easier. Please consult the user
+guide and API reference for service specific details.
+
+#### Terminology:
+
+**Bare-bones clients/commands**: This refers to a modular way of consuming individual operations on JS SDK clients. It results in less code being imported and thus more performant. It is otherwise equivlent to the aggregated clients/commands.
+
+```
+// this imports a bare-bones version of S3 that exposes the .send operation
+import { S3Client } from "@aws-sdk/client-s3"
+
+// this imports just the getObject operation from S3
+import { GetObjectCommand } from "@aws-sdk/client-s3"
+
+//usage
+const bareBonesS3 = new S3Client({...});
+await bareBonesS3.send(new GetObjectCommand({...}));
+```
+
+**Aggregated clients/commands**: This refers to a way of consuming clients that contain all operations on them. Under the hood this calls the bare-bones commands. This imports all commands on a particular client and results in more code being imported and thus less performant. This is 1:1 with v2's style.
+
+```
+// this imports an aggregated version of S3 that exposes the .send operation
+import { S3 } from "@aws-sdk/client-s3"
+
+// No need to import an operation as all operations are already on the S3 prototype
+
+//usage
+const aggregatedS3 = new S3({...});
+await aggregatedS3.getObject({...}));
+```
+
+### Generated Code
+
+The v3 codebase is generated from internal AWS models that AWS services expose. We use [smithy-typescript](https://github.com/awslabs/smithy-typescript) to generate all code in the `/clients` subdirectory. These packages always have a prefix of `@aws-sdk/client-XXXX` and are one-to-one with AWS services and service operations. You should be importing `@aws-sdk/client-XXXX` for most usage.
+
+Clients depend on common "utility" code in `/packages`. The code in `/packages` is manually written and outside of special cases (like credenitials or abort controller) is generally not very useful alone.
+
+Lastly we have higher level libraries in `/lib`. These are javascript specific libraries that wrap client operations to make them easier to work with. Popular examples are `@aws-sdk/lib-dynamodb` which [simplifies working with items in Amazon DynamoDB](https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-dynamodb/README.md) or `@aws-sdk/lib-storage` which exposes the `Upload` function and [simplifies parallel uploads in S3's multipartUpload](https://github.com/aws/aws-sdk-js-v3/blob/main/lib/lib-storage/README.md).
+
+1. `/packages`. This sub directory is where most manual code updates are done. These are published to NPM under `@aws-sdk/XXXX` and have no special prefix.
+1. `/clients`. This sub directory is code generated and depends on code published from `/packages` . It is 1:1 with AWS services and operations. Manual edits should generally not occur here. These are published to NPM under `@aws-sdk/client-XXXX`.
+1. `/lib`. This sub directory depends on generated code published from `/clients`. It wraps existing AWS services and operations to make them easier to work with in Javascript. These are published to NPM under `@aws-sdk/lib-XXXX`
+
+### Paginators
+
+Many AWS operations return paginated results when the response object is too large to return in a single response. In AWS SDK for JavaScript v2, the response contains a token you can use to retrieve the next page of results. You then need to write additional functions to process pages of results.
+
+In AWS SDK for JavaScript v3 we’ve improved pagination using async generator functions, which are similar to generator functions, with the following differences:
+
+- When called, async generator functions return an object, an async generator whose methods (`next`, `throw`, and `return`) return promises for { `value`, `done` }, instead of directly returning { `value`, `done` }. This automatically makes the returned async generator objects async iterators.
+- await expressions and `for await (x of y)` statements are allowed.
+- The behavior of `yield*` is modified to support delegation to async iterables.
+
+The Async Iterators were added in the ES2018 iteration of JavaScript. They are supported by Node.js 10.x+ and by all modern browsers, including Chrome 63+, Firefox 57+, Safari 11.1+, and Edge 79+. If you’re using TypeScript v2.3+, you can compile Async Iterators to older versions of JavaScript.
+
+An async iterator is much like an iterator, except that its `next()` method returns a promise for a { `value`, `done` } pair. As an implicit aspect of the Async Iteration protocol, the next promise is not requested until the previous one resolves. This is a simple, yet a very powerful pattern.
+
+#### Example Pagination Usage
+
+In v3, the clients expose paginateOperationName APIs that are written using async generators, allowing you to use async iterators in a for await..of loop. You can perform the paginateListTables operation from `@aws-sdk/client-dynamodb` as follows:
+
+```
+const {
+  DynamoDBClient,
+  paginateListTables,
+} = require("@aws-sdk/client-dynamodb");
+
+...
+const paginatorConfig = {
+  client: new DynamoDBClient({}),
+  pageSize: 25
+};
+const commandParams = {};
+const paginator = paginateListTables(paginatorConfig, commandParams);
+
+const tableNames = [];
+for await (const page of paginator) {
+  // page contains a single paginated output.
+  tableNames.push(...page.TableNames);
+}
+...
+
+```
+
+Or simplified:
+
+```
+...
+const client = new DynamoDBClient({});
+
+const tableNames = [];
+for await (const page of paginateListTables({ client }, {})) {
+    // page contains a single paginated output.
+    tableNames.push(...page.TableNames);
+}
+...
+```
+
+### Abort Controller
+
+In v3, we support the AbortController interface which allows you to abort requests as and when desired.
+
+The [AbortController Interface](https://dom.spec.whatwg.org/#interface-abortcontroller) provides an `abort()` method that toggles the state of a corresponding AbortSignal object. Most APIs accept an AbortSignal object, and respond to `abort()` by rejecting any unsettled promise with an “AbortError”.
+
+```
+// Returns a new controller whose signal is set to a newly created AbortSignal object.
+const controller = new AbortController();
+
+// Returns the AbortSignal object associated with controller.
+const signal = controller.signal;
+
+// Invoking this method will set controller’s AbortSignal's aborted flag
+// and signal to any observers that the associated activity is to be aborted.
+controller.abort();
+```
+
+#### AbortController Usage
+
+In JavaScript SDK v3, we added an implementation of WHATWG AbortController interface in `@aws-sdk/abort-controller`. To use it, you need to send `AbortController.signal` as `abortSignal` in the httpOptions parameter when calling `.send()` operation on the client as follows:
+
+```
+const { AbortController } = require("@aws-sdk/abort-controller");
+const { S3Client, CreateBucketCommand } = require("@aws-sdk/client-s3");
+
+...
+
+const abortController = new AbortController();
+const client = new S3Client(clientParams);
+
+const requestPromise = client.send(new CreateBucketCommand(commandParams), {
+  abortSignal: abortController.signal,
+});
+
+// The abortController can be aborted any time.
+// The request will not be created if abortSignal is already aborted.
+// The request will be destroyed if abortSignal is aborted before response is returned.
+abortController.abort();
+
+// This will fail with "AbortError" as abortSignal is aborted.
+await requestPromise;
+```
+
+For a full pagination deep dive please check out our [blog post](https://aws.amazon.com/blogs/developer/pagination-using-async-iterators-in-modular-aws-sdk-for-javascript/).
+
+#### AbortController Example
+
+The following code snippet shows how to upload a file using S3's putObject API in the browser with support to abort the upload. First, create a controller using the `AbortController()` constructor, then grab a reference to its associated AbortSignal object using the AbortController.signal property. When the `PutObjectCommand` is called with `.send()` operation, pass in AbortController.signal as abortSignal in the httpOptions parameter. This will allow you to abort the PutObject operation by calling `abortController.abort()`.
+
+```#JavaScript
+const abortController = new AbortController();
+const abortSignal = abortController.signal;
+
+const uploadBtn = document.querySelector('.upload');
+const abortBtn = document.querySelector('.abort');
+
+uploadBtn.addEventListener('click', uploadObject);
+
+abortBtn.addEventListener('click', function() {
+  abortController.abort();
+  console.log('Upload aborted');
+});
+
+const uploadObject = async (file) => {
+  ...
+  const client = new S3Client(clientParams);
+  try {
+    await client.send(new PutObjectCommand(commandParams), { abortSignal });
+  } catch(e) {
+    if (e.name === "AbortError") {
+      uploadProgress.textContent = 'Upload aborted: ' + e.message;
+    }
+    ...
+  }
+}
+```
+
+For a full abort controller deep dive please check out our [blog post](https://aws.amazon.com/blogs/developer/abortcontroller-in-modular-aws-sdk-for-javascript/).
+
+### Middleware Stack
+
+The JavaScript SDK maintains a series of asynchronous actions. These series include actions that serialize input parameters into the data over the wire and deserialize response data into JavaScript objects. Such actions are implemented using functions called middleware and executed in a specific order. The object that hosts all the middleware including the ordering information is called a Middleware Stack. You can add your custom actions to the SDK and/or remove the default ones.
+
+When an API call is made, SDK sorts the middleware according to the step it belongs to and its priority within each step. The input parameters pass through each middleware. An HTTP request gets created and updated along the process. The HTTP Handler sends a request to the service, and receives a response. A response object is passed back through the same middleware stack in reverse, and is deserialized into a JavaScript object.
+
+A middleware is a higher-order function that transfers user input and/or HTTP request, then delegates to “next” middleware. It also transfers the result from “next” middleware. A middleware function also has access to context parameter, which optionally contains data to be shared across middleware.
+
+For example, you can use middleware to add a custom header like S3 object metadata:
+
+```
+const { S3 } = require("@aws-sdk/client-s3");
+const client = new S3({ region: "us-west-2" });
+// Middleware added to client, applies to all commands.
+client.middlewareStack.add(
+  (next, context) => async (args) => {
+    args.request.headers["x-amz-meta-foo"] = "bar";
+    const result = next(args);
+    // result.response contains data returned from next middleware.
+    return result;
+  },
+  {
+    step: "build",
+    name: "addFooMetadataMiddleware",
+    tags: ["METADATA", "FOO"],
+  }
+);
+
+await client.putObject(params);
+```
+
+Specifying the absolute location of your middleware
+The example above adds middleware to `build` step of middleware stack. The middleware stack contains five steps to manage a request’s lifecycle:
+
+- The **initialize** lifecycle step initializes an API call. This step typically adds default input values to a command. The HTTP request has not yet been constructed.
+- The **serialize** lifecycle step constructs an HTTP request for the API call. Example of typical serialization tasks include input validation and building an HTTP request from user input. The downstream middleware will have access to serialized HTTP request object in callback’s parameter `args.request`.
+- The **build** lifecycle step builds on top of serialized HTTP request. Examples of typical build tasks include injecting HTTP headers that describe a stable aspect of the request, such as `Content-Length` or a body checksum. Any request alterations will be applied to all retries.
+- The **finalizeRequest** lifecycle step prepares the request to be sent over the wire. The request in this stage is semantically complete and should therefore only be altered to match the recipient’s expectations. Examples of typical finalization tasks include request signing, performing retries and injecting hop-by-hop headers.
+- The **deserialize** lifecycle step deserializes the raw response object to a structured response. The upstream middleware have access to deserialized data in next callbacks return value: `result.output`.
+  Each middleware must be added to a specific step. By default each middleware in the same step has undifferentiated order. In some cases, you might want to execute a middleware before or after another middleware in the same step. You can achieve it by specifying its `priority`.
+
+```
+client.middlewareStack.add(middleware, {
+  step: "initialize",
+  priority: "high", // or "low".
+});
+```
+
+For a full middleware stack deep dive please check out our [blog post](https://aws.amazon.com/blogs/developer/middleware-stack-modular-aws-sdk-js/).
