Skip to content

Commit b7891d1

Browse files
author
awstools
committed
feat(client-firehose): This release adds support for Data Message Extraction for decompressed CloudWatch logs, and to use a custom file extension or time zone for S3 destinations.
1 parent 0ec2097 commit b7891d1

15 files changed

+514
-410
lines changed

‎clients/client-firehose/README.md

+2-2
Original file line numberDiff line numberDiff line change
@@ -6,9 +6,9 @@
66

77
AWS SDK for JavaScript Firehose Client for Node.js, Browser and React Native.
88

9-
<fullname>Amazon Kinesis Data Firehose API Reference</fullname>
9+
<fullname>Amazon Data Firehose</fullname>
1010

11-
<p>Amazon Kinesis Data Firehose is a fully managed service that delivers real-time
11+
<p>Amazon Data Firehose is a fully managed service that delivers real-time
1212
streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon
1313
OpenSearch Service, Amazon Redshift, Splunk, and various other supportd
1414
destinations.</p>

‎clients/client-firehose/src/Firehose.ts

+2-2
Original file line numberDiff line numberDiff line change
@@ -274,8 +274,8 @@ export interface Firehose {
274274

275275
/**
276276
* @public
277-
* <fullname>Amazon Kinesis Data Firehose API Reference</fullname>
278-
* <p>Amazon Kinesis Data Firehose is a fully managed service that delivers real-time
277+
* <fullname>Amazon Data Firehose</fullname>
278+
* <p>Amazon Data Firehose is a fully managed service that delivers real-time
279279
* streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon
280280
* OpenSearch Service, Amazon Redshift, Splunk, and various other supportd
281281
* destinations.</p>

‎clients/client-firehose/src/FirehoseClient.ts

+2-2
Original file line numberDiff line numberDiff line change
@@ -307,8 +307,8 @@ export interface FirehoseClientResolvedConfig extends FirehoseClientResolvedConf
307307

308308
/**
309309
* @public
310-
* <fullname>Amazon Kinesis Data Firehose API Reference</fullname>
311-
* <p>Amazon Kinesis Data Firehose is a fully managed service that delivers real-time
310+
* <fullname>Amazon Data Firehose</fullname>
311+
* <p>Amazon Data Firehose is a fully managed service that delivers real-time
312312
* streaming data to destinations such as Amazon Simple Storage Service (Amazon S3), Amazon
313313
* OpenSearch Service, Amazon Redshift, Splunk, and various other supportd
314314
* destinations.</p>

‎clients/client-firehose/src/commands/CreateDeliveryStreamCommand.ts

+21-21
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
3232

3333
/**
3434
* @public
35-
* <p>Creates a Kinesis Data Firehose delivery stream.</p>
35+
* <p>Creates a Firehose delivery stream.</p>
3636
* <p>By default, you can create up to 50 delivery streams per Amazon Web Services
3737
* Region.</p>
3838
* <p>This is an asynchronous operation that immediately returns. The initial status of the
@@ -45,7 +45,7 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
4545
* doesn't change, and you can't invoke <code>CreateDeliveryStream</code> again on it.
4646
* However, you can invoke the <a>DeleteDeliveryStream</a> operation to delete
4747
* it.</p>
48-
* <p>A Kinesis Data Firehose delivery stream can be configured to receive records directly
48+
* <p>A Firehose delivery stream can be configured to receive records directly
4949
* from providers using <a>PutRecord</a> or <a>PutRecordBatch</a>, or it
5050
* can be configured to use an existing Kinesis stream as its source. To specify a Kinesis
5151
* data stream as input, set the <code>DeliveryStreamType</code> parameter to
@@ -68,7 +68,7 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
6868
* <p>When you specify <code>S3DestinationConfiguration</code>, you can also provide the
6969
* following optional values: BufferingHints, <code>EncryptionConfiguration</code>, and
7070
* <code>CompressionFormat</code>. By default, if no <code>BufferingHints</code> value is
71-
* provided, Kinesis Data Firehose buffers data up to 5 MB or for 5 minutes, whichever
71+
* provided, Firehose buffers data up to 5 MB or for 5 minutes, whichever
7272
* condition is satisfied first. <code>BufferingHints</code> is a hint, so there are some
7373
* cases where the service cannot adhere to these conditions strictly. For example, record
7474
* boundaries might be such that the size is a little over or under the configured buffering
@@ -78,7 +78,7 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
7878
* <ul>
7979
* <li>
8080
* <p>An Amazon Redshift destination requires an S3 bucket as intermediate location.
81-
* Kinesis Data Firehose first delivers data to Amazon S3 and then uses
81+
* Firehose first delivers data to Amazon S3 and then uses
8282
* <code>COPY</code> syntax to load data into an Amazon Redshift table. This is
8383
* specified in the <code>RedshiftDestinationConfiguration.S3Configuration</code>
8484
* parameter.</p>
@@ -91,16 +91,14 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
9191
* </li>
9292
* <li>
9393
* <p>We strongly recommend that you use the user name and password you provide
94-
* exclusively with Kinesis Data Firehose, and that the permissions for the account are
94+
* exclusively with Firehose, and that the permissions for the account are
9595
* restricted for Amazon Redshift <code>INSERT</code> permissions.</p>
9696
* </li>
9797
* </ul>
98-
* <p>Kinesis Data Firehose assumes the IAM role that is configured as part of the
99-
* destination. The role should allow the Kinesis Data Firehose principal to assume the role,
98+
* <p>Firehose assumes the IAM role that is configured as part of the
99+
* destination. The role should allow the Firehose principal to assume the role,
100100
* and the role should have permissions that allow the service to deliver the data. For more
101-
* information, see <a href="https://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html#using-iam-s3">Grant Kinesis Data
102-
* Firehose Access to an Amazon S3 Destination</a> in the <i>Amazon Kinesis Data
103-
* Firehose Developer Guide</i>.</p>
101+
* information, see <a href="https://docs.aws.amazon.com/firehose/latest/dev/controlling-access.html#using-iam-s3">Grant Firehose Access to an Amazon S3 Destination</a> in the <i>Amazon Firehose Developer Guide</i>.</p>
104102
* @example
105103
* Use a bare-bones client and the command you need to make an API call.
106104
* ```javascript
@@ -165,10 +163,10 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
165163
* Enabled: true || false,
166164
* Processors: [ // ProcessorList
167165
* { // Processor
168-
* Type: "RecordDeAggregation" || "Decompression" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
166+
* Type: "RecordDeAggregation" || "Decompression" || "CloudWatchLogProcessing" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
169167
* Parameters: [ // ProcessorParameterList
170168
* { // ProcessorParameter
171-
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat", // required
169+
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat" || "DataMessageExtraction", // required
172170
* ParameterValue: "STRING_VALUE", // required
173171
* },
174172
* ],
@@ -257,6 +255,8 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
257255
* },
258256
* Enabled: true || false,
259257
* },
258+
* FileExtension: "STRING_VALUE",
259+
* CustomTimeZone: "STRING_VALUE",
260260
* },
261261
* RedshiftDestinationConfiguration: { // RedshiftDestinationConfiguration
262262
* RoleARN: "STRING_VALUE", // required
@@ -297,10 +297,10 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
297297
* Enabled: true || false,
298298
* Processors: [
299299
* {
300-
* Type: "RecordDeAggregation" || "Decompression" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
300+
* Type: "RecordDeAggregation" || "Decompression" || "CloudWatchLogProcessing" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
301301
* Parameters: [
302302
* {
303-
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat", // required
303+
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat" || "DataMessageExtraction", // required
304304
* ParameterValue: "STRING_VALUE", // required
305305
* },
306306
* ],
@@ -361,10 +361,10 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
361361
* Enabled: true || false,
362362
* Processors: [
363363
* {
364-
* Type: "RecordDeAggregation" || "Decompression" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
364+
* Type: "RecordDeAggregation" || "Decompression" || "CloudWatchLogProcessing" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
365365
* Parameters: [
366366
* {
367-
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat", // required
367+
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat" || "DataMessageExtraction", // required
368368
* ParameterValue: "STRING_VALUE", // required
369369
* },
370370
* ],
@@ -405,10 +405,10 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
405405
* Enabled: true || false,
406406
* Processors: [
407407
* {
408-
* Type: "RecordDeAggregation" || "Decompression" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
408+
* Type: "RecordDeAggregation" || "Decompression" || "CloudWatchLogProcessing" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
409409
* Parameters: [
410410
* {
411-
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat", // required
411+
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat" || "DataMessageExtraction", // required
412412
* ParameterValue: "STRING_VALUE", // required
413413
* },
414414
* ],
@@ -443,10 +443,10 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
443443
* Enabled: true || false,
444444
* Processors: [
445445
* {
446-
* Type: "RecordDeAggregation" || "Decompression" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
446+
* Type: "RecordDeAggregation" || "Decompression" || "CloudWatchLogProcessing" || "Lambda" || "MetadataExtraction" || "AppendDelimiterToRecord", // required
447447
* Parameters: [
448448
* {
449-
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat", // required
449+
* ParameterName: "LambdaArn" || "NumberOfRetries" || "MetadataExtractionQuery" || "JsonParsingEngine" || "RoleArn" || "BufferSizeInMBs" || "BufferIntervalInSeconds" || "SubRecordType" || "Delimiter" || "CompressionFormat" || "DataMessageExtraction", // required
450450
* ParameterValue: "STRING_VALUE", // required
451451
* },
452452
* ],
@@ -572,7 +572,7 @@ export interface CreateDeliveryStreamCommandOutput extends CreateDeliveryStreamO
572572
* <p>The specified input parameter has a value that is not valid.</p>
573573
*
574574
* @throws {@link InvalidKMSResourceException} (client fault)
575-
* <p>Kinesis Data Firehose throws this exception when an attempt to put records or to start
575+
* <p>Firehose throws this exception when an attempt to put records or to start
576576
* or stop delivery stream encryption fails. This happens when the KMS service throws one of
577577
* the following exception types: <code>AccessDeniedException</code>,
578578
* <code>InvalidStateException</code>, <code>DisabledException</code>, or

‎clients/client-firehose/src/commands/DeleteDeliveryStreamCommand.ts

+7-4
Original file line numberDiff line numberDiff line change
@@ -29,15 +29,18 @@ export interface DeleteDeliveryStreamCommandOutput extends DeleteDeliveryStreamO
2929
/**
3030
* @public
3131
* <p>Deletes a delivery stream and its data.</p>
32-
* <p>To check the state of a delivery stream, use <a>DescribeDeliveryStream</a>. You can delete a delivery stream only if it is in one of the following states:
32+
* <p>You can delete a delivery stream only if it is in one of the following states:
3333
* <code>ACTIVE</code>, <code>DELETING</code>, <code>CREATING_FAILED</code>, or
3434
* <code>DELETING_FAILED</code>. You can't delete a delivery stream that is in the
35-
* <code>CREATING</code> state. While the deletion request is in process, the delivery
36-
* stream is in the <code>DELETING</code> state.</p>
37-
* <p>While the delivery stream is in the <code>DELETING</code> state, the service might
35+
* <code>CREATING</code> state. To check the state of a delivery stream, use <a>DescribeDeliveryStream</a>. </p>
36+
* <p>DeleteDeliveryStream is an asynchronous API. When an API request to DeleteDeliveryStream succeeds, the delivery stream is marked for deletion, and it goes into the
37+
* <code>DELETING</code> state.While the delivery stream is in the <code>DELETING</code> state, the service might
3838
* continue to accept records, but it doesn't make any guarantees with respect to delivering
3939
* the data. Therefore, as a best practice, first stop any applications that are sending
4040
* records before you delete a delivery stream.</p>
41+
* <p>Removal of a delivery stream that is in the <code>DELETING</code> state is a low priority operation for the service. A stream may remain in the
42+
* <code>DELETING</code> state for several minutes. Therefore, as a best practice, applications should not wait for streams in the <code>DELETING</code> state
43+
* to be removed. </p>
4144
* @example
4245
* Use a bare-bones client and the command you need to make an API call.
4346
* ```javascript

0 commit comments

Comments
 (0)