Skip to content

Commit e0b725c

Browse files
authored
feat(stepfunctions-tasks): add validations for EmrCreateCluster (#28529)
This PR adds validations for `EmrCreateCluster`. ## timeoutDurationMinutes > Minimum value is 5 and maximum value is 1440. https://docs.aws.amazon.com/emr/latest/APIReference/API_SpotProvisioningSpecification.html ## bidPrice and bidPriceAsPercentageOfOnDemandPrice Both `bidPrice` and `bidPriceAsPercentageOfOnDemandPrice` are specified, the error occurs in Step Functions console. ``` Specify at most one of bidPrice or bidPriceAsPercentageOfOnDemandPrice value for the Spot Instance fleet : Master request. (Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ValidationException; Request ID: xxxxxx; Proxy: null) ``` ## targetOnDemandCapacity and targetSpotCapacity > At least one of TargetSpotCapacity and TargetOnDemandCapacity should be greater than 0. For a master instance fleet, only one of TargetSpotCapacity and TargetOnDemandCapacity can be specified, and its value must be 1. https://docs.aws.amazon.com/emr/latest/APIReference/API_InstanceFleetConfig.html ---- *By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
1 parent 9d8b06f commit e0b725c

File tree

20 files changed

+234
-36
lines changed

20 files changed

+234
-36
lines changed

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/LambdaNodeJsLatestIntegDefaultTestDeployAssertD40B5C28.assets.json

+3-3
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/LambdaNodeJsLatestIntegDefaultTestDeployAssertD40B5C28.template.json

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/cdk-integ-lambda-nodejs-latest.assets.json

+6-6
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/cdk-integ-lambda-nodejs-latest.template.json

+1-1
Original file line numberDiff line numberDiff line change
@@ -38,7 +38,7 @@
3838
"S3Bucket": {
3939
"Fn::Sub": "cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"
4040
},
41-
"S3Key": "3c88aacdb6b48767fb52367e6dc0fe01602cfdc730dee4c4e3bebe0cec85ff9e.zip"
41+
"S3Key": "eed45a32a57f32bc36031539054db0b27239d161061c528482bb55be51068664.zip"
4242
},
4343
"Environment": {
4444
"Variables": {

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/cdk.out

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/integ.json

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/manifest.json

+3-3
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/@aws-cdk-testing/framework-integ/test/aws-lambda-nodejs/test/integ.latest.js.snapshot/tree.json

+1-1
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

packages/aws-cdk-lib/aws-stepfunctions-tasks/lib/emr/emr-create-cluster.ts

+19-1
Original file line numberDiff line numberDiff line change
@@ -582,13 +582,17 @@ export namespace EmrCreateCluster {
582582
/**
583583
* The bid price for each EC2 Spot instance type as defined by InstanceType. Expressed in USD.
584584
*
585+
* Cannot specify both `bidPrice` and `bidPriceAsPercentageOfOnDemandPrice`.
586+
*
585587
* @default - None
586588
*/
587589
readonly bidPrice?: string;
588590

589591
/**
590592
* The bid price, as a percentage of On-Demand price.
591593
*
594+
* Cannot specify both `bidPrice` and `bidPriceAsPercentageOfOnDemandPrice`.
595+
*
592596
* @default - None
593597
*/
594598
readonly bidPriceAsPercentageOfOnDemandPrice?: number;
@@ -713,6 +717,8 @@ export namespace EmrCreateCluster {
713717

714718
/**
715719
* The spot provisioning timeout period in minutes.
720+
*
721+
* The value must be between 5 and 1440.
716722
*/
717723
readonly timeoutDurationMinutes: number;
718724
}
@@ -781,14 +787,26 @@ export namespace EmrCreateCluster {
781787
/**
782788
* The target capacity of On-Demand units for the instance fleet, which determines how many On-Demand instances to provision.
783789
*
790+
* If not specified or set to 0, only Spot Instances are provisioned for the instance fleet using `targetSpotCapacity`.
791+
*
792+
* At least one of `targetSpotCapacity` and `targetOnDemandCapacity` should be greater than 0.
793+
* For a master instance fleet, only one of `targetSpotCapacity` and `targetOnDemandCapacity` can be specified, and its value
794+
* must be 1.
795+
*
784796
* @default No targetOnDemandCapacity
785797
*/
786798
readonly targetOnDemandCapacity?: number;
787799

788800
/**
789801
* The target capacity of Spot units for the instance fleet, which determines how many Spot instances to provision
790802
*
791-
* @default No targetSpotCapacity
803+
* If not specified or set to 0, only On-Demand Instances are provisioned for the instance fleet using `targetOnDemandCapacity`.
804+
*
805+
* At least one of `targetSpotCapacity` and `targetOnDemandCapacity` should be greater than 0.
806+
* For a master instance fleet, only one of `targetSpotCapacity` and `targetOnDemandCapacity` can be specified, and its value
807+
* must be 1.
808+
*
809+
* @default No targetSpotCapacity
792810
*/
793811
readonly targetSpotCapacity?: number;
794812
}

packages/aws-cdk-lib/aws-stepfunctions-tasks/lib/emr/private/cluster-utils.ts

+21
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,10 @@ export function EbsConfigurationPropertyToJson(property: EmrCreateCluster.EbsCon
105105
* @param property
106106
*/
107107
export function InstanceTypeConfigPropertyToJson(property: EmrCreateCluster.InstanceTypeConfigProperty) {
108+
if (property.bidPrice && property.bidPriceAsPercentageOfOnDemandPrice) {
109+
throw new Error('Cannot specify both bidPrice and bidPriceAsPercentageOfOnDemandPrice');
110+
}
111+
108112
return {
109113
BidPrice: cdk.stringToCloudFormation(property.bidPrice),
110114
BidPriceAsPercentageOfOnDemandPrice: cdk.numberToCloudFormation(property.bidPriceAsPercentageOfOnDemandPrice),
@@ -146,6 +150,9 @@ function SpotProvisioningSpecificationPropertyToJson(property?: EmrCreateCluster
146150
if (!property) {
147151
return undefined;
148152
}
153+
if (!cdk.Token.isUnresolved(property.timeoutDurationMinutes) && (property.timeoutDurationMinutes < 5 || property.timeoutDurationMinutes > 1440)) {
154+
throw new Error(`timeoutDurationMinutes must be between 5 and 1440, got ${property.timeoutDurationMinutes}`);
155+
}
149156
return {
150157
AllocationStrategy: cdk.stringToCloudFormation(property.allocationStrategy),
151158
BlockDurationMinutes: cdk.numberToCloudFormation(property.blockDurationMinutes),
@@ -160,6 +167,20 @@ function SpotProvisioningSpecificationPropertyToJson(property?: EmrCreateCluster
160167
* @param property
161168
*/
162169
export function InstanceFleetConfigPropertyToJson(property: EmrCreateCluster.InstanceFleetConfigProperty) {
170+
if (!property.targetSpotCapacity && !property.targetOnDemandCapacity) {
171+
throw new Error('At least one of targetSpotCapacity and targetOnDemandCapacity should be greater than 0');
172+
}
173+
if (property.instanceFleetType === EmrCreateCluster.InstanceRoleType.MASTER) {
174+
if (property.targetSpotCapacity && property.targetOnDemandCapacity) {
175+
throw new Error('For a master instance fleet, only one of targetSpotCapacity and targetOnDemandCapacity can be specified');
176+
}
177+
if (property.targetSpotCapacity && property.targetSpotCapacity !== 1) {
178+
throw new Error(`For a master instance fleet, targetSpotCapacity cannot be a number other than 1, got ${property.targetSpotCapacity}`);
179+
}
180+
if (property.targetOnDemandCapacity && property.targetOnDemandCapacity !== 1) {
181+
throw new Error(`For a master instance fleet, targetOnDemandCapacity cannot be a number other than 1, got ${property.targetOnDemandCapacity}`);
182+
}
183+
}
163184
return {
164185
InstanceFleetType: cdk.stringToCloudFormation(property.instanceFleetType?.valueOf()),
165186
InstanceTypeConfigs: cdk.listMapper(InstanceTypeConfigPropertyToJson)(property.instanceTypeConfigs),

0 commit comments

Comments
 (0)