-
Notifications
You must be signed in to change notification settings - Fork 153
feat(idempotency): Add function wrapper and decorator #1262
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
73 commits
Select commit
Hold shift + click to select a range
51104f6
feat: initial idempotency classes
vgphoenixcampos d54e22e
feat: refactor persistence layer classes into their own folder
vgphoenixcampos 6ea753e
feat: rename idempotency config to differentiate from idempotency opt…
vgphoenixcampos c5f2144
feat: added type for a generic function
vgphoenixcampos ab254c6
feat: remove idempotency configuration for this FR
vgphoenixcampos 8677376
feat: refactored type of function to accept any combo of parameters
vgphoenixcampos b955ac4
feat: adding PersistenceLayer
jeffrey-baker-vg c8776d2
feat: PersistenceLayer unit tests for saveInProgress
jeffrey-baker-vg f258d55
feat: added saveSuccess
jeffrey-baker-vg edc9b33
feat: added getRecord
jeffrey-baker-vg dfc72c0
feat: added delete record
jeffrey-baker-vg 4d63218
feat: branch coverage and cleaning up imports
jeffrey-baker-vg d92efb1
feat: added more tests
jeffrey-baker-vg 5258d10
feat: deleted unused methods
jeffrey-baker-vg 5d82215
feat: added comments
jeffrey-baker-vg 95004c2
feat: implement get command for dynamo persistence layer
KevenFuentes9 c5a9335
feat: implement get command for dynamo persistence layer
KevenFuentes9 3d1f52c
feat: allow for data attr to be passed and return in persistence laye…
KevenFuentes9 0a6e839
feat: added implementation for delete, update, put
vgphoenixcampos b9ae754
feat: create condition on put for not in progress status
vgphoenixcampos 2cd4390
feat: use inprogress enum for status
vgphoenixcampos 01bcc99
feat: added error when unable to get record for idempotency key
vgphoenixcampos c1a6c60
feat: added error for conditional write of an existing record
vgphoenixcampos 1011002
feat: tests added for put record on dynamo persistence layer
vgphoenixcampos adcfab3
feat: implemented the idempotency record functions for status, expiry…
KevenFuentes9 9f0535e
test: check if the status is expired
KevenFuentes9 9bf7885
test: idempotency record is not expired and status maintained
KevenFuentes9 ffb2c22
feat: added tests for get record
vgphoenixcampos ec31dbb
feat: add aws-sdk-client-mock jest assertion library
vgphoenixcampos 48dfdec
feat: add unit tests for update record and delete record
vgphoenixcampos f666dfa
feat: remove optional chaining from item made unnecessary with error …
vgphoenixcampos 0945801
feat: remove unused block
vgphoenixcampos e161521
feat: refactored mock child class to be shared amongst dynamo persist…
vgphoenixcampos 9efeae0
test: add path to get the response data from the data record
KevenFuentes9 0fea9f3
feat: added branch to handle conditional check failure
vgphoenixcampos b4165bd
feat: add configuration option to dynamo client creation to remove un…
vgphoenixcampos d1dad17
feat: change how time is measured to seconds
vgphoenixcampos b9afc2a
feat: change type of the response/result to a record
vgphoenixcampos 0ec8301
feat:updated imports
jeffrey-baker-vg 1366614
feat: added save in progress to handle already existing records in dy…
KevenFuentes9 38d7a67
feat: add log message for the already in progress error
KevenFuentes9 dd442e2
feat: change the anyfunction type definition to also include a sync f…
KevenFuentes9 8e97a61
refactor: create constructor object for dynamo persistence layer
vgphoenixcampos afb9523
fix: remove temp eslint disable
vgphoenixcampos 8a031ca
fix: adjust verbiage on test blocks
vgphoenixcampos f476e21
style: put constructor parameters onto one line for readability
vgphoenixcampos a044fa6
fix: update dynamo persistence layer tests to use new construtor opt…
vgphoenixcampos cefae2d
Merge branch 'main' of github.com:jeffrey-baker-vg/aws-lambda-powerto…
vgphoenixcampos dd979a8
fix: remove unneeded eslint ignore from persistence layer
vgphoenixcampos b290d19
style: put parameters for dynamo client command object onto one line …
vgphoenixcampos f76c720
fix: move lib-dynamo dep under the correct package
vgphoenixcampos b4931bf
refactor: change idempotency record to use options object in contructor
vgphoenixcampos 9b4787a
feat: add consistent read to dynamo persistence layer
vgphoenixcampos 08598f9
fix: revert changes to layer-publisher package-lock
vgphoenixcampos b3fe0be
feat added the call to the function in the idempotency handler and ad…
KevenFuentes9 0af6a5e
feat: add logic to invoke function if it has not already been called;…
KevenFuentes9 68ce9a2
chore: move and enhance comment on question
KevenFuentes9 19e01eb
chore: update comments
KevenFuentes9 5afab50
feat: use new record formatting for idempotent function wrapper
KevenFuentes9 78f1a89
test: add test case for issues even getting to save the record in per…
KevenFuentes9 58e1bda
chore: refactoring test suite for idempotent wrapper
KevenFuentes9 8fd2436
chore: clean up comments
KevenFuentes9 2e43938
feat: added decorator for idempotency
KevenFuentes9 9e369f7
chore: get rid of extra private member on the idempotency handler class
KevenFuentes9 cdae838
chore: merging
KevenFuentes9 f5e3b6b
chore: refactor to use class for options
KevenFuentes9 927654f
chore: bring in old version of package-locks
KevenFuentes9 d0d0574
Merge pull request #5 from jeffrey-baker-vg/origin/idempotency-core
KevenFuentes9 1ec1c80
Merge remote-tracking branch 'upstream/main'
KevenFuentes9 e3eb775
chore: update paths for interface
KevenFuentes9 bd2388c
chore: remove env files
KevenFuentes9 eadb129
chore: rename file
KevenFuentes9 780a8ef
chore: renaming test names and function names for idempotency decorat…
KevenFuentes9 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -86,4 +86,4 @@ | |
"dependencies": { | ||
"hosted-git-info": "^6.1.1" | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
/* eslint-disable @typescript-eslint/no-explicit-any */ | ||
|
||
import { AnyFunctionWithRecord, IdempotencyRecordStatus } from './types'; | ||
import { IdempotencyOptions } from './types/IdempotencyOptions'; | ||
import { IdempotencyRecord } from 'persistence'; | ||
import { IdempotencyInconsistentStateError, IdempotencyItemAlreadyExistsError, IdempotencyAlreadyInProgressError, IdempotencyPersistenceLayerError } from './Exceptions'; | ||
|
||
export class IdempotencyHandler<U> { | ||
|
||
public constructor(private functionToMakeIdempotent: AnyFunctionWithRecord<U>, private functionPayloadToBeHashed: unknown, | ||
private idempotencyOptions: IdempotencyOptions, private fullFunctionPayload: Record<string, any>) {} | ||
|
||
public determineResultFromIdempotencyRecord(idempotencyRecord: IdempotencyRecord): Promise<U> | U{ | ||
if (idempotencyRecord.getStatus() === IdempotencyRecordStatus.EXPIRED) { | ||
throw new IdempotencyInconsistentStateError('Item has expired during processing and may not longer be valid.'); | ||
} else if (idempotencyRecord.getStatus() === IdempotencyRecordStatus.INPROGRESS){ | ||
throw new IdempotencyAlreadyInProgressError(`There is already an execution in progress with idempotency key: ${idempotencyRecord.idempotencyKey}`); | ||
} else { | ||
// Currently recalling the method as this fulfills FR1. FR3 will address using the previously stored value https://github.com/awslabs/aws-lambda-powertools-typescript/issues/447 | ||
return this.functionToMakeIdempotent(this.fullFunctionPayload); | ||
} | ||
} | ||
|
||
public async processIdempotency(): Promise<U> { | ||
try { | ||
await this.idempotencyOptions.persistenceStore.saveInProgress(this.functionPayloadToBeHashed); | ||
} catch (e) { | ||
if (e instanceof IdempotencyItemAlreadyExistsError) { | ||
const idempotencyRecord: IdempotencyRecord = await this.idempotencyOptions.persistenceStore.getRecord(this.functionPayloadToBeHashed); | ||
|
||
return this.determineResultFromIdempotencyRecord(idempotencyRecord); | ||
} else { | ||
throw new IdempotencyPersistenceLayerError(); | ||
} | ||
} | ||
|
||
return this.functionToMakeIdempotent(this.fullFunctionPayload); | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
/* eslint-disable @typescript-eslint/no-explicit-any */ | ||
import { IdempotencyOptions } from './types/IdempotencyOptions'; | ||
import { IdempotencyHandler } from './IdempotencyHandler'; | ||
|
||
const idempotent = function (options: IdempotencyOptions) { | ||
return function (_target: any, _propertyKey: string, descriptor: PropertyDescriptor) { | ||
const childFunction = descriptor.value; | ||
descriptor.value = function(record: Record<string, any>){ | ||
const idempotencyHandler: IdempotencyHandler<unknown> = new IdempotencyHandler<unknown>(childFunction, record[options.dataKeywordArgument], options, record); | ||
|
||
return idempotencyHandler.processIdempotency(); | ||
}; | ||
|
||
return descriptor; | ||
}; | ||
}; | ||
|
||
export { idempotent }; | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,11 +1,19 @@ | ||
import type { AnyFunction } from './types/AnyFunction'; | ||
import type { IdempotencyOptions } from './types/IdempotencyOptions'; | ||
|
||
const makeFunctionIdempotent = <U>( | ||
fn: AnyFunction<U>, | ||
_options: IdempotencyOptions | ||
// TODO: revisit this with a more specific type if possible | ||
/* eslint-disable @typescript-eslint/no-explicit-any */ | ||
): (...args: Array<any>) => Promise<U | void> => (...args) => fn(...args); | ||
/* eslint-disable @typescript-eslint/no-explicit-any */ | ||
import { AnyFunctionWithRecord, AnyIdempotentFunction } from './types/AnyFunction'; | ||
import { IdempotencyOptions } from './types/IdempotencyOptions'; | ||
import { IdempotencyHandler } from './IdempotencyHandler'; | ||
|
||
const makeFunctionIdempotent = function <U>( | ||
fn: AnyFunctionWithRecord<U>, | ||
saragerion marked this conversation as resolved.
Show resolved
Hide resolved
|
||
options: IdempotencyOptions | ||
): AnyIdempotentFunction<U> { | ||
const wrappedFn: AnyIdempotentFunction<U> = function (record: Record<string, any>): Promise<U> { | ||
const idempotencyHandler: IdempotencyHandler<U> = new IdempotencyHandler<U>(fn, record[options.dataKeywordArgument], options, record); | ||
|
||
return idempotencyHandler.processIdempotency(); | ||
}; | ||
|
||
return wrappedFn; | ||
}; | ||
|
||
export { makeFunctionIdempotent }; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,11 @@ | ||
// eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
type AnyFunction<U> = (...args: Array<any>) => Promise<U>; | ||
type AnyFunctionWithRecord<U> = (record: Record<string,any>) => Promise<U> | U; | ||
|
||
// eslint-disable-next-line @typescript-eslint/no-explicit-any | ||
type AnyIdempotentFunction<U> = (record: Record<string,any>) => Promise<U>; | ||
|
||
export { | ||
AnyFunction | ||
// AnyFunction, | ||
AnyFunctionWithRecord, | ||
AnyIdempotentFunction | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,4 @@ | ||
export * from './AnyFunction'; | ||
export * from './IdempotencyRecordStatus'; | ||
export * from './PersistenceLayer'; | ||
export * from './IdempotencyRecordOptions'; | ||
export * from './PersistenceLayer'; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor: adding a meaningful and easy to understand error message for each of these Errors would improve the DX and help troubleshooting errors
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I added a message to this when the error is created since this error can happen in more than one type of scenario. That way we can tailor the message by the scenario