Skip to content

PM-1073 trolley webhook handling #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Apr 18, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# EditorConfig is awesome: https://EditorConfig.org

# top-most EditorConfig file
root = true

[*]
indent_style = space
indent_size = 2
end_of_line = lf
charset = utf-8
trim_trailing_whitespace = true
insert_final_newline = true

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ensure there is a newline at the end of the file. This is a common convention to avoid potential issues with some tools and version control systems.

Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
-- CreateEnum
CREATE TYPE "webhook_status" AS ENUM ('error', 'processed', 'logged');

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The enum value 'processing' was changed to 'logged'. Ensure that this change aligns with the intended logic and that all references to 'processing' in the application code are updated accordingly to prevent any inconsistencies or errors.


-- CreateTable
CREATE TABLE "trolley_webhook_log" (
"id" UUID NOT NULL DEFAULT uuid_generate_v4(),
"event_id" TEXT NOT NULL,
"event_time" TIMESTAMP(6) NOT NULL DEFAULT CURRENT_TIMESTAMP,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider specifying the timezone for the TIMESTAMP fields to ensure consistency across different environments. Use TIMESTAMP WITH TIME ZONE if appropriate.

"event_payload" TEXT NOT NULL,
"event_model" TEXT,
"event_action" TEXT,
"status" "webhook_status" NOT NULL,
"error_message" TEXT,
"created_by" VARCHAR(80),
"updated_by" VARCHAR(80),
"created_at" TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP,
"updated_at" TIMESTAMP(6) DEFAULT CURRENT_TIMESTAMP,

CONSTRAINT "trolley_webhook_log_pkey" PRIMARY KEY ("id")
);

-- CreateIndex
CREATE UNIQUE INDEX "trolley_webhook_log_event_id_key" ON "trolley_webhook_log"("event_id");
25 changes: 23 additions & 2 deletions prisma/schema.prisma
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
generator client {
provider = "prisma-client-js"
provider = "prisma-client-js"
previewFeatures = ["extendedIndexes"]
binaryTargets = ["native", "linux-musl-openssl-3.0.x"]
binaryTargets = ["native", "linux-musl-openssl-3.0.x"]
}

datasource db {
Expand Down Expand Up @@ -210,6 +210,27 @@ model winnings {
origin origin? @relation(fields: [origin_id], references: [origin_id], onDelete: NoAction, onUpdate: NoAction)
}

enum webhook_status {
error
processed
logged

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider whether changing the enum value from processing to logged accurately reflects the intended state of the webhook log. If processing was meant to indicate an ongoing process, logged might not convey the same meaning. Ensure that this change aligns with the logic and handling of webhook statuses throughout the application.

}

model trolley_webhook_log {
id String @id @default(dbgenerated("uuid_generate_v4()")) @db.Uuid
event_id String @unique
event_time DateTime @default(now()) @db.Timestamp(6)
event_payload String
event_model String?
event_action String?
status webhook_status
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure yet how we gonna use this and what will be the lifecycle of log status. Will plan this soon...
Please add new enum value in webhook_status as logged and default to it for status field.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kkartunov "processing" is basically "logged". When we receive the event first thing is we insert it in db and mark it as "processing".
After something handles it (if there's any handler to process it) we mark it as "processed" or "error" depending on the status of the handler.
I'll replace "processing" with "logged" to be more straight forward, sounds good?

error_message String?
created_by String? @db.VarChar(80)
updated_by String? @db.VarChar(80)
created_at DateTime? @default(now()) @db.Timestamp(6)
updated_at DateTime? @default(now()) @db.Timestamp(6)
}

enum action_type {
INITIATE_WITHDRAWAL
ADD_WITHDRAWAL_METHOD
Expand Down
3 changes: 2 additions & 1 deletion src/api/api.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,10 @@ import { TopcoderModule } from 'src/shared/topcoder/topcoder.module';
import { OriginRepository } from './repository/origin.repo';
import { TaxFormRepository } from './repository/taxForm.repo';
import { PaymentMethodRepository } from './repository/paymentMethod.repo';
import { WebhooksModule } from './webhooks/webhooks.module';

@Module({
imports: [GlobalProvidersModule, TopcoderModule],
imports: [WebhooksModule, GlobalProvidersModule, TopcoderModule],
controllers: [
HealthCheckController,
AdminWinningController,
Expand Down
17 changes: 17 additions & 0 deletions src/api/webhooks/trolley/handlers/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import { Provider } from '@nestjs/common';
import { PaymentHandler } from './payment.handler';
import { getWebhooksEventHandlersProvider } from '../../webhooks.event-handlers.provider';

export const TrolleyWebhookHandlers: Provider[] = [
getWebhooksEventHandlersProvider(
'trolleyHandlerFns',
'TrolleyWebhookHandlers',
),

PaymentHandler,
{
provide: 'TrolleyWebhookHandlers',
useFactory: (paymentHandler: PaymentHandler) => [paymentHandler],
inject: [PaymentHandler],
},
];
19 changes: 19 additions & 0 deletions src/api/webhooks/trolley/handlers/payment.handler.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import { Injectable } from '@nestjs/common';
import { WebhookEvent } from '../../webhooks.decorators';
import { TrolleyWebhookEvent } from '../trolley.types';

@Injectable()
export class PaymentHandler {
@WebhookEvent(TrolleyWebhookEvent.paymentCreated)
async handlePaymentCreated(payload: any): Promise<any> {
// TODO: Build out logic for payment.created event
console.log('handling', TrolleyWebhookEvent.paymentCreated);

}

@WebhookEvent(TrolleyWebhookEvent.paymentUpdated)
async handlePaymentUpdated(payload: any): Promise<any> {
// TODO: Build out logic for payment.updated event
console.log('handling', TrolleyWebhookEvent.paymentUpdated);
}
}
138 changes: 138 additions & 0 deletions src/api/webhooks/trolley/trolley.service.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
import crypto from 'crypto';
import { Inject, Injectable } from '@nestjs/common';
import { trolley_webhook_log, webhook_status } from '@prisma/client';
import { PrismaService } from 'src/shared/global/prisma.service';

enum TrolleyHeaders {
id = 'x-paymentrails-delivery',
signature = 'x-paymentrails-signature',
created = 'x-paymentrails-created',
}

const trolleyWhHmac = process.env.TROLLEY_WH_HMAC;
if (!trolleyWhHmac) {
throw new Error('TROLLEY_WH_HMAC is not set!');
}

/**
* Service responsible for handling Trolley webhook operations.
*/
@Injectable()
export class TrolleyService {
constructor(
@Inject('trolleyHandlerFns')
private readonly handlers,
private readonly prisma: PrismaService,
) {}

/**
* Validates the webhook signature to ensure the request is authentic.
*
* @param headers - The HTTP request headers containing the signature.
* @param bodyPayload - The raw body payload of the webhook request.
* @returns A boolean indicating whether the signature is valid.
*/
validateSignature(headers: Request['headers'], bodyPayload: string): boolean {
const headerSignature = headers[TrolleyHeaders.signature] ?? '';
if (!headerSignature || !headerSignature.match(/t=\d+,v1=[a-f0-9]{64}/i)) {
return false;
}

const headerSignatureValues = headerSignature.split(',');
const t = headerSignatureValues[0].split('=')[1];
const v1 = headerSignatureValues[1].split('=')[1];

const hmac = crypto.createHmac('sha256', trolleyWhHmac as string);
hmac.update(`${t}${bodyPayload}`);
const digest = hmac.digest('hex');

return digest === v1;
}

/**
* Validates whether the webhook event is unique by checking its ID against the database.
*
* @param headers - The HTTP request headers containing the webhook ID.
* @returns A promise that resolves to a boolean indicating whether the webhook event is unique.
*/
async validateUnique(headers: Request['headers']): Promise<boolean> {
const requestId = headers[TrolleyHeaders.id];

if (!requestId) {
return false;
}

const whEvent = await this.prisma.trolley_webhook_log.findUnique({
where: { event_id: requestId },
});
return !whEvent;
}

/**
* Tracks the webhook events status by Updating or creating a record in the `trolley_webhook_log` table with the given event details.
*
* @param requestId - The unique identifier for the webhook event.
* @param status - The status of the webhook event.
* @param payload - (Optional) The payload associated with the webhook event.
* @param meta - (Optional) Additional metadata for the webhook event, such as event time.
* @returns A promise that resolves to the upserted `trolley_webhook_log` record.
*/
setEventState(
requestId: string,
status: webhook_status,
payload?: any,
meta?: Partial<trolley_webhook_log>,
) {
return this.prisma.trolley_webhook_log.upsert({
where: {
event_id: requestId,
},
create: {
event_id: requestId,
event_payload: JSON.stringify(payload ?? {}),
event_time: meta?.event_time,
event_model: payload?.model ?? '',
event_action: payload?.action ?? '',
status,
created_by: 'system',
} as trolley_webhook_log,
update: {
status,
...meta,
},
});
}

/**
* Handles incoming webhook events by processing the payload and delegating
* the event to the appropriate handler based on the model and action.
*
* @param headers - The headers of the incoming request, containing metadata
* such as the event ID and creation time.
* @param payload - The body of the webhook event, containing details such as
* the model, action, and event-specific data.
*/
async handleEvent(headers: Request['headers'], payload: any) {
const requestId = headers[TrolleyHeaders.id];

try {
await this.setEventState(requestId, webhook_status.logged, payload, {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The event state has been changed from processing to logged. Ensure that this change aligns with the intended logic of the application. If the event should be marked as processing before handling, consider revisiting this change.

event_time: headers[TrolleyHeaders.created],
});

const { model, action, body } = payload;
const handler = this.handlers.get(`${model}.${action}`);
// do nothing if there's no handler for the event (event was logged in db)
if (!handler) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous implementation threw an error when no handler was found. The new implementation silently returns. Consider logging a warning or informational message to track unhandled events for monitoring purposes.

return;
}

await handler(body);
await this.setEventState(requestId, webhook_status.processed);
} catch (e) {
await this.setEventState(requestId, webhook_status.error, void 0, {
error_message: e.message ?? e,
});
}
}
}
6 changes: 6 additions & 0 deletions src/api/webhooks/trolley/trolley.types.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
export enum TrolleyWebhookEvent {
paymentCreated = 'payment.created',
paymentUpdated = 'payment.updated',
}

export type TrolleyEventHandler = (eventPayload: any) => Promise<unknown>;
47 changes: 47 additions & 0 deletions src/api/webhooks/webhooks.controller.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import {
Controller,
Post,
Req,

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The BadRequestException import has been removed but not replaced with any other exception handling for bad requests. Ensure that any necessary exception handling for bad requests is implemented elsewhere in the code.

RawBodyRequest,
ForbiddenException,
} from '@nestjs/common';
import { ApiTags } from '@nestjs/swagger';
import { TrolleyService } from './trolley/trolley.service';
import { Public } from 'src/core/auth/decorators';

@Public()
@ApiTags('Webhooks')
@Controller('webhooks')
export class WebhooksController {
constructor(private readonly trolleyService: TrolleyService) {}

/**
* Handles incoming trolley webhooks.
*
* This method validates the webhook request by checking its signature and ensuring
* it has not been processed before. If validation passes, it processes the webhook
* payload and marks it as processed.
*
* @param request - The incoming webhook request containing headers, raw body, and parsed body.
* @returns A success message if the webhook is processed successfully.
* @throws {ForbiddenException} If the signature is invalid or the webhook has already been processed.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exception type has been changed from BadRequestException to ForbiddenException. Ensure that this change aligns with the intended behavior of the application. If the change is intentional, verify that all parts of the application that handle this exception are updated accordingly.

*/
@Post('trolley')
async handleTrolleyWebhook(@Req() request: RawBodyRequest<Request>) {
Copy link
Contributor

@kkartunov kkartunov Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is all great with approaching & preparing for event handlers.
But what we want for scope of this is something different:

  • handleTrolleyWebhook need to always succeed and return 200 if we able to verify signature. Only if unable to verify signature respond with 403.
  • Make sure the event is logged in trolley_webhook_log with status logged.
  • In case of a repeat validateUnique just return 200 to Trolley and do nothing on our side.
  • we will handle events on our own. Whatever we do with them is not Trolley bussiness but Topcoder stuff.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true. the comment is deprecated. As mentioned in the PR, we're not throwing "processing" errors. Only missing or invalid signature will be thrown.
I'll supress the "unique" error as well - makes sense, right!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm. let me rephrase that:

  • yes, we're always returning 200, and 403 for invalid signature (i was also throwing 400 for duplicate calls, but will supress it)
  • events are already logged as soon as they land
  • ok
  • exactly, that's already happening and I tried to mention it in the PR desc as well. we're on the same side 👍

if (
!this.trolleyService.validateSignature(
request.headers,
request.rawBody?.toString('utf-8') ?? '',
)
) {
throw new ForbiddenException('Missing or invalid signature!');

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the exception from BadRequestException to ForbiddenException may alter the intended HTTP response status code. Ensure that ForbiddenException (HTTP 403) is the correct choice for handling missing or invalid signatures, as it indicates that the request was valid but the server is refusing action, whereas BadRequestException (HTTP 400) indicates that the request itself is malformed. Verify that this change aligns with the desired API behavior.

}

// do not proceed any further if event has already been processed
if (!(await this.trolleyService.validateUnique(request.headers))) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider adding a log entry or some form of notification when a webhook is identified as already processed. This can help with debugging and monitoring.

return;
}

return this.trolleyService.handleEvent(request.headers, request.body);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The try-catch block for error handling has been removed. Consider reintroducing error handling to ensure that any exceptions thrown by handleEvent are properly managed and logged. This will help in diagnosing issues and maintaining system stability.

}
}
5 changes: 5 additions & 0 deletions src/api/webhooks/webhooks.decorators.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { SetMetadata } from '@nestjs/common';

export const WEBHOOK_EVENT_METADATA_KEY = 'WH_EVENT_TYPE';
export const WebhookEvent = (...events: string[]) =>
SetMetadata(WEBHOOK_EVENT_METADATA_KEY, events);
62 changes: 62 additions & 0 deletions src/api/webhooks/webhooks.event-handlers.provider.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
import { Reflector } from '@nestjs/core';
import { WEBHOOK_EVENT_METADATA_KEY } from './webhooks.decorators';

/**
* Factory function to create a map of Trolley webhook event handlers.
*
* This function iterates over the provided handler classes and inspects their methods
* to find those annotated with specific metadata indicating the Trolley webhook events
* they handle. It then binds these methods to their respective event types and stores
* them in a map for easy lookup.
*
* @param reflector - An instance of `Reflector` used to retrieve metadata from methods.
* @param handlerClasses - An array of handler class instances containing methods
* annotated with Trolley webhook event metadata.
* @returns A `Map` where the keys are `TrolleyWebhookEvent` types and the values are
* bound handler functions for those events.
*/
const whEventHandlersFactory = (reflector: Reflector, handlerClasses) => {
const handlersMap = new Map<
string,
(eventPayload: any) => Promise<unknown>
>();

for (const handlerClass of handlerClasses) {
const prototype = Object.getPrototypeOf(handlerClass);
for (const propertyName of Object.getOwnPropertyNames(prototype)) {
const method = prototype[propertyName];
if (typeof method !== 'function' || propertyName === 'constructor') {
continue;
}

const eventTypes = reflector.get<string[]>(
WEBHOOK_EVENT_METADATA_KEY,
method,
);

if (eventTypes?.length > 0) {
eventTypes.forEach((eventType) => {
handlersMap.set(eventType, method.bind(handlerClass));
});
}
}
}

return handlersMap;
};

/**
* Creates a provider object for webhook event handlers.
*
* @param provide - The token that will be used to provide the dependency.
* @param handlersKey - The key used to identify the specific handlers to inject.
* @returns An object defining the provider with a factory function and its dependencies.
*/
export const getWebhooksEventHandlersProvider = (
provide: string,
handlersKey: string,
) => ({
provide,
useFactory: whEventHandlersFactory,
inject: [Reflector, handlersKey],
});
Loading