Skip to content

Add support to clear custom dimensions #136

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Sep 16, 2022
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 40 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,9 +156,9 @@ putDimensions({ Operation: "Aggregator" })
putDimensions({ Operation: "Aggregator", DeviceType: "Actuator" })
```

- **setDimensions**(Record<String, String>[] dimensions...)
- **setDimensions**(Record<String, String> | Record<String, String>[] dimensions, boolean useDefault)

Explicitly override all dimensions. This will remove the default dimensions.
Explicitly override all dimensions. This will remove the default dimensions unless the `useDefault` parameter is set to `true` (defaults to false).

**WARNING**: Every distinct value will result in a new CloudWatch Metric.
If the cardinality of a particular value is expected to be high, you should consider
Expand All @@ -171,9 +171,26 @@ Requirements:
Examples:

```js
setDimensions(
// Overwrites custom dimensions - keeps default dimensions
setDimensions({Operation: "Aggregator"}, true)
```

```js
// Overwrites custom dimensions - removes default dimensions
setDimensions([
{ Operation: "Aggregator" },
{ Operation: "Aggregator", DeviceType: "Actuator" })
{ Operation: "Aggregator", DeviceType: "Actuator" }
])
```

- **resetDimensions**(boolean useDefault)

Explicitly clear all custom dimensions. Set `useDefault` to `true` to keep the default dimensions.

Example:

```js
resetDimensions(false) // this will clear all custom dimensions as well as disable default dimensions
```

- **setNamespace**(String value)
Expand All @@ -185,7 +202,7 @@ Requirements:
- Name Length 1-255 characters
- Name must be ASCII characters only

Examples:
Example:

```js
setNamespace("MyApplication");
Expand All @@ -209,7 +226,24 @@ setTimestamp(new Date().getTime())

- **flush**()

Flushes the current MetricsContext to the configured sink and resets all properties, dimensions and metric values. The namespace and default dimensions will be preserved across flushes. Timestamp will be preserved if set explicitly via `setTimestamp()`.
Flushes the current MetricsContext to the configured sink and resets all properties and metric values. The namespace and default dimensions will be preserved across flushes. Custom dimensions are preserved by default, but this behavior can be changed by setting `logger.flushPreserveDimensions = false`. Timestamp will be preserved if set explicitly via `setTimestamp()`.

Examples:

```js
logger.flush() // custom and default dimensions will be preserved after each flush
```

```js
logger.flushPreserveDimensions = false
logger.flush() // only default dimensions will be preserved after flush()
```

```js
logger.flushPreserveDimensions = false
logger.resetDimensions(false)
logger.flush() // default dimensions are disabled - no dimensions will be preserved after flush()
```

## Configuration

Expand Down
29 changes: 19 additions & 10 deletions src/logger/MetricsContext.ts
Original file line number Diff line number Diff line change
Expand Up @@ -118,12 +118,12 @@ export class MetricsContext {
// This ensures the latest dimension key-value is used as a target member on the root EMF node.
// This operation is O(n^2), but acceptable given sets are capped at 10 dimensions
const incomingDimensionSetKeys = Object.keys(incomingDimensionSet);
this.dimensions = this.dimensions.filter(existingDimensionSet => {
this.dimensions = this.dimensions.filter((existingDimensionSet) => {
const existingDimensionSetKeys = Object.keys(existingDimensionSet);
if (existingDimensionSetKeys.length !== incomingDimensionSetKeys.length) {
return true;
}
return !existingDimensionSetKeys.every(existingDimensionSetKey =>
return !existingDimensionSetKeys.every((existingDimensionSetKey) =>
incomingDimensionSetKeys.includes(existingDimensionSetKey),
);
});
Expand All @@ -136,14 +136,21 @@ export class MetricsContext {
*
* @param dimensionSets
*/
public setDimensions(dimensionSets: Array<Record<string, string>>): void {
this.shouldUseDefaultDimensions = false;

dimensionSets.forEach(dimensionSet => Validator.validateDimensionSet(dimensionSet));

public setDimensions(dimensionSets: Array<Record<string, string>>, useDefault = false): void {
dimensionSets.forEach((dimensionSet) => Validator.validateDimensionSet(dimensionSet));
this.shouldUseDefaultDimensions = useDefault;
this.dimensions = dimensionSets;
}

/**
* Reset all custom dimensions
* @param useDefault Indicates whether default dimensions should be used
*/
public resetDimensions(useDefault: boolean): void {
this.shouldUseDefaultDimensions = useDefault;
this.dimensions = [];
}

/**
* Get the current dimensions.
*/
Expand All @@ -166,7 +173,7 @@ export class MetricsContext {
// otherwise, merge the dimensions
// we do this on the read path because default dimensions
// may get updated asynchronously by environment detection
return this.dimensions.map(custom => {
return this.dimensions.map((custom) => {
return { ...this.defaultDimensions, ...custom };
});
}
Expand All @@ -182,12 +189,14 @@ export class MetricsContext {

/**
* Creates an independently flushable context.
* Custom dimensions are preserved.
* @param preserveDimensions Indicates whether custom dimensions should be preserved
*/
public createCopyWithContext(): MetricsContext {
public createCopyWithContext(preserveDimensions = true): MetricsContext {
return new MetricsContext(
this.namespace,
Object.assign({}, this.properties),
Object.assign([], this.dimensions),
preserveDimensions ? Object.assign([], this.dimensions) : [],
this.defaultDimensions,
this.shouldUseDefaultDimensions,
this.timestamp,
Expand Down
35 changes: 30 additions & 5 deletions src/logger/MetricsLogger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,12 @@ import { Unit } from './Unit';
export class MetricsLogger {
private context: MetricsContext;
private resolveEnvironment: EnvironmentProvider;
public flushPreserveDimensions: boolean;

constructor(resolveEnvironment: EnvironmentProvider, context?: MetricsContext) {
this.resolveEnvironment = resolveEnvironment;
this.context = context || MetricsContext.empty();
this.flushPreserveDimensions = true;
}

/**
Expand All @@ -48,7 +50,7 @@ export class MetricsLogger {

// accept and reset the context
await sink.accept(this.context);
this.context = this.context.createCopyWithContext();
this.context = this.context.createCopyWithContext(this.flushPreserveDimensions);
}

/**
Expand Down Expand Up @@ -83,12 +85,35 @@ export class MetricsLogger {

/**
* Overwrite all dimensions on this MetricsLogger instance.
*
* @param dimensionSets
* @see [CloudWatch Dimensions](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_concepts.html#Dimension)
*
* @param {Array<Record<string, string>> | Record<string, string>} dimensionSetOrSets Dimension sets to overwrite with
* @param {boolean} [useDefault=false] whether to use default dimensions
*/
public setDimensions(dimensionSet: Record<string, string>, useDefault: boolean): MetricsLogger;
public setDimensions(dimensionSet: Record<string, string>): MetricsLogger;
public setDimensions(dimensionSets: Array<Record<string, string>>, useDefault: boolean): MetricsLogger;
public setDimensions(dimensionSets: Array<Record<string, string>>): MetricsLogger;
public setDimensions(
dimensionSetOrSets: Array<Record<string, string>> | Record<string, string>,
useDefault = false,
): MetricsLogger {
if (Array.isArray(dimensionSetOrSets)) {
this.context.setDimensions(dimensionSetOrSets, useDefault);
} else {
this.context.setDimensions([dimensionSetOrSets], useDefault);
}

return this;
}

/**
* Clear all custom dimensions on this MetricsLogger instance
*
* @param useDefault indicates whether default dimensions should be used
*/
public setDimensions(...dimensionSets: Array<Record<string, string>>): MetricsLogger {
this.context.setDimensions(dimensionSets);
public resetDimensions(useDefault: boolean): MetricsLogger {
this.context.resetDimensions(useDefault);
return this;
}

Expand Down
180 changes: 180 additions & 0 deletions src/logger/__tests__/MetricsLogger.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -186,6 +186,129 @@ describe('successful', () => {
expect(actualValue).toBe(expectedValue);
});

test('setDimensions with default dimensions enabled', async () => {
// arrange
const expectedKey = 'key';
const expectedValue = 'value';
const dimensions: Record<string, string> = {};
dimensions[expectedKey] = expectedValue;

// act
logger.putDimensions({ foo: 'bar' });
logger.setDimensions(dimensions, true);
await logger.flush();

// assert
expect(sink.events).toHaveLength(1);

const dimensionSets = sink.events[0].getDimensions();
expect(dimensionSets).toHaveLength(1);

const dimension = dimensionSets[0];
expect(Object.keys(dimension).length).toBe(4);
expect(dimension[expectedKey]).toBe(expectedValue);
});

test('setDimensions with default dimensions disabled', async () => {
// arrange
const expectedKey1 = 'key1';
const expectedValue1 = 'value1';
const expectedKey2 = 'key2';
const expectedValue2 = 'value2';
const dimensions1: Record<string, string> = {};
const dimensions2: Record<string, string> = {};
dimensions1[expectedKey1] = expectedValue1;
dimensions2[expectedKey2] = expectedValue2;

const dimensionsList = [dimensions1, dimensions2];

// act
logger.putDimensions({ foo: 'bar' });
logger.setDimensions(dimensionsList, false);
await logger.flush();

// assert
expect(sink.events).toHaveLength(1);

const dimensionSets = sink.events[0].getDimensions();
expect(dimensionSets).toHaveLength(2);

const dimension1 = dimensionSets[0];
const dimension2 = dimensionSets[1];
expect(Object.keys(dimension1).length).toBe(1);
expect(Object.keys(dimension2).length).toBe(1);

expect(dimension1[expectedKey1]).toBe(expectedValue1);
expect(dimension2[expectedKey2]).toBe(expectedValue2);
});

test('setDimensions with empty dimension set', async () => {
// arrange
const dimensions: Record<string, string> = {};

// act
logger.putDimensions({ foo: 'bar' });
logger.setDimensions(dimensions);
await logger.flush();

// assert
expect(sink.events).toHaveLength(1);

const dimensionSets = sink.events[0].getDimensions();
expect(dimensionSets).toHaveLength(1);
expect(Object.keys(dimensionSets[0]).length).toBe(0);
});

test('resetDimensions with default dimensions enabled', async () => {
// arrange
const expectedKey = 'key';
const expectedValue = 'value';
const dimensions: Record<string, string> = {};
dimensions[expectedKey] = expectedValue;

// act
logger.putDimensions({ foo: 'bar' });
logger.resetDimensions(true);
logger.putDimensions(dimensions);
await logger.flush();

// assert
expect(sink.events).toHaveLength(1);
const dimensionSets = sink.events[0].getDimensions();

expect(dimensionSets).toHaveLength(1);
const dimension = dimensionSets[0];

expect(Object.keys(dimension).length).toBe(4);
expect(dimension[expectedKey]).toBe(expectedValue);
expect(dimension['foo']).toBeUndefined();
});

test('resetDimensions with default dimensions disabled', async () => {
// arrange
const expectedKey = 'key';
const expectedValue = 'value';
const dimensions: Record<string, string> = {};
dimensions[expectedKey] = expectedValue;

// act
logger.putDimensions({ foo: 'bar' });
logger.resetDimensions(false);
logger.putDimensions(dimensions);
await logger.flush();

// assert
expect(sink.events).toHaveLength(1);
const dimensionSets = sink.events[0].getDimensions();

expect(dimensionSets).toHaveLength(1);
const dimension = dimensionSets[0];

expect(Object.keys(dimension).length).toBe(1);
expect(dimension[expectedKey]).toBe(expectedValue);
expect(dimension['foo']).toBeUndefined();
});

test('can set namespace', async () => {
// arrange
const expectedValue = faker.random.word();
Expand Down Expand Up @@ -371,6 +494,63 @@ describe('successful', () => {
}
});

test('configure flush() to not preserve custom dimensions', async () => {
// arrange
const expectedKey = 'dim';
const expectedValue = 'value';
const dimensions: Record<string, string> = {};
dimensions[expectedKey] = expectedValue;

// act
logger.flushPreserveDimensions = false;
logger.putDimensions({ foo: 'bar' });
await logger.flush();

logger.putDimensions(dimensions);
await logger.flush();

// assert
expect(sink.events).toHaveLength(2);

const evt1 = sink.events[0];
expect(Object.keys(evt1.getDimensions()[0]).length).toBe(4);
expect(evt1.getDimensions()[0]['foo']).toBe('bar');

const evt2 = sink.events[1];
expect(Object.keys(evt2.getDimensions()[0]).length).toBe(4);
expect(evt2.getDimensions()[0]['foo']).toBeUndefined();
expect(evt2.getDimensions()[0][expectedKey]).toBe(expectedValue);
});

test('configure flush() to not preserve any dimenions', async () => {
// arrange
logger.flushPreserveDimensions = false;
logger.resetDimensions(false);

// act
logger.putDimensions({ foo: 'bar' });
await logger.flush();

logger.putDimensions({ baz: 'qux' });
await logger.flush();

await logger.flush();

// assert
expect(sink.events).toHaveLength(3);

const evt1 = sink.events[0];
expect(Object.keys(evt1.getDimensions()[0]).length).toBe(1);
expect(evt1.getDimensions()[0]['foo']).toBe('bar');

const evt2 = sink.events[1];
expect(Object.keys(evt2.getDimensions()[0]).length).toBe(1);
expect(evt2.getDimensions()[0]['baz']).toBe('qux');

const evt3 = sink.events[2];
expect(evt3.getDimensions().length).toBe(0);
});

const expectDimension = (key: string, value: string) => {
expect(sink.events).toHaveLength(1);
const dimensionSets = sink.events[0].getDimensions();
Expand Down