Skip to content

Feature/deduplication #58

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
Oct 4, 2016
Merged

Feature/deduplication #58

Changes from 5 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
fd0f3bb
Start work to make the deduplication plugin work like the .NET one
srijken May 13, 2016
0fc7f0c
Cleanup and make sure we have stacks
srijken Sep 19, 2016
7c12490
Better hashcode calculation
srijken Sep 19, 2016
c609e4c
Merge master
srijken Sep 26, 2016
1e57e9e
Use the configured error parser
srijken Sep 26, 2016
93b29f0
Fix lint error
srijken Sep 27, 2016
8a1ebc3
Merge master
srijken Sep 27, 2016
0f8261a
Files from previous commit; merge from master
srijken Sep 27, 2016
e41652d
Increase timeouts
srijken Sep 27, 2016
b3419f1
load default settings / injects in testsuite
srijken Sep 27, 2016
4cfc5fa
use fake XMLHttpRequests
srijken Sep 27, 2016
fff0f69
Add sinon
srijken Sep 27, 2016
09cd805
Remove unused variable
srijken Sep 27, 2016
06d81f2
Create a real exception in the reference id test
srijken Sep 27, 2016
ef44761
Get rid of higher timeout values
srijken Sep 27, 2016
e2968cc
use the same priority as the .NET client
srijken Sep 28, 2016
1bbe4bf
Use += instead of =
srijken Sep 28, 2016
2a35c09
Also take the message into account, to deal with errors without stack…
srijken Sep 28, 2016
f7c9b2d
Add logging
srijken Sep 28, 2016
b26971a
correct the case where no stack trace is found
srijken Sep 28, 2016
bf1baab
use 30 secs as deduplication interval
srijken Oct 3, 2016
be062f7
Add check on error.message
srijken Oct 3, 2016
63c94a1
Fix lint messages
srijken Oct 3, 2016
66f40c3
Fix unit test
srijken Oct 3, 2016
cdf55c8
Run build
srijken Oct 3, 2016
bb0414d
Change some .info calls to .trace
srijken Oct 4, 2016
9651726
Merge branch 'master' into feature/deduplication
srijken Oct 4, 2016
ef5f477
ran the build
srijken Oct 4, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions src/plugins/default/DuplicateCheckerPlugin.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { EventPluginContext } from '../EventPluginContext';
import { Utils } from '../../Utils';

export class DuplicateCheckerPlugin implements IEventPlugin {
public priority: number = 90;
public priority: number = 1010;
public name: string = 'DuplicateCheckerPlugin';

private _mergedEvents: MergedEvent[] = [];
Expand All @@ -28,8 +28,10 @@ export class DuplicateCheckerPlugin implements IEventPlugin {
let hashCode = 0;

while (error) {
if (error.stack_trace && error.stack_trace.length) {
hashCode = (hashCode * 397) ^ Utils.getHashCode(JSON.stringify(error.stack_trace));
hashCode += (hashCode * 397) ^ Utils.getHashCode(error.message);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think we should be checking the message length as well? Os should getHashCode method check that?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added it, just in case :)


if (error.stack_trace && error.stack_trace.length) {
hashCode += (hashCode * 397) ^ Utils.getHashCode(JSON.stringify(error.stack_trace));
}
error = error.inner;
}
Expand All @@ -39,6 +41,7 @@ export class DuplicateCheckerPlugin implements IEventPlugin {

let error = context.event.data['@error'];
let hashCode = getHashCode(error);

if (!hashCode) {
return;
}
Expand All @@ -50,16 +53,19 @@ export class DuplicateCheckerPlugin implements IEventPlugin {
if (merged) {
merged.incrementCount(count);
merged.updateDate(context.event.date);
context.log.info("Ignoring duplicate event with hash: " + hashCode);
context.cancelled = true;
return;
}

if (this._processedHashcodes.some(h => h.hash === hashCode && h.timestamp >= (now - this._interval))) {
context.log.info("Adding event with hash: " + hashCode);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a trace level message

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only have these:

  info(message: string): void;
  warn(message: string): void;
  error(message: string): void;

this._mergedEvents.push(new MergedEvent(hashCode, context, count));
context.cancelled = true;
return;
}

context.log.info("Enqueueing event with hash: " + hashCode + "to cache.");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a trace level message

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only have:

  info(message: string): void;
  warn(message: string): void;
  error(message: string): void;

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Guess we could add trace (https://github.com/DeveloperToolsWG/console-object/blob/master/api.md#consolelogobject--object-) doesn't look like it's supported everywhere. We'd have to wrap it like we are doing the other ones I guess.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

this._processedHashcodes.push({ hash: hashCode, timestamp: now });
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would be nice if we added some minimal logging to this class. Makes it much easier to debug issues

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done


// Only keep the last 50 recent errors.
Expand Down