ng test is inconsistent, running the wrong tests

[jplew]

resubmitting from: angular/angular#22632

I'm submitting a...

[ ] Regression (a behavior that used to work and stopped working in a new release)
[x] Bug report  
[ ] Feature request
[ ] Documentation issue or request
[ ] Support request => Please do not submit support request here, instead see https://github.com/angular/angular/blob/master/CONTRIBUTING.md#question

Current behavior

TL;DR: My test succeeds on the first try, then fails on all subsequent retries.

After I execute ng test from the command-line, I expect my test (should check server if fragment is valid) to pass successfully. It does:

Next, while ng test is still watching my files, I make a trivial change to my spec test (wordbank.service.spec.ts) by inserting a blank line. This relaunches the test, except this time, the same test is now failing:

From now on, while test is watching, any change I make results in the same error message, and I can't get the test to pass. The console reports something strange:

GameMessageComponent should check server if fragment is valid FAILED
Error: Expected one matching request for criteria "Match method: GET, URL: (any)", found none. at
HttpClientTestingBackend.expectOne
(webpack:///./node_modules/@angular/common/esm5/http/testing.js?:427:19)
at UserContext.eval
(webpack:///./src/app/services/wordbank.service.spec.ts?:34:28)

Why is it talking about GameMessageComponent? I am focusing on this single test using fit(), therefore I am not testing my GameMessageComponent in any way. I am only interested in WordbankService. My Karma window is reporting the correct describe() target (WordbankService), but the terminal in which I launched the ng test is saying I'm trying to test GameMessageComponent, which is a component in my app that has nothing to do with this service.

The code for my test (wordbank.service.spec.ts): https://gist.github.com/jplew/140c069c48cf48e8ad48e285d7b4cdd0
The code for the service I'm testing (wordbank.service.ts): https://gist.github.com/jplew/2dfac1d7476ae080166ead216e9e0126
The code for game-message.component.spec.ts: https://gist.github.com/jplew/66081e668e78c3e384c5d4dd72abba9f

I am using fit() rather than it() in order to isolate my test. However, if I replace fit with it and run all 57 of my tests, I get a similar result. My WordbankService test succeeds on the initial pass. The console reports:

Executed 57 of 57 (10 FAILED) (4.876 secs / 4.67 secs) // (some of my other tests are failing right now for unrelated reasons)

No failure was reported for WordBankService on the initial go. Then, I add a space or whatever to one of my files. The test automatically reruns and I get:

WordbankService should check server if fragment is valid FAILED
Error: Expected one matching request for criteria "Match method: GET, URL: (any)", found none...
Executed 57 of 57 (13 FAILED) (4.484 secs / 4.269 secs)

Expected behavior

I expect the same test to yield the same result each time it is run, all else being equal.

Minimal reproduction of the problem with instructions

I'm not sure how to reproduce it, because out of all of my tests, this is the only one that triggers this bug. There is probably some idiosyncrasy in my test that is causing this strange behavior.

What is the motivation / use case for changing the behavior?

This makes my tests completely unreliable. They work as expected when I first launch ng test. But every subsequent test is reporting false information. It appears the result does not correspond to the actual test.
In order for me to test properly, I need to run ng test every single time I make a change.

Environment

Angular CLI version: 1.7.0 & 1.7.3
Angular version: 5.2.7


Browser:
- [x] Chrome (desktop) version Chrome 65.0.3325 
- [ ] Chrome (Android) version XX
- [ ] Chrome (iOS) version XX
- [ ] Firefox version XX
- [ ] Safari (desktop) version XX
- [ ] Safari (iOS) version XX
- [ ] IE version XX
- [ ] Edge version XX
 
For Tooling issues:
- Node version: 9.5.0  
- Platform:  Mac OS X 10.12.6

Others:

[jplew]

I am able to replicate the bug in some other tests. This time, I got all my tests to pass on the first run:

On the second run, however:

The new errors are coming from word.service.spec.ts which follows a similar testing pattern (https://gist.github.com/jplew/8cb79202126c4ebaaa9697d2273b4c40). I must be doing something very wrong with req.flush(x) or expectOne({ method: 'GET' }).

[filipesilva]

Thanks for reporting this issue. This issue was originally reported a long time ago and since then we've had many releases, one of which might have addressed this problem. Please update to the most recent Angular CLI version.

If the problem persists after upgrading, please open a new issue, provide a simple repository reproducing the problem, and describe the difference between the expected and current behavior.

[angular-automatic-lock-bot]

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

_{This action has been performed automatically by a bot.}