Coverage report incorrect #480

ghost · 2022-06-22T01:36:41Z

I've been trying to get jest to work properly with a Vue 2 + Vite project. Jest reports pass/fail as I would expect, but the coverage report seems incorrect. Link to the repro here: https://github.com/qkjosh/vue-jest-coverage

package.json

"devDependencies": {
    "@babel/core": "7.18.5",
    "@babel/preset-env": "^7.18.2",
    "@vue/test-utils": "^1.2.2",
    "@vue/vue2-jest": "^28.0.0",
    "babel-jest": "28.1.1",
    "jest": "28.1.1",
    "jest-environment-jsdom": "28.1.1",
    "vite": "2.9.12",
    "vite-plugin-vue2": "^2.0.1"
  }

Given an incomplete test suite, the coverage report indicates 100% coverage and seems to treat the whole SFC component as one statement. Do I have something completely misconfigured, or is the coverage report broken?

The text was updated successfully, but these errors were encountered:

dreid · 2022-06-24T16:22:56Z

I've also been investigating this, in our case we do not use vite, but it seems to be related to babel 7 switching from source-map to @jridgewell/gen-mapping here:

https://github.com/babel/babel/blob/main/CHANGELOG.md#v71710-2022-04-29

ghost · 2022-06-24T21:13:59Z

After some poking around I also narrowed it down to something in babel (noticed coverage working again after downgrading a few packages). Thanks for that link. Wonder if this is outside scope of vue-jest then?

dreid · 2022-06-24T22:53:08Z

I'm not really sure, I tried stepping through https://github.com/vuejs/vue-jest/blob/master/packages/vue2-jest/lib/generate-source-map.js and it seemed like it was not able to find the originalLine for many lines in the file.

And from investigating the jest transforms cache an indicator of this problem is that the source map for the version with incorrect coverage does not have as much mapping data as the version with correct coverage.

Its possible that vue-jest could address this by switching to https://github.com/jridgewell/trace-mapping instead of source-map or by using https://www.npmjs.com/package/@ampproject/remapping ?

AnthonyRuelle · 2022-06-28T12:17:02Z

same problem on my application for several weeks. It's really very blocking. Do we have a solution?

howard-repos · 2022-07-05T02:02:01Z

Any update?

adelsagemilang · 2022-07-07T10:17:00Z

I've also been investigating this, in our case we do not use vite, but it seems to be related to babel 7 switching from source-map to @jridgewell/gen-mapping here:

https://github.com/babel/babel/blob/main/CHANGELOG.md#v71710-2022-04-29

I have downgraded my @babel/core to 7.17.9 just before switched to @jridgewell/gen-mapping on 7.17.10 but it seems no luck

Anyone with a proposed solution much appreciated

udebella · 2022-07-12T10:24:12Z

I've also been investigating this, in our case we do not use vite, but it seems to be related to babel 7 switching from source-map to @jridgewell/gen-mapping here:
https://github.com/babel/babel/blob/main/CHANGELOG.md#v71710-2022-04-29

I have downgraded my @babel/core to 7.17.9 just before switched to @jridgewell/gen-mapping on 7.17.10 but it seems no luck

Anyone with a proposed solution much appreciated

Downgrading to 7.17.9 does not work because in @babel/core they did not freeze their dependencies. If you have a project that work, just make sure you don't upgrade @babel/core dependencies: removing and recreating your package-lock.json leads to issue. :(

My really ugly workaround is to to prevent @babel/generator to update to a buggy version. So I added inside my package.json:

  "overrides": {
    "@babel/generator": "<7.17.9"
  }

Then doing an npm install fixes broken package-lock.json.
Be careful with this fix, because it forces a version of one dependency inside @babel/core which seems to rely on consistent versions : this workaround could not work or break your project: don't forget to remove the override part from your package.json when this issue is fixed.

Maybe knowing where the issue comes from more precisely will help maintainers?

Edit: This solution works only for npm > 8.3.0 as overrides were introduced in this version. If you still use an older version, you can use @AnthonyRuelle solution
If you use yarn, you can use @BenShelton solution

bdeo · 2022-07-22T12:36:17Z

I'm also having this issue. Is the above PR being considered for a resolution here?

LG0012 · 2022-07-22T13:35:14Z

Same issue

okgo · 2022-07-26T06:33:47Z

Same issue

AnthonyRuelle · 2022-07-26T07:11:14Z

Same issue ... 😩 Le mar. 26 juil. 2022 à 08:34, Yurii Rekechuk ***@***.***> a écrit :

…

Same issue — Reply to this email directly, view it on GitHub <#480 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AWJ7JAH32ITDVPH2NZILRBTVV6BFPANCNFSM5ZOORPAA> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

BenShelton · 2022-07-26T14:19:46Z

Using yarn we were able to fix this by locking both versions of @babel/core and @babel/generator to 7.17.9 (thanks to the suggestion from @udebella )

"devDependencies": {
  "@babel/core": "7.17.9"
},
"resolutions": {
  "@babel/generator": "7.17.9"
},

We only use babel for jest so this doesn't have any other side effects for us, YMMV.

LG0012 · 2022-08-05T05:00:15Z

Any solution how to fix this? Non of the SFC file <script> block code is covered. Only works with .js mixin files.

"dependencies": {
    "babel-jest": "^28.1.3",
    "core-js": "^3.19.3",
    "jest": "^28.1.3",
    "jest-environment-jsdom": "^28.1.3",
    "nuxt": "^2.15.8",
    "vue": "^2.6.14",
    "vue-server-renderer": "^2.6.14",
    "vue-template-compiler": "^2.6.14",
    "vuetify": "^2.6.1",
    "vuex": "^3.6.2",
    "webpack": "^4.46.0"
},
"devDependencies": {
    "@babel/core": "7.18.10",
    "@babel/plugin-proposal-class-properties": "^7.14.5",
    "@babel/preset-env": "^7.15.6",
    "@babel/preset-react": "^7.14.5",
    "@nuxt/types": "~2.15.0",
    "@nuxtjs/router-extras": "^1.1.1",
    "@nuxtjs/vuetify": "^1.12.3",
    "@types/jest": "^27.0.2",
    "@vue/test-utils": "^1.2.2",
    "@vue/vue2-jest": "^28.0.1",
    "babel-core": "^7.0.0-bridge.0"
}

john-maddock · 2022-08-19T10:25:13Z

any potential fixes for this?.... having the same issue :(

AnthonyRuelle · 2022-08-19T11:56:59Z

Waiting for the fix, in the package.json:

Add this into scripts section:
"preinstall": "npx npm-force-resolutions"
Add this section to force babel generator version:
"resolutions": {
"@babel/generator": "7.17.9"
},
On my side it worked and it will be fine while waiting for the version upgrade with the fixes

aztran · 2022-08-23T04:17:10Z

Waiting for the fix, in the package.json:

Add this into scripts section:
"preinstall": "npx npm-force-resolutions"

Add this section to force babel generator version:
"resolutions": {
"@babel/generator": "7.17.9"
},
On my side it worked and it will be fine while waiting for the version upgrade with the fixes

I still having this issue. hope it can be solved ASAP

GrRivero · 2022-09-21T14:18:31Z

I have this issue:

New Project with vue 2 + babel + typescript + jest doesn't work code coverage vue-cli#7270

I tried all the workarounds in this post and no one works for me. Do we have a solution?

datlc-1387 · 2022-10-05T08:22:45Z

I tried downgrade babel version and add sourceMaps: true to file jest.config. It worked.
swc-project/swc#3854 (comment)

GrRivero · 2022-10-05T08:27:21Z

I tried downgrade babel version and add sourceMaps: true to file jest.config. It worked.

Hi! sorry, can you share your package.json or versions of babel?

Thnx

schirrel · 2022-10-13T19:51:06Z

Any update on it?

xdubx · 2022-10-21T11:00:04Z

same bug over here. I even get code red marked that has been demonstrably tested. on vue3

pazderka · 2022-11-08T08:28:24Z

Is there any other workaround? Downgrading @babel/core and @babel/generator didn't work for me, the coverage is still incorrect. Downgrading jest and @vue/vue2-jest to versions 28 solves it partly, it generates valid coverage, but some files are missing for some reason...

EDIT: Installing jest@28 and @vue/vue2-jest@28 along with adding coverageProvider: v8 to jest config did the trick for me.

EvanWoodard · 2022-11-22T04:16:28Z

Well, I have a weird workaround after digging into the @vue/vue2-jest code a bit.

So, it seems like the old template compiler (@vue/component-compiler-utils) and the new one (vue/compiler-sfc) are giving the exact same output except for the source mapping.

Vue2-jest has a weird handler for which one of these compilers it uses. Basically it does a try/catch, where if vue/compiler-sfc exists, it will assume you have a more modern version of vue and use the sfc compiler, and if it errors, it will use the component-compiler-utils. (vue2-jest/lib/process.js) - line 17-22.

What ended up working for me, at least partially (I may still run into some unforeseen error), was to delete the try block and only use the code that was in the catch. So, only use @vue/component-compiler-utils. However, this isn't a persistent solution. Instead, I found a hack to trick the module resolution.

(I am using yarn)
In package.json

"dependencies": {
   ...
},
"resolutions": {
    "@vue/compiler-sfc": "link:./node_modules/.cache/null"
}

Basically, it pipes the require('@vue/compiler-sfc') to null, which then throws the error and uses the other compiler.

My expectation is that the Vue team will eventually align the two compilers, but in the meantime, this seems to fix my jest coverage.

NOTE: This is also using the @vue/cli-plugin-<whatever>@5.0.8 packages and @vue/[email protected]. I'm not sure if this same solution works on previous versions of either/both. Additionally, I needed to change the jest.config to use 'v8' as the coverage provider.

rklos · 2022-11-25T18:47:34Z

Using yarn we were able to fix this by locking both versions of @babel/core and @babel/generator to 7.17.9 (thanks to the suggestion from @udebella )
"devDependencies": {
  "@babel/core": "7.17.9"
},
"resolutions": {
  "@babel/generator": "7.17.9"
},
We only use babel for jest so this doesn't have any other side effects for us, YMMV.

This fix has stopped working quite time ago.
With step by step method I have found that the recent update of the istanbul-lib-instrument had broken something (there is an issue: istanbuljs/istanbuljs#707).

So the next iteration of the workaround is (for npm >= 8):

"overrides": {
  "@babel/core": "7.17.9",
  "@babel/generator": "7.17.9",
  "istanbul-lib-instrument": "5.2.0"
}

GrRivero · 2022-11-28T08:10:46Z

So the next iteration of the workaround is (for npm >= 8):

"overrides": {
  "@babel/core": "7.17.9",
  "@babel/generator": "7.17.9",
  "istanbul-lib-instrument": "5.2.0"
}

This workaround works for me with:

"dependencies": {
    "@types/cli-color": "^2.0.2",
    "@types/i18n": "^0.13.5",
    "ajv": "^8.11.2",
    "axios": "^0.27.2",
    "cli-color": "^2.0.3",
    "colors": "^1.4.0",
    "core-js": "^3.26.1",
    "i18next": "^22.0.6",
    "js-beautify": "^1.14.7",
    "jwt-decode": "^3.1.2",
    "lodash.clonedeep": "^4.5.0",
    "register-service-worker": "^1.7.2",
    "sanitize-html": "^2.7.3",
    "uuid": "^9.0.0",
    "vue": "^2.7.14",
    "vue-class-component": "^7.2.6",
    "vue-i18n": "^8.28.2",
    "vue-json-viewer": "^2.2.22",
    "vue-pdf-embed": "^1.1.5",
    "vue-property-decorator": "^9.1.2",
    "vue-router": "^3.6.5",
    "vue-simple-markdown": "^1.1.5",
    "vue-virtual-scroller": "^1.1.2",
    "vuedraggable": "^2.24.3",
    "vuejs-ace-editor": "^1.0.1",
    "vuetify": "^2.6.12",
    "vuex": "3.6.2",
    "vuex-class": "^0.3.2",
    "winston": "^3.8.2"
},
"devDependencies": {
    "@4tw/cypress-drag-drop": "^2.2.1",
    "@mdi/font": "^7.0.96",
    "@playwright/test": "^1.28.0",
    "@types/concat-stream": "^2.0.0",
    "@types/jest": "^28.1.8",
    "@types/js-beautify": "^1.13.3",
    "@types/jsdom": "^20.0.1",
    "@types/lodash.clonedeep": "^4.5.7",
    "@types/mssql": "^8.1.1",
    "@types/node": "^18.11.9",
    "@types/sanitize-html": "^2.6.2",
    "@types/uuid": "^8.3.4",
    "@typescript-eslint/eslint-plugin": "^5.43.0",
    "@typescript-eslint/parser": "^5.43.0",
    "@vue/cli-plugin-babel": "^4.5.19",
    "@vue/cli-plugin-eslint": "^4.5.19",
    "@vue/cli-plugin-pwa": "^4.5.19",
    "@vue/cli-plugin-typescript": "^4.5.19",
    "@vue/cli-plugin-unit-jest": "^4.5.19",
    "@vue/cli-service": "^4.5.19",
    "@vue/eslint-config-prettier": "^6.0.0",
    "@vue/eslint-config-typescript": "^10.0.0",
    "@vue/test-utils": "^1.3.3",
    "concat-stream": "^2.0.0",
    "cypress": "9.7.0 ",
    "eslint": "^6.8.0",
    "eslint-plugin-prettier": "^3.4.1",
    "eslint-plugin-tsdoc": "^0.2.17",
    "eslint-plugin-vue": "^8.7.1",
    "jest-fetch-mock": "^3.0.3",
    "jest-junit": "^14.0.1",
    "jest-slow-test-reporter": "1.0.0",
    "jest-sonar-reporter": "^2.0.0",
    "jsdom": "^20.0.2",
    "jsdom-worker": "^0.3.0",
    "mssql": "^9.0.1",
    "node-fetch": "^2.6.7",
    "playwright-chromium": "^1.28.0",
    "prettier": "^2.7.1",
    "sass": "^1.56.1",
    "sass-loader": "^10.4.1",
    "ts-jest": "^26.5.6",
    "typescript": "4.6.4",
    "vue-cli-plugin-vuetify": "^2.5.8",
    "vue-template-compiler": "^2.7.14",
    "vuetify-loader": "^1.9.2"
},
"overrides": {
    "@babel/core": "7.17.9",
    "@babel/generator": "7.17.9",
    "istanbul-lib-instrument": "5.2.0"
}

Only the breakpoints in script vue dont works fine when launch jest.

Thanks all!!!

xdubx · 2022-11-29T11:25:19Z

"overrides": {
  "@babel/core": "7.17.9",
  "@babel/generator": "7.17.9",
  "istanbul-lib-instrument": "5.2.0"
}

works on vue3 too. thx @GrRivero @rklos

Envoy49 · 2022-12-23T00:34:58Z

This solved my issue #522, probably worth trying here as well?

Upgrading couple of packages and editing jest.config.ts fixed the issue with coverage.

package.json

"@types/jest": "^29.2.2",
"jest": "^29.3.1",
"jest-environment-jsdom": "29.3.1",
"ts-jest": "^29.0.3",

jest.config.ts

moduleNameMapper: {
  uuid: require.resolve("uuid"),
},
coverageProvider: "v8",

Martin-Finereport · 2022-12-27T07:48:40Z

update jest and @vue/vue2-jest to version 28.1.0 also did the trick for me

AllanOricil · 2023-01-05T10:05:36Z

Why Im I seeing this error message TypeError: Cannot read properties of undefined (reading 'testEnvironmentOptions')
It makes no sense, because I installed jest-environment-jsdom

Envoy49 · 2023-01-05T10:19:42Z

Why Im I seeing this error message TypeError: Cannot read properties of undefined (reading 'testEnvironmentOptions') It makes no sense, because I installed jest-environment-jsdom

This issue is not relevant to this topic at all !!!

AllanOricil · 2023-01-05T10:46:19Z

@Envoy49 This topic talks about fixes for coverage reports. After following one of many fixes from this topic, the coverage reports issue was gone. I updgraded Jest from v27 to v28 || v29, and this error TypeError: Cannot read properties of undefined (reading 'testEnvironmentOptions') started. If it happened to me, it will surelly happen to other people. I had to rollback Jest to v27 because installing jest-environment-json did not fix the issue, as explained in jest's official migration doc.

AllanOricil · 2023-01-05T11:18:46Z

relates: vuejs/vue-cli#7129

petenattress · 2023-06-15T14:30:44Z

I've tried various solutions listed in this thread. Setting sourceMaps: true in our babel.config.js (which is only used by vue-jest) worked for me (on v28 of Jest and vue-jest).

pmrotule · 2023-08-07T12:57:33Z

Personally, the coverage started to be all wrong after I upgraded from Jest v26 to v29 (using Vue 2). All tests are passing, but the coverage is reporting uncovered lines from the template or simply lines that don't exist (i.e. reporting 35-66 as uncovered lines on a vue file having only 44 lines).

I tested all the workarounds mentioned in this issue, none of them has worked. Funny enough, the only thing that worked in my case was reverting the changes made in #486, then all went back to normal: my tests were passing and the coverage report was reporting the right uncovered lines.

Does anyone else experience the same? You can test the revert by changing @vue/vue2-jest version to the following version:

"@vue/vue2-jest": "pmrotule/vue-jest#644ed67e1d7f08729e82518e6033fae6f248d75e"

We're using Vue 2.6.12 and @vue/composition-api 1.7.1

tntim96 · 2023-08-07T22:57:38Z

Not sure about others, but at this stage we're planning to (hopefully) solve this by upgrading to Vue3 ¯_(ツ)_/¯

patrick-g-p · 2024-04-25T18:48:41Z

My coworker noticed this issue still persists with "@vue/vue3-jest": "^29.2.6". Our team was able to fix the odd coverage behavior by adding "coverageProvider": "v8" to our app's Jest config file

pazderka · 2024-04-25T19:40:39Z

My coworker noticed this issue still persists with "@vue/vue3-jest": "^29.2.6". Our team was able to fix the odd coverage behavior by adding "coverageProvider": "v8" to our app's Jest config file

Yea, v8 works when it comes to coverage, but it is a huge tradeoff in favor of test speed and coverage report accuracy… would be really great to fix babel provider.

NikhilVerma mentioned this issue Jul 22, 2022

Fix source maps for vue2-jest #486

Merged

LG0012 mentioned this issue Aug 12, 2022

How to collect coverage from <template> section? #476

Open

okoell mentioned this issue Aug 25, 2022

New Project with vue 2 + babel + typescript + jest doesn't work code coverage vuejs/vue-cli#7270

Open

GrRivero mentioned this issue Nov 28, 2022

SFC Vue files with tests disappear from test coverage reports #422

Open

xdubx mentioned this issue Nov 29, 2022

Coverage (and debugging from IDE) is broken with @vue/vue3-jest #514

Open

cpontvieux-systra mentioned this issue Dec 9, 2022

test utils coverage report broken for Vue2 projects vuejs/vue-test-utils#1977

Closed

This was referenced Dec 22, 2022

Coverage report is complaining about non existing lines of code #522

Closed

Coverage report is complaining about non existing lines of code nuxt/bridge#675

Closed

ulfgebhardt mentioned this issue Jun 1, 2023

refactor(webapp): fix coverage Ocelot-Social-Community/Ocelot-Social#6361

Merged

1 task

StephenHulme mentioned this issue Feb 16, 2024

Confirm coverage of Vue component sanger/limber#1585

Closed

davidclaveau mentioned this issue Mar 19, 2024

Rebuild Testing Suite - Part 2 - Frontend bcgov/gwells#2141

Open

1 task

StephenHulme mentioned this issue Jul 22, 2024

Y24-207 - [BUG] Incorrect test coverage from Vue Single-File Components sanger/limber#1801

Closed

Coverage report incorrect #480

Coverage report incorrect #480

Comments

ghost commented Jun 22, 2022 • edited by ghost Loading

package.json

dreid commented Jun 24, 2022

ghost commented Jun 24, 2022

dreid commented Jun 24, 2022

AnthonyRuelle commented Jun 28, 2022

howard-repos commented Jul 5, 2022

adelsagemilang commented Jul 7, 2022

udebella commented Jul 12, 2022 • edited Loading

bdeo commented Jul 22, 2022

LG0012 commented Jul 22, 2022

okgo commented Jul 26, 2022

AnthonyRuelle commented Jul 26, 2022 via email

BenShelton commented Jul 26, 2022

LG0012 commented Aug 5, 2022

john-maddock commented Aug 19, 2022

AnthonyRuelle commented Aug 19, 2022

aztran commented Aug 23, 2022

GrRivero commented Sep 21, 2022

datlc-1387 commented Oct 5, 2022 • edited Loading

GrRivero commented Oct 5, 2022

schirrel commented Oct 13, 2022

xdubx commented Oct 21, 2022 • edited Loading

pazderka commented Nov 8, 2022 • edited Loading

EvanWoodard commented Nov 22, 2022 • edited Loading

rklos commented Nov 25, 2022 • edited Loading

GrRivero commented Nov 28, 2022

xdubx commented Nov 29, 2022 • edited Loading

Envoy49 commented Dec 23, 2022 • edited Loading

Martin-Finereport commented Dec 27, 2022

AllanOricil commented Jan 5, 2023 • edited Loading

Envoy49 commented Jan 5, 2023

AllanOricil commented Jan 5, 2023

AllanOricil commented Jan 5, 2023

petenattress commented Jun 15, 2023

pmrotule commented Aug 7, 2023

tntim96 commented Aug 7, 2023

patrick-g-p commented Apr 25, 2024

pazderka commented Apr 25, 2024

ghost commented Jun 22, 2022 •

edited by ghost

Loading

udebella commented Jul 12, 2022 •

edited

Loading

datlc-1387 commented Oct 5, 2022 •

edited

Loading

xdubx commented Oct 21, 2022 •

edited

Loading

pazderka commented Nov 8, 2022 •

edited

Loading

EvanWoodard commented Nov 22, 2022 •

edited

Loading

rklos commented Nov 25, 2022 •

edited

Loading

xdubx commented Nov 29, 2022 •

edited

Loading

Envoy49 commented Dec 23, 2022 •

edited

Loading

AllanOricil commented Jan 5, 2023 •

edited

Loading