Save coverage for each page in a test

[grantbdev]

Save coverage for each page visited in a test by keeping a reference to the coverage for each app window loaded and then saving the coverage at the end after coverage is calculated.

The implementation in the PR keeps the behavior of saving coverage after each test since that is the current behavior and I believe it would be necessary for #4. However, I think it would also be possible to initialize the array and event listener in before() and only save coverage in after() if that is desired instead.

Closes #69

[CLAassistant]

All committers have signed the CLA.

[bahmutov]

Hmm, @grantbdev I love it. I just need an example site showing this in action.

[grantbdev]

@bahmutov I can do that. Do you have any preferences on what I base it on or is any minimum repo that has different JS (be it jQuery, React, etc.) loaded/executed across windows sufficient?

[grantbdev]

@bahmutov clarification on what you need for an example site would be appreciated 😄

[matinfo]

very interesting, let go!

[bahmutov]

@grantbdev as an example I would love to see a simple site with 2 pages, each loading its own code. The test could open the first page, then click on a link to go to the second page. The collected coverage object should show coverage for the code loaded on the first page and the code loaded on the second page

[grantbdev]

@bahmutov PR rebased and updated with testing sample code in this package's test suite if that is OK with you.

[3590Duncan]

Any updates on when this could potentially be released?

[bahmutov]

Looks good, I will merge this, test locally and will release it.

[bahmutov]

🎉 This PR is included in version 1.11.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[windmaomao]

I'm not sure this is the right place, but i found out the bug might be related to the following lines for beforeEach.

    // save reference to coverage for each app window loaded in the test
    cy.on('window:load', saveCoverageObject)

    // save reference if visiting a page inside a before() hook
    cy.window({ log: false }).then(saveCoverageObject)

If i have a test file with multiple tests

describe('Button component', () => {
  it('should display label', () => {
    mount(<Button />);
  });
  it('should ..', () => {
    mount(<Button />)
  })

What happens is that window.__coverage__ does not get reset when going from the first it to the second it. So when combining them, it gets a weird behavior, for example, all the counts in the second it would carry all the counts in the first it already.

[grantbdev]

@windmaomao It's been a long time since I worked on this, but I believe the intended behavior is to give the total counts/coverage of all the tests run at once. I think that is also consistent with coverage tracking in other testing libraries where it typically shows the totals for a testing session. If you want to see the coverage from a specific test you would have to run it in isolation.

[windmaomao]

thanks @grantbdev , i appreciate your time. Actually i'm asking the opposite. The window.__coverage__ does cover the overall counts. For example, if i go from test 1 to test 2 within the same unit, it'll show me the the combination of test 1 and 2. However the problem is the save part, it occurs to me (I want to confirm when i get time), the results are "merged" at both test 1 and 2, therefore making test1 results double counted. Hopefully i'm making sense here.

I think i know exactly what i want to get. I want the test 2 results gets merged at the end of this entire unit, but not after test 1 is finished.