Testing Frameworks

[kingcody]

I'm relatively new to unit testing, but I've been working with the specs that come with the generator; trying to learn more and see if I can help with the test coverage. I've found that Jasmine is a little heavy when setting up asynchronous tests. Perhaps I haven't given it enough of a chance.

Mocha on the other hand seems to be a good bit easier when it comes to async. I setup Mocha + Chai + Sinon for the server tests I was working on and really enjoyed the combo. Kinda got me thinking: why not use mocha with karma instead?

I installed karma-mocha and everything seems to work great. So I'm wondering if anyone could outline Jasmine's advantages? Or would it possibly be more advantageous to use Mocha for server and client testing?

[JaKXz]

In brief, I like Jasmine's ability to cover both front and back end cases. I like to write tests with one framework to keep the test suites consistent and (IMO) more readable as a result. Doubtless you've been through the docs and obviously there are a lot of examples there.

For the tests in this generator, I believe the existing combo is mocha + chai, so I'd continue with that combo and it's great that you're a fan of that, increasing test coverage [which] is awesome of you. See if you can integrate istanbul for coverage metrics as a bonus 😄

[kingcody]

@JaKXz thanks so much for your answer! I can totally see the advantage of using the same framework for both server and client. As for mocha + chai, the server test are actually using mocha + should.js ATM. Would a switch to mocha + chai be possible? In my (inexperienced) opinion; chai would seem a better candidate since it provide expect, should, assert.

Also, on the point of a unified testing framework; karma currently uses Jasmine, while the server test are written in mocha. Should we look at unifying them or perhaps use a prompt in the future to allow users to choose?

Btw; thanks for suggesting Istanbul, I'll definitely check it out.

[JaKXz]

Let me get some clarity: are we talking about test coverage for the generator? OR, for a newly generated project? I misspoke earlier and I apologize, generator-angular-fullstack uses mocha for its own testing.

[kingcody]

I was referring to a newly generated project when I spoke earlier. The client side test, ran by karma, use Jasmine. The server side use Mocha + should.js. So my question was, should we look at unifying those tests? Potentially using Mocha + Chai for both server and client test on a generated project. Sorry for the confusion.

[DaftMonk]

I would like to unify them. I prefer mocha and chai for both front and backend, but I think we should support jasmine as well.

[kingcody]

@DaftMonk, by support would you mean a prompt? Or merely making the test in a way that would allow for Jasmine to be easily switched out?

[DaftMonk]

I was thinking a prompt.

[kingcody]

Sounds good. Have you heard of or used Sinon before? More specifically the Sinon-Chai plugin. Would you have any objection to using it in conjunction with Mocha + Chai? My thought was that since Jasmine has some form of spy assertions built in, it would be nice to carry that feature over to the mocha/chai setup.

[kingcody]

@JaKXz thanks again for your input on this.

[JaKXz]

@kingcody no worries, it didn't turn out to be that much. I also meant to mention the mocha + chai + sinon combination. Have fun!

[kingcody]

@DaftMonk I've got a branch based on canary + #482 that uses mocha + chai + sinon for server and client side tests. I've included promise support in the server tests and reduced their code due to removing callbacks and such... Also I switched to using chai should assertions since they don't need globals(jshint) or a var expect = require('chai').expect in every test, let me know if thats a problem. One thing that i see is the test included in generator-ng-component would need to be updated if we were to keep those changes.

In implementing a prompt based testing framework selection, we would need to be able to maintain the choice with .yo-rc.json I'm guessing? Then ng-component would need to be able to support Jasmine or Chai style assertions? Sorry, I've not had much experience with the inner workings of the sub-generators.

[thomporter]

nice job @kingcody! 👍

[kingcody]

If anyone would like to take a look at the work I've done, you can find it here.

I also started to extend some of the existing tests. If anyone has time, I'd like some feedback on what I've done there.

[kingcody]

@JaKXz just wondering, since you seem to have experience with testing, If you might have time to checkout the work I've done on the tests I mention above. If not, it's all good :)

[JaKXz]

@kingcody I've got more experience with Jasmine tbh, but I can give it a shot; no promises, but in any case thanks for your work on it!

Just a thought, maybe there could be a choice to use Jasmine on its own, OR, mocha + sinon + chai? I'm contemplating having to rip out all your good work in future projects ...lol

[kingcody]

In implementing a prompt based testing framework selection, we would need to be able to maintain the choice with .yo-rc.json I'm guessing? Then ng-component would need to be able to support Jasmine or Chai style assertions?

That's my plan 😄

[kingcody]

@JaKXz would you be interested in helping with the Jasmine test coverage?

[JaKXz]

@kingcody I would definitely be interested, and will try to make time for it somehow... but if others in the community can start making PRs then I'd encourage that 😄

[kingcody]

Sounds great @JaKXz. Here are some of the things that I see need to be implemented:

• port current server-side tests to Jasmine alternatives
• port current client-side tests to Mocha alternatives
• unify/refactor grunt tasks for client and server tests using Jasmine or Mocha
• server-side test coverage using Istanbul and Jasmine
• server-side test coverage using Istanbul and Mocha
• client-side test coverage using Istanbul and Jasmine or Mocha
• generator prompt for test selection
• extend ng-component to generate Jasmine or Mocha tests based on .yo-rc.json

Do you think that it would be a good idea to open an issue to invite PR's and track updates to the list of things needed?