Automatically verify puzzle validity

[iximeow]

Since puzzlers' source can be compiled and ran on its own.. it should be possible to automatically extract the puzzle's source and expected output, execute the puzzle, and verify that the output isn't changed between Scala versions.

CircleCI has a free tier that would work pretty well for that purpose, and it shouldn't take long to put together. Think it's worth the effort?

[demobox]

Looking at the relatively slow (unfortunately!) rate at which puzzlers are deprecated, I wouldn't say it's urgent, but it would certainly be nice to have.

Rather than going for some complex scraping, I think it would probably be easier simply to commit an executable test case with each puzzler, though, and then run a CI server against that. One could then even "build" all the puzzlers locally, using an arbitrary Scala version.

Thanks for the suggestion!

[som-snytt]

Could also automatically verify that the puzzler is funny. If Jenkins so much as cracks a smile, you can be reasonably sure it's funny.

[demobox]

If Jenkins so much as cracks a smile, you can be reasonably sure it's funny.

We could also install the Chuck Norris plugin to see what he has to say..? ;-)

[som-snytt]

We need the Chuckle Norris plugin, where the puzzler has to result in a smiley Chuck Norris.

Also, why does Chuck Norris have a leather glove only on his left hand? Anything to do with right-bias in #148?

This issue has regained urgency because of scala/scala3#4553

[demobox]

Closing out old issues - please re-open if you have cycles to potentially look into this!

[som-snytt]

The good news is that while it was often hard to tell with Scala whether some weird behavior was a bug or a limitation, in Dotty, the question is whether it's only puzzling because you're still on 0.25.0-RC1. The behavior may change with the latest nightly, but you're guaranteed it will still be puzzling.