Run libstd test suite in miri

[RalfJung]

We have all these nice #[test] functions in libstd (and liballoc and libcore), wouldn't it be great if we could run them all in miri? Maybe even on CI? miri has a small test suite itself that exercises some bits on libstd, and that lets us find things like #54908, but it's not very exhaustive and it doesn't seem like a good idea to duplicate all that.

So I think we should figure out a way to run the libstd tests in miri. Unfortunately I don't have a very good idea of what it would take. But it sounds like a great project, if someone is looking for one. :D

[Mark-Simulacrum]

Is there some level of general idea what would need to be done for this? I think a list of a few "starter" steps might be good here -- I get the impression this might not be actually too hard?

Basically I think as-is this is fairly unlikely to happen because issues like this tend to languish unfortunately -- but getting a few starter steps together should help immensely.

[RalfJung]

The problem is I have no idea what it takes.^^ I'd start by trying to run cargo miri test on libstd, seeing why that doesn't work and going from there. Probably taking some inspiration from xargo for how to use cargo to build libstd. But figuring out what it takes is probably the most time-consuming part here, and I don't think I'll have time for that soon.

If you think that's too vague to be tracked here, we could open an issue in the miri tracker instead. I just hoped here it might find someone else who is motivated enough to give it a try, that's unlikely to happen on the miri side.

[Mark-Simulacrum]

I think even that much is a good starting point for this - thanks!

[RalfJung]

Here's another issue this could have prevented (or caught earlier): #54957

And this earlier issue would also have been detected: #53529

[RalfJung]

This is done now in the form of https://github.com/RalfJung/miri-test-libstd