Add pytest-custom_exit_code

[stsewd]

This is to allow us to ignore the exit code from pytest when
no tests were collected, when using the -k option, for example.

[humitos]

Why do we need this in community repository? Failing when no test are collected makes sense to me.

Although, I think we need this in commercial instead, where we want to run a test from readthedocsinc but the whole suite fails because it tries to find it in community first and it doesn't find it.

[humitos]

We can move the dependency to corporate under setup.py:options.extras_require.tests.

[stsewd]

@humitos I'm planning to run the test suite with different settings files here as well (#6623), so this is needed here too.

[stsewd]

the whole suite fails because it tries to find it in community first and it doesn't find it.

It failed because the test you wanted to run (sso related) was only run when using another settings file

[humitos]

Hrm... I see your point. I'd like to have a way to use this plugin only when running tests locally, but when running in CI we force a good exit code. Otherwise, we will see the Green bullet and think that everything was OK, when all test could be failing completely. Is it possible to use this only locally?

[stsewd]

Otherwise, we will see the Green bullet and think that everything was OK, when all test could be failing completely

The plugin only overrides the exit code for "no tests were collected" (5). https://docs.pytest.org/en/latest/usage.html