Fail Azure CI for failing tests

[alimcmaster1]

• closes Azure CI should fail on test failures #23453
• passes git diff upstream/master -u -- "*.py" | flake8 --diff

This should hopefully return a failed status on Azure

[codecov]

Codecov Report

Merging #23454 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master   #23454   +/-   ##
=======================================
  Coverage   92.25%   92.25%           
=======================================
  Files         161      161           
  Lines       51200    51200           
=======================================
  Hits        47232    47232           
  Misses       3968     3968

Flag	Coverage Δ
#multiple	90.63% <ø> (ø)	⬆️
#single	42.29% <ø> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf4c0b6...951f3fc. Read the comment docs.

[alimcmaster1]

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=2227&view=logs

Good news the build fails on test failures now :)

I shall add this to windows/linux yml configs and attempt to de-duplicate the logic in the powershell script.

Done - please see latest comment

[pep8speaks]

Hello @alimcmaster1! Thanks for updating the PR.

• There are no PEP8 issues in the file pandas/tests/indexing/test_indexing.py !

Comment last updated on November 04, 2018 at 13:46 Hours UTC

[alimcmaster1]

This now looks like what we want:

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=2409&view=logs

Build status goes to failed on everything but ( "Linux py36_locale_slow" - which doesn't seem to run my failing test in test_indexing.py )

@TomAugspurger - I will now remove my dummy failing test. Any feedback/concerns with this PR?

CC: Azure Folks @vtbassmatt, @davidstaheli - for any comments - as this touches a few files that #23111 does

[TomAugspurger]

Sorry for the delay. This LGTM. Will merge on green.

[TomAugspurger]

Build status goes to failed on everything but ( "Linux py36_locale_slow" - which doesn't seem to run my failing test in test_indexing.py )

that's due to the -m only-slow marker, which skips tests not marked as slow.

[TomAugspurger]

Ahh merge conflict :) I'll fix that up quick.

[alimcmaster1]

Np at all @TomAugspurger my fault - I only just pushed the change to remove my failing test. And aah thanks for the info re the -m only-slow marker

[TomAugspurger]

Merged master and canceled the previous commit's build.

[TomAugspurger]

Thanks for working on this @alimcmaster1!

[jbrockmendel]

@alimcmaster1 any idea if there is a way to do this in bash? xref #30393 the two remaining places where we do this are among the last windows-special-casing left

[alimcmaster1]

Sure let me take a look - unless we are moving the CI to github actions soon? #29715 @datapythonista in which case I might wait for that to happen.

[jreback]

not so soon - things will move gradually and smooothly

[datapythonista]

It will take time, and besides what Jeff said, it's not yet clear that GitHub will give us all the workers we need, so may be we need to keep Azure in the long term.