Skip to content

Fail Azure CI for failing tests #23454

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 10 commits into from
Nov 7, 2018
Merged

Fail Azure CI for failing tests #23454

merged 10 commits into from
Nov 7, 2018

Conversation

alimcmaster1
Copy link
Member

This should hopefully return a failed status on Azure

@codecov
Copy link

codecov bot commented Nov 2, 2018

Codecov Report

Merging #23454 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master   #23454   +/-   ##
=======================================
  Coverage   92.25%   92.25%           
=======================================
  Files         161      161           
  Lines       51200    51200           
=======================================
  Hits        47232    47232           
  Misses       3968     3968
Flag Coverage Δ
#multiple 90.63% <ø> (ø) ⬆️
#single 42.29% <ø> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cf4c0b6...951f3fc. Read the comment docs.

@alimcmaster1
Copy link
Member Author

alimcmaster1 commented Nov 2, 2018

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=2227&view=logs

Good news the build fails on test failures now :)

I shall add this to windows/linux yml configs and attempt to de-duplicate the logic in the powershell script.

Done - please see latest comment

@gfyoung gfyoung added the CI Continuous Integration label Nov 2, 2018
@gfyoung gfyoung requested a review from TomAugspurger November 2, 2018 06:11
@pep8speaks
Copy link

pep8speaks commented Nov 3, 2018

Hello @alimcmaster1! Thanks for updating the PR.

Comment last updated on November 04, 2018 at 13:46 Hours UTC

@alimcmaster1
Copy link
Member Author

alimcmaster1 commented Nov 5, 2018

This now looks like what we want:

https://dev.azure.com/pandas-dev/pandas/_build/results?buildId=2409&view=logs

Build status goes to failed on everything but ( "Linux py36_locale_slow" - which doesn't seem to run my failing test in test_indexing.py )

@TomAugspurger - I will now remove my dummy failing test. Any feedback/concerns with this PR?

CC: Azure Folks @vtbassmatt, @davidstaheli - for any comments - as this touches a few files that #23111 does

@alimcmaster1 alimcmaster1 changed the title WIP: Fail Azure CI for failing tests Fail Azure CI for failing tests Nov 6, 2018
@TomAugspurger
Copy link
Contributor

Sorry for the delay. This LGTM. Will merge on green.

@TomAugspurger
Copy link
Contributor

Build status goes to failed on everything but ( "Linux py36_locale_slow" - which doesn't seem to run my failing test in test_indexing.py )

that's due to the -m only-slow marker, which skips tests not marked as slow.

@TomAugspurger
Copy link
Contributor

Ahh merge conflict :) I'll fix that up quick.

@alimcmaster1
Copy link
Member Author

alimcmaster1 commented Nov 6, 2018

Np at all @TomAugspurger my fault - I only just pushed the change to remove my failing test. And aah thanks for the info re the -m only-slow marker

@TomAugspurger
Copy link
Contributor

Merged master and canceled the previous commit's build.

@TomAugspurger TomAugspurger merged commit 528527b into pandas-dev:master Nov 7, 2018
@TomAugspurger
Copy link
Contributor

Thanks for working on this @alimcmaster1!

@TomAugspurger TomAugspurger added this to the 0.24.0 milestone Nov 7, 2018
JustinZhengBC pushed a commit to JustinZhengBC/pandas that referenced this pull request Nov 14, 2018
tm9k1 pushed a commit to tm9k1/pandas that referenced this pull request Nov 19, 2018
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
Pingviinituutti pushed a commit to Pingviinituutti/pandas that referenced this pull request Feb 28, 2019
@@ -54,3 +54,28 @@ jobs:
inputs:
testResultsFiles: 'test-data-*.xml'
testRunTitle: 'Linux'
- powershell: |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@alimcmaster1 any idea if there is a way to do this in bash? xref #30393 the two remaining places where we do this are among the last windows-special-casing left

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure let me take a look - unless we are moving the CI to github actions soon? #29715 @datapythonista in which case I might wait for that to happen.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not so soon - things will move gradually and smooothly

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will take time, and besides what Jeff said, it's not yet clear that GitHub will give us all the workers we need, so may be we need to keep Azure in the long term.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Azure CI should fail on test failures
7 participants