ENH: add pd.test to enable nose test runnning from the imported session, #4327

[jreback]

closes #4327

copied from numpy nosetester.

[jreback]

any comments / objections? anything further I should add to docs?

[wesm]

Maybe I'm just making trouble, but should we consider switching to pytest?

[jreback]

I actually do like pytest better - fixtures IMHO are much nicer

is there an easy / automated way to convert things?

[jreback]

convert is not the right word / more of adjust

[jreback]

https://pytest.org/latest/nose.html

[jorisvandenbossche]

I also prefer pytest (using it for my personal projects), but the question is maybe also how much effort it is to switch such a big test suite.

Matplotlib is switching from nose to pytest: matplotlib/matplotlib#5405

[jreback]

and xref: #10788

[TomAugspurger]

FYI: http://pytest.readthedocs.org/en/latest/nose.html specifically

Unsupported idioms / known issues

• unittest-style setUp, tearDown, setUpClass, tearDownClass are recognized only on unittest.TestCase classes but not on plain classes. nose supports these methods also on plain classes but pytest deliberately does not. As nose and pytest already both support setup_class, teardown_class, setup_method, teardown_method it doesn’t seem useful to duplicate the unittest-API like nose does. If you however rather think pytest should support the unittest-spelling on plain classes please post to this issue.
• nose imports test modules with the same import path (e.g. tests.test_mod) but different file system paths (e.g. tests/test_mode.py and other/tests/test_mode.py) by extending sys.path/import semantics. pytest does not do that but there is discussion in issue268 for adding some support. Note that nose2 choose to avoid this sys.path/import hackery.
• nose-style doctests are not collected and executed correctly, also doctest fixtures don’t work.
• no nose-configuration is recognized

Just ran them both,

Nose

Ran 5468 tests in 932.244s

FAILED (SKIP=48, errors=4, failures=1)

(Two of those errors are from the branch I'm working on, not sure about the cause of the others)

Pytest

============= 21 failed, 5525 passed, 48 skipped in 958.65 seconds =============

Just glancing at the output the majority of the additional failures came from test_testing.py

[clembou]

Hi,

I noticed an issue when attempting to use the new pd.test() function, at least on a windows machine.
If I try to run it from anywhere within the pandas repo in particular in the root folder I get the following output:

[pandas_dev] C:\Users\jahfe_000\Documents\GitHub\pandas>python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Jan 29 2016, 14:26:21) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import pandas
>>> pandas.test()
Running unit tests for pandas
pandas version 0.17.1+311.g70f79ce
numpy version 1.10.4
pandas is installed in pandas
Python version 2.7.11 |Continuum Analytics, Inc.| (default, Jan 29 2016, 14:26:21) [MSC v.1500 64 bit (AMD64)]
nose version 1.3.7

----------------------------------------------------------------------
Ran 0 tests in 0.000s

OK
<nose.result.TextTestResult run=0 errors=0 failures=0>
>>>

However if I browse out of the pandas folder, it works properly:

[pandas_dev] C:\Users\jahfe_000\Documents\GitHub\pandas>cd ..

[pandas_dev] C:\Users\jahfe_000\Documents\GitHub>python
Python 2.7.11 |Continuum Analytics, Inc.| (default, Jan 29 2016, 14:26:21) [MSC v.1500 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import pandas
>>> pandas.test()
Running unit tests for pandas
pandas version 0.17.1+311.g70f79ce
numpy version 1.10.4
pandas is installed in c:\users\jahfe_000\documents\github\pandas\pandas
Python version 2.7.11 |Continuum Analytics, Inc.| (default, Jan 29 2016, 14:26:21) [MSC v.1500 64 bit (AMD64)]
nose version 1.3.7
C:\Anaconda\envs\pandas_dev\lib\site-packages\matplotlib\backends\qt_compat.py:84: ImportWarning: Not importing directory 'C:\Anaconda\envs\pandas_dev\sip': missing __init__.py
  import sip
....................................S..............S..............S..............S.................F................................................................................................................................................................................................................EE......SS...........S....S.............S.

I think this may be because while in the root folder, pandas.__file__ returns a relative path from the package root. When launched from outside the module path, it returns an absolute path which seems to be processed nicely by NoseTester.

I don't know if this is really important given that people running a develop instance of pandas from the git repo will know how to run the tests separately but I though I would flag this up.

I can replicate this problem on windows after following the steps to setup a dev instance of pandas from the documentation.

[jreback]

@clembou good point.

I think that if this is run in a development version then it should fail loudly (if the extensions are not built), or work. Can you create a separate issue for this (and PR to fix if you can)

[clembou]

@jreback done: #12285.

I'll see if I can find a fix, and attempt to make a PR!

[charris]

Hmm, is there good reason to prefer pytest to nose?

[jreback]

@charris I really like pytest, but the codebase was originally setup using nose, so sort of stuck with it. I mean it works, but we don't really use any of the power of pytest IOW fixtures, so no real reason to switch at this point.

you guys thinking of changing in numpy?

[charris]

@jreback No. I was just surprised that there was a package that others would consider for the role. A quick look does show some interesting features in pytest, but currently nose works well enough for numpy, at least in my opinion ;)