Skip to content

#14854: DatetimeIndex compiled together in test_datetime.py #15266

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from
Closed

#14854: DatetimeIndex compiled together in test_datetime.py #15266

wants to merge 8 commits into from

Conversation

TrigonaMinima
Copy link

@TrigonaMinima TrigonaMinima commented Jan 30, 2017

Partially completes #14854

Grouped all the DatetimeIndex related tests under one file - pandas/tests/indexes/test_datetime.py.

Tests reorganized

There were some tests in other classes which could be shifted to the as well. From pandas/pandas/tseries/tests/test_timeseries.py

  1. Should these also be included in the test_datetime.py in their respective classes or let them be?

  2. In pandas/tests/indexes/test_datetimelike.py there were some tests which were using the DatetimeLike class defined there. I could import that class in the test_datetime.py and transfer the whole TestDatetimeIndex but should it be done?

Mostly passes git diff upstream/master | flake8 --diff

./pandas/tests/indexes/test_datetime.py:1121:5: F811 redefinition of unused 'test_intersection' from line 572
./pandas/tests/indexes/test_datetime.py:1751:5: F811 redefinition of unused 'test_union' from line 589
  1. Should I merge the two test_intersection and test_union functions?

Further TODOs from the following discussion and the #14854 under datetime

  • Split datetimes/test_datetime.py into smaller chunks

  • In datetimes/test_misc.py lines from 167-202 should go under timestamp related tests (?)

  • Transfer of function (test_datetimeindex_accessors) from class TestDatetime64 in datetimes/test_misc.py to class TestDatetimeIndexOps of datetime/test_ops.py (?)

  • Function test_round from datetimes/test_datetime.py to pandas/tests/scalars/test_timestamp.py. (and test_timedelta.py). So anything specifically Timestamp related will go there.

@codecov-io
Copy link

codecov-io commented Jan 30, 2017

Codecov Report

Merging #15266 into master will not impact coverage.

@@           Coverage Diff           @@
##           master   #15266   +/-   ##
=======================================
  Coverage   86.33%   86.33%           
=======================================
  Files         139      139           
  Lines       51149    51149           
=======================================
  Hits        44157    44157           
  Misses       6992     6992

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8452080...6ee2bd9. Read the comment docs.

@jreback
Copy link
Contributor

jreback commented Jan 30, 2017

ok, so the resulting test file is just too big and unweildy.

let's split it out to a sub-dir, IOW

pandas/tests/indexes/datetime

then split into a number of files (you can start by splitting each class to a separate named file for instance)

@jreback jreback added the Testing pandas testing functions or related to the test suite label Jan 30, 2017
@jreback
Copy link
Contributor

jreback commented Jan 30, 2017

nice start!

name things like

test_ops.py (not test_datetimeops (except for the main 'test_datetime.py')

@TrigonaMinima
Copy link
Author

Thanks.

Though, test_datetime.py still looks big.

What about the questions I raised or is this PR ready?

@jreback
Copy link
Contributor

jreback commented Jan 30, 2017

In pandas/tests/indexes/test_datetimelike.py there were some tests which were using the DatetimeLike class defined there. I could import that class in the test_datetime.py and transfer the whole TestDatetimeIndex but should it be done?

no DatetimeLike needs to be defined where it is as PeriodIndex tests use it (though it could be factorted to another file if need be)

@jreback
Copy link
Contributor

jreback commented Jan 30, 2017

There were some tests in other classes which could be shifted to the as well. From pandas/pandas/tseries/tests/test_timeseries.py
test_pass_datetimeindex_to_index from class TestTimeSeries
test_datetimeindex_accessors from class TestDatetime64
test_datetimeindex_diff from class TestDatetime64
test_nanosecond_field from class TestDatetime64
test_datetimeindex_constructor from class TestDatetime64
Should these also be included in the test_datetime.py in their respective classes or let them be?

essentially yes these should be moved.

@jreback
Copy link
Contributor

jreback commented Jan 30, 2017

certainly would like to split up test_datetime.py Ideally split along class lines (though maybe not much there, it might already be a big class).

So can come back and do that later.

@jreback
Copy link
Contributor

jreback commented Jan 30, 2017

also need to add an entry in setup.py (right after pandas.tests.indexes

@TrigonaMinima
Copy link
Author

Now only remaining thing, it seems, is the merging of those 2 different function definitions by same name (test_intersection and test_union) in the pandas/tests/indexes/datetime/test_datetime.py. I didn't do that because you said not to change individual tests.

@jreback
Copy link
Contributor

jreback commented Jan 31, 2017

@TrigonaMinima sure, let's get this passing. Then pls make a list of what changes you think could / should be made. We want to move and then in other commits, fix/change etc.

@TrigonaMinima
Copy link
Author

@jreback I am sorry, I didn't understand you. Changes like? Merging those two functions and what else? Also, other commits under this PR only, right?

@TrigonaMinima
Copy link
Author

TrigonaMinima commented Jan 31, 2017

Okay so here's the list I came up with

  • In datetime/test_misc.py lines from 167-202 should go under timestamp related tests (?)

  • Transfer of function (test_datetimeindex_accessors) from class TestDatetime64 in datetime/test_misc.py to class TestDatetimeIndexOps of datetime/test_ops.py (?)

  • Split datetime/test_datetime.py into smaller chunks

@jreback
Copy link
Contributor

jreback commented Jan 31, 2017

@TrigonaMinima if you can get this passing (iow fix the flake errors) would be great.

@TrigonaMinima
Copy link
Author

TrigonaMinima commented Feb 1, 2017

@jreback the build in Python2.7 is giving errors in all the tests in pandas/tests/indexes due to import errors for datetime module. I am guessing it is trying to import the new datetime directory we have just made. It runs fine for Python3.5.

Any suggestions? Changing the name of the directory is the easiest fix I can think of.

@jreback
Copy link
Contributor

jreback commented Feb 1, 2017

@TrigonaMinima change the directory name to datetimes I think will work

@jreback
Copy link
Contributor

jreback commented Feb 1, 2017

FYI you can do

git diff master | flake8 --diff to see flake errors ahead of time

@TrigonaMinima
Copy link
Author

Yeah. I read the full contributing guide yesterday.

Though, this module name problem wasn't showing up in the output of the command. Which is the right behavior given this isn't exactly a pep8 issue.

@jreback
Copy link
Contributor

jreback commented Feb 1, 2017

@TrigonaMinima yeah, for some reason travis catches slightly different things that local flake8, not really sure why (its only this particular pep, missing a whole module import)

def test_round(self):

# round
dt = Timestamp('20130101 09:10:11')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this is something to add to the list.

I want to create a pandas/tests/scalars/test_timestamp.py (and test_timedelta.py). So anything specifically Timestamp related will go there.

this can be next iteration though.

class TestDatetimeIndex(tm.TestCase):
_multiprocess_can_split_ = True

def test_construction_with_alt(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so ok with splitting this up a bit now. e.g. test_construction (and when you do, copy the class (name), and leave the methods alone (even though they say test_construction_* for now that's ok))

freq='B')
tm.assert_index_equal(result, expected)

def test_astype(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_astype.py

self.assertRaises(ValueError, idx.astype, 'datetime64')
self.assertRaises(ValueError, idx.astype, 'datetime64[D]')

def test_where_other(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_indexing.py

tm.assert_index_equal(result, exp)
self.assertEqual(result.freq, 'D')

def test_fillna_datetime64(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_missing.py

dtype=object)
self.assert_index_equal(idx.fillna('x'), exp)

def test_difference_freq(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_setops. (include union, intersection, difference routines)


self.assertEqual(idx.nanosecond[0], t1.nanosecond)

def test_constructor_coverage(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

another one for test_construction

non_datetime = Index(list('abc'))
self.assertFalse(idx.equals(list(non_datetime)))

def test_union_coverage(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to test_setops

self.assertEqual(result.name, df.index[2])

# GH 10699
def test_datetime64_with_DateOffset(self):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can go to test_ops

idx = date_range('1/1/2000', periods=3, freq='D', tz='Asia/Tokyo',
name='idx')
with tm.assertRaises(ValueError):
result =
Copy link
Contributor

@jreback jreback Feb 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

put insert and delete into test_indexing

idx = date_range('1/1/2000', periods=3, freq='D', tz='Asia/Tokyo',
name='idx')
with tm.assertRaises(ValueError):
result =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

take -> test_indexing

@jreback jreback added this to the 0.20.0 milestone Feb 3, 2017
@jreback jreback closed this in da92a5c Feb 3, 2017
@jreback
Copy link
Contributor

jreback commented Feb 3, 2017

thanks @TrigonaMinima !

ok I am going to push a scalar directory now that is empty, but please continue on moving things!

@jreback jreback mentioned this pull request Feb 3, 2017
6 tasks
@TrigonaMinima TrigonaMinima deleted the issue-14854-datetime branch February 3, 2017 19:38
AnkurDedania pushed a commit to AnkurDedania/pandas that referenced this pull request Mar 21, 2017
xref pandas-dev#14854

Author: TrigonaMinima <[email protected]>

Closes pandas-dev#15266 from TrigonaMinima/issue-14854-datetime and squashes the following commits:

6ee2bd9 [TrigonaMinima] TST: Splitting test_datetime.py into smaller chunks (gh14854)
415a748 [TrigonaMinima] TST: Moving DatetimeIndex related tests from test_timeseries.py and flake8 fixes
c43c7de [TrigonaMinima] TST: proper naming of files
458d141 [TrigonaMinima] TST: splitting test_datetime.py
1ff0819 [TrigonaMinima] TST: fix flake8 errors - test_datetime.py (GH14854)
9311161 [TrigonaMinima] TST: reorg of DatetimeIndex tests from tseries/tests/test_base.py to test_datetime.py (GH14854)
54421a5 [TrigonaMinima] TST: reorg of DatetimeIndex tests from test_datetimelike.py to test_datetime.py (GH14854)
f83814b [TrigonaMinima] TST: reorg of DatetimeIndex tests from test_timeseries.py to test_datetime.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing pandas testing functions or related to the test suite
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants