Skip to content

Move remaining conversion functions to tslibs.conversion #18358

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 19, 2017

Conversation

jbrockmendel
Copy link
Member

Slightly more efficient implementation of _to_i8.

Importantly (or at least satisfyingly) gets pandas_datetime_to_datetimestruct usage centralized in conversion.

@codecov
Copy link

codecov bot commented Nov 18, 2017

Codecov Report

Merging #18358 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18358      +/-   ##
==========================================
- Coverage   91.38%   91.36%   -0.02%     
==========================================
  Files         164      164              
  Lines       49790    49791       +1     
==========================================
- Hits        45501    45493       -8     
- Misses       4289     4298       +9
Flag Coverage Δ
#multiple 89.17% <100%> (ø) ⬆️
#single 39.5% <42.85%> (-0.06%) ⬇️
Impacted Files Coverage Δ
pandas/core/tools/datetimes.py 84.48% <100%> (ø) ⬆️
pandas/core/internals.py 94.54% <100%> (ø) ⬆️
pandas/core/indexes/datetimes.py 95.48% <100%> (ø) ⬆️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.8% <0%> (-0.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cfad581...f19eeba. Read the comment docs.

@codecov
Copy link

codecov bot commented Nov 18, 2017

Codecov Report

Merging #18358 into master will decrease coverage by 0.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master   #18358      +/-   ##
==========================================
- Coverage   91.38%   91.36%   -0.02%     
==========================================
  Files         164      164              
  Lines       49790    49791       +1     
==========================================
- Hits        45501    45493       -8     
- Misses       4289     4298       +9
Flag Coverage Δ
#multiple 89.17% <100%> (ø) ⬆️
#single 39.53% <75%> (-0.03%) ⬇️
Impacted Files Coverage Δ
pandas/core/internals.py 94.54% <100%> (ø) ⬆️
pandas/core/indexes/datetimes.py 95.48% <100%> (ø) ⬆️
pandas/core/tools/datetimes.py 84.48% <100%> (ø) ⬆️
pandas/io/gbq.py 25% <0%> (-58.34%) ⬇️
pandas/core/frame.py 97.8% <0%> (-0.1%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cfad581...0ef759f. Read the comment docs.

@@ -73,6 +73,86 @@ cdef inline int64_t get_datetime64_nanos(object val) except? -1:

return ival


def cast_to_nanoseconds(ndarray arr):
cdef:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add doc-string



def datetime_to_datetime64(ndarray[object] values):
cdef:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same



cdef inline _to_i8(object val):
cdef:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doc-string.

can you rename this to something else. maybe datetime_to_i8

@jreback jreback added Clean Datetime Datetime data dtype labels Nov 19, 2017
@@ -13,7 +13,7 @@ cimport util

import numpy as np

from tslib cimport _to_i8
from tslibs.conversion cimport _maybe_datetimelike_to_i8
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

note to de-private this at some point as well

@@ -73,6 +73,123 @@ cdef inline int64_t get_datetime64_nanos(object val) except? -1:

return ival


def cast_to_nanoseconds(ndarray arr):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's rename this maybe ensure_datetime64ns

if len(iresult) == 0:
return result

unit = get_datetime64_unit(arr.flat[0])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prob should add some checks around this, IOW this could already be ns (TODO ok)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah, this check is done elsewhere, will not be hard to implement.

If it weren't for datetime64[Y] and datetime64[M] we could take the pandas_datetime_to_datetimestruct out of these funcs altogether, avoid converting back and forth.


Returns
-------
result : ndarray witth dtype int64
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo

@jreback jreback added this to the 0.22.0 milestone Nov 19, 2017
@jreback jreback merged commit a172ff9 into pandas-dev:master Nov 19, 2017
@jreback
Copy link
Contributor

jreback commented Nov 19, 2017

thanks!

@jbrockmendel jbrockmendel deleted the tslibs-conversion12 branch December 8, 2017 19:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Clean Datetime Datetime data dtype
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants