ENH: optional ':' separator in ISO8601 strings

[thejohnfreeman]

• closes Loss of nanosecond resolution when constructing Timestamps from str #10041
• tests added / passed
• passes git diff upstream/master | flake8 --diff
• whatsnew entry

Includes some refactoring in the ISO8601 parser. If more tests are desired, please let me know where to put them. Same for the whatsnew entry. This is effectively my first pull request for Pandas.

cc @jreback

[jreback]

hmm, I don't think 12 or 1234 should work (but 123456) should , IOW, you either have to fully specify HHMMSS or not at all. Any ambiuity in partially accepting it as you have here?

cc @chris-b1

[thejohnfreeman]

So the current version accepts 12 in addition to 1. I was thinking we should keep the two digit form.

In [23]: pd.Timestamp('20010101 1')
Out[23]: Timestamp('2001-01-01 01:00:00')

In [24]: pd.Timestamp('20010101 09')
Out[24]: Timestamp('2001-01-01 09:00:00')

In [25]: pd.Timestamp('20010101 12')
Out[25]: Timestamp('2001-01-01 12:00:00')

As for four digits, I was thinking it would make sense to allow "military" times like 0800 (which is also currently accepted).

In [26]: pd.Timestamp('20010101 0800')
Out[26]: Timestamp('2001-01-01 08:00:00')

[jreback]

cc @chris-b1 can you have a look

[chris-b1]

Looks good as far as I can tell. The partial time formats are a bit out there, but unambiguous, so I don't see any harm.

In [24]: pd.Timestamp('2015/01/01 1234')
Out[24]: Timestamp('2015-01-01 12:34:00')

[jreback]

just do this in a 1-liner, pls add the issue number, and move to 0.18.0

[jreback]

can you rebase and push again

[jreback]

thanks @thejohnfreeman !

[thejohnfreeman]

Thanks for the help!

[jreback]

I normally test on windows before merging....but alas

.................................................S.......S.........................................................................................
...................................................
======================================================================
ERROR: test_parsers_iso8601 (pandas.tseries.tests.test_tslib.TestDatetimeParsingWrappers)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\conda\Documents\pandas3.5\pandas\tseries\tests\test_tslib.py", line 736, in test_parsers_iso8601
    raise Exception(date_str)
Exception: 2005-0101

======================================================================
FAIL: test_nanosecond_string_parsing (pandas.tseries.tests.test_tslib.TestTimestampNsOperations)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\conda\Documents\pandas3.5\pandas\tseries\tests\test_tslib.py", line 902, in test_nanosecond_string_parsing
    self.assertEqual(ts.value, expected_value)
AssertionError: 1367392545123456000 != 1367392545123456789

----------------------------------------------------------------------
Ran 9204 tests in 304.769s

FAILED (SKIP=174, errors=1, failures=1)

[jreback]

so if you have the ability to debug on windows, pls have a look :<

[thejohnfreeman]

Just at a glance, the errors make it seem like my changes to the C code weren't built. I didn't have any platform-specific preprocessor guards in the C, so I would expect it to behave the same on each platform. What are the sources of differences between Windows and Linux behavior? Are the numpy and Python datetime types different on each platform? Are there platform-specific tests?

[jreback]

hmm, I just tested this again on windows and its ok. Why don't you rebase and push again.

[jreback]

ok, maybe something funny happened in the build.

thanks again!