PERF: infer_datetime_format without padding #11142

[chris-b1]

Closes #11142

[sinhrks]

I think users expect %y when they pass 2 digits. Should this be zfilled?

[chris-b1]

hmm, I hadn't planned on adding %y because it adds more ambiguity, e.g. '1-1-1'. But I suppose it would be possible to follow the dateutil semantics (%m-%d-%y)

[sinhrks]

Ah, what I meant is to keep %Y without padding, without adding %y as this is ambiguous.

[chris-b1]

Not sure I'm following, could you show an example date that would be effected?
I see what you're saying. I couldn't find any cases where it causes a problem, but I'll remove the padding on the year.

[chris-b1]

@sinhrks pushed changes for your notes. The perf impact on _guess_datetime_format is negligible because it is only ever called once. Nothing showed up in asv, here are a couple timings.

In [1]: s, l = pd.date_range('2014-1-1', periods=10), pd.date_range('2014-1-1', periods=100000)
In [2]: s = s.strftime('%Y-%m-%d')
In [3]: l = l.strftime('%Y-%m-%d')

# master
In [4]: %timeit pd.to_datetime(l, infer_datetime_format=True)
10 loops, best of 3: 30.3 ms per loop

In [5]: %timeit pd.to_datetime(s, infer_datetime_format=True)
1000 loops, best of 3: 300 µs per loop

# PR
In [5]: %timeit pd.to_datetime(l, infer_datetime_format=True)
10 loops, best of 3: 30.3 ms per loop

In [6]: %timeit pd.to_datetime(s, infer_datetime_format=True)
1000 loops, best of 3: 308 µs per loop

[jreback]

thanks!