REF: use consistent pattern in tslibs.vectorized

[jbrockmendel]

DO NOT MERGE - Performance takes a hit

We have edited some of the functions in `tslibs.vectorized to do some pre-processing in order to de-duplicate some code. This new pattern is conducive to refactoring out helper functions for further de-duplication.

So far, moving to the new pattern has been performance-neutral or better. But for the functions which this branch moves to the new pattern, performance takes a hit, particularly for dt64arr_to_periodarray. I am at a loss as to why this would be, hoping more eyeballs will help. cc @WillAyd

asv continuous -E virtualenv -f 1.01 master HEAD -b tslibs
[...]
       before           after         ratio
     [d82e5403]       [9f5d9ef6]
     <master~4>       <ref-vectorized>
+      21.4±0.3ms         53.7±6ms     2.51  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 1011, datetime.timezone(datetime.timedelta(seconds=3600)))
+         248±4μs         613±50μs     2.48  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 3000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      21.9±0.3ms         53.8±6ms     2.46  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 1000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         238±8μs         574±60μs     2.41  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 1011, datetime.timezone(datetime.timedelta(seconds=3600)))
+         268±3μs         617±70μs     2.30  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 7000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         246±5μs         562±70μs     2.28  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 2011, datetime.timezone(datetime.timedelta(seconds=3600)))
+        282±10μs         636±70μs     2.26  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 12000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        274±20μs         616±70μs     2.24  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 11000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         270±6μs         599±60μs     2.22  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 6000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         248±4μs         544±40μs     2.20  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 2000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      23.4±0.4ms         51.3±2ms     2.19  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         278±8μs         605±70μs     2.18  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 8000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         290±9μs        620±100μs     2.14  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 9000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        286±40μs         611±60μs     2.14  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 10000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        259±20μs         538±60μs     2.08  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 1000, datetime.timezone(datetime.timedelta(seconds=3600)))
+       35.5±10ms        73.5±20ms     2.07  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 9000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         300±5μs         610±60μs     2.03  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 4006, datetime.timezone(datetime.timedelta(seconds=3600)))
+         289±8μs         586±60μs     2.03  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 4000, datetime.timezone(datetime.timedelta(seconds=3600)))
+       38.3±10ms        77.3±20ms     2.02  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 10000, datetime.timezone(datetime.timedelta(seconds=3600)))
+       36.6±10ms        70.9±20ms     1.94  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 11000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        26.1±1ms         49.9±1ms     1.91  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2011, datetime.timezone(datetime.timedelta(seconds=3600)))
+       38.8±10ms        72.4±20ms     1.87  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 12000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        27.0±3ms       50.2±0.5ms     1.86  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 3000, datetime.timezone(datetime.timedelta(seconds=3600)))
+       35.3±10ms        64.6±10ms     1.83  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 7000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        35.2±9ms        64.3±10ms     1.83  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 6000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        38.0±1ms        67.7±10ms     1.78  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 4006, datetime.timezone(datetime.timedelta(seconds=3600)))
+         358±7μs         637±70μs     1.78  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 5000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      9.43±0.2μs         16.7±2μs     1.77  tslibs.resolution.TimeResolution.time_get_resolution('s', 100, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
+     3.10±0.04μs         5.49±1μs     1.77  tslibs.fields.TimeGetStartEndField.time_get_start_end_field(1, 'start', 'month', None, 5)
+       37.7±10ms        66.5±10ms     1.76  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 8000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        33.5±5ms         57.0±2ms     1.70  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 4000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      26.5±0.1μs        42.8±10μs     1.61  tslibs.tz_convert.TimeTZConvert.time_tz_localize_to_utc(100, datetime.timezone(datetime.timedelta(seconds=3600)))
+        45.3±8ms        70.5±10ms     1.56  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 5000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      24.7±0.2μs        38.2±10μs     1.54  tslibs.tz_convert.TimeTZConvert.time_tz_localize_to_utc(100, <DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>)
+     1.55±0.08ms       2.36±0.2ms     1.52  tslibs.normalize.Normalize.time_is_date_array_normalized(1000000, datetime.timezone(datetime.timedelta(seconds=3600)))
+         327±4ms        491±100ms     1.50  tslibs.tslib.TimeIntsToPydatetime.time_ints_to_pydatetime('time', 1000000, None)
+      24.7±0.9ms        36.6±10ms     1.48  tslibs.fields.TimeGetDateField.time_get_date_field(1000000, 'dow')
+     3.29±0.06μs         4.87±1μs     1.48  tslibs.fields.TimeGetStartEndField.time_get_start_end_field(1, 'start', 'month', None, 12)
+         338±2ms        497±100ms     1.47  tslibs.tslib.TimeIntsToPydatetime.time_ints_to_pydatetime('time', 1000000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      22.7±0.3μs         32.0±3μs     1.41  tslibs.normalize.Normalize.time_is_date_array_normalized(10000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      12.2±0.5μs         17.1±4μs     1.41  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 2000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      12.0±0.4μs         16.5±4μs     1.38  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 2011, datetime.timezone(datetime.timedelta(seconds=3600)))
+      12.1±0.3μs         16.8±2μs     1.38  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 12000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      12.2±0.2μs         16.6±2μs     1.36  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 11000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      11.7±0.3μs         15.9±1μs     1.35  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 3000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      12.6±0.3μs         16.5±2μs     1.31  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 4000, datetime.timezone(datetime.timedelta(seconds=3600)))
+      8.89±0.6μs         11.3±1μs     1.27  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(0, 2000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
+         115±2μs         143±20μs     1.25  tslibs.normalize.Normalize.time_is_date_array_normalized(10000, <DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>)
+      12.8±0.3μs         15.9±2μs     1.24  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 6000, datetime.timezone(datetime.timedelta(seconds=3600)))
+        61.7±2μs         75.0±7μs     1.22  tslibs.normalize.Normalize.time_is_date_array_normalized(10000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
+     7.58±0.08ms       8.61±0.1ms     1.14  tslibs.normalize.Normalize.time_is_date_array_normalized(1000000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
+     8.80±0.07ms       9.55±0.4ms     1.09  tslibs.normalize.Normalize.time_normalize_i8_timestamps(1000000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
-     3.46±0.05μs       3.06±0.1μs     0.89  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1, 3000, tzlocal())
-      25.2±0.9ms       21.2±0.5ms     0.84  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2011, datetime.timezone.utc)
-      2.39±0.1ms       2.01±0.3ms     0.84  tslibs.normalize.Normalize.time_normalize_i8_timestamps(1000000, datetime.timezone(datetime.timedelta(seconds=3600)))
-      47.0±0.5ms       38.7±0.8ms     0.82  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2011, <DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>)
-        38.5±3ms       31.2±0.4ms     0.81  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 3000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
-        36.4±1ms       29.3±0.7ms     0.81  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2011, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
-        25.7±3ms       20.7±0.2ms     0.80  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 3000, datetime.timezone.utc)
-        25.7±3ms       20.5±0.2ms     0.80  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 3000, None)
-     1.88±0.02ms        607±200μs     0.32  tslibs.tz_convert.TimeTZConvert.time_tz_convert_from_utc(1000000, datetime.timezone.utc)

[jreback]

DO NOT MERGE - Performance takes a hit

I think you can do a draft PR to indicate this (of course comments ok too)

[jbrockmendel]

There was a missing type declaration in the periodarray_to_dt64arr func; now this is perf-improving for that func.

       before           after         ratio
     [71a327c4]       [a84c147f]
     <master>         <ref-vectorized>
-     7.74±0.06μs      7.51±0.05μs     0.97  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1, 1000, <DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>)
-         266±5μs        258±0.8μs     0.97  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 4006, datetime.timezone(datetime.timedelta(seconds=3600)))
-         221±3μs          214±2μs     0.97  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 2011, None)
-     5.25±0.06μs      4.99±0.02μs     0.95  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 3000, None)
-     5.71±0.03μs      5.42±0.04μs     0.95  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 10000, None)
-      8.91±0.1μs      8.44±0.05μs     0.95  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1, 12000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
-     5.43±0.08μs      5.13±0.05μs     0.95  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 7000, datetime.timezone.utc)
-     5.13±0.06μs      4.84±0.05μs     0.94  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(100, 1011, datetime.timezone.utc)
-     2.95±0.04μs      2.75±0.06μs     0.93  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1, 2000, datetime.timezone.utc)
-      21.2±0.5ms       19.8±0.3ms     0.93  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 1000, datetime.timezone.utc)
-         228±2μs        212±0.6μs     0.93  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 2011, datetime.timezone.utc)
-        38.6±3ms       35.8±0.2ms     0.93  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 1000, <DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>)
-        387±20ns          352±6ns     0.91  tslibs.period.PeriodProperties.time_property('M', 'year')
-      9.69±0.3μs      8.81±0.04μs     0.91  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(0, 4006, datetime.timezone(datetime.timedelta(seconds=3600)))
-     3.09±0.06μs      2.77±0.03μs     0.90  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(0, 9000, datetime.timezone.utc)
-        269±10μs          242±1μs     0.90  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(10000, 9000, datetime.timezone.utc)
-      3.16±0.2μs      2.78±0.04μs     0.88  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(0, 4006, datetime.timezone.utc)
-      3.41±0.1μs      2.98±0.08μs     0.88  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(0, 5000, None)
-        25.4±3ms       20.8±0.2ms     0.82  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2000, None)
-        46.9±8ms      37.4±0.06ms     0.80  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2011, <DstTzInfo 'US/Pacific' LMT-1 day, 16:07:00 STD>)
-        27.4±5ms       20.7±0.1ms     0.76  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 2011, datetime.timezone.utc)
-        26.7±6ms       19.8±0.2ms     0.74  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 1011, None)
-       44.1±10ms       30.6±0.2ms     0.70  tslibs.period.TimeDT64ArrToPeriodArr.time_dt64arr_to_periodarr(1000000, 7000, tzfile('/usr/share/zoneinfo/Asia/Tokyo'))
-        4.23±1μs      2.81±0.02μs     0.67  tslibs.period.TimePeriodArrToDT64Arr.time_periodarray_to_dt64arr(0, 1011)
-        9.97±3μs      6.38±0.03μs     0.64  tslibs.period.TimePeriodArrToDT64Arr.time_periodarray_to_dt64arr(0, 6000)

Still need to make sure the normalize funcs are neutral or better

[WillAyd]

lgtm save the red CI pieces. Nice cleanup!