REF: use more explicit to_numpy(object) instead of astype(object) in EA implementation

[jorisvandenbossche]

Somewhat related to #24877 / #43699, but regardless of whether we want to stop astype returning numpy arrays or not, I think when we know we want to get an object-dtype numpy array as result, using to_numpy(object) is more explicit about this when reading the code.

[jbrockmendel]

Can't speak to the general case, but DTA is more performant in astype

In [3]: dti = pd.date_range('2016-01-01', periods=10**5, freq='s', tz="US/Pacific")

In [4]: dta = dti._data

In [5]: %timeit dta.astype(object)
50.8 ms ± 2.13 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [6]: %timeit dta.to_numpy(object)
178 ms ± 6.26 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)

[jorisvandenbossche]

Good catch. That seems to be because in datetimelike __array__ we explicitly go through a list, which shouldn't be needed:

pandas/pandas/core/arrays/datetimelike.py

Lines 313 to 316 in fefccfe

	def __array__(self, dtype: NpDtype | None = None) -> np.ndarray:
	# used for Timedelta/DatetimeArray, overwritten by PeriodArray
	if is_object_dtype(dtype):
	return np.array(list(self), dtype=object)

We can probably call astype there (or do the same than astype does). Astype special cases object dtype for datetime64 to directly call ints_to_pydatetime:

pandas/pandas/core/arrays/datetimelike.py

Lines 410 to 422 in fefccfe

	if is_object_dtype(dtype):
	if self.dtype.kind == "M":
	# *much* faster than self._box_values
	# for e.g. test_get_loc_tuple_monotonic_above_size_cutoff
	i8data = self.asi8.ravel()
	converted = ints_to_pydatetime(
	i8data,
	# error: "DatetimeLikeArrayMixin" has no attribute "tz"
	tz=self.tz, # type: ignore[attr-defined]
	freq=self.freq,
	box="timestamp",
	)
	return converted.reshape(self.shape)

(which is what list(self) will also call, but then iteratively)

[jbrockmendel]

xref #44484 for dispatch logic

[jorisvandenbossche]

Yes, although the behaviour differences I note in that issue are for other dtypes, not object dtype (for object, it's only a matter of performance, as far as I see)

[github-actions]

This pull request is stale because it has been open for thirty days with no activity. Please update and respond to this comment if you're still interested in working on this.

[simonjayhawkins]

Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.