REGR: Comparing timestamp to empty Series raises TypeError starting in 0.25.0

[pganssle]

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• (optional) I have confirmed this bug exists on the master branch of pandas.

---

Minimal reproducing code sample

import pandas as pd

pd.Series() < pd.Timestamp.now()  # Raises TypeError

(Note: This raises a warning because the default dtype is changing to object — which will actually fix the issue — but no warning is raised when you slice a column from an empty DataFrame).

Problem description

Prior to 0.25.0, it was possible to compare a pd.Timestamp to an empty Series (even if that series did not have a dtype of datetime64[ns]), but starting in the 0.25.0 release, this operation now raises TypeError because of mismatched dtypes. Running git bisect, I gather that this change was introduced in GH-26916, but neither the PR or the issue seem to suggest that this change was an explicit goal.

It might be arguable what the correct behavior should be, since it's probably valid that you shouldn't be able to compare a Timestamp to a float64, but it's also obvious what the return value should be: an empty Series of dtype bool.

The reason it would be valuable to allow this behavior is that a lot of people rely on pandas to infer their data types from the input, but no such inference can take place if the data set is empty, in the pre-0.25.0 world, it was possible to design an API like this:

def get_names_in_past(names: List[str], dates: List[datetime]) -> pd.DataFrame:
    df = pd.DataFrame({'names': names, 'dates': dates})
    return df[df['dates'] < pd.Timestamp.now()]

But starting in 0.25.0, when passed an empty list, you'd get a TypeError instead of an empty DataFrame, and you need to special-case the situation where df is empty. There are similar APIs you can imagine where the data are read from a CSV or some other source where the dtype is inferred from the contents that would start failing on empty data.

I think it would be reasonable to short-circuit comparisons with empty Series before the dtypes are compared, to always return an empty boolean Series.

Another reasonable option that I think would be more narrowly-scoped to solve the particular problem of dtype inference without data would be adding a flag to Series to indicate if the dtype is ambiguous. For pd.Series() it would be True and for pd.Series(dtype=float) it would be False. This would be more complicated to implement because you'd need to make sure that it's set correctly when the Series is part of a DataFrame and make sure it's set correctly in .astype, but that would allow you to preserve the error in comparing the wrong types when a type has been explicitly specified while avoiding the need for clunky code that has to explicitly handle empty DataFrames.

[simonjayhawkins]

Thanks @pganssle for the report. #31080 maybe related.