"accident" is a nice word for it. i jumped the gun. my bad.

I renamed this, to avoid confusion.

I've also changed my mind on backporting.
Let's try this for 2.0, otherwise other C/Cython changes can be very hard to backport.

Are we sure that this isn't breaking anything?

Are we sure that this isn't breaking anything?

This is the documented way to share C functions across extension modules. See https://docs.python.org/3/extending/extending.html#using-capsules

That's not what I meant. This is a pretty big pr where it's easy to miss something

The diff is a bit deceiving. The only real large addition to the code base is the 2 new include files. Everything else is just moving things to a better home (ex: function definitions of out parse_helper.h into pd_parser.c).

This actually improves type safety; the parsers.pyx changes before were happy redeclaring functions that are really implemented as returning a double * to functions returning a float64_t *. Guessing Cython relied on letting the linker resolve symbols before, but that returns a compilation warning/error in the current design.

Hmm why would this make backporting Cython changes any different?

The only file git doesn't correctly recognize as being moved is parse_helper.h which was last changed in 2020

Actually, looking at this again, I guess the only case backports would be affected are if a function was added/changed.

IIUC, the new process with the PyCapsule, is to modify the function in the original file, change the declarations in pd_parser/pd_datetime, and then modify pyx imports. Is that right?

I guess leaving it in only 2.1 could be fine, but wasn't sure if you were planning on refactoring more (in which case I'd expect this backported).

In general, I like to keep CI/build stuff as close as possible between the main/release branch.

The only things that I think would make an issue with a backport are:

1. Updating parse_helper.h
2. Adding a new C datetime function
3. Changing the declaration of a C datetime function

I don't think 1. is likely to happen and 2. And 3. aren't really things we should backport

IIUC, the new process with the PyCapsule, is to modify the function in the original file, change the declarations in pd_parser/pd_datetime, and then modify pyx imports. Is that right?

Generally, if we:

1. Update a function within a module (ex: np_datetime.c) then that's it; the capsule changes nothing
2. New/changed function declaration; would need to be reflected in the capsule as well (think of the capsule as the internal API)
3. Decide we want to use the capsule functions in a Cython module that hasn't previously used the capsule; we need to cimport the import_pandas_X function and invoke it; this is the same functionality we already have with cnp.import_array() and PyDateTime_IMPORT

Since it appears this is a reorganization, I suppose this could go in 2.0, but no strong preference on my end. Would you still prefer this go into 2.1 @lithomas1?

No strong preference anymore, let's just put in in main for now, and see if it sticks this time.