CoW: Avoid warning in apply for mixed dtype frame

[phofl]

• closes #xxxx (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

Yikes this is ugly

xref #56019

[jorisvandenbossche]

This is something we want to remove again after CoW is enforced? (i.e. it's not needed for CoW itself after 3.0)
In that case I would maybe add a TODO that this can be removed later, as from just looking at the code it is not obvious that it's only for the warning mode (like it's not behind an if check)

[phofl]

it's not only for the warning mode. Apply currently trigger a copy for every row, which is something that this helps us avoid

[jorisvandenbossche]

Ah, yes of course.

[jorisvandenbossche]

To be honest, I don't understand how this part adds an extra ref after the first row (i.e. the reason you need to reset refs to have one ref to only itself)

[jorisvandenbossche]

Ah, I assume that's because we create a shallow copy of the result:

pandas/pandas/core/apply.py

Lines 1078 to 1084 in 7b528c9

	for i, v in enumerate(series_gen):
	# ignore SettingWithCopy here in case the user mutates
	results[i] = self.func(v, *self.args, **self.kwargs)
	if isinstance(results[i], ABCSeries):
	# If we have a view on v, we need to make a copy because
	# series_generator will swap out the underlying data
	results[i] = results[i].copy(deep=False)

[phofl]

Yep exactly

[jorisvandenbossche]

Would it be cleaner to create a shallow "copy" of the Series that does not keep a reference? (I don't think we have any helper function to do that right now? except for Series(results[i]._values, copy=False, index=.., name=..), like what you did in the eval PR to create Series objects without a reference)

[phofl]

My assumption was that this is optimized for performance and I didn’t want to screw with that. And keeping the ugly parts in one method seemed like a fine compromise

that said, the udf could also set up references that could screw us over

[jorisvandenbossche]

I didn’t want to screw with that

It's already taking a shallow copy always (even when not needed), so having a specialized shallow copy without tracking refs should at best slightly improve things.

The logic about requiring this shallow copy also already lives where the series_generator is used, so I think I find it slightly clearer to consolidate that logic there

Now, actually implementing a Series._shallow_copy_without_refs() is not fully trivial because AFAIK we can't just reuse the existing mgr and block copy methods and reset the refs after the fact, as that then already appended the refs object which is shared with the series generator series with a new ref.
So let's consider that for a follow-up PR if I want to pursue that ;)

[jorisvandenbossche]

I am just going to update some comments here, then

[phofl]

Yeah I'd rather not do too many helpers that make this easy since you really shouldn't be doing this