ENH/INT: libpandas refactor

[jreback]

xref #11960

Create libpandas to support current user-facing API and container API (currently only numpy).

• ImmutableArray, single immutable dtyped container with a pandas.DataType dtype mirroring numpy/python scalars as well as current pandas extension types (Categorical, datetime w/tz), integer/boolean NA support, future types (string w/encoding, var-len), maybe ragged array etc.
  • cached-properties:
    • isnull/notnull
    • is_monotonic_increasing/is_monotonic_decreasing
  • methods:
    • indexing getitem ops (positional, slicing, fancy)
• Array - mutable sub-class of ImmutableArray
  • cached-properties above are now just properties
  • indexing supports setitem as well
• Table - ordered-dict-like container of Array

[jreback]

cc @wesm

[nickeubank]

@jreback @wesm I see this is xref'd to the copy on write PRs, but I'm not quite clear on how they relate -will this include copy-on-write behavior instead of the SettingOnACopy error?

Also, will it obviate anything added through #11500 / should I stop working on that?

[shoyer]

@nickeubank This is a proposed major refactor of pandas internals, which would indeed remove the need for #11500. Probably best to take a break on that for now, though I suppose it depends on how close you are -- this refactor will take quite some time :).

[nickeubank]

@shoyer OK, thanks for heads up. I'm not quite clear on how far I am... I think all the core components are in place but I am still figuring out how many edge-case problems are lurking in the tests that still fail.

Any chance you could re-open #11500? I may play with it a little more, but won't spend much time on it and will probably close if doesn't look like I'm really close. Just happy will make it into next version! :)

[shoyer]

You should have permissions to reopen #11500 yourself. If not, we can take
care of that for you.

On Wed, Jan 13, 2016 at 11:52 AM, Nick Eubank [email protected]
wrote:

@shoyer https://github.com/shoyer OK, thanks for heads up. I'm not
quite clear on how far I am... I think all the core components are in place
but I am still figuring out how many edge-case problems are lurking in the
tests that still fail.

Any chance you could re-open #11500
#11500? I may play with it a
little more, but won't spend much time on it and will probably close if
doesn't look like I'm really close. Just happy will make it into next
version! :)

—
Reply to this email directly or view it on GitHub
#11970 (comment).

[nickeubank]

can't seem to find -- used to be button at bottom, right? not there now, and pushes don't propagate, even after rebasing.

[jreback]

a lot of this is already happening in various forms, closing.