Step by step guide to understand protocol discussion, and decisions to be made #31
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Closed
| - Lack of Python object support (e.g. strings as Python objects in pandas) | ||
| - Booleans represented in a single bit, and some implementations may prefer a one byte representation | ||
| - No support for bfloat type (not sure if any dataframe implementation uses them currently) | ||
| - Limited number of units for timestamps and durations (nano/micro/milli-seconds, and seconds are |
There was a problem hiding this comment.
Arrow supports "extension types" (consisting of one of its native types + metadata on how to interpret it), so as long as the physical storage is supported by Arrow (which I think is the case for all those examples), if needed, all those types could be supported as extension types defined by the dataframe protocol.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This document complements #30, and provides detailed information on what are the different options regarding the dataframe exchange protocol, and what are the decisions that need to be made.
Since there are different backgrounds in the group, the aim is that everybody reading this document should be in the same page understanding the different topics of the discussion.
Publishing as a PR, so people can:
At the end of the document I summarize some of the decisions that IMHO need to be made, and that we can discuss in the call on Thursday. Feel free to propose other points.