Skip to content

TST/COMPAT: update csv test to infer time with pyarrow>=6.0 #44381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Nov 10, 2021
Merged

TST/COMPAT: update csv test to infer time with pyarrow>=6.0 #44381

merged 4 commits into from
Nov 10, 2021

Conversation

jorisvandenbossche
Copy link
Member

Closes #44231

@jorisvandenbossche jorisvandenbossche added Arrow pyarrow functionality IO CSV read_csv, to_csv labels Nov 10, 2021
@simonjayhawkins
Copy link
Member

Thanks @jorisvandenbossche. This has now starting failing on master and so I guess probably 1.3.x too. Will milestone 1.3.5 for backport.

@simonjayhawkins simonjayhawkins added this to the 1.3.5 milestone Nov 10, 2021
simonjayhawkins
simonjayhawkins reviewed Nov 10, 2021
View reviewed changes
pandas/compat/pyarrow.py
@@ -12,6 +12,7 @@
pa_version_under3p0 = _palv < Version("3.0.0")
pa_version_under4p0 = _palv < Version("4.0.0")
pa_version_under5p0 = _palv < Version("5.0.0")
pa_version_under6p0 = _palv < Version("6.0.0")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think also need to add pa_version_under6p0 = True below

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes, thanks

@lithomas1
Copy link
Member

lithomas1 commented Nov 10, 2021 •
edited
Loading

Thanks @jorisvandenbossche. This has now starting failing on master and so I guess probably 1.3.x too. Will milestone 1.3.5 for backport.

The new pyarrow engine should only be present on master, so I guess backport is not needed.

@jorisvandenbossche Do you know if pyarrow will add an option to configure this auto-inference?

@jorisvandenbossche jorisvandenbossche modified the milestones: 1.3.5, 1.4 Nov 10, 2021
@jorisvandenbossche
Copy link
Member Author

Good point.

I am not aware of any discussion about an option to turn it off (looking at https://issues.apache.org/jira/browse/ARROW-11247 now)

@jbrockmendel jbrockmendel mentioned this pull request Nov 10, 2021
Merged
4 tasks
@jorisvandenbossche jorisvandenbossche merged commit 67d3543 into pandas-dev:master Nov 10, 2021
@jorisvandenbossche jorisvandenbossche deleted the test-arrow-time branch November 10, 2021 19:25
@jorisvandenbossche
Copy link
Member Author

The sdist 3.10 build was failing (unrelated I suppose), is that a know failure ? (I don't directly see an issue about it)

nickleus27 pushed a commit to nickleus27/pandas that referenced this pull request Nov 28, 2021
@jorisvandenbossche @nickleus27
TST/COMPAT: update csv test to infer time with pyarrow>=6.0 (pandas-d…
b0c1671 
…ev#44381)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@simonjayhawkins simonjayhawkins simonjayhawkins left review comments

Assignees
No one assigned
Labels
Arrow pyarrow functionality IO CSV read_csv, to_csv
Projects
None yet
Milestone
1.4
Development

Successfully merging this pull request may close these issues.

BUG: test_date_col_as_index_col[pyarrow] failing locally with pyarrow 6.0.0
3 participants
@jorisvandenbossche @simonjayhawkins @lithomas1