Skip to content

Accept empty dataframes in DataFrame.to_parquet #27341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jul 11, 2019

Conversation

CJStadler
Copy link
Contributor

Fixes #27339

I wrote two tests because fastparquet adds a name to the index when it deserializes.

  • closes #xxxx
  • tests added / passed
  • passes black pandas
  • passes git diff upstream/master -u -- "*.py" | flake8 --diff
  • whatsnew entry

@jreback jreback added the IO Parquet parquet, feather label Jul 11, 2019
Copy link
Contributor

@jreback jreback left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small comments, can you add a whatsnew note (bug fixes in io), ping on green.

@@ -471,6 +471,10 @@ def test_partition_cols_supported(self, pa, df_full):
assert len(dataset.partitions.partition_names) == 2
assert dataset.partitions.partition_names == set(partition_cols)

def test_empty_dataframe(self, pa):
df = pd.DataFrame()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add the github issue number as a comment

@@ -566,3 +570,9 @@ def test_error_on_using_partition_cols_and_partition_on(self, fp, df_full):
partition_on=partition_cols,
partition_cols=partition_cols,
)

def test_empty_dataframe(self, fp):
df = pd.DataFrame()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

@CJStadler
Copy link
Contributor Author

Thanks @jreback. Should this go in the whatsnew for 0.25.0 or 0.25.1?

@jreback
Copy link
Contributor

jreback commented Jul 11, 2019

Thanks @jreback. Should this go in the whatsnew for 0.25.0 or 0.25.1?

you can put in 0.25.0

@jreback jreback added this to the 0.25.0 milestone Jul 11, 2019
@jreback jreback added the Compat pandas objects compatability with Numpy or Python functions label Jul 11, 2019
@jreback
Copy link
Contributor

jreback commented Jul 11, 2019

lgtm. @CJStadler ping on green.

@CJStadler
Copy link
Contributor Author

It's green @jreback. Thanks!

@jreback jreback merged commit eeff07f into pandas-dev:master Jul 11, 2019
@jreback
Copy link
Contributor

jreback commented Jul 11, 2019

thanks!

@CJStadler CJStadler deleted the parquet-empty-df branch July 11, 2019 20:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Compat pandas objects compatability with Numpy or Python functions IO Parquet parquet, feather
Projects
None yet
Development

Successfully merging this pull request may close these issues.

DataFrame.to_parquet raises fails when dataframe is empty
3 participants