Skip to content

DOC: update the parquet docstring #20129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 12, 2018
Merged

DOC: update the parquet docstring #20129

Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
6d25e6c
DOC: update the parquet docstring
benman1 Mar 10, 2018
14ed668
update order of sections; return type fixed
benman1 Mar 10, 2018
7b7f8bc
fix issues as suggested in discussion of pull request
benman1 Mar 10, 2018
7ec8537
ignore validation error for **kwargs argument as per jorisvandenbossche
benman1 Mar 10, 2018
39423cd
update links as per feedback on pull request
benman1 Mar 10, 2018
d08f6b2
fix pep8 long line
benman1 Mar 10, 2018
b5329d2
remove return section
jorisvandenbossche Mar 10, 2018
e65840c
Update frame.py
jorisvandenbossche Mar 12, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 26 additions & 3 deletions pandas/core/frame.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1697,19 +1697,42 @@ def to_parquet(self, fname, engine='auto', compression='snappy',

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An extended summary is necessary.

.. versionadded:: 0.21.0

This function writes the dataframe as a parquet file. You
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a link to the parquet docs (copy from io.rst) in References

can choose different parquet backends, and have the option
of compressing.

Parameters
----------
fname : str
string file path
String file path.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC we use lower for these? @jorisvandenbossche

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it is fine as start of the sentence and not being the exact type

engine : {'auto', 'pyarrow', 'fastparquet'}, default 'auto'
Parquet library to use. If 'auto', then the option
``io.parquet.engine`` is used. The default ``io.parquet.engine``
behavior is to try 'pyarrow', falling back to 'fastparquet' if
'pyarrow' is unavailable.
compression : {'snappy', 'gzip', 'brotli', None}, default 'snappy'
Name of the compression to use. Use ``None`` for no compression.
kwargs
Additional keyword arguments passed to the engine
kwargs : dict
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you change this line to **kwargs (without the dict, as you actually cannot pass it as a dict)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Writing it as **kwargs results in a validation error:

Errors found:
	Errors in parameters section
		Parameters {'kwargs'} not documented
		Unknown parameters {'**kwargs'}
		Parameter "**kwargs" has no type

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can ignore that error (the validation script is not yet perfect here :-))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying!

Additional keyword arguments passed to the engine.

Returns
----------
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are to many hyphens.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make the underlines the same length as the title

None

See Also
--------
DataFrame.to_csv : write a csv file.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

link to pd.read_parquet

DataFrame.to_sql : write to a sql table.
DataFrame.to_hdf : write to hdf.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the "See Also" section, please check the section 5 of https://python-sprints.github.io/pandas/guide/pandas_docstring.html


Notes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

----------
Requires either fastparquet or pyarrow libraries.

Examples
----------
>>> df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4]})
>>> df.to_parquet('df.parquet.gzip', compression='gzip')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also show using read_parquet to read this back

"""
from pandas.io.parquet import to_parquet
to_parquet(self, fname, engine,
Expand Down