Skip to content

DOC: update the parquet docstring #20129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 12, 2018
Merged

DOC: update the parquet docstring #20129

Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
6d25e6c
DOC: update the parquet docstring
benman1 Mar 10, 2018
14ed668
update order of sections; return type fixed
benman1 Mar 10, 2018
7b7f8bc
fix issues as suggested in discussion of pull request
benman1 Mar 10, 2018
7ec8537
ignore validation error for **kwargs argument as per jorisvandenbossche
benman1 Mar 10, 2018
39423cd
update links as per feedback on pull request
benman1 Mar 10, 2018
d08f6b2
fix pep8 long line
benman1 Mar 10, 2018
b5329d2
remove return section
jorisvandenbossche Mar 10, 2018
e65840c
Update frame.py
jorisvandenbossche Mar 12, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 29 additions & 3 deletions pandas/core/frame.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1697,19 +1697,45 @@ def to_parquet(self, fname, engine='auto', compression='snappy',

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

An extended summary is necessary.

.. versionadded:: 0.21.0

This function writes the dataframe as a parquet file (see
:func:`pandas.io.parquet.to_parquet`). You can choose different parquet
backends, and have the option of compressing.

Parameters
----------
fname : str
string file path
String file path.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC we use lower for these? @jorisvandenbossche

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it is fine as start of the sentence and not being the exact type

engine : {'auto', 'pyarrow', 'fastparquet'}, default 'auto'
Parquet library to use. If 'auto', then the option
``io.parquet.engine`` is used. The default ``io.parquet.engine``
behavior is to try 'pyarrow', falling back to 'fastparquet' if
'pyarrow' is unavailable.
compression : {'snappy', 'gzip', 'brotli', None}, default 'snappy'
Name of the compression to use. Use ``None`` for no compression.
kwargs
Additional keyword arguments passed to the engine
kwargs : dict
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you change this line to **kwargs (without the dict, as you actually cannot pass it as a dict)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Writing it as **kwargs results in a validation error:

Errors found:
	Errors in parameters section
		Parameters {'kwargs'} not documented
		Unknown parameters {'**kwargs'}
		Parameter "**kwargs" has no type

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can ignore that error (the validation script is not yet perfect here :-))

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying!

Additional keyword arguments passed to the parquet library. See
the documentation for :func:`pandas.io.parquet.to_parquet` for
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you change this link to :ref:`io.parquet` (then it will link to the user guide with more information, the link you added now is to the docstring of to_parquet)

complete details.

Returns
-------
None

See Also
--------
read_parquet : Read a parquet file.
DataFrame.to_csv : Write a csv file.
DataFrame.to_sql : Write to a sql table.
DataFrame.to_hdf : Write to hdf.

Notes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

-----
This function requires either the fastparquet or pyarrow library.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make fastparquet and pyarrow into links to their home page?

The syntax is `pyarrow <https:// ...>`__


Examples
--------
>>> df = pd.DataFrame(data={'col1': [1, 2], 'col2': [3, 4]})
>>> df.to_parquet('df.parquet.gzip', compression='gzip')
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you also show using read_parquet to read this back

"""
from pandas.io.parquet import to_parquet
to_parquet(self, fname, engine,
Expand Down