ENH: pandas read_* wildcard #15904 #16166
Conversation
Codecov Report
@@ Coverage Diff @@
## master #16166 +/- ##
=======================================
Coverage 90.87% 90.87%
=======================================
Files 162 162
Lines 50816 50816
=======================================
Hits 46178 46178
Misses 4638 4638
Continue to review full report at Codecov.
|
Codecov Report
@@ Coverage Diff @@
## master #16166 +/- ##
==========================================
- Coverage 90.87% 90.86% -0.01%
==========================================
Files 162 162
Lines 50816 50819 +3
==========================================
- Hits 46178 46176 -2
- Misses 4638 4643 +5
Continue to review full report at Codecov.
|
doc/source/cookbook.rst
Outdated
|
|
||
| import glob | ||
| frames = [] | ||
| for f in glob.glob('file_*.csv'): |
There was a problem hiding this comment.
idiomatically
result = pd.concat([pd.read_csv(f) for f in glob.glob('file_*.csv')], ignore_index=True)
usually you want the ignore_index=True
doc/source/cookbook.rst
Outdated
| result = pd.concat(frames) | ||
|
|
||
| This performs significantly better than using ``pd.append`` to add each of the files to an existing DataFrame. | ||
| Finally, this strategy will work with the other ``read_`` functions described in the :ref:`io docs<io>`. |
There was a problem hiding this comment.
added, but with (...) to be consistent with a later usage.
doc/source/cookbook.rst
Outdated
| frames.append(pd.read_csv(f)) | ||
| result = pd.concat(frames) | ||
|
|
||
| This performs significantly better than using ``pd.append`` to add each of the files to an existing DataFrame. |
There was a problem hiding this comment.
pd.append is not a thing. Remove the first sentence in any event.
There was a problem hiding this comment.
Removed here and in io.rst
doc/source/cookbook.rst
Outdated
|
|
||
| frames = [] | ||
| files = ['file_0.csv', 'file_1.csv', 'file_2.csv'] | ||
| for f in files: |
There was a problem hiding this comment.
see below for the idiom to do this
doc/source/cookbook.rst
Outdated
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
|
||
| The best way to combine multiple files into a single DataFrame is to read the individual frames one by one, put all | ||
| of the individual frames into a list, and then combine the frames in the list using ``pd.concat``: |
There was a problem hiding this comment.
Changed here and in io.rst
doc/source/cookbook.rst
Outdated
| data = pd.DataFrame(np.random.randn(10, 4)) | ||
| data.to_csv('file_{}.csv'.format(i)) | ||
|
|
||
| frames = [] |
There was a problem hiding this comment.
You can delete this frames now.
There was a problem hiding this comment.
Removed, thanks.
doc/source/cookbook.rst
Outdated
| .. ipython:: python | ||
|
|
||
| import glob | ||
| frames = [] |
There was a problem hiding this comment.
Same with this frames
|
I think this should have a pointer from the |
|
@dwkenefick thanks! The doc build should be done in 20-30 minutes, if you want to check the output here |
* DOC: pandas read_* wildcard pandas-dev#15904 Added example in cookbook about reading multiple files into a dataframe.
* DOC: pandas read_* wildcard pandas-dev#15904 Added example in cookbook about reading multiple files into a dataframe.
git diff upstream/master --name-only -- '*.py' | flake8 --diff