to_csv - allow to create folders.

[OmerJog]

currently to_csv will create a file if it does not exist. However it will not create a folder (nor sub-folders).
In this case it will raise
IOError: [Errno 2] No such file or directory:

The user must wrap his code in order for it to create a folder.

I'm not sure why the function behaves like this.. It would be great if it will also create folders if needed.

Reference from stack overflow:
https://stackoverflow.com/questions/47143836/pandas-dataframe-to-csv-raising-ioerror-no-such-file-or-directory

[jreback]

this is out of scope for pandas itself as it can easily be accomplished by the user

[JivanRoquet]

In this case, pd.to_csv() is also out of scope, since the user can "easily" open the file from Python and stream the dataframe content into it.

[manugarri]

@JivanRoquet i agree with you, pandas has been known for being beginner friendly, and a simple SO search can guarantee that making directories for output files is a very common issue for python users, particularly analysts that know how to use pandas, but dont really now about filesystems.

On the other side, the last pandas community survey indicated that feature creep is an important issue for the community. @jreback maybe makes sense to add a comment with instructions to the docstring of the to_csv function as a compromise?

[mchappychen]

Is possible to make folder drive from colab:
https://stackoverflow.com/questions/60706752/how-to-make-directory-on-the-google-colab
so you is can do it

[MarcoGorelli]

maybe makes sense to add a comment with instructions to the docstring of the to_csv function

A note in the docs might be useful, pull requests welcome

[leonarduschen]

Quoting from #42255 (review)

would also accept a better error message for cases like this (not sure how hard this is to detect that if you create the parent folders). (separate PR)

Do we want this? I'd be happy to make a PR, should be straightforward with os.path.isdir or pathlib.Path.is_dir (don't have to catch IOError), would help a lot of new users

I don't think automatic creation of directories is a great idea to do so automatically, so after the 2 above could consider via a keyword (yes some work for this). but please propose before you do the PR.

After that, maybe we can consider this too?

[jreback]

Do we want this? I'd be happy to make a PR, should be straightforward with os.path.isdir or pathlib.Path.is_dir (don't have to catch IOError), would help a lot of new users

would take this as an enhancement PR.

After that, maybe we can consider this too?

we would need a new keyword to do this (to preserver backward compabitility), but -1 on adding new keywords so unlikely