ENH: Add Stata 119 writer #30959

bashtage · 2020-01-13T11:13:48Z

Add support for writing Stata 119 format files
Rename new writer to StataWriterUTF8 since no longer version-specific
Improve exception message for unsupported files
Fix small issues in to_stata missed in 118

xref Support writing unicode characters in df.to_stata() #23573 (already closed)
tests added / passed
passes black pandas
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

bashtage · 2020-01-13T11:21:25Z

The rename should be easy now since it isn't released yet.

WillAyd

looks good a few minor things

WillAyd · 2020-01-13T16:49:10Z

doc/source/whatsnew/v1.0.0.rst

@@ -224,7 +224,7 @@ Other enhancements
 - :meth:`DataFrame.sort_values` and :meth:`Series.sort_values` have gained ``ignore_index`` keyword to be able to reset index after sorting (:issue:`30114`)
 - :meth:`DataFrame.sort_index` and :meth:`Series.sort_index` have gained ``ignore_index`` keyword to reset index (:issue:`30114`)
 - :meth:`DataFrame.drop_duplicates` has gained ``ignore_index`` keyword to reset index (:issue:`30114`)
- Added new writer for exporting Stata dta files in version 118, ``StataWriter118``.  This format supports exporting strings containing Unicode characters (:issue:`23573`)
+- Added new writer for exporting Stata dta files in version 118 and 119, ``StataWriterUTF8``.  This format supports exporting strings containing Unicode characters (:issue:`23573`)


Already released 1.0 so will want to move to the 1.1 whatsnew when that comes available

Missed that. This is unfortunate. Are deprecation cycles still OK since this renames a class to avoid creating a trivial class for the 119 version?

Hmm so you want to rename what was added in 1.0 to this right? Maybe OK as an exception if you would just immediately deprecate anyway but cc @TomAugspurger for thoughts

That was the idea -- it is a very trivial change to enable the new format, but StataWriter118 doesn't make sense when it supports format 119 too.

I've gone down the deprecation path to 2.0 deprecation. TBH I don't think (m)any will use StataWriter118 directly since to_stata contains virtually all of the functionality.

I can revert this if quick deprecation/renaming is better.

If it's simpler I think OK to just change since this was just released in the RC and not do a deprecation cycle, but @jreback as well for thoughts

Ok, I can get a restored version back pretty quickly. I think avoiding deprecation would be good if it isn't too controversial.

To confirm, StataWriter118 was new in 1.0? If so, then I think we're OK to change the name for 1.0.

@TomAugspurger yes, it is new in 1.0.0. I have moved back to changing the class name for 1.0.0 with no reference for 1.1.0, and so no deprecation.

WillAyd · 2020-01-13T16:51:42Z

pandas/io/stata.py

    >>> writer.write_file()
    """

    _encoding = "utf-8"
-    _dta_version = 118
+
+    def __init__(


Can you annotate new functions?

Can you take a look at what I used and see if it looks right?

annotations are not blocking here, nice to have sure

pandas/core/frame.py

bashtage · 2020-01-13T23:47:12Z

I can squash/rebase if there is a consensus that targetting 1.0.0 is a good idea.

jreback

small comments; do you have sufficient tests coverage for all of the added cases?

pandas/core/frame.py

jreback · 2020-01-14T04:40:58Z

pandas/core/frame.py

        if version == 114:
            if convert_strl is not None:
                raise ValueError("strl is not supported in format 114")
            from pandas.io.stata import StataWriter as statawriter
        else:
            if version == 117:
                from pandas.io.stata import StataWriter117 as statawriter
-            else:
-                from pandas.io.stata import StataWriter118 as statawriter
+            else:  # versions 118 and 119


can you make this just if/elif/else and not a nested if/else

pandas/io/stata.py

jreback · 2020-01-14T04:42:26Z

pandas/io/stata.py

    >>> writer.write_file()
    """

    _encoding = "utf-8"
-    _dta_version = 118
+
+    def __init__(


annotations are not blocking here, nice to have sure

pandas/io/stata.py

jreback · 2020-01-14T04:44:19Z

I think ok to do this for 1.0.0

jreback · 2020-01-14T04:44:45Z

is there an issue that this would close? if so can you update the top

Add support for writing Stata 119 format files Improve exception message for unsupported files Fix small issues in to_stata missed in 118 Add annotations

bashtage · 2020-01-14T09:17:10Z

do you have sufficient tests coverage for all of the added cases?

Coverage of new code (StataWriterUTF8) introduced in 1.0.0 is 100%.

bashtage · 2020-01-14T09:19:15Z

is there an issue that this would close? if so can you update the top

There is no issue open for this. I cross-referenced #23573 which is the closest, but is already closed.

jreback

lgtm. i think its worth providing a fixture for version. ping on green.

pandas/tests/io/test_stata.py

WillAyd · 2020-01-14T16:03:04Z

Thanks @bashtage

Co-authored-by: Kevin Sheppard <[email protected]>

simonjayhawkins added Enhancement IO Stata read_stata, to_stata labels Jan 13, 2020

WillAyd requested changes Jan 13, 2020

View reviewed changes

mroeschke reviewed Jan 13, 2020

View reviewed changes

pandas/core/frame.py Outdated Show resolved Hide resolved

bashtage force-pushed the stata-writer-119 branch 2 times, most recently from 699b44f to 287ca23 Compare January 13, 2020 23:45

jreback requested changes Jan 14, 2020

View reviewed changes

jreback added this to the 1.0.0 milestone Jan 14, 2020

ENH: Add Stata 119 writer

e4eaecd

Add support for writing Stata 119 format files Improve exception message for unsupported files Fix small issues in to_stata missed in 118 Add annotations

bashtage force-pushed the stata-writer-119 branch from 287ca23 to e4eaecd Compare January 14, 2020 09:15

jreback approved these changes Jan 14, 2020

View reviewed changes

pandas/tests/io/test_stata.py Show resolved Hide resolved

bashtage requested a review from WillAyd January 14, 2020 14:22

WillAyd approved these changes Jan 14, 2020

View reviewed changes

WillAyd merged commit d78c061 into pandas-dev:master Jan 14, 2020

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Jan 14, 2020

Backport PR pandas-dev#30959: ENH: Add Stata 119 writer

ba14146

meeseeksmachine mentioned this pull request Jan 14, 2020

Backport PR #30959 on branch 1.0.x (ENH: Add Stata 119 writer) #31009

Merged

simonjayhawkins pushed a commit that referenced this pull request Jan 14, 2020

Backport PR #30959: ENH: Add Stata 119 writer (#31009)

85f2172

Co-authored-by: Kevin Sheppard <[email protected]>

bashtage deleted the stata-writer-119 branch July 28, 2020 14:41

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ENH: Add Stata 119 writer #30959

ENH: Add Stata 119 writer #30959

bashtage commented Jan 13, 2020 •

edited

Loading

bashtage commented Jan 13, 2020

WillAyd left a comment

WillAyd Jan 13, 2020

bashtage Jan 13, 2020

WillAyd Jan 13, 2020

bashtage Jan 13, 2020

WillAyd Jan 13, 2020

bashtage Jan 13, 2020

TomAugspurger Jan 13, 2020

bashtage Jan 13, 2020

WillAyd Jan 13, 2020

bashtage Jan 13, 2020

jreback Jan 14, 2020

bashtage commented Jan 13, 2020

jreback left a comment

jreback Jan 14, 2020

jreback Jan 14, 2020

jreback commented Jan 14, 2020

jreback commented Jan 14, 2020

bashtage commented Jan 14, 2020

bashtage commented Jan 14, 2020

jreback left a comment

WillAyd commented Jan 14, 2020

ENH: Add Stata 119 writer #30959

ENH: Add Stata 119 writer #30959

Conversation

bashtage commented Jan 13, 2020 • edited Loading

bashtage commented Jan 13, 2020

WillAyd left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bashtage commented Jan 13, 2020

jreback left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jreback commented Jan 14, 2020

jreback commented Jan 14, 2020

bashtage commented Jan 14, 2020

bashtage commented Jan 14, 2020

jreback left a comment

Choose a reason for hiding this comment

WillAyd commented Jan 14, 2020

bashtage commented Jan 13, 2020 •

edited

Loading