Pass the errors and kwargs arguments through to astype in the columns… #25888

johnklehm · 2019-03-26T22:52:35Z

… dict logic branch.

codecov · 2019-03-26T23:29:14Z

Codecov Report

Merging #25888 into master will increase coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #25888      +/-   ##
==========================================
+ Coverage   91.47%   91.48%   +<.01%     
==========================================
  Files         175      175              
  Lines       52885    52885              
==========================================
+ Hits        48379    48380       +1     
+ Misses       4506     4505       -1

Flag	Coverage Δ
#multiple	`90.04% <100%> (ø)`	⬆️
#single	`41.82% <0%> (-0.01%)`	⬇️

Impacted Files	Coverage Δ
pandas/core/generic.py	`93.54% <100%> (ø)`	⬆️
pandas/util/testing.py	`89.84% <0%> (+0.1%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95c78d6...7c2e092. Read the comment docs.

codecov · 2019-03-26T23:29:15Z

Codecov Report

Merging #25888 into master will increase coverage by 0.33%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #25888      +/-   ##
==========================================
+ Coverage   91.47%   91.81%   +0.33%     
==========================================
  Files         175      175              
  Lines       52885    52580     -305     
==========================================
- Hits        48379    48274     -105     
+ Misses       4506     4306     -200

Flag	Coverage Δ
#multiple	`90.36% <ø> (+0.32%)`	⬆️
#single	`41.9% <ø> (+0.07%)`	⬆️

Impacted Files	Coverage Δ
pandas/core/generic.py	`93.54% <ø> (ø)`	⬆️
pandas/compat/pickle_compat.py	`69.13% <0%> (-6.48%)`	⬇️
pandas/plotting/_compat.py	`83.33% <0%> (-4.17%)`	⬇️
pandas/core/config_init.py	`96.96% <0%> (-2.24%)`	⬇️
pandas/plotting/_style.py	`77.17% <0%> (-0.49%)`	⬇️
pandas/compat/numpy/__init__.py	`92.85% <0%> (-0.48%)`	⬇️
pandas/core/groupby/grouper.py	`98.16% <0%> (-0.37%)`	⬇️
pandas/plotting/_misc.py	`38.46% <0%> (-0.23%)`	⬇️
pandas/core/computation/engines.py	`88.52% <0%> (-0.19%)`	⬇️
pandas/plotting/_timeseries.py	`65.28% <0%> (-0.18%)`	⬇️
... and 76 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95c78d6...343e26c. Read the comment docs.

jreback · 2019-03-27T01:39:45Z

what exactly are you trying to do? does this close an issue? always always need a test

johnklehm · 2019-03-27T01:59:35Z

If you look at lines 5705 [0], 5732 [1] the calls to astype pass through the errors arg and kwargs. Only when the column names are passed as a dict are you not allowed to set the errors behavior.

How I ran into this was when I am calling astype on one column I can set errors='coerce' but when I am calling astype on a dict of columns this does not work.

I was hoping to get feedback on this approach.

[0] https://github.com/pandas-dev/pandas/pull/25888/files#diff-03b380f521c43cf003207b0711bac67fR5705
[1] https://github.com/pandas-dev/pandas/pull/25888/files#diff-03b380f521c43cf003207b0711bac67fR5732

jreback · 2019-03-27T02:05:49Z

you would be better off to write a test that shows the behavior
writing the code to latch is generally pretty easy once you have good reproducing tests

johnklehm · 2019-03-27T15:45:18Z

Updated with the test @jreback . Side note is that I was tempted to rename Generic to TestGeneric but I refrained. :)

WillAyd

Can you also add a whatsnew note for v0.25?

pandas/tests/generic/test_generic.py

…ck for the absense of the exception we'd normally throw. Assert the resulting dataframe against a reference dataframe. Move the test case to test_dtypes.

johnklehm · 2019-03-29T20:58:19Z

The whatsnew note is added in 0430b38

….oops.

…ication in the prior commit.

johnklehm · 2019-03-30T15:31:44Z

Anything more I need to pursue on this?

pandas/tests/frame/common.py

pandas/tests/frame/test_dtypes.py

doc/source/whatsnew/v0.25.0.rst

pandas/tests/frame/common.py

pandas/tests/frame/test_dtypes.py

…uldn't raise. Add a comment about addressing GH25905. Improve the dict formatting. Simplify the dict key names to make the test easier to read.

WillAyd

Minor nits

pandas/tests/frame/common.py

pandas/tests/frame/test_dtypes.py

pep8speaks · 2019-03-31T00:09:25Z

Hello @johnklehm! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2019-03-31 00:32:36 UTC

johnklehm · 2019-03-31T00:16:22Z

PEP8 E241 is normally off by default along with E226 E242....is the pep8speaks bot implying E241 is adhered to for pandas?

pandas/tests/frame/test_dtypes.py

WillAyd · 2019-03-31T00:19:48Z

@johnklehm yep also need to fix up flake8 errors

…learer what is converted by astype. Swap the position of result and expected to match the style of the rest of the file.

johnklehm · 2019-03-31T01:00:01Z

I see 2 flake8 errors but don't think they're related to my changes:

./build/lib.linux-x86_64-3.7/pandas/_version.py:10:1: F401 'sys' imported but unused
./build/lib.linux-x86_64-3.7/pandas/tests/generic/test_generic.py:207:5: E303 too many blank lines (3)

WillAyd

lgtm @jreback

jreback · 2019-03-31T02:03:25Z

thanks @johnklehm

Pass the errors and kwargs arguments through to astype in the columns…

7c2e092

… dict logic branch.

gfyoung added Error Reporting Incorrect or improved errors from pandas DataFrame DataFrame data structure labels Mar 27, 2019

test_generic.py - Add a test for astype dict errors ignore.

11e5d42

WillAyd mentioned this pull request Mar 28, 2019

"errors" parameter has no effect on pd.DataFrame.astype if a dictionary is passed in as the dtype argument #25905

Closed

WillAyd requested changes Mar 28, 2019

View reviewed changes

pandas/tests/generic/test_generic.py Outdated Show resolved Hide resolved

gfyoung reviewed Mar 28, 2019

View reviewed changes

pandas/tests/generic/test_generic.py Outdated Show resolved Hide resolved

johnklehm added 2 commits March 29, 2019 14:07

Update the test data to relfect pandas-dev#25905. Add an explicit che…

d7c9d5a

…ck for the absense of the exception we'd normally throw. Assert the resulting dataframe against a reference dataframe. Move the test case to test_dtypes.

Add a whatsnew note about the bugfix for astype.

0430b38

johnklehm added 3 commits March 29, 2019 16:10

Keep Python 3.5 happy by removing the f-strings.

19afd08

ValueError inherits from Exception...so not_raises can be simplified.…

b04757a

….oops.

common.py - Fix an unused import vestigal from the not_raises simplif…

3a4849f

…ication in the prior commit.

WillAyd requested changes Mar 30, 2019

View reviewed changes

pandas/tests/frame/common.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

jreback requested changes Mar 30, 2019

View reviewed changes

doc/source/whatsnew/v0.25.0.rst Outdated Show resolved Hide resolved

pandas/tests/frame/common.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

Simplify the test by not checking explicitly for the exception it sho…

1c35cc7

…uldn't raise. Add a comment about addressing GH25905. Improve the dict formatting. Simplify the dict key names to make the test easier to read.

WillAyd requested changes Mar 30, 2019

View reviewed changes

pandas/tests/frame/common.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

johnklehm added 3 commits March 30, 2019 18:58

Remove some unused imports from the previous revisions.

3190b0f

Remove the docstring from the test.

ce115a8

One line per dict formatting.

f2dd79b

WillAyd requested changes Mar 31, 2019

View reviewed changes

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

pandas/tests/frame/test_dtypes.py Outdated Show resolved Hide resolved

PEP8 E241 fixes. Change the literals for the expected df to make it c…

343e26c

…learer what is converted by astype. Swap the position of result and expected to match the style of the rest of the file.

WillAyd approved these changes Mar 31, 2019

View reviewed changes

jreback added this to the 0.25.0 milestone Mar 31, 2019

jreback approved these changes Mar 31, 2019

View reviewed changes

jreback merged commit de3a85c into pandas-dev:master Mar 31, 2019

johnklehm deleted the astype-dict-errors-fix branch March 31, 2019 12:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Pass the errors and kwargs arguments through to astype in the columns… #25888

Pass the errors and kwargs arguments through to astype in the columns… #25888

johnklehm commented Mar 26, 2019

codecov bot commented Mar 26, 2019

codecov bot commented Mar 26, 2019 •

edited

Loading

jreback commented Mar 27, 2019

johnklehm commented Mar 27, 2019

jreback commented Mar 27, 2019

johnklehm commented Mar 27, 2019

WillAyd left a comment

johnklehm commented Mar 29, 2019 •

edited

Loading

johnklehm commented Mar 30, 2019

WillAyd left a comment

pep8speaks commented Mar 31, 2019 •

edited

Loading

johnklehm commented Mar 31, 2019

WillAyd commented Mar 31, 2019

johnklehm commented Mar 31, 2019 •

edited

Loading

WillAyd left a comment

jreback commented Mar 31, 2019

Pass the errors and kwargs arguments through to astype in the columns… #25888

Pass the errors and kwargs arguments through to astype in the columns… #25888

Conversation

johnklehm commented Mar 26, 2019

codecov bot commented Mar 26, 2019

Codecov Report

codecov bot commented Mar 26, 2019 • edited Loading

Codecov Report

jreback commented Mar 27, 2019

johnklehm commented Mar 27, 2019

jreback commented Mar 27, 2019

johnklehm commented Mar 27, 2019

WillAyd left a comment

Choose a reason for hiding this comment

johnklehm commented Mar 29, 2019 • edited Loading

johnklehm commented Mar 30, 2019

WillAyd left a comment

Choose a reason for hiding this comment

pep8speaks commented Mar 31, 2019 • edited Loading

Comment last updated at 2019-03-31 00:32:36 UTC

johnklehm commented Mar 31, 2019

WillAyd commented Mar 31, 2019

johnklehm commented Mar 31, 2019 • edited Loading

WillAyd left a comment

Choose a reason for hiding this comment

jreback commented Mar 31, 2019

codecov bot commented Mar 26, 2019 •

edited

Loading

johnklehm commented Mar 29, 2019 •

edited

Loading

pep8speaks commented Mar 31, 2019 •

edited

Loading

johnklehm commented Mar 31, 2019 •

edited

Loading