MAINT: Remove np.array_equal calls in tests #18047

gfyoung · 2017-10-31T04:47:18Z

Preferable to use tm.assert_numpy_array_equal instead.

gfyoung · 2017-10-31T04:49:12Z

A couple of questions going forward:

There are several instances where they used np.array_equal to check non-array inequality. Would it be useful to create a method called tm.assert_numpy_array_not_equal for this case?
tm.assert_numpy_array_equal enforces that we must be comparing two numpy arrays, which makes it slightly cumbersome at times to compare two arrays (e.g. they are lists). Perhaps we could take a page out of numpy book and create a function called tm.assert_array_equal ?

jreback · 2017-10-31T10:21:05Z

There are several instances where they used np.array_equal to check non-array inequality. Would it be useful to create a method called tm.assert_numpy_array_not_equal for this case?

this has come up a couple of times, sure I guess that would be ok

tm.assert_numpy_array_equal enforces that we must be comparing two numpy arrays, which makes it slightly cumbersome at times to compare two arrays (e.g. they are lists). Perhaps we could take a page out of numpy book and create a function called tm.assert_array_equal ?

no, list comparison is directly handled by assert

jreback

need to add np.array_equal to the black-list in ci/lint.sh (near the end), we check for using black-listed functions and fail the build.

jreback · 2017-10-31T10:23:54Z

pandas/tests/indexes/datetimes/test_date_range.py

@@ -22,7 +22,10 @@

 def eq_gen_range(kwargs, expected):
    rng = generate_range(**kwargs)
-    assert (np.array_equal(list(rng), expected))


I would remove this function and just directly compare these (only 3 uses)

jreback · 2017-10-31T10:24:31Z

pandas/tests/io/parser/test_textreader.py

-        expected = {0: ['a', 'd', 'g', 'l'],
-                    1: ['b', 'e', 'h', 'm'],
-                    2: ['c', 'f', 'i', 'n']}
+        expected = {0: np.array(['a', 'd', 'g', 'l'], dtype=object),


I think assert can do this; don't convert to a numpy array just to compare

The output of Textreader.read is an array, so would need to convert.

jreback · 2017-10-31T10:24:33Z

pandas/tests/io/parser/test_textreader.py

@@ -225,7 +225,7 @@ def test_escapechar(self):
        reader = TextReader(StringIO(data), delimiter=',', header=None,
                            escapechar='\\')
        result = reader.read()
-        expected = {0: ['"hello world"'] * 3}
+        expected = {0: np.array(['"hello world"'] * 3, dtype=object)}


Same as above

gfyoung · 2017-10-31T18:43:37Z

need to add np.array_equal to the black-list in ci/lint.sh (near the end)

There are many instances of this method being called in the source code...I think it shouldn't be allowed in tests BUT be careful with source code though.

jreback · 2017-10-31T19:25:10Z

There are many instances of this method being called in the source code...I think it shouldn't be allowed in tests BUT be careful with source code though.

that line in lint should be

grep -r -E --include '*.py' --exclude testing.py '(numpy|np)\.testing' pandas\tests
(and have to add in the check for array_equal in the regex)

jreback · 2017-10-31T19:26:53Z

e.g.

(pandas) bash-3.2$   grep -r -E --include '*.py' --exclude testing.py '(numpy|np)\.testing|(numpy|np)\.array_equal' pandas/tests
pandas/tests/dtypes/test_cast.py:        assert (np.array_equal(result, arr))
pandas/tests/dtypes/test_cast.py:        assert (np.array_equal(result, expected))

codecov · 2017-11-01T04:42:46Z

Codecov Report

Merging #18047 into master will decrease coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #18047      +/-   ##
==========================================
- Coverage   91.25%   91.23%   -0.02%     
==========================================
  Files         163      163              
  Lines       50115    50114       -1     
==========================================
- Hits        45730    45720      -10     
- Misses       4385     4394       +9

Flag	Coverage Δ
#multiple	`89.04% <ø> (-0.01%)`	⬇️
#single	`40.24% <ø> (-0.06%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/core/frame.py	`97.75% <0%> (-0.1%)`	⬇️
pandas/tslib.py	`100% <0%> (ø)`	⬆️
pandas/compat/pickle_compat.py	`75.6% <0%> (ø)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 881ee30...c7d2e38. Read the comment docs.

codecov · 2017-11-01T04:42:50Z

Codecov Report

Merging #18047 into master will decrease coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #18047      +/-   ##
==========================================
- Coverage   91.25%   91.23%   -0.03%     
==========================================
  Files         163      163              
  Lines       50120    50115       -5     
==========================================
- Hits        45737    45721      -16     
- Misses       4383     4394      +11

Flag	Coverage Δ
#multiple	`89.04% <ø> (-0.01%)`	⬇️
#single	`40.23% <ø> (-0.15%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/gbq.py	`25% <0%> (-58.34%)`	⬇️
pandas/io/html.py	`85.17% <0%> (-0.81%)`	⬇️
pandas/core/frame.py	`97.75% <0%> (-0.1%)`	⬇️
pandas/core/groupby.py	`92.03% <0%> (-0.01%)`	⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 15fa4bd...902f4e9. Read the comment docs.

jreback · 2017-11-01T10:31:44Z

@gfyoung can you do this so we can ensure these don't appear going forward

gfyoung · 2017-11-01T16:49:19Z

@jreback : There are several instances that I did not remove yet because I want to do a more thorough refactoring in light of my comment above

I'll do that in a follow-up PR if that works.

jreback · 2017-11-02T11:28:16Z

that sounds good @gfyoung thanks!

xref pandas-dev#18047

xref #18047

xref pandas-dev#18047

(cherry picked from commit 194dbff)

xref pandas-dev#18047 (cherry picked from commit 27bbea7)

(cherry picked from commit 194dbff)

xref #18047 (cherry picked from commit 27bbea7)

gfyoung added the Testing pandas testing functions or related to the test suite label Oct 31, 2017

gfyoung added this to the 0.21.1 milestone Oct 31, 2017

gfyoung force-pushed the numpy-array-equal-remove branch from c5b4ada to 4d21ad1 Compare October 31, 2017 06:26

jreback requested changes Oct 31, 2017

View reviewed changes

gfyoung force-pushed the numpy-array-equal-remove branch from 4d21ad1 to c7d2e38 Compare November 1, 2017 04:42

gfyoung mentioned this pull request Nov 1, 2017

TST: separate out grouping-type tests #18057

Merged

gfyoung force-pushed the numpy-array-equal-remove branch from c7d2e38 to 9adf991 Compare November 1, 2017 09:27

jreback modified the milestones: 0.21.1, 0.22.0 Nov 1, 2017

gfyoung added 2 commits November 1, 2017 10:11

MAINT: Remove np.array_equal calls in tests

b2a0826

MAINT: Remove eq_gen_range in test_date_range

902f4e9

gfyoung force-pushed the numpy-array-equal-remove branch from 9adf991 to 902f4e9 Compare November 1, 2017 17:11

jreback approved these changes Nov 2, 2017

View reviewed changes

jreback merged commit 194dbff into pandas-dev:master Nov 2, 2017

gfyoung deleted the numpy-array-equal-remove branch November 2, 2017 16:06

gfyoung modified the milestones: 0.22.0, 0.21.1 Nov 3, 2017

gfyoung mentioned this pull request Nov 3, 2017

TST: Remove even more uses np.array_equal in tests #18087

Merged

jreback added a commit to jreback/pandas that referenced this pull request Nov 3, 2017

COMPAT: compare platform return on 32-bit

f2c6260

xref pandas-dev#18047

jreback mentioned this pull request Nov 3, 2017

COMPAT: compare platform return on 32-bit #18090

Merged

jreback added a commit that referenced this pull request Nov 3, 2017

COMPAT: compare platform return on 32-bit (#18090)

27bbea7

xref #18047

ghost pushed a commit to reef-technologies/pandas that referenced this pull request Nov 3, 2017

MAINT: Remove np.array_equal calls in tests (pandas-dev#18047)

af36934

ghost pushed a commit to reef-technologies/pandas that referenced this pull request Nov 3, 2017

COMPAT: compare platform return on 32-bit (pandas-dev#18090)

798401b

xref pandas-dev#18047

1kastner pushed a commit to 1kastner/pandas that referenced this pull request Nov 5, 2017

MAINT: Remove np.array_equal calls in tests (pandas-dev#18047)

7691209

1kastner pushed a commit to 1kastner/pandas that referenced this pull request Nov 5, 2017

COMPAT: compare platform return on 32-bit (pandas-dev#18090)

4bfbca9

xref pandas-dev#18047

No-Stream pushed a commit to No-Stream/pandas that referenced this pull request Nov 28, 2017

MAINT: Remove np.array_equal calls in tests (pandas-dev#18047)

33fef4f

No-Stream pushed a commit to No-Stream/pandas that referenced this pull request Nov 28, 2017

COMPAT: compare platform return on 32-bit (pandas-dev#18090)

5d1299f

xref pandas-dev#18047

TomAugspurger pushed a commit to TomAugspurger/pandas that referenced this pull request Dec 8, 2017

MAINT: Remove np.array_equal calls in tests (pandas-dev#18047)

992b0f9

(cherry picked from commit 194dbff)

TomAugspurger pushed a commit to TomAugspurger/pandas that referenced this pull request Dec 8, 2017

COMPAT: compare platform return on 32-bit (pandas-dev#18090)

e2c596d

xref pandas-dev#18047 (cherry picked from commit 27bbea7)

TomAugspurger pushed a commit that referenced this pull request Dec 11, 2017

MAINT: Remove np.array_equal calls in tests (#18047)

27794fa

(cherry picked from commit 194dbff)

TomAugspurger pushed a commit that referenced this pull request Dec 11, 2017

COMPAT: compare platform return on 32-bit (#18090)

5a006d3

xref #18047 (cherry picked from commit 27bbea7)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

MAINT: Remove np.array_equal calls in tests #18047

MAINT: Remove np.array_equal calls in tests #18047

gfyoung commented Oct 31, 2017

gfyoung commented Oct 31, 2017

jreback commented Oct 31, 2017

jreback left a comment

jreback Oct 31, 2017

gfyoung Nov 1, 2017

jreback Oct 31, 2017

gfyoung Nov 2, 2017

jreback Oct 31, 2017

gfyoung Nov 2, 2017

gfyoung commented Oct 31, 2017

jreback commented Oct 31, 2017

jreback commented Oct 31, 2017

codecov bot commented Nov 1, 2017

codecov bot commented Nov 1, 2017 •

edited

Loading

jreback commented Nov 1, 2017

gfyoung commented Nov 1, 2017

jreback commented Nov 2, 2017

MAINT: Remove np.array_equal calls in tests #18047

MAINT: Remove np.array_equal calls in tests #18047

Conversation

gfyoung commented Oct 31, 2017

gfyoung commented Oct 31, 2017

jreback commented Oct 31, 2017

jreback left a comment

Choose a reason for hiding this comment

jreback Oct 31, 2017

Choose a reason for hiding this comment

gfyoung Nov 1, 2017

Choose a reason for hiding this comment

jreback Oct 31, 2017

Choose a reason for hiding this comment

gfyoung Nov 2, 2017

Choose a reason for hiding this comment

jreback Oct 31, 2017

Choose a reason for hiding this comment

gfyoung Nov 2, 2017

Choose a reason for hiding this comment

gfyoung commented Oct 31, 2017

jreback commented Oct 31, 2017

jreback commented Oct 31, 2017

codecov bot commented Nov 1, 2017

Codecov Report

codecov bot commented Nov 1, 2017 • edited Loading

Codecov Report

jreback commented Nov 1, 2017

gfyoung commented Nov 1, 2017

jreback commented Nov 2, 2017

codecov bot commented Nov 1, 2017 •

edited

Loading