Skip to content

BUG: Switched shapes in ValueError msg in DataFrame construct (#20742) #24725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

3 changes: 3 additions & 0 deletions doc/source/whatsnew/v0.24.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1852,6 +1852,9 @@ Other
^^^^^

- Bug where C variables were declared with external linkage causing import errors if certain other C libraries were imported before Pandas. (:issue:`24113`)
- Changed ``ValueError`` message when constructiong a :class:`DataFrame` with parameters ``columns`` and ``index`` not matching the shape of the input data. New message: "Shape of passed values is (``index``, ``columns``), indices imply (``index``, ``columns``)" (:issue:`20742`)
- Changed ``ValueError`` message when constructiong a :class:`Panel` with parameters ``items``, ``major index`` and ``minor index`` not matching the shape of the input data. New message: "Shape of passed values is (``major``, ``minor``, ``items``), indices imply (``major``, ``minor``, ``items``)" (related to :issue:`20742`)


.. _whatsnew_0.24.0.contributors:

Expand Down
6 changes: 4 additions & 2 deletions pandas/core/internals/managers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1673,8 +1673,10 @@ def create_block_manager_from_arrays(arrays, names, axes):

def construction_error(tot_items, block_shape, axes, e=None):
""" raise a helpful message about our construction """
passed = tuple(map(int, [tot_items] + list(block_shape)))
implied = tuple(map(int, [len(ax) for ax in axes]))
passed = tuple(map(int, list(block_shape) + [tot_items]))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just do a reverse if ndim == 2 here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not a good idea for panels with 3 dimensions (items, major, minor)
with reversed for ndim == 2: (items, major, minor)
current: (major, minor, items)

The current version is the intended behavior, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't care about Panels, they are being removed shortly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this only affect ndim==2


implied = () if len(axes) == 0 \
else tuple(len(ax) for ax in axes[1:] + [axes[0]])
if passed == implied and e is not None:
raise e
if block_shape[0] == 0:
Expand Down
26 changes: 19 additions & 7 deletions pandas/tests/frame/test_constructors.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,25 +386,37 @@ def test_constructor_error_msgs(self):
'B': ['a', 'b', 'c']})

# wrong size ndarray, GH 3105
msg = r"Shape of passed values is \(3, 4\), indices imply \(3, 3\)"
msg = r"Shape of passed values is \(4, 3\), indices imply \(3, 3\)"
with pytest.raises(ValueError, match=msg):
DataFrame(np.arange(12).reshape((4, 3)),
columns=['foo', 'bar', 'baz'],
index=pd.date_range('2000-01-01', periods=3))

# see issue #20742
arr = np.array([[4, 5, 6]])
msg = r"Shape of passed values is \(1, 3\), indices imply \(1, 4\)"
with pytest.raises(ValueError, match=msg):
DataFrame(index=[0], columns=range(0, 4), data=arr)

# see issue #20742
arr = np.array([4, 5, 6])
msg = r"Shape of passed values is \(3, 1\), indices imply \(1, 4\)"
with pytest.raises(ValueError, match=msg):
DataFrame(index=[0], columns=range(0, 4), data=arr)

# higher dim raise exception
with pytest.raises(ValueError, match='Must pass 2-d input'):
DataFrame(np.zeros((3, 3, 3)), columns=['A', 'B', 'C'], index=[1])

# wrong size axis labels
msg = ("Shape of passed values "
r"is \(3, 2\), indices "
r"imply \(3, 1\)")
r"is \(2, 3\), indices "
r"imply \(1, 3\)")
with pytest.raises(ValueError, match=msg):
DataFrame(np.random.rand(2, 3), columns=['A', 'B', 'C'], index=[1])

msg = ("Shape of passed values "
r"is \(3, 2\), indices "
r"is \(2, 3\), indices "
r"imply \(2, 2\)")
with pytest.raises(ValueError, match=msg):
DataFrame(np.random.rand(2, 3), columns=['A', 'B'], index=[1, 2])
Expand Down Expand Up @@ -638,10 +650,10 @@ def _check_basic_constructor(self, empty):
assert frame.values.dtype == np.int64

# wrong size axis labels
msg = r'Shape of passed values is \(3, 2\), indices imply \(3, 1\)'
msg = r'Shape of passed values is \(2, 3\), indices imply \(1, 3\)'
with pytest.raises(ValueError, match=msg):
DataFrame(mat, columns=['A', 'B', 'C'], index=[1])
msg = r'Shape of passed values is \(3, 2\), indices imply \(2, 2\)'
msg = r'Shape of passed values is \(2, 3\), indices imply \(2, 2\)'
with pytest.raises(ValueError, match=msg):
DataFrame(mat, columns=['A', 'B'], index=[1, 2])

Expand Down Expand Up @@ -1805,7 +1817,7 @@ def test_from_records_to_records(self):
tm.assert_frame_equal(DataFrame.from_records(arr2), DataFrame(arr2))

# wrong length
msg = r'Shape of passed values is \(3, 2\), indices imply \(3, 1\)'
msg = r'Shape of passed values is \(2, 3\), indices imply \(1, 3\)'
with pytest.raises(ValueError, match=msg):
DataFrame.from_records(arr, index=index[:-1])

Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/io/json/test_pandas.py
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ def test_frame_from_json_bad_data(self):
json = StringIO('{"columns":["A","B"],'
'"index":["2","3"],'
'"data":[[1.0,"1"],[2.0,"2"],[null,"3"]]}')
msg = r"Shape of passed values is \(2, 3\), indices imply \(2, 2\)"
msg = r"Shape of passed values is \(3, 2\), indices imply \(2, 2\)"
with pytest.raises(ValueError, match=msg):
read_json(json, orient="split")

Expand Down
18 changes: 3 additions & 15 deletions pandas/tests/test_panel.py
Original file line number Diff line number Diff line change
Expand Up @@ -1143,23 +1143,11 @@ def test_from_dict_mixed_orient(self):
assert panel['A'].values.dtype == np.float64

def test_constructor_error_msgs(self):
msg = (r"Shape of passed values is \(3, 4, 5\), "
r"indices imply \(4, 5, 5\)")
msg = (r"Shape of passed values is \(4, 5, 3\), "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leave panel entirely.

r"indices imply \(5, 6, 4\)")
with pytest.raises(ValueError, match=msg):
Panel(np.random.randn(3, 4, 5),
lrange(4), lrange(5), lrange(5))

msg = (r"Shape of passed values is \(3, 4, 5\), "
r"indices imply \(5, 4, 5\)")
with pytest.raises(ValueError, match=msg):
Panel(np.random.randn(3, 4, 5),
lrange(5), lrange(4), lrange(5))

msg = (r"Shape of passed values is \(3, 4, 5\), "
r"indices imply \(5, 5, 4\)")
with pytest.raises(ValueError, match=msg):
Panel(np.random.randn(3, 4, 5),
lrange(5), lrange(5), lrange(4))
lrange(4), lrange(5), lrange(6))

def test_conform(self):
df = self.panel['ItemA'][:-5].filter(items=['A', 'B'])
Expand Down