Skip to content

BUG: Switched shapes in ValueError msg in DataFrame construct (#20742) #24725

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2 changes: 2 additions & 0 deletions doc/source/whatsnew/v0.24.0.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1108,6 +1108,7 @@ update the ``ExtensionDtype._metadata`` tuple to match the signature of your
- :meth:`~pandas.api.types.ExtensionArray.searchsorted` has been added (:issue:`24350`)
- Support for reduction operations such as ``sum``, ``mean`` via opt-in base class method override (:issue:`22762`)
- :func:`ExtensionArray.isna` is allowed to return an ``ExtensionArray`` (:issue:`22325`).
- Cleanup ``ValueError`` message thrown in creation of block manager: list block shape before number of total items in passed input shape and implied shape. (:issue:`20742`)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably shouldn't mention block manager in the release notes. Maybe just something about "fixed (or improved?) error message when ...)" where ... is a user action that would lead to the exception.


**Dtype changes**

Expand Down Expand Up @@ -1853,6 +1854,7 @@ Other

- Bug where C variables were declared with external linkage causing import errors if certain other C libraries were imported before Pandas. (:issue:`24113`)


.. _whatsnew_0.24.0.contributors:

Contributors
Expand Down
8 changes: 7 additions & 1 deletion pandas/core/internals/managers.py
Original file line number Diff line number Diff line change
Expand Up @@ -1674,7 +1674,13 @@ def create_block_manager_from_arrays(arrays, names, axes):
def construction_error(tot_items, block_shape, axes, e=None):
""" raise a helpful message about our construction """
passed = tuple(map(int, [tot_items] + list(block_shape)))
implied = tuple(map(int, [len(ax) for ax in axes]))
if len(passed) == 2:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you add a comment on the reversing. (e.g. this is a user facing error message and this is a DataFrame construction)

you can do len(passed) <= 2

passed = passed[::-1]

implied = tuple(len(ax) for ax in axes)
if len(implied) == 2:
implied = implied[::-1]

if passed == implied and e is not None:
raise e
if block_shape[0] == 0:
Expand Down
24 changes: 17 additions & 7 deletions pandas/tests/frame/test_constructors.py
Original file line number Diff line number Diff line change
Expand Up @@ -386,25 +386,35 @@ def test_constructor_error_msgs(self):
'B': ['a', 'b', 'c']})

# wrong size ndarray, GH 3105
msg = r"Shape of passed values is \(3, 4\), indices imply \(3, 3\)"
msg = r"Shape of passed values is \(4, 3\), indices imply \(3, 3\)"
with pytest.raises(ValueError, match=msg):
DataFrame(np.arange(12).reshape((4, 3)),
columns=['foo', 'bar', 'baz'],
index=pd.date_range('2000-01-01', periods=3))

arr = np.array([[4, 5, 6]])
msg = r"Shape of passed values is \(1, 3\), indices imply \(1, 4\)"
with pytest.raises(ValueError, match=msg):
DataFrame(index=[0], columns=range(0, 4), data=arr)

arr = np.array([4, 5, 6])
msg = r"Shape of passed values is \(3, 1\), indices imply \(1, 4\)"
with pytest.raises(ValueError, match=msg):
DataFrame(index=[0], columns=range(0, 4), data=arr)

# higher dim raise exception
with pytest.raises(ValueError, match='Must pass 2-d input'):
DataFrame(np.zeros((3, 3, 3)), columns=['A', 'B', 'C'], index=[1])

# wrong size axis labels
msg = ("Shape of passed values "
r"is \(3, 2\), indices "
r"imply \(3, 1\)")
r"is \(2, 3\), indices "
r"imply \(1, 3\)")
with pytest.raises(ValueError, match=msg):
DataFrame(np.random.rand(2, 3), columns=['A', 'B', 'C'], index=[1])

msg = ("Shape of passed values "
r"is \(3, 2\), indices "
r"is \(2, 3\), indices "
r"imply \(2, 2\)")
with pytest.raises(ValueError, match=msg):
DataFrame(np.random.rand(2, 3), columns=['A', 'B'], index=[1, 2])
Expand Down Expand Up @@ -638,10 +648,10 @@ def _check_basic_constructor(self, empty):
assert frame.values.dtype == np.int64

# wrong size axis labels
msg = r'Shape of passed values is \(3, 2\), indices imply \(3, 1\)'
msg = r'Shape of passed values is \(2, 3\), indices imply \(1, 3\)'
with pytest.raises(ValueError, match=msg):
DataFrame(mat, columns=['A', 'B', 'C'], index=[1])
msg = r'Shape of passed values is \(3, 2\), indices imply \(2, 2\)'
msg = r'Shape of passed values is \(2, 3\), indices imply \(2, 2\)'
with pytest.raises(ValueError, match=msg):
DataFrame(mat, columns=['A', 'B'], index=[1, 2])

Expand Down Expand Up @@ -1805,7 +1815,7 @@ def test_from_records_to_records(self):
tm.assert_frame_equal(DataFrame.from_records(arr2), DataFrame(arr2))

# wrong length
msg = r'Shape of passed values is \(3, 2\), indices imply \(3, 1\)'
msg = r'Shape of passed values is \(2, 3\), indices imply \(1, 3\)'
with pytest.raises(ValueError, match=msg):
DataFrame.from_records(arr, index=index[:-1])

Expand Down
2 changes: 1 addition & 1 deletion pandas/tests/io/json/test_pandas.py
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ def test_frame_from_json_bad_data(self):
json = StringIO('{"columns":["A","B"],'
'"index":["2","3"],'
'"data":[[1.0,"1"],[2.0,"2"],[null,"3"]]}')
msg = r"Shape of passed values is \(2, 3\), indices imply \(2, 2\)"
msg = r"Shape of passed values is \(3, 2\), indices imply \(2, 2\)"
with pytest.raises(ValueError, match=msg):
read_json(json, orient="split")

Expand Down