We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the master branch of pandas.
import os os.environ["PANDAS_DATA_MANAGER"] = "array" import pandas as pd, numpy as np df = pd.DataFrame(index=np.arange(10)) df["foo"] = np.ones((10, 1)) # ValueError: Expected a 1D array, got an array with shape (10, 1) <details> <summary> Full traceback </summary> ```pytb --------------------------------------------------------------------------- KeyError Traceback (most recent call last) /usr/local/lib/python3.9/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance) 3360 try: -> 3361 return self._engine.get_loc(casted_key) 3362 except KeyError as err: /usr/local/lib/python3.9/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc() /usr/local/lib/python3.9/site-packages/pandas/_libs/index.pyx in pandas._libs.index.IndexEngine.get_loc() pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item() pandas/_libs/hashtable_class_helper.pxi in pandas._libs.hashtable.PyObjectHashTable.get_item() KeyError: 'foo' The above exception was the direct cause of the following exception: KeyError Traceback (most recent call last) /usr/local/lib/python3.9/site-packages/pandas/core/frame.py in _set_item_mgr(self, key, value) 3750 try: -> 3751 loc = self._info_axis.get_loc(key) 3752 except KeyError: /usr/local/lib/python3.9/site-packages/pandas/core/indexes/base.py in get_loc(self, key, method, tolerance) 3362 except KeyError as err: -> 3363 raise KeyError(key) from err 3364 KeyError: 'foo' During handling of the above exception, another exception occurred: ValueError Traceback (most recent call last) /var/folders/bd/43q20k0n6z15tdfzxvd22r7c0000gn/T/ipykernel_14506/151211532.py in <module> 5 6 df = pd.DataFrame(index=np.arange(10)) ----> 7 df["foo"] = np.ones((10, 1)) /usr/local/lib/python3.9/site-packages/pandas/core/frame.py in __setitem__(self, key, value) 3610 else: 3611 # set column -> 3612 self._set_item(key, value) 3613 3614 def _setitem_slice(self, key: slice, value): /usr/local/lib/python3.9/site-packages/pandas/core/frame.py in _set_item(self, key, value) 3795 value = np.tile(value, (len(existing_piece.columns), 1)).T 3796 -> 3797 self._set_item_mgr(key, value) 3798 3799 def _set_value( /usr/local/lib/python3.9/site-packages/pandas/core/frame.py in _set_item_mgr(self, key, value) 3752 except KeyError: 3753 # This item wasn't present, just insert at end -> 3754 self._mgr.insert(len(self._info_axis), key, value) 3755 else: 3756 self._iset_item_mgr(loc, value) /usr/local/lib/python3.9/site-packages/pandas/core/internals/array_manager.py in insert(self, loc, item, value) 872 value = value[0, :] # type: ignore[index] 873 else: --> 874 raise ValueError( 875 f"Expected a 1D array, got an array with shape {value.shape}" 876 ) ValueError: Expected a 1D array, got an array with shape (10, 1)
Using the array manager, a 2d array with only one non-singleton dimension causes errors when assigned to a dataframe.
This does not happen with the BlockManager. I would assume the same thing would work, since these should be pretty equivalent.
This works fine:
The behavior with the block manager is expected.
import os os.environ["PANDAS_DATA_MANAGER"] = "block" import pandas as pd, numpy as np df = pd.DataFrame(index=np.arange(10)) df["foo"] = np.ones((10, 1))
INSTALLED VERSIONS ------------------ commit : 945c9ed766a61c7d2c0a7cbb251b6edebf9cb7d5 python : 3.9.9.final.0 python-bits : 64 OS : Darwin OS-release : 20.6.0 Version : Darwin Kernel Version 20.6.0: Tue Oct 12 18:33:42 PDT 2021; root:xnu-7195.141.8~1/RELEASE_X86_64 machine : x86_64 processor : i386 byteorder : little LC_ALL : None LANG : None LOCALE : None.UTF-8 pandas : 1.3.4 numpy : 1.20.3 pytz : 2021.3 dateutil : 2.8.2 pip : 21.3.1 setuptools : 59.0.1 Cython : None pytest : 6.2.5 hypothesis : None sphinx : 4.1.2 blosc : None feather : None xlsxwriter : None lxml.etree : None html5lib : None pymysql : None psycopg2 : None jinja2 : 3.0.3 IPython : 7.29.0 pandas_datareader: None bs4 : None bottleneck : None fsspec : 2021.11.1 fastparquet : None gcsfs : None matplotlib : 3.5.0 numexpr : 2.7.3 odfpy : None openpyxl : 3.0.9 pandas_gbq : None pyarrow : None pyxlsb : None s3fs : None scipy : 1.7.3 sqlalchemy : None tables : 3.6.1 tabulate : None xarray : 0.20.1 xlrd : 1.2.0 xlwt : None numba : 0.54.1
The text was updated successfully, but these errors were encountered:
The ArrayManager is now deprecated. Closing.
Sorry, something went wrong.
No branches or pull requests
I have checked that this issue has not already been reported.
I have confirmed this bug exists on the latest version of pandas.
I have confirmed this bug exists on the master branch of pandas.
Reproducible Example
Issue Description
Using the array manager, a 2d array with only one non-singleton dimension causes errors when assigned to a dataframe.
This does not happen with the BlockManager. I would assume the same thing would work, since these should be pretty equivalent.
Expected Behavior
This works fine:
The behavior with the block manager is expected.
Installed Versions
The text was updated successfully, but these errors were encountered: