BUG: groupby with both by and level parameters has an inconsistent behavior

[LucasG0]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the master branch of pandas.

Reproducible Example

# With single Index
>>> df = pd.DataFrame({"c1": list("abc"), "c2": range(3)}, index=["x", "x", "y"])
>>> df
  c1  c2
x  a   0
x  b   1
y  c   2
>>> df.groupby(level=0, by="c1").sum(numeric_only=False)
   c1  c2
x  ab   1
y   c   2                                                                                                                                                                                                           >>> df.groupby(level=0, by=0).sum(numeric_only=False)                                                                                                                                                                 c1  c2                                                                                                                                                                                                          x  ab   1                                                                                                                                                                                                          y   c   2

# With MultiIndex
>>> df = pd.DataFrame({"c1": list("abc"), "c2": range(3)}, index=pd.MultiIndex.from_arrays([list("xxy"), [0, 0, 1]]))
>>> df
    c1  c2
x 0  a   0
  0  b   1
y 1  c   2
>>> df.groupby(level=0, by="c1").sum(numeric_only=False)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "D:\Programmation\Utils\Miniconda\lib\site-packages\pandas\core\frame.py", line 7641, in groupby
    dropna=dropna,
  File "D:\Programmation\Utils\Miniconda\lib\site-packages\pandas\core\groupby\groupby.py", line 897, in __init__
    dropna=self.dropna,
  File "D:\Programmation\Utils\Miniconda\lib\site-packages\pandas\core\groupby\grouper.py", line 889, in get_grouper
    if not isinstance(gpr, Grouping)
  File "D:\Programmation\Utils\Miniconda\lib\site-packages\pandas\core\groupby\grouper.py", line 493, in __init__
    ) = index._get_grouper_for_level(mapper, ilevel)
  File "D:\Programmation\Utils\Miniconda\lib\site-packages\pandas\core\indexes\multi.py", line 1483, in _get_grouper_for_level
    grouper = level_values.map(mapper)
  File "D:\Programmation\Utils\Miniconda\lib\site-packages\pandas\core\indexes\base.py", line 5506, in map
    new_values = self._map_values(mapper, na_action=na_action)
  File "D:\Programmation\Utils\Miniconda\lib\site-packages\pandas\core\base.py", line 870, in _map_values
    new_values = map_f(values, mapper)
  File "pandas\_libs\lib.pyx", line 2859, in pandas._libs.lib.map_infer
TypeError: 'numpy.ndarray' object is not callable

Issue Description

• When the index is a single Index, the by parameter is ignored.
• When the index is a MultiIndex, an unclear error is raised.

It might be confusing for someone trying to groupby on columns and index (that is not aware by can actually perform this).

Expected Behavior

I think a cleaner error should be raised, like

ValueError: groupby only accepts one of 'by' or 'level' parameters.

Installed Versions

INSTALLED VERSIONS

commit : 66e3805
python : 3.7.6.final.0
python-bits : 64
OS : Windows
OS-release : 10
Version : 10.0.19041
machine : AMD64
processor : Intel64 Family 6 Model 142 Stepping 11, GenuineIntel
byteorder : little
LC_ALL : None
LANG : None
LOCALE : None.None

pandas : 1.3.5
numpy : 1.19.5
pytz : 2020.1
dateutil : 2.8.1
pip : 20.2.2
setuptools : 49.4.0.post20200813
Cython : None
pytest : None
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fsspec : None
fastparquet : None
gcsfs : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : None
pyarrow : None
pyxlsb : None
s3fs : None
scipy : 1.6.0
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
numba : 0.52.0

[simonjayhawkins]

Thanks @LucasG0 for the report. It may help engagement if the code sample is copy and pastable. i.e. show the code to create the DataFrames.

Expected Behavior

I think a cleaner error should be raised, like

ValueError: groupby only accepts one of 'by' or 'level' parameters.

unless by is a function, where the level parameter appears to work as expected?

df = pd.DataFrame(
    {"c1": list("abc"), "c2": range(3)},
    index=pd.MultiIndex.from_arrays([list("xxy"), [0, 0, 1]]),
)
print(df)

print(df.groupby(level=0, by=lambda x: x == "y").sum(numeric_only=False))

print(df.groupby(by=lambda x: x == "y").sum(numeric_only=False))

    c1  c2
x 0  a   0
  0  b   1
y 1  c   2
       c1  c2
False  ab   1
True    c   2
        c1  c2
False  abc   3

[LucasG0]

It may help engagement if the code sample is copy and pastable. i.e. show the code to create the DataFrames.

Indeed, I edited the original post.

unless by is a function, where the level parameter appears to work as expected?

Except when by is a function, the only working case I noticed is with a dict

>>> df = pd.DataFrame({"c1": list("abc"), "c2": range(3)}, index=pd.MultiIndex.from_arrays([list("xxy"), [0, 0, 1]]))
>>> df
    c1  c2
x 0  a   0
  0  b   1
y 1  c   2
>>> df.groupby(by={"x": "first"}).sum(numeric_only=False)
Empty DataFrame
Columns: [c1, c2]
Index: []
>>> df.groupby(level=0, by={"x": "first"}).sum(numeric_only=False)
       c1  c2
first  ab   1

[simonjayhawkins]

closing as duplicate of #40378 to help keep discussion in one place