Skip to content

Fix regression in setitem when expanding DataFrame with specific column name format #39023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jan 8, 2021

Conversation

phofl
Copy link
Member

@phofl phofl commented Jan 7, 2021

We obviously want cast to object if value is string.

@phofl phofl added Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version labels Jan 7, 2021
@jorisvandenbossche
Copy link
Member

Alternatively, I suppose that is_bool_dtype should not raise an error on this:

In [97]: pd.api.types.is_bool_dtype('0 - Name')
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-97-c990eff26eac> in <module>
----> 1 pd.api.types.is_bool_dtype('0 - Name')

~/scipy/pandas/pandas/core/dtypes/common.py in is_bool_dtype(arr_or_dtype)
   1395         return False
   1396     try:
-> 1397         dtype = get_dtype(arr_or_dtype)
   1398     except TypeError:
   1399         return False

~/scipy/pandas/pandas/core/dtypes/common.py in get_dtype(arr_or_dtype)
   1631         arr_or_dtype = arr_or_dtype.dtype
   1632 
-> 1633     return pandas_dtype(arr_or_dtype)
   1634 
   1635 

~/scipy/pandas/pandas/core/dtypes/common.py in pandas_dtype(dtype)
   1823     # raise a consistent TypeError if failed
   1824     try:
-> 1825         npdtype = np.dtype(dtype)
   1826     except SyntaxError as err:
   1827         # np.dtype uses `eval` which can raise SyntaxError

~/miniconda3/envs/dev/lib/python3.7/site-packages/numpy/core/_internal.py in _commastring(astr)
    177                     raise ValueError(
    178                         'format number %d of "%s" is not recognized' %
--> 179                         (len(result)+1, astr))
    180                 startindex = mo.end()
    181 

ValueError: format number 1 of "b'0 - Name'" is not recognized

@phofl
Copy link
Member Author

phofl commented Jan 7, 2021

You mean catching the numpy error?

@jorisvandenbossche
Copy link
Member

I didn't check exactly what it would entail to fix it, but is_bool_dtype is a function that we control ourselves, so we can ensure it returns False for this case, instead of erroring.
But yeah, since we are already catching errors when trying to create a numpy dtype, we might need to catch some more errors

@phofl
Copy link
Member Author

phofl commented Jan 7, 2021

Yeah, this works too. Thx

@jbrockmendel
Copy link
Member

nice teamwork! @phofl can you add a test specific to is_bool_dtype

@phofl
Copy link
Member Author

phofl commented Jan 7, 2021

Done

@jreback jreback added this to the 1.2.1 milestone Jan 8, 2021
@jreback jreback merged commit 6e707f2 into pandas-dev:master Jan 8, 2021
@jreback
Copy link
Contributor

jreback commented Jan 8, 2021

thanks @phofl

@jreback
Copy link
Contributor

jreback commented Jan 8, 2021

@meeseeksdev backport 1.2.x

meeseeksmachine pushed a commit to meeseeksmachine/pandas that referenced this pull request Jan 8, 2021
…g DataFrame with specific column name format
jreback pushed a commit that referenced this pull request Jan 8, 2021
…e with specific column name format (#39032)

Co-authored-by: patrick <[email protected]>
@phofl phofl deleted the 39010 branch January 8, 2021 08:41
luckyvs1 pushed a commit to luckyvs1/pandas that referenced this pull request Jan 20, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Indexing Related to indexing on series/frames, not to indexes themselves Regression Functionality that used to work in a prior pandas version
Projects
None yet
Development

Successfully merging this pull request may close these issues.

BUG: 1.2. version fails in adding column if number on column name beginning.
4 participants