Skip to content

ERR: better concat error messages #9157 #9172

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from

Conversation

cel4
Copy link
Contributor

@cel4 cel4 commented Dec 30, 2014

closes #9157

This is a first attempt to improve the error message when trying to concat an empty list/set of objects:

pd.concat([]) currently raises following error:

/Users/ch/miniconda/envs/sci33/lib/python3.3/site-packages/pandas/tools/merge.py in __init__(self, objs, axis, join, join_axes, keys, levels, names, ignore_index, verify_integrity, copy)
    765 
    766         if len(objs) == 0:
--> 767             raise ValueError('All objects passed were None')
    768 
    769         # consolidate data & figure out what our result ndim is going to be

ValueError: All objects passed were None

After the tiny fix it is:

/Users/ch/repo/pandas/pandas/tools/merge.py in __init__(self, objs, axis, join, join_axes, keys, levels, names, ignore_index, verify_integrity, copy)
    767 
    768         if len(objs) == 0:
--> 769             raise ValueError('No objects passed')
    770 
    771         if isinstance(objs, dict):

ValueError: No objects passed

count_gen, objs = itertools.tee(objs)
obj_count = len(list(count_gen))

if obj_count == 0:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't len(objects) accomplish this?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, it does look a little hackish. I had issues with objs being a generator. len(list(objs)) would consume the generator, that's why I used the hack with itertools.tee.

@jreback
Copy link
Contributor

jreback commented Jan 2, 2015

needs a test

@jreback jreback added Error Reporting Incorrect or improved errors from pandas Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Jan 2, 2015
@cel4
Copy link
Contributor Author

cel4 commented Jan 2, 2015

Yes, still thinking how to design a good test. I still raise a ValueError, but this time with a slightly different error message. Should I test for the wording of the error message? I am not too happy with this solution since it breaks if someone decides to come up with a better message.

@jreback
Copy link
Contributor

jreback commented Jan 18, 2015

@cel4 you can do a assertRegExp. And it should break if something (e.g. the error message is changed), that's the point.

@cel4
Copy link
Contributor Author

cel4 commented Jan 22, 2015

@jreback, sorry I am very busy at the moment. I am planning to continue the work here in a week or two.

@cel4
Copy link
Contributor Author

cel4 commented Feb 8, 2015

I added a draft version how a test could look like. Would appreciate any comments.

@cel4
Copy link
Contributor Author

cel4 commented Feb 9, 2015

AttributeError: 'TestOrderedMerge' object has no attribute 'assertRegexpMatches' - grr... seems like assertRegexpMatches is not part of the unittest module in the 2.6 version...

@jorisvandenbossche
Copy link
Member

@cel4 In the pandas testing module, there is a assertRaisesRegexp available for this -> tm.assertRaisesRegexp()

@cel4
Copy link
Contributor Author

cel4 commented Feb 11, 2015

Hmh, tests are green. Yet I am not sure if I like this solution. Wouldn't it be better if we also check inpd.concat for empty sequences, so we can raise a even better (less general) error message about what went wrong exactly.

@jreback
Copy link
Contributor

jreback commented Feb 11, 2015

this needs to address my comments above
you can force materializing a generator and thrn just take the len

@sergeny
Copy link

sergeny commented Apr 22, 2015

I propose "ValueError: No objects to concatenate"

@cel4
Copy link
Contributor Author

cel4 commented Apr 22, 2015

@sergeny, I like that suggestion. Unfortunately, I do not have much time at the moment and I totally forgot about that PR. I hope I can finish it soon.

@jreback jreback changed the title Attempt to improve concat error message ERR: better concat error messages #9157 May 9, 2015
@jreback
Copy link
Contributor

jreback commented Jul 28, 2015

closing as stale, but if you would like to reopen and fixup pls do.

@cel4
Copy link
Contributor Author

cel4 commented Aug 14, 2015

Should be ready for a first round of reviews.

@@ -2552,6 +2552,22 @@ def _constructor(self):

tm.assertIsInstance(result, NotADataFrame)

def test_empty_sequence_concat(self):
empty_pat = "[Nn]o objects"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add the issue number here as a comment

@jreback
Copy link
Contributor

jreback commented Aug 14, 2015

looks reasonable. pls add a release note. and squash. (you can put in API changes)

@jreback jreback added this to the 0.17.0 milestone Aug 14, 2015
@cel4
Copy link
Contributor Author

cel4 commented Aug 15, 2015

@jreback, When travis is happy, this should be ready for review once again.

@jreback
Copy link
Contributor

jreback commented Aug 15, 2015

merged via 58db03e

thanks!

@jreback jreback closed this Aug 15, 2015
@cel4 cel4 deleted the concat_error branch August 16, 2015 08:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Error Reporting Incorrect or improved errors from pandas Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Clearer error message for pd.concat if empty iterator is passed
4 participants