Skip to content

BUG: Concat with inner join and empty DataFrame #15328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jesrael opened this issue Feb 7, 2017 · 3 comments
Closed

BUG: Concat with inner join and empty DataFrame #15328

jesrael opened this issue Feb 7, 2017 · 3 comments
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Milestone

Comments

@jesrael
Copy link

jesrael commented Feb 7, 2017

Function concat with parameter join='inner' not return empty DataFrame:

DF_empty = pd.DataFrame()
DF_a = pd.DataFrame({'a': [1, 2]}, index=[0, 1])
print (DF_empty)
Empty DataFrame
Columns: []
Index: []

print (DF_a)
   a
0  1
1  2

print(pd.concat([DF_empty, DF_a], axis=1, join='inner'))
   a
0  1
1  2

But merge works nice:

print (pd.merge(DF_empty, DF_a, left_index=True, right_index=True))
Empty DataFrame
Columns: [a]
Index: []

SO question


print (pd.show_versions())
INSTALLED VERSIONS ------------------ commit: None python: 3.5.1.final.0 python-bits: 64 OS: Windows OS-release: 7 machine: AMD64 processor: Intel64 Family 6 Model 60 Stepping 3, GenuineIntel byteorder: little LC_ALL: None LANG: sk_SK LOCALE: None.None
pandas: 0.19.2
nose: 1.3.7
pip: 9.0.1
setuptools: 20.3
Cython: 0.23.4
numpy: 1.11.0
scipy: 0.17.0
statsmodels: 0.6.1
xarray: None
IPython: 4.1.2
sphinx: 1.3.1
patsy: 0.4.0
dateutil: 2.5.1
pytz: 2016.2
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.5.2
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.8.4
lxml: 3.6.0
bs4: 4.4.1
html5lib: 0.999
httplib2: None
apiclient: None
sqlalchemy: 1.0.12
pymysql: None
psycopg2: None
jinja2: 2.8
boto: 2.39.0
pandas_datareader: 0.2.1
None

<\details>

@jreback
Copy link
Contributor

jreback commented Feb 7, 2017

ok, this is a bug, we normally filter out completely empty frames, but in this case we need to skip that condition

index 3fbd83a..5bf0546 100644
--- a/pandas/tools/merge.py
+++ b/pandas/tools/merge.py
@@ -1689,7 +1689,7 @@ class _Concatenator(object):
                            if sum(obj.shape) > 0 or isinstance(obj, Series)]
 
             if (len(non_empties) and (keys is None and names is None and
-                                      levels is None and join_axes is None)):
+                                      levels is None and join_axes is None and not self.intersect)):
                 objs = non_empties
                 sample = objs[0]
 

pull-request to add some tests (there are very little tests for using join= with concat) and fix would be great.

@jreback jreback added Bug Difficulty Novice Reshaping Concat, Merge/Join, Stack/Unstack, Explode labels Feb 7, 2017
@jreback jreback added this to the 0.20.0 milestone Feb 7, 2017
@jreback jreback changed the title Concat with inner join and empty DataFrame not works BUG: Concat with inner join and empty DataFrame Feb 7, 2017
@abaldenko
Copy link
Contributor

happy to work on that. does it amount to fixing /pandas/tools/merge.py and writing some related tests in pandas/pandas/tests/types/test_concat.py?

@jreback
Copy link
Contributor

jreback commented Feb 9, 2017

tests are in pandas/tools/tests/test_concat.py

and (shortly) this code is in pandas/tools/concat.py (just moving it).

AnkurDedania pushed a commit to AnkurDedania/pandas that referenced this issue Mar 21, 2017
closes pandas-dev#15328

Author: abaldenko <[email protected]>

Closes pandas-dev#15397 from abaldenko/concat_empty_dataframe and squashes the following commits:

47c8735 [abaldenko] BUG: Concat with inner join and empty DataFrame
fc473b7 [abaldenko] BUG: Concat with inner join and empty DataFrame
b86dcb6 [abaldenko] BUG: Concat with inner join and empty DataFrame
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Reshaping Concat, Merge/Join, Stack/Unstack, Explode
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants