Merge resets column types - category is reset

[laufere]

Code Sample, a copy-pastable example if possible

import numpy as np
import pandas as pd
# Generate some random dataframes with a common "id" column to merge on
# Transform some columns into type "category"
some_strings = ['aa', 'bb', 'cc', 'dd']
d1 = {'id': np.arange(100),
      'var1': np.random.randint(0,10,size=100),
      'var2': np.random.uniform(size=100),
      'var3': np.random.randint(0,2,size=100),
      'var4': np.random.choice(some_strings, 100)}
df1 = pd.DataFrame(d1)
df1['var1'] = df1['var1'].astype('category')

d2 = {'id': np.arange(100),
      'var1': np.random.randint(0,10,size=100),
      'var2': np.random.uniform(size=100),
      'var3': np.random.randint(0,2,size=100),
      'var4': np.random.choice(some_strings, 100)}
df2 = pd.DataFrame(d2)
df2['var1'] = df2['var1'].astype('category')
df2['var4'] = df2['var4'].astype('category')

print 'df1'
print df1.info() #Shows categorical columns

print 'df2'
print df2.info() #Shows categorical columns
print pd.merge(df1, df2, on='id').info() #Categorical columns are no longer categorical

Expected Output

The merged df1 and df2 no longer has category columns, and are reset to their original types.

output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 2.7.11.final.0
python-bits: 64
OS: Linux
OS-release: 2.6.32-431.23.3.el6.x86_64
machine: x86_64
processor: x86_64
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8

pandas: 0.17.1
nose: 1.3.7
pip: 7.1.2
setuptools: 18.5
Cython: 0.23.4
numpy: 1.10.1
scipy: 0.16.0
statsmodels: 0.6.1
IPython: 4.0.1
sphinx: 1.3.1
patsy: 0.4.0
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.4.4
matplotlib: 1.5.0
openpyxl: 2.2.6
xlrd: 0.9.4
xlwt: 1.0.0
xlsxwriter: 0.7.7
lxml: 3.4.4
bs4: 4.4.1
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: 1.0.9
pymysql: None
psycopg2: None
Jinja2: None

[TomAugspurger]

Thanks for the report.

FWIW when we fix this, we'll want to make sure that the categories match when merging on the Categorical column. Right now this would "succeed" since the categories get cast as object.

[jreback]

dupe of #10409