Option for crosstab/pivot_table to include empty columns

[hayd]

I don't think you can do this easily atm. crosstab is just a [convenience for pivot_table]((https://github.com/pydata/pandas/blob/master/pandas/tools/pivot.py#L218)...

http://stackoverflow.com/questions/17003034/missing-data-in-pandas-crosstab

a = np.array(['foo', 'foo', 'foo', 'bar', 'bar', 'foo', 'foo'], dtype=object)
b = np.array(['one', 'one', 'two', 'one', 'two', 'two', 'two'], dtype=object)
c = np.array(['dull', 'dull', 'dull', 'dull', 'dull', 'shiny', 'shiny'], dtype=object)

pd.crosstab(a, [b, c], rownames=['a'], colnames=['b', 'c'])
b     one   two       
c    dull  dull  shiny
a                     
bar     1     1      0
foo     2     1      2

# but wants
b     one        two       
c    dull  shiny dull  shiny
a                     
bar     1     0    1      0
foo     2     0    1      2

Maybe we could have an option in pivot_table (and crosstab) to include empty columns.

[hayd]

Also see: https://groups.google.com/forum/#!searchin/pydata/Edwin$20Haver/pydata/Fda9mEGpVKM/rPYnJSUrs3YJ

[jreback]

I think something like this might be done in tools/merge
an outer join produces the Cartesian product of the keys

[hayd]

@jreback I'm not sure how to do that merge trick for multi-dimensional things?

My awful hack was:

def cartesian_product(X):
    lenX = map(len, X)
    cumprodX = np.cumproduct(lenX)
    a = np.insert(cumprodX, 0, 1)
    b = a[-1] / a[1:]
    return [np.tile(np.repeat(x, b[i]), 
                    np.product(a[i]))
               for i, x in enumerate(X)]

faster than itertools/compat.product for larger inputs (slower for smaller). Also works on things which aren't necessarily Series/DataFrames... basically I'm doing

cartesian_product(table.index.levels)

Have that as a component to a PR on the way, easy to drop in something else instead for that part.

[hayd]

this probably could be cythonized...