Implement a DataFrame.view() method to interactively display data

[wavexx]

R has a very rudimentary function (utils::View()) to display data interactively. On X11 it's so broken I basically never used it, yet, it would be very nice if pandas had something similar.

With python, there's tabview: https://github.com/firecat53/tabview/

which can be used as a module as well:

import tabview.tabview as t
t.view([matrix data])

This runs a full-screen curses interfaces with a simple spreadsheet like display, which is actually much nicer than what R offers.

I've started to use and customize the cli wrapper a lot (display X,Y coordinates, etc), to the point that I would like direct integration with DataFrame.

Would such a patch be accepted?

[jreback]

@wavexx are you proposing including tabview as an included library with pandas? or something else?

[wavexx]

On 01/02/2015 06:11 PM, jreback wrote:

@wavexx are you proposing including tabview as an included
library with pandas? or something else?

I'm open for ideas.

It could start with a "view()" method that tries to import tabview, with
tabview being only a suggested package.

But we could add a submodule if having it as part of the package is a
desirable thing. I would be a bit against a flat copy, since I'd like
the development of tabview to continue also as a stand-alone viewer.

I would also love to have some sort of flexible viewer mechanism such as
the "PAGER" environment variable, but for data. Setting TABVIEW=tabview
could allow other tools to emerge. One example would be an X11
alternative if DISPLAY is set. The only downside is that it would
require saving/parsing the data again, so the built-in tabview is still
desiderable when TABVIEW is empty.

[s-celles]

Hello,

a quite ugly solution is available here

import pandas as pd
import tabview.tabview as t
def view(data):
    if isinstance(data, pd.DataFrame) or isinstance(data, pd.Series):
        result = t.view(data.reset_index().transpose().reset_index().transpose().values)
        return(result)
    elif isinstance(data, pd.Panel):
        result = t.view(panel.to_frame().reset_index().transpose().reset_index().transpose().values)
        return(result)
    else:
        raise(NotImplementedError)

import pandas.io.data as web
import datetime
start = datetime.datetime(2010, 1, 1)
end = datetime.datetime(2013, 1, 27)
panel = web.DataReader(["F", "YHOO"], 'yahoo', start, end) # Panel
df = panel.loc[:,:,"F"] # DataFrame
ts=df['Adj Close'] # Series

view(df)

view(ts)

view(panel)
view(panel.transpose(0,2,1))

It "works" with Panel, DataFrame and Series. I hope it can helps.

Kind regards

Sébastien

[wavexx]

Why the double transpose?

with df.reset_index() as df:
    t.view([df.columns.tolist()] + df.values.tolist()

seems to work.

[s-celles]

Because of the double reset_index() (one for index, one for columns)

see http://stackoverflow.com/questions/14189695/reset-index-for-dataframe-columns

[wavexx]

On 01/05/2015 01:29 PM, scls19fr wrote:

Because of the double reset_index() (one for index, one for
columns)

I guessed, but why not just concatenating the columns separately as I
did in the last example?

If Series are the reason, I would put a specialized view() method in its
class.

[s-celles]

I thought that converting numpy array to list and concatenating 2 python lists wasn't a great idea. But timeit could show us that.

In [10]: %timeit df.reset_index().transpose().reset_index().transpose().values
100 loops, best of 3: 6.81 ms per loop

In [11]: %timeit [df.reset_index().columns.tolist()] + df.reset_index().values.tolist()
100 loops, best of 3: 5.21 ms per loop

and I was wrong (it should even be better with "with")

[s-celles]

I wonder if on Pandas side we shouldn't have only

import tabview.tabview as t
t.view(self)

inside Pandas DataFrame, Series and Panel view() methods
and some update of tabview to support Pandas inputs like TabViewer/tabview#59

or

pd.view(self)

with

pd.view = tabview.tabview.view

so it will be easier if other Pandas Panel / DataFrame / Series pagers appears

[wavexx]

The idea of having a generic view method that could be overridden is good. Specialized methods in DataFrame/Series/Panel could reuse it in an uniform way.

This would solve the problem of having a generic data visualizer.

Still, pandas should come with a built-in/working method in order for the feature to catch on.
I'm hoping to fix a couple of issues with tabview in order to be used as a module and then re-visit the proposal in these terms.

[wavexx]

We're pulling in native support for Pandas into tabview itself, so I consider this no longer an issue for Pandas.