Please don't drop pandas.rpy.common.convert_robJ

[zhangyi-hu]

Currently pandas is the only well-known package supporting conversion from rpy2 object to dictionary.
There is warning message for rpy

warnings.warn("The pandas.rpy module is deprecated and will be "
              "removed in a future version. We refer to external packages "
              "like rpy2. "
              "\nSee here for a guide on how to port your code to rpy2: "
              "http://pandas.pydata.org/pandas-docs/stable/r_interface.html",
              FutureWarning, stacklevel=2)

Since the user primarily use rpy2 with pandas dataframe inputs, I strongly recommend keep this conversion functionality publicly available after rpy is retired.

We will be very grateful !!

[jorisvandenbossche]

@hzzyyy Can you show an example of what functionality you exactly refer to here?
And there is no alternative of that functionality built in into rpy2? As this is certainly the idea behind deprecating it in pandas, that you can use rpy2 instead.

See also #9602

[zhangyi-hu]

Thank you @jorisvandenbossche for your quick response,

The function I want to use is
rpy.common.convert_robJ
Where the input of this function is a rpy2 ListVetor representing a named list in R. convert_robJ will convert it into a user friendly dictionary containing pandas dataframes, numpy arrays etc.

I think rpy2 can convert r dataframe but not named list containing dataframe (and possibly matrix etc.).
rpy.common has the following conveters:

_pandas_converters = [
    (robj.DataFrame, _convert_DataFrame),
    (robj.Matrix, _convert_Matrix),
    (robj.StrVector, _convert_vector),
    (robj.FloatVector, _convert_vector),
    (robj.Array, _convert_array),
    (robj.Vector, _convert_list),
]

_converters = [
    (robj.DataFrame, lambda x: _convert_DataFrame(x).toRecords(index=False)),
    (robj.Matrix, lambda x: _convert_Matrix(x).toRecords(index=False)),
    (robj.IntVector, _convert_vector),
    (robj.StrVector, _convert_vector),
    (robj.FloatVector, _convert_vector),
    (robj.Array, _convert_array),
    (robj.Vector, _convert_list),
]

With those converters, it can work with more general data from R.
If you can show me tools in rpy2 with the same functionality, I will be very grateful.

Thanks

[jorisvandenbossche]

cc @lgautier

[zhangyi-hu]

I'm willing to work on this ticket and relocate this function outside of rpy, in case you are in short of hands.
Just let me know where it should sit in pandas.

Thanks

[wesm]

@hzzyyy for things like this, I think we should encourage more and more pandas add-on 3rd party packages. This shifts development burden away from the core maintainership. We would be happy to maintain a list of "pandas-related 3rd party packages" on the pandas website, and additionally we are thinking about creating a dedicated pandas home on GitHub, and packages like this ("rpy_pandas") could go there.

[zhangyi-hu]

@wesm I totally agree and volunteer to work on rpy_pandas when it launches.

[lgautier]

Thanks for the cc @jorisvandenbossche .

@hzzyyy :

Currently pandas is the only well-known package supporting conversion from rpy2 object to dictionary.

Well, the conversion does not look too hard to me:

# Create an example list vector
from rpy2.robjects.vectors import ListVector
l = ListVector({'a':1, 'b':2})

# convert R list to Python dict
d = dict(l.items())

[zhangyi-hu]

@lgautier Thank you for your reply,

In a real example, the object is more complicated. The 'l' in your example will have multiple layers.

For instance, the value corresponding to key 'a' can be another ListVector. With multiple entries.
We need to convert them recursively.

[lgautier]

@hzzyyy : Then it does not appear to be a data frame. A way to achieve this is to redefine the conversion rule for R lists (with the cautionary note that the names in R lists are not necessarily unique), but that's an rpy2 question. The pandas issue tracker might no be the best place to discuss it.

[jorisvandenbossche]

@hzzyyy Is this still an issue?
If yes, can you provide a more concrete example showing functionality in pandas.rpy that you need and cannot easily be reproduced using rpy2?

Closing this in favor of #9602 (but feel free to comment further, here or there)

[zhangyi-hu]

@jorisvandenbossche I made a local copy of the file I need.
Thanks