BUG: [unit test] pandas/io/clipboard/__init__.py:300: PyperclipException

[adrienpacifico]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

# pytest pandas/tests/io/
============================================================================================ short test summary info =============================================================================================
FAILED pandas/tests/io/test_clipboard.py::TestClipboard::test_raw_roundtrip[\U0001f44d...] - pandas.errors.PyperclipException: 
FAILED pandas/tests/io/test_clipboard.py::TestClipboard::test_raw_roundtrip[\u03a9\u0153\u2211\xb4...] - pandas.errors.PyperclipException: 
FAILED pandas/tests/io/test_clipboard.py::TestClipboard::test_raw_roundtrip[abcd...] - pandas.errors.PyperclipException: 
========================================================================================= 3 failed, 232 passed in 1.55s ==========================================================================================

Issue Description

The unit-test

pandas/pandas/tests/io/test_clipboard.py

Line 398 in 047c11d

def test_raw_roundtrip(self, data):

does not pass.

I am using VSCode with the .devcontainer file (thus in a container), with a linux distribution (pop-os).
Current commit I am on is 047c11d

I tried to do apt-get install xclip but it did not solve the issue.

Expected Behavior

The test should pass.

Installed Versions

INSTALLED VERSIONS

commit : 047c11d
python : 3.8.13.final.0
python-bits : 64
OS : Linux
OS-release : 5.8.0-7630-generic
Version : #32160919370720.10~781bb80-Ubuntu SMP Tue Jan 5 21:29:56 UTC 2
machine : x86_64
processor : x86_64
byteorder : little
LC_ALL : C.UTF-8
LANG : C.UTF-8
LOCALE : en_US.UTF-8

pandas : 1.6.0.dev0+89.g047c11d780.dirty
numpy : 1.22.4
pytz : 2022.2.1
dateutil : 2.8.2
setuptools : 65.3.0
pip : 22.2.2
Cython : 0.29.32
pytest : 7.1.3
hypothesis : 6.54.5
sphinx : 4.5.0
blosc : None
feather : None
xlsxwriter : 3.0.3
lxml.etree : 4.9.1
html5lib : 1.1
pymysql : 1.0.2
psycopg2 : 2.9.3
jinja2 : 3.0.3
IPython : 8.5.0
pandas_datareader: 0.10.0
bs4 : 4.11.1
bottleneck : 1.3.5
brotli :
fastparquet : 0.8.3
fsspec : 2021.11.0
gcsfs : 2021.11.0
matplotlib : 3.5.3
numba : 0.55.2
numexpr : 2.8.3
odfpy : None
openpyxl : 3.0.10
pandas_gbq : 0.17.8
pyarrow : 9.0.0
pyreadstat : 1.1.9
pyxlsb : 1.0.9
s3fs : 2021.11.0
scipy : 1.9.1
snappy :
sqlalchemy : 1.4.41
tables : 3.7.0
tabulate : 0.8.10
xarray : 2022.6.0
xlrd : 2.0.1
xlwt : 1.3.0
zstandard : 0.18.0
tzdata : None

[mroeschke]

What was the exception traceback for these tests?

[adrienpacifico]

@mroeschke it is a PyperclipException.

Stacktrace

self = <pandas.tests.io.test_clipboard.TestClipboard object at 0x7f479a61ca00>, data = '👍...'

    @pytest.mark.parametrize("data", ["\U0001f44d...", "Ωœ∑´...", "abcd..."])
    @pytest.mark.xfail(
        reason="Flaky test in multi-process CI environment: GH 44584",
        raises=AssertionError,
        strict=False,
    )
    def test_raw_roundtrip(self, data):
        # PR #25040 wide unicode wasn't copied correctly on PY3 on windows
>       clipboard_set(data)

pandas/tests/io/test_clipboard.py:400: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pandas/io/clipboard/__init__.py:640: in lazy_load_stub_copy
    return copy(text)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <pandas.io.clipboard.init_no_clipboard.<locals>.ClipboardUnavailable object at 0x7f479a3c3460>, args = ('👍...',), kwargs = {}

    def __call__(self, *args, **kwargs):
>       raise PyperclipException(EXCEPT_MSG)
E       pandas.errors.PyperclipException: 
E           Pyperclip could not find a copy/paste mechanism for your system.
E           For more information, please visit
E           https://pyperclip.readthedocs.io/en/latest/#not-implemented-error

pandas/io/clipboard/__init__.py:300: PyperclipException

[mroeschke]

Thanks. Do you get the same errors when you install xsel?

[adrienpacifico]

Thanks. Do you get the same errors when you install xsel?

Yes it does.

The issue is the following:

1. It goes through

   pandas/pandas/io/clipboard/__init__.py

   Line 511 in fe9e5d0

   def determine_clipboard():

   with variable HAS_DISPLAY == False

it then executes

pandas/pandas/io/clipboard/__init__.py

Line 581 in fe9e5d0

return init_no_clipboard()

and thus calls

pandas/pandas/io/clipboard/__init__.py

Line 297 in fe9e5d0

def init_no_clipboard():

and raises the following exception:

def init_no_clipboard():
    class ClipboardUnavailable:
        def __call__(self, *args, **kwargs):
            raise PyperclipException(EXCEPT_MSG)

        def __bool__(self) -> bool:
            return False

    return ClipboardUnavailable(), ClipboardUnavailable()

[mroeschke]

Maybe DISPLAY needs to be set as an environment variable? I see HAS_DISPLAY = os.getenv("DISPLAY", False)

[adrienpacifico]

There is no environment variable, however os.getenv("DISPLAY", False) will set HAS_DISPLAY to False

Changing HAS_DISPLAY = os.getenv("DISPLAY", True) (instead of False) will make the 3 tests as xfailed.

As my environment has (I guess) no display, I feel that it is not what is supposed to be done.

[adrienpacifico]

I see that in the CI, tests are launched with xvfb-run, which probably explain why they are passing.

However, if I try to intall xvfb ( apt-get install xvfb ) in the docker image, I get a strange error:

(pandas-dev) root@61580073cc05:/workspaces/pandas# xvfb-run pandas/tests/io/test_clipboard.py 
/usr/bin/xvfb-run: 184: pandas/tests/io/test_clipboard.py: Permission denied

[adrienpacifico]

Moreover, if I

Xvfb :99 &
export DISPLAY=:99

Then the value of HAS_DISPLAY becomes equal to ':99', and the tests now Xpass!

However, if I get it right we should have:

• Tests launched without display --> should xfail
• Tests launched with display --> should pass

@mroeschke, I'll try to get that behavior by modifiying the tests and the maybe I should make a PR?

Is there a way to see if this test pass somewhere in the CI?

My proposition would be to :

• Run the test if a environment variable DISPLAY is set (current behavior is to xpass it)
• xfail the test if there is no environment variable

Code proposition:

    @pytest.mark.parametrize("data", ["\U0001f44d...", "Ωœ∑´...", "abcd..."])
    @pytest.mark.xfail(
        os.environ.get('DISPLAY') is None, 
        reason="Cannot be runned if a headless system is not put in place with ",
        strict=True,
    )
    def test_raw_roundtrip(self, data):
        # PR #25040 wide unicode wasn't copied correctly on PY3 on windows
        clipboard_set(data)
        assert data == clipboard_get()

[mroeschke]

Thanks for investigating. Yes a PR with that xfail would be fine. Since this modifies the test to work in "local" development environments, as long as the CI still passes (which doesn't use the container), should be a fine addition.