Skip to content

Tab completion in IPython 6 on Categorical shows warning #16409

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wesm opened this issue May 21, 2017 · 11 comments
Closed

Tab completion in IPython 6 on Categorical shows warning #16409

wesm opened this issue May 21, 2017 · 11 comments
Labels
Dependencies Required and optional dependencies
Milestone
0.20.2

Comments

@wesm
Copy link
Member

wesm commented May 21, 2017

With pandas 0.20.1, IPython 6.0.0 (stack from conda-forge) I have:

In [31]: obj = pd.Categorical(['foo', 'bar', 'baz'])

In [32]: obj.<TAB>
/home/wesm/miniconda/envs/book-env/lib/python3.6/site-packages/jedi/evaluate/compiled/__init__d
  getattr(obj, name)

I'm not sure the origin or meaning in the output, but it's a bad smell that may confuse users
@TomAugspurger
Copy link
Contributor

IPython 6 switched over to jedi for completion, which must execute at least some code? Categorical.labels is a property. It looks like we can get around this by specifying __dir__.
I think I've seen it in at least one other place.

@Carreau does this sound like an IPython bug? For a simpler example:

In [32]: import warnings

In [33]: class Foo:
    ...:     @property
    ...:     def bad(self):
    ...:         warnings.warn("bad")
    ...:     @property
    ...:     def good(self):
    ...:         return "good"

In [34]:

In [34]: f = Foo()

/Users/taugspurger/Envs/pandas-dev/bin/ipython:4: UserWarning: bad
  import re
In [35]: f.<TAB>

If so I'll file one on IPython. Either way I'll put up a PR to silence these using __dir__.

@Carreau
Copy link
Contributor

Carreau commented May 21, 2017 via email

@wesm
Copy link
Member Author

wesm commented May 21, 2017

FWIW, the Jedi completion in IPython 6 feels like a gigantic step backwards in UX in the IPython shell. I'm seeing 500ms or more tab completion lag on objects when using pandas; having used IPython since version 0.8.4 and never had this problem, a bit of a bummer =( I suspect pandas is a case of a "big" API that stretches some of the Jedi worst case scenarios. Is there a way to revert to the old completer?

@Carreau
Copy link
Contributor

Carreau commented May 22, 2017 

c.Completer.jedi_compute_type_timeout=<Int>
    Experimental: restrict time (in milliseconds) during which Jedi can compute
    types. Set to 0 to stop computing types. Non-zero value lower than 100ms may
    hurt performance by preventing jedi to build its cache.
    Default: 400

and

c.Completer.use_jedi=<Bool>
    Experimental: Use Jedi to generate autocompletions. Default to True if jedi
    is installed
    Default: True

These two options can be of help, the old completer has not been deactivated and completions are merged with jedi ones.
Actually increasing the jedi timeout may make things faster as if you give enough time to jedi it will save a cache of the type-inference tree on disk and re-use that. Which should make most things ~quasi instantaneous (at least for me on matplotlib).

Dev branch of jedi is also way faster (but still have some bug).

@TomAugspurger
Copy link
Contributor

Thanks @Carreau. ipython/ipython#9094 is the IPython issue for properties executing code.

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 22, 2017
@TomAugspurger
COMPAT: Catch warnings on tab-complete in IPy 6
2d048c9 
Properties may run code with Jedi completion in IPython 6

Closes pandas-dev#16409
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 22, 2017
@TomAugspurger
COMPAT: Catch warnings on tab-complete in IPy 6
da2d290 
Properties may run code with Jedi completion in IPython 6

Closes pandas-dev#16409
@TomAugspurger TomAugspurger mentioned this issue May 22, 2017
Merged
@TomAugspurger TomAugspurger added this to the 0.20.2 milestone May 22, 2017
@TomAugspurger TomAugspurger added Dependencies Required and optional dependencies Usability labels May 22, 2017
@jorisvandenbossche
Copy link
Member

For me it happens on tab completing modules with deprecations as well. Eg pd.lib.<TAB> displays a huge list of warnings (but that might also be related to the way we implemented the shim, but in any case was not visible in IPython < 6.0)

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 23, 2017
@TomAugspurger
COMPAT: Catch warnings on tab-complete in IPy 6
8d9689f 
Properties may run code with Jedi completion in IPython 6

Closes pandas-dev#16409
@DGrady
Copy link
Contributor

DGrady commented May 23, 2017

FWIW I see this (only the first time?) when I try to tab complete on a data frame:

Python 3.6.1 |Anaconda custom (x86_64)| (default, May 11 2017, 13:04:09)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import pandas as pd

In [2]: df = pd.DataFrame()

In [3]: df.gro2017-05-23 10:27:43.062 python[28545:3846149] !!! BUG: The current event queue and the main event queue are not the same. Events will not be handled correctly. This is probably because _TSGetMainT.
upby

I pressed TAB after typing df.gro.

@TomAugspurger
Copy link
Contributor

@DGrady that seems unrelated. Is it just on DataFrames that happens, or any python object? If you could narrow it down a bit, I'm sure the IPython devs would appreciate a bug report.

@DGrady
Copy link
Contributor

DGrady commented May 23, 2017

@TomAugspurger You're right; it's something about my environment. That does not happen with a clean environment with conda create --name completiontest python=3 ipython pandas; sorry to confuse the issue!

@Carreau
Copy link
Contributor

Carreau commented May 23, 2017

@DGrady do you have any Spark/Java thing running ? I think I've seen similar bugs when trying to use IPython with Jython.

@DGrady
Copy link
Contributor

DGrady commented May 23, 2017

@Carreau I do not; I do use Spark, but not from this machine. Just running macOS 10.12.5. Here's the output of conda list for the offending environment:

(arg, the ones with missing channels are <pip>)

# packages in environment at /Users/dgrady/anaconda3:
#
_license                  1.1                      py36_1
alabaster                 0.7.10                   py36_0
anaconda                  custom                   py36_0
anaconda-client           1.6.3                    py36_0
anaconda-navigator        1.5.2                    py36_0
anaconda-project          0.6.0                    py36_0
appnope                   0.1.0                    py36_0
appscript                 1.0.1                    py36_0
asn1crypto                0.22.0                   py36_0
astroid                   1.4.9                    py36_0
astropy                   1.3.2               np112py36_0
awscli                    1.11.48                   
babel                     2.4.0                    py36_0
backports                 1.0                      py36_0
beautifulsoup4            4.6.0                    py36_0
bitarray                  0.8.1                    py36_0
blaze                     0.10.1                   py36_0
bleach                    1.5.0                    py36_0
bokeh                     0.12.5                   py36_1
boto                      2.46.1                   py36_0
botocore                  1.5.11                    
bottleneck                1.2.1               np112py36_0
cffi                      1.10.0                   py36_0
chardet                   3.0.3                    py36_0
chest                     0.2.3                    py36_0
click                     6.7                      py36_0
cloudpickle               0.2.2                    py36_0
clyent                    1.2.2                    py36_0
colorama                  0.3.9                    py36_0
conda                     4.3.18                   py36_0
conda-env                 2.6.0                         0
configobj                 5.0.6                    py36_0
contextlib2               0.5.5                    py36_0
cryptography              1.8.1                    py36_0
curl                      7.52.1                        0
cycler                    0.10.0                   py36_0
cython                    0.25.2                   py36_0
cytoolz                   0.8.2                    py36_0
dask                      0.14.3                   py36_0
datashape                 0.5.4                    py36_0
decorator                 4.0.11                   py36_0
dill                      0.2.6                    py36_0
docutils                  0.13.1                   py36_0
entrypoints               0.2.2                    py36_1
et_xmlfile                1.0.1                    py36_0
fastcache                 1.0.2                    py36_1
flake8                    3.3.0                    py36_0
flask                     0.12.2                   py36_0
flask-cors                3.0.2                    py36_0
freetype                  2.5.5                         2
get_terminal_size         1.0.0                    py36_0
gevent                    1.2.1                    py36_0
graphviz                  2.38.0                        4
greenlet                  0.4.12                   py36_0
h5py                      2.7.0               np112py36_0
hdf5                      1.8.17                        1
heapdict                  1.0.0                    py36_1
html5lib                  0.999                    py36_0
icu                       54.1                          0
idna                      2.5                      py36_0
imagesize                 0.7.1                    py36_0
ipykernel                 4.6.1                    py36_0
ipython                   6.0.0                    py36_1
ipython_genutils          0.2.0                    py36_0
ipywidgets                6.0.0                    py36_0
isort                     4.2.5                    py36_0
itsdangerous              0.24                     py36_0
jbig                      2.1                           0
jdcal                     1.3                      py36_0
jedi                      0.10.2                   py36_2
jinja2                    2.9.6                    py36_0
jmespath                  0.9.1                     
jpeg                      9b                            0
jsonschema                2.6.0                    py36_0
jupyter                   1.0.0                    py36_3
jupyter_client            5.0.1                    py36_0
jupyter_console           5.1.0                    py36_0
jupyter_core              4.3.0                    py36_0
lazy-object-proxy         1.2.2                    py36_0
libiconv                  1.14                          0
libpng                    1.6.27                        0
libtiff                   4.0.6                         3
libxml2                   2.9.4                         0
libxslt                   1.1.29                        0
llvmlite                  0.18.0                   py36_0
locket                    0.2.0                    py36_1
lxml                      3.7.3                    py36_0
markupsafe                0.23                     py36_2
matplotlib                2.0.2               np112py36_0
mccabe                    0.6.1                    py36_0
mistune                   0.7.4                    py36_0
mkl                       2017.0.1                      0
mkl-service               1.1.2                    py36_3
mpmath                    0.19                     py36_1
multipledispatch          0.4.9                    py36_0
mypy                      0.501                     
nbconvert                 5.1.1                    py36_0
nbformat                  4.3.0                    py36_0
networkx                  1.11                     py36_0
nltk                      3.2.3                    py36_0
nose                      1.3.7                    py36_1
notebook                  5.0.0                    py36_0
numba                     0.33.0              np112py36_0
numexpr                   2.6.2               np112py36_0
numpy                     1.12.1                   py36_0
numpydoc                  0.6.0                    py36_0
odo                       0.5.0                    py36_1
olefile                   0.44                     py36_0
openpyxl                  2.4.7                    py36_0
openssl                   1.0.2k                        2
packaging                 16.8                     py36_0
pandas                    0.20.1              np112py36_0
pandocfilters             1.4.1                    py36_0
partd                     0.3.8                    py36_0
path.py                   10.3.1                   py36_0
pathlib2                  2.2.1                    py36_0
patsy                     0.4.1                    py36_0
pep8                      1.7.0                    py36_0
pexpect                   4.2.1                    py36_0
pickleshare               0.7.4                    py36_0
pillow                    4.1.1                    py36_0
pip                       9.0.1                    py36_1
ply                       3.10                     py36_0
prompt_toolkit            1.0.14                   py36_0
psutil                    5.2.2                    py36_0
ptyprocess                0.5.1                    py36_0
py                        1.4.33                   py36_0
pyasn1                    0.2.3                    py36_0
pycodestyle               2.3.1                    py36_0
pycosat                   0.6.2                    py36_0
pycparser                 2.17                     py36_0
pycrypto                  2.6.1                    py36_4
pycurl                    7.43.0                   py36_2
pyflakes                  1.5.0                    py36_0
pygments                  2.2.0                    py36_0
pylint                    1.6.4                    py36_1
pyopenssl                 17.0.0                   py36_0
pyparsing                 2.1.4                    py36_0
pyqt                      5.6.0                    py36_2
pytables                  3.3.0               np112py36_0
pytest                    3.0.7                    py36_0
python                    3.6.1                         2
python-dateutil           2.6.0                    py36_0
python-graphviz           0.5.2                    py36_0
python.app                1.2                      py36_4
pytz                      2017.2                   py36_0
pywavelets                0.5.2               np112py36_0
pyyaml                    3.12                     py36_0
pyzmq                     16.0.2                   py36_0
qt                        5.6.2                         2
qtawesome                 0.4.4                    py36_0
qtconsole                 4.3.0                    py36_0
qtpy                      1.2.1                    py36_0
readline                  6.2                           2
redis                     3.2.0                         0
redis-py                  2.10.5                   py36_0
requests                  2.14.2                   py36_0
rope                      0.9.4                    py36_1
rsa                       3.4.2                     
ruamel_yaml               0.11.14                  py36_1
s3transfer                0.1.10                    
scikit-image              0.13.0              np112py36_0
scikit-learn              0.18.1              np112py36_1
scipy                     0.19.0              np112py36_0
seaborn                   0.7.1                    py36_0
setuptools                27.2.0                   py36_0
simplegeneric             0.8.1                    py36_1
singledispatch            3.4.0.3                  py36_0
sip                       4.18                     py36_0
six                       1.10.0                   py36_0
snowballstemmer           1.2.1                    py36_0
sockjs-tornado            1.0.3                    py36_0
sphinx                    1.5.6                    py36_0
spyder                    3.1.4                    py36_0
sqlalchemy                1.1.9                    py36_0
sqlite                    3.13.0                        0
statsmodels               0.8.0               np112py36_0
sympy                     1.0                      py36_0
terminado                 0.6                      py36_0
testpath                  0.3                      py36_0
tk                        8.5.18                        0
toolz                     0.8.2                    py36_0
tornado                   4.5.1                    py36_0
traitlets                 4.3.2                    py36_0
typed-ast                 1.0.3                     
unicodecsv                0.14.1                   py36_0
wcwidth                   0.1.7                    py36_0
werkzeug                  0.12.2                   py36_0
wheel                     0.29.0                   py36_0
widgetsnbextension        2.0.0                    py36_0
wrapt                     1.10.10                  py36_0
xlrd                      1.0.0                    py36_0
xlsxwriter                0.9.6                    py36_0
xlwings                   0.10.4                   py36_0
xlwt                      1.2.0                    py36_0
xz                        5.2.2                         1
yaml                      0.1.6                         0
yapf                      0.16.0                    
zlib                      1.2.8                         3

TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 25, 2017
@TomAugspurger
COMPAT: Catch warnings on tab-complete in IPy 6
adf9343 
Properties may run code with Jedi completion in IPython 6

Closes pandas-dev#16409
TomAugspurger added a commit that referenced this issue May 26, 2017
@TomAugspurger
COMPAT: Catch warnings on tab-complete in IPy 6 (#16414)
3c9a74b 
Properties may run code with Jedi completion in IPython 6

Closes #16409
TomAugspurger added a commit to TomAugspurger/pandas that referenced this issue May 29, 2017
@TomAugspurger
COMPAT: Catch warnings on tab-complete in IPy 6 (pandas-dev#16414)
75efd8a 
Properties may run code with Jedi completion in IPython 6

Closes pandas-dev#16409
(cherry picked from commit 3c9a74b)
TomAugspurger added a commit that referenced this issue May 30, 2017
@TomAugspurger
COMPAT: Catch warnings on tab-complete in IPy 6 (#16414)
2a008a9 
Properties may run code with Jedi completion in IPython 6

Closes #16409
(cherry picked from commit 3c9a74b)
stangirala pushed a commit to stangirala/pandas that referenced this issue Jun 11, 2017
@TomAugspurger @stangirala
COMPAT: Catch warnings on tab-complete in IPy 6 (pandas-dev#16414)
880108b 
Properties may run code with Jedi completion in IPython 6

Closes pandas-dev#16409
@bilderbuchi bilderbuchi mentioned this issue Aug 9, 2017
Open
@keflavich keflavich mentioned this issue Sep 6, 2017
Closed
@TomAugspurger TomAugspurger mentioned this issue Sep 25, 2017
Closed
@jorisvandenbossche jorisvandenbossche mentioned this issue Aug 13, 2019
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Dependencies Required and optional dependencies
Projects
None yet
Milestone
0.20.2
Development

No branches or pull requests
5 participants
@wesm @Carreau @DGrady @jorisvandenbossche @TomAugspurger