ENH: option to change the number in _dir_additions_for_owner

[TheDerivator]

• closes ENH: raise column number limit for user-completion or add warning #37996
• tests added / passed
• Allows to optionally change the number of columns from a dataframe that are added to dir, which is frequently also the number of columns suggested by tab completion, by using the option pandas.set_option('display.max_dir_additions', 100) to a higher number than 100 or None for an unlimited number of columns. Changing the value of display.max_dir_additions will only take effect for existing dataframes after a column is added or deleted, as the result of dir is cached.

[jreback]

why are we not just using max_columns for this?

[TheDerivator]

why are we not just using max_columns for this?

If you have a dataframe with 10 000 columns you will not want to have them all pretty printed, but you may still want to have all columns suggested by tab completion. (edit: so you need to be able to set them to a different value)

[jreback]

really? you are going to scroll thru them?

i don't really like adding yet another option

if people really want to scroll then just increase the default

[TheDerivator]

No you do not scroll thru them. Instead you type the first two (or so) letters and press the tab button. The editor will then suggest names for columns that start with those letters. This is called tab completion. Except that, currently, it will not suggest names of columns that are beyond the first 100 columns since these are not known to dir.

For instance, if you grab a jupyter notebook and take some random data

import string
import pandas as pd
import numpy as np
columns = [u + v for u in string.ascii_lowercase for v in string.ascii_lowercase]
data = np.random.normal(0, 1, (100, len(columns)))
df = pd.DataFrame(data, columns=columns)

then df.e<tab> will not return any suggestions for columns. (It will suggest things like df.eq though, but because this is a method.) With this pull request, if you set pd.set_option('display.max_dir_items', None) then it will suggest ea etc. until ez. However if you were to set at the same time pd.set_option('display.max_columns', None) then df will print all 26 * 26 columns.

[jreback]

i c ok then

need tests
and update the options documentation
and a ehatsnew note

[jreback]

cc @pandas-dev/pandas-core and @arw2019 if any comments here.

[jreback]

thanks @TheDerivator