DOC: improve readability of signature description (type hints)

[normanius]

Problem description

I'm generally grateful for the efficient documentation of pandas, and I like the advancements in the new layout of recent versions, except for one thing:

The function/object signature at the very top is much harder to read now that the type annotations are printed as well. How about shading the type annotation in gray, or leaving it away altogether? It's enough if the type is described in the "Parameters" section IMHO. Also, the argument could be formatted differently than the default values.

I don't want to unleash a discussion about aesthetics, simply knock the issue away in case it doesn't comply with the guidelines. Thanks anyway!

[WillAyd]

@jorisvandenbossche

[TomAugspurger]

Currently, the type hint and the argument are in the same HTML element. According to sphinx-doc/sphinx#5868 Sphinx 3.0 will have better options. Until then I'm not sure if there's much to be done.

[jorisvandenbossche]

Personally, I am very much in favor of what @normanius suggests. I opened an issue about that myself a while ago: #28585 (but @WillAyd you objected to it then about leaving them away)

And so for the option of leaving them out, that is already possible right now: http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_typehints

For a different styling, that is indeed not yet possible right now, but from the example in sphinx-doc/sphinx#5868 (comment), I also not sure it will be easy in the upcoming release to eg just make the type grey (since the type has the same class as the parameter name)

[TomAugspurger]

I'd be OK with hiding them for now using http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_typehints.

When sphinx 3 is released, I'd be happy with un-hiding them, but de-emphasizing them with CSS if possible.

[jorisvandenbossche]

And to give my reasoning for wanting to hide them: for me, the primary target for type hints are developers (whether developing pandas, or developing other code that uses pandas). And developers can use an IDE which shows them those type hints, or use tools to check them, ... they shouldn't need to look at the online docs for checking the type hint.
On the other hand, the primary target for the online docs are general users. Of course, there is no strict line between "user" and "developer", as any user is writing code and thus developing. But still, I think the average pandas user will not know type hints, and for them, this is simply noise.

To summarize, I think the average pandas user shouldn't need to see / try to understand something like "Union[List[Callable], Tuple[Callable, ...], Mapping[Union[str, int], Callable], NoneType]" (the type hint for formatters in to_string).

[normanius]

Agreeing with Joris, I'd like to add that the (optional) type hints in the signature and the type description in the parameter section are only partially redundant, which can be confusing. Users probably refer to the latter source by preference (because more descriptive and always present in the docs), rendering the other less useful. The two "type languages" (e.g. "Index or array-like" vs. "Optional[Collection]") also make it difficult for beginners to get comfy with the docs.

How about adding a "type hints" button as a compromise? (Hoping that the doc-generator somehow supports collapsibles out of the box?)

---

[TomAugspurger]

Long-term I think a toggle is best. I don't know if it's possible today (but it'd be great if someone could experiment).

[TomAugspurger]

@jorisvandenbossche did #33312 fix this?

Right now at https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.concat.html I see this:

[jorisvandenbossche]

Hmm, that's clearly not working there (and it's IMO also good example of why it would be better to now show those type hints in that form ..)

Now, the example I tested when doing the PR is actually working: https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.DataFrame.html (dev docs, no type hints) vs https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.html (with type hints)

But eg also for https://pandas.pydata.org/pandas-docs/dev/reference/api/pandas.DataFrame.to_string.html it's not working

[krassowski]

One thing that I managed to get working is creating a custom object and then telling static type checker that it should be considered to be of a different type. For example:

def search(engine: Literal['google', 'bing', 'yandex', 'wikipedia']):
    pass

can be replaced with:

_SearchEngineType = Literal['google', 'bing', 'yandex', 'wikipedia']


class SearchEngineType:
    """Either of: 'google', 'bing', 'yandex', 'wikipedia'"""


SearchEngineT = SearchEngineType()
SearchEngineT.__supertype__ = _SearchEngineType


def search(engine: SearchEngineT):
    """Sends a search query to an engine.

    Parameters:
        engine: the engine
    """
    pass

which results in:

I then move types to a different file and page in the documentation which gives a nice separation (but the user can still easily find out what is behind each type by clicking the link!).

Not saying that it will work for pandas, but I hope that it might give you some inspiration. The core idea is to hide the complexity behind the type definition which is optimized towards static type checkers (and not humans!) by leveraging the mechanism which is used by typing.NewType, i.e. overwriting obj.__supertype__.

This currently only works with 'sphinx_autodoc_typehints' extension (and I've hidden the standard autodoc types with
autodoc_typehints = 'none').

The only thing that might confuse the user is the presence of class prefix in the type definition.

[simonjayhawkins]

This is now fixed.