DOC: Add notice and example for CategoricalDtype with different categories_dtype

[luke396]

• closes DOC: CategoricalDtype equality semantics aren't completely described #57259 (Replace xxxx with the GitHub issue number)
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

[VladimirFokow]

Thanks @luke396 !

I was hoping that this part would be edited:

Two instances of :class:`~pandas.api.types.CategoricalDtype` compare equal
whenever they have the same categories and order. When comparing two
unordered categoricals, the order of the ``categories`` is not considered.

to be a complete and definitive definition, covering all cases right here.

(instead of adding examples and having the user then guess the actual rules)

[VladimirFokow]

More generally, I am confused:

A categorical's type is fully described by
1. ``categories``: a sequence of unique values and no missing values
2. ``ordered``: a boolean

... If the CategoricalDtype is "fully described" by categories and order,
and categories is defined as a "sequence of unique values and no missing values"
--> [] is a "sequence of unique values and no missing values"
(yes, it's empty, but it is a sequence of 0 unique values and doesn't contain NaNs (the missing values))
--> there is no dtype anywhere in this definition..
So should this description be corrected as well?

These should be good precise definitions giving an accurate conceptual understanding.

[luke396]

Two instances of :class:`~pandas.api.types.CategoricalDtype` compare equal
whenever they have the same categories and order. When comparing two
unordered categoricals, the order of the ``categories`` is not considered.

Thank you, @VladimirFokow, for your advice on the PR. I hadn't considered it as thoroughly as you did. At the moment, the example in the file doesn't suit my needs well, which is why the PR is still in progress. Have to admit it, I actually don't have the same accurate and comprehensive understanding of Categorical as you do.

These should be good precise definitions giving an accurate conceptual understanding.

You've shown great consideration in enhancing the docstring, and your insights could be used to open a new pull request aimed at improving the general description further.

[VladimirFokow]

Maybe they meant that a Categorical👈 (not CategoricalDtype) is partly described by .categories...
But in this case, categories is NOT a sequence.
It's an Index (containing the categories, and of a certain dtype which is determined... automatically? - even if constructed from a numpy array of objects yet all integers - the dtype of this index will be int, not object...

How is this dtype determined - completely disregarding the numpy dtype and just looking at the actual values? How to operate with it - can we change it / what are the best practices when dealing with problems connected to it)

---

upd: Maybe what they meant here is a purely conceptual description, because after it they say:

This information can be stored in a CategoricalDtype.

[luke396]

Hi @rhshadrach, could you please review the PR and provide some comments for improvement?

[rhshadrach]

Thanks for the PR!

[luke396]

@rhshadrach Thanks for your prompt review! I have updated the PR based on your comments.

[rhshadrach]

lgtm

[rhshadrach]

/preview

[github-actions]

Website preview of this PR available at: https://pandas.pydata.org/preview/pandas-dev/pandas/57273/

[rhshadrach]

Thanks @luke396