find_MAP is failing in V4

[ricardoV94]

While working on #4770 I found that find_MAP is failing quite often after the V4 refactoring.

Most issues seem related to gradients of discrete vars and/or models with both discrete and continuous unobserved variables. The failing tests in here are marked with an xfail for the time being.

[michaelosthege]

Could it be a problem with the initialization too? (crossref #4752)
I found "find_MAP" to be quite brittle when the initialization is bad. Not only because optimization is hard, but also because the automatic convergence detection often makes it give up after a handful of iterations.

[ricardoV94]

I have the impression it's more than that. But it was some time since I had a look

[twiecki]

@michaelosthege is this fixed now?

[ricardoV94]

This is not fixed. The tests marked with xfail above are still failing in main

[michaelosthege]

Yeah we need the get_moment implementations on the most important distributions plus the switch of the default strategy.

I opened #5087, but currently don't have any time to continue working on it for the near future. Anybody please feel free to take over.

[ricardoV94]

I double checked, the issue is not (just) the starting points. We are getting Type/Shape Errors somewhere inside the scipy optimization library

[ricardoV94]

After #5320 there is only one test that is failing: https://github.com/pymc-devs/pymc/blob/6191c61d0f913c2c5f18effd3074548f9d266186/pymc/tests/test_starting.py#L49

Some shape error comes from deep inside the scipy optimization routine. Only the first of the two calls to find_MAP seems to be raising, suggesting it is something to do with the exclusion of the discrete variable, which does not happen in the second call:

https://github.com/pymc-devs/pymc/blob/6191c61d0f913c2c5f18effd3074548f9d266186/pymc/tests/test_starting.py#L62-L63