Rebase ipython_directive on top of recent ipython updated version

[ghost]

ipython/ipython#4570

We'd prefer to get rid of the duplication, but we have some tweaks in place
which ipython does not.

I rebased everythingon top of ipython to generae a clean diff, if they pick it up
we can make the transition, otherwise we synched for now.

cc @takluyver, if you're interested, check the diff between 2e3c608 and 82b8c3a
for an unobstructed view.

@jorisvandenbossche, can you check this out and let me know if I missed anything?
I included the cython magic fixes you pointed out.

related #5221

[jorisvandenbossche]

Some early feedback (not yet tested thoroughly):

• ipython prompt counting is not working on some pages (stays on [1]), and there is also no output on those pages.
• io.rst unicode error is not yet solved .. (on windows)
• Out prompt is not always aligned, but that was something we also encountered in Update IPython directive ipython/ipython#4570 (Update IPython directive ipython/ipython#4570 (commits))

[ghost]

Thanks, I'll look at those tomorrow.

[ghost]

@cpcloud , the cython fix you put in ipython_directive should find it's way upstream,
do you remember what the issue was?

[ghost]

Ok, that was very strange (and a nightmare to track down). a snippet I add long ago to faq.rst which demonstrates... monkey patching (q @jreback gloating) borked the numbering by ipython directive.

Took another crack at GH5142, after rereading the issues.

@jorisvandenbossche, seal of approval?

[jreback]

@y-p monkey patching pandas is ok :) its just other packages BY pandas which is a problem :)

[ghost]

Yep, except when that break stuff as well. case in point.

[jorisvandenbossche]

@y-p I can't test it today anymore, but I will do tomorrow.

[ghost]

Thanks, no rush. You've just proven to have a better eye for it.

[jorisvandenbossche]

A big +1 on this!

I did build the docs, and it seems good:

• all prompt / code blocks seem OK (correct alignment, counting, etc)
  • there is a small difference with ipython master, with your version the Out [] prompts are red, with ipython's version they are blue :-) doesn't matter, but don't know why.
  • In the current version of the docs, we didn't have Out [] prompts. Now there are, if we want there is an option to change that, but for me this is OK (it's a little more crowdier, but it's what users also see in there console).
• regarding the io.rst / unicode issue:
  • it builds without error on windows!
  • however, the output in the html is not yet good (I get such questionmarks), although this is less important. I am a complete encoding noob, so I don't really understand the reasoning of your changes, but this commit (https://github.com/y-p/pandas/commit/a9b0ef5388ba68f450667e89c04c0a15f297089a) doesn't matter (doesn't change anything in the output/building).
• okexcept is working fine, errors in code are sent to console
• no warnings anymore! thanks to the moving of the template (well, there were 7 warnings, but all related to references to the deleted api.rst)

So as long you don't forget to put back the api docs before merging, this is looking good.

[ghost]

Twice, I told myself twice to remember to get rid of that commit before pushing. sigh

The coloring and Out prompts are not intentional but no harm I guess, would rather move on.
The question marks are due to using decode(foo, errors='replace') which always succeds and returns
unicode.
I'm still unclear about the unicode issue. The current docs contain a string which is not valid utf8
and I fully expected that to create all kinds of trouble, not sure why it doesn't.

I'll give the unicode bit another look, I was sure the issue was solvable. Am still holding out
for some of this to get merged upstream into ipython.

Thanks.

[jorisvandenbossche]

In every case, the question marks in the output is better than not being able to build it (as long as the docs on the website are build on linux where it is not a problem).

And indeed, the prompts/coloring is certainly not an issue to me, just remarked the difference.

[jorisvandenbossche]

I don't know if it is useful, but it seems a little bit related: in matplotlib/numpydoc they had also an issue with special characters: numpy/numpydoc#1, matplotlib/matplotlib#2529