[WIP] Don't move README to index on check if there exists an index file

[gregmuellegger]

I removed the logic to move the README to be the index of the documentation.

However this doesn't work straight away, as I tested this with https://github.com/wdv4758h/python-best-practices/tree/468cec7d100af41d4f35baa1728e31e7ba04fb7d
The problem here is that the conf.py is inserted automatically and assumes a index file to exist. The build output is this:

Exception occurred:
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/environment.py", line 1272, in get_doctree
    f = open(doctree_filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/checkouts/latest/_build/doctrees-readthedocs/index.doctree'
The full traceback has been saved in /tmp/sphinx-err-mhhzzjwp.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!
Traceback (most recent call last):
  File "/home/gregor/projects/readthedocs.org/readthedocs/projects/tasks.py", line 129, in run
    outcomes = self.build_docs()
  File "/home/gregor/projects/readthedocs.org/readthedocs/projects/tasks.py", line 353, in build_docs
    outcomes['html'] = self.build_docs_html()
  File "/home/gregor/projects/readthedocs.org/readthedocs/projects/tasks.py", line 369, in build_docs_html
    success = html_builder.build()
  File "/home/gregor/projects/readthedocs.org/readthedocs/doc_builder/backends/sphinx.py", line 146, in build
    bin_path=project.venv_bin(version=self.version.slug)
  File "/home/gregor/projects/readthedocs.org/readthedocs/doc_builder/base.py", line 121, in run
    return self.build_env.run(*args, **kwargs)
  File "/home/gregor/projects/readthedocs.org/readthedocs/doc_builder/environments.py", line 304, in run
    return self.run_command_class(cls=self.command_class, cmd=cmd, **kwargs)
  File "/home/gregor/projects/readthedocs.org/readthedocs/doc_builder/environments.py", line 335, in run_command_class
    raise BuildEnvironmentWarning(msg)
BuildEnvironmentWarning: Command python /home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/bin/sphinx-build -T -b readthedocs -d _build/doctrees-readthedocs -D language=en . _build/html failed:
Running Sphinx v1.3.1
making output directory...
loading translations [en]... done
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [readthedocs]: targets for 2 source files that are out of date
updating environment: 2 added, 0 changed, 0 removed
reading sources... [ 50%] README
reading sources... [100%] misc

WARNING: master file /home/gregor/projects/readthedocs.org/user_builds/python-best-practices/checkouts/latest/index.rst not found
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/gregor/projects/readthedocs.org/user_builds/python-best-practices/checkouts/latest/README.rst:: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [ 33%] README

Traceback (most recent call last):
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/cmdline.py", line 245, in main
    app.build(opts.force_all, filenames)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/application.py", line 264, in build
    self.builder.build_update()
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/__init__.py", line 245, in build_update
    'out of date' % len(to_build))
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/__init__.py", line 316, in build
    self.write(docnames, list(updated_docnames), method)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/__init__.py", line 354, in write
    self._write_serial(sorted(docnames), warnings)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/__init__.py", line 362, in _write_serial
    self.write_doc(docname, doctree)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/html.py", line 445, in write_doc
    self.handle_page(docname, ctx, event_arg=doctree)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/html.py", line 786, in handle_page
    output = self.templates.render(templatename, ctx)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/jinja2glue.py", line 138, in render
    return self.environment.get_template(template).render(context)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/jinja2/environment.py", line 989, in render
    return self.environment.handle_exception(exc_info, True)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/jinja2/environment.py", line 754, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/jinja2/_compat.py", line 37, in reraise
    raise value.with_traceback(tb)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/themes/basic/page.html", line 10, in top-level template code
    {%- extends "layout.html" %}
  File "/home/gregor/projects/readthedocs.org/readthedocs/templates/sphinx/layout.html", line 1, in top-level template code
    {% extends "!layout.html" %}
  File "/home/gregor/projects/readthedocs.org/readthedocs/templates/sphinx/sphinx_rtd_theme/layout.html", line 121, in top-level template code
    {% block menu %}
  File "/home/gregor/projects/readthedocs.org/readthedocs/templates/sphinx/sphinx_rtd_theme/layout.html", line 122, in block "menu"
    {% set toctree = toctree(maxdepth=4, collapse=theme_collapse_navigation, includehidden=True) %}
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/jinja2/sandbox.py", line 355, in call
    return __context.call(__obj, *args, **kwargs)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/html.py", line 776, in <lambda>
    ctx['toctree'] = lambda **kw: self._get_local_toctree(pagename, **kw)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/builders/html.py", line 712, in _get_local_toctree
    docname, self, collapse, **kwds))['fragment']
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/environment.py", line 1238, in get_toctree_for
    doctree = self.get_doctree(self.config.master_doc)
  File "/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/envs/latest/lib/python3.4/site-packages/sphinx/environment.py", line 1272, in get_doctree
    f = open(doctree_filename, 'rb')
FileNotFoundError: [Errno 2] No such file or directory: '/home/gregor/projects/readthedocs.org/user_builds/python-best-practices/checkouts/latest/_build/doctrees-readthedocs/index.doctree'

Any ideas how we could solve this easily and elegantly?

Related #1550.

[gregmuellegger]

Could we test in the auto-generated conf.py if a index.rst exists? Or is it possible that the source directory is different to the directory that the conf.py is in?

[ericholscher]

I think requiring an index file is a reasonable thing -- We also have logic for auto-creating an index file if it doesn't exist, which should be triggered in this case, no?

[ericholscher]

https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/doc_builder/base.py#L90

[ericholscher]

Closing this for #1597 -- which I think is the proper solution.