Redis and Elasticsearch in installation instructions

[kwisatz]

The installation instructions at http://read-the-docs.readthedocs.org/en/latest/install.html do not mention Redis or Elasticsearch connectivity or setup. However, the server's logs clearly show how that app is trying to connect to both services.

Would it be possible to add some sentences on whether these services are actually required and what for?

[ericholscher]

If you would like to support docs around local installs, we would welcome the contribution. As it stands, we don't have time to officially support it.

[tcosta84]

I second that! I just found out I needed Elastic Search. How can I contribute with the docs?

[gregmuellegger]

@tcosta84 You can fork the readthedocs.org repository here on GitHub and start modifying the relevant files in the docs directory. Then commit and push your changes to a branch in your fork and propose a pull request when you think your changes are worthwhile to be integrated into readthedocs.

[deatharse]

@tcosta84 @kwisatz It would be great if you could do this as it would help others like myself who has reached a blocker with the CNAME.

[kwisatz]

I could contribute, but it's still unclear to me whether those services are really required and again, what for? I'm happy to write a few sentences, but I'd need to know the context ;)

[tcosta84]

This week I'm going to install RTD locally so I will be able to contribute at least on understanding all the stack components. The only thing I'm sure is that Elastic Search is necessary to enable full text search. If it's disabled you won't be able to do a search. As for Redis, I know it is used as a cache but I'm not sure if you can ignore iit without errors.

[gregmuellegger]

Like @tcosta84 said, Elastic Search is required for a working search index, but the site will still work without it (except search). Redis is used as a message broker and result storage for Celery, the task queue in charge. It's needed to queue build jobs and other tasks. So its quite vital for a production install.

[deatharse]

I've managed to get it all installed and working except for ES. I put the file referenced http://read-the-docs.readthedocs.org/en/latest/rtfd.html#elastic-search-setup in checkouts/readthedocs.org/readthedocs and prepended it with:

import os
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "settings.__init__")

however when I run it, it fails with the trace

Traceback (most recent call last):
  File "es.py", line 8, in <module>
    index.create_index(index_name)
  File "/usr/share/readthedocs/checkouts/readthedocs.org/readthedocs/search/indexes.py", line 106, in create_index
    self.es.indices.create(index=index, body=body)
  File "/usr/share/readthedocs/local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 69, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/usr/share/readthedocs/local/lib/python2.7/site-packages/elasticsearch/client/indices.py", line 102, in create
    params=params, body=body)
  File "/usr/share/readthedocs/local/lib/python2.7/site-packages/elasticsearch/transport.py", line 307, in perform_request
    status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
  File "/usr/share/readthedocs/local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 89, in perform_request
    self._raise_error(response.status, raw_data)
  File "/usr/share/readthedocs/local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 105, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: TransportError(400, u'IndexCreationException[[readthedocs-20150918112956] failed to create index]; 
nested: IllegalArgumentException[Custom Analyzer [default_icu] failed to find tokenizer under name [icu_tokenizer]]; ')

[tcosta84]

@alasdairdc how did you get all working? Did you create local_settings.py and managed to execute ./manage.py migrate without errors?

Can you please describe how did you get at this point?

[deatharse]

@tcosta84 well the only thing I dont have working is elasticsearch for pages and i have noticed the same error reported in #1045 how far have your got? I did try to document the steps i took (for myself) but I will verify them just now in a VBox and post them here.

[deatharse]

@tcosta84 Not had a chance to verify it yet but you can find the steps I took here: https://gist.github.com/alasdairdc/25ecb68e692fdd615bc9#file-rtd-production

[RichCastar]

@alasdairdc Did you ever get ES working with readthedocs?

[midacts]

ehhhh doesnt look like it is working?

[RichardLitt]

@midacts Can you open a new issue, or clarify whether you have the same problem?

[midacts]

@RichardLitt Looks like this answered it. Under the "Elastic Search Setup" section.

Thanks

[deatharse]

@RichCastar Apologies for the late response, your remark slipped through the net. I hadn't gotten it to work and abandoned trying partially due to the resource requirements for ES. Glad the docs have been updated and people are having more success now.