Add admin interface for reindexing a project search

[dojutsu-user]

Related issue - #4654

• On Project admin, a command that reindexes all active versions (project queryset and all attached active versions)
• On Version admin, a command that reindexes by version queryset instead
• Drop management command, these aren't good interfaces for site admins and are mostly hidden
• The management command seems to just do all versions active or not, we should probably test for active versions instead.
• There should also be an admin function to wipe the index for a version and for a project

[dojutsu-user]

This PR is in very initial phase.
I have just used the management command from admin interface.
Work is in progress.

[codecov]

Codecov Report

Merging #4992 into master will decrease coverage by 0.05%.
The diff coverage is 38.46%.

@@            Coverage Diff            @@
##           master   #4992      +/-   ##
=========================================
- Coverage   76.76%   76.7%   -0.06%     
=========================================
  Files         158     158              
  Lines        9953    9965      +12     
  Branches     1245    1246       +1     
=========================================
+ Hits         7640    7644       +4     
- Misses       1980    1988       +8     
  Partials      333     333

Impacted Files	Coverage Δ
readthedocs/projects/admin.py	85.59% <38.46%> (-5.92%)	⬇️

[codecov]

Codecov Report

Merging #4992 into master will decrease coverage by 0.7%.
The diff coverage is 20.83%.

@@            Coverage Diff             @@
##           master    #4992      +/-   ##
==========================================
- Coverage   77.04%   76.33%   -0.71%     
==========================================
  Files         158      158              
  Lines       10108    10021      -87     
  Branches     1274     1250      -24     
==========================================
- Hits         7788     7650     -138     
- Misses       1989     2037      +48     
- Partials      331      334       +3

Impacted Files	Coverage Δ
readthedocs/projects/admin.py	72.02% <18.42%> (-19.49%)	⬇️
readthedocs/builds/admin.py	52.63% <23.52%> (-43.21%)	⬇️
readthedocs/vcs_support/backends/svn.py	29.5% <0%> (-9.89%)	⬇️
readthedocs/core/utils/__init__.py	74.73% <0%> (-5.68%)	⬇️
readthedocs/vcs_support/backends/hg.py	64.51% <0%> (-3.18%)	⬇️
readthedocs/notifications/storages.py	82.53% <0%> (-3.18%)	⬇️
readthedocs/doc_builder/config.py	91.89% <0%> (-2.56%)	⬇️
readthedocs/projects/querysets.py	80.58% <0%> (-1.07%)	⬇️
readthedocs/core/views/serve.py	87.25% <0%> (-0.99%)	⬇️
readthedocs/projects/exceptions.py	84.21% <0%> (-0.79%)	⬇️
... and 29 more

[dojutsu-user]

I think we should keep the management command because it is used in these tests

    readthedocs.org/readthedocs/search/tests/test_views.py

[dojutsu-user]

@agjohnson
I don't know how to write the tests relating about elastic search.
Can you please give me some advice on this.

[ericholscher]

This is a good addition. We should simplify the admin actions so they are just wrappers around a small utility function that does the logic. It will make it easier to maintain, and also port to our upgraded search (once we figure out how to ship it)

[dojutsu-user]

@ericholscher
After the update of the ES, I have updated the indexing/un-indexing function for Project Admin but I am not able to find a way to do the same for Version Admin.
Can you please guide me on this.

[ericholscher]

You should be able to call https://github.com/rtfd/readthedocs.org/blob/master/readthedocs/search/signals.py#L44

[ericholscher]

Here is a bit of a snippet:

from datetime import datetime, timedelta
from readthedocs.search.documents import PageDocument
from readthedocs.builds.models import Version
from readthedocs.search.signals import index_html_file

days = 1
since = datetime.now() - timedelta(days=days)

vs = Version.objects.filter(builds__date__gte=since, active=True).distinct()
to_index = PageDocument().get_queryset().filter(version__in=vs)
print("Indexing %s" % len(to_index))
index_html_file(to_index)

[dojutsu-user]

@ericholscher
Can this issue waits for sometime? I am learning elasticsearch (completed one udemy course and reading the docs of elastic.... but it's not enough yet) and would love to try this again in a few days.
Also, I am looking forward to add "Auto-complete feature for search" in RTD.

[dojutsu-user]

Closing this.
#5343