Skip to content
This repository was archived by the owner on Apr 8, 2025. It is now read-only.

Add dumping of domain data #48

Closed
wants to merge 3 commits into from
Closed

Add dumping of domain data #48

wants to merge 3 commits into from

Conversation

ericholscher
Copy link
Member

@ericholscher ericholscher commented Aug 17, 2018

This will let us index more interesting data about Sphinx domains in our search.

This will output something like this in the JSON:

    "objects": {
        "/api/v2/version/": [
            "http",
            "get",
            "",
            "get--api-v2-version-"
        ],
        "/api/v2/build/(int:id)/": [
            "http",
            "get",
            "",
            "get--api-v2-build-(int-id)-"
        ],
        "/api/v2/project/(int:id)/active_versions/": [
            "http",
            "get",
            "",
            "get--api-v2-project-(int-id)-active_versions-"
        ],
        "/api/v2/version/(int:id)/": [
            "http",
            "get",
            "",
            "get--api-v2-version-(int-id)-"
        ],
        "/api/v2/project/": [
            "http",
            "get",
            "",
            "get--api-v2-project-"
        ],
        "/api/v2/build/": [
            "http",
            "get",
            "",
            "get--api-v2-build-"
        ],
        "/api/v2/project/(int:id)/": [
            "http",
            "get",
            "",
            "get--api-v2-project-(int-id)-"
        ]
    },

Copy link
Member

@stsewd stsewd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, I don't understand this code to the %100, I read http://www.sphinx-doc.org/en/stable/extdev/domainapi.html to have a little more of context p:

Is the example the full output of https://github.com/rtfd/readthedocs.org/blob/master/docs/api/v2.rst?

@@ -148,6 +150,25 @@ def rtd_render(self, template, render_context):
app.builder.templates)


def geneate_search_objects(app, env):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Little typo geneate -> generate

@@ -148,6 +150,25 @@ def rtd_render(self, template, render_context):
app.builder.templates)


def geneate_search_objects(app, env):
domain_objects = {}
for domainname, domain in sorted(app.env.domains.items()):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what's the reason to sort the data before? At the end, we save all this in a not-sorted dict

@safwanrahman
Copy link
Member

safwanrahman commented Aug 26, 2018

I have build documents of readthedocs using the built in JSONHTMLBuilder. It seems like the searchindex.fjson file may already have all the objects we need, but I am confused about the data its providing.

"objects":{
      "":{
         "/api/v1/user/{id}/":[
            7,
            0,
            1,
            "get--api-v1-user-id-"
         ]
      }""
}

https://gist.github.com/safwanrahman/b7151e8fe47ceab145c700995e075cf6

@safwanrahman
Copy link
Member

So I have type of data by reading the source code . Its in following order
index of docnames, key of objtypes, priority, shortanchor (if any).

we already get this file in our build. I think we can parse the information from there and build our index. what do you think @ericholscher ?

@ericholscher
Copy link
Member Author

@safwanrahman We can perhaps just use the existing searchindex.json that is produced in the HTML build. I imagine we might want to output more data in the future, but if we can use that, it's fine.

@ericholscher
Copy link
Member Author

Closing this, as I think all the data we need is already in objects.inv for this PR. So it's already on disk 👍

@stsewd stsewd deleted the get-domain-data branch October 23, 2018 15:02
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants