Skip to content

Re-work around design document for APIv3 #5845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jun 26, 2019
Merged

Re-work around design document for APIv3 #5845

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
48e2eda
Re-work around design document for APIv3
humitos Jun 25, 2019
a6e929a
Improve title
humitos Jun 26, 2019
cd8c163
Add table of contents
humitos Jun 26, 2019
02ce6ec
Remove open questions
humitos Jun 26, 2019
cb6c87a
Explain more Out of roadmap
humitos Jun 26, 2019
71d07cf
Expand introduction
humitos Jun 26, 2019
250ee1f
Better Goals and Non-Goals
humitos Jun 26, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
248 changes: 106 additions & 142 deletions docs/development/design/apiv3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,204 +1,168 @@
Design of APIv3
===============
=======================
APIv3 Design Document
=======================

This document describes the design,
some decisions already made and built (current Version 1 of APIv3)
and an implementation plan for next Versions of APIv3.

APIv3 will be designed to be easy to use and useful to perform read and write operations as the main two goals.

It will be based on Resources as APIv2 but considering the ``Project`` resource as the main one,
from where most of the endpoint will be based on it.

.. contents::
:local:
:backlinks: none
:depth: 1

Problems with APIv2
-------------------


* No authentication
* It's read-only
* Not designed for slugs
* Useful APIs not exposed (only for internal usage currently)
* Error reporting is a mess
* Relationships between API resources is not obvious
* Footer API endpoint returns HTML


Proposed improves
+++++++++++++++++


Use authentication
~~~~~~~~~~~~~~~~~~

* Pros:

* queries can be personalized depending on the user
* allows us to perform write actions

* Cons:

* harder to implement
* requires a lot of time for good testing and QA


Questions:

#. Do we want make auth a requirement?
#. Should we expose some endpoints as Read Only if not authenticated?
#. How we do communicate users about deprecation if they are Anonymous? Is it enough to add a note in the docs saying that "Auth is preferred for communication"?
#. How are we going to expose the ``Token`` required for Auth when building if auth is mandatory?


Read and Write
~~~~~~~~~~~~~~

* Pros:
Goals
-----

* most of the actions can be performed by a script instead of by
accessing readthedocs.org with a browser
* Easy to use for our users (access most of resources by ``slug``)
* Useful to perform read and write operations
* Authentication/Authorization

* Cons:
* Authentication based on scoped-tokens
* Handle Authorization nicely using an abstraction layer

* we have to deal with authorization
* open the door to possibilities of security holes
* Cover most useful cases:

* Integration on CI (check build status, trigger new build, etc)
* Usage from public Sphinx/MkDocs extensions
* Allow creation of flyout menu client-side
* Simplify migration from other services (import projects, create multiple redirects, etc)

Questions:

#. Do we want to allow the user to perform **all the write actions**
that are possible to do from the dashboard via the API?


Design it for slugs
~~~~~~~~~~~~~~~~~~~

* Pros:

* knowing the slug of your project (which is presented to the user)
you can perform all the actions or retrieve all the data

* Cons:

* it will be a mixture between most of the endpoint using ``slug``
and to retrieve details of a Build it will be an ``id``


Expose internal endpoints
~~~~~~~~~~~~~~~~~~~~~~~~~

There are some endpoints that we are using internally like
``/api/v2/search/``, ``/api/v2/footer/`` and
``/api/v2/sustainability/``.


* Pros:

* allow to build custom footer
* allow to build custom search autocomplete widget

* Cons:
Non-Goals
---------

* n/a
* Filter by arbitrary and non-useful fields
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Again, this isn't clear, we need to define this more.


Questions:
* "Builds with ``exit_code=1``"
* "Builds containing ``ERROR`` on their output"
* "Projects created after X datetime"
* "Versions with tag ``python``"

#. Do we want to add ``/api/v2/sustainability/`` to APIv3 or that
should be part of the new Ad Server that we are building?
* Cover *all the actions* available from the WebUI


Proper status codes for error reporting
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Problems with APIv2
-------------------

* Pros:
There are several problem with our current APIv2 that we can list:

* user knows what it's happening and can take decisions about it
* No authentication
* It's read-only
* Not designed for slugs
* Useful APIs not exposed (only for internal usage currently)
* Error reporting is a mess
* Relationships between API resources is not obvious
* Footer API endpoint returns HTML

* Cons:

* n/a
Implementation stages
---------------------

Version 1
+++++++++

Relationship between API resources
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The first implementation of APIv3 will cover the following aspects:

* Pros:
.. note::

* browse-able API by accessing to the default resource (Project)
* knowing the project's slug you can perform all the actions related to it
This is currently implemented and live. Although, it's only for internal testing.

* Cons:
* Authentication

* more data is returned with all the links to the endpoints
* all endpoints require authentication via ``Authorization:`` request header
* detail endpoints are available for all authenticated users
* only Project's maintainers can access listing endpoints
* personalized listing

* Read and Write

Make footer API returns JSON
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* edit attributes from Version (only ``active`` and ``privacy_level``)
* trigger Build for a specific Version

* Pros:
* Accessible by slug

* users can build their own version menu flyout
* cleaner than returning a HTML and injecting it
* Projects are accessed by ``slug``
* Versions are accessed by ``slug``
* ``/projects/`` endpoint is the main one and all of the other are nested under it
* Builds are accessed by ``id``, as exception to this rule
* access all (active/non-active) Versions of a Project by ``slug``
* get latest Build for a Project (and Version) by ``slug``
* filter by relevant fields

* Cons:
* Proper status codes to report errors

* we need to adapt our theme for this
* Browse-able endpoints

* browse is allowed hitting ``/api/v3/projects/`` as starting point
* ability to navigate clicking on other resources under ``_links`` attribute

Questions:
* Rate limited

#. Do we want an specific endpoint for the footer?
#. The flyout could be built by querying 2 or 3 of the regular
endpoint. Would this add too much overhead to the page loading
time?

Version 2
+++++++++

Use cases
---------
Second iteration will polish issues found from the first step,
and add new endpoints to allow *import a project and configure it*
without the needed of using the WebUI as a main goal.

We want to cover
++++++++++++++++
After Version 2 is deployed,
we will invite users that reach us as beta testers to receive more feedback
and continue improving it by supporting more use cases.

This iteration will include:

* Return all **my** projects
* Access all the resources by knowing the project's slug
* Ability to filter by fields
* Minor changes to fields returned in the objects
* Import Project endpoint
* Edit Project attributes ("Settings" and "Advanced settings-Global settings" in the WebUI)
* Trigger Build for default version
* Allow CRUD for Redirect, Environment Variables and Notifications (``WebHook`` and ``EmailHook``)
* Documentation

* all the active versions of specific project

* Data about builds
Version 3
+++++++++

* latest build for project
* latest build for a particular version of a project
* status of a particular build
Third iteration will implement granular permissions.
Keeping in mind how Sphinx extension will use it:

* Perform write actions like
* ``sphinx-version-warning`` needs to get *all active Versions of a Project*
* An extension that creates a landing page, will need *all the subprojects of a Project*

* add a Domain,
* add User as mantainer,
* import a new Project under my username,
* set the language of the Project,
* trigger a Build,
* activate/deactivate a Version to be built,
* and all the actions you can perform from the Admin tab.
To fulfill these requirements, this iteration will include:

* Retrieve all the information needed to create a custom version menu flyout
* Scope-based authorization token


Considering some useful cases for the corporate site:
Version 4
+++++++++

* Give access to a doc page (``objects.inv``, ``/design/core.html``)
* Specific endpoint for our flyout menu (returning JSON instead of HTML)


We do NOT want to cover
+++++++++++++++++++++++
Out of roadmap
--------------

* Random filtering over a whole and not useful Resource
These are some features that we may want to build at some point.
Although, they are currently out of our near roadmap because they don't affect too many users,
or are for internal usage only.

* "All the ``stable`` versions"
* "Builds with ``exit_code`` equal to 257"
* CRUD for Domain
* Add User as maintainer
* Give access to a documentation page (``objects.inv``, ``/design/core.html``)
* Internal Build process
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should mention why all of these are out of the roadmap.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put them out of the roadmap for now because, in case we build them, we need to decide in which Version they will be assigned. Also, they do not affect too many users or are only for internal usage only.

I'm updating the paragraph mentioning this.



Technical aspects that would be good to have
--------------------------------------------
Nice to have
------------

* Rate limit
* ``Request-ID`` header
* `JSON minified by default`_ (maybe with ``?pretty=true``)
* `JSON schema and validation`_ with docs_
Expand Down