Improve import project user experience #1695

agjohnson · 2015-09-25T00:41:54Z

This makes a number of changes to a fairly deficient UI. It still doesn't
provide an interface for importing multiple projects at once, but that could be
added easily once we have a public task for importing projects.

Frontend changes:

Combines import intermediate page, and github/bitbucket import pages into a single UI.
The new import page:
- Lists remote repos foremost
- Has a backup "Manual import" button
- Removes overly verbose interactions -- that is, don't explain the UI with words
- Makes import page into API driven interface
- Repos can filter by clicking an organization
- On no connected accounts, import page onboards user into connecting bitbucket/github
- Adds more graphical elements, like repo avatars
- Display errors on syncing problems
- Paginates

Backend changes:

Combine the two in the backend modeling
Combine update tasks
Tasks can return error messages
Shared model has common fields moved from JSON blob
No more eval, data is JSON serialized

Still needed:

Rename models, views, and fix migration. OAuth* is a stupid name
Handle messaging around admin access better

Also, note: CSS changes particular to RTD.org are in core.css, which general styling is done in the LESS source. This is because RTD.com should avoid this styling, as list styling is not compatible.

Fixes #1670

Adds filtering handling, better error reporting to all failure points, and redirects to a populated for to importing the project.

This finally starts combining the import project pages into a single page. This page has a list of remote repositories, CTA buttons to signup when you aren't, and a backup button to manual import.

This adds matches to the model and serializer

When the repo is private and we don't have admin access, don't automatically import the repo, as we can't set up webhooks/etc.

This intercepts the task after return to reapply the local task state, and include the error message which resulted as a retval to the task on exception.

ericholscher · 2015-09-28T16:26:13Z

readthedocs/projects/static-src/projects/js/import.js

@@ -0,0 +1,252 @@
+var ko = require('knockout'),
+    $ = require('jquery'),
+    tasks = require('../../../../core/static-src/core/js/tasks'),


This is super ugly. Is there not a better way to path this stuff?

Agreed, I hate this pattern as well. It's silly that browserify doesn't include the path it's running from by default, giving you addressable modules of readthedocs/core/js/whatever.js. I've patched the browserify instance to handle this and it's much prettier.

ericholscher · 2015-09-28T16:37:08Z

Looks good to me. I only had a couple small nits, and a couple questions/comments. 👍

agjohnson · 2015-09-28T17:02:17Z

readthedocs/oauth/models.py

+        # if projects:
+        #    repo.matches = [project.slug for project in projects]
+        # else:
+        #    repo.matches = []


@ericholscher just realized i meant to ask about this, and if we felt this extra lookup is important. I take it this was added to cover old repositories that were imported before github integration?

Yea, we don't have any standard of repo URL in the project, so they can have a number of formats. (eg. git://, https://, .git, no .git)

I could also be okay saying that if you didn't import via github, we don't disallow import again from github connected accounts. The most correct solution here is an intermediary table Project <-> RemoteRepo of course, which we'll have to make the same decision on when we go to run migrations there.

Yea, I think we need explicit mapping, and doing real tracking/resolution would require forcing GH auth and requiring that, which I'm not super excited to do.

I'd say we just poll whats already available in the RemoteRepository table when we do, don't worry about forcing auth or making sure it's up to date. The problem there will be of course how we handle syncing the repos and ensuring relations of Project <-> RemoteRepository stick.

I'll add the ghetto url stuff back in for now here

ericholscher · 2015-09-28T20:49:01Z

readthedocs/restapi/views/model_views.py

+    renderer_classes = [JSONRenderer, BrowsableAPIRenderer]
+    serializer_class = RemoteOrganizationSerializer
+    model = RemoteOrganization
+    queryset = RemoteOrganization.objects.all()


These querysets aren't filtering by user at all, which means they return every project for every user.

Also adds test cases for user filtering on repo/orgs

agjohnson · 2015-10-02T21:05:30Z

Okay, addressed the incorrect template issue and filtering in the last few commits. I believe that should wrap up work on this for now. There is definitely some room for improvement on filtering -- I see the next version of this having a fuzzy search bar.

Improve import project user experience

agjohnson added the PR: work in progress Pull request is not ready for full review label Sep 25, 2015

agjohnson added 29 commits September 24, 2015 18:23

Allow public repositories to show when default privacy is private

790cf91

Somewhat working example of API driven list

383812e

Add OAuth combination models

f7cf808

Add data migration for OAuth models

89fa008

Add missing constants

6174067

Redo html layout and add styling to import page

6141c79

Project service import filtering, error reporting, and form redirect

71f06b8

Adds filtering handling, better error reporting to all failure points, and redirects to a populated for to importing the project.

Style updates to project import page

38fece2

Add styling for buttons using icons

5a868ea

Move projectimport to projects app

61da2fb

Don't bold icons

618f531

Import path on project import.js

7632b80

Repo icon on project import

1274409

Add combination github and bitbucket sync API endpoints

1449e2b

Update import HTML and move loader gif to static collect source

63eed28

Remove Github/Bitbucket oauth models and related views

878cb69

Drop github/bitbucket migration

10acc6f

Use a constant for project import providers

bd2bc7e

Tune remote import template

64806e8

Combine import project forms into a single interface

3072372

This finally starts combining the import project pages into a single page. This page has a list of remote repositories, CTA buttons to signup when you aren't, and a backup button to manual import.

Drop unused templates for import, repoint view template

ba641c8

Manual import button changes

4f94890

Fix import fields on oauth models

bf49fe2

Find matches on oauth repo and display link to project instead

2ae3156

This adds matches to the model and serializer

View icon on repo list, and turn buttons into links

fdf35c7

Class cleanup

52fa9ca

Adds a locked designation for repositories

6c02f0a

When the repo is private and we don't have admin access, don't automatically import the repo, as we can't set up webhooks/etc.

Add exception to public task return, allowing for failures

406e04d

This intercepts the task after return to reapply the local task state, and include the error message which resulted as a retval to the task on exception.

Improve github/bitbucket sync error responses a little bit

817c5cc

agjohnson added PR: ready for review and removed PR: work in progress Pull request is not ready for full review labels Sep 26, 2015

ericholscher reviewed Sep 28, 2015
View reviewed changes

agjohnson reviewed Sep 28, 2015
View reviewed changes

agjohnson added 3 commits September 28, 2015 10:03

Update javascript include paths in gulpfile

988860b

Cleanup on project import js

38a0045

Drop redundant model view queryset

dbb495c

agjohnson mentioned this pull request Sep 28, 2015

Audit MEDIA_PATH assets, use static asset tag instead #1705

Closed

Add fuzzy url search to remote repos back

fa48743

ericholscher reviewed Sep 28, 2015
View reviewed changes

agjohnson added 4 commits October 2, 2015 12:07

Fix issue with fuzzy query

e90b501

Add privacy changes for remote repos/orgs

ef2ca84

Also adds test cases for user filtering on repo/orgs

Fix filter on user query and api tests

8fba74a

Add account context data on correct view

c9f6023

agjohnson mentioned this pull request Oct 2, 2015

Unable to import renamed GitHub repository #1723

Closed

Add default scopes for GH into base settings

e3f4c62

ericholscher added a commit that referenced this pull request Oct 5, 2015

Merge pull request #1695 from rtfd/import-improve-ux

17fef85

Improve import project user experience

ericholscher merged commit 17fef85 into master Oct 5, 2015

jaraco mentioned this pull request Aug 30, 2016

Paginated import UI is unusably slow #2392

Closed

stsewd deleted the import-improve-ux branch August 15, 2018 22:21

takluyver mentioned this pull request May 20, 2020

Better experience when many projects are available to import readthedocs/ext-theme#13

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve import project user experience #1695

Improve import project user experience #1695

agjohnson commented Sep 25, 2015

ericholscher Sep 28, 2015

agjohnson Sep 28, 2015

ericholscher commented Sep 28, 2015

agjohnson Sep 28, 2015

ericholscher Sep 28, 2015

agjohnson Sep 28, 2015

ericholscher Sep 28, 2015

agjohnson Sep 28, 2015

ericholscher Sep 28, 2015

agjohnson commented Oct 2, 2015

Improve import project user experience #1695

Improve import project user experience #1695

Conversation

agjohnson commented Sep 25, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ericholscher commented Sep 28, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agjohnson commented Oct 2, 2015