Skip to content

Alternate approach to enhancing Python 3 support #2918

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 49 commits into from
Closed
Show file tree
Hide file tree
Changes from 42 commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
3b0c307
Add Python 3.6 to tox.ini
gthank May 30, 2017
cf7c021
Update version of slumber.
gthank May 30, 2017
f2f0a88
Remove dependency on Distutils2; it's obsolete.
gthank May 30, 2017
ee016b2
Upgrade dnspython dependency.
gthank May 30, 2017
1ea4274
Update haystack(ish) dependencies.
gthank May 30, 2017
e26b059
Update to latest django-celery.
gthank May 30, 2017
be9f1ed
Upgrade to newer pytest-django for better Python3.
gthank May 30, 2017
09150cc
Upgrade django-allauth to public release containing change from pinne…
gthank May 30, 2017
96c2f6d
Use modern Unipath.
gthank May 30, 2017
ff6813b
Use release of django-cors-middleware w/ signal support.
gthank May 30, 2017
eee0978
Explicitly depend on six, rather than waiting for a dependency to bri…
gthank May 30, 2017
f201d15
Make a pass w/ python-modernize.
gthank May 30, 2017
64116bc
Make a pass with futurize on stage 2.
gthank May 30, 2017
633877b
Apply Eric's Unicode version-parsing fix.
gthank May 30, 2017
975b7e4
Guarantee StringIO is fed Unicode.
gthank May 30, 2017
87af24f
Apply Eric's fix for Unicode in auth.
gthank May 30, 2017
1c22682
Clean up bytes/unicode distinction when mocking results of subprocess…
gthank May 31, 2017
496b9c9
Explicitly mock readthedocs.doc_builder.environments.api_v2.command a…
gthank May 31, 2017
7fa4239
Use string.ascii_lowercase instead of string.lowercase when uniquifying.
gthank May 31, 2017
5ca33e6
Remove stray trailing comma in imports; that's an error in Python 3.
gthank May 31, 2017
806b065
Use assertTrue instead of assertTrueok_
gthank May 31, 2017
a4a1bab
Ignore tags file from exuberant ctags.
gthank May 31, 2017
0bd9294
unittest.assertItemsEqual went away in Python 3, use six.assertCountE…
gthank May 31, 2017
0b70237
Undo a .next() -> next() from futurize; it was an unfortunate name co…
gthank May 31, 2017
cbcdbcb
Don't try to use BaseException.message
gthank May 31, 2017
2401401
Fix file pathing error caused by differences between 2 and 3.
gthank May 31, 2017
959cebd
Fix bytes/str in test_version_commit_name, and be explicit.
gthank Jun 1, 2017
448e1d4
Use assertContains so Django will do the hard work of Unicode/bytes r…
gthank Jun 1, 2017
77ffbd8
Explicitly wrap forms paramter from django-formtools in a list.
gthank Jun 1, 2017
76fe20e
Temporarily rely on latest master of readthedocs-build
gthank Jun 1, 2017
fc8eccd
Fix a couple more references to BaseException.message
gthank Jun 1, 2017
d18a19a
six.string_type -> six.binary_type because we're trying to invoke .de…
gthank Jun 1, 2017
136c59a
Use @python_2_unicode_compatible to handle __str__/__unicode__ on Dja…
gthank Jun 1, 2017
1a75d68
Fix more BaseException.message shenanigans.
gthank Jun 1, 2017
36b83da
Coerce cmd_input to bytes where necessary before feeding to proc.comm…
gthank Jun 1, 2017
988c21c
Mock return values from direct subprocess interaction as bytes explic…
gthank Jun 1, 2017
04c2566
Enable Python 3.6 in Travis.
gthank Jun 1, 2017
f351f85
Tell Travis we want python 3.6 too
gthank Jun 1, 2017
76e1117
Make Python 2.7 and Python 3.6 separate Travis builds.
gthank Jun 1, 2017
e46be85
More Travis-CI tweakery.
gthank Jun 1, 2017
0d73ebe
Fix wonky syntax.
gthank Jun 1, 2017
1d055ef
Try to simplify the tox/travis interactions.
gthank Jun 5, 2017
6e8027b
Clean up some import order issues.
gthank Jun 8, 2017
f972a55
Parse the URL using the right name.
gthank Jun 8, 2017
717da9a
Pin prospector to previous minor release
agjohnson Jun 8, 2017
ea3096f
Flag three tests as known failues under Py3.
gthank Jun 9, 2017
dd6b557
Remove unnecessary 'cast' to unicode string.
gthank Jun 9, 2017
027f734
Make version-parsing code use only ASCII chars, in accordance with PE…
gthank Jun 9, 2017
3d5191b
Merge branch 'master' into master
gthank Jun 14, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -39,3 +39,4 @@ public_*
private_*
.rope_project/
readthedocs/htmlcov
tags
3 changes: 2 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
language: python
python:
- 2.7
- 3.6
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe TOX handles the Python env, so having this both places means we run all of the tests 2x. I think we can remove this.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or, we can remove the 2.7, and run them all in 3.6 -- I see travis fails with "no 3.6 found" when running under 2.7.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I added 3.6 to get around the weird error in 2.7, but that exploded the build matrix, so I tried to add the tox-travis plugin to make the integration smarter, but never got it properly configured. If 2.7 is working under 3.6, I think the right play is to pull out the tox-travis plugin and just tell travis to use 3.6

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps default to 2.7 for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason, Travis fails and says it can't find the Python 3.6 interpreter when we do it on 2.7. The 2.7 tests seem to work fine when we tell Travis to use 3.6. Apparently just some quirk with the Travis and tox interactions.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, strange. Perhaps this is another motivator for switching to our newer standard tox.ini/travis.yml config, like: https://github.com/rtfd/sphinx_rtd_theme

If this is working as expected, probably no strong reason to switch just yet though

sudo: false
env:
- TOX_ENV=py27
- TOX_ENV=py36
- TOX_ENV=docs
- TOX_ENV=lint
- TOX_ENV=eslint
Expand Down
4 changes: 3 additions & 1 deletion readthedocs/api/base.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
"""API resources"""
from __future__ import absolute_import
from builtins import object
import logging
import json
import redis
Expand Down Expand Up @@ -95,7 +97,7 @@ def sync_versions(self, request, **kwargs):
except Exception as e:
return self.create_response(
request,
{'exception': e.message},
{'exception': str(e)},
response_class=HttpApplicationError,
)
return self.create_response(request, deleted_versions)
Expand Down
1 change: 1 addition & 0 deletions readthedocs/api/client.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Slumber API client"""
from __future__ import absolute_import
import logging

from slumber import API
Expand Down
2 changes: 2 additions & 0 deletions readthedocs/api/utils.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
"""Utility classes for api module"""
from __future__ import absolute_import
from builtins import object
import logging

from django.core.paginator import Paginator, InvalidPage
Expand Down
1 change: 1 addition & 0 deletions readthedocs/bookmarks/admin.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Django admin interface for `~bookmarks.models.Bookmark`."""

from __future__ import absolute_import
from django.contrib import admin
from readthedocs.bookmarks.models import Bookmark

Expand Down
1 change: 1 addition & 0 deletions readthedocs/bookmarks/migrations/0001_initial.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from __future__ import absolute_import
from django.db import models, migrations
from django.conf import settings

Expand Down
8 changes: 6 additions & 2 deletions readthedocs/bookmarks/models.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,17 @@
"""Models for the bookmarks app."""

from __future__ import absolute_import
from builtins import object
from django.db import models
from django.contrib.auth.models import User
from django.utils.encoding import python_2_unicode_compatible
from django.utils.translation import ugettext_lazy as _, ugettext

from readthedocs.builds.models import Version
from readthedocs.projects.models import Project


@python_2_unicode_compatible
class Bookmark(models.Model):

"""A user's bookmark of a ``Project``, ``Version``, and page."""
Expand All @@ -23,11 +27,11 @@ class Bookmark(models.Model):
date = models.DateTimeField(_('Date'), auto_now_add=True)
url = models.CharField(_('URL'), max_length=255, null=True, blank=True)

class Meta:
class Meta(object):
ordering = ['-date']
unique_together = ('user', 'project', 'version', 'page')

def __unicode__(self):
def __str__(self):
return ugettext(u"Bookmark %(url)s for %(user)s (%(pk)s)") % {
'url': self.url,
'user': self.user,
Expand Down
1 change: 1 addition & 0 deletions readthedocs/bookmarks/urls.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""URL config for the bookmarks app."""

from __future__ import absolute_import
from django.conf.urls import url
from readthedocs.bookmarks.views import BookmarkListView
from readthedocs.bookmarks.views import BookmarkAddView, BookmarkRemoveView
Expand Down
1 change: 1 addition & 0 deletions readthedocs/bookmarks/views.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Views for the bookmarks app."""

from __future__ import absolute_import
from django.contrib.auth.decorators import login_required
from django.http import HttpResponse, HttpResponseRedirect
from django.http import HttpResponseBadRequest
Expand Down
1 change: 1 addition & 0 deletions readthedocs/builds/admin.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Django admin interface for `~builds.models.Build` and related models."""

from __future__ import absolute_import
from django.contrib import admin
from readthedocs.builds.models import Build, VersionAlias, Version, BuildCommandResult
from guardian.admin import GuardedModelAdmin
Expand Down
1 change: 1 addition & 0 deletions readthedocs/builds/constants.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Constants for the builds app."""

from __future__ import absolute_import
from django.utils.translation import ugettext_lazy as _

BUILD_STATE_TRIGGERED = 'triggered'
Expand Down
6 changes: 4 additions & 2 deletions readthedocs/builds/forms.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
"""Django forms for the builds app."""

from __future__ import absolute_import
from builtins import object
from django import forms

from readthedocs.builds.models import VersionAlias, Version
Expand All @@ -9,7 +11,7 @@

class AliasForm(forms.ModelForm):

class Meta:
class Meta(object):
model = VersionAlias
fields = (
'project',
Expand All @@ -27,7 +29,7 @@ def __init__(self, instance=None, *args, **kwargs):

class VersionForm(forms.ModelForm):

class Meta:
class Meta(object):
model = Version
fields = ['active', 'privacy_level', 'tags']

Expand Down
1 change: 1 addition & 0 deletions readthedocs/builds/migrations/0001_initial.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from __future__ import absolute_import
from django.db import models, migrations
import readthedocs.builds.version_slug
import taggit.managers
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from __future__ import absolute_import
from django.db import models, migrations
import readthedocs.builds.models

Expand Down
21 changes: 14 additions & 7 deletions readthedocs/builds/models.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
"""Models for the builds app."""

from __future__ import absolute_import
from builtins import object
import logging
import re
import os.path
Expand All @@ -8,6 +10,7 @@
from django.core.urlresolvers import reverse
from django.conf import settings
from django.db import models
from django.utils.encoding import python_2_unicode_compatible
from django.utils.translation import ugettext_lazy as _, ugettext

from guardian.shortcuts import assign
Expand All @@ -33,6 +36,7 @@
log = logging.getLogger(__name__)


@python_2_unicode_compatible
class Version(models.Model):

"""Version of a ``Project``."""
Expand Down Expand Up @@ -77,7 +81,7 @@ class Version(models.Model):

objects = VersionManager.from_queryset(VersionQuerySet)()

class Meta:
class Meta(object):
unique_together = [('project', 'slug')]
ordering = ['-verbose_name']
permissions = (
Expand All @@ -86,7 +90,7 @@ class Meta:
('view_version', _('View Version')),
)

def __unicode__(self):
def __str__(self):
return ugettext(u"Version %(version)s of %(project)s (%(pk)s)" % {
'version': self.verbose_name,
'project': self.project,
Expand Down Expand Up @@ -299,6 +303,7 @@ def get_bitbucket_url(self, docroot, filename, source_suffix='.rst'):
)


@python_2_unicode_compatible
class VersionAlias(models.Model):

"""Alias for a ``Version``."""
Expand All @@ -310,14 +315,15 @@ class VersionAlias(models.Model):
blank=True)
largest = models.BooleanField(_('Largest'), default=False)

def __unicode__(self):
def __str__(self):
return ugettext(u"Alias for %(project)s: %(from)s -> %(to)s" % {
'project': self.project,
'from': self.from_slug,
'to': self.to_slug,
})


@python_2_unicode_compatible
class Build(models.Model):

"""Build data."""
Expand Down Expand Up @@ -348,14 +354,14 @@ class Build(models.Model):

objects = BuildQuerySet.as_manager()

class Meta:
class Meta(object):
ordering = ['-date']
get_latest_by = 'date'
index_together = [
['version', 'state', 'type']
]

def __unicode__(self):
def __str__(self):
return ugettext(u"Build %(project)s for %(usernames)s (%(pk)s)" % {
'project': self.project,
'usernames': ' '.join(self.project.users.all()
Expand Down Expand Up @@ -395,6 +401,7 @@ def failed(self):
return not self.successful


@python_2_unicode_compatible
class BuildCommandResult(BuildCommandResultMixin, models.Model):

"""Build command for a ``Build``."""
Expand All @@ -410,13 +417,13 @@ class BuildCommandResult(BuildCommandResultMixin, models.Model):
start_time = models.DateTimeField(_('Start time'))
end_time = models.DateTimeField(_('End time'))

class Meta:
class Meta(object):
ordering = ['start_time']
get_latest_by = 'start_time'

objects = RelatedBuildQuerySet.as_manager()

def __unicode__(self):
def __str__(self):
return (ugettext(u'Build command {pk} for build {build}')
.format(pk=self.pk, build=self.build))

Expand Down
1 change: 1 addition & 0 deletions readthedocs/builds/signals.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Build signals"""

from __future__ import absolute_import
import django.dispatch


Expand Down
1 change: 1 addition & 0 deletions readthedocs/builds/urls.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""URL configuration for builds app."""

from __future__ import absolute_import
from django.conf.urls import url

from .views import builds_redirect_detail, builds_redirect_list
Expand Down
1 change: 1 addition & 0 deletions readthedocs/builds/utils.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Utilities for the builds app."""

from __future__ import absolute_import
import re


Expand Down
7 changes: 5 additions & 2 deletions readthedocs/builds/version_slug.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,12 +16,15 @@
another number would be confusing.
"""

from __future__ import absolute_import
from builtins import range
import math
import re
import string
from operator import truediv
from django.db import models
from django.utils.encoding import force_text
from six.moves import range


# Regex breakdown:
Expand Down Expand Up @@ -89,15 +92,15 @@ def uniquifying_suffix(self, iteration):
uniquifying_suffix(26) == '_ba'
uniquifying_suffix(52) == '_ca'
"""
alphabet = string.lowercase
alphabet = string.ascii_lowercase
length = len(alphabet)
if iteration == 0:
power = 0
else:
power = int(math.log(iteration, length))
current = iteration
suffix = ''
for exp in reversed(range(0, power + 1)):
for exp in reversed(list(range(0, power + 1))):
digit = int(truediv(current, length ** exp))
suffix += alphabet[digit]
current = current % length ** exp
Expand Down
2 changes: 2 additions & 0 deletions readthedocs/builds/views.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
"""Views for builds app."""

from __future__ import absolute_import
from builtins import object
import logging

from django.shortcuts import get_object_or_404
Expand Down
5 changes: 4 additions & 1 deletion readthedocs/cdn/purge.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
"""Utility to purge MaxCDN files, if configured."""

from __future__ import absolute_import
from builtins import range
import logging

from django.conf import settings
from six.moves import range

log = logging.getLogger(__name__)

Expand All @@ -14,7 +17,7 @@

def chunks(in_list, chunk_size):
"""Yield successive n-sized chunks from l."""
for i in xrange(0, len(in_list), chunk_size):
for i in range(0, len(in_list), chunk_size):
yield in_list[i:i + chunk_size]

if CDN_USERNAME and CDN_KEY and CDN_SECRET and CDN_SERVICE == 'maxcdn':
Expand Down
1 change: 1 addition & 0 deletions readthedocs/comments/admin.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""ModelAdmin configurations for comments app."""

from __future__ import absolute_import
from django.contrib import admin
from .models import DocumentNode, DocumentComment, NodeSnapshot

Expand Down
1 change: 1 addition & 0 deletions readthedocs/comments/backend.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Storage backends for the comments app."""

from __future__ import absolute_import
import json

from django.core import serializers
Expand Down
1 change: 1 addition & 0 deletions readthedocs/comments/migrations/0001_initial.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# -*- coding: utf-8 -*-
from __future__ import unicode_literals

from __future__ import absolute_import
from django.db import models, migrations
from django.conf import settings

Expand Down
Loading