PR (#1)

* iostream: check that stream is open before trying to read (tornadoweb#2670)

* curl_httpclient: fix disabled decompress_response

by setting None (NULL) instead of "none" for ENCODING

reported by Andrey Oparin <[email protected]>

* tests: run test_gzip for curl_httpclient also

move simple_httpclient test_gzip to the shared httpclient tests,
to test the decompress_response option for curl_httpclient as well

* mypy: Enable no_implicit_optional

"Implicit-optional" mode is on by default, but that default is intended to change in the indefinite future (python/peps#689, python/typing#275). Go ahead and change to the future explicit use of Optional.

* gen.with_timeout: Don't log CancelledError after timeout

See also: commit a237a99

* Fix ReST syntax

* locks: Remove redundant CancelledError handling

CancelledError is now always considered "quiet" (and
concurrent.futures.CancelledError is no longer the same as
asyncio.CancelledError).

* ci: Re-enable nightly python

Fixes tornadoweb#2677

* gen: Clean up docs for with_timeout

Mark CancelledError change as 6.0.3

* *: Modernize IO error handling

Where possible, replace use of errno with the exception hierarchy
available since python 3.3. Remove explicit handling of EINTR which
has been automatic since python 3.5

* netutil: Ignore EADDRNOTAVAIL when binding to localhost ipv6

This happens in docker with default configurations and is generally
harmless.

Fixes tornadoweb#2274

* test: Skip test_source_port_fail when running as root

Root is always allowed to bind to low port numbers, so we can't
simulate failure in this case. This is the last remaining failure when
running tests in docker.

* docs: Add notice about WindowsSelectorEventLoop on py38

Fixes tornadoweb#2608

* Bump version of twisted to pick up security fix

* Release notes for 6.0.3

* SSLIOStream: Handle CertificateErrors like other errors

Fixes: tornadoweb#2689

* Update database backend reference

* Strip documentation about removed argument

* Mark Template autoescape kwarg as Optional

* httputil: cache header normalization with @lru_cache instead of hand-rolling

Tornado is now py3-only so @lru_cache is always available.

Performance is about the same. Benchmark below. Python 3.7 on Linux.

before, cached:   0.9121252089971676
before, uncached: 13.358482279989403

after, cached:    0.9175888689933345
after, uncached:  11.085199063003529

```py
from time import perf_counter

names = [f'sOMe-RanDOM-hEAdeR-{i}' for i in range(1000)]

from tornado.httputil import _normalize_header
start = perf_counter()
for i in range(10000):
    # _normalize_header.cache_clear()
    for name in names:
        _normalize_header(name)
print(perf_counter() - start)

from tornado.httputil import _NormalizedHeaderCache
start = perf_counter()
_normalized_headers = _NormalizedHeaderCache(1000)
for i in range(10000):
    # _normalized_headers = _NormalizedHeaderCache(1000)
    for name in names:
        _normalized_headers[name]
print(perf_counter() - start)
```

* httputil: use compiled re patterns

This is slightly faster than using the builtin cache, e.g.:

With benchmark below (Python 3.7, Linux):

before: 0.7284867879934609
after:  0.2657967659761198

```py
import re
from time import perf_counter

line = 'HTTP/1.1'

_http_version_re = re.compile(r"^HTTP/1\.[0-9]$")

start = perf_counter()
for i in range(1000000):
    _http_version_re.match(line)
print(perf_counter() - start)

start = perf_counter()
for i in range(1000000):
    re.match(r"^HTTP/1\.[0-9]$", line)
print(perf_counter() - start)
```

* test: Disable TLS 1.3 in one test

This test started failing on windows CI with an upgrade to python
3.7.4 (which bundles a newer version of openssl). Disable tls 1.3 for
now.

Possibly related to tornadoweb#2536

* spelling corrections

* maintainance -> maintenance

* recieving -> receiving

* tests: replace remaining assertEquals() with assertEqual()

assertEquals() is deprecated, and python3.7/pytest can warn about it

* httputil.parse_body_arguments: allow incomplete url-escaping

support x-www-form-urlencoded body with values consisting of
encoded bytes which are not url-encoded into ascii
(it seems other web frameworks often support this)

add bytes qs support to escape.parse_qs_bytes,
leave str qs support for backwards compatibility

* Clear fewer headers on 1xx/204/304 responses

This function is called on more than just 304 responses; it’s
important to permit the Allow header on 204 responses.  Also, the
relevant RFCs have changed significantly.

Fixes tornadoweb#2726.

Signed-off-by: Anders Kaseorg <[email protected]>

* Fix extra data sending at HEAD response with Transfer-Encoding: Chunked

* Omit Transfer-Encoding header for HEAD response

* Add test for unescaping with groups

* Fix unescaping of regex routes

Previously, only the part before the first '(' would be correctly
unescaped.

* Use HTTPS link for tornado website.

* Simplify chained comparison.

* build(deps): bump twisted from 19.2.1 to 19.7.0 in /maint

Bumps [twisted](https://github.com/twisted/twisted) from 19.2.1 to 19.7.0.
- [Release notes](https://github.com/twisted/twisted/releases)
- [Changelog](https://github.com/twisted/twisted/blob/trunk/NEWS.rst)
- [Commits](twisted/twisted@twisted-19.2.1...twisted-19.7.0)

Signed-off-by: dependabot[bot] <[email protected]>

* Dead link handling

Added an extra set for handling dead links, and reporting.

One consequence of this is that using this script will "work" offline, but will report that some all the links were not fetched.

* ci: Pin version of black

A new release of black changed the way some of our files are formatted,
so use a fixed version in CI.

* demos: Fix lint in webspider demo

Updates tornadoweb#2765

* build: Revamp test/CI configuration

Reduce tox matrix to one env per python version, with two extra builds
for lint and docs. Delegate to tox from travis-ci.

Add 3.8 to testing. Simplify by dropping coverage reporting and
"no-deps" test runs.

* process: correct docs of fork_processes exit behavior

fixes tornadoweb#2771

* Remove legacy Python support in speedups.c

* ci: Don't run full test suite on python 3.5.2

* web: Update hashing algorithm in StaticFileHandler (tornadoweb#2778)

Addresses tornadoweb#2776.

* build: Run docs and lint on py38

This requires moving some noqa comments due to 3.8's changes to the
ast module.

* lint: Upgrade to new version of black

* lint: Use newer mypy

This required some minor code changes, mainly some adjustments in tests
(which are now analyzed more thoroughly in spite of being mostly
unannotated), and some changes to placement of type:ignore comments.

* use bcrypt's checkpw instead of ==

* Fix case of JavaScript, GitHub and CSS.

* Fix syntax error in nested routing example

* test: Add gitattributes for test data files

This ensures that the tests pass on Windows regardless of the user's git
CRLF settings.

* test: Use selector event loop on windows.

This gets most of the tests working again on windows with py38.

* test: Add some more skips on windows

Alternate resolvers behave differently on this platform for unknown
reasons.

* test: Add hasattr check for SIGCHLD

This name is not present on all platforms

* testing: Add level argument to ExpectLog

This makes it possible for tests to be a little more precise, and also
makes them less dependent on exactly how the test is run (runtests.py
sets the logging level to info, but when running tests directly from
an editor it may use the default of warnings-only).

CI only runs the tests with runtests.py, so this might regress, but I'm
not building anything to prevent that yet (options include running the
tests differently in CI or making ExpectLog always use a fixed log
configuration instead of picking up the current one)

* ci: Add python 3.8 to windows CI

* asyncio: AnyThreadEventLoopPolicy should always use selectors on windows

* iostream: resolve reads that may be completed while closing

fixes issue that a read may fail with StreamClosedError
if stream is closed mid-read

* avoid premature _check_closed in _start_read

_start_read can resolve with _try_inline_read, which can succeed even if the stream has been closed
if the buffer has been populated by a prior read

preserve the fix for asserts being hit when dealing with closed sockets

* catch UnsatisfiableReadError in close

* iostream: Add tests for behavior around close with read_until

Updates tornadoweb#2719

* iostream: Expand comments around recent subtle changes

* Fix Google OAuth example (from 6.0 OAuth2Mixin->authorize_redirect is an ordinary synchronous function)

* Add Python 3.8 clasifier to setup.py

* Standardize type documentation for HTTPRequest init

* travis-ci.com doesn't like it when you have matrix and jobs

.org still allows this for some reason

* Master branch release notes for version 6.0.4

* maint: Bump bleach version for a security fix

* iostream: Update comment

Update comment from tornadoweb#2690 about ssl module exceptions.

* Added default User-Agent to the simple http client if not provided.

The User-Agent format is "Tornado\{Tornado_Version}".

If self.request.user_agent isn't set and self.request.headers has
no User-Agent in it's keys the default User-Agent is added.

Fixes: tornadoweb#2702

* Revert "docs: Use python 3.7 via conda for readthedocs builds"

This reverts commit e7e31e5.

We were using conda to get access to python 3.7 before rtd supported
it in their regular builds, but this led to problems pinning a
specific version of sphinx. See
readthedocs/readthedocs.org#6870

* fix new E741 detected cases

* fix typos

* revert genericize change

* stop ping_callback

* fix types for max_age_days and expires_days parameters

* test: Add a sleep to deflake a test

Not sure why this has recently started happening in some environments,
but killing a process too soon causes the wrong exit status in some
python builds on macOS.

* ci: Drop tox-venv

Its README says it is mostly obsolete due to improvements in
virtualenv. Using it appears to cause problems related to
pypa/setuptools#1934 because virtualenv
installs the wheel package by default but venv doesn't.

* ci: Allow failures on nightly python due to cffi incompatibility

* template: Clarify docs on escaping

Originally from tornadoweb#2831, which went to the wrong branch.

* test: Use default timeouts in sigchild test

The 1s timeout used here has become flaky with the introduction of a
sleep (before the timeout even starts).

* auth: Fix example code

Continuation of tornadoweb#2811

The oauth2 version of authorize_redirect is no longer a coroutine, so
don't use await in example code. The oauth1 version is still a
coroutine, but one twitter example was incorrectly calling it with
yield instead of await.

* platform: Remove tornado.platform.auto.set_close_exec

This function is obsolete: Since python 3.4, file descriptors created
by python are non-inheritable by default (and in the event you create
a file descriptor another way, a standard function os.set_inheritable
is available).

The windows implementation of this function was also apparently
broken, but this went unnoticed because the default behavior on
windows is for file descriptors to be non-inheritable.

Fixes tornadoweb#2867

* iostream,platform: Remove _set_nonblocking function

This functionality is now provided directly in the `os` module.

* test: Use larger time values in testing_test

This test was flaky on appveyor. Also expand comments about what
exactly the test is doing.

* Remove text about callback (removed) in run_on_executor

* curl_httpclient: set CURLOPT_PROXY to NULL if pycurl supports it

This restores curl's default behaviour: use environment variables.

This option was set to "" to disable proxy in
905a215 but curl uses environment
variables by default.

* httpclient_test: Improve error reporting

Without this try/finally, if this test ever fails, errors can be
reported in a confusing way.

* iostream_test: Improve cleanup

Closing the file descriptor without removing the corresponding handler
is technically incorrect, although the default IOLoops don't have a
problem with it.

* test: Add missing level to ExpectLog call

* asyncio: Improve support Python 3.8 on Windows

This commit removes the need for applications to work around the
backwards-incompatible change to the default event loop. Instead,
Tornado will detect the use of the windows proactor event loop and
start a selector event loop in a separate thread.

Closes tornadoweb#2804

* asyncio: Rework AddThreadSelectorEventLoop

Running a whole event loop on the other thread leads to tricky
synchronization problems. Instead, keep as much as possible on the
main thread, and call out to a second thread only for the blocking
select system call itself.

* test: Add an option to disable assertion that logs are empty

Use this on windows due to a log spam issue in asyncio.

* asyncio: Refactor selector to callbacks instead of coroutine

Restarting the event loop to "cleanly" shut down a coroutine introduces
other problems (mainly manifesting as errors logged while running
tornado.test.gen_test). Replace the coroutine with a pair of callbacks
so we don't need to do anything special to shut down without logging
warnings.

* docs: Pin version of sphinxcontrib-asyncio

The just-released version 0.3.0 is incompatible with our older pinned
version of sphinx.

* docs: Pin version of sphinxcontrib-asyncio

The just-released version 0.3.0 is incompatible with our older pinned
version of sphinx.

* Added arm64 jobs for Travis-CI

* CLN : Remove utf-8 coding cookies in source files

On Python 3, utf-8 is the default python source code encoding. so, the
coding cookies on files that specify utf-8 are not needed anymore.

	modified:   tornado/_locale_data.py
	modified:   tornado/locale.py
	modified:   tornado/test/curl_httpclient_test.py
	modified:   tornado/test/httpclient_test.py
	modified:   tornado/test/httputil_test.py
	modified:   tornado/test/options_test.py
	modified:   tornado/test/util_test.py

* Allow non-yielding functions in `tornado.gen.coroutine`'s type hint (tornadoweb#2909)

`@gen.coroutine` deco allows non-yielding functions, so I reflected that in the type hint.

Requires usage of `@typing.overload` due to python/mypy#9435

* Update super usage (tornadoweb#2912)

On Python 3, super does not need to be called with arguments where as on
Python 2, super needs to be called with a class object and an instance.

This commit updates the super usage using automated regex-based search
and replace. After the automated changes were made, each change was
individually checked before committing.

* Update links on home page

* Updated http links to the https versions when possible.
* Updated links to Google Groups to match their new URL format.
* Updated links to other projects to match their new locations.
* And finally, updated link to FriendFeed to go to the Wikipedia page, because friendfeed.com is just a redirect to facebook.com now :-( :-(

* Modified ".travis.yml" to test it's own built wheel

Signed-off-by: odidev <[email protected]>

* tests: httpclient may turn all methods into GET for 303 redirect

* websocket_test: test websocket_connect redirect raises exception

instead of "uncaught exception" and then test timeout

* websocket: set follow_redirects to False

to prevent silent failure when the websocket client gets a 3xx
redirect response, because it does not currently support redirects

Partial fix for issue tornadoweb#2405

* simple_httpclient: after 303 redirect, turn all methods into GET

not just POST (but still not HEAD)

following the behavior of libcurl > 7.70

* httpclient_test: add test for connect_timeout=0 request_timeout=0

* simple_httpclient: handle connect_timeout or request_timeout of 0

Using a connect_timeout or request_timeout of 0 was effectively
invalid for simple_httpclient: it would skip the actual request
entirely (because the bulk of the logic was inside "if timeout:").
This was not checked for or raised as an error, it just behaved
unexpectedly.

Change simple_httpclient to always assert these timeouts are not None
and to support the 0 value similar to curl (where request_timeout=0
means no timeout, and connect_timeout=0 means curl default 300 seconds
which is very very long for a tcp connection).

* httpclient: document connect_timeout/request_timeout 0 value

not exactly true for curl_httpclient (libcurl uses a connect_timeout
of 300 seconds if no connect timeout is set) but close enough

* test: update Travis-CI matrix pypy version to 3.6-7.3.1

* httpclient_test: new test for invalid gzip Content-Encoding

this caused an infinite loop in simple_httpclient

* http: fix infinite loop hang with invalid gzip data

* test: Refactor CI configuration

- Add osx and windows builds on travis
- Stop running -full test suites on every python version on arm64
- Use cibuildwheel to build for all python versions in one job per
  platform
- Bring a single test configuration and linters up to a first "quick"
  stage before starting the whole matrix
- Push the resulting wheels (and sdist) to pypi on tag builds

* Add release notes for 6.1, bump version to 6.1b1

* ci: Switch from testpypi to real pypi

* Add deprecation notice for Python 3.5

* Update how to register application with Google

* Fix await vs yield in the example

* gen: Expliclty track contextvars, fixing contextvars.reset

The asyncio event loop provides enough contextvars support out of the
box for basic contextvars functionality to work in tornado coroutines,
but not `contextvars.reset`. Prior to this change, each yield created
a new "level" of context, when an entire coroutine should be on the
same level. This is necessary for the reset method to work.

Fixes tornadoweb#2731

* test: Add a timeout to SyncHTTPClient test

* asyncio: Manage our own thread instead of an executor

Python 3.9 changed the behavior of ThreadPoolExecutor at interpreter
shutdown (after the already-tricky import-order issues around
atexit hooks). Avoid these issues by managing the thread by hand.

* ci,setup: Add python 3.9 to tox, cibuildwheel and setup.py

* Bump version to 6.1b2

* Set version to 6.1 final

* ci: Work around outdated windows root certificates

* Bump main branch to 6.2.dev1

* Remove appveyor configs

* Drop support for python 3.5

* iostream: Add platform assertion for mypy

Without this mypy would fail when run on windows.

* maint: Prune requirements lists

Remove dependencies that are rarely used outside of tox. The main
motivation is to give dependabot less to worry about when an indirect
dependency has a security vulnerability.

* *: Update black to newest version

* Update mypy to latest version

* docs: Upgrade to latest version of sphinx

This version attempts to resolve types found in type annotations,
but in many cases it can't find them so silence a bunch of warnings.
(Looks like deferred annotation processing will make this better but
we won't be able to use that until we drop Python 3.6)

* docs: Pin specific versions of requirements

* docs: Stop using autodoc for t.p.twisted

This way we don't have to install twisted into the docs build
environment. Add some more detail while I'm here.

* platform: Deprecate twisted and cares resolvers

These were most interesting when the default resolver blocked
the main thread. Now that the default is to use a thread pool,
there is little if any demand for alternative resolvers just to
avoid threads.

* Issue tornadoweb#2954: prevent logging error messages for not existing translation files

Every not existing translation file for the existing locales logged an error message:
Cannot load translation for 'ps': [Errno 2] No such file or directory: '/usr/share/locale/ps/LC_MESSAGES/foo.mo'

* WaitIterator: don't re-use _running_future

When used with asyncio.Future, WaitIterator may skip indices in some
cases. This is caused by multiple _return_result calls after another,
without having the chain_future call finish in between. This is fixed
here by not hanging on to the _running_future anymore, which forces
subsequent _return_result calls to add to _finished, instead of causing
the previous result to be silently dropped.

Fixes tornadoweb#2034

* Fix return type of _return_result

* docs: fix simple typo, authentiate -> authenticate

There is a small typo in tornado/netutil.py.

Should read `authenticate` rather than `authentiate`.

* Avoid 2GB read limitation on SSLIOStream

* Remove trailing whitespace

* locale: Format with black

* wsgi: Update docstring example for python 3

Fixes tornadoweb#2960

* Remove WebSocketHandler.stream.

It was no longer used and always set to None.

* Add 'address' keyword control binded address tornadoweb#2969

* format code according to result of flake8 check

* Add comment explaining workaround

* change comment

* should use python3 unicode in 'blog' demo tornadoweb#2977

* leave previous versionchanged

* leave previous versionchanged

* write_message method of WebSocketClientConnection now accepts dict as input

* write_message method of WebSocketClientConnection now accepts dict as input

* Uppercase A in Any

* BaseIOStream.write(): support typed memoryview

Making sure that ``len(data) == data.nbytes`` by casting
memoryviews to bytes.

* Allowed set max_body_size to 0

* fix line too long

* fix E127

* what

* But this is not beautiful

* Is this okay

* build(deps): bump jinja2 from 2.11.2 to 2.11.3 in /docs

Bumps [jinja2](https://github.com/pallets/jinja) from 2.11.2 to 2.11.3.
- [Release notes](https://github.com/pallets/jinja/releases)
- [Changelog](https://github.com/pallets/jinja/blob/master/CHANGES.rst)
- [Commits](pallets/jinja@2.11.2...2.11.3)

Signed-off-by: dependabot[bot] <[email protected]>

* build(deps): bump pygments from 2.7.2 to 2.7.4 in /docs

Bumps [pygments](https://github.com/pygments/pygments) from 2.7.2 to 2.7.4.
- [Release notes](https://github.com/pygments/pygments/releases)
- [Changelog](https://github.com/pygments/pygments/blob/master/CHANGES)
- [Commits](pygments/pygments@2.7.2...2.7.4)

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: Ben Darnell <[email protected]>
Co-authored-by: Zachary Sailer <[email protected]>
Co-authored-by: Pierce Lopez <[email protected]>
Co-authored-by: Robin Roth <[email protected]>
Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Martijn van Oosterhout <[email protected]>
Co-authored-by: Michael V. DePalatis <[email protected]>
Co-authored-by: Remi Rampin <[email protected]>
Co-authored-by: Ran Benita <[email protected]>
Co-authored-by: Semen Zhydenko <[email protected]>
Co-authored-by: Anders Kaseorg <[email protected]>
Co-authored-by: Bulat Khasanov <[email protected]>
Co-authored-by: supakeen <[email protected]>
Co-authored-by: John Bampton <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Jeff van Santen <[email protected]>
Co-authored-by: Bruno P. Kinoshita <[email protected]>
Co-authored-by: Gareth T <[email protected]>
Co-authored-by: Min RK <[email protected]>
Co-authored-by: bn0ir <[email protected]>
Co-authored-by: James Bourbeau <[email protected]>
Co-authored-by: Recursing <[email protected]>
Co-authored-by: Flavio Garcia <[email protected]>
Co-authored-by: Ben Darnell <[email protected]>
Co-authored-by: marc <Marc>
Co-authored-by: agnewee <[email protected]>
Co-authored-by: Jeff Hunter <[email protected]>
Co-authored-by: 依云 <[email protected]>
Co-authored-by: odidev <[email protected]>
Co-authored-by: Sai Rahul Poruri <[email protected]>
Co-authored-by: jack1142 <[email protected]>
Co-authored-by: Amit Patel <[email protected]>
Co-authored-by: Debby <[email protected]>
Co-authored-by: = <=>
Co-authored-by: Eugene Toder <[email protected]>
Co-authored-by: Florian Best <[email protected]>
Co-authored-by: Alexander Clausen <[email protected]>
Co-authored-by: Tim Gates <[email protected]>
Co-authored-by: bfis <[email protected]>
Co-authored-by: Eugene Toder <[email protected]>
Co-authored-by: youguanxinqing <[email protected]>
Co-authored-by: kriskros341 <[email protected]>
Co-authored-by: Mads R. B. Kristensen <[email protected]>
Co-authored-by: Sakuya <[email protected]>

Author: 41 people

.flake8

@@ -10,4 +10,8 @@ ignore =
     E402,
     # E722 do not use bare except
     E722,
+    # flake8 and black disagree about
+    # W503 line break before binary operator
+    # E203 whitespace before ':'
+    W503,E203
 doctests = true

.gitattributes

@@ -0,0 +1,4 @@
+# Tests of static file handling assume unix-style line endings.
+tornado/test/static/*.txt text eol=lf
+tornado/test/static/dir/*.html text eol=lf
+tornado/test/templates/*.html text eol=lf

.travis.yml

@@ -1,89 +1,101 @@
 # https://travis-ci.org/tornadoweb/tornado
-dist: trusty
 
-# Use containers instead of full VMs for faster startup.
-sudo: false
+os: linux
+dist: xenial
+language: python
+addons:
+  apt:
+    packages:
+      - libgnutls-dev
 
-matrix:
-    fast_finish: true
+env:
+  global:
+    - CIBW_BUILD="cp3[6789]*"
+    - CIBW_TEST_COMMAND="python3 -m tornado.test"
+    - CIBW_TEST_COMMAND_WINDOWS="python -m tornado.test --fail-if-logs=false"
 
-language: python
-# For a list of available versions, run
-#     aws s3 ls s3://travis-python-archives/binaries/ubuntu/14.04/x86_64/
-python:
-    - 2.7
-    - pypy2.7-5.8.0
-    - 3.4
-    - 3.5
-    - 3.6
-    - nightly
-    - pypy3.5-5.8.0
+# Before starting the full build matrix, run one test configuration
+# and the linter (the `black` linter is especially likely to catch
+# first-time contributors).
+stages:
+  - quick
+  - test
+
+jobs:
+  fast_finish: true
+  include:
+    # We have two and a half types of builds: Wheel builds run on all supported
+    # platforms and run the basic test suite for all supported python versions.
+    # Sdist builds (the "and a half") just build an sdist and run some basic
+    # validation. Both of these upload their artifacts to pypi if there is a
+    # tag on the build and the key is available.
+    #
+    # Tox builds run a more comprehensive test suite with more configurations
+    # and dependencies (getting all these dependencies installed for wheel
+    # builds is a pain, and slows things down because we don't use as much
+    # parallelism there. We could parallelize wheel builds more but we're also
+    # amortizing download costs across the different builds). The wheel builds
+    # take longer, so we run them before the tox builds for better bin packing
+    # in our allotted concurrency.
+    - python: '3.8'
+      arch: amd64
+      services: docker
+      env: BUILD_WHEEL=1
+    - python: '3.8'
+      arch: arm64
+      services: docker
+      env: BUILD_WHEEL=1  ASYNC_TEST_TIMEOUT=15
+    - os: windows
+      env: PATH=/c/Python38:/c/Python38/Scripts:$PATH BUILD_WHEEL=1
+      language: shell
+      before_install:
+        - choco install python --version 3.8.0
+        # Windows build images have outdated root certificates; until that's
+        # fixed use certifi instead.
+        # https://github.com/joerick/cibuildwheel/issues/452
+        - python -m pip install certifi
+        - export SSL_CERT_FILE=`python -c "import certifi; print(certifi.where())"`
+    - os: osx
+      env: BUILD_WHEEL=1
+      language: shell
+
+    - python: '3.8'
+      arch: amd64
+      env: BUILD_SDIST=1
+
+    - python: '3.6'
+      env: TOX_ENV=py36-full
+    - python: '3.7'
+      env: TOX_ENV=py37-full
+    - python: '3.8'
+      env: TOX_ENV=py38-full
+    - python: '3.9-dev'
+      env: TOX_ENV=py39-full
+    - python: nightly
+      env: TOX_ENV=py3
+    - python: pypy3.6-7.3.1
+      # Pypy is a lot slower due to jit warmup costs, so don't run the "full"
+      # test config there.
+      env: TOX_ENV=pypy3
+    # Docs and lint python versions must be synced with those in tox.ini
+    - python: '3.8'
+      env: TOX_ENV=docs
+
+    # the quick stage runs first, but putting this at the end lets us take
+    # advantage of travis-ci's defaults and not repeat stage:test in the others.
+    - python: '3.8'
+      env: TOX_ENV=py38,lint
+      stage: quick
 
 install:
-    # On nightly, upgrade setuptools first to work around
-    # https://github.com/pypa/setuptools/issues/1257
-    - if [[ $TRAVIS_PYTHON_VERSION == 'nightly' ]]; then travis_retry pip install -U setuptools; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == 2* ]]; then travis_retry pip install mock monotonic; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == 'pypy' ]]; then travis_retry pip install mock; fi
-    # TODO(bdarnell): pycares tests are currently disabled on travis due to ipv6 issues.
-    #- if [[ $TRAVIS_PYTHON_VERSION != 'pypy'* ]]; then travis_retry pip install pycares; fi
-    - if [[ $TRAVIS_PYTHON_VERSION != 'pypy'* ]]; then travis_retry pip install pycurl; fi
-    # Twisted runs on 2.x and 3.3+, but is flaky on pypy.
-    - if [[ $TRAVIS_PYTHON_VERSION != 'pypy'* ]]; then travis_retry pip install Twisted; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == '2.7' || $TRAVIS_PYTHON_VERSION == '3.6' ]]; then travis_retry pip install sphinx sphinx_rtd_theme; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == '3.6' ]]; then travis_retry pip install flake8; fi
-    # On travis the extension should always be built
-    - if [[ $TRAVIS_PYTHON_VERSION != 'pypy'* ]]; then export TORNADO_EXTENSION=1; fi
-    - travis_retry python setup.py install
-    - travis_retry pip install codecov virtualenv
-    # Create a separate no-dependencies virtualenv to make sure all imports
-    # of optional-dependencies are guarded.
-    - virtualenv ./nodeps
-    - ./nodeps/bin/python -VV
-    - ./nodeps/bin/python setup.py install
-    - curl-config --version; pip freeze
+  - if [[ -n "$TOX_ENV" ]]; then pip3 install tox; fi
+  - if [[ -n "$BUILD_WHEEL" ]]; then pip3 install cibuildwheel; fi
+  - if [[ -n "$BUILD_WHEEL" || -n "$BUILD_SDIST" ]]; then pip3 install twine; fi
 
 script:
-    # Run the tests once from the source directory to detect issues
-    # involving relative __file__ paths; see
-    # https://github.com/tornadoweb/tornado/issues/1780
-    - unset TORNADO_EXTENSION && python -m tornado.test
-    # For all other test variants, get out of the source directory before
-    # running tests to ensure that we get the installed speedups module
-    # instead of the source directory which doesn't have it.
-    - cd maint
-    # Copy the coveragerc down so coverage.py can find it.
-    - cp ../.coveragerc .
-    - if [[ $TRAVIS_PYTHON_VERSION != 'pypy'* ]]; then export TORNADO_EXTENSION=1; fi
-    - export TARGET="-m tornado.test.runtests"
-    # Travis workers are often overloaded and cause our tests to exceed
-    # the default timeout of 5s.
-    - export ASYNC_TEST_TIMEOUT=15
-    # We use "python -m coverage" instead of the "bin/coverage" script
-    # so we can pass additional arguments to python.
-    # coverage needs a function that was removed in python 3.6 so we can't
-    # run it with nightly cpython. Coverage is very slow on pypy.
-    - if [[ $TRAVIS_PYTHON_VERSION != nightly && $TRAVIS_PYTHON_VERSION != 'pypy'* ]]; then export RUN_COVERAGE=1; fi
-    - if [[ "$RUN_COVERAGE" == 1 ]]; then export TARGET="-m coverage run $TARGET"; fi
-    - python $TARGET
-    - if [[ $TRAVIS_PYTHON_VERSION == 2* ]]; then python $TARGET --ioloop=tornado.platform.select.SelectIOLoop; fi
-    - python -O $TARGET
-    - LANG=C python $TARGET
-    - LANG=en_US.utf-8 python $TARGET
-    - if [[ $TRAVIS_PYTHON_VERSION == 3* ]]; then python -bb $TARGET; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == 2* ]]; then python $TARGET --httpclient=tornado.curl_httpclient.CurlAsyncHTTPClient; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == 2* ]]; then python $TARGET --ioloop=tornado.platform.twisted.TwistedIOLoop; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == 2* ]]; then python $TARGET --resolver=tornado.platform.twisted.TwistedResolver; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == 2* ]]; then python $TARGET --ioloop=tornado.ioloop.PollIOLoop --ioloop_time_monotonic; fi
-    #- if [[ $TRAVIS_PYTHON_VERSION != pypy* ]]; then python $TARGET --resolver=tornado.platform.caresresolver.CaresResolver; fi
-    - if [[ $TRAVIS_PYTHON_VERSION != 'pypy3' ]]; then ../nodeps/bin/python -m tornado.test.runtests; fi
-    # make coverage reports for Codecov to find
-    - if [[ "$RUN_COVERAGE" == 1 ]]; then coverage xml; fi
-    - export TORNADO_EXTENSION=0
-    - if [[ $TRAVIS_PYTHON_VERSION == 3.6 ]]; then cd ../docs && mkdir sphinx-out && sphinx-build -E -n -W -b html . sphinx-out; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == '2.7' || $TRAVIS_PYTHON_VERSION == 3.6 ]]; then cd ../docs && mkdir sphinx-doctest-out && sphinx-build -E -n -b doctest . sphinx-out; fi
-    - if [[ $TRAVIS_PYTHON_VERSION == '3.6' ]]; then flake8; fi
+  - if [[ -n "$TOX_ENV" ]]; then tox -e $TOX_ENV -- $TOX_ARGS; fi
+  - if [[ -n "$BUILD_WHEEL" ]]; then cibuildwheel --output-dir dist && ls -l dist; fi
+  - if [[ -n "$BUILD_SDIST" ]]; then python setup.py check sdist && ls -l dist; fi
 
 after_success:
-    # call codecov from project root
-    - if [[ "$RUN_COVERAGE" == 1 ]]; then cd ../ && codecov; fi
+  - if [[ ( -n "$BUILD_WHEEL" || -n "$BUILD_SDIST" ) && -n "$TRAVIS_TAG" && -n "$TWINE_PASSWORD" ]]; then twine upload -u __token__ dist/*; fi

MANIFEST.in

@@ -1,6 +1,7 @@
 recursive-include demos *.py *.yaml *.html *.css *.js *.xml *.sql README
 recursive-include docs *
 prune docs/build
+include tornado/py.typed
 include tornado/speedups.c
 include tornado/test/README
 include tornado/test/csv_translations/fr_FR.csv

README.rst

@@ -45,4 +45,4 @@ Documentation
 -------------
 
 Documentation and links to additional resources are available at
-http://www.tornadoweb.org
+https://www.tornadoweb.org