Merge pull request #1 from pandas-dev/master

updates pandas

Author: datajanko

.coveragerc

@@ -1,24 +1,23 @@
-Contributing to pandas
-======================
+# Contributing to pandas
 
 Whether you are a novice or experienced software developer, all contributions and suggestions are welcome!
 
-Our main contribution docs can be found [here](https://github.com/pandas-dev/pandas/blob/master/doc/source/contributing.rst), but if you do not want to read it in its entirety, we will summarize the main ways in which you can contribute and point to relevant places in the docs for further information.
+Our main contributing guide can be found [in this repo](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst) or [on the website](https://pandas-docs.github.io/pandas-docs-travis/development/contributing.html). If you do not want to read it in its entirety, we will summarize the main ways in which you can contribute and point to relevant sections of that document for further information.
+
+## Getting Started
 
-Getting Started
----------------
 If you are looking to contribute to the *pandas* codebase, the best place to start is the [GitHub "issues" tab](https://github.com/pandas-dev/pandas/issues). This is also a great place for filing bug reports and making suggestions for ways in which we can improve the code and documentation.
 
-If you have additional questions, feel free to ask them on the [mailing list](https://groups.google.com/forum/?fromgroups#!forum/pydata) or on [Gitter](https://gitter.im/pydata/pandas). Further information can also be found in our [Getting Started](https://github.com/pandas-dev/pandas/blob/master/doc/source/contributing.rst#where-to-start) section of our main contribution doc.
+If you have additional questions, feel free to ask them on the [mailing list](https://groups.google.com/forum/?fromgroups#!forum/pydata) or on [Gitter](https://gitter.im/pydata/pandas). Further information can also be found in the "[Where to start?](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#where-to-start)" section.
+
+## Filing Issues
+
+If you notice a bug in the code or documentation, or have suggestions for how we can improve either, feel free to create an issue on the [GitHub "issues" tab](https://github.com/pandas-dev/pandas/issues) using [GitHub's "issue" form](https://github.com/pandas-dev/pandas/issues/new). The form contains some questions that will help us best address your issue. For more information regarding how to file issues against *pandas*, please refer to the "[Bug reports and enhancement requests](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#bug-reports-and-enhancement-requests)" section.
 
-Filing Issues
--------------
-If you notice a bug in the code or in docs or have suggestions for how we can improve either, feel free to create an issue on the [GitHub "issues" tab](https://github.com/pandas-dev/pandas/issues) using [GitHub's "issue" form](https://github.com/pandas-dev/pandas/issues/new). The form contains some questions that will help us best address your issue. For more information regarding how to file issues against *pandas*, please refer to the [Bug reports and enhancement requests](https://github.com/pandas-dev/pandas/blob/master/doc/source/contributing.rst#bug-reports-and-enhancement-requests) section of our main contribution doc.
+## Contributing to the Codebase
 
-Contributing to the Codebase
-----------------------------
-The code is hosted on [GitHub](https://www.github.com/pandas-dev/pandas), so you will need to use [Git](http://git-scm.com/) to clone the project and make changes to the codebase. Once you have obtained a copy of the code, you should create a development environment that is separate from your existing Python environment so that you can make and test changes without compromising your own work environment. For more information, please refer to our [Working with the code](https://github.com/pandas-dev/pandas/blob/master/doc/source/contributing.rst#working-with-the-code) section of our main contribution docs.
+The code is hosted on [GitHub](https://www.github.com/pandas-dev/pandas), so you will need to use [Git](http://git-scm.com/) to clone the project and make changes to the codebase. Once you have obtained a copy of the code, you should create a development environment that is separate from your existing Python environment so that you can make and test changes without compromising your own work environment. For more information, please refer to the "[Working with the code](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#working-with-the-code)" section.
 
-Before submitting your changes for review, make sure to check that your changes do not break any tests. You can find more information about our test suites can be found [here](https://github.com/pandas-dev/pandas/blob/master/doc/source/contributing.rst#test-driven-development-code-writing). We also have guidelines regarding coding style that will be enforced during testing. Details about coding style can be found [here](https://github.com/pandas-dev/pandas/blob/master/doc/source/contributing.rst#code-standards).
+Before submitting your changes for review, make sure to check that your changes do not break any tests. You can find more information about our test suites in the "[Test-driven development/code writing](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#test-driven-development-code-writing)" section. We also have guidelines regarding coding style that will be enforced during testing, which can be found in the "[Code standards](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#code-standards)" section.
 
-Once your changes are ready to be submitted, make sure to push your changes to GitHub before creating a pull request. Details about how to do that can be found in the [Contributing your changes to pandas](https://github.com/pandas-dev/pandas/blob/master/doc/source/contributing.rst#contributing-your-changes-to-pandas) section of our main contribution docs. We will review your changes, and you will most likely be asked to make additional changes before it is finally ready to merge. However, once it's ready, we will merge it, and you will have successfully contributed to the codebase!
+Once your changes are ready to be submitted, make sure to push your changes to GitHub before creating a pull request. Details about how to do that can be found in the "[Contributing your changes to pandas](https://github.com/pandas-dev/pandas/blob/master/doc/source/development/contributing.rst#contributing-your-changes-to-pandas)" section. We will review your changes, and you will most likely be asked to make additional changes before it is finally ready to merge. However, once it's ready, we will merge it, and you will have successfully contributed to the codebase!

.github/CONTRIBUTING.md

@@ -0,0 +1,2 @@
+custom: https://pandas.pydata.org/donate.html
+tidelift: pypi/pandas

.github/FUNDING.yml

@@ -1,4 +1,5 @@
 - [ ] closes #xxxx
 - [ ] tests added / passed
+- [ ] passes `black pandas`
 - [ ] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`
 - [ ] whatsnew entry

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1 @@
+To report a security vulnerability to pandas, please go to https://tidelift.com/security and see the instructions there.

.github/SECURITY.md

@@ -57,10 +57,19 @@ dist
 # wheel files
 *.whl
 **/wheelhouse/*
+pip-wheel-metadata
 # coverage
 .coverage
 coverage.xml
 coverage_html_report
+.mypy_cache
+*.pytest_cache
+# hypothesis test database
+.hypothesis/
+__pycache__
+# pytest-monkeytype
+monkeytype.sqlite3
+
 
 # OS generated files #
 ######################
@@ -88,8 +97,8 @@ scikits
 *.c
 *.cpp
 
-# Performance Testing #
-#######################
+# Unit / Performance Testing #
+##############################
 asv_bench/env/
 asv_bench/html/
 asv_bench/results/
@@ -98,13 +107,14 @@ asv_bench/pandas/
 # Documentation generated files #
 #################################
 doc/source/generated
+doc/source/user_guide/styled.xlsx
+doc/source/reference/api
 doc/source/_static
 doc/source/vbench
 doc/source/vbench.rst
 doc/source/index.rst
 doc/build/html/index.html
 # Windows specific leftover:
 doc/tmp.sv
-doc/source/styled.xlsx
-doc/source/templates/
 env/
+doc/source/savefig/

.gitignore

@@ -2,9 +2,3 @@
 
 scanner:
     diff_only: True  # If True, errors caused by only the patch are shown
-
-pycodestyle:
-    max-line-length: 79
-    ignore:  # Errors and warnings to ignore
-        - E731
-        - E402

.pep8speaks.yml

@@ -0,0 +1,17 @@
+repos:
+-   repo: https://github.com/python/black
+    rev: stable
+    hooks:
+    -   id: black
+        language_version: python3.7
+-   repo: https://gitlab.com/pycqa/flake8
+    rev: 3.7.7
+    hooks:
+    -   id: flake8
+        language: python_venv
+        additional_dependencies: [flake8-comprehensions]
+-   repo: https://github.com/pre-commit/mirrors-isort
+    rev: v4.3.20
+    hooks:
+    -   id: isort
+        language: python_venv

.pre-commit-config.yaml

@@ -1,6 +1,4 @@
-sudo: false
 language: python
-# Default Python version is usually 2.7
 python: 3.5
 
 # To turn off cached cython files and compiler cache
@@ -23,86 +21,36 @@ env:
 
 git:
     # for cloning
-    depth: 1000
+    depth: false
 
 matrix:
     fast_finish: true
     exclude:
       # Exclude the default Python 3.5 build
       - python: 3.5
+
     include:
-    - os: osx
-      language: generic
-      env:
-        - JOB="3.5_OSX" TEST_ARGS="--skip-slow --skip-network"
-    - dist: trusty
-      env:
-        - JOB="2.7_LOCALE" LOCALE_OVERRIDE="zh_CN.UTF-8" SLOW=true
-      addons:
-        apt:
-          packages:
-          - language-pack-zh-hans
-    - dist: trusty
-      env:
-        - JOB="2.7" TEST_ARGS="--skip-slow" LINT=true
-      addons:
-        apt:
-          packages:
-          - python-gtk2
-    # In allow_failures
-    - dist: trusty
-      env:
-        - JOB="3.5_CONDA_BUILD_TEST" TEST_ARGS="--skip-slow --skip-network" CONDA_BUILD_TEST=true
-    - dist: trusty
-      env:
-        - JOB="3.6" TEST_ARGS="--skip-slow --skip-network" PANDAS_TESTING_MODE="deprecate" CONDA_FORGE=true COVERAGE=true
-    # In allow_failures
     - dist: trusty
       env:
-        - JOB="2.7_SLOW" SLOW=true
-    # In allow_failures
-    - dist: trusty
-      env:
-        - JOB="3.6_PIP_BUILD_TEST" TEST_ARGS="--skip-slow" PIP_BUILD_TEST=true
-      addons:
-        apt:
-          packages:
-          - xsel
-    # In allow_failures
+        - JOB="3.7" ENV_FILE="ci/deps/travis-37.yaml" PATTERN="(not slow and not network)"
+
     - dist: trusty
       env:
-        - JOB="3.6_NUMPY_DEV" TEST_ARGS="--skip-slow --skip-network" PANDAS_TESTING_MODE="deprecate"
-    # In allow_failures
+        - JOB="3.6, locale" ENV_FILE="ci/deps/travis-36-locale.yaml" PATTERN="((not slow and not network) or (single and db))" LOCALE_OVERRIDE="zh_CN.UTF-8"
+
     - dist: trusty
       env:
-        - JOB="3.6_ASV" ASV=true
+        - JOB="3.6, coverage" ENV_FILE="ci/deps/travis-36-cov.yaml" PATTERN="((not slow and not network) or (single and db))" PANDAS_TESTING_MODE="deprecate" COVERAGE=true
+
     # In allow_failures
     - dist: trusty
       env:
-        - JOB="3.6_DOC" DOC=true
+        - JOB="3.6, slow" ENV_FILE="ci/deps/travis-36-slow.yaml" PATTERN="slow"
+
     allow_failures:
       - dist: trusty
         env:
-          - JOB="3.5_CONDA_BUILD_TEST" TEST_ARGS="--skip-slow --skip-network" CONDA_BUILD_TEST=true
-      - dist: trusty
-        env:
-          - JOB="2.7_SLOW" SLOW=true
-      - dist: trusty
-        env:
-          - JOB="3.6_PIP_BUILD_TEST" TEST_ARGS="--skip-slow" PIP_BUILD_TEST=true
-        addons:
-          apt:
-            packages:
-            - xsel
-      - dist: trusty
-        env:
-          - JOB="3.6_NUMPY_DEV" TEST_ARGS="--skip-slow --skip-network" PANDAS_TESTING_MODE="deprecate"
-      - dist: trusty
-        env:
-          - JOB="3.6_ASV" ASV=true
-      - dist: trusty
-        env:
-          - JOB="3.6_DOC" DOC=true
+          - JOB="3.6, slow" ENV_FILE="ci/deps/travis-36-slow.yaml" PATTERN="slow"
 
 before_install:
   - echo "before_install"
@@ -115,41 +63,36 @@ before_install:
   - pwd
   - uname -a
   - git --version
-  - git tag
+  - ./ci/check_git_tags.sh
+  # Because travis runs on Google Cloud and has a /etc/boto.cfg,
+  # it breaks moto import, see:
+  # https://github.com/spulec/moto/issues/1771
+  # https://github.com/boto/boto/issues/3741
+  # This overrides travis and tells it to look nowhere.
+  - export BOTO_CONFIG=/dev/null
 
 install:
   - echo "install start"
   - ci/prep_cython_cache.sh
-  - ci/install_travis.sh
+  - ci/setup_env.sh
   - ci/submit_cython_cache.sh
   - echo "install done"
 
 before_script:
-  - ci/install_db_travis.sh
+  # display server (for clipboard functionality) needs to be started here,
+  # does not work if done in install:setup_env.sh (GH-26103)
   - export DISPLAY=":99.0"
-  - ci/before_script_travis.sh
+  - echo "sh -e /etc/init.d/xvfb start"
+  - sh -e /etc/init.d/xvfb start
+  - sleep 3
 
 script:
   - echo "script start"
-  - ci/run_build_docs.sh
-  - ci/script_single.sh
-  - ci/script_multi.sh
-  - ci/lint.sh
-  - ci/asv.sh
-  - echo "checking imports"
-  - source activate pandas && python ci/check_imports.py
-  - echo "script done"
-
-after_success:
-  -  ci/upload_coverage.sh
+  - source activate pandas-dev
+  - ci/run_tests.sh
 
 after_script:
   - echo "after_script start"
-  - source activate pandas && pushd /tmp && python -c "import pandas; pandas.show_versions();" && popd
-  - if [ -e /tmp/single.xml ]; then
-    ci/print_skipped.py /tmp/single.xml;
-    fi
-  - if [ -e /tmp/multiple.xml ]; then
-    ci/print_skipped.py /tmp/multiple.xml;
-    fi
+  - source activate pandas-dev && pushd /tmp && python -c "import pandas; pandas.show_versions();" && popd
+  - ci/print_skipped.py 
   - echo "after_script done"

.travis.yml

@@ -0,0 +1,54 @@
+Copyright 2017- Paul Ganssle <[email protected]>
+Copyright 2017- dateutil contributors (see AUTHORS file)
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+
+       http://www.apache.org/licenses/LICENSE-2.0
+
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+
+The above license applies to all contributions after 2017-12-01, as well as
+all contributions that have been re-licensed (see AUTHORS file for the list of
+contributors who have re-licensed their code).
+--------------------------------------------------------------------------------
+dateutil - Extensions to the standard Python datetime module.
+
+Copyright (c) 2003-2011 - Gustavo Niemeyer <[email protected]>
+Copyright (c) 2012-2014 - Tomi Pieviläinen <[email protected]>
+Copyright (c) 2014-2016 - Yaron de Leeuw <[email protected]>
+Copyright (c) 2015-     - Paul Ganssle <[email protected]>
+Copyright (c) 2015-     - dateutil contributors (see AUTHORS file)
+
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+
+    * Redistributions of source code must retain the above copyright notice,
+      this list of conditions and the following disclaimer.
+    * Redistributions in binary form must reproduce the above copyright notice,
+      this list of conditions and the following disclaimer in the documentation
+      and/or other materials provided with the distribution.
+    * Neither the name of the copyright holder nor the names of its
+      contributors may be used to endorse or promote products derived from
+      this software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
+CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
+EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
+PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
+PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
+LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
+NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+The above BSD License Applies to all code, even that also covered by Apache 2.0.

LICENSES/DATEUTIL_LICENSE

@@ -0,0 +1,2 @@
+YEAR: 2013-2016
+COPYRIGHT HOLDER: Hadley Wickham; RStudio; and Evan Miller