Update Contributing Environment section

ci/environment-dev.yaml

@@ -0,0 +1,14 @@
+name: pandas-dev
+channels:
+  - defaults
+  - conda-forge
+dependencies:
+  - Cython
+  - NumPy
+  - moto
+  - pytest
+  - python-dateutil
+  - python=3
+  - pytz
+  - setuptools
+  - sphinx

ci/requirements_all.txt → ci/requirements-optional-conda.txt

@@ -1,28 +1,27 @@
-pytest>=3.1.0
-pytest-cov
-pytest-xdist
-flake8
-sphinx=1.5*
-nbsphinx
-ipython
-python-dateutil
-pytz
-openpyxl
-xlsxwriter
-xlrd
-xlwt
-html5lib
-patsy
 beautifulsoup4
-numpy
-cython
-scipy
+blosc
+bottleneck
+fastparquet
+feather-format
+html5lib
+ipython
+ipykernel
+jinja2
+lxml
+matplotlib
+nbsphinx
 numexpr
+openpyxl
+pyarrow
+pymysql
 pytables
-matplotlib
+pytest-cov
+pytest-xdist
+s3fs
+scipy
 seaborn
-lxml
 sqlalchemy
-bottleneck
-pymysql
-Jinja2
+xarray
+xlrd
+xlsxwriter
+xlwt

ci/requirements-optional-pip.txt

@@ -0,0 +1,27 @@
+# This file was autogenerated by scripts/convert_deps.py
+# Do not modify directlybeautifulsoup4
+blosc
+bottleneck
+fastparquet
+feather-format
+html5lib
+ipython
+jinja2
+lxml
+matplotlib
+nbsphinx
+numexpr
+openpyxl
+pyarrow
+pymysql
+tables
+pytest-cov
+pytest-xdist
+s3fs
+scipy
+seaborn
+sqlalchemy
+xarray
+xlrd
+xlsxwriter
+xlwt

ci/requirements_dev.txt

@@ -1,8 +1,10 @@
+# This file was autogenerated by scripts/convert_deps.py
+# Do not modify directly
+Cython
+NumPy
+moto
+pytest
 python-dateutil
 pytz
-numpy
-cython
-pytest>=3.1.0
-pytest-cov
-flake8
-moto
+setuptools
+sphinx

doc/source/contributing.rst

@@ -103,39 +103,11 @@ want to clone your fork to your machine::
 
     git clone https://github.com/your-user-name/pandas.git pandas-yourname
     cd pandas-yourname
-    git remote add upstream git://github.com/pandas-dev/pandas.git
+    git remote add upstream https://github.com/pandas-dev/pandas.git
 
 This creates the directory `pandas-yourname` and connects your repository to
 the upstream (main project) *pandas* repository.
 
-Creating a branch
------------------
-
-You want your master branch to reflect only production-ready code, so create a
-feature branch for making your changes. For example::
-
-    git branch shiny-new-feature
-    git checkout shiny-new-feature
-
-The above can be simplified to::
-
-    git checkout -b shiny-new-feature
-
-This changes your working directory to the shiny-new-feature branch.  Keep any
-changes in this branch specific to one bug or feature so it is clear
-what the branch brings to *pandas*. You can have many shiny-new-features
-and switch in between them using the git checkout command.
-
-To update this branch, you need to retrieve the changes from the master branch::
-
-    git fetch upstream
-    git rebase upstream/master
-
-This will replay your commits on top of the latest pandas git master.  If this
-leads to merge conflicts, you must resolve these before submitting your pull
-request.  If you have uncommitted changes, you will need to ``stash`` them prior
-to updating.  This will effectively store your changes and they can be reapplied
-after updating.
 
 .. _contributing.dev_env:
 
@@ -145,60 +117,53 @@ Creating a development environment
 An easy way to create a *pandas* development environment is as follows.
 
 - Install either :ref:`Anaconda <install.anaconda>` or :ref:`miniconda <install.miniconda>`
+- Make sure your conda is up to date (`conda update conda`)
 - Make sure that you have :ref:`cloned the repository <contributing.forking>`
 - ``cd`` to the *pandas* source directory
 
-Tell conda to create a new environment, named ``pandas_dev``, or any other name you would like
-for this environment, by running::
-
-      conda create -n pandas_dev --file ci/requirements_dev.txt
-
+We'll now kick off a three-step process:
 
-For a python 3 environment::
+1. Install the build dependencies
+2. Build and install pandas
+3. Install the optional dependencies
 
-      conda create -n pandas_dev python=3 --file ci/requirements_dev.txt
-
-.. warning::
-
-   If you are on Windows, see :ref:`here for a fully compliant Windows environment <contributing.windows>`.
-
-This will create the new environment, and not touch any of your existing environments,
-nor any existing python installation. It will install all of the basic dependencies of
-*pandas*, as well as the development and testing tools. If you would like to install
-other dependencies, you can install them as follows::
+.. code-block:: none
 
-      conda install -n pandas_dev -c pandas pytables scipy
+   # Create and activate the build environment
+   conda env create -f ci/environment-dev.yaml
+   conda activate pandas-dev
 
-To install *all* pandas dependencies you can do the following::
+   # Build and install pandas
+   python setup.py build_ext --inplace -j 4
+   python -m pip install -e .
 
-      conda install -n pandas_dev -c conda-forge --file ci/requirements_all.txt
+   # Install the rest of the optional dependencies
+   conda install -c defaults -c conda-forge --file=ci/requirements-optional-conda.txt
 
-To work in this environment, Windows users should ``activate`` it as follows::
+At this point you should be able to import pandas from your locally built version::
 
-      activate pandas_dev
+   $ python  # start an interpreter
+   >>> import pandas
+   >>> print(pandas.__version__)
+   0.22.0.dev0+29.g4ad6d4d74
 
-Mac OSX / Linux users should use::
+.. warning::
 
-      source activate pandas_dev
+   If you are on Windows, see :ref:`here for a fully compliant Windows environment <contributing.windows>`.
 
-You will then see a confirmation message to indicate you are in the new development environment.
+This will create the new environment, and not touch any of your existing environments,
+nor any existing python installation.
 
 To view your environments::
 
       conda info -e
 
-To return to your home root environment in Windows::
-
-      deactivate
-
-To return to your home root environment in OSX / Linux::
+To return to your root environment::
 
-      source deactivate
+      conda deactivate
 
 See the full conda docs `here <http://conda.pydata.org/docs>`__.
 
-At this point you can easily do an *in-place* install, as detailed in the next section.
-
 .. _contributing.windows:
 
 Creating a Windows development environment
@@ -224,29 +189,57 @@ Here are some references and blogs:
 - https://blog.ionelmc.ro/2014/12/21/compiling-python-extensions-on-windows/
 - https://support.enthought.com/hc/en-us/articles/204469260-Building-Python-extensions-with-Canopy
 
-.. _contributing.getting_source:
+.. _contributing.pip:
 
-Making changes
---------------
+If you aren't using conda for you development environment, follow these instructions.
+You'll need to have python3.5 installed on your system.
+
+.. code-block:: none
+
+   # Create a virtual environment
+   # Use an ENV_DIR of your choice. We'll use ~/virtualenvs/pandas-dev
+   # Any parent directories should already exist
+   python3 -m venv ~/virtualenvs/pandas-dev
+   # Activate the virtulaenv
+   . ~/virtualenvs/pandas-dev/bin/activate
 
-Before making your code changes, it is often necessary to build the code that was
-just checked out.  There are two primary methods of doing this.
+   # Install the build dependencies
+   python -m pip install -r ci/requirements_dev.txt
+   # Build and install pandas
+   python setup.py build_ext --inplace -j 4
+   python -m pip install -e .
+
+   # Install additional dependencies
+   python -m pip install -r ci/requirements-optional-pip.txt
+
+Creating a branch
+-----------------
+
+You want your master branch to reflect only production-ready code, so create a
+feature branch for making your changes. For example::
+
+    git branch shiny-new-feature
+    git checkout shiny-new-feature
 
-#. The best way to develop *pandas* is to build the C extensions in-place by
-   running::
+The above can be simplified to::
 
-      python setup.py build_ext --inplace
+    git checkout -b shiny-new-feature
 
-   If you startup the Python interpreter in the *pandas* source directory you
-   will call the built C extensions
+This changes your working directory to the shiny-new-feature branch.  Keep any
+changes in this branch specific to one bug or feature so it is clear
+what the branch brings to *pandas*. You can have many shiny-new-features
+and switch in between them using the git checkout command.
 
-#. Another very common option is to do a ``develop`` install of *pandas*::
+To update this branch, you need to retrieve the changes from the master branch::
 
-      python setup.py develop
+    git fetch upstream
+    git rebase upstream/master
 
-   This makes a symbolic link that tells the Python interpreter to import *pandas*
-   from your development directory. Thus, you can always be using the development
-   version on your system without being inside the clone directory.
+This will replay your commits on top of the latest pandas git master.  If this
+leads to merge conflicts, you must resolve these before submitting your pull
+request.  If you have uncommitted changes, you will need to ``stash`` them prior
+to updating.  This will effectively store your changes and they can be reapplied
+after updating.
 
 
 .. _contributing.documentation:
@@ -342,30 +335,6 @@ Requirements
 
 First, you need to have a development environment to be able to build pandas
 (see the docs on :ref:`creating a development environment above <contributing.dev_env>`).
-Further, to build the docs, there are some extra requirements: you will need to
-have ``sphinx`` and ``ipython`` installed. `numpydoc
-<https://github.com/numpy/numpydoc>`_ is used to parse the docstrings that
-follow the Numpy Docstring Standard (see above), but you don't need to install
-this because a local copy of numpydoc is included in the *pandas* source
-code. `nbsphinx <https://nbsphinx.readthedocs.io/>`_ is required to build
-the Jupyter notebooks included in the documentation.
-
-If you have a conda environment named ``pandas_dev``, you can install the extra
-requirements with::
-
-      conda install -n pandas_dev sphinx ipython nbconvert nbformat
-      conda install -n pandas_dev -c conda-forge nbsphinx
-
-Furthermore, it is recommended to have all :ref:`optional dependencies <install.optional_dependencies>`.
-installed. This is not strictly necessary, but be aware that you will see some error
-messages when building the docs. This happens because all the code in the documentation
-is executed during the doc build, and so code examples using optional dependencies
-will generate errors. Run ``pd.show_versions()`` to get an overview of the installed
-version of all dependencies.
-
-.. warning::
-
-   You need to have ``sphinx`` version >= 1.3.2.
 
 Building the documentation
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -386,10 +355,10 @@ If you want to do a full clean build, do::
     python make.py clean
     python make.py html
 
-Starting with *pandas* 0.13.1 you can tell ``make.py`` to compile only a single section
-of the docs, greatly reducing the turn-around time for checking your changes.
-You will be prompted to delete ``.rst`` files that aren't required. This is okay because
-the prior versions of these files can be checked out from git. However, you must make sure
+You can tell ``make.py`` to compile only a single section of the docs, greatly
+reducing the turn-around time for checking your changes. You will be prompted to
+delete ``.rst`` files that aren't required. This is okay because the prior
+versions of these files can be checked out from git. However, you must make sure
 not to commit the file deletions to your Git repository!
 
 ::

scripts/convert_deps.py

@@ -0,0 +1,29 @@
+"""
+Convert the conda environment.yaml to a pip requirements.txt
+"""
+import yaml
+
+exclude = {'python=3'}
+rename = {'pytables': 'tables'}
+
+with open("ci/environment-dev.yaml") as f:
+    dev = yaml.load(f)
+
+with open("ci/requirements-optional-conda.txt") as f:
+    optional = [x.strip() for x in f.readlines()]
+
+required = dev['dependencies']
+required = [rename.get(dep, dep) for dep in required if dep not in exclude]
+optional = [rename.get(dep, dep) for dep in optional if dep not in exclude]
+
+
+with open("ci/requirements_dev.txt", 'wt') as f:
+    f.write("# This file was autogenerated by scripts/convert_deps.py\n")
+    f.write("# Do not modify directly\n")
+    f.write('\n'.join(required))
+
+
+with open("ci/requirements-optional-pip.txt", 'wt') as f:
+    f.write("# This file was autogenerated by scripts/convert_deps.py\n")
+    f.write("# Do not modify directly")
+    f.write("\n".join(optional))