⬆️ Bump pandas to 0.25.0, and update other data science deps

Manually updating various PyData/data science dependencies in Pipfile, references to release notes and diff commits as follows:

- [dask](https://github.com/dask/dask) from 1.2.2 to 2.1.0
  - [Release notes](https://docs.dask.org/en/latest/changelog.html)
  - [Commits](dask/dask@v1.2.2...v2.1.0)
- [geopandas](https://github.com/geopandas/geopandas) from 0.5.0 to 0.5.1:
  - [Release notes](https://github.com/geopandas/geopandas/releases/tag/v0.5.1)
  - [Commits](geopandas/geopandas@v0.5.0...v0.5.1)
- [matplotlib](https://github.com/matplotlib/matplotlib) from 3.1.0 to 3.1.1:
  - [Release notes](https://github.com/matplotlib/matplotlib/releases/tag/v3.1.1)
  - [Commits](matplotlib/matplotlib@v3.1.0...v3.1.1)
- [numpy](https://github.com/numpy/numpy) from 1.16.4 to 1.17.0rc2:
  - [Release notes](https://github.com/numpy/numpy/releases/tag/v1.17.0rc2)
  - [Commits](numpy/numpy@v1.16.4...v1.17.0rc2)
- [pandas](https://github.com/pandas-dev/pandas) from 0.24.2 to 0.25.0:
  - [Release notes](https://github.com/pandas-dev/pandas/releases/tag/v0.25.0)
  - [Changelog](https://github.com/pandas-dev/pandas/blob/master/RELEASE.md)
  - [Commits](pandas-dev/pandas@v0.24.2...v0.25.0)
- [rasterio](https://github.com/mapbox/rasterio) from 1.0.23 to 1.0.24.
  - [Release notes](https://github.com/mapbox/rasterio/releases/tag/1.0.24)
  - [Changelog](https://github.com/mapbox/rasterio/blob/master/CHANGES.txt)
  - [Commits](rasterio/rasterio@1.0.23...1.0.24)
- [tqdm](https://github.com/tqdm/tqdm) from 4.32.1 to 4.32.2:
  - [Release notes](https://github.com/tqdm/tqdm/releases/tag/v4.32.2)
  - [Commits](tqdm/tqdm@v4.32.1...v4.32.2)

Quickfix in data_prep.ascii_to_xyz with pandas 0.25.0 introducing new json_normalize behaviour (see https://pandas.pydata.org/pandas-docs/version/0.25/whatsnew/v0.25.0.html#json-normalize-with-max-level-param-support, pandas-dev/pandas#23843). Caught unit test failing when trying to do the math Z=ELEVATION-BOTTOM because pandas.Series parsed the dictionary's key (Z) directly into the name as converters.Z, wow just wow...

Author: weiji14

Pipfile

@@ -8,26 +8,26 @@ black = "==19.3b0"
 chainer = "==6.2.0"
 comet-ml = "==2.0.5"
 cupy-cuda100 = "==6.2.0"
-dask = {extras = ["array"],version = "==1.2.2"}
+dask = {extras = ["array"],version = "==2.1.0"}
 descartes = "==1.1.0"
-geopandas = "==0.5.0"
+geopandas = "==0.5.1"
 joblib = "==0.13.2"
 jupyterlab = "==1.0.0a10"
 jupytext = "==1.1.4rc1"
 livelossplot = "==0.4.1"
-matplotlib = "==3.1.0"
+matplotlib = "==3.1.1"
 netcdf4 = "==1.4.1"
-numpy = "==1.16.4"
+numpy = "==1.17.0rc2"
 onnx_chainer = "==1.4.1"
 optuna = "==0.13.0"
-pandas = "==0.24.2"
+pandas = "==0.25.0"
 pygmt = {editable = true, ref = "0.0.1a0-40-g29220bd", git = "https://github.com/weiji14/pygmt.git"}
 quilt = "==2.9.15"
-rasterio = "==1.0.23"
+rasterio = "==1.0.24"
 rtree = "==0.8.3"
 salem = "==0.2.4"
 scikit-image = "==0.15.0"
-tqdm = "==4.32.1"
+tqdm = "==4.32.2"
 
 [dev-packages]
 behave = "==1.2.6"

Pipfile.lock

@@ -683,7 +683,7 @@
     "\n",
     "    # Read json file first\n",
     "    j = json.loads(open(pipeline_file).read())\n",
-    "    jdf = pd.io.json.json_normalize(j, record_path=\"pipeline\")\n",
+    "    jdf = pd.io.json.json_normalize(data=j, record_path=\"pipeline\", max_level=0)\n",
     "    jdf = jdf.set_index(keys=\"type\")\n",
     "    reader = jdf.loc[\"readers.text\"]  # check how to read the file(s)\n",
     "\n",

data_prep.ipynb

@@ -286,7 +286,7 @@ def ascii_to_xyz(pipeline_file: str) -> pd.DataFrame:
 
     # Read json file first
     j = json.loads(open(pipeline_file).read())
-    jdf = pd.io.json.json_normalize(j, record_path="pipeline")
+    jdf = pd.io.json.json_normalize(data=j, record_path="pipeline", max_level=0)
     jdf = jdf.set_index(keys="type")
     reader = jdf.loc["readers.text"]  # check how to read the file(s)