Skip to content

Fix for BUG: multi-index excel header fails if all numeric #11328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions asv_bench/asv.conf.json
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@
"numexpr": [],
"pytables": [],
"openpyxl": [],
"xlsxwriter": [],
"xlrd": [],
"xlwt": []
},
Expand Down
10 changes: 10 additions & 0 deletions asv_bench/benchmarks/frame_methods.py
Original file line number Diff line number Diff line change
Expand Up @@ -930,6 +930,16 @@ def time_frame_xs_row(self):
self.df.xs(50000)


class frame_sort_index(object):
goal_time = 0.2

def setup(self):
self.df = DataFrame(randn(1000000, 2), columns=list('AB'))

def time_frame_sort_index(self):
self.df.sort_index()


class series_string_vector_slice(object):
goal_time = 0.2

Expand Down
46 changes: 46 additions & 0 deletions asv_bench/benchmarks/gil.py
Original file line number Diff line number Diff line change
Expand Up @@ -320,3 +320,49 @@ def time_nogil_kth_smallest(self):
def run(arr):
algos.kth_smallest(arr, self.k)
run()

class nogil_datetime_fields(object):
goal_time = 0.2

def setup(self):
self.N = 100000000
self.dti = pd.date_range('1900-01-01', periods=self.N, freq='D')
self.period = self.dti.to_period('D')
if (not have_real_test_parallel):
raise NotImplementedError

def time_datetime_field_year(self):
@test_parallel(num_threads=2)
def run(dti):
dti.year
run(self.dti)

def time_datetime_field_day(self):
@test_parallel(num_threads=2)
def run(dti):
dti.day
run(self.dti)

def time_datetime_field_daysinmonth(self):
@test_parallel(num_threads=2)
def run(dti):
dti.days_in_month
run(self.dti)

def time_datetime_field_normalize(self):
@test_parallel(num_threads=2)
def run(dti):
dti.normalize()
run(self.dti)

def time_datetime_to_period(self):
@test_parallel(num_threads=2)
def run(dti):
dti.to_period('S')
run(self.dti)

def time_period_to_datetime(self):
@test_parallel(num_threads=2)
def run(period):
period.to_timestamp()
run(self.period)
20 changes: 20 additions & 0 deletions asv_bench/benchmarks/series_methods.py
Original file line number Diff line number Diff line change
Expand Up @@ -71,3 +71,23 @@ def setup(self):
def time_series_nsmallest2(self):
self.s2.nsmallest(3, take_last=True)
self.s2.nsmallest(3, take_last=False)


class series_dropna_int64(object):
goal_time = 0.2

def setup(self):
self.s = Series(np.random.randint(1, 10, 1000000))

def time_series_dropna_int64(self):
self.s.dropna()

class series_dropna_datetime(object):
goal_time = 0.2

def setup(self):
self.s = Series(pd.date_range('2000-01-01', freq='S', periods=1000000))
self.s[np.random.randint(1, 1000000, 100)] = pd.NaT

def time_series_dropna_datetime(self):
self.s.dropna()
2 changes: 1 addition & 1 deletion ci/install_conda.sh
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ bash miniconda.sh -b -p $HOME/miniconda || exit 1
conda config --set always_yes yes --set changeps1 no || exit 1
conda update -q conda || exit 1
conda config --add channels conda-forge || exit 1
conda config --add channels http://conda.binstar.org/pandas || exit 1
conda config --add channels http://conda.anaconda.org/pandas || exit 1
conda config --set ssl_verify false || exit 1

# Useful for debugging any issues with conda
Expand Down
2 changes: 2 additions & 0 deletions ci/requirements-2.7.pip
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,5 @@ blosc
httplib2
google-api-python-client == 1.2
python-gflags == 2.0
pathlib
py
Empty file added ci/requirements-2.7_SLOW.pip
Empty file.
1 change: 1 addition & 0 deletions ci/requirements-3.4.build
Original file line number Diff line number Diff line change
Expand Up @@ -2,3 +2,4 @@ python-dateutil
pytz
numpy=1.8.1
cython
libgfortran
5 changes: 3 additions & 2 deletions doc/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -299,8 +299,9 @@
intersphinx_mapping = {
'statsmodels': ('http://statsmodels.sourceforge.net/devel/', None),
'matplotlib': ('http://matplotlib.org/', None),
'python': ('http://docs.python.org/', None),
'numpy': ('http://docs.scipy.org/doc/numpy', None)
'python': ('http://docs.python.org/3', None),
'numpy': ('http://docs.scipy.org/doc/numpy', None),
'py': ('http://pylib.readthedocs.org/en/latest/', None)
}
import glob
autosummary_generate = glob.glob("*.rst")
Expand Down
5 changes: 3 additions & 2 deletions doc/source/io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,10 @@ for some advanced strategies

They can take a number of arguments:

- ``filepath_or_buffer``: Either a string path to a file, URL
- ``filepath_or_buffer``: Either a path to a file (a :class:`python:str`,
:class:`python:pathlib.Path`, or :class:`py:py._path.local.LocalPath`), URL
(including http, ftp, and S3 locations), or any object with a ``read``
method (such as an open file or ``StringIO``).
method (such as an open file or :class:`~python:io.StringIO`).
- ``sep`` or ``delimiter``: A delimiter / separator to split fields
on. With ``sep=None``, ``read_csv`` will try to infer the delimiter
automatically in some cases by "sniffing".
Expand Down
34 changes: 33 additions & 1 deletion doc/source/whatsnew/v0.17.1.txt
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ Highlights include:

Enhancements
~~~~~~~~~~~~
- ``DatetimeIndex`` now supports conversion to strings with astype(str)(:issue:`10442`)

- Support for ``compression`` (gzip/bz2) in :method:`DataFrame.to_csv` (:issue:`7615`)

Expand All @@ -27,6 +28,10 @@ Enhancements
Other Enhancements
^^^^^^^^^^^^^^^^^^

- ``pd.read_*`` functions can now also accept :class:`python:pathlib.Path`, or :class:`py:py._path.local.LocalPath`
objects for the ``filepath_or_buffer`` argument. (:issue:`11033`)
- Improve the error message displayed in :func:`pandas.io.gbq.to_gbq` when the DataFrame does not match the schema of the destination table (:issue:`11359`)

.. _whatsnew_0171.api:

API changes
Expand All @@ -37,17 +42,31 @@ API changes
- Regression from 0.16.2 for output formatting of long floats/nan, restored in (:issue:`11302`)
- Prettyprinting sets (e.g. in DataFrame cells) now uses set literal syntax (``{x, y}``) instead of
Legacy Python syntax (``set([x, y])``) (:issue:`11215`)
- Indexing with a null key will raise a ``TypeError``, instead of a ``ValueError`` (:issue:`11356`)

.. _whatsnew_0171.deprecations:

Deprecations
^^^^^^^^^^^^

- The ``pandas.io.ga`` module which implements ``google-analytics`` support is deprecated and will be removed in a future version (:issue:`11308`)
- Deprecate the ``engine`` keyword from ``.to_csv()``, which will be removed in a future version (:issue:`11274`)


.. _whatsnew_0171.performance:

Performance Improvements
~~~~~~~~~~~~~~~~~~~~~~~~

- Checking monotonic-ness before sorting on an index (:issue:`11080`)
- ``Series.dropna`` performance improvement when its dtype can't contain ``NaN`` (:issue:`11159`)


- Release the GIL on most datetime field operations (e.g. ``DatetimeIndex.year``, ``Series.dt.year``), normalization, and conversion to and from ``Period``, ``DatetimeIndex.to_period`` and ``PeriodIndex.to_timestamp`` (:issue:`11263`)


- Improved performance to ``to_excel`` (:issue:`11352`)

.. _whatsnew_0171.bug_fixes:

Bug Fixes
Expand All @@ -58,13 +77,19 @@ Bug Fixes

- Bug in ``HDFStore.select`` when comparing with a numpy scalar in a where clause (:issue:`11283`)

- Bug in tz-conversions with an ambiguous time and ``.dt`` accessors (:issues:`11295`)

- Bug in tz-conversions with an ambiguous time and ``.dt`` accessors (:issue:`11295`)
- Bug in comparisons of Series vs list-likes (:issue:`11339`)


- Bug in ``DataFrame.replace`` with a ``datetime64[ns, tz]`` and a non-compat to_replace (:issue:`11326`, :issue:`11153`)



- Bug in list-like indexing with a mixed-integer Index (:issue:`11320`)

- Bug in ``pivot_table`` with ``margins=True`` when indexes are of ``Categorical`` dtype (:issue:`10993`)
- Bug in ``DataFrame.plot`` cannot use hex strings colors (:issue:`10299`)



Expand All @@ -88,5 +113,12 @@ Bug Fixes


- Bugs in ``to_excel`` with duplicate columns (:issue:`11007`, :issue:`10982`, :issue:`10970`)

- Fixed a bug that prevented the construction of an empty series of dtype
``datetime64[ns, tz]`` (:issue:`11245`).

- Bug in ``read_excel`` with multi-index containing integers (:issue:`11317`)

- Bug in ``to_excel`` with openpyxl 2.2+ and merging (:issue:`11408`)

- Bug in ``DataFrame.to_dict()`` produces a ``np.datetime64`` object instead of ``Timestamp`` when only datetime is present in data (:issue:`11327`)
Loading