to_html formatter not called for float values in a mixed-type column (2) #26000

simonjayhawkins · 2019-04-05T11:46:23Z

closes to_html formatter not called for float values in a mixed-type column #13021
tests added / passed
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

resubmit of #25983

…tter

…tter-2

simonjayhawkins · 2019-04-05T12:34:48Z

considering #24134 (comment) , this PR may need to revert some of #24134 which i think caused the regression in #25984

simonjayhawkins · 2019-04-05T19:41:14Z

i'm going to close this for now, since while the implementation of ExtensionArrayFormatter relies on formatting options AND a formatters parameter, it is not possible to add the shortcut for custom formatters. Changing this is outside the scope of this PR so i'll circle back to this once that's resolved.

…tter-2

simonjayhawkins · 2019-04-05T21:11:49Z

@jreback ; gone with bypassing format_array altogether when formatters are passed . it's a bit hacky, but on the bright side it closes the original issue, gets round the leading space bug #26002 and gets an additional test in play.

codecov · 2019-04-05T23:21:11Z

Codecov Report

❗ No coverage uploaded for pull request base (master@77e6556). Click here to learn what that means.
The diff coverage is 100%.

@@            Coverage Diff            @@
##             master   #26000   +/-   ##
=========================================
  Coverage          ?   91.86%           
=========================================
  Files             ?      180           
  Lines             ?    50717           
  Branches          ?        0           
=========================================
  Hits              ?    46592           
  Misses            ?     4125           
  Partials          ?        0

Flag	Coverage Δ
#multiple	`90.46% <100%> (?)`
#single	`41.1% <50%> (?)`

Impacted Files	Coverage Δ
pandas/io/formats/format.py	`98.02% <100%> (ø)`
pandas/core/arrays/integer.py	`96.3% <100%> (ø)`
pandas/core/arrays/interval.py	`93.1% <100%> (ø)`
pandas/core/arrays/categorical.py	`95.92% <100%> (ø)`
pandas/core/indexes/base.py	`96.71% <100%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 77e6556...5ecf91a. Read the comment docs.

codecov · 2019-04-05T23:21:12Z

Codecov Report

Merging #26000 into master will decrease coverage by <.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master   #26000      +/-   ##
==========================================
- Coverage   91.82%   91.82%   -0.01%     
==========================================
  Files         175      175              
  Lines       52551    52555       +4     
==========================================
+ Hits        48256    48257       +1     
- Misses       4295     4298       +3

Flag	Coverage Δ
#multiple	`90.38% <100%> (ø)`	⬆️
#single	`40.74% <50%> (-0.12%)`	⬇️

Impacted Files	Coverage Δ
pandas/io/formats/format.py	`97.9% <100%> (ø)`	⬆️
pandas/io/gbq.py	`75% <0%> (-12.5%)`	⬇️
pandas/core/frame.py	`96.79% <0%> (-0.12%)`	⬇️
pandas/util/testing.py	`90.72% <0%> (+0.1%)`	⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 181f972...9c1354c. Read the comment docs.

jreback · 2019-04-06T16:18:53Z

pandas/io/formats/format.py


    def get_result(self):
        fmt_values = self._format_strings()
        return _make_fixed_width(fmt_values, self.justify)

    def _format_strings(self):
+        # shortcut
+        if self.formatter is not None and self.shortcut:


why do you need shortcut at all? isn't self.formatter is not None enough?

the implementation of ExtensionArrayFormatter relies a formatter parameter and not shortcutting. so yes we need a shortcut parameter from to_string etc unless the ExtensionArrayFormatter implementation is changed.

I would rather do that, this is too hacky

changing the ExtensionArrayFormatter is unlikely to be trivial.

…tter-2

jreback · 2019-04-07T21:38:33Z

pandas/io/formats/format.py

@@ -938,6 +938,10 @@ def get_result(self):
        return _make_fixed_width(fmt_values, self.justify)

    def _format_strings(self):
+        # shortcut


can you give some more explanation here

jreback · 2019-06-08T20:18:17Z

@simonjayhawkins can you merge master, what's the status?

…tter-2

simonjayhawkins · 2019-06-13T12:39:25Z

@simonjayhawkins can you merge master, what's the status?

changing ExtensionArrayFormatter is necessary but outside the scope of this PR.

did make a change here in this PR and ci passing. I think that is more an indication of missing tests than a solution.

will do a precursor PR instead. I'm not really happy with the implementation of ExtensionArrayFormatter but don't know why it was done this way. There presumably needs to a mechanism for 3rd party creators of EA types to display a repr. not sure if this was the reason and that is the solution though.

…tter-2

simonjayhawkins · 2019-06-18T15:08:38Z

while the implementation of ExtensionArrayFormatter relies on formatting options AND a formatters parameter, it is not possible to add the shortcut for custom formatters.

changed formatters to be a pre-formatter instead of a shortcut so this is no longer an issue. @jorisvandenbossche

simonjayhawkins · 2019-06-18T15:11:02Z

latest commit also closes #26002 (again), two spaces between columns now as for other DataFrame columns

jreback · 2019-06-19T00:26:29Z

lgtm. @jorisvandenbossche @TomAugspurger if you want to have a look.

jreback · 2019-06-21T02:22:10Z

@simonjayhawkins is this orthogonal to your formatters refactor?

simonjayhawkins · 2019-06-21T06:05:27Z

@jreback

The two PRs are addressing orthogonal issues, but the implementations are not 100% orthogonal.

The other PR is a more important issue. This PR addresses a case which is probably rarely encountered by users.

This should probably be put on hold while the Extension Array formatting is discussed, since this PR does change some Extension Array formatting code.

I could probably pull out a couple of these changes into a separate PR. Changing the formatter parameter to format_array to be optional is probably not contentious and adding a test for na_rep of IntegarNA would be a good thing.

TomAugspurger · 2019-07-02T20:25:53Z

@simonjayhawkins there's a merge conflict.

Is this ready for 0.25? Haven't had time to look at the change, but I'm happy to trust others if the tests are passing.

simonjayhawkins · 2019-07-02T21:04:26Z

Is this ready for 0.25?

i don't believe it is wise to merge this at this time, so closing for now and will revisit at a later date.

simonjayhawkins added 6 commits April 4, 2019 00:02

to_html formatter not called for float values in a mixed-type column

71e8b31

changes to test as requested

2a2bb57

Merge remote-tracking branch 'upstream/master' into GenericArrayForma…

1e5615b

…tter

Merge branch 'master' into GenericArrayFormatter

4ca48a1

Merge remote-tracking branch 'upstream/master' into GenericArrayForma…

4ef3149

…tter

Merge remote-tracking branch 'upstream/master' into GenericArrayForma…

d7a8510

…tter-2

jreback added IO HTML read_html, to_html, Styler.apply, Styler.applymap Output-Formatting __repr__ of pandas objects, to_string labels Apr 5, 2019

simonjayhawkins closed this Apr 5, 2019

simonjayhawkins reopened this Apr 5, 2019

simonjayhawkins added 2 commits April 5, 2019 22:06

shortcut format_array

4b60e4b

Merge remote-tracking branch 'upstream/master' into GenericArrayForma…

5ac441b

…tter-2

simonjayhawkins added 2 commits April 5, 2019 23:12

add shortcut parameter to format_array

8a64459

add whatsnew for pandas-dev#26002

9c1354c

simonjayhawkins marked this pull request as ready for review April 5, 2019 22:45

jreback requested changes Apr 6, 2019

View reviewed changes

simonjayhawkins added 4 commits April 6, 2019 23:48

remove shortcut parameter from format_array

d0df1d6

Merge remote-tracking branch 'upstream/master' into GenericArrayForma…

c74b0aa

…tter-2

remove whatsnew for pandas-dev#26002

4262113

defer to GenericArrayFormatter for IntervalArray

f0cf9b7

jreback requested changes Apr 7, 2019

View reviewed changes

Merge remote-tracking branch 'upstream/master' into GenericArrayForma…

d6bee41

…tter-2

simonjayhawkins mentioned this pull request Jun 13, 2019

REF: remove ExtensionArrayFormatter #26833

Merged

4 tasks

simonjayhawkins added 2 commits June 18, 2019 11:33

Merge remote-tracking branch 'upstream/master' into GenericArrayForma…

1c535a1

…tter-2

pre-format instead of shortcut

5ecf91a

simonjayhawkins mentioned this pull request Jun 18, 2019

Refactor format_array in pandas\io\formats\format.py #26837

Open

jreback added this to the 0.25.0 milestone Jun 19, 2019

jreback approved these changes Jun 19, 2019

View reviewed changes

simonjayhawkins added the Prio-low label Jun 21, 2019

jreback removed this from the 0.25.0 milestone Jun 28, 2019

simonjayhawkins closed this Jul 2, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

to_html formatter not called for float values in a mixed-type column (2) #26000

to_html formatter not called for float values in a mixed-type column (2) #26000

simonjayhawkins commented Apr 5, 2019 •

edited

Loading

simonjayhawkins commented Apr 5, 2019

simonjayhawkins commented Apr 5, 2019

simonjayhawkins commented Apr 5, 2019

codecov bot commented Apr 5, 2019 •

edited

Loading

codecov bot commented Apr 5, 2019

jreback Apr 6, 2019

simonjayhawkins Apr 6, 2019

jreback Apr 6, 2019

simonjayhawkins Apr 6, 2019

jreback Apr 7, 2019

jreback commented Jun 8, 2019

simonjayhawkins commented Jun 13, 2019

simonjayhawkins commented Jun 18, 2019

simonjayhawkins commented Jun 18, 2019

jreback commented Jun 19, 2019

jreback commented Jun 21, 2019

simonjayhawkins commented Jun 21, 2019

TomAugspurger commented Jul 2, 2019 •

edited

Loading

simonjayhawkins commented Jul 2, 2019

to_html formatter not called for float values in a mixed-type column (2) #26000

to_html formatter not called for float values in a mixed-type column (2) #26000

Conversation

simonjayhawkins commented Apr 5, 2019 • edited Loading

simonjayhawkins commented Apr 5, 2019

simonjayhawkins commented Apr 5, 2019

simonjayhawkins commented Apr 5, 2019

codecov bot commented Apr 5, 2019 • edited Loading

Codecov Report

codecov bot commented Apr 5, 2019

Codecov Report

jreback Apr 6, 2019

Choose a reason for hiding this comment

simonjayhawkins Apr 6, 2019

Choose a reason for hiding this comment

jreback Apr 6, 2019

Choose a reason for hiding this comment

simonjayhawkins Apr 6, 2019

Choose a reason for hiding this comment

jreback Apr 7, 2019

Choose a reason for hiding this comment

jreback commented Jun 8, 2019

simonjayhawkins commented Jun 13, 2019

simonjayhawkins commented Jun 18, 2019

simonjayhawkins commented Jun 18, 2019

jreback commented Jun 19, 2019

jreback commented Jun 21, 2019

simonjayhawkins commented Jun 21, 2019

TomAugspurger commented Jul 2, 2019 • edited Loading

simonjayhawkins commented Jul 2, 2019

simonjayhawkins commented Apr 5, 2019 •

edited

Loading

codecov bot commented Apr 5, 2019 •

edited

Loading

TomAugspurger commented Jul 2, 2019 •

edited

Loading