BUG: extra leading space in to_string when index=False #29670

charlesdong1991 · 2019-11-17T12:51:05Z

closes Bug on .to_string(index=False) #24980 , xref pd.Series.to_string(index=False) behavior changed to include leading spaces #28538
tests added / passed
passes black pandas
passes git diff upstream/master -u -- "*.py" | flake8 --diff
whatsnew entry

I still see people posting issue for the change of behaviour in to_string(index=False) on pandas such as #28538 , so I think it might be worth it to reopen my previous stalled PR #25000 and see if could get it solved this time.

All the changes have been made based on the reviews in PR #25000 and for detailed summary and explanation, please check #25000 (comment)
Feel free to take a look and any review and comments are very welcomed!

…index_false

alimcmaster1 · 2019-11-17T14:46:50Z

pandas/io/formats/format.py

@@ -1095,7 +1106,7 @@ def format_array(
    space: Optional[Union[str, int]] = None,
    justify: str = "right",
    decimal: str = ".",
-    leading_space: Optional[bool] = None,
+    leading_space: bool = "compat",


this isn't a boolean? Can you run mypy on your changes

oh, yeah, didn't see your comment when I committed my latest change. I changed already

alimcmaster1 · 2019-11-17T14:49:02Z

pandas/tests/io/formats/test_format.py

+    "inputs, expected",
+    [([" a", " b"], " a\n b"), ([".1", "1"], ".1\n 1"), (["10", "-10"], " 10\n-10")],
+)
+def test_to_string_index_false_corner_case(inputs, expected):


I think these cases should just be added as param's to your test method below test_format_remove_leading_space_series

sure, this was just some corner cases that was found during the previous PR. will move.

alimcmaster1 · 2019-11-17T14:54:02Z

pandas/io/formats/format.py

@@ -937,13 +943,18 @@ def to_latex(
    def _format_col(self, i: int) -> List[str]:
        frame = self.tr_frame
        formatter = self._get_formatter(i)
+        if self.index:
+            leading_space = "compat"


Seems like there was already some discussion around this here https://github.com/pandas-dev/pandas/pull/25000/files#r252237505 -

emm, i think i addressed it already in this PR?

alimcmaster1 · 2019-11-17T14:58:49Z

pandas/io/formats/format.py

@@ -1110,7 +1121,7 @@ def format_array(
    space
    justify
    decimal
-    leading_space : bool, optional
+    leading_space : bool, default is 'compat'
        Whether the array should be formatted with a leading space.


same comment as previous here what is the intended type of this?

yeah, I didn't notice this so just copy paste the change in my previous PR. my bad, changed.

alimcmaster1 · 2019-11-17T15:02:43Z

thanks for the pr @charlesdong1991 - some comments above. We can re-open the previous PR if helpful? #25000

charlesdong1991 · 2019-11-17T15:11:30Z

thanks for your comment, @alimcmaster1
I already have this PR open and there has been dramatic changes since back then, so I think it is fine just to leave the old PR as is. But if you think it's helpful, feel free to re-open the stalled PR, and I don't think I am able to do it.

charlesdong1991 · 2019-12-23T15:14:24Z

Finally the annotation error is gone, it's green now 😅

pls take a look and see if this PR is useful.

…index_false

charlesdong1991 · 2019-12-28T08:39:29Z

Hello, @jreback
I would appreciate a lot if you would also like to take a look at this PR which was closed long ago, but since other people posted the issue again, I reopened it with this PR.

All comments in the last closed PR were addressed.

…index_false

simontorres · 2020-01-07T16:56:27Z

So, this PR seems ok. Anyone with the right permissions to fix conflict an merge? I've been forced to keep using version 0.23.0

jreback · 2020-01-07T16:57:53Z

@simontorres there are quite a number of comments on this
it is not so simple

i find it hard to believe this blocking an update of pandas

simontorres · 2020-01-07T17:03:01Z

I was talking out ignorance. I'm sorry

charlesdong1991 · 2020-01-07T17:03:16Z

Hi, @simontorres i resolved the conflict, and as @jreback said, there are some comments in the PR, and pls feel free to take over if you could find a better solution for this, and then i could close this PR to make way for you!

simontorres · 2020-01-07T17:13:20Z

Thanks @charlesdong1991 and @jreback . I don't think I can find a better solution but I will see if I can help.

charlesdong1991 · 2020-01-07T18:05:49Z

I am sure you could help with the issue and pandas 👍 @simontorres

also @jreback feel free to close this PR if you find it hard to approach!

alimcmaster1 · 2020-01-07T22:00:11Z

Thanks @charlesdong1991 and @jreback . I don't think I can find a better solution but I will see if I can help.

@simontorres
Could you provide a repo of the issue you faced? That way we can ensure we are covering your use case with a test case in pandas.

simontorres · 2020-01-08T15:14:36Z

Introduction/Application

One example is this Imaging Spectrograph called GoodmanHTS, there are two elements that can move a camera and a dispersor, their angle are recorded in the image's header. There are certain named modes that I'm interested in that can be used define file names, for instance. So this is a shorter version of a longer table.

import pandas

columns = ['grating_freq', 'wavmode', 'camtarg', 'grttarg', 'ob_filter']
spec_mode = [['400', 'm1', '11.6', '5.8', 'None'],
             ['400', 'm2', '16.1', '7.5', 'GG455'],
             ['600', 'UV', '15.25', '7.0', 'None'],
             ['600', 'Blue', '17.0', '7.0', 'None'],
             ['600', 'Mid', '20.0', '10.0', 'GG385'],
             ['600', 'Red', '27.0', '12.0', 'GG495'],
             ['930', 'm1', '20.6', '10.3', 'None'],
             ['930', 'm2', '25.2', '12.6', 'None'],
             ['930', 'm3', '29.9', '15.0', 'GG385'],
             ['930', 'm4', '34.6', '18.3', 'GG495'],
             ['930', 'm5', '39.4', '19.7', 'GG495']]

modes_data_frame = pandas.DataFrame(spec_mode, columns=columns)

The problem

The information I get is the Grating Frequency (for instance 930), the Camera Target Angle (camtarg, 20.6), the Grating Target Angle (grttarg, 10.3) and the Order Blocking Filter (. When I use pandas to find the match in the following way.

mode = modes_data_frame[
    ((modes_data_frame['grating_freq'] == 930) &
     (modes_data_frame['camtarg'] == 20.6) &
     (modes_data_frame['grttarg'] == 10.3) &
     (modes_data_frame['ob_filter'] == None))]

if mode.empty:
    central_wavelength = get_central_wavelength(*some_args)
    return 'Custom_{:d}nm'.format(central_wavelength)
else:
    return mode['wavmode'].to_string(index=False)

it should return 'm1' instead it returns ' m1'
This worked until version 0.23.0 and I reported it on #24980

WillAyd · 2020-02-12T00:34:22Z

This hasn't gotten traction in a while so closing as stale, but @charlesdong1991 or @simontorres ping if you plan to pick up and can continue from there

charlesdong1991 added 6 commits December 3, 2018 17:43

remove \n from docstring

7e461a1

fix conflicts

1314059

Merge remote-tracking branch 'upstream/master'

8bcb313

Merge remote-tracking branch 'upstream/master' into reopen_to_string_…

0f1d700

…index_false

Leading space in to_string when index=False

7d96cd3

add whatsnew note

41bceb9

alimcmaster1 requested changes Nov 17, 2019

View reviewed changes

fix typing

2e46bb4

charlesdong1991 added 2 commits November 17, 2019 16:12

change doc

fe63d83

combine tests

af9d721

alimcmaster1 added the Output-Formatting __repr__ of pandas objects, to_string label Nov 17, 2019

charlesdong1991 added 8 commits November 17, 2019 16:43

fix typing hinting

b9affd0

type hinting

17941b6

remove type hinting

e5fa13e

merge master and fix conflict

eaccb40

add type annotation

e027044

remove commetn

c63cc82

black reformat

5e6a730

fix annotation

d75b4da

Merge remote-tracking branch 'upstream/master' into reopen_to_string_…

67d4ae8

…index_false

Merge remote-tracking branch 'upstream/master' into reopen_to_string_…

fb88243

…index_false

jreback mentioned this pull request Jan 7, 2020

Bug on .to_string(index=False) #24980

Closed

merge master and fix conflict

9b8b8f7

simontorres mentioned this pull request Jan 8, 2020

Update pandas soar-telescope/goodman_pipeline#314

Closed

WillAyd closed this Feb 12, 2020

onshek mentioned this pull request Sep 3, 2020

BUG: extra leading space in to_string when index=False #36094

Merged

5 tasks

simontorres pushed a commit to simontorres/goodman_pipeline that referenced this pull request Oct 1, 2020

Removed pandas constraints, bugs seems fixed in pandas-dev/pandas#29670

e58ac51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BUG: extra leading space in to_string when index=False #29670

BUG: extra leading space in to_string when index=False #29670

charlesdong1991 commented Nov 17, 2019 •

edited

Loading

alimcmaster1 Nov 17, 2019

charlesdong1991 Nov 17, 2019

alimcmaster1 Nov 17, 2019

charlesdong1991 Nov 17, 2019

alimcmaster1 Nov 17, 2019

charlesdong1991 Nov 17, 2019

alimcmaster1 Nov 17, 2019

charlesdong1991 Nov 17, 2019 •

edited

Loading

alimcmaster1 commented Nov 17, 2019

charlesdong1991 commented Nov 17, 2019

charlesdong1991 commented Dec 23, 2019

charlesdong1991 commented Dec 28, 2019 •

edited

Loading

simontorres commented Jan 7, 2020

jreback commented Jan 7, 2020

simontorres commented Jan 7, 2020

charlesdong1991 commented Jan 7, 2020

simontorres commented Jan 7, 2020

charlesdong1991 commented Jan 7, 2020 •

edited

Loading

alimcmaster1 commented Jan 7, 2020

simontorres commented Jan 8, 2020

WillAyd commented Feb 12, 2020

BUG: extra leading space in to_string when index=False #29670

BUG: extra leading space in to_string when index=False #29670

Conversation

charlesdong1991 commented Nov 17, 2019 • edited Loading

alimcmaster1 Nov 17, 2019

Choose a reason for hiding this comment

charlesdong1991 Nov 17, 2019

Choose a reason for hiding this comment

alimcmaster1 Nov 17, 2019

Choose a reason for hiding this comment

charlesdong1991 Nov 17, 2019

Choose a reason for hiding this comment

alimcmaster1 Nov 17, 2019

Choose a reason for hiding this comment

charlesdong1991 Nov 17, 2019

Choose a reason for hiding this comment

alimcmaster1 Nov 17, 2019

Choose a reason for hiding this comment

charlesdong1991 Nov 17, 2019 • edited Loading

Choose a reason for hiding this comment

alimcmaster1 commented Nov 17, 2019

charlesdong1991 commented Nov 17, 2019

charlesdong1991 commented Dec 23, 2019

charlesdong1991 commented Dec 28, 2019 • edited Loading

simontorres commented Jan 7, 2020

jreback commented Jan 7, 2020

simontorres commented Jan 7, 2020

charlesdong1991 commented Jan 7, 2020

simontorres commented Jan 7, 2020

charlesdong1991 commented Jan 7, 2020 • edited Loading

alimcmaster1 commented Jan 7, 2020

simontorres commented Jan 8, 2020

Introduction/Application

The problem

WillAyd commented Feb 12, 2020

charlesdong1991 commented Nov 17, 2019 •

edited

Loading

charlesdong1991 Nov 17, 2019 •

edited

Loading

charlesdong1991 commented Dec 28, 2019 •

edited

Loading

charlesdong1991 commented Jan 7, 2020 •

edited

Loading