groupby-apply got different output with for loop in same action

[princelai]

data csv download link

https://mega.nz/#!XjZWyYDT!EezOrZJglcwqytA5bIgc6-VoiEMGB7R0Tu8CG-QjV5w

Problem description

I have a Stock Time Series, I only take 4 ts_code(stock code) and other 3 useful columns to recurrent problem，you can download throgh link.

read data

df = pd.read_csv('stock.csv', index_col='trade_date')
df.index = pd.to_datetime(df.index)

problem code

dtday = df.groupby('ts_code').apply(lambda x: (x.MA1 > x.MA2) & (x.MA2 > x.MA3))

right code

l = []
for k,v in df.groupby('ts_code'):
    v2 = (v.MA1 > v.MA2) & (v.MA2 > v.MA3)
    v2.name = k
    l.append(v2)
dtday2 = pd.concat(l, axis=1).fillna(False).T

result 1

print(dtday.sum(axis=1))
ts_code
000001.SZ    87
002624.SZ    62
002625.SZ    36
600519.SH    28
dtype: int64

result 2

print(dtday2.sum(axis=1))
000001.SZ    87
002624.SZ     2
002625.SZ    44
600519.SH    39
dtype: int64

you can plot the any stock line to verificate

df.loc[df['ts_code'] == '002625.SZ', ['MA1', 'MA2', "MA3"]].plot()

I only want to count how many day the MA1 > MA2 > MA3, I think that 2 code are exactly the same，but the result show NO.

why?

Expected Output

Output of pd.show_versions()

INSTALLED VERSIONS

commit: None
python: 3.7.2.final.0
python-bits: 64
OS: Linux
OS-release: 4.19.30-1-MANJARO
machine: x86_64
processor:
byteorder: little
LC_ALL: None
LANG: en_US.UTF-8
LOCALE: en_US.UTF-8
pandas: 0.24.2
pytest: None
pip: 19.0.3
setuptools: 40.8.0
Cython: 0.29.2
numpy: 1.16.2
scipy: 1.2.1
pyarrow: None
xarray: None
IPython: 7.3.0
sphinx: 1.8.5
patsy: 0.5.1
dateutil: 2.8.0
pytz: 2018.9
blosc: None
bottleneck: None
tables: 3.4.4
numexpr: 2.6.9
feather: None
matplotlib: 3.0.3
openpyxl: 2.6.1
xlrd: 1.2.0
xlwt: None
xlsxwriter: None
lxml.etree: 4.3.2
bs4: 4.7.1
html5lib: None
sqlalchemy: 1.3.1
pymysql: 0.9.3
psycopg2: None
jinja2: 2.10
s3fs: None
fastparquet: None
pandas_gbq: None
pandas_datareader: None
gcsfs: None

[otro-mas1]

"I only want to count how many day the MA1 > MA2 > MA3, I think that 2 code are exactly the same，but the result show NO."

do you want this?

df3 = df.loc[(df['MA1'] > df['MA2']) & (df['MA2'] > df['MA3']), ]
df3.ts_code.value_counts()

[jreback]

your use of groupby-apply is completely non-idiomatic, modifying an external variable from inside the .apply. there are many better ways to do this.

In any event, closing as fixed on master by #24748