Test failure pandas.tests.io.test_sql.TestXSQLite

[jbrockmendel]

I'm seeing a test failure locally on master (83436af) that doesn't appear to be reflected in the CI.

Strangely, python -m pytest pandas/tests/io/test_sql.py goes through just fine. It gets triggered when I run:

>>> import matplotlib        # import order matters here
>>> matplotlib.use('Agg')  # needed to avoid matplotlib errors on a headless machine
>>> import pandas as pd
>>> pd.test()
[...]
___________________________________________________________________ TestXSQLite.test_write_row_by_row ____________________________________________________________________

self = <pandas.tests.io.test_sql.TestXSQLite object at 0x7f99def27850>

    def test_write_row_by_row(self):
    
        frame = tm.makeTimeDataFrame()
        frame.iloc[0, 0] = np.nan
        create_sql = sql.get_schema(frame, 'test')
        cur = self.conn.cursor()
        cur.execute(create_sql)
    
        cur = self.conn.cursor()
    
        ins = "INSERT INTO test VALUES (%s, %s, %s, %s)"
        for idx, row in frame.iterrows():
            fmt_sql = format_query(ins, *row)
            tquery(fmt_sql, cur=cur)
    
        self.conn.commit()
    
        result = sql.read_sql("select * from test", con=self.conn)
        result.index = frame.index
>       tm.assert_frame_equal(result, frame)

pandas/tests/io/test_sql.py:2134: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
pandas/util/testing.py:1402: in assert_frame_equal
    obj='DataFrame.iloc[:, {idx}]'.format(idx=i))
pandas/util/testing.py:1281: in assert_series_equal
    obj='{obj}'.format(obj=obj))
pandas/_libs/testing.pyx:59: in pandas._libs.testing.assert_almost_equal
    cpdef assert_almost_equal(a, b,
pandas/_libs/testing.pyx:173: in pandas._libs.testing.assert_almost_equal
    raise_assert_detail(obj, msg, lobj, robj)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

obj = 'DataFrame.iloc[:, 2]', message = 'DataFrame.iloc[:, 2] values are different (1.0 %)'
left = '[-0.59787048, 1.75727252, -0.1310937, -0.24085753, -1.55551175, 1.56304803, 0.43139791, 0.65542659, 0.0821268, -2.194...3087, 1.24262929, 0.25190051, -0.07253108, -0.8719974, -0.68566296, 0.00038078, -0.15632703, -0.46625691, -0.01641763]'
right = '[-0.597870480786, 1.75727252312, -0.131093699304, -0.240857530072, -1.55551175168, 1.56304803297, 0.431397907647, 0.6...725310846663, -0.871997401837, -0.685662960253, 0.000380776752185, -0.156327025786, -0.466256909406, -0.0164176348789]'
diff = None

    def raise_assert_detail(obj, message, left, right, diff=None):
        if isinstance(left, np.ndarray):
            left = pprint_thing(left)
        if isinstance(right, np.ndarray):
            right = pprint_thing(right)
    
        msg = """{obj} are different
    
    {message}
    [left]:  {left}
    [right]: {right}""".format(obj=obj, message=message, left=left, right=right)
    
        if diff is not None:
            msg += "\n[diff]: {diff}".format(diff=diff)
    
>       raise AssertionError(msg)
E       AssertionError: DataFrame.iloc[:, 2] are different
E       
E       DataFrame.iloc[:, 2] values are different (1.0 %)
E       [left]:  [-0.59787048, 1.75727252, -0.1310937, -0.24085753, -1.55551175, 1.56304803, 0.43139791, 0.65542659, 0.0821268, -2.19483669, 1.29691261, 0.86777117, 0.36392125, -0.57871272, 1.51058845, 1.09966946, -0.55036466, -0.40261691, -0.08746403, -0.45147578, 1.41894647, -0.4797129, 0.37331928, 0.26579615, -0.83709729, 0.31505209, -0.92100097, 0.10158214, -0.53096049, 0.06649923, -2.00748014, 2.22979451, 0.08193706, 0.14799922, 0.17588662, 0.86689148, -1.09017582, -1.81591557, 0.54031046, -0.11075987, 1.01837036, 1.98109779, -0.4993707, -1.59367359, -2.08201023, 0.88399937, 1.24248965, -0.37544702, -0.48737033, -1.3391138, -2.8575393, 0.89456229, -0.05170464, 1.13538934, 1.01720956, 0.16289824, -0.37986802, -0.89600403, -0.20111217, -0.05674187, 0.21197277, 0.62103332, -0.17917755, 0.88623787, 0.89353538, 0.89794712, -0.81026691, 0.04248671, -0.62944966, -1.07986145, -0.21194148, 0.71853091, 0.82763943, -1.53930274, 0.79953673, 0.65268344, 0.38126291, 1.68621206, -0.58234383, 0.53573623, -1.87181172, 1.81584145, 0.01156531, 1.00901096, -0.04814026, -0.06214, 0.55437294, -0.05109543, -0.40641954, -0.56006971, -0.14203087, 1.24262929, 0.25190051, -0.07253108, -0.8719974, -0.68566296, 0.00038078, -0.15632703, -0.46625691, -0.01641763]
E       [right]: [-0.597870480786, 1.75727252312, -0.131093699304, -0.240857530072, -1.55551175168, 1.56304803297, 0.431397907647, 0.655426586864, 0.0821268019634, -2.19483669218, 1.29691260774, 0.867771165078, 0.363921245537, -0.578712721937, 1.5105884463, 1.09966945821, -0.550364664379, -0.402616908529, -0.0874640312425, -0.451475782528, 1.41894646672, -0.479712901824, 0.373319283832, 0.265796151564, -0.837097285667, 0.315052093362, -0.921000967868, 0.101582144906, -0.530960487754, 0.0664992299507, -2.00748013934, 2.22979451164, 0.0819370606265, 0.147999223147, 0.175886617188, 0.866891479712, -1.09017582435, -1.81591556915, 0.540310455382, -0.110759871621, 1.01837036341, 1.98109778942, -0.499370703418, -1.59367359033, -2.08201023199, 0.883999370748, 1.24248965221, -0.375447016032, -0.487370326597, -1.3391138024, -2.85753930098, 0.894562293166, -0.0517046435105, 1.13538933705, 1.01720956219, 0.162898242598, -0.379868021698, -0.896004027455, -0.201112174114, -0.056741869755, 0.211972767677, 0.621033324798, -0.179177554443, 0.886237870665, 0.89353538222, 0.897947121995, -0.810266911765, 0.0424867143869, -0.6294496634, -1.0798614477, -0.211941482626, 0.718530908141, 0.82763942996, -1.53930273634, 0.799536732242, 0.65268344093, 0.381262905684, 1.68621205795, -0.582343833696, 0.535736231547, -1.87181171881, 1.81584145215, 0.0115653145115, 1.00901096046, -0.0481402575178, -0.0621399986677, 0.554372937832, -0.0510954278629, -0.406419542776, -0.560069714991, -0.142030873399, 1.24262929489, 0.251900511488, -0.0725310846663, -0.871997401837, -0.685662960253, 0.000380776752185, -0.156327025786, -0.466256909406, -0.0164176348789]

pandas/util/testing.py:1105: AssertionError

Py2.7.12, Ubuntu 16.04

[gfyoung]

that doesn't appear to be reflected in the CI

@jbrockmendel : What do you mean by this? I'm pretty sure that this test would get covered in the CI.

[jbrockmendel]

I mean I don't see the error in Travis et al. Figured it was worth reporting just in case.

[gfyoung]

Fair enough. Local test failures are a little more difficult to debug since we have test software that says otherwise...that being said, I cannot reproduce this locally at the moment.

Also, this failure appears to be related to some rounding issues, given that the numbers look pretty close but not exact. I'll close for now, but if anyone else has this problem, we can reopen.

[TomAugspurger]

Got a failure on travis: https://travis-ci.org/pandas-dev/pandas/jobs/275540546#L1608

[gfyoung]

Of course, it would rear it's head after the issue gets closed. 😂

[TomAugspurger]

I checked a few in the history and didn't see any others :)

[MaximilianKoestler]

@TomAugspurger This definitely happened before. I got the same error a few days ago on travis:
https://travis-ci.org/pandas-dev/pandas/jobs/274599916
Force-pushing and restarting the ci magically fixed it.

[jorisvandenbossche]

Since it seems just some rounding errors, I would just check for less decimals in the assert ? -> #17538