Skip to content

TST: skipping a gbq test for now #14643

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jreback opened this issue Nov 12, 2016 · 4 comments
Closed

TST: skipping a gbq test for now #14643

jreback opened this issue Nov 12, 2016 · 4 comments
Labels
Testing pandas testing functions or related to the test suite
Milestone

Comments

@jreback
Copy link
Contributor

jreback commented Nov 12, 2016

I am xfailing most of these tests for now till we have the testing sorted: d9e75c7

@jreback jreback added IO Google Testing pandas testing functions or related to the test suite labels Nov 12, 2016
@jreback jreback added this to the 0.19.2 milestone Nov 12, 2016
@parthea
Copy link
Contributor

parthea commented Nov 12, 2016

At first glance, it appears that the test started failing 3 days ago. The test was written based on the assumption that BigQuery's streaming system caches the table schema for up to 2 minutes. This is mentioned in the following stackoverflow post.
http://stackoverflow.com/questions/25279116/cannot-insert-new-value-to-bigquery-table-after-updating-with-new-column-using-s

This assumption is documented in gbq.py.

# <https://code.google.com/p/google-bigquery/issues/detail?id=191>`__

It appears that this assumption is no longer valid. Another user mentioned a similar issue in Google BigQuery Issue 191:

Our team has noticed that updates are taking over 2 minutes now (more like ~10 minutes) as of the past 12 hours.

pandas.io.gbq.to_gbq() will also fail for end users under the following condition:

  • Users set the if_exists parameter of pandas.io.gbq.to_gbq() to 'replace'
  • The schema differs from the remote BigQuery schema
  • The BigQuery system cache is greater than 2 minutes (i.e. change in schema is not reflected in BigQuery).
    if_exists : {'fail', 'replace', 'append'}, default 'fail'
        'fail': If table exists, do nothing.
        'replace': If table exists, drop it, recreate it, and insert data.
        'append': If table exists, insert data. Create if does not exist.

@jreback
Copy link
Contributor Author

jreback commented Nov 13, 2016

ok we can mark this disabled so it's not run by default (but u can still run locally)
maybe also add s comment about the timeout as you describe above

@parthea
Copy link
Contributor

parthea commented Dec 29, 2016

I'd like to keep this issue open because this exact issue is currently open on the BigQuery bug tracking site.
https://code.google.com/p/google-bigquery/issues/detail?id=771

@jreback
Copy link
Contributor Author

jreback commented Feb 14, 2017

closed #15381

let's migrate this (and others) to pandas-gbq

@jreback jreback closed this as completed Feb 14, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Testing pandas testing functions or related to the test suite
Projects
None yet
Development

No branches or pull requests

2 participants