Skip to content

Accept SQLAlchemy Connection as con param in pd.read_sql_query() #8533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
davidljung opened this issue Oct 10, 2014 · 1 comment
Closed

Accept SQLAlchemy Connection as con param in pd.read_sql_query() #8533

davidljung opened this issue Oct 10, 2014 · 1 comment
Labels
Duplicate Report Duplicate issue or pull request IO SQL to_sql, read_sql, read_sql_query

Comments

@davidljung
Copy link
Contributor

Currently pd.read_sql_query() accepts an sqlalchemy.engine.base.Engine instance as the con parameter. Using that, it presumably creates a new sqlalchemy.engine.base.Connection to connect to the DB server and issue the query.
However, as most (all?) DB servers associate a session with a connection, this precludes issuing queries for an existing session/connection.

For example, it is common to break up large queries into a series of steps that create TEMPorary intermediate result tables, possibly joining some into further intermediate tables, and then querying for the final result. The lifetime and scope of a TEMP table is that of the session/connection.

Hence, something like:

con = engine.connect()
con.execute('CREATE TEMP TABLE tmptable ...')
result = pd.read_sql_query('SELECT * FROM tmptable ..', engine)

will fail as the tmptable isn't available to the connection/session created by read_sql_query().

Concatenating the SQL statements into one query string with ';' statement seperator, or using the SQLAlchemy text class to attempt to create the table and query it in one SQL query also fails as leads to SQLAlchemy claiming the query doesn't return results (perhaps because it doesn't being with "SELECT"?)

This, and other similar use-cases involving multiple queries within a single session/connection context, could be easily supported by allowing a sqlalchemy.engine.base.Connection to be passed to the read_sql_query() and similar functions, whereby they would use the supplied connection rather than creating a new one. An added benefit would be the reduction in overhead of repeatedly creating and tearing down connections to the DB server and also allowing users to implement connection pooling when necessary.

(Issue created at suggestion of @joris in this StackOverflow question )

@jorisvandenbossche jorisvandenbossche added the IO SQL to_sql, read_sql, read_sql_query label Oct 10, 2014
@jorisvandenbossche
Copy link
Member

Actually, there was already an issue for this: #7877. So closing as a duplicate (but certainly comment there!)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Duplicate Report Duplicate issue or pull request IO SQL to_sql, read_sql, read_sql_query
Projects
None yet
Development

No branches or pull requests

2 participants