ENH: SQL through SQLAlchemy - performance

[mangecoeur]

This issue is to track performance measurement and improvements to the SQLalchemy-based SQL io support in pandas.

TODO

• Get perfromance measurements working
• Determine bottlenecks
• Address bottleneck

[mangecoeur]

First improvements to write performance #6417

[jreback]

ideally you can write these as vbenches ?

[cpcloud]

@mangecoeur GitHub has checkboxes! They're pretty neat.

[jreback]

should add to the ecosystem page? https://github.com/yhat/pandasql

[jreback]

@jorisvandenbossche for 0.14.1?

maybe add some vbenches in 0.14.0 if you guys can

[balancap]

I am not an SQL and SQLAlchemy expert, but I have observed that for any reading operation, a couple of requests to the database are needed before the SELECT

• SHOW FULL TABLES
• SHOW CREATE TABLE

I think it would be interesting to be able to optionally desactivate them, as from what I have measured, they can be costly if the database server is distant and has big latency.

I have not look yet at the implementation inside Pandas, but do you think it could be possible ?

[jorisvandenbossche]

@balancap created a new issue about this: #7396 (I suppose what I report there is the reason for these requests you report here. Do you think that could be correct?).

[mroeschke]

We have SQL benchmarks in our ASV suite. Closing