-
Notifications
You must be signed in to change notification settings - Fork 417
Disabling prepared statements entirely with SQLAlchemy? #1041
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Prepared statements are an integral part of PostgreSQL extended query protocol which asyncpg uses. That said, I'm not sure why you would want to disable those as that flow should work just fine with pgbouncer. Can you give a more concrete example of a problem you're seeing? |
elprans
added a commit
that referenced
this issue
Aug 16, 2023
It appears that PgBouncer's `transaction` pooling mode does not consider implicit transactions properly, and so in a [`Parse`, `Flush`, `Bind`, `Execute`, `Sync`] sequence, `Flush` would be (incorrectly) considered by PgBouncer as a transaction boundary and it will happily send the following `Bind` / `Execute` messages to a different backend process. This makes it so that when `statement_cache_size` is set to `0`, asyncpg assumes a pessimistic stance on prepared statement persistence and does not rely on them even in implicit transactions. The above message sequence thus becomes `Parse`, `Flush`, `Parse` (a second time), `Bind`, `Execute`, `Sync`. This obviously has negative performance impact due to the extraneous `Parse`. Fixes: #1058 Fixes: #1041
elprans
added a commit
that referenced
this issue
Aug 16, 2023
It appears that PgBouncer's `transaction` pooling mode does not consider implicit transactions properly, and so in a [`Parse`, `Flush`, `Bind`, `Execute`, `Sync`] sequence, `Flush` would be (incorrectly) considered by PgBouncer as a transaction boundary and it will happily send the following `Bind` / `Execute` messages to a different backend process. This makes it so that when `statement_cache_size` is set to `0`, asyncpg assumes a pessimistic stance on prepared statement persistence and does not rely on them even in implicit transactions. The above message sequence thus becomes `Parse`, `Flush`, `Parse` (a second time), `Bind`, `Execute`, `Sync`. This obviously has negative performance impact due to the extraneous `Parse`. Fixes: #1058 Fixes: #1041
elprans
added a commit
that referenced
this issue
Aug 17, 2023
…1065) It appears that PgBouncer's `transaction` pooling mode does not consider implicit transactions properly, and so in a [`Parse`, `Flush`, `Bind`, `Execute`, `Sync`] sequence, `Flush` would be (incorrectly) considered by PgBouncer as a transaction boundary and it will happily send the following `Bind` / `Execute` messages to a different backend process. This makes it so that when `statement_cache_size` is set to `0`, asyncpg assumes a pessimistic stance on prepared statement persistence and does not rely on them even in implicit transactions. The above message sequence thus becomes `Parse`, `Flush`, `Parse` (a second time), `Bind`, `Execute`, `Sync`. This obviously has negative performance impact due to the extraneous `Parse`. Fixes: #1058 Fixes: #1041
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have some SQL strings with multiple commands (i.e.
INSERT a; INSERT B;
), which cannot be made into prepared statements.When using SQLAlchemy, it seems like
asyncpg
is always called in such a way that prepared statements are always generated.There is an issue on a related subject, where the
prepared_statement_cache_size
option can be used to avoid cacheing the statements. But this setting does not stop them from being used entirely. Most of the above issue is only about changing the way such statements are named/identified/stored. Even with this set to0
, the statements are still created and called.The only way I can find to get around this problem is to call asyncpg more directly:
Is there any nicer solution for this? I'd like to skip
get_raw_connection
,.cursor()
and_connection
, and use SQLAlchemy more as intended, but without generating any prepared statements.Also, a somewhat related question, does SQLAlchemy provide a way of accessing asyncpg's
copy_to_table
other than the above? I can't find much documentation about this, nor anything specific toCOPY
in the source.The text was updated successfully, but these errors were encountered: