Skip to content

fixes for cloud fetch - part un #356

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 16, 2024

Conversation

andrefurlan-db
Copy link
Contributor

@andrefurlan-db andrefurlan-db commented Feb 16, 2024

This is a first set of fixes for cloud fetch.

  • Throw when failed to download file
  • Retry properly while downloading file
  • Add a bunch of debug logs
  • Added URL redacting
  • Prevent thread issues

TODO:

  • check rows context to avoid duplicates
  • http connection pools for cloud storage, proxies, etc.

Backported to version 2

backport to version 2

Signed-off-by: Andre Furlan <[email protected]>
@andrefurlan-db andrefurlan-db marked this pull request as ready for review February 16, 2024 02:23
@andrefurlan-db andrefurlan-db requested review from benc-db and jackyhu-db and removed request for arikfr February 16, 2024 02:25
@andrefurlan-db andrefurlan-db changed the title fixes for cloud fetch fixes for cloud fetch - part un Feb 16, 2024
Signed-off-by: Andre Furlan <[email protected]>
@andrefurlan-db andrefurlan-db merged commit 6a348ec into databricks:2.9.4 Feb 16, 2024
andrefurlan-db added a commit to andrefurlan-db/databricks-sql-python that referenced this pull request Feb 21, 2024
* fixes for cloud fetch

Signed-off-by: Andre Furlan <[email protected]>
---------

Signed-off-by: Andre Furlan <[email protected]>
Co-authored-by: Raymond Cypher <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants