Cloud Fetch download handler #127
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mattdeekay
requested review from
susodapop,
arikfr,
yunbodeng-db and
andrefurlan-db
as code owners
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
60b036d to
884bc0f
Compare
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
2ffb2ba to
d0e0037
Compare
Signed-off-by: Matthew Kim <[email protected]>
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
d0e0037 to
d2ffe1e
Compare
yunbodeng-db
requested review from
rcypher-databricks
and removed request for
arikfr
…frame markers Signed-off-by: Matthew Kim <[email protected]>
susodapop
reviewed
Jun 21, 2023
susodapop
reviewed
Jun 21, 2023
- Linting - Type annotations - Use response.ok - Log exception - Remove semaphore and only use threading.event - reset() flags method - Fix tests after removing semaphore - Link expiry logic should be in secs - Decompress data static function - link_expiry_buffer and static public methods - Docstrings and comments Signed-off-by: Matthew Kim <[email protected]>
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
f92860e to
17c813c
Compare
Nope, will be part of the full e2e PR |
Signed-off-by: Matthew Kim <[email protected]>
susodapop
approved these changes
Jun 27, 2023
Signed-off-by: Matthew Kim <[email protected]>
Signed-off-by: Matthew Kim <[email protected]>
mattdeekay
force-pushed
the
cloudfetch-downloader
branch
from
9872995 to
e8d2f77
Compare
susodapop
pushed a commit
to unj1m/databricks-sql-python
that referenced
this pull request
Sep 19, 2023
* Cloud Fetch download handler Signed-off-by: Matthew Kim <[email protected]> * Issue fix: final result link compressed data has multiple LZ4 end-of-frame markers Signed-off-by: Matthew Kim <[email protected]> * Addressing PR comments - Linting - Type annotations - Use response.ok - Log exception - Remove semaphore and only use threading.event - reset() flags method - Fix tests after removing semaphore - Link expiry logic should be in secs - Decompress data static function - link_expiry_buffer and static public methods - Docstrings and comments Signed-off-by: Matthew Kim <[email protected]> * Changing logger.debug to remove url Signed-off-by: Matthew Kim <[email protected]> * _reset() comment to docstring Signed-off-by: Matthew Kim <[email protected]> * link_expiry_buffer -> link_expiry_buffer_secs Signed-off-by: Matthew Kim <[email protected]> --------- Signed-off-by: Matthew Kim <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
CloudFetch download handler. Each handler will run on a thread and a CloudFetch download manager will maintain a thread pool.