Code

SI: Implement put operations#67

Code

Merged

susodapop merged 41 commits intosidatabricks/databricks-sql-python:sifrom

PECO-397databricks/databricks-sql-python:PECO-397Copy head branch name to clipboard

Dec 30, 2022

Commits

Commits on Nov 15, 2022

• Basic PUT operation. Currently this never executes because the server

  Show description for 28c3a59
  

  Jesse Whitehouse

  committed
  28c3a59

  Copy full SHA for 28c3a59

• Bump Spark CLI service protocol version being used.

  Show description for 1b245b1
  

  Jesse Whitehouse

  committed
  1b245b1

  Copy full SHA for 1b245b1

• Log when attempting a staging operation

  Show description for 1239def
  

  Jesse Whitehouse

  committed
  1239def

  Copy full SHA for 1239def

Commits on Nov 16, 2022

• Fix failing unit tests since function signature for ExecuteResponse changed

  Show description for b605cce
  

  Jesse Whitehouse

  committed
  b605cce

  Copy full SHA for b605cce

• Add e2e test for put.

  Show description for 3ed84d8
  

  Jesse Whitehouse

  committed
  3ed84d8

  Copy full SHA for 3ed84d8

• Bail on tests if staging_ingestion_user is not set

  Show description for 57b8a34
  

  Jesse Whitehouse

  committed
  57b8a34

  Copy full SHA for 57b8a34

• Black client.py

  Show description for 7812278
  

  Jesse Whitehouse

  committed
  7812278

  Copy full SHA for 7812278

Commits on Nov 17, 2022

• Add unit test that sanity checks _handle_staging_operation is called

  Show description for 6b76439
  

  Jesse Whitehouse

  committed
  6b76439

  Copy full SHA for 6b76439

• Fix imports so that this module can be run independently:

  Show description for 3df7c89
  

  Jesse Whitehouse

  committed
  3df7c89

  Copy full SHA for 3df7c89

• Implement GET operation

  Show description for 8f0a02e
  

  Jesse Whitehouse

  committed
  8f0a02e

  Copy full SHA for 8f0a02e

Commits on Nov 23, 2022

• Refactor client.py into distinct methods for each ingestion command type

  Show description for 55525cb
  

  Jesse Whitehouse

  committed
  55525cb

  Copy full SHA for 55525cb

• Update pypoetry so I can develop on Python 3.10

  Show description for 157ac3d
  

  Jesse Whitehouse

  committed
  157ac3d

  Copy full SHA for 157ac3d

• Applied PR feedback around explicit response codes.

  Show description for 0739ccc
  

  Jesse Whitehouse

  committed
  0739ccc

  Copy full SHA for 0739ccc

• Applying PR feedback

  Show description for d3a3651
  

  Jesse Whitehouse

  committed
  d3a3651

  Copy full SHA for d3a3651

• PR feedback

  Show description for 72f917e
  

  Jesse Whitehouse

  committed
  72f917e

  Copy full SHA for 72f917e

• Black client.py

  Show description for fba64b7
  

  Jesse Whitehouse

  committed
  fba64b7

  Copy full SHA for fba64b7

• Refactor e2e test to use a single teste for PUT, GET, and REMOVE

  Show description for c27a3d6
  

  Jesse Whitehouse

  committed
  c27a3d6

  Copy full SHA for c27a3d6

• Make REMOVE command work

  Show description for 19ca706
  

  Jesse Whitehouse

  committed
  19ca706

  Copy full SHA for 19ca706

• These methods don't need to know the `operation`

  Show description for 0167bd9
  

  Jesse Whitehouse

  committed
  0167bd9

  Copy full SHA for 0167bd9

• Remove single quote that broke query

  Show description for 85e4d7c
  

  Jesse Whitehouse

  committed
  85e4d7c

  Copy full SHA for 85e4d7c

• Remove unneeded argument

  Show description for 713002d
  

  Jesse Whitehouse

  committed
  713002d

  Copy full SHA for 713002d

• Expect operation to succeed

  Show description for fc06ef8
  

  Jesse Whitehouse

  committed
  fc06ef8

  Copy full SHA for fc06ef8

• Black PySQLStagingIngestionTestSuite only

  Show description for cafa17d
  

  Jesse Whitehouse

  committed
  cafa17d

  Copy full SHA for cafa17d

• Tidy up comments in e2e test

  Show description for a508a1c
  

  Jesse Whitehouse

  committed
  a508a1c

  Copy full SHA for a508a1c

• Basic e2e test scaffolded in. Currently fails.

  Show description for ce80df0
  

  Jesse Whitehouse

  committed
  ce80df0

  Copy full SHA for ce80df0

• Only allow ingestion commands when base_uploads_path is specified

  Show description for 36885a4
  

  Jesse Whitehouse

  committed
  36885a4

  Copy full SHA for 36885a4

• Restrict local file operations to descendents of uploads_base_path

  Show description for c0c09d4
  

  Jesse Whitehouse

  committed
  c0c09d4

  Copy full SHA for c0c09d4

Commits on Dec 20, 2022

• Remove per PR feedback

  Show description for f612795
  

  Jesse Whitehouse

  committed
  f612795

  Copy full SHA for f612795

• Add check for null local_file per PR feedback

  Show description for e609ef3
  

  Jesse Whitehouse

  committed
  e609ef3

  Copy full SHA for e609ef3

• Open output stream _after_ successful HTTP request

  Show description for cdbe2d6
  

  Jesse Whitehouse

  committed
  cdbe2d6

  Copy full SHA for cdbe2d6

• Resolve relative paths before comparing row.localFile to uploads_base_path

  Show description for 34a0362
  

  Jesse Whitehouse

  committed
  34a0362

  Copy full SHA for 34a0362

• Add test that PUT fails if file exists in staging location and OVERWRITE not set

  Show description for c8a64c7
  

  Jesse Whitehouse

  committed
  c8a64c7

  Copy full SHA for c8a64c7

• Add tests: operations fail to modify another user's staging location

  Show description for d48d3f3
  

  Jesse Whitehouse

  committed
  d48d3f3

  Copy full SHA for d48d3f3

• Add test that ingestion command fails if local file is blank

  Show description for e0037e0
  

  Jesse Whitehouse

  committed
  e0037e0

  Copy full SHA for e0037e0

• Add test that invalid staging path will fail at server

  Show description for 3fa5d84
  

  Jesse Whitehouse

  committed
  3fa5d84

  Copy full SHA for 3fa5d84

Commits on Dec 22, 2022

• Basic usage example (needs tweaking)

  Show description for 4824b68
  

  Jesse Whitehouse

  committed
  4824b68

  Copy full SHA for 4824b68

• Add samples of GET and REMOVE

  Show description for 469f35f
  

  Jesse Whitehouse

  committed
  469f35f

  Copy full SHA for 469f35f

Commits on Dec 28, 2022

• Refactor to allow uploads_base_path to be either a single string object

  Show description for bdb948a
  

  Jesse Whitehouse

  committed
  bdb948a

  Copy full SHA for bdb948a

Commits on Dec 29, 2022

• Refactor uploads_base_path to staging_allowed_local_path

  Show description for 0261b7a
  

  Jesse Whitehouse

  committed
  0261b7a

  Copy full SHA for 0261b7a

Commits on Dec 30, 2022

• Fix mypy static type failures

  Show description for 00d8a49
  

  Jesse Whitehouse

  committed
  00d8a49

  Copy full SHA for 00d8a49

• Black src files

  Show description for 7a602e6
  

  Jesse Whitehouse

  committed
  7a602e6

  Copy full SHA for 7a602e6