Skip to content

ValueError: need at least one array to concatenate When Returned Dataset is Emtpy #492

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hbc-acai opened this issue Jan 17, 2025 · 0 comments

Comments

@hbc-acai
Copy link

I notice when a query returns empty dataset, the fetchall function fails with ValueError: need at least one array to concatenate. The expected behavior is to return an empty list.

python 3.8
databricks-sql-connector==3.7.1
pyarrow==17.0.0

   sql_query = f"""
                SELECT Column FROM table
                WHERE 1=0"""

host = os.getenv("DATABRICKS_HOST")
http_path = os.getenv("DATABRICKS_HTTP_PATH")

connection = sql.connect(
  server_hostname=host,
  http_path=http_path)
    with conn.cursor() as cursor:
        cursor.execute(sql_query)
        rows = cursor.fetchall()
  File "c:\Users\acai\.venv\lib\site-packages\databricks\sql\client.py", line 1066, in fetchall
    return self.active_result_set.fetchall()
  File "c:\Users\acai\.venv\lib\site-packages\databricks\sql\client.py", line 1431, in fetchall
    return self._convert_arrow_table(self.fetchall_arrow())
  File "c:\Users\acai\.venv\lib\site-packages\databricks\sql\client.py", line 1302, in _convert_arrow_table
    df = table_renamed.to_pandas(
  File "pyarrow\\array.pxi", line 885, in pyarrow.lib._PandasConvertible.to_pandas
  File "pyarrow\\table.pxi", line 5002, in pyarrow.lib.Table._to_pandas
  File "c:\Users\acai\.venv\lib\site-packages\pyarrow\pandas_compat.py", line 800, in table_to_dataframe
    blocks = [
  File "c:\Users\acai\O.venv\lib\site-packages\pyarrow\pandas_compat.py", line 801, in <listcomp>
    _reconstruct_block(item, column_names, ext_columns_dtypes)
  File "c:\Users\acai\.venv\lib\site-packages\pyarrow\pandas_compat.py", line 743, in _reconstruct_block
    arr = pandas_dtype.__from_arrow__(arr)
  File "c:\Users\acai\.venv\lib\site-packages\pandas\core\arrays\integer.py", line 121, in __from_arrow__
    return IntegerArray._concat_same_type(results)
  File "c:\Users\acai\\.venv\lib\site-packages\pandas\core\arrays\masked.py", line 271, in _concat_same_type
    data = np.concatenate([x._data for x in to_concat])
  File "<__array_function__ internals>", line 200, in concatenate
ValueError: need at least one array to concatenate

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant