Skip to content

BUG: Cannot connect to sql #51086

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
2 of 3 tasks
Edimo05 opened this issue Jan 31, 2023 · 6 comments
Closed
2 of 3 tasks

BUG: Cannot connect to sql #51086

Edimo05 opened this issue Jan 31, 2023 · 6 comments
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member

Comments

@Edimo05
Copy link

Edimo05 commented Jan 31, 2023

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import dask.dataframe as dd

engine = create_engine("mssql+pyodbc://{user}:{pw}@{sv}:1433/{db}?driver=ODBC+DRIVER+17+for+SQL+Server"
                       .format(user="sa",
                               pw="sa",
                               db="table",
                               sv='server.com'), pool_pre_ping=True)

# Get the list of CSV files in the folder
csv_files = [f for f in os.listdir(dir_folder) if f.endswith('.csv')]

# Loop through the CSV files in the folder
for file in csv_files:
    # Read the CSV file into a Dask DataFrame
    df = dd.read_csv(f'{dir_folder}/{file}', usecols = cols, dtype = converters)

	# Fill empty rows with 0
    df = df.fillna(0)

    # Rename the columns
    df = df.rename(columns = new_column_names)

    # Write the DataFrame to the SQL table
    df.to_sql(table, engine, if_exists='append', index=False)

Issue Description

Issue related to connection, but as far as I know, I need t create the connection and send it through to_sql.

image

Expected Behavior

It supposed to connect the database and load the dataframe

Installed Versions

python : 3.10.4.final.0 pandas : 1.4.2
@Edimo05 Edimo05 added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Jan 31, 2023
@phofl
Copy link
Member

phofl commented Jan 31, 2023

Can you post in #40686?

@jbrockmendel
Copy link
Member

isnt this a dask issue?

@leo-schick
Copy link

@jbrockmendel no, it is a issue with pandas not supporting SQLAlchemy>=2.0 yet

@jlonge4
Copy link

jlonge4 commented Apr 4, 2023

@Edimo05 @phofl does this still exist?

@phofl
Copy link
Member

phofl commented Apr 4, 2023

Dask only supports strings as uri, no engines. So this works as expected

@phofl phofl closed this as completed Apr 4, 2023
@jlonge4
Copy link

jlonge4 commented Apr 4, 2023

@phofl thanks! I dug through and couldn't think of a single reason why not so that works 😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Needs Triage Issue that has not been reviewed by a pandas team member
Projects
None yet
Development

No branches or pull requests

5 participants