Inconsistent index between plain DataFrame and read_sql DataFrame

[weikhor]

• closes BUG: inconsistent index between plain DataFrame and read_sql DataFrame #47608
• Tests added and passed if fixing a bug or adding a new feature
• All code checks passed.
• Added type annotations to new arguments/methods/functions.
• Added an entry in the latest doc/source/whatsnew/vX.X.X.rst file if fixing a bug or adding a new feature.

[mroeschke]

Would be good to have the sql smoke test from the original issue as well.

[phofl]

Do we want this? This seems to be deliberate.

I think the if isn't needed at all, if we want a RangeIndex?

[phofl]

I am -1 on this change without deprecation and -0 on this change in general.

data = {1: ["foo"], 2: ["bar"]}
exp = DataFrame(data, columns=["a", "b"])

DataFrame()

These operations create regular Index objects, not a RangeIndex, this change would cause inconsistencies.

You can also see this on the number of tests you had to change for this

[weikhor]

ok. I try to search other method.

[weikhor]

Hi @phofl, when I try this.
1)

data = {"a": [], "b": []}
exp = pd.DataFrame(data, columns=["a", "b"])
print(exp)
print(exp.index)

Empty DataFrame
Columns: [a, b]
Index: []
RangeIndex(start=0, stop=0, step=1)

data = {1: ["foo"], 2: ["bar"]}
exp = pd.DataFrame(data, columns=["a", "b"])
print(exp)
print(exp.index)

Empty DataFrame
Columns: [a, b]
Index: []
Index([], dtype='object')

On current main branch, index type created is not same. Is this expected?

[phofl]

Can you create an overview of all possible cases and then add it to the issue? We should aim for consistency if we make a change

E.g. reading empty csv for example should be consistent too

[weikhor]

Hi @phofl Possible cases of empty dataframe is as follow. I based on documentation

class pandas.DataFrame(data=None, index=None, columns=None, dtype=None, copy=None)

Parameters
    data : ndarray (structured or homogeneous), Iterable, dict, or DataFrame
           Dict can contain Series, arrays, constants, dataclass or list-like objects. If data is a dict, column order follows insertion- 
           order. If a dict contains Series which have an index defined, it is aligned by its index.

1. Iterable

data = []
df = pd.DataFrame(data, columns=["A", "B"])
print(df.index)

Index([], dtype='object')

2. Dict

a) using series

data = {'a': pd.Series([]), 'B': pd.Series([])}
df = pd.DataFrame(data, columns=["A", "B"])
print(df.index)

RangeIndex(start=0, stop=0, step=1)

b) using list

data = {"A": [], "B": []}
df = pd.DataFrame(data)
print(df.index)

RangeIndex(start=0, stop=0, step=1)

c) using array

data = {'A': np.array([]), 'B': np.array([])}
df = pd.DataFrame(data)
print(df.index)

d) based on columns

data = {1: ["foo"], 2: ["bar"]}
df = pd.DataFrame(data, columns=["A", "B"])
print(df.index)

Index([], dtype='object')

4. Read csv

df = pd.read_csv("empty.csv")
print(df.index)

Index([], dtype='object')

[weikhor]

Based on https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html,

index  : Index or array-like
         Index to use for resulting frame. Will default to RangeIndex if no indexing information part of input data and no index 
         provided.

The default is RangeIndex when no index provided

[mroeschke]

Thanks for your investigation so far @weikhor! I think your findings also intersect with #47959 so would be good to document your findings there as well.

I think this topic needs more discussion in the issues before moving forward with a fix, so closing. But thanks for your effort here so far.