BUG: using at indexer reassign existing field error

[agodbug]

Pandas version checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pandas.

• I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
df = pd.DataFrame({'a': [1,2,3], 'b':[4,5,6]})
df.at[slice(0, 1, None), 'c'] = 7  # GOOD
df.at[slice(0, 1, None), 'c'] = 8  # BAD

```python-traceback
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/Caskroom/miniconda/4.6.14/miniconda3/envs/test/lib/python3.9/site-packages/pandas/core/indexing.py", line 2281, in __setitem__
    return super().__setitem__(key, value)
  File "/usr/local/Caskroom/miniconda/4.6.14/miniconda3/envs/test/lib/python3.9/site-packages/pandas/core/indexing.py", line 2236, in __setitem__
    self.obj._set_value(*key, value=value, takeable=self._takeable)
  File "/usr/local/Caskroom/miniconda/4.6.14/miniconda3/envs/test/lib/python3.9/site-packages/pandas/core/frame.py", line 3869, in _set_value
    loc = self.index.get_loc(index)
  File "/usr/local/Caskroom/miniconda/4.6.14/miniconda3/envs/test/lib/python3.9/site-packages/pandas/core/indexes/range.py", line 388, in get_loc
    self._check_indexing_error(key)
  File "/usr/local/Caskroom/miniconda/4.6.14/miniconda3/envs/factor/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 5637, in _check_indexing_error
    raise InvalidIndexError(key)
pandas.errors.InvalidIndexError: slice(0, 1, None)

### Issue Description

It shows us create a dataframe, then try add a column c and  assign some value to special index using **.at[]** operation.
first time it is GOOD, but then next time i try to fill the other value even in the same index we got an InvalidIndexError.

### Expected Behavior

no errors, and result should be:
   a  b    c
0  1  4  8.0
1  2  5  8.0
2  3  6  NaN

### Installed Versions

<details>

INSTALLED VERSIONS
------------------
commit           : e8093ba372f9adfe79439d90fe74b0b5b6dea9d6
python           : 3.9.1.final.0
python-bits      : 64
OS               : Darwin
OS-release       : 20.6.0
Version          : Darwin Kernel Version 20.6.0: Wed Jun 23 00:26:31 PDT 2021; root:xnu-7195.141.2~5/RELEASE_X86_64
machine          : x86_64
processor        : i386
byteorder        : little
LC_ALL           : None
LANG             : None
LOCALE           : None.UTF-8

pandas           : 1.4.3
numpy            : 1.21.2
pytz             : 2021.3
dateutil         : 2.8.2
setuptools       : 58.0.4
pip              : 21.2.4
Cython           : 0.29.24
pytest           : None
hypothesis       : None
sphinx           : None
blosc            : None
feather          : None
xlsxwriter       : None
lxml.etree       : None
html5lib         : None
pymysql          : 1.0.2
psycopg2         : None
jinja2           : 3.0.2
IPython          : 7.28.0
pandas_datareader: None
bs4              : 4.10.0
bottleneck       : 1.3.4
brotli           : 1.0.9
fastparquet      : None
fsspec           : 2021.10.1
gcsfs            : None
markupsafe       : 2.0.1
matplotlib       : 3.5.1
numba            : 0.55.1
numexpr          : 2.8.1
odfpy            : None
openpyxl         : 3.0.9
pandas_gbq       : None
pyarrow          : 4.0.1
pyreadstat       : None
pyxlsb           : None
s3fs             : None
scipy            : 1.7.1
snappy           : None
sqlalchemy       : 1.3.24
tables           : 3.7.0
tabulate         : 0.8.9
xarray           : None
xlrd             : 2.0.1
xlwt             : None
zstandard        : None

</details>

[phofl]

Hi, thanks for your report.

This is buggy and should not work at all imo. At is for single values, you are setting 2 values in both cases, which should raise

[phofl]

cc @jbrockmendel

[jbrockmendel]

df.at[slice(0, 1, None), 'c'] = 7 should raise