Django: upgrade to 4.2

[humitos]

Run django-upgrade (https://github.com/adamchainz/django-upgrade) to automatically upgrade our code to match 4.2.

I made each commit to be valuable for review. Note the first commit contains all the auto-changes from django-upgrade. So, that's probably a good one to skip.

Important notes from the release notes

I read all the release notes and wrote down the ones that I found important (e.g. affect our application) here as a summary.

Django 4.0

https://docs.djangoproject.com/en/4.2/releases/4.0/

• use scrypt, but PASSWORD_HASHERS is not defined in our settings
  https://docs.djangoproject.com/en/4.2/topics/auth/passwords/#scrypt-usage
• RedisCache backend built-in

Django 4.1

https://docs.djangoproject.com/en/4.2/releases/4.1/

• We can now use SECRET_KEY_FALLBACKS to rotate our key as we wanted

Django 4.2

https://docs.djangoproject.com/en/4.2/releases/4.2/

• It supports psycopg version 3.1.8 or higher
  https://www.psycopg.org/psycopg3/docs/basic/from_pg2.html

  There are some breaking changes that we should take a look at.
  We can leave this for a next iteration, tho.

Related:

• https://github.com/readthedocs/readthedocs-ext/pull/548
• https://github.com/readthedocs/readthedocs-corporate/pull/1629
• https://github.com/readthedocs/readthedocs-ops/pull/1382
• Django: upgrade to 4.2 ext-theme#195
• Use django-upgrade pre-commit hook common#185
• Install testing.txt requirements common#184

[humitos]

I pushed a new commit that fixes a bunch of tests, but there are still 11 that are failing. Some of them are related to privacy levels that I wasn't able to fix 🤷🏼

Besides, tests using fake_paths_by_regex are failing when checking os.path.exists.

This is pretty close. I'm pinging @stsewd here in case he knows what I'm missing at this point.

[humitos]

All tests are passing ✨ 🍰 !

@stsewd @ericholscher I'd really appreciate an approve here so we can move forward with this and test it in production. It seems we don't have a ton of things to deploy this Tuesday, so it may be a good opportunity to do a deploy without many other changes together.

[ericholscher]

This looks pretty straightforward with the test header refactor, re_path removal, and admin changes. I had a couple questions, but overall seems like it shouldn't impact much.

[ericholscher]

Some of this PR seems to remove re_path, but this just changes the import?

[humitos]

Yes. re_path was moved around and it's now under a different import path.

The usage of re_path here cannot be changed by path directly because we have non-simple/standard regex that django-upgrade can detect and adapt to just path. That's my bet, tho.

[humitos]

I did some QA yesterday by spinning up some instances in production and routing some traffic to them. I was able to fix some obvious issues and after that the instances were healthy. I'll merge all these Django upgrade PRs so we can start testing them more locally before deploying them.