Django KeyError <class 'bytes'> in version 1.4.0rc1

[dev-artisan]

Hi,
Just updated my dev server to the latest version (1.4.0rc1) and my Django server crashed when trying to connect to the local MySQL DB installed on Ubuntu server v16
The error occurs in the code below from django/db/backends/mysql/base.py

def get_new_connection(self, conn_params):
        conn = Database.connect(**conn_params)
        conn.encoders[SafeText] = conn.encoders[six.text_type]
        conn.encoders[SafeBytes] = conn.encoders[bytes]
        return conn

Specifically this line
conn.encoders[SafeBytes] = conn.encoders[bytes]

I've reverted back to 1.13.14 and all is well in the world again, hope this helps
Thanks
Omar

[methane]

Thank you for testing RC. It's intended, minor backward incompatible chage.
And the code is removed in Django almost two years ago.
django/django@5b95d42#diff-dbe1511db54b8fff2cb72a199cf37b4c

You need to update Django if you want to use new mysqlclient.

[keviny]

How to resolve this?

[methane]

Update Django.

[dev-artisan]

How to resolve this?

If you're like me and want to stay on long term Django release 1.11, don't update this library to version 1.4.
Stick to version 1.3.14 until you decide to update to Django 2

[fsh905]

I met the same problem,

File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 276, in get_new_connection
    conn.encoders[SafeBytes] = conn.encoders[bytes]
KeyError: <type 'str'>

salved by update mysqlclient from 1.4.1 to 1.3.14

[beamerblvd]

I would like to formally protest to the "Upgrade Django" sentimentality, on two fronts.

First, 1.4.0 is a minor release. According to Semver, introducing backwards-incompatible changes in a minor release is not permitted. By putting a breaking change in 1.4.0, and then denying this is a problem and telling people to upgrade Django, this project is publicly stating that it does not care about / does not adhere to Semver.

Second, Django 1.11 is still a supported version of Django. It is in LTS. Telling people to "Upgrade Django" might be warranted if this error appeared only in a very old, unsupported version of Django (such as 1.5, 1.6, etc.), but Django 1.11 is still LTS until April 2020. In fact, Django 1.11 is the newest version of Django that is in LTS. Django 2.0 and 2.1 support will actually end before Django 1.11 support ends. Django 2.2 isn't out yet. So, one can rightly argue that Django 1.11 is the "current" version of Django, support-wise.

Literally tens of thousands, maybe hundreds of thousands, of developers and sys admins worldwide depend on either Django 1.11 or MySQLClient-Python, and a substantial subset of those depend on both of them together. This cavalier disregard for those developers by introducing a breaking change in a minor release is disheartening.

In my strongly-held opinion, this breaking change should be reverted and that revert immediately released in 1.4.2, and the breaking change should be introduced only in MySQLClient-Python 2.0, in accordance with Semver.

[methane]

Django touched many undocumented or private part of mysqlclient. It caused this crash. But I don't think it's a serious backward incompatibility of mysqlclient-python. Backward compatibility is supported only when public APIs are used, and don't relay on internal strucutre.

Additionally, Django 1.11 doesn't support mysqlclient-1.3.14+ already. See these commit and comment.

• django/django@2ea1e0e
• Fixed #30013 -- Fixed DatabaseOperations.last_executed_query() with mysqlclient 1.3.14+. django/django#10726 (comment)

Since Django 1.11 doesn't support mysqlclient-1.4 at first, there are no reason to add workaround for Django 1.11 in mysqlclient 1.4.

If Django maintainer want Django works with mysqlclient-1.3.14+ and 1.4.1+, I am OK to add some workaround in mysqlclient.

If you want to use Django 1.11 with mysqlclient 1.4+, discuss with Django maintainers about it and backport django/django#10726 to Django 1.11.

[methane]

Second, Django 1.11 is still a supported version of Django.

It is not trully supported. It is security fix only mode. It doesn't fix any issues including this.

the breaking change should be introduced only in MySQLClient-Python 2.0, in accordance with Semver.

It is impossible. When I change query slightly for bugfix, it can be "breaking change" for some people.
And even removing unused private variable broke Django 1.11!

If I follow semver strongly, it means I need to increment major number every time. It doesn't make happy anyone.
It is difficult to know when really big breaking change comes (e.g. dropping Python 2 support).
And people need to pin version anyway.

So this project never follow semver. It doesn't make any sense.

---

But I'm thinking how to work around these breakage, not for semver, but for Django 1.11 users.

If you're interested in contributing this project for Django user, please send pull request to add
travis test running Django test. I don't have motivation for it because I'm not a Django user.

[methane]

Everyone complain when theoricary backward-compatible change caused crash
because of Django implementation. But no one contributes.

OSS maintainer is very frustrating job.

[kierun]

OSS maintainer is very frustrating job.

Yes, it is. And your efforts are appreciated (at least by me!) as well as all your past hard work. If you want, I can test your PR with my setup which has the same error.

To be fair, Django developers should use you API not private variables. Doing so is beyond stupid.

LTR are great because you do not have to justify work to "update to the latest release" every time a new one is released. It is bad because you have to do a whole chunk later on but at least it's a one off.

[methane]

1.4.2post1 fixed this issue. Passes Django 1.11 test suite.