Possible performance regression in GitPython 3.0.0

[jeblair]

I have observed that the test run time on the Zuul project, which makes heavy use of GitPython, has increased by a factor of 1.5 (30 minutes under 2.1.13 to 45 minutes under 3.0.0).

[jeblair]

It looks like the problem may be that many more GitPython actions now result in a call to "git rev-parse" to find the config file location.

I constructed a test script based on one of our unit tests and see the following commands under 2.1.13:

DEBUG:git.cmd:Popen(['git', 'init'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=<valid stream>)
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=<valid stream>)
DEBUG:git.cmd:Popen(['git', 'reset', '--hard', 'HEAD', '--'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'clean', '-x', '-f', '-d'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'clone', '/tmp/upstream', '/tmp/downstream'], cwd=/home/corvus/git/zuul/zuul, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'remote', 'prune', '--dry-run', 'origin'], cwd=/tmp/downstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/downstream, universal_newlines=False, shell=None, istream=<valid stream>)

And the same under 3.0.0:

DEBUG:git.cmd:Popen(['git', 'init'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'rev-parse', '--git-path', 'config'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'rev-parse', '--git-path', 'config'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=<valid stream>)
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=<valid stream>)
DEBUG:git.cmd:Popen(['git', 'reset', '--hard', 'HEAD', '--'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'clean', '-x', '-f', '-d'], cwd=/tmp/upstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'clone', '/tmp/upstream', '/tmp/downstream'], cwd=/home/corvus/git/zuul/zuul, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'rev-parse', '--git-path', 'config'], cwd=/tmp/downstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'rev-parse', '--git-path', 'config'], cwd=/tmp/downstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'rev-parse', '--git-path', 'config'], cwd=/tmp/downstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'remote', 'prune', '--dry-run', 'origin'], cwd=/tmp/downstream, universal_newlines=False, shell=None, istream=None)
DEBUG:git.cmd:Popen(['git', 'cat-file', '--batch-check'], cwd=/tmp/downstream, universal_newlines=False, shell=None, istream=<valid stream>)

It looks like #894 introduced the change here: https://github.com/gitpython-developers/GitPython/pull/894/files#diff-c276fc3c4df38382ec884e59657b869dR450-R458

[Byron]

Does @bdauvergne have an idea how the original issue could be fixed without repeated requests? Is caching possible?

For now I would revert the commit as the performance impact seems substantial, but hope it can be re-added with some sort of caching in place.

This seems to favour performance over fixing an issue with correctness, but I hope we will get a cached version of the reverted fix at some point.