Inspecting the index during a pre-commit is empty

[robin-moss]

I've written a small script to work with pre-commit, which when run locally seems to work but when I run it through pre-commit the git index is empty and I get errors.

The issue I'm seeing is when I run the script via PyCharm the checks pass, but when it's run automatically by pre-commit the meta_file_in_index fails because the index appears to be empty.

Any advice on what I'm doing wrong or suggestions for digging deeper into what's going on would be greatly appreciated.

What I've done so far:

• Validated it's the right path
• Validating it's on the same branch

The issue raised with pre-commit: pre-commit/pre-commit#2184

Source Code:

#!/usr/bin/env python3
import os
import sys

from git import Repo, IndexFile, DiffIndex


def filter_assets(path: str):
    lower_case = path.lower()
    return lower_case.startswith("assets")


def path_to_meta_file(path: str):
    return path + ".meta"

def meta_file_exists(meta_file: str):
    return os.path.isfile(meta_file)

def meta_file_in_index(meta_file: str, index: IndexFile):
    return any(x for x in index.entries.keys() if x[0] == meta_file)

def meta_file_in_diff(meta_file: str, diff: DiffIndex):
    return any(x for x in diff if x.a_path == meta_file)


def run():
    files = [path for path in sys.argv[1:] if filter_assets(path)]
    print(os.getcwd())
    repo = Repo(os.getcwd())
    index = repo.index
    modified = index.diff(None)
    errors = []
    for file in files:
        meta_file = path_to_meta_file(file)
        if not meta_file_exists(meta_file):
            errors.append(f"{file} does not have a matching meta file {meta_file}")
            continue

        if not meta_file_in_index(meta_file, index):
            errors.append(f"{meta_file} is not in the git index")
            continue

        if meta_file_in_diff(meta_file, modified):
            errors.append(f"{meta_file} has been changed but is not staged")

    if len(errors) > 0:
        print("At least one file has an issue with it's meta file:")
        print("  * " + "\n  * ".join(errors))
        sys.exit(1)


if __name__ == "__main__":
    run()

[Byron]

Thanks for the clear issue description, very informative!

It's not the first time I hear that git hooks aren't working as they should depending on how executes them. Most often they fail in cron jobs though.

Since the index appears to be empty, I can imagine it get the wrong path. Now it's mentioned that the 'right path' was validated already, so is it correct that that repo.git_dir is what it should be?

If it is, you might try to re-throw this OSError as it's the only reason it would yield an empty index. Unfortunately it doesn't check the actual cause of the error, it should only yield an empty index if the file isn't present yet, which should barely ever be happening.

[robin-moss]

So adding the code:

    lfd = LockedFD(index._file_path)
    lfd.open(write=False, stream=False)

led to a stack trace (below), which mentions that there is already an index.lock, which would actually make sense as we're in a "pre-commit" hook so in the middle of a Git operation 🤦. Is it possible to run in a "readonly" state that wouldn't require the index lock?

Otherwise I'll have to reconsider my approach to checking the index

Stack trace:

Traceback (most recent call last):
  File "/tmp/tmpgwk46x0q/repo_aps1mss/py_env-python3.9/lib/python3.9/site-packages/gitdb/util.py", line 323, in open
    fd = os.open(self._lockfilepath(), lockmode, int("600", 8))
FileExistsError: [Errno 17] File exists: '/home/robin.moss/repo/.git/index.lock'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/tmp/tmpgwk46x0q/repo_aps1mss/py_env-python3.9/bin/meta_check", line 8, in <module>
    sys.exit(run())
  File "/tmp/tmpgwk46x0q/repo_aps1mss/py_env-python3.9/lib/python3.9/site-packages/unity_pre_commit/meta_check.py", line 41, in run
    lfd.open(write=False, stream=False)
  File "/tmp/tmpgwk46x0q/repo_aps1mss/py_env-python3.9/lib/python3.9/site-packages/gitdb/util.py", line 330, in open
    raise IOError("Lock at %r could not be obtained" % self._lockfilepath()) from e
OSError: Lock at '/home/robin.moss/repo/.git/index.lock' could not be obtained

[Byron]

I have created a new release which simply won't take a read lock anymore - it's not required at all, to solve this particular problem.

[robin-moss]

Thank you, that fixed my issue :)