-
-
Notifications
You must be signed in to change notification settings - Fork 398
some files are locked until python interpreter exits #596
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
The repository's object db keeps a few files open (mapped into memory) so it doesn't have to keep opening and closing files whenever it needs to look up an object. But this is not tied to the python interpreter, but to the lifetime of the repository. If you force collecting the repository, those file handles will be closed. You haven't said which files are the problem, what OS you're on or what the specific error message says, so I'm assuming you're on Windows as that is the OS where we would lock files by keeping them open. If this is not the case, the please provide that information. |
Sorrry for not being detailed enough. I'm on Windows10 64Bit with Python 3.5 (32bit) or Python 2.7 Stackless (32bit). The files are in
I'm new to git/pygit2, can you explain me what do you mean by force collecting the repository? |
That's a python/garbage-collection thing. The repository object isn't necessarily freed immediately when you stop using it. The interpreter may decide to do it at a later time if it doesn't think there is a lot of memory pressure. |
Ok, thank you for taken the time and the explanation. |
* Basing on #243, we are having some difficulty with pygit2 Repository file handlers being open. in `pygit.py` we instantiate the Repository class, which holds references to `pack` and `.idx` git objects and keeps files open. If taf and cleanup is attempted from same subprocess, Repository handler won't be deleted, and cleanup error occurs. Calling manual garbage collection did not help. This seems to be a known issue throughout the git community, see [1] and [2]. [1] - libgit2/pygit2#596 [2] - gitpython-developers/GitPython#546 Several workarounds were proposed by the community, but it seems that we can't fully replicate those workarounds. In [2] they attempt to workaround the issue by deleting git cache. That works for them because their git implementation has file handlers open in pure python, while we only do subprocess git calls. Our TODO: if these windows specific error issues keep persisting, figure out how to garbage collect pygit2 Repository handler correctly. For now settle on error handling, as this issue is specific to a subprocess call that calls taf/updater. Though we should keep track of progress of these known bugs, so we can fix them if they get addressed by pygit2 or gitpython
* Basing on #243, we are having some difficulty with pygit2 Repository file handlers being open. in `pygit.py` we instantiate the Repository class, which holds references to `pack` and `.idx` git objects and keeps files open. If taf and cleanup is attempted from same subprocess, Repository handler won't be deleted, and cleanup error occurs. Calling manual garbage collection did not help. This seems to be a known issue throughout the git community, see [1] and [2]. [1] - libgit2/pygit2#596 [2] - gitpython-developers/GitPython#546 Several workarounds were proposed by the community, but it seems that we can't fully replicate those workarounds. In [2] they attempt to workaround the issue by deleting git cache. That works for them because their git implementation has file handlers open in pure python, while we only do subprocess git calls. Our TODO: if these windows specific error issues keep persisting, figure out how to garbage collect pygit2 Repository handler correctly. For now settle on error handling, as this issue is specific to a subprocess call that calls taf/updater. Though we should keep track of progress of these known bugs, so we can fix them if they get addressed by pygit2 or gitpython
* Basing on #243, we are having some difficulty with pygit2 Repository file handlers being open. in `pygit.py` we instantiate the Repository class, which holds references to `pack` and `.idx` git objects and keeps files open. If taf and cleanup is attempted from same subprocess, Repository handler won't be deleted, and cleanup error occurs. Calling manual garbage collection did not help. This seems to be a known issue throughout the git community, see [1] and [2]. [1] - libgit2/pygit2#596 [2] - gitpython-developers/GitPython#546 Several workarounds were proposed by the community, but it seems that we can't fully replicate those workarounds. In [2] they attempt to workaround the issue by deleting git cache. That works for them because their git implementation has file handlers open in pure python, while we only do subprocess git calls. Our TODO: if these windows specific error issues keep persisting, figure out how to garbage collect pygit2 Repository handler correctly. For now settle on error handling, as this issue is specific to a subprocess call that calls taf/updater. Though we should keep track of progress of these known bugs, so we can fix them if they get addressed by pygit2 or gitpython chore: changelog
* Basing on #243, we are having some difficulty with pygit2 Repository file handlers being open. in `pygit.py` we instantiate the Repository class, which holds references to `pack` and `.idx` git objects and keeps files open. If taf and cleanup is attempted from same subprocess, Repository handler won't be deleted, and cleanup error occurs. Calling manual garbage collection did not help. This seems to be a known issue throughout the git community, see [1] and [2]. [1] - libgit2/pygit2#596 [2] - gitpython-developers/GitPython#546 Several workarounds were proposed by the community, but it seems that we can't fully replicate those workarounds. In [2] they attempt to workaround the issue by deleting git cache. That works for them because their git implementation has file handlers open in pure python, while we only do subprocess git calls. Our TODO: if these windows specific error issues keep persisting, figure out how to garbage collect pygit2 Repository handler correctly. For now settle on error handling, as this issue is specific to a subprocess call that calls taf/updater. Though we should keep track of progress of these known bugs, so we can fix them if they get addressed by pygit2 or gitpython chore: changelog
* fix: reraise original exception and windows specific cleanup errors * Basing on #243, we are having some difficulty with pygit2 Repository file handlers being open. in `pygit.py` we instantiate the Repository class, which holds references to `pack` and `.idx` git objects and keeps files open. If taf and cleanup is attempted from same subprocess, Repository handler won't be deleted, and cleanup error occurs. Calling manual garbage collection did not help. This seems to be a known issue throughout the git community, see [1] and [2]. [1] - libgit2/pygit2#596 [2] - gitpython-developers/GitPython#546 Several workarounds were proposed by the community, but it seems that we can't fully replicate those workarounds. In [2] they attempt to workaround the issue by deleting git cache. That works for them because their git implementation has file handlers open in pure python, while we only do subprocess git calls. Our TODO: if these windows specific error issues keep persisting, figure out how to garbage collect pygit2 Repository handler correctly. For now settle on error handling, as this issue is specific to a subprocess call that calls taf/updater. Though we should keep track of progress of these known bugs, so we can fix them if they get addressed by pygit2 or gitpython chore: changelog * fix: move clean up handling to `on_rm_error` * Cause of Windows Error permissions is in `on_rm_error`, which can trigger and fail in multiple places in updater.py. Since we only put out a warning for that, log the warning when calling os.unlink
Hello,
i make a temporary bare repository, add a remote and fetch references. Then i want to delete the temporary folder, but it fails because of locked files. I've attached the following script to demonstrate what i mean:
filelocktest.zip
The text was updated successfully, but these errors were encountered: