Tuesday, April 28, 2015

Git gc out of memory and repository slowness - tip for solution

On our project we are using a git repository with current size around 9GB. At some point we started facing issues with slowing the access to repository down (many operations took several minutes to complete).

When trying to research the issue we tracked this to behavior of git gc being executed automatically in background without ever actually finishing the job. The error which appeared when git gc was executed manually was related to out of memory.

We temporarily disabled automatic git garbage collect which solved the slowness but it was far from ideal solution.

In meantime I was also playing with various memory allocation and pack size settings on my local clone but all those just made things worse.

At some point I realized that for historical reasons we have in the repository also bunch of binary files like zip archives, database backups, nuget packages etc. which raised a suspicion.

Researching it more I found following solution which helps exclude those huge binary files from packing.

Create or edit file .git/info/attributes and put following content to it (one line per excluded file type):

*.zip -delta
*.bak -delta

Hope this helps :-)