# Technical Documentation: microsoft/VFSForGit > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/microsoft/VFSForGit) ### Overview (`Readme.md`) # VFS for Git ## What is VFS for Git? VFS stands for Virtual File System. VFS for Git virtualizes the file system beneath your Git repository so that Git and all tools see what appears to be a regular working directory, but VFS for Git only downloads objects as they are needed. VFS for Git also manages the files that Git will consider, to ensure that Git operations such as `status`, `checkout`, etc., can be as quick as possible because they will only consider the files that the user has accessed, not all files in the repository. Note: for new deployments, we strongly recommend you consider [Scalar](https://github.com/microsoft/scalar) instead of VFS for Git. By combining the lessons from operating VFS for Git at scale with new developments in Git, Scalar offers a clearer path forward for all large monorepos. ## Installing VFS for Git VFS for Git requires Windows 10 Anniversary Update (Windows 10 version 1607) or later. To install, use [`winget`](https://github.com/microsoft/winget-cli) to install the [`microsoft/git` fork of Git](https://github.com/microsoft/git) and VFS for Git using: ``` winget install --id Microsoft.Git winget install --id Microsoft.VFSforGit ``` You will need to continue using the `microsoft/git` version of Git, and it will notify you when new versions are available. ## Building VFS for Git If you'd like to build your own VFS for Git Windows installer: * Install Visual Studio 2022 Community Edition or higher (https://www.visualstudio.com/downloads/). * Include the following workloads: * .NET desktop development * Desktop development with C++ * Include the following additional components: * Windows 10 or 11 SDK (10.0+) * Install the [.NET 10 SDK](https://dotnet.microsoft.com/download/dotnet/10.0) * Install [`vcpkg`](https://learn.microsoft.com/vcpkg/get-started/get-started) (or use the copy bundled with Visual Studio's C++ workload) * Create a folder to clone into, e.g. `C:\Repos\VFSForGit` * Clone this repo into the `src` subfolder, e.g. `C:\Repos\VFSForGit\src` * Run `src\scripts\Build.bat` (defaults to a Debug build) * You can also build in Visual Studio by opening `src\GVFS.sln` (do not upgrade any projects) and building. However, the very first build will fail, and the second and subsequent builds will succeed. This is because the build requires a prebuild code generation step. For details, see the build script in the previous step. Visual Studio 2022 will [automatically prompt you to install these dependencies](https://devblogs.microsoft.com/setup/configure-visual-studio-across-your-organization-with-vsconfig/) when you open the solution. The .vsconfig file that is present in the root of the repository specifies all required components. The installer can now be found at `C:\Repos\VFSForGit\out\GVFS.Installers\bin\[Debug|Release]\win-x64\SetupGVFS..exe` AI coding assistants working in this repo: see [AGENTS.md](AGENTS.md) for project-specific build/test guidance. ## Trying out VFS for Git * VFS for Git requires a Git service that supports the [GVFS protocol](Protocol.md). For example, you can create a repo in [Azure DevOps](https://azure.microsoft.com/services/devops/), and push some contents to it. There are two constraints: * Your repo must not enable any clean/smudge filters * Your repo must have a `.gitattributes` file in the root that includes the line `* -text` * `gvfs clone ` * Please choose the **Clone with HTTPS** option in the `Clone Repository` dialog in Azure Repos, not **Clone with SSH**. * `cd \src` * Run Git commands as you normally would * `gvfs unmount` when done ## Note on naming This project was formerly known as GVFS (Git Virtual File System). You may occasionally see collateral, including code and protocol names, which refer to the previous name. # Licenses The VFS for Git source code in this repo is available under the MIT license. See [License.md](License.md). VFS for Git relies on the PrjFlt filter driver, formerly known as the GvFlt filter driver, available as a prerelease NuGet package. VFS for Git also includes or links to third-party native libraries at build time. See [THIRD-PARTY-NOTICES.md](THIRD-PARTY-NOTICES.md) for details on these dependencies and their licenses. --- ### Overview (`GVFS/GVFS.Platform.Windows/Readme.md`) # Windows file system virtualization The purpose of this document is to give a high level overview of how the virtualization on Windows works. ProjFS is the file system level driver that is used to intercept file system calls and then call out to a user mode process, in this case GVFS.Mount.exe to get virtualized information or to notify of file system events. There are two interfaces that are exposed by ProjFS. `IVirtualizationInstance` which has all the notifications callbacks and methods that can be called to manipulate the state of the virtual file system. `IRequiredCallbacks` are the methods that are required for virtualization to work. This interface is passed to the `StartVirtualizing` method. ## Required Callbacks - `IRequiredCallbacks` interface The methods of this interface are required for virtualization and are for enumerating directories and getting placeholder and file data in order for ProjFS to project the files or provide the file content. ## Virtualization - `IVirtualizationInstance` interface The ProjFS managed library provide the implementation of this interface with `VirtualizationInstance` which is created in the constructor of the `WindowsFileSystemVirtualizer`. In addition to the root directory of the working directory which will be the virtualization root, it allows the caller to control thread counts, the negative path cache, and notification mappings. ### Negative path cache The negative path cache is a feature in ProjFS that allows it to cache paths that VFSForGit has returned as not found. This gave significant performance benefit because ProjFS no longer needed to make the call to the user mode process (GVFS.Mount.exe) to find out that the path doesn't exist. There is also a method on the `VirtualizationInstance` called `ClearNegativePathCache` that VFSForGit needs to call when it is changing the projection so that paths that may not have exists at the previous commits will now show up. ### Notification Mappings Notification mappings are used to set what notification callbacks will be called for a certain path. Any path in the virtualization root can have different notifications setup for it using bitwise OR-ed values of `NotificationType` from ProjFS. VFSForGit has the combined values in the `Notifications` class for specific files and folders. The `WindowsFileSystemVirtulizer` turns off notifications for the `.git` directory except for some specific files like the `index` file or the folder `refs/heads/`. This helps the performance of git because for most file system access to the `.git` directory will be close to NTFS speed. ### Notification callbacks The notifications callbacks are used to let the user mode process know about various file system actions that are about to take place or have taken place. These are used by VFSForGit to keep the modified paths (the files that git should be keeping up to date) and placeholders correct based on what has happened on the file system. ### Command methods There are other methods that VFSForGit can use on the `VirtualizationInstance` to interact with the files/folders in the virtualization. A couple examples of these methods are listed. #### `MarkDirectoryAsPlaceholder` Used to change a directory to a placeholder so that ProjFS will start asking VFSForGit what the contents of that folder should be and merging that with what is on disk. This is called in a couple of places when a new folder is created. 1. When git creates a folder, it will only add the files that are in the modified paths so there might be files that git will not write to the new folder and need to start being projected. 2. When using the sparse feature and a folder is created that is in the repository but not in the sparse set for projection. The new folder gets added to the sparse set of folders and that new folder needs to start projecting the files and folders in it. #### `DeleteFile` and `UpdateFileIfNeeded` Used by VFSForGit when the projection changes to update with new file data and SHA1 or delete the placeholders that ProjFS has on disk so that it will match the new projection and files will have the correct content when read. Since what is on disk takes priority, these methods can fail if called after a file has been marked dirty, converted to a full file, or turned into a tombstone. ## File/Folder states Files and folders in the projection can be in various states to keep the virtual state of the file system. ### Virtual Files and folders in this state have nothing on disk. They show when a directory is enumerated and ProjFS gets the list of files and folders from VFSForGit to satisfy the enumeration request. ### Placeholder This is a file or folder that is on disk with a specific reparse point that means it doesn't have all the data. For files that means it has the attributes for the file but not the content on disk. There is a SHA1 stored as the `contentId` in the placeholder so that ProjFS can pass that back to VFSForGit to get the content. For folders it means that ProjFS will ask VFSForGit what the contents of the folder should be and merges that with what is on disk to give the view of the file system for that folder. ### Hydrated Placeholder A file that has been read and the contents for the file have been retrieved from VFSForGit and is now on disk. This means any future reads are passthrough to the file system for native file system performance. ### Dirty Placeholder When a placeholder that is hydrated or not has its attributes changed and it is now different from what the provider (VFSForGit) gave. This comes into play when trying to update or delete placeholders. When it is dirty and the code didn't pass `AllowDirtyMetadata`, the update or delete will fail with a `UpdateFailureReason` of `DirtyMetadata`. ### Full file File has been written to or opened for write. This means the file will no longer be updated by VFSForGit and is a regular NTFS file. The path will be added to the modified paths of VFSForGit and git will be the process updating/deleting the file. ### Tombstone This is created when an item is deleted to track what items have been deleted so that they won't get projected again because when there is not a item then ProjFS uses the items from the projection. These need to be deleted when the projection changes so that the correct files and folders will be projected. ## Diagram ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Example In the `src` folder which is the virtualization root after an initial `gvfs clone` there is a file (`file1.txt`). *** ### Enumerate `src` 1. ProjFS calls `StartDirectoryEnumerationCallback`. 2. VFSForGit creates an `ActiveEnumeration` from the current projection and adds to list. 3. ProjFS calls `GetDirectoryEnumerationCallback`. 4. VFSForGit gets the `ActiveEnumeration` by the enumeration `Guid` and add to the enumeration results via the `IDirectoryEnumerationResults` interface. 5. ProjFS calls `EndDirectoryEnumerationCallback` when done. 6. VFSForGit removes the `ActiveEnumeration` from list. State of the files and folders are still all virtual, same as before the enumeration. *** ### Read attributes on `src/file1.txt` 1. ProjFS calls `GetPlaceholderInfoCallback`. 2. If path is not projected, return not found. 3. VFSForGit 1. Calls `GetProjectedFileInfo` and if null returns not found. 2. Calls `WritePlaceholderInfo` to create the placeholder file. 3. Adds the placeholder to the placeholder database. *** ### Read content on `src/file1.txt` 1. ProjFS reads the on-disk placeholder data and calls `GetFileDataCallback`. 2. VFSForGit 1. Uses the `contentId` which will be the blob's SHA1 to get the file content. 2. Calls `CreateWriteBuffer` to create an `IWriteBuffer`. 3. Copies the data to the `IWriteBuffer.Stream`. *** ### Write to `src/file1.txt` 1. ProjFS remove the reparse point so file is a NTFS file. 2. ProjFS calls `OnNotifyFilePreConvertToFull`. 3. VFSForGit if path is projected 1. Adds path to the modified paths so git will keep it up to date. 2. Removes path from the placeholder list. *** ### Delete `src/file1.txt` 1. ProjFS replaces file with tombstone file 2. ProjFS calls `OnNotifyFileHandleClosedFileModifiedOrDeleted` 3. VFSForGit 1. Adds path to the modified paths so git will keep it up to date. 2. Removes path from the placeholder list. At this point `file1.txt` is still in the projection and will be return by enumeration requests but because ProjFS has the tombstone file and that is given precedence over projected files ProjFS will not return `file1.txt` for the enumeration. --- ### Overview (`GVFS/GVFS.UnitTests/Readme.md`) # GVFS Unit Tests ## Unit Test Projects ### GVFS.UnitTests * Targets .NET Core * Contains all unit tests that are .NET Standard compliant ### GVFS.UnitTests.Windows * Targets .NET Framework * Contains all unit tests that depend on .NET Framework assemblies * Has links (in the `NetCore` folder) to all of the unit tests in GVFS.UnitTests GVFS.UnitTests.Windows links in all of the tests from GVFS.UnitTests to ensure that they pass on both the .NET Core and .Net Framework platforms. ## Running Unit Tests **Option 1: `Scripts\RunUnitTests.bat`** `RunUnitTests.bat` will run both GVFS.UnitTests and GVFS.UnitTests.Windows **Option 2: Run individual projects from Visual Studio** GVFS.UnitTests and GVFS.UnitTests.Windows can both be run from Visual Studio. Simply set either as the StartUp project and run them from the IDE. ## Adding New Tests ### GVFS.UnitTests or GVFS.UnitTests.Windows? Whenever possible new unit tests should be added to GVFS.UnitTests. If the new tests are for a .NET Framework assembly (e.g. `GVFS.Platform.Windows`) then they will need to be added to GVFS.UnitTests.Windows. ### Adding New Test Files When adding new test files, keep the following in mind: * New test files that are added to GVFS.UnitTests will not appear in the `NetCore` folder of GVFS.UnitTests.Windows until the GVFS solution is reloaded. * New test files that are meant to be run on both .NET platforms should be added to the **GVFS.UnitTests** project. --- ### Overview (`overlays/libgit2/README.md`) # libgit2 vcpkg Overlay Port This overlay replaces the default vcpkg `libgit2` port to pin a specific version and optionally carry patches ahead of upstream releases. ## Current version **libgit2 v1.9.4** — latest v1.9.x release (2026-05-22). Matches the version pinned by the official vcpkg port. v1.9.x is the last of the libgit2 1.x line. ## Patches - `dependencies.diff` — adjusts CMake dependency resolution for vcpkg (copied from official vcpkg port, required for PCRE discovery) - `non-elevated-admin-owner.diff` — support non-elevated admin user ownership check on Windows ([libgit2/libgit2#7200](https://github.com/libgit2/libgit2/pull/7200)). Allows non-elevated processes run by Administrators group members to be considered the owner of repos owned by that group. Related to [libgit2/libgit2#6279](https://github.com/libgit2/libgit2/issues/6279). This overlay carries the version that was **merged to libgit2 `main`** (2026-06-07), which fixes handle-leak, static-handle, and uninitialized-token bugs present in the original PR #7200 diff. See the provenance table below. The patch is **not yet in any libgit2 release** (latest is v1.9.4; expected to ship upstream in v2.0, no ETA), so it remains necessary until the pinned port moves to a libgit2 that includes it. Additional patches can be added to the `PATCHES` list in `portfile.cmake` to apply fixes that haven't shipped in an official libgit2 release yet. ## File provenance All files were copied from the [official vcpkg libgit2 port](https://github.com/microsoft/vcpkg/tree/master/ports/libgit2) and then modified as noted below. | File | Source | Changes | |------|--------|---------| | `vcpkg.json` | Official vcpkg port | Unchanged | | `dependencies.diff` | Official vcpkg port | Unchanged | | `portfile.cmake` | Official vcpkg port | Removed patches not needed for MSVC x64: `c-standard.diff` (C99 inline keyword — MSVC handles natively), `cli-include-dirs.diff` (CLI tool build — we set `BUILD_CLI=OFF`), `mingw-winhttp.diff` (MinGW only) | | `non-elevated-admin-owner.diff` | libgit2 `main` (commits `cc477ee` + `f9f36a6` + `44c05e5`, merge `e805a16`) | Matches the version merged to `main` on 2026-06-07, not the original [PR #7200](https://github.com/libgit2/libgit2/pull/7200) diff. Maintainer follow-ups fixed a `linked_token` handle leak, replaced a non-thread-safe `static HANDLE` with a per-call local, corrected the `*linked_token = NULL` failure path (with `CloseHandle`), and hoisted `current_user_sid()` so `linked_token` is always populated before both branches use it. | | `README.md` | New | VFSForGit-specific documentation | When updating to a new libgit2 version, compare these files against the official vcpkg port to pick up any new patches or portfile changes. ## How overlays work A vcpkg overlay port **completely replaces** the official port — it doesn't layer on top. When `--overlay-ports=overlays` is specified, vcpkg uses this directory's `portfile.cmake` and `vcpkg.json` instead of the registry's. ## Usage ``` vcpkg install --triplet x64-windows-static-aot ^ --overlay-triplets=triplets --overlay-ports=overlays ``` --- ### Overview (`GVFS/GVFS.Virtualization/Projection/Readme.md`) # GitIndexProjection ## Overview This document is to help give developers a better understanding of the `GitIndexProjection` class and associated classes and the design and architectural decisions that went into it. In simplest terms the purpose of the `GitIndexProjection` class is to parse the `.git/index` file and build an in-memory tree representation of the directories and files that are used when a file system request comes from the virtual file system driver. GVFS.Mount.exe keeps an instance of this class in-memory for the lifetime of the process. This helps VFSForGit quickly return file system operations such as enumeration or on-demand hydration. VFSForGit uses the [skip worktree bit](https://git-scm.com/docs/git-update-index#_skip_worktree_bit) to know what to include in the projection data and what files git will be keeping up to date. Currently VFSForGit only supports using [version 4 of the index](https://git-scm.com/docs/git-update-index#Documentation/git-update-index.txt---index-versionltngt). Details on the index format and version 4 can be found [here](https://github.com/microsoft/git/blob/031fd4b93b8182761948aa348565118955f48307/Documentation/technical/index-format.txt). This code was designed for incredibly large repositories (over 3 million files and 500K folders), there are multiple internal classes that are used to help with the prioritized objectives of: 1. Keep git commands functioning correctly. 2. Keep end-to-end time as short as possible. 3. Keep the memory footprint as small as possible. Some things used to acheive these are: 1. Use `unsafe` code and `fixed` pointers for speed. 2. Keep object pools so that the overhead of allocating is a one-time up-front cost. 3. Keep all folder and files names in a byte array with an index and length to avoid converting them all to .NET strings. 4. Multiple threads and sychronization. ### Processes These are some of the processes that use the `GitIndexProjection`. #### Enumeration Enumeration is tracked on a per call basis with a `Guid` and an `ActiveEnumeration` so that multiple enumerations can run and be restarted without affecting each other. 1. Request comes to start a directory enumeration via the callback `IRequiredCallbacks.StartDirectoryEnumerationCallback` 2. Take a projection read lock 3. Try to get the projected items for the folder from the cache 4. If not in cache, get projected items from the tree and add folder data to the cache 5. Convert projeted items to `ProjectedFileInfo` objects 6. Release the read lock #### File Placeholder 1. Request comes to get placeholder information via the callback `IRequiredCallbacks.GetPlaceholderInfoCallback` 2. If the path is in the projection and placeholders can get created 3. Take a projection read lock 4. Try to get the projected item for the parent folder from the cache 5. Try get the child item from the parent folder data child entries 6. Populate the size if not set 7. Release the read lock #### File Data 1. Request comes for file data via the callback `IRequiredCallbacks.GetFileDataCallback` 2. Get the SHA1 from the contentId 3. Get the BLOB data looking in the following places in this order 1. Check in the loose objects 2. Use LibGit2 to try and get the object 3. Try to download object from server and save to the loose objects 4. Write BLOB content using the `IWriteBuffer` returned by a call to the virtualization instance's `CreateWriteBuffer` method. #### git command 1. User runs git command 2. git invokes pre-command hook 1. Check for valid command 2. Obtain GVFS Lock if needed by using the named pipe message of "AquireLock" (`NamedPipeMessages.AcquireLock.AcquireRequest`). 3. If command is fetch or pull, run prefetch of commits 3. git invokes virtual-filesystem hook 1. Requests the list of modified paths from the GVFS.Mount process using the named pipe message "MPL" (`NamedPipeMessages.ModifiedPaths.ListRequest`). 4. git reads the index setting the skip-worktree bit based when path is not in the list of modified paths 5. When git needs to read an object it checks in this order 1. pack files 2. loose objects 3. if enabled gvfs-helper to try and download via the gvfs protocol 4. retry pack files 5. if enabled use the read-object hook to have GVFS.Mount.exe download the object using the named pipe message "DLO" (`NamedPipeMessages.DownloadObject.DownloadRequest`). 6. if enabled check promisor remote 6. If git changes the index it will write out the new index and the invoke the post-index-change hook using the named pipe message "PICN" (`NamedPipeMessages.PostIndexChanged.NotificationRequest`). This will wait for the hook to return before continuing. This is important because the hook is when the projection is updated and needs to be complete before git continues or it may see the wrong projection. 1. Invalidate the projection state 2. This wakes up the index parsing thread 3. Once the parsing and updating of placeholders is complete the hook returns 7. git invokes post-command hook using the named pipe message "ReleaseLock" (`NamedPipeMessages.ReleaseLock.Request`). 1. Release the GVFS Lock if needed ## Internal Classes ### `FileTypeAndMode` Class only used for file systems that support file mode since that is in the git index and is needed when the file is created on disk. ### `PoolAllocationMultipliers` Class used to hold the multipliers that are applied to the various pools in the code. These numbers come from running with various sized repos and determining what was best for keeping the pools at reasonable sizes. ### `ObjectPool` Class that is a generic pool of some type of object that will dynamically grow or can be shrunk to free objects when too many get allocated. All objects for the pool are created at the time the pool is expanded. The `LazyUTF8String.BytePool` is a specialized pool to allow the use of a pointer into the allocated `byte[]`. ### `FolderEntryData` Abstract base class for data about an item that is in a folder. Contains the name and a flag for whether the entry is a folder. `FolderData` and `FileData` are the derived classes for this class. ### `FolderData` Class containing the data about a folder in the projection. Includes the child entries as a `SortedFolderEntries` object, a flag to indicate the children's sizes have been populated, and a flag to indicate if the folder should be included in the projection (This is when using sparse mode). ### `FileData` Class containing the data about a file in the projection. Includes the size and the SHA1. The SHA1 is stored as 2 `ulong` and an `uint` for performance and memory usage. ### `LazyUTF8String` Class used to keep track of the string from the index that is in the `BytePool` and converts from the `BytePool` to a `string` on when needed by either calling the `GetString` method or `Compare` when one string is not all ASCII. ### `SortedFolderEntries` Class used to keep the list entries for a folder, either `FolderData` or `FileData` objects) in sorted order. This class keeps the static pool of both `FolderData` and `FileData` objects for reuse. Couple of things to note: 1. This is using the `Compare` method of the `LazyUTF8String` class for a performance optimization since most of the time the paths in the index are ASCII and the code can do byte by byte comparison and __not__ have to convert to a `string` object and then compare which is a performance and memory hit. 2. When getting the index of the name in the sorted entries it will return the bitwise complement of the index where the item should be inserted. This was done to avoid making one call to determine if the name exists and a second call to get the index for insertion. ### `SparseFolderData` Class used to keep the sparse folder information. It contains a flag for whether the folder should be recursed into for projection, the depth of the folder, and the children in a name, data `Dictionary`. When sparse mode is enable this data is used to determine which folders should be included in the projection. A root instance (`rootSparseFolder`) is kept in the `GitIndexProjection` which is not recursive and only files in the root folder are being projected when there aren't any other sparse folders. When sparse folders are added via the `SparseVerb`, the children of the root instance are inserted or removed accordingly. For example when `gvfs sparse --set foo/bar/example;other` runs, there will be 2 sparse folders, `foo/bar/example` and `other`. ``` `rootSparseFolder` in the `GitIndexProjection` would have: Children: |- foo (IsRecursive = false, Depth = 0) | Children: | |- bar (IsRecursive = false, Depth = 1) Children: | |- example (IsRecursive = true, Depth = 2) | |- other (IsRecursive = true, Depth = 0) ``` This will cause the root folder to have files and folders for `foo` and `other`. `foo` will only have the `bar` folder and all its files, but no other folders will be projected. The `foo/bar` folder will only have the `example` folder and all its files, but no other folders will be projected. The `foo/bar/example` and `other` folders will have all child files and folders projected recursively. ### `GitIndexEntry` Class used to store the data from the index about a single entry. There is only one instance of this class used during index parsing and it is reused for each index entry. The reason for this is that version 4 of the git index has the [path prefix compressed](https://github.com/microsoft/git/blob/f5992bb185757a1654ce31424611b4d05bda3400/Documentation/technical/index-format.txt#L116) and the previous path is needed to create the path for the current entry. The code in this class is heavily optimized to make parsing the index and the paths as fast as possible. ### `GitIndexParser` Class that is responsible for parsing the git index based of version 4. Please see [`index-format.txt`](https://github.com/microsoft/git/blob/f5992bb185757a1654ce31424611b4d05bda3400/Documentation/technical/index-format.txt) for detailed information of this format. This is used to both validate the index and build the projection. It currently ignores all index extensions and is only for getting the paths and building the tree using the `FolderData` and `FileData` classes. The index is read in chunks of 512K which gave the best performance. ## Other classes ### `ProjectedFileInfo` Class used to hold the data that is used by `FileSystemVirtualizer` when enumerating or creating placeholders. ## `GitIndexProjection` Class used to hold the projection data and keep it up to date. This code uses and can be called from multiple threads. It is using `ReaderWriterLockSlim` to synchronize access to the projection and ResetEvents for waiting and notification of events. There are caches for a variety of objects that are used. ### Initialization Found in the `Initialize` method and does the following: 1. Take a projection write lock 2. Build the projection 3. Release the write lock 4. Update placeholders if needed 5. Start the index parsing thread ### Index parsing thread There is a thread started when the class is initialized that waits to be woken up to parse the index. Events are used to indicate when the parsing is complete to make sure that the projection is in a good state before using it. When woken the parsing thread will: 1. Check if it needs to stop 2. Take the projection write lock 3. Copy the index file and rebuild the projection while projection is invalid 4. Release projection write lock 5. If the projection was updated clear the negative path cache and update placeholders 6. Set event indicating projection parsing is complete ## GVFS.PerfProfiling project This project is used to specifically test the memory and performance of parsing the index and building the projection. There are three tests that can be ran: `ValidateIndex`, `RebuildProjection`, and `ValidateModifiedPaths`. The `IProfilerOnlyIndexProjection` interface is used to expose the methods for use in this project only. Options can be used to limit which tests run. Each test runs 11 times skipping the first run and getting the average of the last 10. Memory is tracked and displayed as well to make sure it stays consistent. --- ### Getting Started (`docs/getting-started.md`) Getting Started =============== Repository Requirements ----------------------- VFS for Git will work with any Git service that supports the [GVFS protocol](/Protocol.md). For example, you can create a repo in [Azure DevOps](https://azure.microsoft.com/services/devops/), and push some contents to it. There are two constraints: * Your repo must not enable any clean/smudge filters * Your repo must have a `.gitattributes` file in the root that includes the line `* -text` Cloning ------- The `clone` verb creates a local enlistment of a remote repository using the [GVFS protocol](https://github.com/microsoft/VFSForGit/blob/master/Protocol.md). ``` gvfs clone [options] [] ``` Create a local copy of the repository at ``. If specified, create the `` directory and place the repository there. Otherwise, the last section of the `` will be used for ``. At the end, the repo is located at `/src`. ### Options These options allow a user to customize their initial enlistment. * `--cache-server-url=`: If specified, set the intended cache server to the specified ``. All object queries will use the GVFS protocol to this `` instead of the origin remote. If the remote supplies a list of cache servers via the `/gvfs/config` endpoint, then the `clone` command will select a nearby cache server from that list. * `--prefetch-cache-server-url=`, `--get-cache-server-url=`, `--post-cache-server-url=`, and `--sizes-cache-server-url=`: Prefer the specified absolute cache server URL for `/gvfs/prefetch`, loose-object GET requests, batched-object POST requests, or `/gvfs/sizes`, respectively. If a dedicated server fails, VFS for Git retries the request against the server selected by `--cache-server-url`. Sizes requests retain their additional fallback to the origin server when the global cache does not support `/gvfs/sizes`. These values are saved in the local Git configuration as `gvfs.prefetch.cache-server`, `gvfs.get.cache-server`, `gvfs.post.cache-server`, and `gvfs.sizes.cache-server`. They continue to apply to later mount, hydration, and prefetch operations. * `--branch=`: Specify the branch to checkout after clone. * `--local-cache-path=`: Use this option to override the path for the local VFS for Git cache. If not specified, then a default path inside `:\.gvfsCache\` is used. The default cache path is recommended so multiple clones of the same remote repository share objects on the same device. ### Advanced Options The options below are not intended for use by a typical user. These are usually used by build machines to create a temporary enlistment that operates on a single commit. * `--single-branch`: Use this option to only download metadata for the branch that will be checked out. This is helpful for build machines that target a remote with many branches. Any `git fetch` commands after the clone will still ask for all branches. * `--no-prefetch`: Use this option to not prefetch commits after clone. This is not recommended for anyone planning to use their clone for history traversal. Use of this option will make commands like `git log` or `git pull` extremely slow and is therefore not recommended. Mounting and Unmounting ----------------------- Before running Git commands in your VFS for Git enlistment or reading files and folders inside the enlistment, a `GVFS.Mount` process must be running to manage the virtual file system projection. A mount process is started by a successful `gvfs clone`, and the enlistment is registered with `GVFS.Service` to auto-mount in the future. The `gvfs status` command checks to see if a mount process is currently running for the current enlistment. The `gvfs mount` command will start a new mount process and register the enlistment for auto-mount in the future. The `gvfs unmount` command will safely shut down the mount process and unregister the enlistment for auto-mount. Removing a VFS for Git Clone ---------------------------- Since a VFS for Git clone has a running `GVFS.Mount` process to track the Git index and watch updates from the ProjFS filesystem driver, you must first run `gvfs unmount` before deleting your repository. This will also remove the repository from the auto-mount feature of `GVFS.Service`. If you have deleted the enlistment or its `.gvfs` folder, then you will likely see alerts saying "Failed to auto-mount at path `X`". To remove this enlistment from the auto-mount feature, remove the appropriate line from the `C:\ProgramData\GVFS\GVFS.Service\repo-registry` file. --- ### Index (`docs/index.md`) VFS for Git: Virtualized File System for Git ============================================ VFS stands for Virtual File System. VFS for Git virtualizes the file system beneath your Git repository so that Git and all tools see what appears to be a regular working directory, but VFS for Git only downloads objects as they are needed. VFS for Git also manages the files that Git will consider, to ensure that Git operations such as `status`, `checkout`, etc., can be as quick as possible because they will only consider the files that the user has accessed, not all files in the repository. Installing ---------- * VFS for Git requires Windows 10 Anniversary Update (Windows 10 version 1607) or later * Run the latest VFS for Git and Git for Windows installers from https://github.com/Microsoft/VFSForGit/releases Documentation ------------- * [Getting Started](getting-started.md): Get started with VFS for Git. Includes `gvfs clone`. * [Troubleshooting](troubleshooting.md): Collect diagnostic information or update custom settings. Includes `gvfs diagnose`, `gvfs config`, `gvfs upgrade`, and `gvfs cache-server`. * [Frequently Asked Questions](faq.md) --- ### Faq (`docs/faq.md`) Frequently Asked Questions ========================== Here are some questions that users often have with VFS for Git, but are unrelated to [troubleshooting issues](troubleshooting.md). ### Why does `gvfs clone` create a `/src` folder? VFS for Git integrates with ProjFS to keep track of changes under this `src` folder. Any activity in this folder is assumed to be important to Git operations. By creating the `src` folder, we are making it easy for your build system to create output folders outside the `src` directory. We commonly see systems create folders for build outputs and package downloads. VFS for Git creates these folders during its builds. Your build system may create build artifacts such as `.obj` or `.lib` files next to your source code. These are commonly "hidden" from Git using `.gitignore` files. Having such artifacts in your source tree creates additional work for Git because it needs to look at these files and match them against the `.gitignore` patterns. By following the pattern VFS for Git tries to establish and placing your build intermediates and outputs parallel with the `src` folder and not inside it, you can help optimize Git command performance for developers in the repository by limiting the number of files Git needs to consider for many common operations. ### Why the name change? This project was formerly known as GVFS (Git Virtual File System). It is undergoing a rename to VFS for Git. While the rename is in progress, the code, protocol, built executables, and releases may still refer to the old GVFS name. See https://github.com/Microsoft/VFSForGit/projects/4 for the latest status of the rename effort. ### Why only Windows? Wasn't there a macOS version coming? We were working hard to deliver a macOS version, and there are still many remnants of that effort in the codebase. We were heavily dependent upon the macOS KAUTH kernel extensions to provide equivalent functionality of ProjFS on Windows. Unfortunately, [Apple deprecated this feature in Catalina](https://developer.apple.com/support/kernel-extensions/). All of the recommended alternatives were either not appropriate for our scenario or were not fast enough. We transitioned our large repository strategy to focus on using [`git sparse-checkout`](https://github.blog/2020-01-17-bring-your-monorepo-down-to-size-with-sparse-checkout/) instead of filesystem virtualization. We then forked the VFS for Git codebase to create [Scalar](https://github.com/microsoft/scalar). Those investments in a cross-platform tool paid off since Scalar could launch quickly. ### Why are you abandoning VFS for Git? We will continue supporting VFS for Git as long as there is a need for it. Through our experience, we have found that it is appropriate for only a very small number of extremely large repos. For instance, the Windows OS repository will depend on VFS for Git for the foreseeable future, and we will continue supporting them. This includes updating VFS for Git with new versions of Git. The basic issue with VFS for Git is that users who don’t know exactly what’s going on will frequently get into bad states, such as populating too much of their working directory or not properly enabling the ProjFS feature (this is a larger problem for users using older versions of Windows). The Windows OS team developed a lot of tribal knowledge for how to avoid known issues with VFS for Git. We prefer users adopting Scalar because that is where we are investing most of our engineering efforts. The system is simpler and has a better “offramp” onto core Git, if one decides that they want to get there. We are also investing in making the Git client do more of the heavy lifting and reducing how much is really a “Scalar” feature and how much is just a Git feature. --- ### Troubleshooting (`docs/troubleshooting.md`) Troubleshooting =============== Deleting a VFS for Git repo --------------------------- You must follow these steps to delete a VFS for Git repository. If you have attempted deletion before un-mounting jump to [Recovering from an attempt to delete without un-mounting](#Recovering-from-an-attempt-to-delete-without-un-mounting). 1. Un-mount the repo. Since a VFS for Git clone has a running `GVFS.Mount` process to track the Git index and watch updates from the ProjFS filesystem driver, you must first run `gvfs unmount` before deleting your repository. This will also remove the repository from the auto-mount feature of `GVFS.Service`. Make sure the current working directory of your shell is not in the VFS for Git repo and that no other processes are using files in it. For example: ``` C:\Users\you\big_repo\src\> cd ..\.. C:\Users\you\> gvfs unmount big_repo ``` 1. Clean up the remaining folder. (Do not try to delete the repo before it is un mounted.) Once un-mounted you can fully clean up the old repo by deleteing it. Following the example from above: ``` C:\Users\you\> rmdir /S /Q big_repo ``` ### Recovering from an attempt to delete without un-mounting If you have attempted to delete the repo or its `.gvfs` folder, then you will likely see alerts saying "Failed to auto-mount at path `X`". 1. Manually remove this repo from the auto-mount feature, remove the appropriate line from the `C:\ProgramData\GVFS\GVFS.Service\repo-registry` file. 1. Ensure there is no currently running mount process for the repo. 1. Open Task Manager. 1. Go the `Details` tab. 1. Right click in the header row (on `Name` for instance) to choose `Select Columns`. 1. Check the `Command line` column which will show the full command line for each process. 1. Look for a `GVFS.Mount.exe` process that has your repo in question in the command line arguments. 1. If you find said process, right click and choose `End Task` to end it. 1. Proceed with removing the directory as described above to clean up the folder with `rmdir /S /Q REPO`. Upgrade ------- The `GVFS.Service` process checks for new versions of VFS for Git daily and will prompt you for upgrade using a notification. To check manually, run `gvfs upgrade` to see if an upgrade is available. Run `gvfs upgrade --confirm` to actually perform the upgrade, if you wish. ### Upgrade fails with **Symptom:** `gvfs upgrade` fails with the following error: > ERROR: Could not launch upgrade tool. File copy error - The specified path, file name, or both are too long" **Fix:** There is a known issue with VFS for Git v1.0.20112.1 where the `gvfs upgrade` command fails with a long-path error. The root cause is a [recursive directory copy](https://github.com/microsoft/VFSForGit/pull/1672) that loops the contents of that directory into the copy and it never ends. The only fix is to [manually upgrade to version v1.0.20154.3](https://github.com/microsoft/VFSForGit/releases/tag/v1.0.20154.3). Common Issues ------------- We are constantly improving VFS for Git to prevent known issues from reoccurring. Please make sure you are on the latest version using `gvfs upgrade` before raising an issue. Please also keep your version of Windows updated as that is the only way to get updates to the ProjFS filesystem driver. ### TRY THIS FIRST: `gvfs repair` Some known issues can get your enlistment into a bad state. Running `gvfs repair` will detect known issues and the output will mention if the issue is actionable or not. Then, `gvfs repair --confirm` will actually make the changes it thinks are necessary. If `gvfs repair` detects a problem but says it cannot fix the problem, then that's an excellent message to include when creating an issue. ### TRY THIS NEXT: `gvfs unmount`, `gvfs mount`, or restart Sometimes the `GVFS.Mount` process gets in a bad state and simply needs to restart to auto-heal. Since VFS for Git also interacts directly with the ProjFS filesystem driver, sometimes a system restart can help. ### Mismatched Git version **Symptom:** Enlistment fails to mount with an error such as ``` Warning: Installed git version does not match supported version of ``` **Fix:** VFS for Git is tightly coupled to [a custom version of Git](https://github.com/microsoft/git). This error happens when the installed version of Git does not match the one that was included in the VFS for Git installer. Please download and install the matching version of Git from [the VFS for Git releases page](https://github.com/microsoft/vfsforgit/releases). ### 404 Errors, or "The Git repository with name or identifier X does not exist..." If your `gvfs clone ` command fails with this error, then check if you can access `` in a web browser. If you cannot see the repository in the web, then you do not have permissions to read that repository. These issues cannot be resolved by the VFS for Git team and must be done by your repository administrators. If you _can_ see the repository in the web, then likely you have a stale credential in your credential manager that needs updating. VFS for Git _should_ attempt to renew your credential. If it does not, then go to Windows Credential Manager and delete the Git credential for that URL. ### ProjFS Installation Issue **Symptoms:** - VFS for Git will not mount - The mount logs (or mount CLI) have a ProjFS related error. Examples: - "Service prjflt was not found on computer". - "Could not load file or assembly 'ProjectedFSLib.Managed.dll' or one of its dependencies. The specified module could not be found." - "Attaching the filter resulted in: 2149515283" - "StartVirtualizationInstance failed: 80070057(InvalidArg)" **Fix:** The easiest way to fix ProjFS issues is to completely remove ProjFS, and then rely on `GVFS.Service` to re-install (or re-enable) ProjFS. 1. If `C:\Program Files\GVFS\ProjectedFSLib.dll` is present, delete it 2. Determine if inbox ProjFS is currently enabled: *In an Administrator Command Prompt* ```powershell -NonInteractive -NoProfile -Command "&{Get-WindowsOptionalFeature -Online -FeatureName Client-ProjFS}"``` Check the value of `State:` in the output to see if inbox ProjFS is enabled. 3. If ProjFS **is enabled** - Restart machine - Attempt to mount - If mount fails, manually disable ProjFS: - Control Panel->Programs->Turn Windows features on or off - Uncheck "Windows Projected File System" - Click OK - Restart machine - (When `GVFS.Service` starts, it will automatically re-enable the optional feature) - `gvfs mount` repo - If the mount still fails, restart one more time and try `gvfs mount` again 4. If ProjFS **is not enabled** - Manually remove ProjFS and GvFlt From an Administrator command prompt: - `sc stop GVFS.Service` - `sc stop gvflt` - `sc delete gvflt` - `sc stop prjflt` - `sc delete prjflt` - `del c:\Windows\System32\drivers\prjflt.sys` - `sc start GVFS.Service` - `gvfs mount` repo If the above steps do not resolve the issue, and the user is on Windows Server, ensure they have [the latest version of GVFS installed](https://github.com/microsoft/vfsforgit/releases). ### Unable to Move/Rename directories inside GVFS repository. **Symptom:** User is not able to rename or move a partial directory inside their VFS for Git enlistment. If rename or move is done using Windows Explorer, the user might see an error alert with message ```Error 0x80070032: The request is not supported.``` **Fix:** Partial directories are directories that originate from the virtual projection. They exist on disk and still contain ProjFS reparse data. When an application enumerates a partial directory ProjFS calls VFS for Git's enumeration callbacks. VFS for Git does not support rename or move of partial directories inside an enlistment. However it supports rename/move of a regular directory. User can copy the partial directory and paste it inside the enlistment. The newly pasted directory is a regular directory and can be renamed or moved around inside the enlistment. The original partial directory can now be deleted. Diagnosing Issues ----------------- The `gvfs diagnose` command collects logs and config details for the current repository. The resulting zip file helps root-cause issues. When run inside your repository, creates a zip file containing several important files for that repository. This includes: * All log files from `gvfs` commands run in the enlistment, including maintenance steps. * Log files from the `GVFS.Service`. * Configuration files from your `.git` folder, such as the `config` file, `index`, `hooks`, and `refs`. * A summary of your Git object database, including the number of loose objects and the names and sizes of pack-files. As the `diagnose` command completes, it provides the path of the resulting zip file. This zip can be sent to the support team for investigation. Modifying Configuration Values ------------------------------ ### Cache Server URL Cache servers are a feature of the GVFS protocol to provide low-latency access to the on-demand object requests. This modifies the `gvfs.cache-server` setting in your local Git config file. Run `gvfs cache-server --get` to see the current cache server. Run `gvfs cache-server --list` to see the available cache server URLs. Run `gvfs cache-server --set=` to set your cache server to ``. Individual GVFS protocol endpoints can prefer dedicated cache servers through these local Git configuration values: | Configuration | Requests | | --- | --- | | `gvfs.prefetch.cache-server` | `/gvfs/prefetch` | | `gvfs.get.cache-server` | Loose-object GET requests under `/gvfs/objects` | | `gvfs.post.cache-server` | Batched-object POST requests to `/gvfs/objects` | | `gvfs.sizes.cache-server` | `/gvfs/sizes` | Each value must be an absolute URL. A dedicated endpoint server is attempted before `gvfs.cache-server`; if that request fails, VFS for Git falls back to the global cache server. Sizes requests also fall back from the global cache to the origin server when `/gvfs/sizes` is not supported. `gvfs cache-server --get` and `--set` operate on the global `gvfs.cache-server` value. Setting the global server does not clear the four endpoint-specific values. Inspect or change those values with `git config --local []`. ### System-wide Config The `gvfs config` command allows customizing some behavior. 1. Set system-wide config settings using `gvfs config `. 2. View existing settings with `gvfs config --list`. 3. Remove an existing setting with `gvfs config --delete `. The `usn.updateDirectories` config option, when `true`, will update the [USN journal entries](https://docs.microsoft.com/en-us/windows-server/administration/windows-commands/fsutil-usn) of directories when the names of subdirectories or files are modified, even if the directory is still only in a "projected" state. This can be particularly important when using incremental build systems such as microsoft/BuildXL. However, there is a 10-15% performance penalty on some Git commands when this option is enabled. The `gvfs config` command is also used for customizing the feed used for VFS for Git upgrades. This is so large teams can bundle a custom installer or other tools along with VFS for Git upgrades. --- --- METRICS --- - Files Extracted: 9 - Estimated Token Budget: ~13102 tokens - Canonical Reference: https://codewiki.google/github.com/microsoft/VFSForGit