Dev

This page is for documenting gin internals as they are being developed. In the future, it may become a full developer guide to how the gin client uses git and git annex to perform all the operations it supports.

gin ls

UPDATE 2017-07-18: This is outdated and inaccurate now. It needs to be updated to address the new way status is determined since we no longer use annex in version 6 mode.

The gin ls command should print the status of all files in a given directory with respect to git and git annex. There are 6 cases that need to be identified:

1. The file is checked into git annex and its contents are available locally. #CASE1
2. The file is checked into git annex and its contents are not available locally. In this case, there should be a local placeholder file which contains the file's unique hash. #CASE2
3. The file is checked into git or git annex and it has local modifications that have not been committed. #CASE3
4. The file is checked into git or git annex and it has local modifications that have been committed but not pushed. #CASE4
5. The file is checked into git or git annex and it has remote modifications that have not been pulled. #CASE5
6. The file is not being tracked. #CASE6

Below we describe how the client uses git and git annex commands to identify the case for a given file.

Conditions

For each file we need to check:

• Is it in git annex? Y/N
• If N: Is it in git? Y/N

The commands used to answer the above questions also help identify the other conditions (e.g., local modifications).

First question answered by:

• git annex whereis file

Second question is answered by:

• git ls-files file

Cases

file is in annex:

• git annex whereis file will return successfully and will give us the file info From this info we can determine whether:
  • File contents are synced locally. It will have more than one remotes and one remote, which is the local directory, will be marked with here: true. #CASE1
  • File contents are not synced yet. All remotes will be marked with here: false. #CASE2
  • File contents are added or committed but not synced. It will have only one remote, marked here: true, and no others. #CASE4

file is not in annex:

• git annex whereis file will return successfully but will return no info. This means that the file exists and it is either in git or it is untracked. If the command returns with an error, the file does not exist (we don't really care about the missing file case). Following this we need to check whether:
  • git ls-files file returns the file. This means that the file is checked into git.
  • git ls-files file returns nothing. This means the file is not tracked.

Regardless of whether file is in git or git annex:

• git (annex) status file will return M if the file is locally modified and the changes have not been committed. The command will return A if the file has been added but not committed. We consider this the same case. #CASE3
• In order to detect files that have changed locally (that have been committed) and have not been pushed, we can read the output of git diff --name-only origin/master (or the default remote). #CASE4
• If files are changed remotely and have not been pulled locally, this can be detected by performing a git diff --name-only ..origin/master (or the default remote). However, this first requires a git fetch, to update the refs from the remotes (without merging). There may be a more straightforward way to do this with annexed files. #CASE5
• git (annex) status file will return ? if the file exists and is not being tracked. #CASE6

Notes and observations

• When annexed files are unlocked and checked out, git may show them as modified, even though they are not. If a file is detected as being annexed, we should ignore the git status of that file and use only the status reported by git annex.
• In the client, we shouldn't distinguish between annexed files and files checked into git.
• There may be a more straightforward way to detect remote changes for annexed files without requiring a fetch, for #CASE5.