Wsrtjtyk

We have a git repository for a scientific software where we need to maintain a certain folder structure for our data files.
These folders should remain empty, everything that will be put there should not be tracked by git. However, it is necessary that those folders exist.

The solution to accomplish this was to add a .gitignore file into every directory which looks like this:

*

!.gitignore

which means everything inside this folder is ignored except for the .gitignore file.

This works very well.

We maintain all our data on one particular server.
Our scientists use this server often for their calculations.
It would be very convenient to be able to replace the data folders from the git repository which currently contain only the .gitignore file with a symbolic link to the full data files on this server. The data files on the server also have a .gitignore file which looks exactly the same as in every repository.

I wrote a bash script to do this which looks like this:

rm -r path/to/empty/data/in/repository/name

ln -sfn /absolute/path/to/data/on/server/ path/to/empty/data/in/repository

Now the software runs perfectly and you have access to all the data without copying it into your git repository.

However, git now gets confused.

If I run git status only my changes are listed as expected. It does not complain about the new symbolic links which replaced the existing directories.

As soon as I run git add . to stage my changes the symbolic links appear as new file: and the .gitignore files in the replaced folder are listed as deleted:.

This seems like a problem to me because as soon as somebody pushes his code changes that he made on the server the symbolic links would get uploaded (I guess) and the .gitignore files would get removed and thus the folder structure would not remain.

Is it possible to tell git that it should compare the content of the symbolic linked folders rather than the symbolic link itself?

PS: I know this seems like a software design issue with the static folder structure which is inside git but I do not want to discuss this here. We are all scientists and no programmers and the software is now developed for over 10 years by many different people. It is not possible to change the code to make it more flexible.

EDIT: This bash code reproduces the problem:

cd ~            #setup

mkdir tmp

cd tmp



mkdir server    #server data folder (this one is full of data)

mkdir server/data

printf '*n!.gitignore' > server/data/.gitignore

printf 'data file 1' > server/data/data1.txt

printf 'data file 2' > server/data/data2.txt



mkdir repo      #repo data folder (this one only contains .gitignore file)

mkdir repo/data

printf '*n!.gitignore' > repo/data/.gitignore



cd repo              # create a dummy repo

git init

git add .

git commit -am"commit 1"



git status



cd ..              # replace data folder with server/data folder which hase exactly the same content

rm -r repo/data/

ln -sfn ~/tmp/server/data/ ./repo/



cd repo

git status

At the end git status should ideally not list any changes in the repository.

EDIT:
I found a workaround: instead of linking the whole directory I'm now linking the content of the directory:

ln -sfn /absolute/path/to/data/on/server/* path/to/empty/data/in/repository/

this works because the symbolic links are irgnored due to the .gitignore file.
Drawback is that it only works with existing data. As soon as there is a new file in the server directory I have to run the bash script again.

Git tracks symbolic links. What you're trying to achieve can be done with bind mounts.

Replace the final ln -sfn ~/tmp/server/data/ ./repo/ with sudo mount --bind $PWD/repo
$HOME/tmp/server/data/