Git¶
git repository¶
HANDE can be downloaded by cloning the repository from github:
$ git clone https://github.com/hande-qmc/hande
We periodically tag releases
A private git repository, where much of the day-to-day development work takes place, is currently located at a private repository at github and can be cloned using:
$ git clone https://github.com/hande-qmc/hande-dev
or if using ssh keys,
$ git clone git@github.com:hande-qmc/hande-dev
If you would like access, please speak to one of the developers. The rest of
this guide assumes you used the default remote name during the clone (i.e.
origin). If this is not the case, we assume you are capable of
appropriately adjusting the commands given in the rest of the guide.
Note
Bug fixes and similar work are applied to both public and private repositories. New features are often developed in the private repository (which hooks into our buildbot server for regression testing), whilst we iron them out. Once we are happy that new features are ready for production use, they will also be migrated to the public repository.
Precepts¶
- All development happens in branches.
- Branches belong to a relevant namespace (feature/XXX indicates XXX is a branch (name) for a new feature, he/XXX for a HANDE enhancement (he), bug_fix/XXX for a bug fix, config/XXX for a new config file, etc).
- Branches are merged into master after review. Merging between development branches should be avoided.
- Branches should be reviewed by one other person (at least) before merging into master.
- To review, send a pull request email (see git request-pull) to all developers (perhaps including a summary of work in the branch, which is not generated by request-pull!). This should be viewed as starting a conversation on the work.
- Make changes prompted by the review and resend the pull request. (This might take a few iterations.)
- After a happy conclusion to the review, merge into master.
Notes:
- We would like each commit to at least compile but don’t expect each commit to be perfect in its own right! This is extremely useful for using git-bisect when investigating regression errors.
- New functionality should be incorporated by new tests. I intend to spend a day soon creating new tests and checking the code coverage (lcov is a wonderful tool) of the test suite.
See http://nvie.com/posts/a-successful-git-branching-model/ for a popular variant on this approach.
The hope is that this approach will lead to better code and also (with a little work) everyone will be more familiar/comfortable with the code that they’re not directly working on themselves.
Branch namespaces¶
A (non-exhaustive!) list of namespaces we use for branches:
- he/XXX
- for an enhancement to HANDE (usually a modification to existing algorithms).
- bug_fix/XXX
- for a bug fix to a specific area of the codebase.
- opt/XXX
- for optimisation work (please include performance details in the commit message!).
- feature/XXX
- for a new feature (generally bigger than an enhancement).
- doc/XXX
- for fixes/enhancements solely to the documentation. (Often this kind of work is coupled to feature/enhancement development work and the documentation is updated directly in the relevant branches consisting mainly of changes to the source code.)
- config/XXX
- for new configuration file(s)/updates to existing configurations.
Obviously there is some overlap between the he, feature and (to a lesser extent) opt namespaces. Broadly speaking, new algorithms or changes to existing algothims which require a new input options are best suited to the feature namespace, speed/memory improvements to opt/ and other improvements (code tidying, logging, etc.) to the he namespace.
How to generate a pull request¶
First push your work to the relevant branch on the git sever and then generate template text for the pull request:
$ git request-pull startref origin [endref]
where startref (endref) is the commit you want to be reviewed from (to) and origin is the name of remote configured to the git sever. startref and endref can be any way of referring to a specific commit and endref defaults to HEAD if not given. Usually the branch would have been created from master, in which case you can simply do (even if master has been committed to since the branch was created):
$ git request-pull master origin
which generates (for example):
$ git request-pull master origin
The following changes since commit 7a58a8d1a8f2e8af15df1c9946e7596078649d79:
Updated the config files for cx2. (2013-12-09 11:07:52 +0000)
are available in the git repository at:
git@tyc-svn.cmth.ph.ic.ac.uk:hubbard_fciqmc config/cx2
for you to fetch changes up to 1a5522648378f406d3e5fbd87e22e3768da490bc:
Fixed typo cx2 config comment (2013-12-13 14:35:42 +0000)
----------------------------------------------------------------
William Vigor (1):
Fixed typo cx2 config comment
config/cx2 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
Copy and paste this text into your email client and send the pull request to hande-dev@imperial.ac.uk (possibly with some additional text describing motivation/benchmark results/etc). If sendmail/exim4/other MTA is set up properly (naturally the CMTH ones are) then
$ git request-pull master origin | mail -s "Pull request" hande-dev@imperial.ac.uk
works as one would expect.
Merging to master¶
Here’s a workflow to make merging to master simple. Remember that with git it’s extremely difficult to make permanently destructive changes so if it goes wrong it can be fixed.
Before you start make sure your code compiles and passes the test suite. Do not merge broken code into master.
Now make sure your master branch is up to date. Here I do this in a fetch then a pull just to see what else has changed. I do a diff to be sure I’m the same as the origin master.
[master]$ git fetch
remote: Counting objects: 340, done.
remote: Compressing objects: 100% (182/182), done.
remote: Total 200 (delta 137), reused 47 (delta 16)
Receiving objects: 100% (200/200), 96.89 KiB, done.
Resolving deltas: 100% (137/137), completed with 58 local objects.
From tyc-svn.cmth.ph.ic.ac.uk:hubbard_fciqmc
c17ef9e..2d8e130 master -> origin/master
...
[master]$ git pull
Updating c17ef9e..2d8e130
Fast-forward
lib/local/parallel.F90 | 9 ++-------
src/full_diagonalisation.F90 | 30 ++++++++++++------------------
2 files changed, 14 insertions(+), 25 deletions(-)
[master]$ git diff origin/master
The blank output from this indicates we’re at origin/master.
I’m going to merge the branch bug_fix/rdm_init. Crucially we use the –no-ff flag to ensure that the merge creates a commit on master; this keeps the history clean (by keeping development work in logical chunks after merging) and also makes it very easy to roll-back and revert an entire feature if problems are encounted.
[master]$ git merge --no-ff bug_fix/rdm_init
Merge made by the 'recursive' strategy.
src/fciqmc_data.f90 | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
[master]$ git log --graph --oneline --decorate | head
* 647b7dd (HEAD, master) Merge branch 'bug_fix/rdm_init'
|\
| * 3c67d81 (bug_fix/rdm_init) Fix uninitialised doing_exact_rdm_eigv breaking fci
* | 2d8e130 (origin/master, origin/HEAD) Merge branch 'bug_fix/small_fci_mpi'
|\ \
This shows that a new commit has been created on master.
At this point it’s possible that the merge needed some manual intervention. It’s fine to make these changes directly and commit them in the merge to your local master. If the merge is starting to get messy it might be best to rebase first to make it easier.
Very importantly, you should now compile the code and run the tests, even if the merge completed without any problems — there might be unintented effects. Only continue if the code compiles and the tests pass. If you need to make changes at this point, you can modify your local existing merge commit with
[master]$ git commit --amend
Now we’ve made sure that the code works, all we do is push to the main repo
[master]$ git push origin master
Counting objects: 12, done.
Delta compression using up to 12 threads.
Compressing objects: 100% (7/7), done.
Writing objects: 100% (7/7), 705 bytes, done.
Total 7 (delta 5), reused 0 (delta 0)
To git@tyc-svn.cmth.ph.ic.ac.uk:hubbard_fciqmc.git
2d8e130..647b7dd master -> master
[master]$ git log --graph --oneline --decorate | head
* 647b7dd (HEAD, origin/master, origin/HEAD, master) Merge branch 'bug_fix/rdm_init'
|\
| * 3c67d81 (bug_fix/rdm_init) Fix uninitialised doing_exact_rdm_eigv breaking fci
* | 2d8e130 Merge branch 'bug_fix/small_fci_mpi'
|\ \
Almost there. We now ought to clean up the namespace to avoid old branch names hanging around (the code of course will always stay).
[master]$ git branch --delete bug_fix/rdm_init
[master]$ git push origin --delete bug_fix/rdm_init
The list of branches merged into HEAD can be found by doing
[master]$ git branch --all --merged
All done!
Unwanted experimental branches¶
Occasionally (frequently?!) we have tried something which didn’t work out. If we don’t want to keep any of the history, we can simply delete the local (and if necessary) remote branches:
$ git branch --delete unwanted_branch
$ git push origin --delete unwanted_branch
But what about branches that we don’t intend to continue working on in the near future, would like to keep around but without cluttering up the main repository, making it unclear which branches need some TLC before merging? We have a separate repository where such branches can be sent, to be resurrected if desired later. The repository is at ch-hande@git.uis.cam.ac.uk:hande_graveyard.git. To push a local branch there:
$ git remote add graveyard ch-hande@git.uis.cam.ac.uk:hande_graveyard.git
$ git push remote graveyard unwanted_branch
and then delete the branch (both local and remote) from the main repository using the same commands as before. If the branch is not local, then you can either check it out and then do the push and delete (easier) or use a refspec:
$ git push graveyard refs/remotes/origin/unwanted_branch:refs/heads/unwanted_branch
where origin/unwanted_branch is the remote branch to be moved to the graveyard repository. The branch on origin can then be deleted as before.