The git tutorial

Introduction

This is the "advanced" git tutorial. Basic knowledge about git is required. It builds upon the ATLAS git workflow tutorial which you should already have followed in the preparation for this tutorial. There is also the HSF git tutorial which offers additional or alternative training material.

Quick start:

1. Setup atlas

   setupATLAS
   lsetup git
   

2. Setup athena

   asetup athena,main,latest
   

3. Download tutorial script git_tutorial2.sh
4. Create mock-up repository:

   sh git_tutorial2.sh --create-repo
   

5. Enter directory of mock-up repository ("./Main/source/test")
6. Start with sparse_checkout (optional) or merge1 tutorial
7. To retry a tutorial part, restore the starting state:

   sh ../../../git_tutorial2.sh -t the_tutorial_merge1_start
   

8. To recreate the entire repository:

   sh ../../../git_tutorial2.sh --wipe-clean
   

The script contains all instructions (identical to the instructions on atlas-software.docs.cern.ch):

sh ../../../git_tutorial2.sh -i

or to see only the instructions for one part:

sh ../../../git_tutorial2.sh -i sparse_checkout

or list the tutorial parts (and some basic instructions):

sh ../../../git_tutorial2.sh

Directory layout:

the script git_tutorial2.sh --create-repo should create the following sub-directories:

./Main/build
./Main/source/test/

The repository is in

./Main/source/test

When going through the worktree tutorial the following extra sub-directories should be created:

./WT
./WT/FixIntegrationTest
./WT/FixIntegrationTest/source
./WT/FixIntegrationTest/build
./WT/MaintenanceMain
./WT/MaintenanceMain/source
./WT/MaintenanceMain/build

Tutorial parts:

• the_tutorial_sparse_checkout [~2 min] (optional)
• the_tutorial_merge1 [5-20min]
• the_tutorial_merge2 [4-15min]
• the_tutorial_rebase1 [4-15min]
• the_tutorial_rebase2 [3-15min]
• the_tutorial_compare_history [~2 min]
• the_tutorial_squash_commits [3-10min]
• the_tutorial_interactive_rebase [~15min] (optional)
• the_tutorial_worktree [10-30min] (optional)
• EXTRA: the_tutorial_athena_example_worktree [~10min] (optional)

The tutorials may take significantly longer, but You presumably will not get through the tutorial in less than the indicated times.

You can go back to the starting condition the_tutorial_merge1, the_tutorial_merge2, ... parts by running git_tutorial2.sh with the argument --tutorial ...part..._start e.g.

sh ...path-to-script.../git_tutorial2.sh --tutorial the_tutorial_merge1_start

There is no defined starting condition for the sparse checkout tutorial. It can Be done at any time but will interfere with ongoing tutorial parts.

The script will give You these instructions by executing

sh ...path-to-script.../git_tutorial2.sh --instructions

or only show the instructions for a particular part by adding the tutorial part e.g.

sh ...path-to-script.../git_tutorial2.sh --instructions merge1

or with "the_tutorial_merge1_start".

When executing the command without any argument You will get the the list of tutorial parts and some basic instructions:

sh git_tutorial2.sh --tutorial

Notes:

• If You want to do the sparse checkout tutorial part, You should start with that since it has an impact on the other tutorial parts The sparse checkout does not depend on a particular initial state and You cannot reset the starting state for this tutorial part. Just read the instructions. In case You want to redo this part from scratch there is only "--wipe-clean" (see below).

• The tutorial parts merge I+II and rebase I+II have a lot of overlap, nevertheless it is instructive to do both. You could short-cut one or the other occasionally by making use of the provided "solutions", but it makes sense to try to resolve the conflicts by Yourself.

• The part compare_history assumes that the merge and rebase parts have been completed, and just is about comparing the two approaches.

• The squash commits and interactive rebase tutorial assumes that the merge I+II part have been completed.

• The worktree part will use sparse checkouts. But it is not a problem if You skipped the sparse checkout part.

• You can restore the starting condition for a certain tutorial part by issuing the command sh ...path-to-script.../git_tutorial2.sh --tutorial the_tutorial_....._start If You do the tutorial in order, and do a part for the first time, then the starting condition should be in the correct state i.e. You only have to restore the starting condition if You do the tutorial parts in a different order or if You want to retry a tutorial part.

• If You messed up You can recreate the mock-up repository by executing sh ...path-to-script.../git_tutorial2 --wipe-clean But that will remove Your personal work on this repository. You can still restore the prepared starting conditions for the various tutorial parts as described above.

Sparse checkout (optional)

About:

• Use sparse checkout to only checkout a fraction of the full source tree of the repository,
• Test atlas extension for sparse checkouts.

Tasks:

• Activate sparse checkout in git.
• Use atlas git extension to add "WorkDir", "Pkg1", "Base" and "Alg" to the sparse checkout.

Detailed description:

Some people like sparse checkouts because it limits the number of files in the working directory to those which are relevant for the current task. This also reduces the amount of disk space of a checkout, and depending on the disk or file system this may also speedup operations. Other people rather want to see always all files and prefer a full checkout. The latter simplifies some operations like investigating files which would not be part of a sparse checkout, and some tools or IDEs may not properly handle sparse checkouts.

Activate sparse checkout in git (disable "cone" pattern matching for filtering the sparse checkout because git-atlas does not handle these patterns)

git sparse-checkout init  --no-cone

Use atlas extension to add packages to the sparse checkout

git atlas addpkg WorkDir
git atlas addpkg Pkg1
git atlas addpkg Base
git atlas addpkg Alg

The content of a sparse checkout is controlled by the file

./.git/info/sparse-checkout

Now this file should have the following content

./Projects/WorkDir/
./Test/Pkg1/
./Test/Base/
./Test/Alg/

This file can be edited manually but git atlas addpkg ... or git atlas rmpkg ... are more convenient. The directories in the sparse checkout can also be listed with

git sparse-checkout list

The git sparse-checkout can also be used to manipulate the sparse checkout (see "git sparse-checkout --help" for the full list of options)

Finally checkout everything that is listed in the sparse checkout for the branch (fix-sphere-integration-test) (should already be done)

git checkout -f  fix-sphere-integration-test

You should see the following files

./Projects/WorkDir/CMakeLists.txt
./Test/Base/Base/Shape.h
./Test/Base/CMakeLists.txt
./Test/Pkg1/CMakeLists.txt
./Test/Pkg1/Pkg1/Sphere.h
./Test/Pkg1/test/test_Pkg1.cxx
./Test/Alg/Alg/Integrator.h
./Test/Alg/CMakeLists.txt
./Test/Alg/test/test_Alg.cxx

find ./Projects -type f
find ./Test -type f

The mock-up repository should contain additionally the following files

./Test/Pkg2/CMakeLists.txt
./Test/Pkg2/Pkg2/Cube.h
./Test/Pkg2/test/test_Pkg2.cxx

But due to the sparse checkout these files should NOT be part of Your working directory.

To list all files in the current branch also those which are not part of the checkout:

git ls-files *

To search in all files of the branch not just those which are in the sparse checkout

git grep "Cube::Cube" HEAD

To look at a file which is not in the sparse checkout:

git show HEAD:Test/Pkg2/Pkg2/Cube.h

The next tutorial part would be the_tutorial_merge1_start

Merge conflicts

About:

• Update Your development branch to a more recent upstream release using merge.
• Resolve merge conflicts

Tasks:

• Create a new branch fix-sphere-integration-test_merge for the tutorial parts merge1, and merge2 (later in the tutorial these specific branches are used to compare merge and rebase).
• Conserve current state in a new branch tmp_beforeMerge_2025-01-12T2101
• merge in changes from nightly/main/2025-01-12T2101, and resolve merge conflicts.
• "the" solution can be found in solution/merge1

Detailed explanation and help:

Situation: You developed a new feature in Your feature branch "fix-sphere-integration-test". After a long break You come back to work and want to continue Your development. You first will update Your branch to the most recent version. Typically that would be "nightly/main/...date string...." matching the (latest) nightly You have setup. You could also update to the "main" branch of the ATLAS athena repository, but that would not necessarily compile together with the nightly You have locally setup. There might be a situation in which You have to update to the "main" branch that, but that also means that You cannot easily compile locally and test Your changes.

One possibility to do the update is to merge the nightly branch into Your feature branch. That is what You should do in this tutorial part.

To simplify getting back to the state before a merge, it makes sense to create a branch which conserves the current state i.e. the state before merging, which You can delete later (e.g. create a branch tmp_beforeMerge_2025-01-12T2101). There are other possibilities to recover an old state. But this makes Your life easier in case You want to verify that nothing got lost after merging or because You screwed up and want to redo the merge. After that, merge in the changes from the latest nightly i.e. nightly/main/2025-01-12T2101. After the merge Your branch should look like solution/merge1 If You do not know what to do You find detailed instructions below.

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_merge1_start

In later tutorial parts we want to compare two approaches to update the working branch. Therefore the merge tutorial should be executed in its separate branch fix-sphere-integration-test_merge:

git checkout -b fix-sphere-integration-test_merge fix-sphere-integration-test

First memorise the current state in a new branch in case something goes wrong. There are other possibilities but branches are cheap and this is convenient:

git branch tmp_beforeMerge_2025-01-12T2101

Then merge in the changes from the branch with the most recent version:

git merge nightly/main/2025-01-12T2101

In this tutorial the merge will not succeed but produce conflicts which need to be resolved. You should see messages like "CONFLICT (content): Merge conflict in ... " and "git status" should show "both modified: ... " .

resolve conflicts:

• edit all the files which have conflicts.
• in this example there should be conflicts in "Test/Pkg2/Pkg2/Cube.h" and "Test/Alg/Alg/Integrator.h"
• "git merge" should tell, or "git status" should also remind You.
• In the files there are markers which indicate the text blocks with conflicts: start: "<<<<<<<" mid: "=======" end: ">>>>>>>" The part between "<<<<<<" and "=======" should be Your version prior to the merge The part between "=======" and ">>>>>>>" should be the version of the branch being merged in. The final version should likely be a combination of the two edits. The solution is not necessarily trivial.
• A conflict resolution has been prepared in a specific branch which can be used as a reference for comparisons or to actually resolve the merge conflicts.
• There can also be merge conflicts in files which are not part of the sparse checkout if You use sparse checkouts e.g. in case You committed changes to git in packages which You have removed from the sparse checkout later on, or the merge-source and Your tree already deviated prior to Your edits. In this case there are conflicts in Pkg2 which is not part of the sparse checkout if You did the sparse_checkout tutorial part.

If You do not know what to do, since You do not remember much about the changes gather some information. Let us start with the conflict in Cube.h. You could look at the commits which effect the file:

git log -- Test/Pkg2/Pkg2/Cube.h

Your edits: change from float to double

You can look at the changes of the commits. In this case it is the top most commit, so You can just look at the diff HEAD^..HEAD

git diff HEAD^..HEAD -- Test/Pkg2/Pkg2/Cube.h

Then You would look for relevant changes in the other branch:

git log nightly/main/2025-01-12T2101 -- Test/Pkg2/Pkg2/Cube.h

merged in changes: compiler warning fix concerning unused variables

In this case the commit is the second top most of all commits in "nightly/main/2025-01-12T2101". To look at the changes You can look at the diff using either the hash or the relative reference "nightly/main/2025-01-12T2101~2..nightly/main/2025-01-12T2101~1":

git diff nightly/main/2025-01-12T2101~2..nightly/main/2025-01-12T2101~1 -- Test/Pkg2/Pkg2/Cube.h

...edit files and resolve conflicts ...

(optional) to compare with the prepared conflict resolution

git diff solution/merge1  -- Test/Pkg2/Pkg2/Cube.h

(optional) use the prepared conflict resolution (though You should try first to solve the conflict by Yourself)

git checkout -f  solution/merge1 -- Test/Pkg2/Pkg2/Cube.h

stage the conflict resolution

git add Test/Pkg2/Pkg2/Cube.h

now the same for the other conflict:

git log -- Test/Alg/Alg/Integrator.h

Your edits: change from float to double, removed debug output

git log nightly/main/2025-01-12T2101 -- Test/Alg/Alg/Integrator.h

edits merged in: debug output was streamlined

...edit files and resolve conflicts ..., and compare with the prepared solution.

git diff solution/merge1 -- Test/Alg/Alg/Integrator.h

Beware, a merge may add stuff which is not desired but does not produce conflicts

(optional) use the prepared conflict resolution

git checkout -f  solution/merge1 -- Test/Alg/Alg/Integrator.h

stage the conflict resolution

git add Test/Alg/Alg/Integrator.h

conclude the merge

git merge --continue

(optional) create a check point to get back to the situation after tutorial merge1. This allows to restore this state as the starting condition for the tutorial part merge2.

git tag tutorial/merge1_end

The next tutorial part would be the_tutorial_merge2_start

Merge conflicts II

About:

• Similar to the_tutorial_merge1
• Update Your development branch to an even more recent upstream release using merge.
• Resolve merge conflicts
• Preparation to compare with the alternative approach, rebase, which is part of the next tutorial

Tasks:

• Perform some "developments" by cherry-picking "edit-follow-naming-convention".
• Before merging conserve the current state in tmp_beforeMerge_${AtlasBuildStamp}.
• Merge in changes from the nightly You have setup "nightly/main/${AtlasBuildStamp}"
• Compare with the prepared solution

Detailed explanation:

You continue developing Your feature, which takes a couple of days. Before the merge request You want to update Your branch to at least the latest nightly which You have setup already. To have an easy possibility to restore the situation before the merge You create a branch tmp_beforeMerge_${AtlasBuildStamp}, which You will delete later on. Then You use merge to update Your feature branch to "nightly/main/${AtlasBuildStamp}"

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_merge2_start

To simulate some development cherry-pick "edit-follow-naming-convention".

git cherry-pick edit-follow-naming-convention

Conserve the current state before merging for verification or to restore the situation before the merge:

git branch tmp_beforeMerge_${AtlasBuildStamp}

Then merge in the changes from the latest nightly:

git merge nightly/main/${AtlasBuildStamp}

Resolve conflicts like before (understand the developments in Your feature branch and the branch You try to merge; focus on the files with conflicts)

(optionally) You can compare with the prepared solution in "solution" to see whether You resolved the conflicts "correctly":

git diff solution  -- Test/Pkg1/Pkg1/Sphere.h
git diff solution  -- Test/Pkg2/Pkg2/Cube.h

(optionally) to enforce the version of the proposed solution:

git checkout -f  solution  -- Test/Pkg1/Pkg1/Sphere.h

Stage the changes:

git add Test/Pkg1/Pkg1/Sphere.h

git checkout -f  solution  -- Test/Pkg2/Pkg2/Cube.h

Stage the changes:

git add Test/Pkg2/Pkg2/Cube.h

conclude the merge

git merge --continue

(optional) You can compare Your version with the provided solution:

git diff solution

create a check point to get back to the situation after tutorial merge2

git tag tutorial/merge2_end

The next tutorial part would be the_tutorial_rebase1_start

Rebase conflict resolution

About:

• Use rebase instead of merge to include changes from the main branch.
• Very similar to merge1 but instructive to try both, merge and rebase.

Tasks:

• To compare the merge and rebase approach, do the rebase tutorial parts in a new branch fix-sphere-integration-test_rebase.
• Conserve the current state in a new branch tmp_beforeRebase_2025-01-12T2101, which You can delete later.
• Rebase Your work branch on the latest version nightly/main/2025-01-12T2101.
• "The" solution can be found in solution/rebase1

Detailed explanation:

To incorporate changes from other branches into Your working branch there is the possibility to merge in changes, which You did in merge1 and merge2, and there is the possibility to "rebase" Your changes on a new base version, which this tutorial is about. Rebase will add all Your changes on-top of that version one-by-one. To compare later on the merge and rebase approach, do the rebase tutorial in a new branch fix-sphere-integration-test_rebase. To facilitate verification that no changes got lost, or to return to the state before the rebase, conserve Your working branch in a temporary branch tmp_beforeRebase_2025-01-12T2101. Then use rebase to get the changes from nightly/main/2025-01-12T2101. In case of conflicts resolve the conflicts and compare with the prepared solutions. Since rebase will apply Your commits one after the other on a new base version each commit may have its own conflicts.

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_rebase1_start

Start from the prepared feature branch but do the rebase tutorial in a specific branch (to compare with the merge approach later)

git checkout -b fix-sphere-integration-test_rebase fix-sphere-integration-test

Keep current state in case something goes wrong or for verification

git branch tmp_beforeRebase_2025-01-12T2101

Rebase Your branch on a "more recent nightly"

git rebase nightly/main/2025-01-12T2101

Resolve 1st conflict. When using rebase Your edits are put on top of the new base version and the edits which produce conflicts are not yet committed. Thus the commit history of the current branch does not tell You anything about the intend of the edits. Only the error message of the rebase does, and the conflict marker comments in the files with conflicts.The conserved version in tmp_beforeRebase_2025-01-12T2101 might be helpful.

... edit files with conflicts ...

(Optionally) compare with the provided solution the solution can be found in solution/rebase1_remove_output

git diff solution/rebase1_remove_output -- Test/Alg/Alg/Integrator.h

beware, the rebase may add stuff which is not desired but does not produce conflicts

or use the provided solution

git checkout -f solution/rebase1_remove_output -- Test/Alg/Alg/Integrator.h

stage changes

git add Test/Alg/Alg/Integrator.h

continue the rebase

git rebase --continue

In case something goes seriously wrong, there is "git rebase --abort" which will bring You back to the state before the rebase. But all Your work after starting the rebase will be discarded.

Resolve 2nd conflict ... edit files with conflicts ...

(Optionally) compare with the provided solution the solution can be found in merge/fix-sphere-integration-test/2025-01-12T2101

git diff solution/rebase1_change_to_double -- Test/Pkg2/Pkg2/Cube.h

or use the provided solution

git checkout -f  solution/rebase1_change_to_double -- Test/Pkg2/Pkg2/Cube.h

stage changes

git add Test/Pkg2/Pkg2/Cube.h

continue the rebase

git rebase --continue

create a check point to get back to the situation after tutorial rebase1

git tag tutorial/rebase1_end

The next tutorial part would be the_tutorial_rebase2_start

Rebase conflict resolution II

About:

• like the previous tutorial rebase1, and similar to the second merge tutorial.

Tasks:

• Simulate some edits by cherry picking edit-follow-naming-convention.
• Before the rebase conserve the current state in tmp_beforeRebase_${AtlasBuildStamp}.
• Update Your working branch by rebasing on the most recent nightly nightly/main/${AtlasBuildStamp}.
• Compare with the prepared solution solution/rebase2.

Detailed explanation:

This is the same as merge2 but using rebase instead and very similar to rebase1. You continue working on Your feature, which is simulated by cherry-picking edit-follow-naming-convention. Before updating to the latest nightly You conserve the current state in a new branch tmp_beforeRebase_${AtlasBuildStamp}, which You can delete later. Then You rebase on "nightly/main/${AtlasBuildStamp}". Finally compare with the solution solution/rebase2.

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_rebase2_start

Your additional developments are simulated by cherry-picking

git cherry-pick edit-follow-naming-convention

Conserve current state in case something goes wrong

git branch tmp_beforeRebase_${AtlasBuildStamp}

Rebase to the latest nightly

git rebase nightly/main/${AtlasBuildStamp}

Resolve conflicts. ... edit files with conflicts ... (Optional) compare with the provided solution which is in "solution/rebase2_double_precision"

git diff solution/rebase2_double_precision  -- Test/Pkg1/Pkg1/Sphere.h

or use the provided solution

git checkout -f  solution/rebase2_double_precision  -- Test/Pkg1/Pkg1/Sphere.h

Stage changes

git add Test/Pkg1/Pkg1/Sphere.h

Compare with solution

git diff solution/rebase2_double_precision  -- Test/Pkg2/Pkg2/Cube.h

or use solution

git checkout -f  solution/rebase2_double_precision  -- Test/Pkg2/Pkg2/Cube.h

Stage changes

git add Test/Pkg2/Pkg2/Cube.h

Continue the rebase

git rebase --continue

Compare Your version with the provided solution:

git diff solution

Create a check point to get back to the situation after tutorial rebase2

git tag tutorial/rebase2_end

The next tutorial part would be the_tutorial_compare_history_start

Compare commit history after merge and rebase

About:

• Understand the different histories created by merge and rebase

Tasks:

• After executing merge1, merge2, rebase1, and rebase2, compare the histories resulting from the two approaches

Detailed explanation:

Look at the history resulting from merge and rebase. For this we look at the log, including the history graph drawing and decorations with tags and branch names.

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_compare_history_start

Look at the history resulting from the merge and rebase tutorial, annotate the commits with the tag and branch names and also show the relation between the nodes:

git log --format=oneline --decorate --graph  fix-sphere-integration-test_merge

git log --format=oneline --decorate --graph  fix-sphere-integration-test_rebase

You should see that rebase creates a much simpler history than merge. All Your (simulated edits) would be one after the other, rather than interleaved with commits from the merges. Thus, if You want to keep the commits as separate commits for e.g. creating multiple dependent merge requests, rather use rebase. If You will "squash" the commits anyway (see next tutorial part), merge may produce fewer merge conflicts.

The next tutorial part would be the_tutorial_squash_commits_start

Squash all commits of a feature branch into a single commit

About:

Sometimes the local history of a new feature becomes complicated, but for the outside world the history is of no interest. In such cases the commit history could be squashed into a single commit e.g. for a merge request.

Tasks:

• Create a branch "squash-test-feature" from "tutorial/merge2_end"
• Inspect the history, and understand which commits belong to Your developments.
• Create a new branch "squash-test-final" from "nightly/main/${AtlasBuildStamp}"
• Merge "squash-test-feature" into "into squash-test-final" with the "--squash" option.
• Compare "squash-test-final" and "squash-test-feature" to ensure that nothing got lost.

Detailed description

• For the tutorial create the branch "squash-test-feature" based on "tutorial/merge2_end"
• Development was started using the base "fix-sphere-integration-test-base"
• After some development which happened immediately after, and which can be seen in fix-sphere-integration-test-dev, You updated to "nightly/main/2025-01-12T2101", applied one additional fix "Follow naming convention", and finally to prepare for a merge request You updated to "nightly/main/${AtlasBuildStamp}" (tutorial parts merge1 + merge2).
• The commit history got complicated and for the merge request You want to squash all commits into a single one using the squash feature of merge.
• For this, You create a new branch "squash-test-final" which is based on the most recent version You updated Your developments to, which is "nightly/main/${AtlasBuildStamp}".
• Finally You merge Your development branch "squash-test-feature" into "squash-test-final" with the "--squash" option, and verify that nothing got lost.

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_squash_commits_start

For the this tutorial create the branch "squash-test-feature" from "tutorial/merge2_end"

git checkout -b squash-test-feature tutorial/merge2_end

Inspect the history, and try to understand which commits belong to Your developments.

git log --format=oneline --decorate --graph squash-test-feature

The state before You started development is in "fix-sphere-integration-test-base", most of the development is in fix-sphere-integration-test-dev, And then You applied one final fix "Follow naming convention"

Checkout a new branch which will be used for the merge request it should be based on the most recent version You updated Your development branch to with either merge or rebase.

git checkout -b squash-test-final nightly/main/${AtlasBuildStamp}

Now merge in Your feature and squash all commits

git merge --squash squash-test-feature

This will only stage all changes. The commit still needs to be made:

git commit

Write a commit message which comprises the developments of all the commits that got squashed into this single one.

Finally check that nothing was lost

git diff squash-test-feature

and lock at the new history

git log --format=oneline --decorate --graph

which should now look like "nightly/main/${AtlasBuildStamp}" but with the one extra commit You just have created.

The next tutorial part would be the_tutorial_interactive_rebase_start

Interactive rebase to cleanup the commit history (optional)

About:

• Use an interactive rebase to clean up the commit history: fix commit messages, fix changes, squash commits.

Tasks:

• Create a new branch "interactive-rebase" for the interactive rebase tutorial based on fix-sphere-integration-test_rebase
• Create a temporary branch tmp_beforeInteractiveRebase to conserve the current state (for verification or recovery).
• Start the interactive rebase from commit "interactive-rebase-first-commit".
• Fix a typo in the include guard in Test/Pkg1/Pkg1/Sphere.h, which was added in commit "Add sphere".
• Fix the commit message of commit "Increase precision". Change the message to "Increase precision of integration."
• Squash the commits "Follow naming convention" and "Underflow protection", because "Follow naming convention" is just a fix of the commit "Underflow protection" and in the cleaned history You do not want to have commits which require fixes.

Detailed description:

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_interactive_rebase_start

Create a new branch for the interactive rebase tutorial which should be based on "fix-sphere-integration-test_rebase"

git checkout -b interactive-rebase fix-sphere-integration-test_rebase

Create a temporary branch You can delete later to conserve the current state for later verification or recovery

git branch tmp_beforeInteractiveRebase

Now use the interactive rebase "rebase -i" to improve the commit history. In this tutorial You only marginally improve the commit history, but it should give You some ideas how the interactive rebase can be used. The starting commit would be "interactive-rebase-first-commit", which is a couple of commits earlier than Your current head. To alter the commit history starting from (but not including) this commit, execute:

git rebase -i interactive-rebase-first-commit

This will open an editor which contains a compact version of the commit history and some instructions. You can re-order the individual commits or request certain actions on the commits, like "pick", "drop", "edit", "squash". "pick" would just use the commit unmodified. "drop" would remove a commit. "edit" will pause the rebase and allow for all kinds of actions: use "git commit --amend" to modify a commit; add extra commits, or revert commits. Such changes may cause "merge" conflicts once the rebase is resumed. So, such changes may cause extra work. If the interactive rebase is started it will pause according to Your markings, to allow You to apply some modifications. Once You are happy with the changes to the commit (or the addition of commits or reverts), resumed the interactive rebase with "git rebase --continue". If You screwed up You can use "git rebase --abort", which will restore the state just before the rebase and discard all changes of this interactive rebase.

a) Your first task is to fix a typo in the commit "Add sphere". Find the commit in the editor and mark the commit for editing by replacing "pick" by "edit". b) The second task is to fix the commit message of commit "Increase precision". So, mark the corresponding line for editing as well. c) The final task is to squash the two commits "Follow naming convention", "Underflow protection". The action "squash" will squash a commit with the commit above, so You have to put the commits in the correct order. In this case You have to place "Follow naming convention" directly under "Underflow protection". Then mark the commit "Follow naming convention" for squashing by replacing "pick" with "squash".

Once, all the modifications You want to perform are marked in this steering file, save the file and exit the editor. This will start the interactive rebase. The interactive rebase will start from the first commit i.e. interactive-rebase-first-commit in this case, then play out the actions. The interactive rebase will pause on lines marked "edit", "squash" or in case there are merge conflicts which may happen if You edit files, add commits or You changed the order of commits in the steering file. When the rebase pauses perform the desired or necessary edits, or add extra commits, reverts, cherry-picks etc. , and resume the rebase with "git rebase --continue", or use "git rebase --abort" and retry the interactive rebase from the beginning

In this example the interactive rebase will pause after applying "Add sphere." because You marked this commit for editing. You should now edit "Test/Pkg1/Pkg1/Sphere.h", and replace "PKG1_SPHER_H" with "PKG1_SPHERE_H".

Check Your changes, stage them, rewrite the commit (the commit message is fine, so no editing of the message needed) with

git diff
git add Test/Pkg1/Pkg1/Sphere.h
git commit --amend --no-edit

and resume the rebase

git rebase --continue

After that, the rebase will pause once the commit "Increase precision" is applied, because You also marked this commit for editing. Now, just modify the commit message and change it to "Increase precision of integration." and e.g. look at the (truncated) history:

git commit --amend -m "Increase precision of integration."
git log -n 2

or use the editor by omitting -m "..." (recommended, but here the message is short and simple). Then resume the rebase

git rebase --continue

The rebase will pause again when squashing the commit "Follow naming convention" with "Underflow protection". Unfortunately it will produce a merge conflict. Git will tell You about the commit which causes the problem. So, You should have the information at hand to inspect the intention of the conflicting commit and solve the conflict. The conflict is in "Test/Pkg1/Pkg1/Sphere.h".

Here instead of actually solving the merge conflict, we just start from the version before the conflict and redo the edits the conflicting commit was about. In general this strategy will not work well, but in this case the conflicting commit is simple. First, the version before the conflict is checked out.

git checkout -f HEAD -- Test/Pkg1/Pkg1/Sphere.h

And then just redo the intended edits: replace save_sqrt by saveSqrt in Test/Pkg1/Pkg1/Sphere.h.

Check the changes:

git diff -- Test/Pkg1/Pkg1/Sphere.h

Note: if You just would have used the second of the two options which show up in "Test/Pkg1/Pkg1/Sphere.h" to solve the conflict, You would already incorporate changes of a later commit, which You "want" to keep separate in this tutorial. If You are satisfied with the changes, stage them

git add Test/Pkg1/Pkg1/Sphere.h

Check that You have not missed other conflicts and just resume the rebase

git status

git rebase --continue

When squashing commits the interactive rebase will offer to change the commit message. The editor will contain the commit messages of all commits that are being squashed. In most cases some editing is necessary. Write the full commit message for the squashed commits. In this case the second commit only corrects the first one and the message of the second commit can just be removed. Save the message, exit the editor, and the rebase will resume automatically.

Unfortunately the edits will lead to another conflict, which needs to be resolved. Since the changes are again simple one can use the same strategy and just checkout the version prior to the conflict and apply the changes of the conflicting commit manually which is just the change from float to double (look at the changes introduced by the conflicting commit). And again, this strategy works here, but in general may not be so useful.

git checkout -f HEAD -- Test/Pkg1/Pkg1/Sphere.h

Manually apply the changes, Check, and if You are satisfied stage them. Then verify that no conflicts remain and resume the rebase

git diff -- Test/Pkg1/Pkg1/Sphere.h
git add Test/Pkg1/Pkg1/Sphere.h
git status

git rebase --continue

The rest works fine. Finally we check that there are no unintended differences and compare with the version prior to the rebase:

git diff tmp_beforeInteractiveRebase

You applied one fix to a typo, so You should see this difference wrt. the version before the rebase. The full solution should look like the version in solution/interactive-rebase-unclean-history

git diff solution/interactive-rebase-unclean-history

but the history is not cleaned there. Also check the commit history that it looks as intended. (Optionally) finally tag Your solution for future investigations.

git tag tutorial/interactive_rebase_end

The next tutorial part would be the_tutorial_worktree_start

Worktrees (optional)

About:

• Work simultaneously on a feature development and some maintenance in different directories but with only one local repository clone using worktrees.
• Use sparse checkouts to limit the checked out files to the relevant ones.

Tasks:

• FEATURE: create directories to develop the feature "${top}/WT/FixIntegrationTest/source/", "${top}/WT/FixIntegrationTest/build", where "${top}" refers to the top level directory in which You created the mock-up directory.
• FEATURE: re-enter the mock-up repository (Main/source/test/)
• FEATURE: create a worktree where the source code should be in "${top}/WT/FixIntegrationTest/source/athena-fix-integration-test" and the feature should be developed in the branch "wt-fix-integration-test" based on "worktree-test-start", suppress the checkout and setup a sparse checkout containing "WorkDir", "Pkg1", "Base", "Alg".

• FEATURE: simulate some developments by cherry-picking a couple of commits "worktree-test-start..solution~3"

• MAIN: switch to a new worktree to do some maintenance on main, which shall happen in the directories: "${top}/WT/MaintenanceMain/source/", "${top}/WT/MaintenanceMain/build" in a new worktree "wt-maintenance-main" also based on "worktree-test-start". Use a sparse checkout of "WorkDir", "Pkg1", "Pkg2", "Base", "Alg".

• MAIN: simulate the development by cherry picking "main~2" and "main~1"

• FEATURE: switch back to the feature worktree to continue the development, which is identical to the rebase1 tutorial.

• MAIN: before starting rebase2, switch back to MAIN and finish the development which is simulated by cherry-picking "main".
• FEATURE: after that finish the feature development (identical to rebase2)

Detailed description:

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_worktree_start

Go to the top level directory in which You created the mock-up repository. That should be the directory which contains the "Main" folder. The variable "top" is likely not defined in Your shell, so You have to adjust the command

cd ${top}

FEATURE: Create directories for the "feature" development (adjust the command}

mkdir -p ${top}/WT/FixIntegrationTest/source/
mkdir -p ${top}/WT/FixIntegrationTest/build

FEATURE: Now go back to the directory which contains the git repository:

cd ${top}/Main/source/test/

FEATURE: Now create a new git worktree - which starts in a new branch "wt-fix-integration-test", - the working directory should be e.g. "../../../WT/FixIntegrationTest/FixIntegrationTest/source/athena-fix-integration-test". The directory which will contain the source code cannot be named "athena" because this name is used as a unique identifier for the git worktree. So, it must be a unique name. - The source branch should be based on "worktree-test-start" - We use a sparse checkout to not checkout all files (60k files in case of athena). So, when creating the worktree the checkout is suppressed.

git worktree add --no-checkout -b wt-fix-integration-test ${top}/WT/FixIntegrationTest/source/athena-fix-integration-test worktree-test-start

All the branches tags etc. will be in the repository in "...top level../Main/source/test/". The worktrees are just directories into which different branches are checked out (note: You cannot use the same branch in multiple worktrees at the same time). The advantages of worktrees are: reduced disk space usage; all branches, tags, remotes are visible in every worktree. A disadvantage might be that Your local repository gets more cluttered with development branches etc. .

FEATURE: Now You would work on the this new feature in the worktree, so enter the directory

cd ${top}/WT/FixIntegrationTest/source/athena-fix-integration-test

FEATURE: Since the checkout was suppressed the directory should be mostly empty. Now activate and setup the sparse checkout (disable "cone" pattern matching for git-atlas)

git sparse-checkout init --no-cone

FEATURE: and add the desired packages e.g. using the atlas extension (or "git sparse-checkout set Projects/WorkDir/ Test/{Pkg1,Base,Alg}/")

git atlas addpkg WorkDir
git atlas addpkg Pkg1
git atlas addpkg Base
git atlas addpkg Alg

FEATURE: The sparse checkout file should now contain Projects/WorkDir/ Test/Pkg1/ Test/Base/ Test/Alg/

git sparse-checkout list

FEATURE: Check out the files part of the sparse checkout (should already be done).

git checkout -f

FEATURE: Simulate some developments by cherry picking a few commits

git cherry-pick worktree-test-start..solution~3

MAIN: Suddenly, You are tasked to fix some problems in main. So, You have to interrupt Your current work on "wt-fix-integration-test". The work may require a different athena release (in this tutorial the same athena release is used). To avoid having to clone the athena repository again, You can just create a new worktree. Go to the top level of this tutorial which should be cd ../../../../ and create the new working directory "WT/MaintenanceMain/"

mkdir -p ${top}/WT/MaintenanceMain/source/
mkdir -p ${top}/WT/MaintenanceMain/build

MAIN: now again go to the main repository and create a worktree

cd ${top}/Main/source/test/

MAIN: the development should happen in the working directory "..toplevel.../MaintenanceMain/source/athena-maintenance-main" (remember to use a unique directory name at the lowest level) and in the branch "wt-maintenance-main". In this tutorial the source branch is again "worktree-test-start". Remember to add "--no-checkout" if You want to use a sparse checkout (otherwise for athena this will check out 60k files)

git worktree add --no-checkout -b wt-maintenance-main ${top}/WT/MaintenanceMain/source/athena-maintenance-main worktree-test-start

You currently have the following work trees:

git worktree list

MAIN: Go into the work directory

cd ${top}/WT/MaintenanceMain/source/athena-maintenance-main

MAIN: the sparse-checkout settings should be inherited from the main repository. To check:

git config core.sparsecheckout

MAIN: But the sparse checkout needs to be initialised for the worktree

git sparse-checkout init --no-cone

MAIN: Then add packages to the sparse checkout using the atlas extension (or "git sparse-checkout set Projects/WorkDir/ Test/{Pkg1,Pkg2,Base,Alg}/")

git atlas addpkg WorkDir
git atlas addpkg Pkg1
git atlas addpkg Pkg2
git atlas addpkg Base
git atlas addpkg Alg

MAIN: All files of these packages should already be checked out, otherwise:

git checkout -f

MAIN: To simulate some edits on Your side

git cherry-pick main~2
git cherry-pick main~1

MAIN: You can compile if You want (one test will fail because the test is going to be fixed in "wt-fix-integration-test"; not relevant for this tutorial):

cd ../../build
cmake ../source/athena-maintenance-main/Projects/WorkDir && make && make test

FEATURE: While cmake is running You could continue Your development (in e.g. a different shell; or wait for the compilation to finish). Move back to the directory in which You are developing the new feature:

cd ${top}/WT/FixIntegrationTest/source/athena-fix-integration-test

FEATURE: Now You fix the integration test as You did in the rebase (I+II) tutorial. You can use the provided solutions rather than fixing the conflicts Yourself).

git rebase nightly/main/2025-01-12T2101

FEATURE: resolve 1st conflict

git checkout -f  rebase/fix-sphere-integration-test/2025-01-12T2101_remove_output -- Test/Alg/Alg/Integrator.h
git add Test/Alg/Alg/Integrator.h

FEATURE: continue rebase

git rebase --continue

FEATURE: resolve 2nd conflict

git checkout -f  merge/fix-sphere-integration-test/2025-01-12T2101 -- Test/Pkg2/Pkg2/Cube.h
git add --sparse Test/Pkg2/Pkg2/Cube.h

FEATURE: continue rebase

git rebase --continue

FEATURE: continue development

git cherry-pick edit-follow-naming-convention

FEATURE: You can compile if You want

cd ../../build
cmake ../source/athena-fix-integration-test/Projects/WorkDir && make && make test

MAIN: While the feature compilation is running You can continue your maintenance work. So go back to the corresponding directory:

cd ${top}/WT/MaintenanceMain/source/athena-maintenance-main

MAIN: To simulate some development cherry-pick "main"

git cherry-pick main

MAIN: Create a check point tag for the maintenance work

git tag tutorial/worktree_maintenance_main

MAIN: You can compile if You want:

cd ../../build
cmake ../source/athena-maintenance-main/Projects/WorkDir && make && make test

FEATURE: While this is running You can continue Your feature development. So move back:

cd ${top}/WT/FixIntegrationTest/source/athena-fix-integration-test

FEATURE: update to "latest" main

git rebase nightly/main/${AtlasBuildStamp}

FEATURE: resolve conflicts

git checkout -f  solution/rebase2_double_precision  -- Test/Pkg1/Pkg1/Sphere.h
git add Test/Pkg1/Pkg1/Sphere.h

git checkout -f  solution/rebase2_double_precision  -- Test/Pkg2/Pkg2/Cube.h
git add --sparse Test/Pkg2/Pkg2/Cube.h

FEATURE: continue rebase

git rebase --continue

FEATURE: If You want You can compile:

cd ../../build
cmake ../source/athena-fix-integration-test/Projects/WorkDir && make && make test

Go back to the worktree and create a check point

cd ${top}/WT/FixIntegrationTest/source/athena-fix-integration-test
git tag tutorial/worktree_fix_integration_end

go back to main repository

cd ${top}/Main/source/test/

The next tutorial part would be the_tutorial_athena_example_worktree_start

Athena worktree and sparse checkout example (optional)

About:

• Similar to the worktree tutorial, but now using the full athena repository

Tasks:

• Clone the athena repository in "${top}/athena-example-Main/source"
• Activate sparse checkouts
• Create a worktree in the directory ${top}WT/athena-example/source/athena_example in the branch AthExHive based on "nightly/main/${AtlasBuildStamp}"
• Add "WorkDir" and AthExHive to the sparse checkout for the worktree for AthExHive

Detailed description:

To restore the initial condition for this tutorial part execute (not necessary if You do the tutorial in order and do this part for the first time)

sh git_tutorial2.sh --tutorial the_tutorial_athena_example_worktree_start

This just demonstrates how to use sparse checkouts and worktrees in conjunction with the full athena repository otherwise it is a simpler version of the worktree tutorial.

If You do not have a local clone of Your athena repository fork create one in e.g. athena-example-Main/source

mkdir ${top}/athena-example-Main
mkdir ${top}/athena-example-Main/source
cd ${top}/athena-example-Main/source

To avoid checking out all 60k files prior to setting up the sparse checkout pass the argument --no-checkout (if the repository is exclusively used in with worktrees, You can also just clone a "bare" repository without a checkout area).

git clone --no-checkout https://:@gitlab.cern.ch:8443/${USER}/athena.git
cd athena

Add the official atlas repository as a remote:

git remote add upstream https://:@gitlab.cern.ch:8443/atlas/athena.git
git fetch upstream

Activate sparse checkout for this new repository

git config core.sparsecheckout true

Create source and build directories for Your new work area and a directory for the worktree. Make sure that the lowest level directory which is used as the worktree has a unique name (athena is not).

mkdir -p  ../../../WT/athena-example/source/athena_example
mkdir -p  ../../../WT/athena-example/build/

Set up the git worktree for this new directory and pass "--no-checkout", because sparse checkout will be used, which will be setup after that

git worktree add --no-checkout -b AthExHive ../../../WT/athena-example/source/athena_example nightly/main/${AtlasBuildStamp}

Now enter the new worktree

cd ../../../WT/athena-example/source/athena_example/

The worktree will inherit the sparse checkout config flag, but the sparse checkout needs to be initialized for the work tree. Check that sparse checkout is active:

git config core.sparsecheckout

Initialise the sparse checkout

git sparse-checkout init

Now use the atlas extension to add the "WorkDir", which is needed for building, and the packages You need to work which is in this tutorial AthExHive

git atlas addpkg WorkDir
git atlas addpkg AthExHive

The packages should already be checked out, if not

git checkout -f

implement Your changes and build the packages in Your sparse checkout

cd ../../build/

cmake ../source/athena_example/Projects/WorkDir && make
make test

go back to main repository

cd ${top}/Main/source/test/