Run git pull over all subdirectories

0 votes
asked Aug 16, 2010 by petah

How can I update multiple git repositories from their shared parent's directory without cd'ing into each repo's root directory? I have the following which are all separate git repositories (not submodules):

/plugins/cms
/plugins/admin
/plugins/chart

I want to update them all at once or at least simplify my current workflow:

cd ~/plugins/admin
git pull origin master
cd ../chart
git pull

etc.

16 Answers

0 votes
answered Aug 16, 2010 by jamie-wong

This should happen automatically, so long as cms, admin and chart are all parts of the repository.

A likely issue is that each of these plugins is a git submodule.

Run git help submodule for more information.

EDIT

For doing this in bash:

cd plugins
for f in cms admin chart
do 
  cd $f && git pull origin master && cd ..
done
0 votes
answered Aug 16, 2010 by vonc

Original answer 2010:

If all of those directories are separate git repo, you should reference them as submodules.

That means your "origin" would be that remote repo 'plugins' which only contains references to subrepos 'cms', 'admin', 'chart'.

A git pull followed by a git submodule update would achieve what your are looking for.


Update January 2016:

With Git 2.8 (Q1 2016), you will be able to fetch submodules in parallel (!) with git fetch --recurse-submodules -j2.
See "How to speed up / parallelize downloads of git submodules using git clone --recursive?"

0 votes
answered Aug 19, 2012 by leo

Run the following from the parent directory, plugins in this case:

find . -type d -depth 1 -exec git --git-dir={}/.git --work-tree=$PWD/{} pull origin master \;

To clarify:

  • find . searches the current directory
  • -type d to find directories, not files
  • -depth 1 for a maximum depth of one sub-directory
  • -exec {} \; runs a custom command for every find
  • git --git-dir={}/.git --work-tree=$PWD/{} pull git pulls the individual directories

To play around with find, I recommend using echo after -exec to preview, e.g.:

find . -type d -depth 1 -exec echo git --git-dir={}/.git --work-tree=$PWD/{} status \;

Note: if the -depth 1 option is not available, try -mindepth 1 -maxdepth 1.

0 votes
answered Jan 13, 2015 by cphyc

If you have a lot of subdirs with git repositories, you can use parallel

ls | parallel -I{} -j100 '
  if [ -d {}/.git ]; then
    echo Pulling {}
    git -C {} pull > /dev/null && echo "pulled" || echo "error :("
  else
     echo {} is not a .git directory
  fi
'
0 votes
answered Aug 3, 2015 by chirag-visavadiya

You can try this

find . -type d -name .git -exec sh -c "cd \"{}\"/../ && pwd && git pull" \;

Also, you can add your customized output by adding one more && argument like.

find . -type d -name .git -exec sh -c "cd \"{}\"/../ && pwd && git pull && git status" \;
0 votes
answered Aug 5, 2015 by ingo-blechschmidt

A bit more low-tech than leo's solution:

for i in */.git; do ( echo $i; cd $i/..; git pull; ); done

This will update all Git repositories in your working directory. No need to explicitly list their names ("cms", "admin", "chart"). The "cd" command only affects a subshell (spawned using the parenthesis).

0 votes
answered Aug 6, 2015 by cmcginty

Most compact method, assuming all sub-dirs are git repos:

ls | parallel git -C {} pull
0 votes
answered Aug 16, 2015 by dmitry-mitskevich
ls | xargs -I{} git -C {} pull

To do it in parallel:

ls | xargs -P10 -I{} git -C {} pull
0 votes
answered Aug 17, 2015 by evadeflow

The mr utility (a.k.a., myrepos) provides an outstanding solution to this very problem. Install it using your favorite package manager, or just grab the mr script directly from github and put it in $HOME/bin or somewhere else on your PATH. Then, cd to the parent plugins folder shared by these repos and create a basic .mrconfig file with contents similar to the following (adjusting the URLs as needed):

# File: .mrconfig
[cms]
checkout = git clone 'https://<username>@github.com/<username>/cms' 'cms'

[admin]
checkout = git clone 'https://<username>@github.com/<username>/admin' 'admin'

[chart]
checkout = git clone 'https://<username>@github.com/<username>/chart' 'chart'

After that, you can run mr up from the top level plugins folder to pull updates from each repository. (Note that this will also do the initial clone if the target working copy doesn't yet exist.) Other commands you can execute include mr st, mr push, mr log, mr diff, etc—run mr help to see what's possible. There's a mr run command that acts as a pass-through, allowing you to access VCS commands not directly suported by mr itself (e.g., mr run git tag STAGING_081220015). And you can even create your own custom commands that execute arbitrary bits of shell script targeting all repos!

mr is an extremely useful tool for dealing with multiple repos. Since the plugins folder is in your home directory, you might also be interested in vcsh. Together with mr, it provides a powerful mechanism for managing all of your configuration files. See this blog post by Thomas Ferris Nicolaisen for an overview.

0 votes
answered Aug 6, 2016 by p-pang

I use this

for dir in $(find . -name ".git")
do cd ${dir%/*}
    echo $PWD
    git pull
    echo ""
    cd - > /dev/null
done

Github

Welcome to Q&A, where you can ask questions and receive answers from other members of the community.
Website Online Counter

...