0 0 0 0


How to wait in a bash script for several subprocesses spawned from that script to finish and return exit code !=0 when any of the subprocesses ends with code !=0 ?

Simple script:

for i in `seq 0 9`; do
  doCalculations $i &

The above script will wait for all 10 spawned subprocesses, but it will always give exit status 0 (see help wait). How can I modify this script so it will discover exit statuses of spawned subprocesses and return exit code 1 when any of subprocesses ends with code !=0?

Is there any better solution for that than collecting PIDs of the subprocesses, wait for them in order and sum exit statuses?

Best Answer:

To parallelize this...

for i in $(whatever_list) ; do
   do_something $i

Translate it to this...

for i in $(whatever_list) ; do echo $i ; done | ## execute in parallel...
   export -f do_something ## export functions (if needed)
   export PATH ## export any variables that are required
   xargs -I{} --max-procs 0 bash -c ' ## process in batches...
      echo "processing {}" ## optional
      do_something {}
  • If an error occurs in one process, it won't interrupt the other processes, but it will result in a non-zero exit code from the sequence as a whole.
  • Exporting functions and variables may or may not be necessary, in any particular case.
  • You can set --max-procs based on how much parallelism you want (0 means "all at once").
  • GNU Parallel offers some additional features when used in place of xargs -- but it isn't always installed by default.
  • The for loop isn't strictly necessary in this example since echo $i is basically just regenerating the output of $(whatever_list). I just think the use of the for keyword makes it a little easier to see what is going on.
  • Bash string handling can be confusing -- I have found that using single quotes works best for wrapping non-trivial scripts.
  • You can easily interrupt the entire operation (using ^C or similar), unlike the the more direct approach to Bash parallelism.

Here's a simplified working example...

for i in {0..5} ; do echo $i ; done |xargs -I{} --max-procs 2 bash -c '
   echo sleep {}
   sleep 2s

Copyright © 2011 Dowemo All rights reserved.    Creative Commons   AboutUs