Is there a way to parallelize a bash for loop? -
i have simple script pulls smart data series of hard drives , writes timestamped log file later logged , parsed relevant data.
filename="filename$( date '+%y_%m_%d_%h%m' ).txt" in {a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p} smartctl -a /dev/sd$i >> /path/to/location/$filename done
since takes several seconds run, find way parallelize it. i've tried appending '&' end of single line in loop, causes text file written haphazardly sections finish rather sequentially , in readable manner. there way fork seperate processes each drive pipe output orderly text file?
also, assume setting filename variable have moved loop in order forks able access it. causes issue if script runs long enough roll on new minute (or two) , script becomes sequentially datestamped fragments rather 1 contiguous file.
with gnu parallel this:
parallel -k 'smartctl -a /dev/{}' ::: b c d e f g h j k l m n o p > path/to/output
the -k
option keeps
output in order. add -j 8
if want run, say, 8 @ time, else 1 per core @ time. or -j 16
if want run them @ once...
parallel -j 16 -k 'smartctl ....
of course, if in bash
can too:
parallel -j 16 -k 'smartctl -a /dev/{}' ::: {a..o} > path/to/output
Comments
Post a Comment