Execute a command as soon as per line of piped input?

I intend to run a java command as soon as for every single suit of ls | grep pattern -. In this instance, I assume I could do find pattern -exec java MyProg '{}' \; yet I'm interested concerning the basic instance - exists a very easy means to claim "run a command as soon as for every single line of typical input"? (In fish or bash.)

2019-05-18 23:14:16
Source Share
Answers: 5

GNU Parallel is created that sort of jobs. The most basic use is:

cat stuff | grep pattern | parallel java MyProg

Watch the introductory video to read more: http://www.youtube.com/watch?v=OpaiGYxkSuQ

2019-05-21 16:38:30

In Bash or any kind of various other Bourne - design shell (ash, ksh, zsh, ):

while read -r line; do command "$line"; done

read -r reviews a solitary line from typical input (read without -r analyzes backslashes, you do not desire that). Hence you can do either of the following:

$ command | while read -r line; do command "$line"; done  

$ while read -r line; do command "$line"; done <file
2019-05-21 09:56:24

I concur with Keith, xargs is one of the most basic device for the work.

I generally make use of a 3 action strategy.

  • do the standard things till you have something you would love to collaborate with
  • prepare the line with awk so it obtains the proper syntax
  • after that allowed xargs execute it, possibly with the aid of bash.

There are smaller sized and also faster means, yet in this manner generally functions.

A straightforward instance:

ls | 
grep xls | 
awk '{print "MyJavaProg --arg1 42 --arg2 "$1"\0"}' | 
xargs -0 bash -c

the 2 first lines picks some documents to collaborate with, after that awk prepares a wonderful string with a command to execute and also some debates and also $1 is the first column input from the pipeline. And also ultimately I see to it that xargs sends this string to bash that simply execute it.

It is a little excessive, yet this dish has actually aided me in a great deal of areas given that it is really adaptable.

2019-05-21 09:06:22

That is what xargs does.

... | xargs command
2019-05-21 08:06:05

When managing possibly unsanitized inputs, I such as to see the whole work 'defined' line by line for aesthetic examination prior to I run it (specifically when it is something devastating like cleansing individuals is mail box's).

So what I do is create a checklist of parameters (ie. usernames), feed it to a documents in one - document - per - line style, similar to this:


Then I open the checklist in vim, and also maul it with search and also change expressions till I get a checklist of complete commands that require to get implemented, similar to this:

/bin/rm -fr /home/johndoe  
/bin/rm -fr /home/jamessmith 

This means if your regex is insufficient, you will certainly see in what command will certainly have possible troubles (ie. /bin/rm -fr johnnyo connor). In this manner you can undo your regex, and also attempt it once more with an extra trusted variation of it. Call mangling is infamous for this, due to the fact that it is upsetting treatment of all the side instances like Van Gogh, O'Connors, St. Clair, Smith - Wesson.

Having set hlsearch serves for doing this in vim, as it will certainly highlight all the suits, so you can conveniently detect if it does not match, or suits in an unplanned means.

As soon as your regex is excellent and also it captures all the instances you can examine for/think of, after that I generally transform it to a sed expression so it can be totally automated for an additional run.

For instances where the variety of lines of input stops you from doing an aesthetic examination, I very advise resembling the command to the screen (or even better, a log) prior to it implements, so if it mistakes out, you recognize specifically which command created it to fall short. After that you can return to your initial regex and also readjust one more time.

2019-05-19 12:17:53