how to split input to two pipes
I would like to do something equivalent to this
some-expensive-command > /tmp/mytempfile
grep -v "pattern" /tmp/mytempfile >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
preferably elegant and without the need for the tempfile. I was thinking about piping through tee, but the best I can think of might combine two of the three lines and still require the intermediate storage:
some-expensive-command | tee /tmp/mytempfile | grep -v "pattern" >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
command-line pipe
add a comment |
I would like to do something equivalent to this
some-expensive-command > /tmp/mytempfile
grep -v "pattern" /tmp/mytempfile >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
preferably elegant and without the need for the tempfile. I was thinking about piping through tee, but the best I can think of might combine two of the three lines and still require the intermediate storage:
some-expensive-command | tee /tmp/mytempfile | grep -v "pattern" >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
command-line pipe
So you want one command's output saved to fileoutput.txtand that same output redirected to another command for further processing ? Is that what you're trying to do ?
– Sergiy Kolodyazhnyy
Nov 17 '18 at 22:29
add a comment |
I would like to do something equivalent to this
some-expensive-command > /tmp/mytempfile
grep -v "pattern" /tmp/mytempfile >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
preferably elegant and without the need for the tempfile. I was thinking about piping through tee, but the best I can think of might combine two of the three lines and still require the intermediate storage:
some-expensive-command | tee /tmp/mytempfile | grep -v "pattern" >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
command-line pipe
I would like to do something equivalent to this
some-expensive-command > /tmp/mytempfile
grep -v "pattern" /tmp/mytempfile >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
preferably elegant and without the need for the tempfile. I was thinking about piping through tee, but the best I can think of might combine two of the three lines and still require the intermediate storage:
some-expensive-command | tee /tmp/mytempfile | grep -v "pattern" >> output.txt
grep "pattern" /tmp/mytempfile | yet-another-command
command-line pipe
command-line pipe
asked Nov 17 '18 at 22:07
Hagen von EitzenHagen von Eitzen
5211
5211
So you want one command's output saved to fileoutput.txtand that same output redirected to another command for further processing ? Is that what you're trying to do ?
– Sergiy Kolodyazhnyy
Nov 17 '18 at 22:29
add a comment |
So you want one command's output saved to fileoutput.txtand that same output redirected to another command for further processing ? Is that what you're trying to do ?
– Sergiy Kolodyazhnyy
Nov 17 '18 at 22:29
So you want one command's output saved to file
output.txt and that same output redirected to another command for further processing ? Is that what you're trying to do ?– Sergiy Kolodyazhnyy
Nov 17 '18 at 22:29
So you want one command's output saved to file
output.txt and that same output redirected to another command for further processing ? Is that what you're trying to do ?– Sergiy Kolodyazhnyy
Nov 17 '18 at 22:29
add a comment |
2 Answers
2
active
oldest
votes
The way question reads it sounds like you want one stdin redirected to two different commands. If that's the case, take advantage of tee plus process substitution:
some-expensive-command | tee >(grep 'pattern' > output.txt) >(grep -v 'pattern' | another-command)
Another way to look at this is by recognizing that grep is line pattern matching tool, so by reading line at a time and using that same line in multiple commands we can achieve exactly the same effect:
rm output.txt # get rid of file so that we don't add old and new output
some-expensive-command | while IFS= read -r line || [ -n "$line" ]; do
printf "%sn" "$line" | grep 'pattern' >> output.txt
printf "%sn" "$line" | grep -v 'pattern' | another-command
done
# or if another-command needs all of the output,
# place `| another-comand` after `done` clause
Another way is to abandon grep and use something more powerful, like awk:
some-expensive-command | awk '/pattern/{print >> "output.txt"}; !/pattern/{print}' | another-command.
Practically speaking, don't worry about using temporary files, so long as you clean them up after using. If it works, it works.
add a comment |
Use bash Process Substitution:
some-command | tee >(grep "pat" | another-command >>out1) | grep -v "pat" >>out2
The process substitution assigns some-command’s output to grep "pat"’s input, thus saving you the tempfile. Of course the data is still saved in a file (it’s always), just that you don’t have to take care of that. If you don’t want to save another-command’s output in a file but rather print it I recommend to simply switch the two command lists.
Another nice source of information: man bash/EXPANSION
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "89"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1093843%2fhow-to-split-input-to-two-pipes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
The way question reads it sounds like you want one stdin redirected to two different commands. If that's the case, take advantage of tee plus process substitution:
some-expensive-command | tee >(grep 'pattern' > output.txt) >(grep -v 'pattern' | another-command)
Another way to look at this is by recognizing that grep is line pattern matching tool, so by reading line at a time and using that same line in multiple commands we can achieve exactly the same effect:
rm output.txt # get rid of file so that we don't add old and new output
some-expensive-command | while IFS= read -r line || [ -n "$line" ]; do
printf "%sn" "$line" | grep 'pattern' >> output.txt
printf "%sn" "$line" | grep -v 'pattern' | another-command
done
# or if another-command needs all of the output,
# place `| another-comand` after `done` clause
Another way is to abandon grep and use something more powerful, like awk:
some-expensive-command | awk '/pattern/{print >> "output.txt"}; !/pattern/{print}' | another-command.
Practically speaking, don't worry about using temporary files, so long as you clean them up after using. If it works, it works.
add a comment |
The way question reads it sounds like you want one stdin redirected to two different commands. If that's the case, take advantage of tee plus process substitution:
some-expensive-command | tee >(grep 'pattern' > output.txt) >(grep -v 'pattern' | another-command)
Another way to look at this is by recognizing that grep is line pattern matching tool, so by reading line at a time and using that same line in multiple commands we can achieve exactly the same effect:
rm output.txt # get rid of file so that we don't add old and new output
some-expensive-command | while IFS= read -r line || [ -n "$line" ]; do
printf "%sn" "$line" | grep 'pattern' >> output.txt
printf "%sn" "$line" | grep -v 'pattern' | another-command
done
# or if another-command needs all of the output,
# place `| another-comand` after `done` clause
Another way is to abandon grep and use something more powerful, like awk:
some-expensive-command | awk '/pattern/{print >> "output.txt"}; !/pattern/{print}' | another-command.
Practically speaking, don't worry about using temporary files, so long as you clean them up after using. If it works, it works.
add a comment |
The way question reads it sounds like you want one stdin redirected to two different commands. If that's the case, take advantage of tee plus process substitution:
some-expensive-command | tee >(grep 'pattern' > output.txt) >(grep -v 'pattern' | another-command)
Another way to look at this is by recognizing that grep is line pattern matching tool, so by reading line at a time and using that same line in multiple commands we can achieve exactly the same effect:
rm output.txt # get rid of file so that we don't add old and new output
some-expensive-command | while IFS= read -r line || [ -n "$line" ]; do
printf "%sn" "$line" | grep 'pattern' >> output.txt
printf "%sn" "$line" | grep -v 'pattern' | another-command
done
# or if another-command needs all of the output,
# place `| another-comand` after `done` clause
Another way is to abandon grep and use something more powerful, like awk:
some-expensive-command | awk '/pattern/{print >> "output.txt"}; !/pattern/{print}' | another-command.
Practically speaking, don't worry about using temporary files, so long as you clean them up after using. If it works, it works.
The way question reads it sounds like you want one stdin redirected to two different commands. If that's the case, take advantage of tee plus process substitution:
some-expensive-command | tee >(grep 'pattern' > output.txt) >(grep -v 'pattern' | another-command)
Another way to look at this is by recognizing that grep is line pattern matching tool, so by reading line at a time and using that same line in multiple commands we can achieve exactly the same effect:
rm output.txt # get rid of file so that we don't add old and new output
some-expensive-command | while IFS= read -r line || [ -n "$line" ]; do
printf "%sn" "$line" | grep 'pattern' >> output.txt
printf "%sn" "$line" | grep -v 'pattern' | another-command
done
# or if another-command needs all of the output,
# place `| another-comand` after `done` clause
Another way is to abandon grep and use something more powerful, like awk:
some-expensive-command | awk '/pattern/{print >> "output.txt"}; !/pattern/{print}' | another-command.
Practically speaking, don't worry about using temporary files, so long as you clean them up after using. If it works, it works.
edited Nov 18 '18 at 1:33
answered Nov 17 '18 at 22:58
Sergiy KolodyazhnyySergiy Kolodyazhnyy
71.8k9148314
71.8k9148314
add a comment |
add a comment |
Use bash Process Substitution:
some-command | tee >(grep "pat" | another-command >>out1) | grep -v "pat" >>out2
The process substitution assigns some-command’s output to grep "pat"’s input, thus saving you the tempfile. Of course the data is still saved in a file (it’s always), just that you don’t have to take care of that. If you don’t want to save another-command’s output in a file but rather print it I recommend to simply switch the two command lists.
Another nice source of information: man bash/EXPANSION
add a comment |
Use bash Process Substitution:
some-command | tee >(grep "pat" | another-command >>out1) | grep -v "pat" >>out2
The process substitution assigns some-command’s output to grep "pat"’s input, thus saving you the tempfile. Of course the data is still saved in a file (it’s always), just that you don’t have to take care of that. If you don’t want to save another-command’s output in a file but rather print it I recommend to simply switch the two command lists.
Another nice source of information: man bash/EXPANSION
add a comment |
Use bash Process Substitution:
some-command | tee >(grep "pat" | another-command >>out1) | grep -v "pat" >>out2
The process substitution assigns some-command’s output to grep "pat"’s input, thus saving you the tempfile. Of course the data is still saved in a file (it’s always), just that you don’t have to take care of that. If you don’t want to save another-command’s output in a file but rather print it I recommend to simply switch the two command lists.
Another nice source of information: man bash/EXPANSION
Use bash Process Substitution:
some-command | tee >(grep "pat" | another-command >>out1) | grep -v "pat" >>out2
The process substitution assigns some-command’s output to grep "pat"’s input, thus saving you the tempfile. Of course the data is still saved in a file (it’s always), just that you don’t have to take care of that. If you don’t want to save another-command’s output in a file but rather print it I recommend to simply switch the two command lists.
Another nice source of information: man bash/EXPANSION
edited Nov 17 '18 at 22:32
answered Nov 17 '18 at 22:15
dessertdessert
22.8k56398
22.8k56398
add a comment |
add a comment |
Thanks for contributing an answer to Ask Ubuntu!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2faskubuntu.com%2fquestions%2f1093843%2fhow-to-split-input-to-two-pipes%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
So you want one command's output saved to file
output.txtand that same output redirected to another command for further processing ? Is that what you're trying to do ?– Sergiy Kolodyazhnyy
Nov 17 '18 at 22:29