How can I use a bash variable as a regex argument to gsub() and match()?
1
I have a script
#! /bin/bash
awk -v pat="$1" '{ if(match($0, pat)) {print $0} }' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep); print $0}' "$3"
I would like to run it like
myscript '..+' ' ' ./inputfile
where $1 is supposed to be a regex understandable by awk (GNU awk).
Specifying pat as a regex expression as arguments to gsub() and match() will not work. I was wondering how I can make it work? Thanks.
text-processing awk regular-expression variable
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
add a comment |
1
I have a script
#! /bin/bash
awk -v pat="$1" '{ if(match($0, pat)) {print $0} }' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep); print $0}' "$3"
I would like to run it like
myscript '..+' ' ' ./inputfile
where $1 is supposed to be a regex understandable by awk (GNU awk).
Specifying pat as a regex expression as arguments to gsub() and match() will not work. I was wondering how I can make it work? Thanks.
text-processing awk regular-expression variable
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
add a comment |
1
1
1
I have a script
#! /bin/bash
awk -v pat="$1" '{ if(match($0, pat)) {print $0} }' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep); print $0}' "$3"
I would like to run it like
myscript '..+' ' ' ./inputfile
where $1 is supposed to be a regex understandable by awk (GNU awk).
Specifying pat as a regex expression as arguments to gsub() and match() will not work. I was wondering how I can make it work? Thanks.
text-processing awk regular-expression variable
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
I have a script
#! /bin/bash
awk -v pat="$1" '{ if(match($0, pat)) {print $0} }' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep); print $0}' "$3"
I would like to run it like
myscript '..+' ' ' ./inputfile
where $1 is supposed to be a regex understandable by awk (GNU awk).
Specifying pat as a regex expression as arguments to gsub() and match() will not work. I was wondering how I can make it work? Thanks.
text-processing awk regular-expression variable
text-processing awk regular-expression variable
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
edited Nov 13 '18 at 15:56
Jeff Schaller
39.3k1054125
39.3k1054125
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
asked Nov 13 '18 at 14:34
TimTim
26.4k75248457
26.4k75248457
add a comment |
add a comment |
2 Answers
2
active
oldest
votes
3
You’re going about things correctly, as far as passing variables is concerned. To make the expression work you need to double the backslashes, at least with Gawk:
myscript '\.\.+' ' ' ./inputfile
Note that you can simplify the AWK parts:
#!/bin/bash
awk -v pat="$1" 'match($0, pat)' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep)} 1' "$3"
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
Thanks. Yes my typo. (1) Could you explain the reason of doubling backslash? Is it correct that bash will remove the backslash in$1inawk -v pat="$1"? Does it work if gsub() sees the pattern argument without any backslash? Does gsub() not require its pattern argument to be enclosed in/.../? (2) Is it possible to write a bash script, so that I can just specify a regex as an argument to the script in the same way as I specify it to awk directly?
– Tim
Nov 13 '18 at 14:56
Sorry, ignore partially my (1). I was thinking of something else. I need to specify literal dot to awk, so I need literal backslash to escape it.
– Tim
Nov 13 '18 at 15:22
(1) See the link in my updated answer re the double backslashes. Bash doesn’t remove the backslash, but AWK needs two because it processes them twice.gsub()doesn’t need/.../. (2) You could double the backslashes before calling AWK...
– Stephen Kitt
Nov 13 '18 at 15:49
Also: gnu.org/software/gawk/manual/html_node/Gory-Details.html
– glenn jackman
Nov 13 '18 at 16:03
notice that the behavior wrt."."is different betweengawkandmawk(the default on debian); ex.echo foo.bar | awk '{gsub(".","X")}1'will print different things depending on whichawkwas used. The values of variables set on the command line are treated exactly as if they were enclosed in", and the standard leaves the behavior of awk wrt. non-standard escapes (other thann,t, etc) undefined.
– mosvy
Nov 13 '18 at 16:03
|
show 3 more comments
2
Just a quick answer with a different approach : script written in Awk:
#!/usr/bin/gawk -f
BEGIN { find=ARGV[1]; repl=ARGV[2]; delete ARGV[1]; delete ARGV[2]}
{ print gensub(find,repl,"g",$0) }
and then the usual Unix filter behavior:
$ chmod 755 myawkscript
$ ls |myawkscript 'w(.)' '{1}'
{y}{w}{s}{r}{p}t
$ myawkscript '(w+)' '{1}' myawkscript
#!/{usr}/{bin}/{gawk} -{f}
{BEGIN} { {find}={ARGV}[{1}]; {repl}={ARGV}[{2}]; .....
{ {print} {gensub}({find},{repl},"{g}",${0}) }
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f481470%2fhow-can-i-use-a-bash-variable-as-a-regex-argument-to-gsub-and-match%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
3
You’re going about things correctly, as far as passing variables is concerned. To make the expression work you need to double the backslashes, at least with Gawk:
myscript '\.\.+' ' ' ./inputfile
Note that you can simplify the AWK parts:
#!/bin/bash
awk -v pat="$1" 'match($0, pat)' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep)} 1' "$3"
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
Thanks. Yes my typo. (1) Could you explain the reason of doubling backslash? Is it correct that bash will remove the backslash in$1inawk -v pat="$1"? Does it work if gsub() sees the pattern argument without any backslash? Does gsub() not require its pattern argument to be enclosed in/.../? (2) Is it possible to write a bash script, so that I can just specify a regex as an argument to the script in the same way as I specify it to awk directly?
– Tim
Nov 13 '18 at 14:56
Sorry, ignore partially my (1). I was thinking of something else. I need to specify literal dot to awk, so I need literal backslash to escape it.
– Tim
Nov 13 '18 at 15:22
(1) See the link in my updated answer re the double backslashes. Bash doesn’t remove the backslash, but AWK needs two because it processes them twice.gsub()doesn’t need/.../. (2) You could double the backslashes before calling AWK...
– Stephen Kitt
Nov 13 '18 at 15:49
Also: gnu.org/software/gawk/manual/html_node/Gory-Details.html
– glenn jackman
Nov 13 '18 at 16:03
notice that the behavior wrt."."is different betweengawkandmawk(the default on debian); ex.echo foo.bar | awk '{gsub(".","X")}1'will print different things depending on whichawkwas used. The values of variables set on the command line are treated exactly as if they were enclosed in", and the standard leaves the behavior of awk wrt. non-standard escapes (other thann,t, etc) undefined.
– mosvy
Nov 13 '18 at 16:03
|
show 3 more comments
3
You’re going about things correctly, as far as passing variables is concerned. To make the expression work you need to double the backslashes, at least with Gawk:
myscript '\.\.+' ' ' ./inputfile
Note that you can simplify the AWK parts:
#!/bin/bash
awk -v pat="$1" 'match($0, pat)' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep)} 1' "$3"
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
Thanks. Yes my typo. (1) Could you explain the reason of doubling backslash? Is it correct that bash will remove the backslash in$1inawk -v pat="$1"? Does it work if gsub() sees the pattern argument without any backslash? Does gsub() not require its pattern argument to be enclosed in/.../? (2) Is it possible to write a bash script, so that I can just specify a regex as an argument to the script in the same way as I specify it to awk directly?
– Tim
Nov 13 '18 at 14:56
Sorry, ignore partially my (1). I was thinking of something else. I need to specify literal dot to awk, so I need literal backslash to escape it.
– Tim
Nov 13 '18 at 15:22
(1) See the link in my updated answer re the double backslashes. Bash doesn’t remove the backslash, but AWK needs two because it processes them twice.gsub()doesn’t need/.../. (2) You could double the backslashes before calling AWK...
– Stephen Kitt
Nov 13 '18 at 15:49
Also: gnu.org/software/gawk/manual/html_node/Gory-Details.html
– glenn jackman
Nov 13 '18 at 16:03
notice that the behavior wrt."."is different betweengawkandmawk(the default on debian); ex.echo foo.bar | awk '{gsub(".","X")}1'will print different things depending on whichawkwas used. The values of variables set on the command line are treated exactly as if they were enclosed in", and the standard leaves the behavior of awk wrt. non-standard escapes (other thann,t, etc) undefined.
– mosvy
Nov 13 '18 at 16:03
|
show 3 more comments
3
3
3
You’re going about things correctly, as far as passing variables is concerned. To make the expression work you need to double the backslashes, at least with Gawk:
myscript '\.\.+' ' ' ./inputfile
Note that you can simplify the AWK parts:
#!/bin/bash
awk -v pat="$1" 'match($0, pat)' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep)} 1' "$3"
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
You’re going about things correctly, as far as passing variables is concerned. To make the expression work you need to double the backslashes, at least with Gawk:
myscript '\.\.+' ' ' ./inputfile
Note that you can simplify the AWK parts:
#!/bin/bash
awk -v pat="$1" 'match($0, pat)' "$3"
awk -v pat="$1" -v rep="$2" '{gsub(pat, rep)} 1' "$3"
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
edited Nov 13 '18 at 16:25
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
answered Nov 13 '18 at 14:50
Stephen KittStephen Kitt
166k24371451
166k24371451
Thanks. Yes my typo. (1) Could you explain the reason of doubling backslash? Is it correct that bash will remove the backslash in$1inawk -v pat="$1"? Does it work if gsub() sees the pattern argument without any backslash? Does gsub() not require its pattern argument to be enclosed in/.../? (2) Is it possible to write a bash script, so that I can just specify a regex as an argument to the script in the same way as I specify it to awk directly?
– Tim
Nov 13 '18 at 14:56
Sorry, ignore partially my (1). I was thinking of something else. I need to specify literal dot to awk, so I need literal backslash to escape it.
– Tim
Nov 13 '18 at 15:22
(1) See the link in my updated answer re the double backslashes. Bash doesn’t remove the backslash, but AWK needs two because it processes them twice.gsub()doesn’t need/.../. (2) You could double the backslashes before calling AWK...
– Stephen Kitt
Nov 13 '18 at 15:49
Also: gnu.org/software/gawk/manual/html_node/Gory-Details.html
– glenn jackman
Nov 13 '18 at 16:03
notice that the behavior wrt."."is different betweengawkandmawk(the default on debian); ex.echo foo.bar | awk '{gsub(".","X")}1'will print different things depending on whichawkwas used. The values of variables set on the command line are treated exactly as if they were enclosed in", and the standard leaves the behavior of awk wrt. non-standard escapes (other thann,t, etc) undefined.
– mosvy
Nov 13 '18 at 16:03
|
show 3 more comments
Thanks. Yes my typo. (1) Could you explain the reason of doubling backslash? Is it correct that bash will remove the backslash in$1inawk -v pat="$1"? Does it work if gsub() sees the pattern argument without any backslash? Does gsub() not require its pattern argument to be enclosed in/.../? (2) Is it possible to write a bash script, so that I can just specify a regex as an argument to the script in the same way as I specify it to awk directly?
– Tim
Nov 13 '18 at 14:56
Sorry, ignore partially my (1). I was thinking of something else. I need to specify literal dot to awk, so I need literal backslash to escape it.
– Tim
Nov 13 '18 at 15:22
(1) See the link in my updated answer re the double backslashes. Bash doesn’t remove the backslash, but AWK needs two because it processes them twice.gsub()doesn’t need/.../. (2) You could double the backslashes before calling AWK...
– Stephen Kitt
Nov 13 '18 at 15:49
Also: gnu.org/software/gawk/manual/html_node/Gory-Details.html
– glenn jackman
Nov 13 '18 at 16:03
notice that the behavior wrt."."is different betweengawkandmawk(the default on debian); ex.echo foo.bar | awk '{gsub(".","X")}1'will print different things depending on whichawkwas used. The values of variables set on the command line are treated exactly as if they were enclosed in", and the standard leaves the behavior of awk wrt. non-standard escapes (other thann,t, etc) undefined.
– mosvy
Nov 13 '18 at 16:03
Thanks. Yes my typo. (1) Could you explain the reason of doubling backslash? Is it correct that bash will remove the backslash in
$1 in awk -v pat="$1"? Does it work if gsub() sees the pattern argument without any backslash? Does gsub() not require its pattern argument to be enclosed in /.../? (2) Is it possible to write a bash script, so that I can just specify a regex as an argument to the script in the same way as I specify it to awk directly?– Tim
Nov 13 '18 at 14:56
Thanks. Yes my typo. (1) Could you explain the reason of doubling backslash? Is it correct that bash will remove the backslash in
$1 in awk -v pat="$1"? Does it work if gsub() sees the pattern argument without any backslash? Does gsub() not require its pattern argument to be enclosed in /.../? (2) Is it possible to write a bash script, so that I can just specify a regex as an argument to the script in the same way as I specify it to awk directly?– Tim
Nov 13 '18 at 14:56
Sorry, ignore partially my (1). I was thinking of something else. I need to specify literal dot to awk, so I need literal backslash to escape it.
– Tim
Nov 13 '18 at 15:22
Sorry, ignore partially my (1). I was thinking of something else. I need to specify literal dot to awk, so I need literal backslash to escape it.
– Tim
Nov 13 '18 at 15:22
(1) See the link in my updated answer re the double backslashes. Bash doesn’t remove the backslash, but AWK needs two because it processes them twice.
gsub() doesn’t need /.../. (2) You could double the backslashes before calling AWK...– Stephen Kitt
Nov 13 '18 at 15:49
(1) See the link in my updated answer re the double backslashes. Bash doesn’t remove the backslash, but AWK needs two because it processes them twice.
gsub() doesn’t need /.../. (2) You could double the backslashes before calling AWK...– Stephen Kitt
Nov 13 '18 at 15:49
Also: gnu.org/software/gawk/manual/html_node/Gory-Details.html
– glenn jackman
Nov 13 '18 at 16:03
Also: gnu.org/software/gawk/manual/html_node/Gory-Details.html
– glenn jackman
Nov 13 '18 at 16:03
notice that the behavior wrt.
"." is different between gawk and mawk (the default on debian); ex. echo foo.bar | awk '{gsub(".","X")}1' will print different things depending on which awk was used. The values of variables set on the command line are treated exactly as if they were enclosed in ", and the standard leaves the behavior of awk wrt. non-standard escapes (other than n,t, etc) undefined.– mosvy
Nov 13 '18 at 16:03
notice that the behavior wrt.
"." is different between gawk and mawk (the default on debian); ex. echo foo.bar | awk '{gsub(".","X")}1' will print different things depending on which awk was used. The values of variables set on the command line are treated exactly as if they were enclosed in ", and the standard leaves the behavior of awk wrt. non-standard escapes (other than n,t, etc) undefined.– mosvy
Nov 13 '18 at 16:03
|
show 3 more comments
2
Just a quick answer with a different approach : script written in Awk:
#!/usr/bin/gawk -f
BEGIN { find=ARGV[1]; repl=ARGV[2]; delete ARGV[1]; delete ARGV[2]}
{ print gensub(find,repl,"g",$0) }
and then the usual Unix filter behavior:
$ chmod 755 myawkscript
$ ls |myawkscript 'w(.)' '{1}'
{y}{w}{s}{r}{p}t
$ myawkscript '(w+)' '{1}' myawkscript
#!/{usr}/{bin}/{gawk} -{f}
{BEGIN} { {find}={ARGV}[{1}]; {repl}={ARGV}[{2}]; .....
{ {print} {gensub}({find},{repl},"{g}",${0}) }
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
add a comment |
2
Just a quick answer with a different approach : script written in Awk:
#!/usr/bin/gawk -f
BEGIN { find=ARGV[1]; repl=ARGV[2]; delete ARGV[1]; delete ARGV[2]}
{ print gensub(find,repl,"g",$0) }
and then the usual Unix filter behavior:
$ chmod 755 myawkscript
$ ls |myawkscript 'w(.)' '{1}'
{y}{w}{s}{r}{p}t
$ myawkscript '(w+)' '{1}' myawkscript
#!/{usr}/{bin}/{gawk} -{f}
{BEGIN} { {find}={ARGV}[{1}]; {repl}={ARGV}[{2}]; .....
{ {print} {gensub}({find},{repl},"{g}",${0}) }
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
add a comment |
2
2
2
Just a quick answer with a different approach : script written in Awk:
#!/usr/bin/gawk -f
BEGIN { find=ARGV[1]; repl=ARGV[2]; delete ARGV[1]; delete ARGV[2]}
{ print gensub(find,repl,"g",$0) }
and then the usual Unix filter behavior:
$ chmod 755 myawkscript
$ ls |myawkscript 'w(.)' '{1}'
{y}{w}{s}{r}{p}t
$ myawkscript '(w+)' '{1}' myawkscript
#!/{usr}/{bin}/{gawk} -{f}
{BEGIN} { {find}={ARGV}[{1}]; {repl}={ARGV}[{2}]; .....
{ {print} {gensub}({find},{repl},"{g}",${0}) }
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
Just a quick answer with a different approach : script written in Awk:
#!/usr/bin/gawk -f
BEGIN { find=ARGV[1]; repl=ARGV[2]; delete ARGV[1]; delete ARGV[2]}
{ print gensub(find,repl,"g",$0) }
and then the usual Unix filter behavior:
$ chmod 755 myawkscript
$ ls |myawkscript 'w(.)' '{1}'
{y}{w}{s}{r}{p}t
$ myawkscript '(w+)' '{1}' myawkscript
#!/{usr}/{bin}/{gawk} -{f}
{BEGIN} { {find}={ARGV}[{1}]; {repl}={ARGV}[{2}]; .....
{ {print} {gensub}({find},{repl},"{g}",${0}) }
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
answered Nov 13 '18 at 16:58
JJoaoJJoao
7,1691928
7,1691928
add a comment |
add a comment |
draft saved
draft discarded
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f481470%2fhow-can-i-use-a-bash-variable-as-a-regex-argument-to-gsub-and-match%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
