Problems of saving commands in variables ---- rethinking special characters inside a variable in bash

 1 example

$ ls

'my dir'

$ ls 'my dir'

test1  test2  test3

$ cmd="ls 'my dir'"

$ echo $cmd

ls 'my dir'

$ $cmd

ls: cannot access "'my": No such file or directory

ls: cannot access "dir'": No such file or directory

$ "$cmd"

-bash: ls 'my dir': command not found

2 Bash processes quotation marks only once

Special characters precedences:

1. Backslash 
2. Outmost quotation mark pair
3. Star (*)
4. space

In the above example.

2.1 Analysis 1

$ $cmd

ls: cannot access "'my": No such file or directory

ls: cannot access "dir'": No such file or directory

As there are no quotation marks, bash puts the content of $cmd (ls 'my dir')into the next step where SPACES are taken as the delimiter between command and arguments. 

Bash doesn't process quotes inside a variable!

ls 'my dir'

command: ls

arg1: 'my

arg2: dir'

As a result, there are two arguments for command ls. 

2.2 Analysis 2

$ "$cmd"

-bash: ls 'my dir': command not found

The outmost quotes pair makes $cmd as the only one part for this line. So bash takes the whole content of  ls 'my dir'  as  a command name. 

3 Problems of "quotes inside a variable are not eaten by bash"

A command's args need to be quoted if they have spaces or other special characters like &, while bash does NOT process these quotes if the args are saved in variables.

e.g.

$ curl --data 'name=john&password=1234' http://some.site.com

If  we save the command line as a variable:

$ cmd="curl --data 'name=john&password=1234' http://some.site.com"

$ $cmd

This would NOT work as expected because the data arg becomes 

'name=john&password=1234' 

, with two extra quotation marks that we should have been eaten by bash.

4 Solutions

4.1 Use backslash instead of quotes

cmd="curl --data name=john\&password=1234 http://some.site.com"

As we avoid quotes inside the variable.

Update: this would not work at all, as bash doesn't process \ either inside a variable.

Any character, e.g. quotation mark, backslash, &, .... in a variable is literal, with no special meaning for bash anymore.

4.2 Use a function instead of a variable to wrap command

function cmd()

{

    curl --data 'name=john&password=1234' http://some.site.com

}

$ cmd

4.3 Use eval to force bash to eat the quotes inside a variable

$ cmd="curl --data 'name=john&password=1234' http://some.site.com"

$ eval $cmd

This lets bash to take $cmd as a literal command line. 

Using eval is not recommended generally. So we would not use it here.

4.4 A real-word example

We have some curl command lines which share a common args, so we want to save the common part and reuse it everywhere.

Option 1:

CURL="curl -k --noproxy '*' -H 'Ceontent-Type: application/json'"

$CURL --data 'json data including spaces' http://site.com

This would not work!

Option 2:

CURL="curl -k --noproxy \* -H Ceontent-Type:\ application/json"

$CURL --data 'json data including spaces' http://site.com

This works but looks ugly.

Update: This would not work at all. As bash doesn't process \ inside a variable either.

Option 3:

function CURL()

{

    curl -k --noproxy '*' -H 'Ceontent-Type: application/json' "$@"

}

CURL --data 'json data including spaces' http://site.com

For more info about $@, "$@", $*, "$*", please refer to here. Simply put, only "$@" respects the original positional args.