expansions and substitutions

substitutions"/>

expansions and substitutions

为什么80%的码农都做不了架构师？>>>   

How bash does expansions and substitutions?

    Before executing your commands,bash checks whether there are any syntax elements in the command line that should be interpreted rather than taken literally.After splitting the command line into tokens(words),bash scans for these special elements and interprets them,resulting in a changed command line:the elements are said to be expanded or subistituted to new text and maybe new tokens(words).

    Bash performs expansions and substitutions in a defined order.This explains why globbing(pathname expansion),for example,is safe to use on filenames with spaces(becasue it happens after the final word splitting).

    The order is (from first to last):

1. brace expansion
2. tlide expansion
3. parameter expansion    arithemetic expansion    command expansion
4. word spliting
5. pathname expansion

    Process substitution is performed simultaneously with parameter expansion,command substitution and arithmetic expansion.It is only performed when the underlying operating system supports it.

    The 3 steps parameter expansion, arimethic expansion and command expansion happen at the same time in a left-to-right fashion on the commandline.This means:

i=1 

echo $i $((i++)) $i

will output 1 1 2 not 1 1 1

expansions and substitutions in details

What is brace expansion?

{string1,string2,......,stringN}

{<START>..<END>}

{<START>..<END>..<INCR>}(bash 4)

<PREAMBLE>{........}

{........}<POSTSCRIPT>

<PREAMBLE>{........}<POSTSCRIPT>

    Brace expansion is used to generate arbitrary strings.The specified strings are used to generate all possible combinations with the optional surrounding preambles and postscripts.

    Usually it's used to generate mass-arguments for a command,that follow a specific naming-scheme.

    It is the very first step in expansion-handling,it's important to understand that.When you use

echo {a,b}$PATH

    then the brace expansion does not expand the variable -- this is in a later step.Brace expansion just makes it being:

echo a$PATH b$PATH

    Another common pitfall is to assume that a range like {1..200} can be expressed with variables using a=1;b=2;{$a..$b}.Due to what I described above,it simply is not possible,because it's the very first step in doing expansions.A possible way to achieve this,if you really can't handle this in another way,is using the eval command,which basically evaluates a commandline twice:

what is eval?

eval [arg ...]
              The  args  are read and concatenated together into a single com‐
              mand.  This command is then read and executed by the shell,  and
              its  exit status is returned as the value of eval.  If there are
              no args, or only null arguments, eval returns 0.

    the eval method requires that the entire command be properly escaped to avoid unexpected expansions.If the sequence expansion is to be assigned to an array,another method is using declaration commands:

declare -a 'pics=  (img{'"$a..$b"'}.png)'; mv "${pics[@]}" ../imgs

What is declare?

declare: declare [-aAfFgilrtux] [-p] [name[=value] ...]
    Set variable values and attributes.
    
    Declare variables and give them attributes.  If no NAMEs are given,
    display the attributes and values of all variables.
    
    Options:
      -f    restrict action or display to function names and definitions
      -F    restrict display to function names only (plus line number and
        source file when debugging)
      -g    create global variables when used in a shell function; otherwise
        ignored
      -p    display the attributes and value of each NAME
    
    Options which set attributes:
      -a    to make NAMEs indexed arrays (if supported)
      -A    to make NAMEs associative arrays (if supported)
      -i    to make NAMEs have the `integer' attribute
      -l    to convert NAMEs to lower case on assignment
      -r    to make NAMEs readonly
      -t    to make NAMEs have the `trace' attribute
      -u    to convert NAMEs to upper case on assignment
      -x    to make NAMEs export
    
    Using `+' instead of `-' turns off the given attribute.
    
    Variables with the integer attribute have arithmetic evaluation (see
    the `let' command) performed when the variable is assigned a value.
    
    When used in a function, `declare' makes NAMEs local, as with the `local'
    command.  The `-g' option suppresses this behavior.
    
    Exit Status:
    Returns success unless an invalid option is supplied or an error occurs.

This method is significantly safer,but one must still be careful to control the values of $a and $b.

Both the exact quoting,and explicitly including "-a" are important.

The brace expansion is present in two basic forms,strings list and ranges

Strings list

{string1,string2,...,stringN}

Without the optional preamble and postscript strings,the result is just a space-separated list of the given strings:

$ echo {I,want,my,money,back}
I want my money back

With preamble or postscript strings, the result is a space-separated list of all possible combinations of preamble, specified strings and postscript:

$ echo _{I,want,my,money,back}
_I _want _my _money _back$ echo {I,want,my,money,back}_
I_ want_ my_ money_ back_$ echo _{I,want,my,money,back}-
_I- _want- _my- _money- _back-

The brace expansion is only performed,if the given string list is really a list of strings.i.e.if there's minimum one "," (comma)! Something like {money} doesn't expand to something special,it's really only the text "{money}".

Ranges

{<START>..<END>}

Brace expansion using ranges is written giving the startpoint and the endpoint of the range.This is a "sequence expression".The sequence can be of two types:

• integers (optionally zero padded, optionally with a given increment)
• characters

$ echo {5..12}
5 6 7 8 9 10 11 12$ echo {c..k}
c d e f g h i j k

When you mix these both types,brace expansion is not performed.

$ echo {5..k}
{5..k}

When you zeropad one of the numbers(or both) in a range,then the generated range is zeropadded,too:

$ echo {01..10}
01 02 03 04 05 06 07 08 09 10

Similar to the expansion using stringlists,you can add preamble and postscript strings:

$ echo 1.{0..9}
1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9$ echo ---{A..E}---
---A--- ---B--- ---C--- ---D--- ---E---

Combining and nesting

When you combine more brace expansions,you effectively use a brace expansion as preamble or postscript for another one.Let's generate all possible combinations of uppercase letters and digits:

$ echo {A..Z}{0..9}
A0 A1 A2 A3 A4 A5 A6 A7 A8 A9 B0 B1 B2 B3 B4 B5 B6 B7 B8 B9 C0 C1 C2 C3 C4 C5 C6
C7 C8 C9 D0 D1 D2 D3 D4 D5 D6 D7 D8 D9 E0 E1 E2 E3 E4 E5 E6 E7 E8 E9 F0 F1 F2 F3
F4 F5 F6 F7 F8 F9 G0 G1 G2 G3 G4 G5 G6 G7 G8 G9 H0 H1 H2 H3 H4 H5 H6 H7 H8 H9 I0
I1 I2 I3 I4 I5 I6 I7 I8 I9 J0 J1 J2 J3 J4 J5 J6 J7 J8 J9 K0 K1 K2 K3 K4 K5 K6 K7
K8 K9 L0 L1 L2 L3 L4 L5 L6 L7 L8 L9 M0 M1 M2 M3 M4 M5 M6 M7 M8 M9 N0 N1 N2 N3 N4
N5 N6 N7 N8 N9 O0 O1 O2 O3 O4 O5 O6 O7 O8 O9 P0 P1 P2 P3 P4 P5 P6 P7 P8 P9 Q0 Q1
Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 R0 R1 R2 R3 R4 R5 R6 R7 R8 R9 S0 S1 S2 S3 S4 S5 S6 S7 S8
S9 T0 T1 T2 T3 T4 T5 T6 T7 T8 T9 U0 U1 U2 U3 U4 U5 U6 U7 U8 U9 V0 V1 V2 V3 V4 V5
V6 V7 V8 V9 W0 W1 W2 W3 W4 W5 W6 W7 W8 W9 X0 X1 X2 X3 X4 X5 X6 X7 X8 X9 Y0 Y1 Y2
Y3 Y4 Y5 Y6 Y7 Y8 Y9 Z0 Z1 Z2 Z3 Z4 Z5 Z6 Z7 Z8 Z9

Brace expansions can be nested,but too much of it usually makes you losing overview a bit.

Here's a sample to generate the alphabet,first the uppercase letters,then the lowercase ones:

$ echo {{A..Z},{a..z}}
A B C D E F G H I J K L M N O P Q R S T U V W X Y Z a b c d e f g h i j k l m n o p q r s t u v w x y z

Common use and examples

wget {1,2,3,4,5,6}.htmlwget {1..6}.htmlmkdir /home/bash/test/{foo,bar,baz,cat,dog}/home/bash/test/{foo,bar,baz,cat,dog}somecommand -v{,,,,}

/home/bash/test/{foo,bar,baz,cat,dog} generates /home/bash/test/foo /home/bash/test/bar /home/bash/test/baz /home/bash/test/cat /home/bash/test/dog

somecommand -v{,,,,} generates somecommand -v -v -v -v -v.

New in Bash 4.0

Zero padded number expansion

Prefix either of the numbers in a numeric range with 0 to pad the expanded numbers with the correct amount of zeros:

$ echo {0001..5}
0001 0002 0003 0004 0005

Increment

It is now possible to specify an increment using ranges:

{<START>..<END>..<INCR>}

<INCR> is numeric,you can use a negative integer but the correct sign is deduced from the order of <START> and <END> anyways.

$ echo {1..10..2}
1 3 5 7 9
$ echo {10..1..2}
10 8 6 4 2

Interesting feature:The increment specification also works for letter-ranges:

$ echo {a..z..3}
a d g j m p s v y

What is Tilde expansion?

~
~/...~NAME
~NAME/...~+
~+/...~-
~-/...

The Tilde expansion is used to expand to several specific pathnames:

• home directories
• current working directory
• previous working directory

Tilde expansion is only performed,when the tlide-construct is at the beginning of a word,or a separte word.

If there's nothing to expand,i.e. in case of a wrong username or any other error condition,the tlide construct is not replaced,it says what it is.

Tlide expansion is also performed everytime a variable is assigned:

• after the first =:TARGET=~moonman/share
• after every :(colon) in the assigned value:TARGET=file:~moonman/share

This way you can correctly use the the tlide expansino in your PATH:

PATH=~/mybins:~peter/mybins:$PATH

Spaces in the referenced pathes?A construct like...

~/"my directory"

…is perfectly valid and works!

Home directory

~
~<NAME>

This form expands to the home-directory of the current user(~) or the home directory of the given user(<NAME>).

If the given user doesn't exist(or if his home directory isn't determinable,for some reason),it doesn't expand to something else,it stays what it is.The requested home directory is found by asking the operating system for the associated home for <NAME>.

To find the home directory of the current user(~),Bash has a precedence:

• expand to the value of HOME if it's defined
• expand to the home directory of the user executing the shell(operating system)

That means,the variable HOME can override the "real" home directory,at least regarding tilde expansion.

Current working directory

~+

This expands to the value of the PWD variable,which holds the correct working directory.

echo "CWD is $PWD"

is equivalent to (note is must be a separate word!)

echo "CWD is" ~+

Previous working directory

~-

This expands to the value of the OLDPWD variable,which holds the previous working directory(the one before the last cd).if OLDPWD is unset(never changed the directory ),it is not expanded.

$ pwd
/home/bash
$ cd /etc
$ echo ~-
/home/bash

What is parameter expansion?

one core functionality of Bash is to manage parameters.A parameter is an entity that stores values and is referenced  by a name,a number or a special symbol.

• Parameters referenced by a name are called variables.(this also applies to arrays)
• Parameters referenced by a number are called positional parameters and reflect the arguments given to a shell.
• Parameters referenced by a special symbol are auto-set parameters that have different special meanings and uses.

Parameter expansion is the procedure to get the value from the referenced entity,like expanding a variable to print its value.On expansion time you can do very nasty things with the parameter or its value.These things are described here.

If you saw some parameter expansion syntax somewhere,and you need to check what it can be,try the overview section below!

Arrays can be special cases for parameter expansion,every applicable description mentions arrays below.

Simple usage

• $PARAMETER
• ${PARAMETER}

The easiest form is to just use a parameter's name within braces.This is identical to using $Foo like you see it everywhere,but has the advantage that it can be immediately followed by characters that would be interpreted as part of the parameter name otherwise.Compare these two expressions,where we want to print a word with a trailing 's'.

echo "The plural of $WORD is most likely $WORDs"
echo "The plural of $WORD is most likely ${WORD}s"

Why does the first one fail?It prints nothing,because a parameter(variable) named "WORDs" is undefined and thus printed as ""(nothing).Without using braces for parameter expansion,Bash will interpret the sequence of all valid characters from the introducing "$" up to the last valid character as name of the parameter.When using braces you just force bash to only interpret the name inside your braces.

Also,please remember,that parameter names are (like nearly everything in UNIX@) case sentitive!

The second form with the curly braces is also needed to access positional parameters(arguments to script) beyond $9:

echo "Argument  1 is: $1"
echo "Argument 10 is: ${10}"

Arrays:

For arrays you always need the braces.The arrays are expanded by individual indexes or mass arguments.An individual index behaves like a normal parameter.

Purpose

An array is a parameter that holds mappings from keys to values.Arrays are used to store a collection of parameters into a parameter.Arrays(in any programming language) are a useful and common composite data structure,and one of the most important scripting features in Bash and other shells.

Here is an abstract representation of an array named NAMES.The indexes go from 0 to 3.

NAMES0: Peter1: Anna2: Greg3: Jan

Instead of using 4 separate variables,multiple related variables are grouped grouped together into elements of the array,accessible by their key.If you want the second name,ask for index 1 of the array NAMES.

转载于: