ÿþ<html> <head> <title>Albert van der Sel: Unix shell scripting</title> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"> </head> <body bgcolor="#FFFFFF" link="blue" alink="blue" vlink="blue"> <h1>A very simple note on Unix/Linux Shell scripting.</h1> <B>Version</B> : 1.1 - Ready<br> <B>Date</B> : 20/01/2013<br> <B>Level</B> : Basic<br> <B>By</B> : Albert van der Sel<br> <hr/> <br> This note is a quick intro in Unix/Linux <B>shell scripting</B>. Because it's fast, it also inevitably means that it is short,<br> and incomplete.<br> <br> It's not for experienced SysAdmins.<br> No..., but it might be useful for someone who wants to get into basic shell scripting.., <I>fast.</I><br> <br> You do do not need to know much beforehand. I <I>only</I> want you to understand what a terminal session is, and how<br> to get to a "prompt" (instead of only working with GUI's). It would also help if you know a bit about unix/linux<br> file-permissions (the "-rwxrwxrwx" stuff), and a little bit of "vi", the "editor" to create textfiles.<br> But, on those subjects, a microscopic intro is included in chapter 1 too.<br> <br> Scripting is the subject of chapter 2. So what's in chapter 1? I think it is a sort of "foundation",<br> since for shell scripting, knowledge of tools like "sed", "awk", "tr", "grep", and a few others, is really needed.<br> Furthermore, some knowledge about filemodes (security), and how to use the "chmod" command to make a file<br> executable (for example), is pretty much mandatory too.<br> <br> <br> <font face="arial" size=2 color="blue"> <B>Main Contents:<br> <br> Chapter 1. A "foundation" first.<br> <br> &nbsp 1.1 A few general things to know first.<br> &nbsp 1.2 Filemodes (or "permissions").<br> &nbsp 1.3 The "chmod" command to change access rights (the "flags").<br> &nbsp 1.4 A few words on viewing and creating/modifying files.<br> &nbsp 1.5 Wildcards.<br> &nbsp 1.6 The "find" and "grep" commands.<br> &nbsp 1.7 Pipeline and Redirecting.<br> &nbsp 1.8 The "sed" and "tr" commands.<br> &nbsp 1.9 The "awk" command.<br> <br> Chapter 2. A few basic notes on Shell scripting.<br> <br> &nbsp2.1 Some "housekeeping".<br> &nbsp2.2 Variables.<br> &nbsp2.3 Conditionals: The "if" and "case" statements.<br> &nbsp2.4 Several types of "Loops."<br> &nbsp2.5 Parameters.<br> <br> </B> <br> <br> <br> <font face="arial" size=2 color="blue"> <h2>Chapter 1. A "foundation" first.</h2> <font face="arial" size=2 color="black"> <font face="arial" size=2 color="red"> <h3>1.1 A few general things to know first.</h3> <font face="arial" size=2 color="black"> If you started a "ssh" terminal to a remote Unix or Linux host, then after logging on, usually<br> a simple "$" prompt appears if you are an "ordinary" user of the system.<br> Ofcourse, you might also have a unix, or linux system, installed on a (local) personal system.<br> <br> By convention, if you are "root" (or the superuser, or the big boss), the prompt often is a "#" symbol.<br> <br> You are probabably interested in "where you are", or "your path", in the filesystem, so you immediately enter the "pwd"<br> command to find out your location. Big chance, you are in the "/home/yourusername" directory. However, if you are somewhere else,<br> there is no problem at all. If not in the home directory, you can always try the "cd ~" command, which usually gets<br> you to your homedir (if you have one).<br> <br> If you indeed have a simple "$" prompt, you might try this command: PS1='$PWD>' which should change the prompt, and show you<br> the "path" inside the prompt,like <B>"/home/mary>"</B>, which might be a bit more comfortable.<br> But if it didn't work, then don't worry about it.<br> <br> But many Linux distro's already per default shows a friendly prompt like <B>"[albert@STARBOSS /home/albert $]"</B>.<br> which shows your account, the machine name, and location in the filesystem.<br> <br> This "PS1" thing, is the name of one of the "environment variables" which are defined for your session.<br> To see all those variables, try the "env" command. Or to see just one, like the "path" variable, try "echo $PATH"<br> <br> <font face="courier" size=2 color="blue"> $ <B>pwd</B><br> <br> /home/mary<br> <br> $ <B>PS1='$PWD>'</B><br> <br> /home/mary> <B>env</B><br> ..<br> shows all the environment variables you have now..<br> <br> $ <B>echo $PATH</B><br> <br> Shows the directories (the paths) which your shell will search if you enter any command..<br> <font face="arial" size=2 color="black"> <br> When you entered the commands as shown above, you are having a <B>interactive session</B> with your "shell", which is an interpreter for your commands.<br> It responds to the commands you key in, but it <B>also starts and executes</B> your shell scripts.<br> <br> It depends a bit on what Operating System you work, but often the socalled "bash" shell is pretty default on Linux, while<br> on many Unix systems, "sh", "korn shell" or "bash" are often pretty default too. Most shells do not differ too much, at least<br> not for the scope of this note. Some folks say that some shells are a (tiny) bit better equiped for certain tasks,<br> but when it comes to "just" entering interactive commands, or creating shell scripts, the differences are not dramatic.<br> <br> Any Unix or Linux Operating system, has hundreds (or even thousends) of commands. There are real System Administration commands<br> for example, to set kernel parameters, to tune memory, or to add disks etc.. etc..<br> They can be used in shell scripts too, as many SysAdmins do. However, the "typical" general purpose scripts are often concerned around<br> processing of files in the most general sense. For example, you as a scripter, must process files so that their format get changed,<br> or add/remove parts from the files, or send them to another machine based on some criteria, or do something based on the last access date/time etc.. etc..<br> <br> <font face="arial" size=2 color="red"> <h3>1.2 Filemodes (or "permissions").</h3> <font face="arial" size=2 color="black"> This is an important subject. In any Operating System, files and directories are protected by access rights.<br> In Unix/Linux, we have "filemodes" and "acl's", but the filemodes are most often associated with access rights.<br> <br> In any directory, try the "ls -al" command, in order to get an extended file listing (with all attributes) in that directory.<br> Now, you might see stuff like this:<br> <br> <font face="courier" size=2 color="blue"> $ ls -al<br> <br> -rw-r----- mary dba 18 jun 1231 readme.txt<br> -rwxrwxrwx mary dba 21 apr 2211 foreverybody.txt<br> -rwx------ mary dba 28 feb 8744 testscript1.ksh<br> -rwxr-x--- mary dba 19 feb 7251 testscript2.ksh<br> <br> <font face="arial" size=2 color="black"> I want you to focus on the "-rwx rwx rwx" stuff here. In Unix/Linux, we have the following file/directory permissions (or also called "filemodes"):<br> <br> <font face="courier" size=2 color="blue"> r = read.....(like being able to "read" or "open" a file, but not being able to modify it)<br> w = write....(being able to "write" or "change" a file)<br> x = execute..(being able to "execute" a file if it's indeed executable)<br> - = nothing granted<br> <font face="arial" size=2 color="black"> <br> Now, you will notice three "blocks" of three "flags" each, say for example "rwx rwx rwx" or for example "rw- rw- r--".<br> If a flag is a "-" somewhere in such a string, it means that a permission is not granted.<br> <br> - The first three flags, apply to the "user" rights (which is, in the above listing, "mary).<br> - The second three flags, apply to the "group" (where the user is member of, like "dba" in the above listing).<br> - The third three flags, apply to "Everyone" (or sometimes also called "the world")<br> <br> <font face="courier" size=2 color="brown"> -------------------<br> |user |group|world|<br> -------------------<br> |r w x|r w x|r w x|<br> -------------------<br> <font face="arial" size=2 color="black"> <br> So, as a few examples:<br> <br> A file like "readme.txt" has rw (read and write) for mary, and r (read) for the dba group (where mary is member of). Everybody else has no access.<br> A file like "testscript2.ksh", has rwx (read and write and execute) for mary, and r (read) and x (execute) for the dba group.<br> <br> So, as another example, if you see some file (in some directory) listed like so:<br> <br> <font face="courier" size=2 color="blue"> -rw-rw---- mary hr 19 dec 44788 personell.doc<br> <font face="arial" size=2 color="black"> <br> Then it means that the user mary has rw permissions to the file "personell.doc", and so has the group "hr". But nobody else can access that file.<br> <br> <br> <font face="arial" size=2 color="red"> <h3>1.3 The "chmod" command to change access rights (the "flags").</h3> <font face="arial" size=2 color="black"> If you are the owner of a file, you must be able to change the access rights to that file. Maybe you want the group<br> where you are member of, to be able to read the file too. Or, maybe you have created a script, but it's not executable yet,<br> meaning that the "x" flag is still missing. For this, you can use the "chmod" command.<br> It's real easy. Just add (+) or remove (-) the "flags" as you want them to be.<br> <br> Examples:<br> <br> Suppose a file has the following flags, but you want the group to have "rw" as well:<br> <font face="courier" size=2 color="blue"> <br> $ <B>ls -al</B> <br> <br> -rw------- mary dba 22 dec 2231 readme.txt<br> <br> $ <B>chmod g+rw</B> readme.txt<br> <br> $ <B>ls -al</B> <br> <br> -rw-rw---- mary dba 22 dec 2231 readme.txt<br> <br> <font face="arial" size=2 color="black"> Suppose a file has the following flags, but you want it to be executable for you as well:<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B> <br> <br> -rw------- mary dba 22 dec 2231 myscript.ksh<br> <br> $ <B>chmod u+x</B> myscript.ksh<br> <br> $ <B>ls -al</B> <br> <br> -rwx------ mary dba 22 dec 2231 myscript.ksh<br> <br> <font face="arial" size=2 color="black"> Suppose a file has the following flags, but you think that the group has too many rights. So, you want the "w" and "x" be<br> revoked for the group:<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B> <br> <br> -rwxrwx--- mary dba 22 dec 2231 myscript.ksh<br> <br> $ <B>chmod g-wx</B> myscript.ksh<br> <br> $ <B>ls -al</B> <br> <br> -rwxr----- mary dba 22 dec 427 myscript.ksh<br> <font face="arial" size=2 color="black"> <br> So, "chmod" can be used with the <B>"+"</B> to add permissions, and with a <B>"-"</B> to revoke permissions.<br> In general use "chmod" like so:<br> <br> <font face="courier" size=2 color="brown"> chmod [u][g][a] + or - [r][w][x] filename (where u=user; g=group; a=world or everyone)<br> <font face="arial" size=2 color="black"> <br> <br> There is another way (or quicker way) to get the same results. Instead of explicitly using u/g/a and r/w/x in the command,<br> you can simply use "numbers" to denote the total of flags. Here is how it goes. First, watch this internal unix definition:<br> <br> <font face="courier" size=2 color="blue"> 1 = execute<br> 2 = write<br> 4 = read <br> <br> <font face="arial" size=2 color="black"> Execute and read adds up to: 1+4=5.<br> Read and write are: 4+2=6.<br> Read, write and exec adds up to: 4+2+1=7<br> <br> So, if you would do "chmod 640 filename" it means 6 (rw) for the user, 4 (r) for the group, 0 (nothing) for everyone.<br> Here are a few examples:<br> <br> Suppose you see the following flags on a file, namely "-rwxrwxrwx" which is too much. You want it to be "-rwxr-x---"<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B> <br> <br> -rwxrwxrwx mary dba 22 dec 427 myscript.ksh<br> <br> $ <B>chmod 750</B> myscript.ksh<br> <br> $ <B>ls -al</B> <br> <br> -rwxr-x--- mary dba 22 dec 427 myscript.ksh<br> <br> <font face="arial" size=2 color="black"> So, the "7" for the user is "rwx" which is 4+2+1. The "5" for the group is "r-x" which is 4+1=5.<br> You see? Setting flags this way is pretty quick and efficient.<br> <br> Suppose you see the following flags on a file, namely "-rw-------". You want it to be "-rw-r-----"<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B> <br> <br> -rw------- mary dba 22 dec 1527 myfile.doc<br> <br> $ <B>chmod 640</B> myfile.doc<br> <br> $ <B>ls -al</B> <br> <br> -rw-r----- mary dba 22 dec 1527 myfile.doc<br> <br> <font face="arial" size=2 color="black"> Although you can "easily" calculate it yourself, here are a few "popular" filemodes:<br> <br> <font face="courier" size=2 color="blue"> rwxrwxrwx=777<br> rwxrwxr-x=775<br> rw-rw-rw-=666<br> rw-r--r--=644<br> rw-r-----=640<br> r-xr-xr-x=555<br> rwx------=700<br> rw-------=600<br> r-x------=500<br> <font face="arial" size=2 color="brown"> <br> <B>Security note:</B> <br> Now, this is just a humble note for learning shell scripting. So, I think you are probably working on a private system.<br> On a private system, you can do want you like ofcourse.<br> <br> However, in general, always the <B>"principle of least priviledge"</B> should be followed.<br> This means: grant no more permissions on objects than is absolutely neccessary.<br> So for example, for "everyone" (or the "world") having access to files should not be done, or be highly exceptional.<br> <br> <br> <font face="arial" size=2 color="red"> <h3>1.4 A few words on viewing and creating/modifying files.</h3> <font face="arial" size=2 color="black"> Shell scripts contain "plain text". In other words, they are just "ascii" files.<br> If we want to create a new script, or if we want to modify an existing script, we need an "editor".<br> <br> One of the oldest and well-know editors, is "vi" (sometimes it's named "vim" on Linux systems).<br> So, we need a bit of knowledge of vi.<br> <br> However, sometimes you just want to <B>quickly browse through</B> an ascii file. For this, we do not neccessarily need an editor.<br> The "cat" and "more" commands, enables us to view the contents of any ascii file (like a script).<br> <br> But there exists not only ascii files on the filesystem. Also, many binary files live there too.<br> So, if it's not obvious from the name of the file, what type of file it is, we need a way to check the <I>filetype</I>.<br> <br> <B><U>Determining the type of file:</U></B><br> <br> For this we can use the "file" command. It cleary shows you what the filetype is, like ASCII or binary.<br> For example:<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B><br> <br> -r-x------ root bin 17 jan... 886 myscript.sh<br> -r-x------ root bin 23 feb..23674 gatekeeper<br> <br> $ <B>file</B> myscript.sh<br> <br> ASCII text<br> <br> $ <B>file</B> gatekeeper<br> <br> gatekeeper: ELF 32-bit LSB executable<br> <br> <font face="arial" size=2 color="black"> Not all systems display exactly the same output, but it should be clear from the output whether or not we are dealing<br> with a text file or not. In the example above, the file "myscript.sh" is a text file which we can safely browse using "cat" or "more".<br> <br> <B><U>using the "cat" and "more" commands:</U></B><br> <br> These two commands are great for viewing the content of flat files, like shell scripts.<br> They are very easy to use.<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B><br> <br> -r-x------ mary dba 17 jan... 886 myscript.sh<br> -r-x------ mary dba 23 feb..34518 presentation.ppt<br> <br> $ <B>cat</B> myscript.sh<br> <br> <font face="arial" size=2 color="black"> The full content is displayed. However, if the file is large, maybe it does not fit on your screen, so maybe you only see<br> the last "screen" of that file.<br> Redirecting the "ouput" to the "more" command, shows you "one screen at the time" everytime you press Enter.<br> <br> <font face="courier" size=2 color="blue"> $ <B>cat</B> myscript.sh <B>| more</B><br> <br> <font face="arial" size=2 color="black"> Here we used the "|" pipeline symbol, to <B>connect</B> two programs: "cat" and "more".<br> Usually, "cat" just simply dumps the content of a file to your screen. But by using <B> a "pipe"</B>, "cat" now knows that<br> it must send the data to the "more" command, which shows the content "one screen at the time".<br> <br> Using the "more" command by itself, is often even better.<br> It simply dumps the content of a flat file to your screen, one page at the time.<br> <br> <font face="courier" size=2 color="blue"> $ <B>more</B> myscript.sh<br> <br> <font face="arial" size=2 color="black"> If you want to end the output before it's done, simply press "Ctrl-C".<br> <br> <B><U>A few words on the "vi" editor:</U></B><br> <br> "Vi" is not the easiest editor around. Everybody starting with vi, will make many mistakes in the beginning.<br> That's really "normal".<br> Here are a few notes, that gets you started. But, folks have created tutorials which are at least 30 pages or so.<br> <br> Using a normal account (not "root"), you probably have a homedir like "/home/mary" or so.<br> But maybe it's a good idea to create your first files in "/tmp", which is a sort of temporary workspace for everybody.<br> Ofcourse, "/tmp" is <B>not the place for "permanent storage"</B>. Don't put anything in there of value.<br> Later on, after the first "trials" with vi, store your files at a better place, like your homedir.<br> <br> First, take notice of this. Vi uses two main "modi":<br> <br> <B>1. Command mode: If you press "Esc"</B><br> <ul> <li> Then you can enter <B>":"</B> and <B>"q!"</B> (from quit !). You will see those symbols at the left-bottom of the screen.<br> This tells vi to exit without saving anything. This is handy if you made a "mess" of the text.</li> <li>Or, if happy with the typed text, you can enter ":" and a "wq!" (write quit !) and vi exits with saving the file.</li> </ul> <B>2. Insert mode: If you press the "i" or "a" or "Ins".</B><br> <br> In Insert mode, you can enter text. If ready, you can press Esc to go to Command mode to quit, or quit and save the file.<br> <br> You can "toggle" between Insert and Command mode. Just press "esc" or "i" when you want.<br> <br> Important: once you have a few lines of text, press Esc, and then you can "freely" use the cursorkeys to navigate through<br> the text, and place the cursor where you want it to be.<br> <br> <B>Fig 1. An example using "vi" (or "vim").</B><br> <br> <img src="unixshell1.jpg" align="centre"/> <br> <br> Shall we try to create a file?<br> <br> <font face="courier" size=2 color="blue"> $ cd /tmp<br> <br> <font face="arial" size=2 color="black"> This "moves" you to the "/tmp" directory, which you can verify using the "pwd" command.<br> Now start creating your first scriptfile called "myscript.sh". So use the command "vi myscript.sh" (remember that vi might be named "vim" on your system).<br> <br> <font face="courier" size=2 color="blue"> $ vi myscript.sh<br> <br> <font face="arial" size=2 color="black"> Your editor starts. Essentially, you have an empty screen. Now press "i" to go to "Ins" mode which enables you to enter text.<br> Now, using your highest concentration possible ;-), you type in:<br> <br> export Myvar="Hello Mary!" (press enter)<br> echo $Myvar (Now press Esc, and type :wq! to save the file)<br> <br> Hopefully it worked. If so, congratulations !<br> Now, do the following:<br> <br> <font face="courier" size=2 color="blue"> $ ls -al myscript.sh<br> <br> -rw-r----- mary dba 13 jan... 36 myscript.sh<br> <br> $ chmod u+x myscript.sh<br> <br> $ ls -al myscript.sh<br> <br> -rwxr----- mary dba 13 jan... 36 myscript.sh<br> <br> $ ./myscript.sh<br> <br> <font face="arial" size=2 color="black"> You see that I made the script executable (for the user/owner) by using the "chmod u+x" command.<br> Indeed, the "x" flag was added. Then I <B>called</B> the script, that is, <B>executed</B> the script.<br> <br> Note:<br> If you could not start "vi" (or "vim"), then maybe it's not on your system. However, that's not very likely.<br> You might try this from the prompt: <B>export EDITOR=vi</B><br> Try again if you can start "vi" this time.<br> <br> <font face="arial" size=2 color="brown"> <B>Optional exercise: Now as a further "vi" excercise:</B><br> <br> Start "vi" again using a filename of "test" (vi test).<br> <br> -Press "i"<br> -Now type the 1st line: &nbsp aaa aaa aaa (press Enter)<br> -Now type the 2nd line: &nbsp bbb bbb bbb (press Enter)<br> -Now type the 3rd line: &nbsp ccc ccc ccc (press Enter)<br> -Now type the 4rd line: &nbsp ddd ddd ddd <br> -Press Esc<br> -Type :wq! to save it, en exit vi. Now, we do not save it any further, so if you mangle the file<br> in the following exercises, just quit vi and load the file again.<br> <br> .Move the cursor around using the "arrow keys". Note that you can freely move around.<br> .Now place the cursor at the third "a". Press "d". Is the third "a" removed? (Maybe you must use "x" instead of "d").<br> .Press "i" and type "a" and press Esc again. You have that third "a" back again?<br> .Now place the cursor somewhere at second line. Press "d". Press "d" again. What happens?<br> .Press Esc. Place the cursor at the 6th a. Press "a", press "a" again. What happens?<br> .Press Esc. Place the cursor at the 6th b. Press "i", press "b". What happens?<br> <br> Maybe you should play around a bit in "this style".<br> <br> <br> <font face="arial" size=2 color="red"> <h3>1.5 Wildcards.</h3> <font face="arial" size=2 color="black"> You do not have "practisize" the following example. If you read it, and understand what's happening, then that's fine enough for me.<br> <br> Suppose in the "/home/abert" directory, we see the following:<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B><br> <br> -rw-r------ albert dba 17 jan... 886 myscript.sh<br> -rw-r------ albert dba 23 feb..23674 abc.dat<br> -rw-r------ albert dba 11 apr... 982 personel.txt<br> -rw-r------ albert dba 19 aug..43674 config.dat<br> -rw-r------ albert dba 17 jan... 886 ftpfiles.sh<br> -rw-r------ albert dba 23 sep..73674 qm.txt<br> -rw-r------ albert dba 25 jan... 221 def.dat<br> -rw-r------ albert dba 15 jan..88674 accpres.ppt<br> drwxr-x---- albert dba 17 jun.....12 test<br> etc..<br> <font face="arial" size=2 color="black"> <br> So, suppose <I>a whole lot of files</I> are inside that directory. Now, suppose there are a lot of subdirs too.<br> Here, we only see the subdir "test", but let's our imagination work here.<br> <br> Famous "wildcards" in Unix (and many other Operating Systems) are the "*" and "?" characters.<br> <br> Suppose I only want a listing of ".sh" files. Then I can use the "*" in the following way:<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B> *.sh<br> <br> -rw-r------ albert dba 17 jan... 886 myscript.sh<br> -rw-r------ albert dba 17 jan... 886 ftpfiles.sh<br> <br> <font face="arial" size=2 color="black"> So, the "*" can replace one or more characters. It's a wildcard for one or more characters. It is as if we say<br> to the shell "I don't care what comes before the ".sh". Anything is OK. Just give me the listing."<br> <br> Now, you can place the "*" anywhere you think it is usefull. For example, you can get a listing of "*script.sh" too.<br> This returns all files which has "script.sh" in their names, no matter what comes in front of it.<br> And something like this works too: "perso*". This returns everything with a name that starts with "perso",<br> no matter what is behind that string.<br> <br> The "?" charcter is usefull to replace one (or more) character(s) by this wildcard. But, the more "?" inserted, the more<br> it starts to look like the use of "*". For example, you only want to see the accounting reports from 1990 up to 1999:<br> <br> <font face="courier" size=2 color="blue"> $ <B>ls -al</B> acc199?rprts.doc<br> <br> <font face="arial" size=2 color="black"> Now, we only used the "ls" command up to now. But the same principle applies with all commands.<br> For example:<br> <br> <font face="courier" size=2 color="blue"> $ <B>cp</B> *.txt /tmp &nbsp &nbsp# copy the *.txt files from "/home/albert" to "/tmp"<br> <br> <font face="arial" size=2 color="black"> Or, if I am logged on to the machine STARBOSS and I want to copy *.sh files to machine STARGATE, using the "scp" utility:<br> <br> <font face="courier" size=2 color="blue"> $ <B>scp</B> *.sh albert@STARGATE:/data/scripts<br> <br> <br> <font face="arial" size=2 color="red"> <h3>1.6 The "find" and "grep" commands.</h3> <font face="arial" size=2 color="black"> These are two very powerfull commands. You can't leave home without them.<br> <br> <B>&#8658; The "find" command:</B><br> <br> This one is really great. For now, we will only see the simplest example of this command, namely just "finding" files.<br> But, very powerfull "exec" constructs exists, in combination with "find".<br> <br> Let's start simple. Suppose in large directory tree, you want to find a specific file (or a set of files), and get it in a listing.<br> Just use find. Take a look at these examples:<br> <br> <font face="courier" size=2 color="blue"> $ <B>find</B> . -name "myfile.txt" -print<br> <br> <font face="arial" size=2 color="black"> Here, the "." (the dot) tells "find" that it should start as from the current directory, and find "myfile.txt" throughout all subdirs too.<br> <br> <font face="courier" size=2 color="blue"> $ <B>find</B> . -name "*.dat" -print<br> <br> <font face="arial" size=2 color="black"> Again, the "." tells "find" that it should start as from the current directory, and find all files with the ".dat" extension,<br> throughout all subdirs too.<br> <br> <B>&#8658; The "grep" command:</B><br> <br> You will find this one in allmost all scripts. It's very usable since it <B>filters</B> information from a bulk of data.<br> Let's take a look at a few examples.<br> <br> You probably often have used the "ps -ef" command. On almost all Unixes and Linux distro's, it gives a <B>full</B> process listing.<br> Now, that can be a listing of hundreds of processes on a busy system.<br> <br> In scripts, or from the prompt, you are often interested in one kind of process only. Then filter the data with "grep".<br> <br> <font face="courier" size=2 color="blue"> $ <B>ps -ef</B><br> <br> Gives an extended listing of all processes...<br> <br> $ <B>ps -ef | grep -i ora</B><br> <br> Shows only those processes with the string "ora" in their name (if such processs is present).<br> <br> $ <B>ps -ef | grep -i syslog</B><br> <br> Shows only those processes with the string "syslog" in their name (if such processs is present).<br> <br> <font face="arial" size=2 color="black"> The "-i" argument means "case insensitive". You can leave it out, but then grep does NOT ignore "case".<br> You can use "grep" to search files too, for a certain string. For example:<br> <br> <font face="courier" size=2 color="blue"> $ <B>grep Sally</B> customers.txt<br> <br> <font face="arial" size=2 color="black"> This will return only the records containing "Sally" from the file "customers.txt".<br> <br> <br> <font face="arial" size=2 color="red"> <h3>1.7 Pipeline and Redirecting.</h3> <font face="arial" size=2 color="black"> <B>&#8658; Connecting programs through a pipe:</B><br> <br> We already have seen examples of this in section 1.6. There we saw that the "ps -ef" command<br> produces a list of all running processes. It's default output device is the connected console (your screen),<br> which is called "stdio" (standard io).<br> <br> However, using the "pipe" symbol "|", we can connect programs in such a way, that the <B>output</B> of the first program,<br> becomes the <B>input</B> of the second program. So, instead that "ps -ef" pushes it's list to your screen, it becomes the input<br> for "grep".<br> <br> Usually, only two programs are connected this way. But you can add a second "pipe", and a third "pipe" etc..<br> For example:<br> <br> <B>program1 | program2 | program3</B><br> <br> The output of program1 becomes the input for program2, and the output of program2 becomes the input for program3.<br> <br> Some examples:<br> <br> <font face="courier" size=2 color="blue"> -- The output of "ps -ef" is fed to the "more" command:<br> <br> $ <B>ps -ef | more</B><br> <br> -- The output of "ps -ef" is fed to the "grep" command:<br> <br> $ <B>ps -ef | grep -i syslog</B><br> <br> -- The output of the "cat" is fed to the "grep" command, which is then fed to the print command:<br> <br> $ <B>cat customers.txt | grep "New York" | lpr</B><br> <font face="arial" size=2 color="black"> <br> <B>&#8658; Redirecting output to a file:</B><br> <br> Many commands produces some sort of listing, which often will be printed on your screen.<br> But you can "redirect" the output <B>to a file</B> instead, using the <B>">"</B> redirection symbol.<br> <br> Here are a few examples:<br> <br> <font face="courier" size=2 color="blue"> $ <B>cat customers.txt | grep -i Sally > /tmp/sally.txt</B><br> <br> $ <B>find . -name "*.dat" -print > /home/albert/all_dat_files.txt</B><br> <br> <br> <font face="arial" size=2 color="red"> <h3>1.8 The "sed" and "tr" commands.</h3> <font face="arial" size=2 color="black"> <B>&#8658; The amazing "sed" utility:</B><br> <br> Again, here is a utility with lots of power. However, most people only seem to use the "s" or "substitute" command.<br> Well, that counts for us too. It's just that replacing stuff (in a file, or string) is simply a often sought feature.<br> <br> How does it work?<bt> <br> Let's create a small file with data. Goto "/tmp" (or your homedir). Then, using vi, create a file called "oldfile.txt"<br> having the following content. If you wonder about the "numbers" before some names: thats on purpose.<br> <br> <font face="courier" size=2 color="blue"> Sally Amsterdam<br> 5Henry New York<br> 99Carl Washington<br> Miranda Berlin<br> 77Albert Amsterdam<br> <br> <font face="arial" size=2 color="black"> Now imagine that it's a file containing thousends of records. Suppose that all "Berlin" fields must be "substituted" by "Munchen".<br> That's a simple task for "sed"<br> <br> <font face="courier" size=2 color="blue"> $ <B>sed 's/Berlin/Munchen/' oldfile.txt > newfile.txt</B><br> <br> $ <B>cat newfile.txt</B><br> <br> Sally Amsterdam<br> 5Henry New York<br> 99Carl Washington<br> Miranda Munchen<br> 77Albert Amsterdam<br> <br> <font face="arial" size=2 color="black"> So, when using the "s" (substitute) command, use "sed" like so:<br> <br> <B>sed 's/oldstring/newstring/' originalfile > newfile</B><br> <br> The "/" is the "delimiter" which seperates the oldstring from the newstring (within the command), which is almost always OK.<br> So, this delimiter has nothing to do with the delimiter of fields inside the file (like a space, a tab, a ";" etc..).<br> It's only there so that sed can discriminate between "oldstring" and "newstring" on the commandline.<br> <br> Sometimes however, the default delimiter "/" does not work. Just suppose you have a file listing all sorts of filesystem paths.<br> Here, a lot of slashes "/" can be expected. So, if you want to replace a certain path by another path (in the file), another delimiter<br> is needed. In such a case, you could try the "_" delimiter like in:<br> <br> <font face="courier" size=2 color="blue"> $ <B>sed 's_/home/albert_/home/mary_' OldfileWithPaths.txt > NewfileWithPaths.txt</B><br> <br> <font face="arial" size=2 color="black"> Remember that the oldfile.txt (which you created with vi) has some records with names with numbers in front of them?<br> Now, this could be seen in reality too. Maybe you get a file and you are asked to remove "all polution" (like numbers) from it,<br> so that the endresult is just a clean file with just simply names and cities, and all numbers removed.<br> <br> This is how you can do it:<br> <br> <font face="courier" size=2 color="blue"> $ <B>sed 's/[0-9]*//' oldfile.txt > newfile2.txt</B><br> <br> $ <B>cat newfile2.txt</B><br> <br> Sally Amsterdam<br> Henry New York<br> Carl Washington<br> Miranda Berlin<br> Albert Amsterdam<br> <br> <font face="arial" size=2 color="black"> Notice that we do not exactly have a "/oldstring/newstring/" really. Actually we only want to <B>remove</B> numbers.<br> So we want to "remove" stuff, and that stuff is not replaced by anything. That's why the "//" is empty.<br> <br> The [0-9]* stuff is great in all Unix shells and programs. Anything that looks like a "range" [0-9], <I>is taken</I> as a range.<br> At runtime, it's expanded in all numbers 0,1,2,3,4,5,6,7,8,9. So, that's great !<br> <br> The <B>*</B> is a wildcard, meaning "everything" here.<br> <br> In effect we say to "sed": look at all fields ("*"). If you see anything in [0-9], replace it by "null" ("//").<br> <br> <B>&#8658; The "tr" command: </B><br> <br> The "tr" command is especially usefull for <B>replacing characters</B> in a file, or on any other input delivered to "tr", for example,<br> through a pipe from another program.<br> <br> Now you might say that "tr" then has a lot in common with "sed". A bit indeed, but they do have different scopes.<br> <br> "Sed" for example, is great for replacing a word (like a name) by another word, throughout a file.<br> On the other hand, "tr" is great for replacing a certain "feature, like replace small case letters to captital letters,<br> or removing to two consequetive spaces to one space, throughout a file.<br> <br> Now that we have seen the use of a range (like [0-9]) when we discussed "sed", we don't let go of such fantastic feature,<br> since it can be used wih "tr" too (and that's true with most other unix utilities).<br> <br> So, how is "tr" used? Some examples will clarify.<br <br> Go to your homedir, or to "tmp".<br> Use vi to create a textfile named "trtest.old" with the content as shown below.<br> Note: Take notice of the upper and lower casing of characters. That's on purpose. The extra two spaces in "PE TER" is on purpose too. <br> <br> After you are done creating the file, we should have this content:<br> <br> <font face="courier" size=2 color="blue"> $ cat trtest.old<br> <br> SALly<br> hENDRIck<br> PE TER<br> alBErt<br> <br> <font face="arial" size=2 color="black"> Now let's see if we can replace all Upper case letters to lower case. We can use the following command for that:<br> <br> <font face="courier" size=2 color="blue"> $ <B> cat trtest.old | tr "[A-Z]" "[a-z]" > trtest.new</B><br> <br> <font face="arial" size=2 color="black"> But let's try another notation ! It's another way to write the same command.<br> Actually, It's quite stunning. Take a look at this:<br> <br> <font face="courier" size=2 color="blue"> $ <B>tr "[A-Z]" "[a-z]" < trtest.old > trtest.new</B><br> <br> <font face="arial" size=2 color="black"> Isn't cool to write commands in such a way? The "tr"command gets its <B>input</B> through the use of the "&#60;" symbol.<br> This input is the file "trtest.old". Using the usual "redirection" symbol ">", the output goes to the file "trtest.new".<br> <br> We have been busy on modifying files, but both "sed" and "tr" can get their input from a "pipe" as well, like so:<br> <br> <font face="courier" size=2 color="blue"> $ <B>echo "AB CD efgHIJ KLM" | tr "[A-Z]" "[a-z]" </B><br> <br> ab cd efghij klm<br> <br> <font face="arial" size=2 color="black"> Here, the "echo" command pushes its data through the pipe to "tr", which replaces all Upper case with lower case characters.<br> <br> <br> <font face="arial" size=2 color="red"> <h3>1.9 The "awk" command.</h3> <font face="arial" size=2 color="black"> The "awk" command is again such a utility with gigantic capabilities. It's almost a sort of "perl" environment by it's own!<br> Someone could write a 50 page introduction on "awk" with ease. And there are quite a few of them.<br> <br> Awk is simply too large to do it justice in such a note as this. But, don't worry. Mosts scripts only use the<br> basic capabilities of "awk".<br> <br> For the basic functionality of "awk", you might describe it as a processor for anything that produces rows and fields<br> that you want to be processed and/or formatted in some way.<br> <br> We already have seen commands which were connected by a pipe. Indeed, often "awk" receives input that way and then starts<br> filtering and/or formatting the data the way you want it to be.<br> Often, in scripts you will find statements like this:<br> <br> <B>"some program" | awk '{print $1 $3}'</B><br> <br> Here, "some program" pushes records through the pipe to awk. In the example above, awk then only prints the first and third record.<br> <br> In this setup, "$0" represents the full record, "$1" the first field, "$2" the second field etc..<br> <br> The program that delivers data to awk, can be anything. Like for example "ps -ef" which produces a process list, or a command<br> which lists Logical volumes with their attributes, or "cat", which opens a file and pushes the records to awk etc.. etc..<br> <br> Here are a few examples:<br> <br> <font face="courier" size=2 color="blue"> $ <B>echo "A B C D" | awk '{ print $1 $3 }'</B><br> <font face="arial" size=2 color="black"> <br> <font face="arial" size=2 color="brown"> <B>Question:</B><br> Explain why we see AC as output.<br> <font face="arial" size=2 color="black"> <br> But it's not true that awk is always used as "just a utility" in a "pipe". It can operate by itself too, directly.<br> Let's create a file with vi. Name it "custlist.txt", and put the following records in it:<br> <br> <font face="courier" size=2 color="blue"> Customer Month Debt<br> KLM June 1000<br> GM June 5000<br> KLM December 2000<br> Murphys November 10000<br> <br> <font face="arial" size=2 color="black"> Let's try this:<br> <br> <font face="courier" size=2 color="blue"> $ <B>awk '/KLM/ { print $0 }' custlist.txt</B><br> <br> <font face="arial" size=2 color="black"> If you run that, you only get the records containting "KLM". So here awk functions as a filter for searching<br> certain records from a file.<br> <br> <font face="arial" size=2 color="brown"> <B>Question:</B><br> Can you think of a simpler, but equivalent command, by just using "grep"?<br> <font face="arial" size=2 color="black"> <br> <br> Right. Now we have a good basis of general commands and concepts. Although we have not yet dealt with a number<br> of other usefull commands (like "wc -l" and others), I will explain those commands while we go through the example scripts.<br> Let's finally start with Chapter 2.<br> <br> <br> <br> <font face="arial" size=2 color="blue"> <h2>Chapter 2. Basic shell scripting.</h2> <font face="arial" size=2 color="black"> As said before in the introduction, chapter 1 really is the basis for Chapter 2.<br> <br> As you will see, "shell scripting" is easy. The <B>real</B> trick is, that you understand those remarkable shell utilities<br> like "sed", "tr", "awk", "grep" and a couple of others. So if you are in doubt, return to Chapter 1.<br> <br> <br> <font face="arial" size=2 color="red"> <h3>2.1 Some "housekeeping" first.</h3> <font face="arial" size=2 color="black"> Here you see a simple example of a script.<br> <br> <font face="courier" size=2 color="blue"> #!/bin/sh<br> ###############################################################################<br> # Purpose.......:Script wich searches the "myapp logfile" if any critical errors <br> # ...............are found. If found, a mail is send to the Operator.<br> # Author.......: Mickey Mouse<br> # Version......: 0.1<br> # Date.........: 11/11/2009<br> # Modifications:-------------------------------------<br> #...............-------------------------------------<br> #...............-------------------------------------<br> ###############################################################################<br> <br> <br> cat /apps/myapp/logs/myapp.log | grep -i "Error" > /tmp/errlog.txt<br> <br> if [ `cat /tmp/errlog.txt | wc -l` -gt 0 ]<br> then<br> &nbsp mailx -s "Critical error found in myapp.log" operator@mycompany.org<br> fi<br> <br> <font face="arial" size=2 color="black"> <B>&#8658; The "directive":</B><br> <br> First, do you notice the first line "#!/bin/sh"? It tells your shell which "interpreter or shell" must execute<br> this script. It's always recommended to use it, to denote the right interpreter.<br> The #! is a <B>directive</B> for your shell which interpreter (or program loader) to use.<br> <br> Since scripts are fairly portable across shells, you often "get away" with it, if you have left it out.<br> However, it's always possible that your script uses specific features from some shell, and there things<br> might go wrong.<br> <br> So, suppose you always work with the "/usr/bin/ksh" shell, and your scripts are intended for the korn shell,<br> then use "#!/usr/bin/ksh" as a <B>directive</B>.<br> <br> It's actually quite "universal". If you would create perl scripts, and you want to call the script just by it's name<br> from the unix/linux prompt, then use something like "#!/usr/bin/perl". In this case, your unix shell immediately knows it must<br> run perl with this script.<br> <br> <B>&#8658; Some "housekeeping":</B><br> <br> Any line that starts with "#", is a "comment" (except #!).<br> <br> - You should always start your scripts with a block of usefull coments, like "the purpose" of your script,<br> the "version", the "Author", the "last modification date" etc..<br> It's really a help for someone who must support or maintain your scripts at a later time.<br> <br> Now, the example above is so small, that actually such a block might seem a bit overdone.<br> In this case... Yes. But for your <I>serious production scripts</I>, it's really a "must" for obvious reasons.<br> <br> - Also, if your script is large, then provide some comment on steps which you think deserves it (like what seems as complex code),<br> using the "#" character preceding the comment.<br> <br> - You do not explicitly have to put the statement "exit" or "return" (or something) anywhere, to end the script.<br> Only, for example, if using an "if" test, some condition turns out to be true, which makes running the script "non desired" or<br> even "impossible", you might use "exit" to terminate the script right away.<br> <br> For example, if a script must be run under a certain user account, you might test the useraccount first.<br> <br> <font face="courier" size=2 color="blue"> if [ `whoami` != root ]<br> then<br> &nbsp echo "You must run this script as root."<br> &nbsp <B>exit</B><br> fi<br> <font face="arial" size=2 color="black"> <br> Here, the `whoami` will return the account of the user. If it's not "root", the script "exits" immediately.<br> A further explanation on how to use the "if" statement with "string comparison", will be done in section 2.3.<br> <br> <br> <B>&#8658; Explaining the example script at the beginning of Chapter 2:</B><br> <br> You see that I use the "cat" command first, to open and read all lines of the "/apps/myapp/logs/myapp.log" file.<br> <br> Then, using a pipe "|", all records will be pushed to the "grep" command, which will <only> filter out the lines<br> containing the string "Error"<br> <br> Then we use "redirection" so that grep places it's output into the file "/tmp/errlog.txt".<br> So we know for sure, that /tmp/errlog.txt will only contain error messages and nothing else from the "myapp.log" file.<br> <br> So, if no errors were found, the "/tmp/errlog.txt" will not have any lines, that is, it contains "0" lines.<br> <br> Next, something interesting happens. Ofcourse, in shell scripts all sorts of "loop" constructs are possible, as well as<br> forms of "decision logic". The "If" statement is a form of "decision logic".<br> Later on we will see the "If" many times.<br> <br> In this example it works like so:<br> <br> "If [number of lines in /tmp/errlog.txt > 0] then<br> &nbsp "send a mail to the operator that errors were found"<br> <br> The "If" logic generally works like this: If .. then .. else ..<br> However, in this particular example, we do not need the "else" block.<br> <br> Now, you still might wonder what exactly the [`cat /tmp/errlog.txt | wc -l` -gt 0] does.<br> <br> The "wc -l" command will count the <B>"number of lines"</B> of anything that it receives from a pipe. Now you see that the "cat" command<br> is used which opens and reads everything out of "/tmp/errlog.txt", and pushes those records to "wc -l".<br> <br> <font face="arial" size=2 color="brown"> Consequently, "wc -l" might have counted zero or more lines, since "/tmp/errlog.txt" might have zero or more lines.<br> <br> If the number of counts > 0, (or denoted as "-gt 0"), the If condition is "true" and a mail is send.<br> If the number of counts = 0, then nothing happens.<br> <font face="arial" size=2 color="black"> <br> Now, I only have one small little thing to explain.<br> <br> Ofcourse, you can always use a command like this <B>cat /tmp/errlog.txt | wc -l</B> in a script.<br> <br> However, this time we have the command in the "if [ ]" block, so we <I>must</I> tell unix that it can really <B>run it</B>, even if it's within<br> the [ ] brackets. You can achieve this by placing the command within the `` quotes.<br> On most keyboards, you find this quote in the upper-left corner, just below the Esc key.<br> <br> <B>Exercise:</B> <br> Create, with vi (or other editor), a file in "/tmp" and call it "errlog.txt". Just put a line of arbitrary text in "errlog.txt".<br> <br> Next, create the file <B>"test.sh"</B> in "/tmp", and put the following code in it:<br> <br> <font face="courier" size=2 color="blue"> if [ `cat /tmp/errlog.txt | wc -l` -gt 0 ]<br> then<br> &nbsp echo "the number of lines in errlog.txt is greater than 0"<br> fi<br> <br> <font face="arial" size=2 color="black"> Once the script file is finished and saved, make it executable:<br> <br> <font face="courier" size=2 color="blue"> $ chmod +x test.sh<br> <br> <font face="arial" size=2 color="black"> You can execute "test.sh" using:<br> <br> <font face="courier" size=2 color="blue"> $ ./test.sh<br> <br> <font face="arial" size=2 color="black"> <B>Now, experiment with the script when errlog.txt has 0 lines, or > 0 lines.</B><br> <br> Question: Can you explain when you get the message "the number of lines in errlog.txt is greater than 0"?<br> <br> <br> <font face="arial" size=2 color="red"> <h3>2.2 Variables.</h3> <font face="arial" size=2 color="black"> We do not have to create a script to see how variables work in a Unix/Linux shell. Statements from the prompt, will demonstrate them just as well.<br> <br> <br> <B>&#8658; Variable assignments:</B><br> <br> Try the following statements:<br> <br> <font face="courier" size=2 color="blue"> $ me="Albert"<br> $ echo $me<br> Albert<br> <br> $ var1=5<br> $ echo $var1<br> 5<br> <br> $ now=`date`<br> $ echo $now<br> Thu Jan 17 20:31:32 2013<br> <br> <font face="arial" size=2 color="black"> <br> So, a variable is set with an assignment of <B>variablename=value</B>. Be sure not to have any white space before or after<br> the equals sign "=". In the first example, I used double quotes around the string "Albert". Actually, double quotes is only<br> neccessary when white space is present in the text string.<br> <br> Once a variable is set, you always refer to it using a "$" character in front of the name, like in "$me".<br> <br> The first two variables are just "static" values, like a name or a number. The third example is really great!<br> In this example, the <B>date</B> command is used (which just returns the current "date"), and is placed between <B>``</B> quotes.<br> The `` quotes always signals to the shell, that it must "expand" this variable, that is, to run that command.<br> <br> Or what about this example. Again, we define a variable named "list" in the form of a unix command, which will be executed<br> at runtime (that is, when you press Enter, or if it's in a script, when the script is run).<br> <br> <font face="courier" size=2 color="blue"> $ list=`ls -al | wc -l`<br> $ echo $list<br> 37<br> <font face="arial" size=2 color="black"> <br> As we know, "ls -al" produces a listing of all files in the current directory. So, when that list is pushed to "wc -l", we just<br> get the counts returned (that is, the number of files).<br> <br> Here is another example:<br> <br> <font face="courier" size=2 color="blue"> $ CountOracle=`ps -ef | grep -i ora | wc -l`<br> $ echo $CountOracle<br> 187<br> <br> <font face="arial" size=2 color="black"> <br> Note:<br> <br> A variable will be expanded (to its value) in a string with double quotes (but not in single quotes). Here is what I mean:<br> <br> <font face="courier" size=2 color="blue"> $ name="Albert"<br> $ echo "Hi there $name"<br> <br> Hi there Albert<br> <font face="arial" size=2 color="black"> <br> <B>&#8658; Exporting a variable:</B><br> <br> Variables "declared" as above, are local to your current shell instance. They are not garanteed to exist for other scripts and commands.<br> <br> To create variables that are Global to your <I>session</I> you have to "export" a variable.<br> Variables that are marked for export are called <B>environment variables</B>, and are available for any command or script<br> during your session.<br> <br> In the most "common shells" (like "sh", "bash", and "korn"), use the "export" command like so:<br> <br> <font face="courier" size=2 color="blue"> $ export me="Albert"<br> $ echo $me<br> Albert<br> <br> $ export ORACLE_HOME=/apps/oracle/product/10.2<br> $ echo $ORACLE_HOME<br> <br> /apps/oracle/product/10.2<br> <br> <font face="arial" size=2 color="black"> In the second example, the $ORACLE_HOME variable functions like a sort of "alias" to some part of the directory tree.<br> These sort of variables have great advantages. Suppose that there would exist a "bin" directory like "/apps/oracle/product/10.2/<B>bin</B>,<br> containing all sorts of programs like for example the program "sqlplus".<br> <br> Once the ORACLE_HOME variable is defined as above, you can call those utilities using a "shortcut", like for example:<br> <br> <font face="courier" size=2 color="blue"> $ $ORACLE_HOME\bin\sqlplus<br> <br> <font face="arial" size=2 color="black"> <br> Note:<br> <br> Sometimes "a slight detour" is used to export a variable. It amounts to the same thing, but it needs more typing:<br> <br> <font face="courier" size=2 color="blue"> $ ORACLE_HOME=/apps/oracle/product/10.2; export ORACLE_HOME<br> <br> <br> <font face="arial" size=2 color="red"> <h3>2.3 Conditionals: The "if" and "case" statements.</h3> <font face="arial" size=2 color="black"> Shell scripting, like almost all other scripting environments, uses the "If" or "case" statements, to implement "decision logic".<br> <br> - use the "if" statement if the logic is based on some condition that's either "true" or "false".<br> - Use the "case" statement if the logic is based on some value that's one of a number of possible values.<br> <br> <font face="arial" size=2 color="blue"> <h4>2.3.1 The "if" statement:</h3> <font face="arial" size=2 color="black"> In many occasions, your script will perform certain actions <I>if</I> a <B>certain condition</B> is "true".<br> If that condition is "not true", or "false", optionally an alternative action is done. Here, is the general "If" format is:<br> <br> <font face="courier" size=2 color="blue"> <B>If</B> [ "evaluate a certain test between brackets" ]<br> <B>then</B><br> &nbsp statement(s) if the test=true<br> <B>else</B><br> &nbsp other statement(s) if the test=false<br> <B>fi</B><br> <br> <font face="arial" size=2 color="black"> Always end the "if" statement with "fi", which functions as an "endmarker" to the shell. Note that "fi" is "if" backwards.<br> The "else" branch is optional. If you do not need it, you can skip it. This is just what was done in the example in section 2.1.<br> <br> The "[ ]" alsways contains some sort of <B>"test"</B>, which evaluates to "true" or "false". Here are a few examples<br> on what can be "tested":<br> <br> <font face="courier" size=2 color="blue"> <B>1: Tests on objects:</B><br> <br> [ -f file ]... the "-f" means "test if the file exists"<br> [ -d dir ].... the "-d" means "test if the directory exists"<br> [ -s file ]... the "-s" means "test if the file > 0 bytes"<br> <br> (there are more similar tests...)<br> <br> <B>2: Tests on nummeric variables:</B><br> <br> [ $x -lt $y ]... means "test if $x is lower than or equal $y"<br> [ $x -le $y ]... means "test if $x is lower than $y"<br> [ $x -eq $y ]... means "test if $x and $y are equal"<br> [ $x -gt $y ]... means "test if $x is greater than $y"<br> [ $x -ge $y ]... means "test if $x is greater than or equal $y"<br> <br> <B>3: Tests on string variables:</B><br> <br> [ "$var" == "string value" ]... means "test if $var contains the same characters as "string value"<br> [ "$var" != "string value" ]... means "test if $var does NOT have the same characters as "string value"<br> <br> Note the space after "[" and the space before "]".<br> <br> <br> <font face="arial" size=2 color="black"> <B>----> Intermezzo: The "touch" command:</B><br> <br> As an intermezzo, we need to explain the "touch" command first, before we do any exercise. It's not always neccessary to use "vi"<br> if you "just need a file". If an "empty" file (0 bytes) is good enough, you can simply use the "touch" command, like so:<br> <br> <font face="courier" size=2 color="blue"> $ cd /tmp<br> $ touch errlog.txt<br> <br> <font face="arial" size=2 color="black"> It means that the file "errlog.txt" will be created, although it's completely "empty".<br> <br> <B>----> End Intermezzo</B><br> <br> <br> <font face="arial" size=2 color="brown"> <B>Exercise:</B><br> <br> <br> test on the existence of a file (-f parameter):<br> <br> - Go to "/tmp" (or your homedir, using "cd ~")<br> - Use vi to create a shell script called "rmlog.sh"<br> - Put the following code into "rmlog.sh"<br> <br> <font face="courier" size=2 color="blue"> if [ -f /tmp/myapp.log ]<br> then<br> &nbsp rm /tmp/myapp.log<br> else<br> &nbsp echo "no log file found"<br> fi<br> <br> <font face="arial" size=2 color="brown"> It's important to keep a space between the "[", and the string "-f /tmp/myapp.log", and to have a space before the last "]" too.<br> <br> - After the script has been saved, use "chmod +x rmlog.sh" to make it executable.<br> - Run the script using "<B>./rmlog.sh</B>"<br> - Go to "/tmp" and use the command "touch myapp.log"<br> - Run the script again.<br> - Go to "/tmp" and check if the logfile still exists or not (using "ls -al myapp.log") .<br> <br> Note: the "rm" command removes a file.<br> </B> <br> <font face="arial" size=2 color="black"> <br> The exercise above, dealt with the <B>test on the existence</B> of an object, such as a file.<br> Lets walk trough a <B>test on nummeric variables</B>, this time. If we would make this script:<br> <br> <font face="courier" size=2 color="blue"> var1=1<br> var2=2<br> <br> if [ $var1 -le $var2 ] <br> then <br> &nbsp echo "Var1 is smallest" <br> else <br> &nbsp echo "Var2 is smallest" <br> fi <br> <br> <font face="arial" size=2 color="black"> I think you agree that if we would run this script, we would see the message: "Var1 is smallest" <br> <br> So remember: for <B>"nummerical"</B> comparisons in the "[ ]", use the "lt", "le", "eq", "gt", and "ge" operators.<br> <br> It's important to realize that, up to now, we have seen "If" tests with respect "on the existence" of an object, and<br> comparisons between nummeric variables.<br> <br> How about string tests? That is, suppose a variable contains some string (some characters), and we need to test that.<br> It's a bit more of the same, but I want to you have a <U>sharp eye</U> on the fact that <B>this time</B>, we use the "==" and "!=" operators.<br> <br> Here are a few simple examples:<br> <br> <font face="courier" size=2 color="blue"> <B>- If you need to determine if variable s contents is equal to a certain fixed string:</B><br> <br> var1=Mary<br> <br> if [ "$var1" == "Mary" ]<br> then<br> &nbsp echo "It's Mary"<br> fi<br> <br> <B>- If you need to determine if variable s contents is Not equal to a certain fixed string:</B><br> <br> var1=Harry<br> <br> if [ "$var1" != "Mary" ]<br> then<br> &nbsp echo "It's Not Mary"<br> fi<br> <br> <font face="arial" size=2 color="blue"> <h4>2.3.2 The "case" statement:</h3> <font face="arial" size=2 color="black"> With the "if" statement, you can handle most "decision tree's" in your scripts. However, suppose the number<br> of options is really fixed. In such cases, using the "case" statement is better.<br> <br> In the following example we will also use the <B>"read"</B> command. The "read" command makes it possible to get<br> user input from the keyboard, which usually will be placed into some variable.<br> <br> You really should try the example below. It's quite cool. Just create a file with vi, name it for example "inputtest.sh",<br> and put the following code in that file:<br> <br> Example:<br> <br> <font face="courier" size=2 color="blue"> echo "Please type yes or no:"<br> read answer<br> <B>case</B> $answer in<br> &nbsp yes|Yes|y)<br> &nbsp &nbsp echo "You said yes.."<br> &nbsp &nbsp ;;<br> &nbsp no|n)<br> &nbsp &nbsp echo "You said no.."<br> &nbsp &nbsp ;;<br> &nbsp q*|Q*)<br> &nbsp &nbsp #Probably the user wants to stop<br> &nbsp &nbsp exit<br> &nbsp &nbsp ;;<br> &nbsp *)<br> &nbsp &nbsp echo "If none of the above was typed, this is the default.."<br> &nbsp &nbsp ;;<br> <B>esac</B><br> <br> <font face="arial" size=2 color="black"> Note that, just as was the case with the if statement, the "case" statement ends with "esac" which is "case" backwards.<br> Also note that every "option" must be terminated by ";;".<br> <br> In the example above, you see indeed a number of options, which represents what the user could key in, like "y" or "n" etc..<br> The case statement should also have a "default" option, expressed by "*)" which then handles every case which is not covered<br> by the regular options. However, it's not required.<br> <br> Example:<br> <br> Here is another example. This example you probably can't try yourself. But it's a nice illustration on how a Unix system administrator<br> might program a tape robot, before the backup takes place. <br> <br> <font face="courier" size=2 color="blue"> DAYNAME=`date +%a`;export DAYNAME<br> <br> case $DAYNAME in<br> Mon)<br> &nbsp tapeutil -f /dev/smc0 move 256 4121<br> &nbsp tapeutil -f /dev/smc0 move 4096 256<br> &nbsp ;;<br> Tue) <br> &nbsp tapeutil -f /dev/smc0 move 256 4116<br> &nbsp tapeutil -f /dev/smc0 move 4101 256 <br> &nbsp ;;<br> Wed) <br> &nbsp tapeutil -f /dev/smc0 move 256 4117<br> &nbsp tapeutil -f /dev/smc0 move 4100 256<br> &nbsp ;;<br> Thu) <br> &nbsp tapeutil -f /dev/smc0 move 256 4118<br> &nbsp tapeutil -f /dev/smc0 move 4099 256<br> &nbsp ;;<br> Fri) <br> &nbsp tapeutil -f /dev/smc0 move 256 4119<br> &nbsp tapeutil -f /dev/smc0 move 4098 256 <br> &nbsp ;;<br> Sat) <br> &nbsp tapeutil -f /dev/smc0 move 256 4120<br> &nbsp tapeutil -f /dev/smc0 move 4097 256<br> &nbsp ;;<br> esac<br> <br> <font face="arial" size=2 color="black"> Don't worry about the exact purpose of the script. I only want you to see that I start with a variable DAYNAME,<br> which, using the date command with a special parameter (not on all unixes it works that way), produces strings like<br> "Mon", "Tue" etc.. So, if the script runs on monday, DAYNAME=Mon. If the script runs on tuesday, DAYNAME=Tue etc..<br> <br> Now look at the "case" logic. If for example DAYNAME=Mon, the "tapeutil" command removes certain tapes from certain slots,<br> and replaces them with new tapes.<br> The same happens with different slots for the other days.<br> <br> So, actually, here the "case" statement was a great help for a Unix system administrator.<br> <br> Again, notice that the "case" statement works great if there are a limited number of options to choose from (like Mon, Tue etc..).<br> <br> <br> <font face="arial" size=2 color="red"> <h3>2.4 Loops.</h3> <font face="arial" size=2 color="black"> Any scripting environment uses "loop constructs" in order to <B>repeat actions</B>. Ofcourse, unix shell scripts have that too.<br> <br> Three "main" loop constructs are possible:<br> <br> <font face="courier" size=2 color="blue"> for (variable in list)<br> &nbsp do Statements<br> done<br> <br> while (condition is true)<br> &nbspdo Statements<br> done<br> <br> until (variable reaches a value, or becomes true)<br> &nbspdo Statements<br> done<br> <br> <font face="arial" size=2 color="black"> For the "while" and "for" loops, a couple of examples is shown below.<br> <br> <B>&#8658; The "For" loop:</B><br> <br> This one is very good, if you must work through a <B>"list"</B> of values, or objects (like files).<br> Once the list is determined, the number of loops is "fixed". For example, you have a list of "100 files" or so, where the statements in the "for" loop must operate on.<br> Here are a few typical examples:<br> <br> <font face="courier" size=2 color="blue"> for i in eat run jump play<br> do<br> &nbsp echo "See Mary $i"<br> done<br> <br> Output:<br> <br> See Mary eat<br> See Mary run<br> See Mary jump<br> See Mary play<br> <br> <font face="arial" size=2 color="black"> Here we have a fixed list of 4 values. The variable $i takes on every value sequentially. Now you may ask: Why not "for $i in.."?<br> Remember that you declare a variable as for example: "var1=5". So, only <I>after</I> it gets a value, <I>then</I> you refer to it as $var1.<br> <br> Here is another "for" loop:<br> <font face="courier" size=2 color="blue"> <br> for file in `ls`<br> do<br> &nbsp mv $file $file.sql<br> done<br> <br> <font face="arial" size=2 color="black"> You know that "ls -al" produces a "long listing" of files, meaning that you get a list of filenames, with their sizes, owner etc..<br> On the other hand, just a simple list of filenames can be obtained using the <B>"ls"</B> command "as is", without the parameters "-al".<br> As we know, the shell will expand the expression <B>`ls`</B> (note the quotes) at runtime, which produces a list of filenames<br> inside some directory.<br> After the list has been produced, the "for" loop opens, and every file is "renamed" to "filename".sql, using the "mv" (move) command.<br> <br> <B>&#8658; The "While" loop:</B><br> <br> The "while" loop is used in such a way, that the loop keeps on "looping" while some "condition" remains true.<br> Here is a very simple example:<br> <br> <font face="courier" size=2 color="blue"> i=1<br> <br> while [ $i -le 10 ]<br> do<br> &nbsp echo "I have printed this $i times.."<br> &nbsp i=$(( $i + 1 ))<br> done<br> <br> <font face="arial" size=2 color="black"> Initially, the "counter" "i" is set to 1. Then the while loop starts. The condition "$i lower than or equal to 10", is true,<br> since 1 < 10. So, the "echo" statement within the loop is done. Then, the counter "i" is incremented by "1".<br> So now, "i=2". Next, the condition for the while loop is tested again. Again, it is "true" since 2 < 10.<br> So now the loop runs for the second time, and the "echo" statement within the loop is executed.<br> <br> The loop goes on and on, until the counter "i" reaches the value of "10". This time, the condition for the while loop is not "true" anymore,<br> and so the loop stops.<br> <br> By the way, the statement to increment a counter as I did in <B>"i=$(( $i + 1 ))"</B> might vary accross shells.<br> If your shell complains about it, try <B>((i++))</B> instead, which is a different way of saying <B>"increment i by one"</B>.<br> <br> Here is another example:<br> <br> <font face="courier" size=2 color="blue"> keeplooping=1<br> <br> echo "type y to loop on, or q to quit"<br> <br> while [ $keeplooping -eq 1 ] ; do<br> &nbsp read quitnever<br> &nbsp if [ "$quitnever" = "y" ] <br> &nbsp then <br> &nbsp &nbsp echo "type y to loop on, or q to quit"<br> &nbsp fi<br> &nbsp if [ "$quitnever" = "q" ] <br> &nbsp then<br> &nbsp &nbsp <B>break;</B><br> &nbsp fi<br> done<br> <br> <font face="arial" size=2 color="black"> It's a bit of a strange example. If you want to try it, I hope it runs on your system without modifications.<br> If it does not run right away.. Well, don't worry too much about it. It's only an example.<br> <br> But, it's interresting to walk through it, because in the loop, we read user input (thanks to the "read" command), and we have two "if" blocks.<br> <br> First, notice that the variable "keeplooping" is set to "1". The while loop starts and tests if $keeplooping = 1.<br> This is true, so the loop goes for the first run. Then we read what the user types in: "y" or "q".<br> What the user types in, will go into the variable "$quitnever". It "$quitnever"=y, then $keeplooping stays "1".<br> So, we go to the next run of the loop. So, if the user only types in "y", the loop will never stop.<br> <br> Now, suppose that the user types "q". Then "$quitnever" = "q". This means that we enter the second "if" statement.<br> Here, we see the <B>"break"</B> command, which always terminates the "while" loop.<br> <br> We already knew that the "while" loop keeps looping while the "condition" is true. However, as we see here,<br> there is another way to stop the "while" loop, and that's by using the "break" command in some "if" block.<br> <br> <br> <font face="arial" size=2 color="red"> <h3>2.5 Parameters or arguments for scripts.</h3> <font face="arial" size=2 color="black"> Often, you will use a parameter (or argument) <I>to supply to</I> a script.<br> Suppose you have a script that "does something" with a report, but it works with other<br> (similar) reports as well. Now, you never want to "hardcode" a reportname inside the script.<br> So, you build your script to be able to use "arguments" (supplied to the script), like in this example:<br> <br> <font face="courier" size=2 color="blue"> $ ./processreport.sh ReportFeb2011.txt<br> <font face="arial" size=2 color="black"> <br> So, at another time, you can for example re-use the script as in:<br> <br> <font face="courier" size=2 color="blue"> $ ./processreport.sh ReportJan2012.txt<br> <font face="arial" size=2 color="black"> <br> The arguments have certain variable "names" at runtime, which can be very useful:<br> <br> <font face="courier" size=2 color="brown"> Here are a few common ones...:<br> <br> <TABLE border=1 BGCOLOR=#81DAF5> <TR> <TD><font face="courier" size=2><B>The shell program itself</B></TD> <TD><font face="courier" size=2><B>$0</B></TD> </TR> <TR> <TD><font face="courier" size=2>argument 1 through 9</TD> <TD><font face="courier" size=2>$1 .. $9</TD> </TR> <TR> <TD><font face="courier" size=2>nth argument</TD> <TD><font face="courier" size=2>${n}</TD> </TR> <TR> <TD><font face="courier" size=2>number of supplied parameters</TD> <TD><font face="courier" size=2>$#</TD> </TR> <TR> <TD><font face="courier" size=2>every parameter</TD> <TD><font face="courier" size=2>$*</TD> </TR> <TR> <TD><font face="courier" size=2>errorcode returned by last executed cmd</TD> <TD><font face="courier" size=2>$?</TD> </TR> <TR> <TD><font face="courier" size=2>pid of last background command</TD> <TD><font face="courier" size=2>$!</TD> </TR> </TABLE> <font face="arial" size=2 color="black"> <br> Here are a few examples:<br> <br> <font face="courier" size=2 color="blue"> Here is a simple Bash script which prints out all its arguments. <br> <br> #!/bin/bash<br> # <br> # Print all arguments (version 1)<br> #<br> <br> for arg in $*<br> do<br> echo Argument $arg<br> done<br> <br> echo Total number of arguments was $#<br> <br> <font face="arial" size=2 color="black"> The `$*' symbol stands for the entire list of arguments and `$#' is the total number of arguments.<br> <br> So, in the above example you could have started the script like for example<br> <br> <font face="courier" size=2 color="blue"> $ ./testscript.sh bike car airplane<br> <font face="arial" size=2 color="black"> <br> The next example is a script that expects a parameter that can be either "start" or "stop",<br> in order to start/stop a certain application.<br> So, you would use the control script as in:<br> <br> <font face="courier" size=2 color="blue"> $ ./control.ksh start<br> $ ./control.ksh stop<br> <font face="arial" size=2 color="black"> <br> While the script "control.ksh" might contain something like:<br> <br> <font face="courier" size=2 color="blue"> #!/bin/ksh<br> <br> # purpose: script that will start or stop the app.<br> <br> case "$1" in<br> start )<br> echo "Start the app.. A moment please.."<br> su - appowner -c '/prj/app/appenviron.sh -e ENV1 -c "app.sh -t start"'<br> ;;<br> stop )<br> echo "Stop the app.. A moment please"<br> su - appowner -c '/prj/app/appenviron.sh -e ENV1 -c "app.sh -t stop"'<br> ;;<br> * )<br> echo "Usage: $0 (start | stop)"<br> exit 1<br> esac<br> <br> <font face="arial" size=2 color="black"> <br> <br> <br> <h3>Well...That's about it! A short note indeed, but hopefully it was usefull...</h3> <br> <br> <br> <br> </body> </html>