pantz.org banner
Expect examples and tips
Posted on 06-24-2012 21:42:19 UTC | Updated on 07-16-2018 23:51:27 UTC
Section: /software/expect/ | Permanent Link

Expect is an automation and testing tool used to automate a process that receives interactive commands. If you can connect to any machine using ssh, telnet, ftp, etc then you can automate the process with an expect script. This works even with local programs where you would have to interact with a script or program on the command line.

When I first starting using Expect I wanted a short but detailed description on how Expect performed its magic. I had to read the Expect O'Reilly book to gather all of the information I needed, but that was not very short. I'm going to give my shortened version of how it works, and answer the questions I had at the time I was learning it.

Installing and getting started

Expect is written as a Tcl extension and can be installed on almost any Unix type distribution. The package on most distributions should just be called "expect" so look for that when installing it. For example on Linux distros that use apt-get to install packages just issue "sudo apt-get install expect".

After Expect is installed find where it was installed to by issuing the command "which expect". This should give you the path to the binary. If not try becoming root and issuing the same command. We need to know the path of the binary as it is the first line of our script. Note this path as we will be needing it soon.

How expect works

Expect scripts are written like many other scripts. Like Bash or Perl the binary is called at the top "#!/usr/bin/expect" and the scripts commands below it. Expect is an extention to the Tcl scripting language so it uses all of Tcl's syntax.

Most scripts begin by spawning the process they want to interact with. This is done with the "spawn" command. Like "spawn ssh user@host or spawn ftp host". Then you can begin checking for output with the command "expect". Expect uses regular expressions to find patterns in the output, and when it matches a pattern you send it command with the "send" command. In the simplest form that is how expect works. You start the process and look for patterns in the output. Then send commands based on the matches from the output.

Most used commands and descriptions

Below I will list the commands (with descriptions) I use the most in Expect. They should get you through the simple scripts that most people need.

Expect scripts design

Before we get to actual Expect script examples let me lay out how I design my scripts and what I have learned works best for me.

  1. I love giving good feedback when an error condition is reached. To do this with expect try to use the eof and timeout keywords, or default pattern keyword in each expect statement. If you set a timeout and eof message and action in each expect block you can use a send_user message and tell the user what part of the script failed or if the process exited. If you use exit as your action for the keywords then you can be sure the script does not go any further. This helps make debugging what went wrong faster at the expense of a few extra lines per statement.I think its worth it.
  2. I try not to use the while loops with the expect command. I see people do this a lot and many times it is not needed. The expect command itself is a loop. It will keep looping through the output looking for a match. You can act on any matches and keep performing more matches and actions. Expect has a default timeout of 10 secs when looping through looking for a match. If there is never a match it will timeout and you can set an action to this timeout if you want.
  3. To set your Expect scripts apart from other scripts use the file extension .exp
  4. Always try to follow a send command with an expect statement if possible. This helps first with timing where the next command will not be executed until that last one has been completed and checked with the expect statement. You don't need sleep statements if you have the luxury of knowing what output you should be seeing. You can be sure the command completed correctly if you know what should be there when it is finished.

Example #1.

The first example will be logging into a linux box with ssh to see if the given accounts password works. By doing this we are also checking other things like if your machine is up and if you can login to it and get a prompt back. So in actuality this script does a bunch of things.

#!/usr/bin/expect
set timeout 9
set username [lindex $argv 0]
set password [lindex $argv 1]
set hostname [lindex $argv 2]
log_user 0

if {[llength $argv] == 0} {
  send_user "Usage: scriptname username \'password\' hostname\n"
  exit 1
}

send_user "\n#####\n# $hostname\n#####\n"

spawn ssh -q -o StrictHostKeyChecking=no $username@$hostname

expect {
  timeout { send_user "\nFailed to get password prompt\n"; exit 1 }
  eof { send_user "\nSSH failure for $hostname\n"; exit 1 }
  "*assword"
}

send "$password\r"

expect {
  timeout { send_user "\nLogin failed. Password incorrect.\n"; exit 1}
  "*\$ "
}

send_user "\nPassword is correct\n"
send "exit\r"
close

Let me describe what is going on in this script line by line.

Line 1 executes the start of the script with the path to the expect binary.

Line 2 sets a timeout for each expect statement to 9 seconds.

Line 3,4,5 set variables for username, password, and hostname which are taken from the command line for when the script is run.

Line 6 turns off the output to STDOUT (printing the output to the screen). If we remove this line you will see the whole login process.

Line 8 is the first if statement. It checks to see if any arguments have been given to the script and if not it will print out how to use the script. If you do this with every script you will always be able to remind yourself of how to use it.

Line 13 prints a banner with the hosts name.

Line 15 starts the ssh process and turns on quiet mode and turns off host key checking (only turn off key checking on a trusted network). It uses the variables we give it on the command line to ssh to the host.

Line 17 is our first expect statement. First we set our timeout message in case our expect statement can not find a match for what its looking for. According to our timeout we set this will occur in 9 seconds. This means it will loop through the ssh output for 9 seconds and if it can not find a match it will print this error an then exit as we have told it to do. We also set our EOF value here. If our ssh session does not connect or gets disconnected EOF will be returned and our error message will print. This will then exit like we tell it to do. The last and most important part of this statement is line 20 the pattern to match which is "*assword". This is a regular expression that looks for 0 or more characters with the letters assword after it. Many password login prompts for ssh show up as "Password: " so this should match that. If it does match then it moves to the next statement. If it does not match we will hit the timeout, then our error message will print and the script will exit.

Line 23 will actually send the password we specified on the command line to ssh. We know we can send the password now because we verified we had a matching password prompt.

Line 25 starts our next expect statement. We set the timeout error message to say that the login must be incorrect. Why? Because the matching loop has started again looking for output. This time we are looking for a command prompt. If we can not match a command prompt in the output then that means we never got one. That means our password must have failed. Line 27 after the timeout line is the matching expression "*\$ ". The prompt I was matching for looked like this "user@hostname:~$ ". So the match says 0 or more of any character (*) and then a dollar sign and then a space. Notice the dollar sign has a \ before it. We have to escape this because it is a special character for regular expressions which means end of line match. If we don't put the \ before the $ it will still work, but the match is to broad and matches anything. We want it to be a specific match to be sure it is a correct prompt so we want to look for any character with a $ and a space. Your prompts are likely to vary so change your matching expression to meet your needs.

Line 30 sends output to the screen to tell the person the we got a correct prompt back. We can send this because we know we successfully got this far and our prompt matched from the previous expect statement.

Line 31 and 32. Line 31 sends the exit command to the linux machine. Line 32 then closes the connection to the ssh process. That's it!

This script uses command line arguments for a reason. If you can build your scripts like this then you can use it to check many different accounts across many different hosts. It does not lock you into one host and account per script. If you have a list of hosts you can use it in a bash loop and check many machines at once. Like: for x in server1 server2; do ./script.exp opt1 opt2 $x; done

Notice we have to put the password on the command line with this script. First off make sure you put your passwords in single quotes. That removes any issue with special characters being interpreted by your shell. Second, is a bad idea to use password prompts on the command line because many shells (like bash) keep a history of your commands (like ~/.bash_history). That means your password is sitting in a file in plaintext in your home dir. That's bad. To help mitigate this with bash you can put the following line in your ~/.bashrc file or ~/.profile. "export HISTCONTROL=ignorespace" . This will then execute on login and after it is run will allow you to put a space before any command and it will not be kept in your history. The password will still be on your screen after you have run it but usually this is scrolled off soon after the script is run. Typing "clear" will scroll it off the current screen, but just keep it in mind.

Example #2

This is a more complicated example that has to deal with different situations depending on the device type. The following script will SSH into 2 different types of Cisco routers and switches and change the SNMP password. The reason this is more complicated is that the 2 different types of devices uses 2 different operating systems with some similar and some different commands.

#!/usr/bin/expect
set timeout 15
set hostname [lindex $argv 0]
set snmpshapass [lindex $argv 1]
set snmpaespass [lindex $argv 2]
set username [lindex $argv 3]
set password [lindex $argv 4]
set enable [lindex $argv 5]
set send_slow {10 .001}
log_user 0

if {[llength $argv] == 0} {
  send_user "Usage: scriptname hostname \'snmpshapass\' \'snmpaespass\' username \'userpassword\' \'enablepassword\'\n"
  send_user "For Cisco Nexus devices just give hostname snmpshapass and snmpaespass if you have ssh keys installed\n"
  exit 1
}

send_user "\n#####\n# $hostname\n#####\n"

if { [info exists $username] } { 
  spawn ssh -q -o StrictHostKeyChecking=no $username@$hostname
} else {
  spawn ssh -q -o StrictHostKeyChecking=no $hostname
}

expect {
  timeout { send_user "\nFailed to get password prompt\n"; exit 1 }
  eof { send_user "\nSSH failure for $hostname\n"; exit 1 }
  "*#" {}
  "*assword:" {
    send "$password\r"
  }
}

send "\r"

expect {
  default { send_user "\nCould not get into enabled mode. Password problem?\n"; exit 1 }
  "*#" {}
  "*>" {
    send "enable\r"
    expect "*assword"
    send "$enable\r"
    expect "*#"
  }
}

send "show ver | inc Cisco\r"

expect {
  default { send_user "\nFailed to determine OS or get back correct prompt while changing pass.\n"; exit 1 }
  "Nexus" {
    send "config t\r"
    expect "*(config)#"
    send snmp-server user snmpUser network-operator auth sha $snmpshapass priv AES-128 $snmpaespass\r"
    expect "*(config)#"
    send "exit\r"
    expect "*#"
    send "copy run start\r"
    expect "100%"
    expect "*#"
  }
  "IOS" {
    send "config t\r"
    expect "*(config)#"
    send snmp-server user snmpUser snmpGroup v3 auth sha $snmpshapass priv AES 128 $snmpaespass\r"
    expect "*(config)#"
    send "exit\r"
    expect "*#"
    send "write mem\r"
    expect "*#"
  }
}

send "exit\r"
send_user "\nSuccessfully changed SNMP password on $hostname\n"
close

I will not go through this script line by line like it did the first one but I will describe what it is doing at some key parts. You notice this script checks for a specific command line option on line 19. If the username variable is set then it will execute the ssh line with a username. If no username is given then it assumes your using ssh keys and uses the username you logged in with. Nexus devices allow ssh keys per user, IOS does not.

After starting the ssh session we have to make sure we are logged and in enabled mode. Cisco and Nexus devices show your in enabled mode with the prompt matching "*#". When we login to a Nexus device with ssh keys we are already at that prompt, this is not true for IOS devices or Nexus devices that do not use keys. Line 25 is our first expect statement and first checks to see if have the correct prompt "*#". If so then nothing is executed and the expect statement is short circuited with {}. If we don't match that prompt expect moves onto the second match line "*assword". That will match an IOS device or a Nexus device that is not configured with ssh keys. It will then give the correct password. If that fails we get the error message.

On line 36 we use expect again to see if we in the correct mode by checking for the prompt matching "*#". For a Nexus device with ssh keys setup we are already at that prompt so we short circuit again and move on to the next area. If there is no match we move to looking for the correct IOS prompt "*>". If we find it then we issue an enable command and give the password to finally get to the prompt we have been looking for that matches "*#". We check for that pattern one last time. if it matches then we move onto the next block.

On line 47 we send the command "show ver | inc Cisco\r". This command works for both devices and will give us a line that we can match on that shows us which operating system we are using. It will either say Nexus in the line or IOS in the line. We will use this output in our next expect statement

On line 49 we start our expect statement and check for one of 2 OS types. Either IOS or Nexus. When one matches we enter configuration mode and run the command to change the SNMP password for that device type. After the command is run we write our configuration to memory and make sure we get back to prompt successfully. The save command takes a few seconds to run. By making sure we get back to a prompt after we issue the save command it assures we have given the save command the right amount of time to run successfully. It is possible there could be an error in the save, but right now we are not checking for that.

Then we finally exit out of the router or switch. If we got this far and did not hit any exit timeouts we set then it is very likely this worked, so we send our success message. Then close our process down.

Tips and Gotchas

Del.icio.us! | Digg Me! | Reddit!

Related stories


RSS Feed RSS feed logo

About


3com

3ware

alsa

alsactl

alsamixer

amd

android

apache

areca

arm

ati

auditd

awk

badblocks

bash

bind

bios

bonnie

cable

carp

cat5

cdrom

cellphone

centos

chart

chrome

cifs

cisco

cloudera

comcast

commands

comodo

compiz-fusion

corsair

cpufreq

cpufrequtils

cpuspeed

cron

crontab

crossover

cu

cups

cvs

database

dbus

dd

dd_rescue

ddclient

debian

decimal

dhclient

dhcp

diagnostic

diskexplorer

disks

dkim

dns

dos

dovecot

drac

dsniff

dvdauthor

e-mail

echo

editor

emerald

ethernet

expect

ext3

ext4

fat32

fedora

fetchmail

fiber

filesystems

firefox

firewall

flac

flexlm

floppy

flowtools

fonts

format

freebsd

ftp

gdm

gmail

gnome

google

greasemonkey

greylisting

growisofs

grub

hacking

hadoop

harddrive

hba

hex

hfsc

html

html5

http

https

hulu

idl

ie

ilo

intel

ios

iperf

ipmi

iptables

ipv6

irix

javascript

kde

kernel

kickstart

kmail

kprinter

krecord

kubuntu

kvm

lame

ldap

linux

logfile

lp

lpq

lpr

maradns

matlab

memory

mencoder

mhdd

mkinitrd

mkisofs

moinmoin

motherboard

mouse

movemail

mplayer

multitail

mutt

myodbc

mysql

mythtv

nagios

nameserver

netflix

netflow

nginx

nic

ntfs

ntp

nvidia

odbc

openbsd

openntpd

openoffice

openssh

openssl

openvpn

opteron

parted

partimage

patch

perl

pf

pfflowd

pfsync

photorec

php

pop3

pop3s

ports

postfix

power

procmail

proftpd

proxy

pulseaudio

putty

pxe

python

qemu

r-studio

raid

recovery

redhat

router

rpc

rsync

ruby

saltstack

samba

schedule

screen

scsi

seagate

seatools

sed

sendmail

sgi

shell

siw

smtp

snort

solaris

soundcard

sox

spam

spamd

spf

sql

sqlite

squid

srs

ssh

ssh.com

ssl

su

subnet

subversion

sudo

sun

supermicro

switches

symbols

syslinux

syslog

systemrescuecd

t1

tcpip

tcpwrappers

telnet

terminal

testdisk

tftp

thttpd

thunderbird

timezone

ting

tls

tools

tr

trac

tuning

tunnel

ubuntu

unbound

vi

vpn

wget

wiki

windows

windowsxp

wireless

wpa_supplicant

x

xauth

xfree86

xfs

xinearama

xmms

youtube

zdump

zeromq

zic

zlib