This sounds super easy. Just echoing some lines into root owned file using using sudo. You figure something like "sudo echo blah >> /tmp/test" would work fine. You would be wrong. Let me demonstrate. You need sudo access to the root user to do this. Here is an example of the problem:
user@host$ touch /tmp/test user@host$ sudo chown root: /tmp/test user@host$ ls -la /tmp/test -rw-r--r-- 1 root root 0 2011-07-16 14:34 /tmp/test user@host$ sudo echo blah >> /tmp/test bash: /tmp/test: Permission denied
The echo program is running as root because we used sudo, but the shell that's redirecting echo's output to the root owned file is still running as the unprivileged user. The current shell does the redirection before sudo starts. To fix this we need to run the whole pipeline under sudo.
user@host$ echo "echo blah >> /tmp/test" | sudo bash user@host$ cat /tmp/bash blah
Doing this as an ssh command to another machine adds some more complication but to get it to work use the following command. This works best if you have ssh keys setup and NOPASSWD access to sudo to root. On machines with remote root login disabled you are forced to use sudo to perform ugly commands like the following.
ssh user@host 'sudo su root -c "echo \"echo test >> /tmp/test\" | sudo bash"'
The above command uses "sudo su root" because of an issue I saw with OpenBSD. If you did not put in the root user then you would get the error "su: no such login class:". Putting in the root user fixes this issue.
If you have more than one line to execute on a remote server then I would suggest just putting all of the commands you want to execute into a script and copy the script to the machine with scp and then executing with another ssh command. This saves you from executing that nasty ssh line above and writing a whole script filled with ssh commands.
So Adobe decided to change the way it uses temp files in Flash for its newer 10.2r152 version in Linux. Before when you went to YouTube and watched a video it would temporarily put the video in your /tmp dir in the format /tmp/FlashXX*. It stayed there while you kept the webpage open. You could then copy the video from /tmp to wherever for your own use. Now the temp file is nowhere to be found. To take its place how about a shell oneliner to download a YouTube video from the command line?
Here's a oneliner bash script that uses wget, sed, awk, and tr to download a video from YouTube. Works on BSD, Linux, and Mac. Just put the code below in a file (I'll call mine ytd.sh) and make your file executable chmod +x ytd.sh.
#!/bin/bash
## Get Youtube Video using wget
## (compatible with Linux, BSD and OSX)
#
## Two arguments
# $1 Youtube URL
# $2 You name for the video
#
wget -c --no-check-certificate --user-agent="" $1 -qO- | \
sed 's/\\u0026/\&/g'| \
awk '/fmt_url_map/{gsub(/[\|\"]/,"\n");print}' | \
sed -n "/^fmt_url_map/,/videoplayback/p" | \
sed -e :a -e '$q;N;2,$D;ba' | tr -d '\n' | \
sed -e "s/\(.*\),\(.\)\{1,3\}/\1/;s/\\\//g" | \
wget -c --no-check-certificate --user-agent="" -i - -O $2.flv
After putting in in a file and making it executable just go get your YouTube url and fire off the download. Here's an example.
./ytd.sh 'http://www.youtube.com/watch?v=J---aiyznGQ' keycat
This will make the file keycat.flv in your dir. Use a recent VLC or Mplayer to play the file.
Its very likely that this script will be out of date (broken) soon but hopefully it will be a good blueprint for anyone else in the future. Thanks to our friends at calomel.org for the fun code!
Below are one off lines that perform different tasks at a command prompt. The commands can use any number of programs usually found on some type of Unix machine. Things like awk or perl.
Warning: These lines were tested on Linux with a bash shell using the GNU versions of each piece of software. Your mileage may vary with your version of *nix and your shell. The different programs (mostly GNU) used here may be different versions or not the GNU version on your system. This means they may work differently or use (or not have) different command line options. If need be check your man pages to adapt the commands below to your OS.
Any awk statements below assume the awk defaults which is spiting fields on spaces unless specified with -F. If using the awk statement and you want to work on a file instead of stdin then put a filename at the end of the awk statement. FYI: Awk refers to lines from files as "Records" and each "Record" has "fields". "Fields" are sections of characters (words) with delimiters between them.
Print the total number of lines that have the name Bill.
awk '/Bill/{n++}; END {print n+0}' filename
Print line numbers using a tab instead of a space.
awk '{print FNR "\t" $0}' filename
Use awk to pull the seventh field (ex. url string) of each line of the logfile (logfile should be separated by spaces). Sort the fields then find the unique fields and count them. Then do a reverse sort on numeric count. Filter out anything but JPEG files and only give me the top 10 of that list. This is for things like counting unique hostnames or urls from a logfile.
awk '{print $7}' logfile | sort | uniq -c | sort -rn | grep "\.jpg" | head
Print the last field of each line of the file.
awk '{ print $NF }' filename
Count the lines in a file. Just like "wc -l".
awk 'END{print NR}' filename
Total the number of lines that contain the name Jose
awk '/Jose/{n++}; END {print n+0}' filename
Add up the numbers at the eighth position (field) of each line. Print the total.
awk '{ sum += $8 } END { print sum }' filename
If the fourth field starts with a number then print that fourth field.
awk '$4 ~ /^[0-9]/ { print $4 }' filename
Find and replace "dog" or "cat" or "bird" with "pet" on every line and print it out.
awk '{gsub(/dog|cat|bird,"pet");print}' filename
Find and replace "dog" with "cat" in every file with extension txt.
awk '{gsub("dog", "cat", $0); print > FILENAME}' *.txt
Find every line that begins with cat. In that line replace furry with nothing. Change the file called filename inline (-i).
sed -i '/^cat/{s/furry//}' filename
Find cat by itself on it's own line even if there are spaces or tabs before it or after it. Replace it with dog. Then print the line.
awk '{sub(/^[ \t]*cat .*$/,"dog");print}' filename
Find any line starting with the defined shell variable SHELLVAR (notice ' ' around it so it's evaluated). When the line is found substitute in foo or boo or coo with bar. Edit the file inline.
sed -i '/^'${SHELLVAR}'/s/\(foo\|boo\|coo\)/bar/' filename
From a unix os: convert DOS newlines (CR/LF) (removes the ^M) to Unix format. Works when each line ends with ^M (Ctrl-M).
awk '{sub(/\r$/,"");print}' filename
Remove all carriage returns from file and rewrite the edits back to same file. tr uses the octal form for cr.
tr -d "\015" < filename | tee > filename
From a unix os: Convert Unix newlines (LF) to DOS format.
awk '{sub(/$/,"\r");print} filename
Delete the leading whitespace (spaces or tabs) from front of each line. Text will end up flush left.
awk '{sub(/^[ \t]+/, ""); print}' filename
Delete the trailing whitespace (spaces or tabs) from end of each line.
awk '{sub(/[ \t]+$/, "");print}' filename
Delete leading and trailing whitespace from each line.
awk '{gsub(/^[ \t]+|[ \t]+$/,"");print}' filename
Delete the trailing whitespace (spaces or tabs) from end of each line.
awk '{sub(/[ \t]+$/, "");print}' filename
Insert 6 blank spaces at beginning of each line.
awk '{sub(/^/, " ");print}' filename
Substitute "dog" with "cat" on lines that don't contain the word "pet".
awk '!/pet/{gsub(/dog/, "cat")};{print}' filename
Print the first 2 fields with a space between the output. Split the fields on the colon (field separator).
awk -F: '{print $1 " " $2}' filename
Swap the first 2 fields.
awk '{tmp = $1; $1 = $2; $2 = tmp; print}' filename
Remove the second field in each line and then print it.
awk '{ $1 = ""; print }' filename
Print the first 6 lines of a file.
awk 'NR <= 6' filename
Print the last line of a file
awk 'END{print}' filename
Print the lines matching the regular expression. (emulates grep).
awk '/regex_here/' filename
Print the lines that don't match the regular expression. (emulates grep -v).
awk '!/regex_here/' filename
Print the line before the regular expression match.
awk '/regex_here/{print x};{x=$0}' filename
Print the line after the regular expression match.
awk '/regex_here/{getline; print}' filename
Print the lines less than 50 characters.
awk 'length < 50' filename
Print the lines 20 through 30.
awk 'NR==20,NR==30' filename
Print the line 50.
awk 'NR==50 {print;exit}' filename
Print lines between the match starting at "Dog" and ending at "Cat".
awk '/Dog/,/Cat/' filename
Find a program by name from process listing that is not awk and kill it. Or try the programs pkill or killall.
ps aux | awk '/program_name/ && !/awk/ {print $2}' > kill
Create a 2 meg file (in 1 kilobyte blocks) in /tmp called zero. 1k can be changed to 1M for meg or 1G for gig in my version of dd.
dd if=/dev/zero of=/tmp/zero bs=1k count=2000
From the root dir (/) find all files with the .txt extention and delete them. Using xargs is faster than find's -exec.
find / -type f -name "*.txt" -print | xargs rm
To delete a file who's file name is a pain to define (ex. ^H^H^H) find it's inode number with the command "ls -il". Use the line below to find and delete a file who's (for example) inode number is 128128.
find . -inum 128128 | xargs rm
Mark the end of each line with a dollar sign so you can see where the filename ends. Good for finding file names with spaces at the end.
ls -lA | cat -e
To delete files with control characters in them like ^M or ^L use the control-V shell feature. This tells many shells to interpret the next input character as a literal character (instead of as a control character). Like below to delete a file with a space before the ctrl-L " ^L" you would issue the following keystrokes in this order (separated by commas) r,m, ,\, ,ctrl-v,ctrl-l. The \ escapes the space. The command looks like:
rm \ ^L
Synchronize files in a directory between 2 hosts using the program rsync. host1's /disk01 (source) is the remote host and /disk01 (destination) is a local directory. The destination is always made to look like the source even if files need to be deleted in the destination (--delete). The source's data is never touched. The source is always named first and the destination is always named second. Trailing / on the source as means copy the contents of this directory to the destination. Without the trailing / on the source you get the directory name copied with all it's files in it. Below uses ssh as the remote-shell program as the transport. It also turns on the lowest grade encryption to speed up the transfer.
rsync -av --delete --rsh="ssh -c arcfour -l root" host1.domain.lan:/disk01/ /disk01/
Take a file with a list of hostnames and login via ssh and get disk usage. SSH keys will need to be setup for auto login. Each command is back grounded so all commands are executed one after another.
for HOST in $(< ListOfHosts); do ssh $HOST `df -h` & done
Set group id and sticky bits on a dir.
chmod g=swrx,+t DIR/
Use wget grab all of a certain type of file listed on a web page and put them in the current dir. The example will use jpeg's.
wget -r -l1 --no-parent -A "*.jpg" http://www.website.com/pictures/
