Archive for the ‘bash’ Category

Two ways of generating white noise

Wednesday, January 14th, 2009

Goal: generate a white noise .wav fille of 1.5 hours
In matlab:


% create a matrix with random values:
>> noise = 2 * rand(90 * 60 * 44100, 2, 'double') - 1;

% write the wav file with a sample frequency o 44.1kHz
>> wavwrite(noise, 44100, 16, 'noise.wav');

This process takes a while and it consumes quite some of your system RAM.

More efficient, in bash:

$ dd if=/dev/urandom count=5400 bs=176400 | sox -s -c 2 -r 44100 -t raw -w - noise.wav

This took only 2 minutes!

Remove duplicate files

Wednesday, December 3rd, 2008

This is a slightly modified version of the script published here. It allows you to scan files for duplicates based on md5 checksums.

#!/bin/bash
# rd - remove dupliactes

# find the files using the specified 'find arguments'
find "$@" -type f -print0 |

# calculate checksum for each file
xargs -0 -n1 md5sum |

# sort on the checksum
sort --key=1,32 |

# show remove command for each duplicate file
awk 'dup[$1]++{print "rm -f " $2}'

exit 0

The script is safe to use, it is not able to actually delete files itself. Instead, it generates a script that does the risky stuff.

Usage
To see what files are marked as duplicate in the current working directory:

$ rd .
rm -f ./config_backup_2008-11-06_11.30.01.tar.bz2
rm -f ./config_backup_2008-11-07_11.30.01.tar.bz2
rm -f ./config_backup_2008-11-08_11.30.01.tar.bz2

If you like the result, you can execute the generated commands. This can by piping the output to the shell:

$ rd . | sh

Processing the rd command might take some time. So you can also copy and paste the output in the terminal when there are a lot of (big) files.
Since the script passes all arguments to the find command. It’s also possible to fine tune the find command. For example, you only want to remove duplicates in the current directory, without searching in sub directories:

#  rd . -maxdepth 1

I’m using the script to remove duplicate backup sets.

vmcp

Monday, December 10th, 2007

Vmcp (vm-copy) is a simple script that allows you to copy/clone a VMware virtual computer image. Just specify the source directory and the destination directory. It will first just copy the wmware directory recursively en then change the configuration option ‘displayName’ in the vmx file.

#!/bin/sh

source=$1
dest=$2

echo copying virtual machine
cp -rv $source $dest

vmxsourcefile=`ls $source/*.vmx`
vmxdestfile=`ls $dest/*.vmx`

grep -v displayName $vmxsourcefile > $vmxdestfile
echo displayName = \"$dest\" >> $vmxdestfile

exit 0

Preparing plogger thumbnails

Friday, October 12th, 2007

I recently started to use Plogger to publish family pictures. The photo blogger package is nice, but still in beta stage. The latter doesn’t go unnoticed when setting it up.

One of the things that has been bothering me is the fact that thumbnails are not only generated during import, but also during the first visit of the new albums. Especially on my very slow web host, it can take up to minutes to view a single picture that hasn’t been processed before.

A solution is a shell script that finds out what pages to download after adding a new set of pictures. It will then download those pages, forcing Plogger to generate all the necessary cache files. The script should be executed form the web server itself. It requires 3 lines (upper part of the script) of configuration: the url and filesystem path to the plogger installation and a log file that keeps track of the url’s already downloaded. If you want to redo the downloading, just remove this log file.

Reminder: Local documentation!

Friday, September 21st, 2007

I’ve been searching for some ‘getopt’ examples on the internet. Many tutorials show the same basic tricks. I completely looked over the local documentation, which in fact has very nice examles for bash and tcsh:

#!/bin/bash

# A small example program for using the new getopt(1) program.
# This program will only work with bash(1)
# An similar program using the tcsh(1) script language can be found
# as parse.tcsh

# Example input and output (from the bash prompt):
# ./parse.bash -a par1 'another arg' --c-long 'wow!*\?' -cmore -b " very long "
# Option a
# Option c, no argument
# Option c, argument `more'
# Option b, argument ` very long '
# Remaining arguments:
# --> `par1'
# --> `another arg'
# --> `wow!*\?'

# Note that we use `"$@"' to let each command-line parameter expand to a
# separate word. The quotes around `$@' are essential!
# We need TEMP as the `eval set --' would nuke the return value of getopt.
TEMP=`getopt -o ab:c:: --long a-long,b-long:,c-long:: \
     -n 'example.bash' -- "$@"`

if [ $? != 0 ] ; then echo "Terminating..." >&2 ; exit 1 ; fi

# Note the quotes around `$TEMP': they are essential!
eval set -- "$TEMP"

while true ; do
        case "$1" in
                -a|--a-long) echo "Option a" ; shift ;;
                -b|--b-long) echo "Option b, argument \`$2'" ; shift 2 ;;
                -c|--c-long)
                        # c has an optional argument. As we are in quoted mode,
                        # an empty parameter will be generated if its optional
                        # argument is not found.
                        case "$2" in
                                "") echo "Option c, no argument"; shift 2 ;;
                                *)  echo "Option c, argument \`$2'" ; shift 2 ;;
                        esac ;;
                --) shift ; break ;;
                *) echo "Internal error!" ; exit 1 ;;
        esac
done
echo "Remaining arguments:"
for arg do echo '--> '"\`$arg'" ; done

MySQL re-installation

Saturday, August 4th, 2007

De-installation is sometimes not so easy with APT. For example when you screwed up your mysql installation or lost mysql passwords you may want to reinstall mysql and start from scratch again. You probably installed mysql server using:

$ sudo apt-get install mysql-server

Then you might think that you need this command for removal:

$ sudo apt-get remove mysql-server

However, this is not the case. The mysql-server package is a sort of meta-package that makes sure that the latest (and greatest) available mysql server version will be installed. Currently, this means that you actually installed mysql-server version 5.0. Above command will only remove the meta-package, while mysql server 5 happily continues running on your system. To de-install the mysql-server, remove the package, the meta package pulled in:

$ sudo apt-get remove mysql-server-5.0

To also remove the configuration and the binary data, use instead:

$ sudo apt-get remove --purge mysql-server-5.0
# and for ubuntu Gutsy:
$ sudo apt-get purge mysql-server-5.0

Now that mysql server is ‘really’ gone, you can start over and install the mysql-server package:

$ sudo apt-get install mysql-server

‘Compact’ bash prompt

Monday, July 30th, 2007

Especially when working with cakephp, the following line in your .bash_profile script might come in handy:

HOSTCOLOR="31m"
PS1="\u@\[\e[${HOSTCOLOR}\]\h\[\e[0m\]:\w\n\[\e[33m\]\!\[\e[0m\] \$ "

It will print out the current working directory on the first line and the actual bash prompt on the second line. Making it less likely that your command will wrap lines after the first few typed characters.
It changes the command prompt from:

command line long

….into:

command line 'short'

The login prompt now shows the following info:

  • username
  • hostname in red (in my bash script, it will print red for one group of servers and blue for other servers)
  • current working directory
  • command history number, ie: use !520 to execute the previous command

Extracting layers from a XCF (gimp) file

Saturday, July 21st, 2007

This is a script that extracts the layers of a XCF file to individual PNG files. It uses xcftools (debian/ubuntu package):

#!/bin/bash

xcffile=$1

layers=`xcfinfo $xcffile  | awk '{ print $NF }'`

destdir=`dirname $xcffile`/layers
mkdir $destdir

for layer in $layers
do
        echo saving $layer to $destdir/$layer.png
        xcf2png -o $destdir/$layer.png $xcffile $layer
done

exit 0

Batch conversion of images

Friday, December 15th, 2006

For batch image conversion, file renaming, etc, I usually use these kind of one-liners:

$ for i in `ls *.bmp` ; do echo convert $i ${i%.*}.png ;done
convert screendump_01.bmp screendump_01.png
convert screendump_02.bmp screendump_02.png
convert screendump_03.bmp screendump_03.png

When you’re satisfied with the result, have it executed by the shell:

$ for i in `ls *.bmp` ; do echo convert $i ${i%.*}.png ;done | sh

Scripting the Unscriptable

Wednesday, October 11th, 2006

I’ve been looking for different ways to send keystrokes, mouse movements and clicks to GUI applications. Under Windows, this is straight forward with AutoIt. The BASIC-like language allows you to read files, start applications, send keystrokes move the mouse etc. A nice feature is to wait for certain dialogs, for example:

; wait for tightvnc window
WinWaitActive("New TightVNC Connection")
; send hostname and port number
Send($hostname & ":" & $displaynumber)

This makes sure that the keystrokes actually end up in the right window. Mouse coordinates can be specified relative to the application window. Using these features allows you to write ‘reasonably’ robust code. But don’t expect your AutoIt script to run flawlessly on other PC’s. There are a lot of dependencies, like dialogs that look different in another version. Internationalization might cause the window title to read “Nieuwe TightVNC Verbinding” (Dutch) and the above code snippet will fail. Even user preferences like font sizes, window themes and screen resolution might result in unexpected behavior.

Anyway, it’s just fun to figure out which sequences to send to an application and let it process a batch of documents!
Ok, this works pretty nice under windows. But what about my favorite platform?
(more…)