androidcodeexamples.com
Offcode ltd logo
ANDROID
APACHE
THUMBNAILS
SCRIPTS
GFX
World-class examples brought to you by Offcode ltd
APACHE
Getting started Upload to server Image server Image manipulation Image cache reCAPTCHA Front Page
androidcodeexamples.com

Apache shell script: image cache

Image cache is useful if many requests are being made to an image that needs some tuning. Whether it's resized, compressed more aggressively or manipulated somehow and then accessed, it's desirable that such conversion operation is not made every time the user refers to it.

Storing images in a single directory is dangerous in the long-term. Even your most successful service will be in deep trouble when the directory contains more than 64000+ entries! We've coded the names into 2 level directory hierarchy. It means that the first letter and the second one will actually point into the directory where the real image is stored. The first/second characters better be quite random. Perhaps these could be picked from the middle of the name, or they could be a few letter hash created from the name, similar to a CRC or something. It's pointless to create subdirectories if all items end up being in a few directories. A sample directory tree looks like below, see the hex notation:

imageserviceagent@server888:~/imagestorage/47$ ls
2d  43  46  49  4d  50  53  57  62  67  6b  6e  72  75
33  44  47  4b  4e  51  55  59  63  68  6c  6f  73  77
41  45  48  4c  4f  52  56  61  65  69  6d  70  74  79

This simple example shows how an entry may be made out of the parameter(s) sent to the server. In the first code snipplet, the system checks if this image already exists. It uses the 'xxd' to generate a few strings that form the path for the image:

cd /home/imageserviceagent/ 2>/dev/null > /dev/null

MYNAME=`/bin/echo "$QUERY_STRING" | /usr/bin/xxd -ps -c 100 | /usr/bin/head -1`
DIRA=`/bin/echo "$MYNAME" | /usr/bin/cut -b 1,2`
DIRB=`/bin/echo "$MYNAME" | /usr/bin/cut -b 3,4`

if [ "$DIRA" = "" ]; then
    /bin/echo "Error"
    /bin/echo ""
    exit
fi

if [ "$DIRB" = "" ]; then
    /bin/echo "Error"
    /bin/echo ""
    exit
fi

if [ -f $DIRA/$DIRB/$MYNAME.jpg ]; then
    DT=`stat -t -c %y $DIRA/$DIRB/$MYNAME.jpg`
    MODI=`date -R -u -d "$DT" | cut -d'+' -f1`
    LEN=`/usr/bin/stat -c %s $DIRA/$DIRB/$MYNAME.jpg`
    /bin/echo "Last-Modified: \"$MODI\"GMT"
    /bin/echo "Content-Length: $LEN"
    /bin/echo "Content-type: image/jpeg"
    /bin/echo ""
    /bin/cat $DIRA/$DIRB/$MYNAME.jpg
    exit
fi

The next code snipplet illustrates automatic directory tree handling. It converts a png image into a jpg and creates the directories if they don't exist. Eventually it returns the converted image back to the user:

if [ -f $FILEN.png ]; then
    /usr/bin/convert $FILEN.png -resize 10% $FILEN.jpg > /dev/null 2>/dev/null
    /bin/rm $FILEN.png 2>/dev/null >/dev/null
    if [ -d $DIRA ]; then
        if [ -d $DIRA/$DIRB ]; then
            cp $FILEN.jpg $DIRA/$DIRB/$MYNAME.jpg 2>/dev/null >/dev/null
        else
            /bin/mkdir $DIRA/$DIRB 2>/dev/null >/dev/null
            cp $FILEN.jpg $DIRA/$DIRB/$MYNAME.jpg 2>/dev/null >/dev/null
        fi
    else
        /bin/mkdir $DIRA 2>/dev/null >/dev/null
        /bin/mkdir $DIRA/$DIRB 2>/dev/null >/dev/null
        cp $FILEN.jpg $DIRA/$DIRB/$MYNAME.jpg 2>/dev/null >/dev/null
    fi
    LEN=`/usr/bin/stat -c %s $FILEN.jpg`
    /bin/echo "Content-Length: $LEN"
    /bin/echo "Content-type: image/jpeg"
    /bin/echo ""
    /bin/cat $FILEN.jpg
    /bin/rm $FILEN.jpg 2>/dev/null >/dev/null
    /bin/echo "$INFO" "$HTTP_REFERER" >> /home/imageserviceagent/infolog.log
    exit
fi

Image cache maintenance

If you let people from the net or apps to create, manipulate, modify etc images, it maybe perhaps a good idea to check in the script that if 'df' tells more than 80% of the storage is full, let no people upload more data.

It's always a good practise to delete old entries. Linux has very powerful tools for those. The script below should not be launched from the net, but should be among the server start-up scripts which run always. It removes more than day old .jpg images from the /daily directory and more than week old entries from the /weekly -directory:

#!/bin/bash

# androidcodeexamples.com

while :
do
    sleep 60
    find /home/imageserviceagent/daily/*/*/*.jpg -mtime +1 -exec rm -rf {} \; 2> /dev/null > /dev/null
    sleep 60
    find /home/imageserviceagent/weekly/*/*/*.jpg -mtime +7 -exec rm -rf {} \; 2> /dev/null > /dev/null
    sleep 3600
done