History

Floodscatter was originally built years ago as an edge application to route requests away from core server banks. It was initially build as a proof of concent using PHP3. The prototype was online in a matter of 4 hours or so. At the same time I was slowing getting more into Java development and could see some advantages of writing this type of application as a Java servlet. It was re-written with a much larger feature set, and optimized to make use of the Java servlet containers resident capability. Popular cached objects were selected from the database and loaded into the container memory for faster serving without having to communicate with the database/ disk. Nowdays a similar implementation could be done with PHP + APC, or utilizing memcache which Google Apps supports and FloodScatter makes use of.

Who might want to use?

There are a number of different type of users who may find this application useful. First off people too cheap to pay for bandwidth. This application allows you to offload a lot of the constant requests for web objects (images/css/js/etc.). People with higher traffic sites, or sites that may soon become overwelmed because of a "flood" of traffic (eg. getting dugg).

Limits

Google currently allocates 500MB of storage, of which we try and share between clients, we allow a max of 5MB of files cached per account. This seems reasonable at this time. There is no doubt at some point Google will start raising all the limits. Also there is a 1MB limit for files which is enforced by Google.

How to use?

1) Signup for an account (it uses google accounts)
2) Create a site (you must specify the site domain)
3) Create a URL using our url format which tells FloodScatter how to handle certain things

URL Format

The URL format goes as follow:
http://floodscatter.appspot.com/cds/$1/$2/$3/$4

$1 = Site ID (it's displayed in your members area beside the site domain)
$2 = Cache Mode (1 = key based, 2 = time based)
$3 = Cache Mode Data (either a cache key, or expiry time in seconds)
$4 = URL of web object to cache

A sample URL is:
http://floodscatter.appspot.com/cds/1/1/610378193e693d9c41c7b6a6051b06c8/www.google.ca/intl/en_ca/images/logo.gif

This url is telling FloodScatter to use the siteid "1", that the cache mode is "key" based, and the key is "610378193e693d9c41c7b6a6051b06c8", the desired cached object is the google logo.

FloodScatter will continue to serve the exact same cached file (it will not re-request the file from the target server) until the key changes.

Optionally you could set the cache mode to "2", and the cache mode data to "3600" and FloodScatter will continue to serve the object for 1 hour, at that point it will flush/dump it.

** FYI Cache modes have not been fully implemented yet.. We are working to implement them **

Cache Modes - What mode is better?

That's tough, depending on the type of site your trying to implement it into, one may be a better choice than the other. I tend to use key based caching, as the dynamic website, when it creates the HTML on the fly, typically creates the key (I use a hash) of some kind of useful information. Such as the last modified date/time of the file. If the file source is modified, the modified date/time will change and a new hash key will be generated to force FloodScatter to re-request the file from the source server. Time based caching is pretty straight forward, if you dynamically generate any images on your site (such as graphs/ etc) every hour or 30 minutes, you can set the expiry similar to that so FloodScatter will continue refreshing at similar times as you generate new images. Cache keys would also work great in that situation.

How to implement?

I find that a lot of application developers typically like to build into their application a bit of "potential scalability" that is, everyone would like to think their webapp will be the next big thing, so why not think big as well. Thinking big is as easy as, let's have a configuration value for the image/js/css files. By default perhaps it's configured to: "www.yourgreatsite.com/images/", but it could easily be set to another server that's job is to serve static content. Perhaps using even a different webserver, optimized to serve static files. Apache without any modules, or perhaps lighttpd. I have also done implmentations where the application dynamically creates all the static file tags using a generator function. This function is programmed to look at the meta information about the file and generate the cache "key" from something unique about it, that may chance, but the function would catch the change right away, and hence modified files would quickly propogate down to FloodScatter.

Comments/ Features/ Issues

Have something you don't like, or would like to see, or isn't working? Let us know.