Caching tips

Caching is all about keeping data closest to where it is needed. It is first and foremost a performance optimization, however you have to ask yourself a couple of questions about the data:

1. Is it read-only, read-mostly or write-mostly?
2. How often does the data change?
3. What is the tolerance in each case for data staleness?

Data that falls into the category of read-only/mostly, has a low rate of change and a reasonable tolerance for data staleness are ideal candidates for caching. A key performance metric is cache-hits versus cache-misses. Obviously, you want to look for cache-hits to get a good return on the investment. If you get few cache-hits and lots of cache-misses - be careful, you may need to rethink your caching strategy as you may have made performance worse rather than better.

For caching web-objects for a global web-site, edge caching providers such as Akamai are the way to go. This works well for static objects such as images, CSS, JS, HTML etc. It moves the content closer to the end user. Also, recall that every point across the network may have some degree of cache-ability (e.g. proxy servers, routers, browsers).

Ensure the HTTP cache-control headers are set appropriately to reduce trips back to the server when the version in the browser cache could be plenty fresh (enough)

For caching data on the app server that is sourced from some back-end service, database or EIS system, memcached or JBossCache work well. Clustering/cache replication may not always be required. Clustering is a good option if the data source is the end-user - if the data source is a database, you can recover the cache quite easily - if the data source is the end user (e.g. session information), you cannot very well ask them, "My cache seems to have gone down the toilet taking your session information with it - tell me again, how many shares of IBM did you want to buy again? (sorry)"

You can reduce a significant amount of load on back-end resources through caching - but only if the data is a good candidate for caching. Hibernate/eh-cache may work pretty well here in addition to memcached and JBossCache.

Oracle Coherence in my opinion has evolved way beyond it's caching roots. Using Coherence, you can build a huge data grid across commodity hardware. Each node in the grid has it's share of responsibilities and is backed up by n other nodes for redundancy. Operations against the data can move to where the data is rather than move the data to where the operation occurs (our traditional way of approaching data processing for the past gozillion years).

Tomcat Controller - updated

Update 12/4/08
Thanks to everyone's comments, we took a deeper look and found what we were looking for in JMX. We plan on leveraging a single JVM/Tomcat instance with seperate contexts. The Tomcat Admin web application has the MBeans we were looking for enabling us to dynamically add/configure data sources.

Original post
I have a requirement that calls for breaking the product I am designing into multiple JVMs (Tomcat instances to be precise). The original plan was to have a single Tomcat instance with multiple contexts however that plan had to be shelved for various reasons. One of the Tomcat instances is to support the Management GUI. To cut a long story short, the Management GUI is required to manage the life-cycle (i.e. stop/start) of another Tomcat instance containing the core engine of the product.

I explored several options for approaching this problem. My first attempt was to look at OSGi. I have been eager to explore OSGi for quite a while now so this gave me a perfect opportunity. I managed to knock out a couple of simple bundles and deploy them to Equinox. Next step, try and build a bundle out of Tomcat. Unfortunately, this didn’t quite go to plan. Tomcat seems to have an issue with authority – it wouldn’t play nicely in an OSGi world leaving me with a stack trace of exception I didn’t have the time or energy to track down. I did think about switching to Jetty at this point, but this would be a very disruptive change and one that our schedule couldn’t allow.

Moving on, I recalled that Eclipse WTP does a pretty good job in managing server definitions and controlling their lifecycles. I proceeded to download the source for WTP and found a bunch of code called Server Tools. I tried to reverse engineer the structure from the code but was quite astonished by the size of the code base. I figured it would be too much to weed through to get to what I was looking for. The Server Tools is pretty well structured code – it’s just overkill for my needs.

Next step, there has to be something in Ant I can use. I remember using the Ant Catalina tools from years ago. These are basically Ant tasks that expose the controls of Tomcat through the Manager application to Ant. The tasks didn’t really give me what I was looking for. Sure, they could be used to control individual contexts but didn’t really give me anything to control the server instance itself.

I was really looking to avoid Runtime.exec() if I could. There had to be something that worked at a higher level of abstraction. I continued to dig around Ant and recalled the Java task. This sounded promising. Of course, I really wanted to leverage the class behind the Java task itself and not deal with build.xml, launching Ant and all that malarchy. The Java task provides me with what I wanted. Here’s the prototype code I came up with called the Tomcat Controller. I think the Java task fits my needs perfectly:

TomcatController.java

import java.io.File;

import org.apache.tools.ant.Project;
import org.apache.tools.ant.taskdefs.Java;
import org.apache.tools.ant.types.Path;

public class TomcatController {

  public enum State {
    STARTING,
    STARTED,
    STOPPING,
    STOPPED
  };
 
  State state = State.STOPPED;
 
  State getState() {
    return state;
  }
 
  private TomcatServerConfiguration config;
 
  public TomcatController(TomcatServerConfiguration config) {
    this.config = config;
  }
 
  // Asynchronously start this Tomcat
  public synchronized void start() {
    state = State.STARTING;
    Java javaTask = createTomcatJavaTask();
    javaTask.createArg().setValue("start");
    for (String s : config.getJvmArgs())
      javaTask.createJvmarg().setValue(s);
    javaTask.setFork(true);
    javaTask.setSpawn(true);

    System.out.println(javaTask.getCommandLine());
    javaTask.execute();   
    state = State.STARTED;// Not quite true - Tomcat may still be starting at this point
  }
 
  // Block while this Tomcat is stopping 
  public synchronized void stop() {
    state = State.STOPPING;
    Java javaTask = createTomcatJavaTask();
    javaTask.createArg().setValue("stop");
    javaTask.setFork(true);
    javaTask.setSpawn(false);

    System.out.println(javaTask.getCommandLine());
    javaTask.execute();
    state = State.STOPPED;   
  }

  private Java createTomcatJavaTask() {
    Project project = new Project();
    project.setBaseDir(new File(config.getTomcatHome()+File.separator+"bin"));
    Java javaTask = new Java();
    javaTask.setProject(project);
    javaTask.setDir(new java.io.File(config.getTomcatHome()+File.separator+"bin"));
    javaTask.setTaskName("tomcat");
    javaTask.setClassname("org.apache.catalina.startup.Bootstrap");
   
    Path classpath = new Path(project);
    classpath.createPathElement().setPath(config.getTomcatHome()+File.separator+"bin");
    classpath.createPathElement().setPath(config.getTomcatHome()+File.separator+"bin"+File.separator+"bootstrap.jar");
    classpath.createPathElement().setPath(config.getTomcatHome()+File.separator+"bin"+File.separator+"commons-logging-api.jar");
    javaTask.setClasspath(classpath);
    javaTask.createJvmarg().setValue("-Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager");
    javaTask.createJvmarg().setValue("-Djava.util.logging.config.file="+config.getTomcatHome()+File.separator+"conf"+File.separator+"logging.properties");
    javaTask.createJvmarg().setValue("-Djava.endorsed.dirs="+config.getTomcatHome()+File.separator+"common"+File.separator+"endorsed");
    javaTask.createJvmarg().setValue("-Dcatalina.base="+config.getTomcatHome());
    javaTask.createJvmarg().setValue("-Dcatalina.home="+config.getTomcatHome());
    javaTask.createJvmarg().setValue("-Djava.io.tmpdir="+config.getTomcatHome()+File.separator+"temp");
    return javaTask;
  }
 
  public static void main(String[] args) throws InterruptedException {
   
    // Quick test - should really be in a JUnit test
    TomcatServerConfiguration config = new TomcatServerConfiguration();
    config.setName("Engine");
    config.setTomcatHome("c:\\apache-tomcat-5.5.26");
    config.addJvmArg("-Xmx512m");
    config.addJvmArg("-Dcom.sun.management.jmxremote");
    config.addJvmArg("-Dcom.sun.management.jmxremote.port=9128");
    config.addJvmArg("-Dcom.sun.management.jmxremote.ssl=false");
    config.addJvmArg("-Dcom.sun.management.jmxremote.authenticate=false");            
    TomcatController tc = new TomcatController(config);
    tc.start();
    Thread.currentThread().sleep(20000);
    tc.stop();
  }
}

TomcatServerConfiguration

import java.util.ArrayList;
import java.util.List;


public class TomcatServerConfiguration {

 private String name;
 private String tomcatHome;
 private List<String> jvmArgs = new ArrayList<String>();

 public TomcatServerConfiguration() {

 }
 public String getName() {
   return name;
 }
 public TomcatServerConfiguration setName(String name) {
   this.name = name;
   return this;
 }

 public String getTomcatHome() {
   return tomcatHome;
 }
 public TomcatServerConfiguration setTomcatHome(String tomcatHome) {
   this.tomcatHome = tomcatHome;
   return this;
 }

 public List<String> getJvmArgs() {
   return jvmArgs;
 }

 public void addJvmArg(String arg) {
   jvmArgs.add(arg);
 }

 public void clearJvmArgs() {
   jvmArgs.clear();
 }
}