domenica 22 luglio 2012

How To implement indexes in Redis using Jedis driver

"Redis is an open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hasheslists, sets and sorted sets.

You can run atomic operations on these types, like appending to a string; incrementing the value in a hash; pushing to a list; computing set intersection, union and difference; or getting the member with highest ranking in a sorted set.
In order to achieve its outstanding performance, Redis works with an in-memory dataset. Depending on your use case, you can persist it either by dumping the dataset to disk every once in a while, or by appending each command to a log." [link]
Redis is simple to use and has powerful low level API, through which developers can implement complex tasks.
It is a key-value store: to search for something we must know the corresponding entry key.
However, sometimes is needed to retrive entries by value, for example when we want a subset of HASHES (representing POJOs) having the same value in a specific field. In those cases there is not a single command to do that, so a workaround is required.

My solution is to implement a sort of "index" of the field of interest, using SORTED SET type offered by Redis.
"Redis Sorted Sets are, similarly to Redis Sets, non repeating collections of Strings. The difference is that every member of a Sorted Set is associated with score, that is used in order to take the sorted set ordered, from the smallest to the greatest score. While members are unique, scores may be repeated."

In the following Java code, methods take a Jedis instance as parameter, used to communicate with Redis datastore.
An example of pojo class:

public class PojoClass {
 private Long id;
 private int number;
 private String text;
 /* getters and setters omitted */
}

Utility class contains methods to make id and pojoName from a pojo instance; a pojoId identify an unique istance of a pojo class in the datastore.

public class Utility {

 public static String getPojoName( Object pojo ) {
  return pojo.getClass().getName();
 }
 
 public static String getId( Object pojo ) {
  try { return ""+pojo.getClass().getField("id").get(pojo); }
  catch (Exception ignored) { }
  return null;
 }
 
 public static String getPojoId( Object pojo ) {
  return getPojoName(pojo)+"_"+getId(pojo);
 }

}

IndexManager methods:
  • calculateScore(Object value)
  • persistEntry(Object pojoInstance, String fieldName, Object fieldValue, Jedis jedis)
  • findEntries(String pojoName, String fieldName, Object fieldValue, Jedis jedis)
  • removeEntry(String remPojoId, String fieldName, Jedis jedis)


calculateScore() calculates corrispective double value of a primitive Java type value; if input parameter is an instance of String, return its hashcode (NB: it may not be unique).
public static double calculateScore(Object value) {
  Class<?> valueClass = value.getClass();
  Double score = 0.0;
  if ( valueClass.equals(Short.class)    ||
    valueClass.equals(Integer.class) ||
    valueClass.equals(Long.class)   ||
    valueClass.equals(Float.class)   ||
    valueClass.equals(Double.class)  ){
   score = Double.parseDouble(""+value);
  } else
  if ( valueClass.equals(Character.class)  ){
   score = ((Integer)value).doubleValue() ;
  } else
  if ( valueClass.equals(Boolean.class) ){
   score = ((Boolean)value)? 1.0 : 0.0;
  } else
  if ( valueClass.equals(Byte.class) ){
   score = ((Byte)value).doubleValue();
  }
  if ( valueClass.equals(String.class) ){
   score = Double.parseDouble(""+((String)value).hashCode());
  } 
  return score;
 }

Command ZADD must be used to insert "index" entries in the datastore:
public void persistEntry(String pojoName, String fieldName, Object fieldValue, Jedis jedis) {
  if (!value.getClass().equals(String.class)) {    
    // if String.class : value must be persisted in index entry
    jedis.zadd(pojoName+":"+fieldName, this.calculateScore(fieldValue), value);
  } else {      
    jedis.zadd(pojoName+":"+fieldName, this.calculateScore(fieldValue), "");
  }
}
findEntries() is useful to retrive entries with a specific field value:
public Set<String> findEntries(String pojoName, String fieldName, Object fieldValue, Jedis jedis) {
  String key = pojoName+":"+fieldName;
  double minmaxScore = this.calculateScore(fieldValue);
  Set<String >keys = new HashSet<String>();
    if (fieldValue instanceof String) {
      Set<String> keys_tmp = jedis.zrangeByScore(key, minmaxScore, minmaxScore);
      for (String k : keys_tmp) {
        String[] valueAndReferredEntity = k.split("\\$\\$\\$");
        if (valueAndReferredEntity[0].equals(""+fieldValue)) {
          keys.add(valueAndReferredEntity[1]);
        }
      }
    } else {
      keys = jedis.zrangeByScore(key, minmaxScore, minmaxScore);
    }
    return keys;
}

When an istance of a PojoClass is removed from the DB, its indexes must be updated:

public void removeEntry(String remId, String remPojoName, 
     String fieldName, Object fieldValue, Jedis jedis) {
  Set<String> keys = null;
  String key = remPojoName+":"+fieldName;
  double minmaxScore = Utility.calculateScore(fieldValue);
  keys = new HashSet<String>();
  if (fieldValue instanceof String) {
    keys = jedis.zrangeByScore(key, minmaxScore, minmaxScore);
    for (String k : keys) {
    String[] valueAndReferredEntity = k.split("\\$\\$\\$");
      if (valueAndReferredEntity[0].equals(""+fieldValue)) {
        jedis.zrem(key, k);
      }
    }
  } else {
    keys = jedis.zrangeByScore(key, minmaxScore, minmaxScore);
    for (String k : keys) {
      if (k.equals(""+remId)) { jedis.zrem(key, k); }
    }
  }
}

NB: Indexes should also be updated when indexed field values of a PojoInstance changes.


0 commenti: