Friday, December 10, 2010

Remote Lazy Loading with Hibernate and Spring

The solution that I want to show in this blog entry solves the problem described in my previous entry for the environment I'm currently working in, which consists of various front-end and back-end frameworks. The important ones for this solution are already mentioned in the title and they are Hibernate and Spring.

During my search for a solution to this problem I found a lot of different posts and forum entries dedicated to the problem. Programmers seem to look for the solution, but either they gave up, or if they found a solution they were not willing to share it with the community (at least not the complete solution). Hopefully this post will fill the gap.

As mentioned already before, the solution to the problem is in any case the use of an aspect. Spring offers AOP functionality which allows to intersect method calls in every spring managed beans. For what we want in this case, this is not enough. We do not want to intersect spring managed beans, but hibernate managed entity objects.

The magic word in this case is "load-time-weaver". Spring offers the possibility to weave its aspects into every kind of objects when its load-time-weaving functionality is enabled.

That's basically all on prerequisites that you'll need, so lets see some code. Starting with the Aspect itself

@Aspect
@Component
@Configurable(autowire = Autowire.BY_TYPE)
public class OnDemandPreloader {


  @Autowired
  private PreloadService preloadService;


  @Pointcut("execution(public * path.to.model..*.get*(..))")
  public void possibleLazyInitializationException() {
  }


  @Around(value = "possibleLazyInitializationException()")
  public Object preloadOnDemand(ProceedingJoinPoint pjp) throws Throwable {
    Object result = pjp.proceed();
    if (result instanceof AbstractPersistentCollection) {
      // The object is either a map, a set or a list (basically @OneToMany or @ManyToMany)
      AbstractPersistentCollection collection = (AbstractPersistentCollection) result;
      if (!collection.wasInitialized() && collection.getSession() == null) {
        // Calling the getter to the object would lead to a LazyInitializationException, therfore reload it
        Object target = pjp.getTarget();
        String getterName = pjp.getSignature().getName();
        if (target instanceof BaseEntity) {
          BaseEntity entity = (BaseEntity) target;
          Object preloadedCollection = preloadService.preloadCollection(entity, getterName);
          setPreloadedResult(target, getterName, preloadedCollection);
          result = pjp.proceed();
        }
      }
    } else if (result instanceof BaseEntity) {
      // The Object is another BaseEntity
      BaseEntity bdo = (BaseEntity) result;
      if (bdo instanceof HibernateProxy && !Hibernate.isInitialized(bdo)
          && ((HibernateProxy) bdo).getHibernateLazyInitializer().getSession() == null) {
        // Calling the getter to the object would lead to a LazyInitializationException, therfore reload it
        Object target = pjp.getTarget();
        String getterName = pjp.getSignature().getName();
        if (target instanceof BaseEntity) {
          BaseEntity entity = (BaseEntity) target;
          Object preloadedObject = preloadService.preloadBaseEntity(entity, getterName);
          setPreloadedResult(target, getterName, preloadedObject);
          result = pjp.proceed();
        }
      }
    }
    return result;
  }


  private void setPreloadedResult(Object target, String getterName, Object preloadedContent) throws IllegalArgumentException {
    if (target == null || getterName == null || preloadedContent == null) {
      // "Something is null! Could not execute lazy loading"
    } else {
      String setterName = getterName.replaceFirst("get", "set");
      if (!setterName.startsWith("set")) {
        // "Setter doesn't start with 'set'"
      } else {
        try {
          Method getter = target.getClass().getMethod(getterName);
          Class getterType = (getter != null ? getter.getReturnType() : null);
          if (preloadedContent instanceof Collection) {
            setPreloadedCollection(target, setterName, (Collection) preloadedContent);
          } else if (preloadedContent instanceof Map) {
            setPreloadedMap(target, setterName, (Map) preloadedContent);
          } else if (preloadedContent instanceof BaseEntity) {
            setPreloadedBaseEntity(target, setterName, (BaseEntity) preloadedContent, getterType);
          }
        } catch (Exception e) {
          // Do something
      }
    }
  }


  private void setPreloadedBaseEntity(Object target, String setterName, BaseEntity preloadedContent, Class paramType) throws [..] {
      Method setter = target.getClass().getMethod(setterName, (paramType != null ? paramType : Hibernate.getClass(preloadedContent)));
      setter.invoke(target, preloadedContent);
  }


  private void setPreloadedMap(Object target, String setterName, Map preloadedMap) throws [..] {
      Method setter = target.getClass().getMethod(setterName, Map.class);
      setter.invoke(target, preloadedMap);
  }


  private void setPreloadedCollection(Object target, String setterName, Collection preloadedCollection) throws [..] {
    if (preloadedCollection instanceof List) {
      Method setter = target.getClass().getMethod(setterName, List.class);
      setter.invoke(target, preloadedCollection);
    } else if (preloadedCollection instanceof Set) {
      Method setter = target.getClass().getMethod(setterName, Set.class);
      setter.invoke(target, preloadedCollection);
    } else {
      [..]
    }
  }


  public void setPreloadService(PreloadService preloadService) {
    this.preloadService = preloadService;
  }
}  
This class is quite simple. The @Pointcut annotated method defines just where the aspect should be applied. In this case to each getter in the model (of course this could be enhanced by avoiding simple types). An improvement to this could be to use a special annotation like "@LazyLoadable" on every getter that could be a candidate for lazy loading, but for simplicity it is stated like this.

The @Around annotated method first checks if the requested resource is of type "AbstractPersistentCollection"(A list, set or map) or of type "BaseEntity"(the mother of all entity classes in your domain; This class is probably annotated with "@mappedSuperClass" and contains for example an ID, or a date saving the last change of the entity; In short, everything that could be interesting for every entity in your model). Depending on the type it checks whether a call to its getter would cause the LazyInitializationException(check for isInitialized and if the session is there). If so it calls the preloadService to take care of the lazy reloading and with its result it calls an apropriate setPreloaded*() method which puts the preloaded object into place using the dedicated setter (!CAUTION! This process assumes coding standards, like for getter and setter names the same base-method name is used just differing by its prefix which is "get" or "set" eg. getName and setName)

In this way after the aspect the requested resource is in place and the normal call to its getter can return it. This brings us to the preloadService, which is quite a simple spring bean. Its purpose is to open the transaction and forward the call to the DAO.


@Named
public class PreloadServiceImpl implements PreloadService {


  @Inject
  private PreloadDao preloadDao;


  @Override
  @Transactional(readOnly = true, propagation = Propagation.REQUIRED)
  public Object preloadCollection(BaseEntity entity, String getterName) {
    if (entity == null || getterName == null || !getterName.startsWith("get")) {
      return null;
    }
    return preloadDao.preloadCollection(entity, getterName);
  }


  @Override
  @Transactional(readOnly = true, propagation = Propagation.REQUIRED)
  public BaseEntity preloadBaseEntity(BaseEntity entity, String getterName) {
    if (entity == null || getterName == null || !getterName.startsWith("get")) {
      return null;
    }
    return preloadDao.preloadBaseEntity(entity, getterName);
  }
}

Nothing special in here; Just a bean that forwards to the DAO. This structure is just to maintain the structure of the rest of the project.

The last is the DAO which reloads an entity und tries in some way to initialize(fetch) the content in order to return it.
@Named
public class PreloadDaoHibernateImpl implements PreloadDao {


  @Inject
  SessionFactory sessionFactory;


  @Override
  public Object preloadCollection(BaseEntity entity, String getterName) {
    if (entity != null && getterName != null) {
      // Reload current entity
      sessionFactory.getCurrentSession().load(entity, entity.getId());
      try {
        Method getter = entity.getClass().getMethod(getterName, (Class[]) null);
        Object answer = getter.invoke(entity, (Object[]) null);
        if (answer instanceof Collection) {
          return fetch((Collection) answer);
        } else if (answer instanceof Map) {
          return fetch((Map) answer);
        } else {
          // "Result is not a Collection!"
        }
      } catch (Exception e) {
      }
    }
    return null;
  }


  @Override
  public BaseEntity preloadBaseEntity(BaseEntity entity, String getterName) {
    if (entity != null && getterName != null) {
      // Reload current entity
      sessionFactory.getCurrentSession().load(entity, entity.getId());
      try {
        Method getter = entity.getClass().getMethod(getterName, (Class[]) null);
        Object answer = getter.invoke(entity, (Object[]) null);
        if (answer instanceof BaseEntity) {
          return fetch((BaseEntity) answer);
        }else 
          // "Result is not a BaseEntity!"
        } catch (Exception e) {
      }
    }
    return null;
  }


  private Map fetch(Map map) {
    map.values();
    return map;
  }


  private Collection fetch(Collection collection) {
    collection.iterator();
    return collection;
  }


  private BaseEntity fetch(BaseEntity answer) {
    answer.getId();
    return answer;
  }
}


The basic idea behind is to load the entity into the session and call the getter that originally would have failed at that call. Depending on the type of object now try to initialize it.

And that is basically all you need to code. Just take it, adapt it to the names and namespaces used in your project and that's it.

Well, not completely. As said before now we need to load-time-weave the application. In Spring there is for sure something like the "application-context.xml" which contains an entry like "</context:annotation-config> as nowadays beans are defined using annotations. To this line you have to add the following:
<aop:aspectj-autoproxy>
<context:annotation-config>
<context:spring-configured>
<context:load-time-weaver>
And if you haven't already put the OnDemadPreLoader.java into a Spring scanned package add it to like
<context:component-scan base-package="some.package.lazy" />
The latter alows Spring to find your aspekt, whereas the other configurations enable load-time-weaving. Now as a last file you have to create under src/main/java/resources (or wherever your "application-context.xml" is located) a folder META-INF containing a file named aop.xml.
<!DOCTYPE aspectj PUBLIC "-//AspectJ//DTD//EN" "http://www.eclipse.org/aspectj/dtd/aspectj.dtd">
<aspectj>
  <weaver options="-verbose -showWeaveInfo">
    <include within="path.to.model.hibernate.*" />
    <include within="path.to.onDemandPreloader.lazy.*" />
    <exclude within="*..*javassist*" />
    <exclude within="*..*CGLIB*" />
  </weaver>
  <aspects>
    <aspect name="path.to.onDemandPreloader.lazy.OnDemandPreloader" />
  </aspects>
</aspectj>

This just says where you want to weave in your aspect and what aspect it is. Furthermore it uses the configurable annotation together with the autowired one to inject dependencies.

Now we are almost done. In my case there is just one more thing, which is inside the server.xml that tomcat uses you have to put like

<Context docBase="projectName" path="/projectName" reloadable="true" source="org.eclipse.jst.jee.server:projectName">
  <Loader loaderClass="org.springframework.instrument.classloading.tomcat.TomcatInstrumentableClassLoader" />
</Context>

which replaces the original TomCat classloader with a spring one. and thats it basically. Just start your tomCat now with the javaagent set to the instrumented one and it should work
-javaagent:./src/main/webapp/WEB-INF/lib/spring-instrument-3.0.2.RELEASE.jar
Here are some useful links where I found the concept to this ideas:

LazyInitializationException and Hibernate

If you ever really used Hibernate in a web project sooner or later you'll see one of this nasty LazyInitializationException telling you that you try to access an uninitialized collection or that the session is closed.
(The documentation says: Indicates access to unfetched data outside of a session context. For example, when an uninitialized proxy or collection is accessed after the session was closed.)


Normally this means that the program tries to follow a 1:n (or m:n) relationship which was decleared to be fetched LAZY as it is needed. 
A very simple example for such a declaration is stated below.


@Entity
@Table(name = "person")
public class Person {
[..]
@OneToMany(targetEntity = Address.class, mappedBy = "person", fetch = FetchType.LAZY)
private List addresses;
[..]
}


A very simple solution to this problem would be to just check if it is possible to follow the relationship without exception. If not, then reopen the session to Hibernate and reconnect the object to the database. Once reconnected all the needed data can be fetched and the object can be detached and sent to the client.

However, this would have to be performed prior to every single "get*" call on every entity in your domain model (Except the once that return non-entity types like String, int or long) which would mean of course a lot of code in each getter to do always the same or at least similar stuff.

At this point a programmer has to look for a different solution and it is the point where aspect oriented programming comes in handy. An aspect could be put around every getter in your domain model to dynamically check for the presence of the Hibernate session and just in case reopen it and fetch the needed contents.

Basically it should look like the following pseudo-code (adopted from a Java solution):

@Pointcut("execution(public * path.to.model..*.get*(..))")
public void possibleLazyInitializationException() {
}

@Around(value = "possibleLazyInitializationException()")
public Object preloadOnDemand(ProceedingJoinPoint pjp) {
  result= pjp.getTheRequestedAttributeReference;
  if (result == Collection,Map,BaseDomainObject && result.isNotInitialized && result.getSession == null){
    reopenHibernateSession;
    reload(pjp.getTargetObject);
    initializeTheResult;
    useItsSetterToPutItOnPlace;
  }
}

After this Aspect the normal getter method will find the object in place, so it can be called using the getter as usual.
(I planned to post now my own solution to it, however it is very specific to my setup, so I decided to put it in a dedicated blog entry)

Friday, April 09, 2010

LaTeX - What was the shortcut of that symbol again?

Who of you LaTeX-users has never been in the situation that you needed some strange symbol, or a greek letter or whatever and you could not remember its LaTeX shortcut?

No there is a simple and smart solution to this problem Detexify tries to determine the symbol your looking for based on your freehand drawing of it.


In the sample above it correctly recognized the greek letter "Phi" from a 2-second drawing.

Try it out

Wednesday, April 07, 2010

Correct Citation with Latex and BibTex

Starting with LaTeX can be a very annoying and quite time consuming process, but when you finally have your environment up and running (MAC or Windows)you probably notice that there is no need to have a word processor on your machine anymore (except for compliance to others).

As I'm a programmer Eclipse is already on my computer, so the choice of my editorial environment felt to that in combination to TexLipse but the hints below work for every environment.

Let's imagine for a moment that you want to cite in your text the book named "Secure Programming with Static Analysis". So the first step is to create an empty file called "myReferences.bib" (or something else with extension .bib). Now you need to find the correct citation entry for that book, which is usually done using some specialized web sites. In the area of Computer science there are at least the once below offering this service


Google Scholar simplifies this task because it is aware of most of the websites listed above and therefore it lists in first place for the book we are looking for the corresponding website of http://portal.acm.org/.

From there it is possible to get the correct citation for the book which is:

@book{Chess2007spw,
 Address = {Erewhon},
 Author = {Chess, Band West, J.},
 Edition = {1.ed.},
 Isbn = {0321424778},
 Publisher = {Addison Wesley Professional},
 Title = {Secure Programming with Static Analysis},
 Year = {2007}}

This citation can now be copied into the .bib file created beforehand.
The last steps to make it visible in the LaTeX document are to add the style and bibliography to the document as follows:

\bibliographystyle{plain}    % (uses file "plain.bst")
\bibliography{myReferences}  % expects file "myReferences.bib"

and to cite it somewhere in the text as follows:

[..]from the book \cite{Chess2007spw}.

And that's it basically

Wednesday, March 17, 2010

HowTo: Remove recursively all the ".svn" from a directory

Most of the times when you are working on a project you need to collaborate with other people, or you want at least that if your computer crashes not to loose all your work.
In this case it is essential to work with some sort of version control system. Nowadays the most important versioning systems are CVS, SVN and GIT.

In my case the choice is most of the times subversion(SVN) because of its simpleness, its seamless integration into the Eclipse framework due to "SubClipse" and because there are thousands of free SVN-hosts around, of which Google Code is probably the most well known.

This works (more or less) fine and should not be the topic of this post. The problem starts when you want to give your work away without disconnecting from your repository. If you just copy your project and (zip it and) send it to someone else you'll probably send all the ".svn" staff hidden into each folder of your project, and in a reasonably big project this could be even more than 100, which makes it unfeasible to do this work by hand.

For all the Unix users, like the Linux guys and the Mac guys, there is a really simple solution using the terminal. The following command finds recursively all the ".svn"'s in the subtree you are launching the command including "." (Find, starting from "." all directories who's name is ".svn")
find . -type d -name .svn
this command can now be passed to the remove command (rm)
rm -rf `find . -type d -name .svn`
The flags in this case tell to the "rm" command
  • -r --> recursively (all the sub(folders|files))
  • -f --> force (do not ask for confirmation, just delete)

If you are not an experienced Unix user be careful with the "rm" command in combination with these two flags, they could remove your entire system when used in the "right" way :-P

Sunday, March 07, 2010

JavaScript-Engine in a Java program

Some days ago I was experimenting on an easy way to evaluate a calculation saved within a String variable of my program.
After some searches and a very useful hint I finally found the Java "ScriptEngineManager" introduced in Java 6 (JavaDoc). This engine is a very useful extension to the Java programming language as it allows to natively import the functionality of (in this case) JavaScript (documentation).

With this knowledge I wrote this simple test-program to show the possibilities of this technology:

import javax.script.ScriptEngine;
import javax.script.ScriptEngineManager;
import javax.script.ScriptException;


public class ScriptTest {


  public static void main(String[] args) {
ScriptEngineManager mgr = new ScriptEngineManager();
ScriptEngine jsEngine = mgr.getEngineByName("JavaScript");
  try {
  Number ret = (Number) jsEngine.eval("10 + 5 * (2 +3 *6)");
  System.out.println(ret.doubleValue());

  Boolean ret1 = (Boolean) jsEngine.eval("100 >= (6*5)");
  System.out.println(ret1.booleanValue());

jsEngine.put("Hallo", "Servus");
String ret2 = (String) jsEngine.eval("Hallo");
System.out.println(ret2);
} catch (ScriptException ex) {
ex.printStackTrace();
}
}
}

Monday, March 01, 2010

Microsoft Office 2007: "This copy of Microsoft Office is not genuine"

Some time ago Microsoft published an Update named "Office Genuine Advantage (KB949810)".
This update causes some Versions to display a message like the one below:
This copy of Microsoft Office is not genuine.


Please excuse this interruption. This copy of Microsoft Office did not pass validation. Click Learn More for details and for help identifying the best way to get genuine Microsoft Office.
When your Office shows a message like this it means that the version you are using is indexed by MS as stolen or unauthorized copied.
Of course there exist several reasons why this could happen; either someone copied your serial and published it in some strange forums, or you have bought your PC with MS Office pre-installed and the first owner still uses its version, or you simply downloaded it from somewhere.

There exists a very simple solution to come over this problem, and if you have not stolen your copy feel free to use this solution in order to remove this annoying pup-up.
  1. Close Office completely
  2. Go to C:\windows\system32
  3. Locate the following three files:
    OGAAddin.dll
    OGACheckControl.dll
    OGAEXEC.exe
  4. Rename all of them into *.old to obtain
    OGAAddin.dll.old
    OGACheckControl.dll.old
    OGAEXEC.exe.old
  5. Re-run Office
  6. (delete the *.old files)
Now the pop-up doesn't show up again and you can use your copy as before the update