Menu

NHibernate (7)

Workaround for HQL SELECT TOP 1 in a subquery

HQL doesn’t seem to support clauses like “SELECT TOP N…”, which can cause headaches when for example you need to get the data for the newest record from a table. One way to resolve this would be to do something like “SELECT * FROM X WHERE ID in (SELECT ID FROM X WHERE Date IN (SELECT MAX(Date) FROM X))”, a doubly nested query which looks complicated even in this simple example and gets out of control when query conditions need to be more complex.

What is the alternative? Use EXISTS – as in “a newer record doesn’t exist”. It still looks a bit ugly but at least it’s manageable. The above query would then look like this: “SELECT * FROM X AS X1 WHERE NOT EXISTS(SELECT * FROM X AS X2 WHERE X2.Date > X1.Date)”

Note that this works only for “SELECT TOP 1”. For a greater number there doesn’t seem to be a solution at all.

Repository moved temporarily to '/viewvc/nhibernate/trunk/'; please relocate

When trying to switch my local Subversion copy of the NHibernate source to a different tag (from 3.1GA to trunk, in this case), I got this error:

Repository moved temporarily to '/viewvc/nhibernate/trunk/'; please relocate

The frustrating thing was that I was trying to relocate to exactly this url. And if I tried others, it said that I should relocate to them… I searched the net in vain for the solution, the only information I got is that I should re-configure my apache server (thanks a bunch!)

The problem is, in fact, simple: the URL is wrong. I thought I could just copy the repository’s URL from my web browser, like I do with other sites. Not here: there’s a separate entry for direct SVN access. So instead of using this url:

http://nhibernate.svn.sourceforge.net/viewvc/nhibernate/

use this one:

https://nhibernate.svn.sourceforge.net/svnroot/nhibernate

It does seem like a simple problem but the solution wasn’t so easy to find.

How to make LINQ to NHibernate eager-load joined properties like the Criteria API

In terms of “lazyness” of a property, there are currently three different ways in which it can be mapped in NHibernate:

  • lazy=”false” means that it’s not lazy at all – the property’s content will be loaded along with its owner object. This means additional data is always loaded when you load an object, and may mean additional sql queries, too.
  • lazy=”proxy” means that the object contained in the property is loaded when any of its public members is accessed. This property will contain an instance of a proxy, which is an object that knows how to initialize itself on-access. It performs the initialization not by loading its properties but by loading an instance of a real object and redirecting its properties and methods to it. This is why everything on the class that is to be proxied needs to be virtual: proxy is an instance of a class derived from it, which is dynamically generated and has every public member overridden to support lazy-loading.
  • lazy=”no-proxy” means that the property is lazy-loaded, but without a proxy. From what I’ve seen, here the lazy property iself is manipulated on the owner object so that it facilitates on-access loading. In any case, there’s no proxy and no duplicate instances. This feature is currently (in v3.0.0) buggy and it seems to work the same as the first option (lazy=”false”), just as it did in 2.0 when it was unsupported.

Each of the options has its bad sides: with proxies, you get duplicate objects and must make everything virtual on your data classes. In the non-lazy option, for each eager property a join is usually added in the sql query so that the property’s values are loaded at the same time, and the same goes for the property’s properties etc. Even worse, HQL queries don’t respect this joining method, they load the main table in one SQL query and then execute an additional query or two (or dozen) for each record to collect its related data – this is called the “N+1 selects” problem. Needles to say, using HQL for such queries is madness: in these cases, it is best to switch to Criteria API which does the joins properly.

And the bad sides of the no-proxy option? It doesn’t work… Other than that, it seems the perfect solution: no joins, no duplicate objects. If you ask me, I don’t want my data to be loaded on-demand at all. I don’t want the application to decide when it will load its data: if I fill a datagrid with one hundred objects and then the grid triggers lazy-loading on each of the objects in turn, this will create chaos. No, I want the application to break if it accesses data that was not explicitly loaded. But with the current implementation I have no choice: it’s either eager or proxy, and I’m choosing eager, for better or worse.

How about LINQ queries? In 2.x it was implemented over the Criteria API which means it knew how to join-load additional records. Not so in 3.x: now it behaves like HQL, N+1 selects all over the place.

So, what is there to do? It seems the only option left is to write all queries with explicit fetch statements for every non-lazy property… This would definitely solve the execution performance issue, but development performance would suffer: if I add a new non-lazy property, I’d have to rewrite all queries where it appears.

Ok, but if we’re using LINQ, it’s a dynamical query, right? It can be modified to include all required fetches. After some research, it turns out that there’s a solution that (at least on the outside) looks even elegant: use an extension method to do this. So, you would do something like:

session.Query<Person>().Where(…).EagerFetchAllNonLazyProperties()

or:

(from p in session.Query<Person> where … select …).EagerFetchAllNonLazyProperties()

Here’s one way this method could be implemented. Note that this is a somewhat hacked implementation and that there are probably some unsupported cases – one thing that is suspicious to me is that Criteria API joined the eager properties recursively while only the first level is covered here, so be careful. But it’s a good start... Preliminary tests were very promising ;).

public static IQueryable<TOriginating> EagerFetchAll<TOriginating>
  (this IQueryable<TOriginating> query)
{
  // hack the session reference out of the provider - or is
  // there a better way to do this?
  ISession session = (ISession)typeof(NhQueryProvider)
    .GetField("_session", System.Reflection.BindingFlags.Instance
      | System.Reflection.BindingFlags.NonPublic)
    .GetValue(query.Provider);

  IClassMetadata metaData = session.SessionFactory
    .GetClassMetadata(typeof(TOriginating));

  for(int i = 0; i < metaData.PropertyNames.Length; i++)
  {
    global::NHibernate.Type.IType propType = metaData.PropertyTypes[i];

    // get eagerly mapped associations to other entities
    if (propType.IsAssociationType && propType.IsEntityType
      && !metaData.PropertyLaziness[i])
    {
      ParameterExpression par = Expression.Parameter(typeof(TOriginating), "p");

      Expression propExp = Expression.Property(par, metaData.PropertyNames[i]);

      Expression callExpr = Expression.Call(null,
        typeof(EagerFetchingExtensionMethods).GetMethod("Fetch")
          .MakeGenericMethod(typeof(TOriginating), propType.ReturnedClass),
        // first parameter is the query, second is property access expression
        query.Expression, Expression.Lambda(propExp, par)
      );

      LambdaExpression expr = Expression.Lambda(callExpr, par);

      Type fetchGenericType = typeof(NhFetchRequest<,>)
        .MakeGenericType(typeof(TOriginating), propType.ReturnedClass);
      query = (IQueryable<TOriginating>)Activator.CreateInstance
        (fetchGenericType, query.Provider, callExpr);
    }
  }

  return query;
}

Collection owner not associated with session? Not quite.

I hate when this happens. I upgraded to NHibernate 2.0 and then quickly afterward to 2.1.0 (you guessed it: because of LINQ). I had to change a couple of things to support it in my company’s application framework and it all seemed to work well – until I discovered that deleting any entity that has a one-to-many relation with cascade=”all-delete-orphan” stopped functioning. It died with a cryptic error message of “collection owner not associated with session”… If I changed to cascade=”all” it worked, but this is not the point, it wasn’t broken earlier. Of course, I tried looking all over the web and apart from a page in Spanish (which wouldn’t be helpful even if it was in English) came up blank. Tried moving to NHibernate 2.1.2 - which is not that simple since we’re using a slightly modified version of NHibernate (a reason more to suspect that the solution to this problem would be hard to find). So here’s a short post for anyone stumbling upon a similar problem.

In the end, I traced it to this behaviour: the collection owner is not found in the session because NHibernate tries to find it using ID = 0, while it’s original ID was 48. The logic is somewhat strange here, because the method receives the original collection owner (which is in the session), retrieves its ID (which was for some reason reset to 0) and then tries to find it using this wrong ID. Moreover, there’s a commented-out code that says “// TODO NH Different behavior” that would seem to do things properly (I checked it, it’s still standing in the NHibernate trunk as is). But the real reason why this happened is that blasted zero in the ID: further debugging (thankfully, there’s a full source for NHibernate available), revealed that it was reset because “use_identifier_rollback” was turned on in the configuration. Well… I probably set this to experiment with it and forgot. Turning it off solved the problem for me… Luckily, I didn’t really need this rollback functionality - as it’s not exactly what it seems to be: it doesn’t rollback identifiers when the transaction is rolled back, it rolls them back when entities are deleted! Why the second feature made more sense to implement than the first one is a mystery to me...

NHibernate 2.1 updates schema metadata without being asked to

In NHibernate 2.1, the session factory is set up to access the database immediately when you build it. This is done by a Hbm2ddl component to update something called SchemaMetaData: I’m not sure what this is all about, but I am certain that such behaviour is not nice. The previous version of NHibernate didn’t do it, so I expect the new one to behave likewise unless I explicitly order the change.

The solution for this is to add a line to your hibernate.cfg.xml file that says:

<property name="hbm2ddl.keywords">none</property>

Note that completely omitting this setting will actually enable the feature… Did I already mention I don’t like it? I don’t, so much that I decided not to change config files but to hardcode it disabled. I use one global method to load the NHibernate configuration, so this is easy. The code looks something like this:

_configuration = new global::NHibernate.Cfg.Configuration();
_configuration.Configure();
_configuration.SetProperty("hbm2ddl.keywords", "none");

NHibernate queued adds on lazy-load collections

I don’t know if this behaviour is documented (well, yeah, the (N)Hibernate documentation is pretty thin but it’s improving), I wasn’t fully aware of it and this caused a bug… I’m posting this in hope it may save for someone else the time I have lost today :).

In our software we use a custom NHibernate collection type that is derived from AbstractPersistentCollection and keeps track of “back references”: that is, references to the record that owns the collection. It does this automatically when an object is added to the collection.

Now, on collections mapped as lazy-loading, Add() operations are allowed even if the collection is not initialized (i.e. the collection just acts as a proxy). When adding an object to a collection in this state, a QueueAdd() method is called that stores the added object in a secondary collection. Once a lazy initialization is performed, this secondary collection is merged into the main one (I believe it’s the DelayedAddAll() method that does this). This can be hard to debug because lazy load is transparently triggered if you just touch the collection with the debugger (providing the session is connected at that moment), and everything gets initialized properly.

Our backreference was initialized at the moment the object was really added into the main collection. But this is not enough, we had to support queued adds - that is, the cases when QueueAdd returns true. The other alternative is to disable delayed adds by commenting out the places where QueueAdd is called – I don’t know if this is possible, there seems to be some code that supports it. We decided to support delayed add, and it seems to work. The modification looks something like this (this is the PersistentXyz class):

int IList.Add(object value) 
{ 
    if (!QueueAdd(value)) 
    { 
        Write(); 
        return ((IList) bag).Add(value); 
    } 
    else 
    { 
        // if the add was queued, we must set the back reference explicitly 
        if (BackReferenceController != null) 
        { 
            BackReferenceController.SetBackReference(value); 
        } 
        return -1; 
    } 
}

Supporting triggers that occasionally generate values with NHibernate

NHibernate supports fields that are generated by the database, but in a limited way. You can mark a field as generated on insert, on update, or both. In this case, NHibernate doesn’t write the field’s value to the database, but creates a select statement that retrieves its value after update or insert.

Ok, but what if you have a trigger that updates this field in some cases and sometimes doesn’t? For example, you may have a document number that is generated for some types of documents, and set by the user for other types. You cannot do this with NHibernate in a regular way – but there is a workaround…

It is possible to map multiple properties to the same database column. So, if you make a non-generated property that is writable, and a generated read-only property, this works. You have to be careful, though, because the non-generated property’s value won’t be refreshed after database writes.

A more secure solution would be to make one of the properties non-public and implement the other one to support both functionalities. Like this:

//
// This field is used only to send a value to the database trigger:
// the value set here will be written to the database table and can be consumed by
// the trigger. But it will not be refreshed if the value was changed by the trigger.
// 
private int? setDocumentNumber;	

private int? _documentnumber;

//
// The public property that works as expected, generated but not read-only
// public int? DocumentNumber { get { return _documentnumber; } set { // NHibernate is indifferent to this property's value (it will not // be written to the database), so we have to update the setDocumentNumber // field which is regularly mapped _documentnumber = value; setDocumentNumber = value; } }

Here’s the NHibernate mapping for these two:

<property name="DocumentNumber" generated="always" insert="false" update="false"/>
<property name="setDocumentNumber" column="DocumentNumber" access="field"/>
Subscribe to this RSS feed