With the migration capabilities we have between Acceleo 2 and Acceleo 3, comparing those two has proven rather easy; comparing them with Xpand has been a little more trouble for me as I am not familiar to this tool. I still attained a result I believe to be interesting.
I didn't make in-depth comparisons between the tooling of each of these three tools, focusing on the generation performance only. GUI comparison may come later, when I have the courage :).
What I did compare was the generation performance for a given set of generation templates that I ported to all three of the technologies. The result is that all three tools (Acceleo 2, Acceleo 3 and Xpand 1.0.1) generate the very same java code, whitespaces excepted.
Each of the times outlined below is the average of three runs on the same machine :
- Intel Core 2 Duo 64 bits
- Dual core, 2GHz by core
- 4 Go RAM
- Windows seven 64bits
The version of Java installed on this machine was Java SE 1.5.0_22 64bits.
The Eclipse platform version was Eclipse Helios 3.6.1 (64 bits still). Each tool was installed within its own Eclipse with only its dependencies and the dependencies of the tests.
I've run two distinct use cases for this benchmarking :
- "Small" corresponding to the generation with a small model as input. I used UML.genmodel for this, which in turn links to UML.ecore, Ecore.ecore and Ecore.genmodel. When counting the content of the two main models used by this transformation, UML.genmodel and UML.ecore (just a basic count of eAllContents), I get a total of 14 420 elements.
- "Medium" corresponding to the generation of medium models as input. For this, I generated a metamodel with a fixed number of classes, enumerations and datatypes, then the corresponding genmodel. Like above, a basic count of the model content yields a total of 50 944 elements.
As far as generated files go, the "small" use case generates 217 java classes for a total of 200 921 lines of code. The "medium" use case generates 1 206 java files for a total of 838 273 lines of code.
All measures have been taken "at worst", launching the generation in a new Eclipse runtime (thus, nothing in cache, no plugin loaded yet). The times given here are in seconds, and they correspond to the average of three runs. One measure comprises the time to load the model, the time to evaluate the generation and the time to serialize the result.
Without further ado, here are the figures :
| Acceleo 2 | Acceleo 3 | Xpand | |
| Small | 50 s | 10 s | 25 s |
| Medium | 215 s | 25 s | 212 s |
Being an Acceleo developper myself, I can only be pleased by these results ... Even though the test might be seend as biased as a result :p. All of the files needed to replay these can be downloaded from here. On that matter, as I am not familiar with Xpand, please don't hesitate to optimize the templates if at all possible (though the Acceleo 3 templates themselves don't use the latest evolutions of Acceleo 3.1).
