Just how much heavier is Catalyst than Dancer2?

May 5, 2014 by David Farrell

Within the Perl community it is received wisdom that Catalyst is a heavyweight web framework with many dependencies and that Dancer2 is a micro web framework that’s more agile than a Cirque du Soleil acrobat. But is it true?

Tech Specs

The comparison was between Catalyst 5.90062 and Dancer2 0.14. For local tests the machine used was a 2011 MacBook Air running Fedora 19 and Perl 5.16.3.

Number of Dependencies

The greatest perceived difference between Catalyst and Dancer2 is the number of dependencies each has, with Catalyst being thought to have “too many”. For example at the recent German Perl workshop, Dancer2 development lead Sawyer X joked:

We’re not pulling off half of CPAN like maybe a different web framework [Catalyst]

To compare the two frameworks, we need to compare all of their dependencies; not just those first-order dependencies used by the framework, but also those used by the modules used by the framework and so on. Fortunately this is an easy comparison to make using Stratopan.

I created two stacks, one for Catalyst and one for Dancer2. Their respective dependency graphs are shown below:

Tabulating the Stratopan data for the two stacks we get:

Stratopan reveals that while Catalyst required 15 more modules than Dancer2 (+52%), when considering recursive dependencies, the Catalyst stack is only 19% larger than the Dancer2 stack. Interestingly, over 61% of the 96 distributions Dancer2 uses are used by Catalyst.

Testing

When installing a CPAN module, the module tests usually take the longest time of the installation process and can contribute to the perceived “size” of the module. As the maturer framework, perhaps Catalyst simply has more tests than Dancer2, and therefore it’s installation process takes longer?

To check for this, I tested both frameworks installation tests:

$ perl Makefile.PL
$ make
$ make test

Catalyst’s test results:

Files=166, Tests=3374, 179 wallclock secs ( 0.85 usr  0.16 sys + 172.95 cusr  4.56 csys = 178.52 CPU)

Dancer2’s results:

Files=78, Tests=1112, 21 wallclock secs ( 0.38 usr  0.07 sys + 19.68 cusr  1.46 csys = 21.59 CPU)

These results show that Catalyst ran 3,374 tests over 3 minutes compared to Dancer2’s 1,112 tests over 21 seconds. So while Catalyst did run more tests, it was also slower in executing them; Dancer2 executed 53 tests per second and Catalyst managed 19 tests per second.

What would explain this discrepancy? Perhaps Dancer2 has more trivial tests that run quicker than Catalyst’s tests. As a control I checked the code coverage of each framework’s test suite using Devel::Cover. Dancer2’s total test coverage was 84.7% whilst Catalyst’s was 85.5% - a negligible difference.

Conclusion

So is Catalyst’s “heavyweight” reputation deserved? Whilst it does not have a significantly greater number of dependencies than Dancer2, during installation Dancer2’s test suite runs far faster than Catalyst’s and with a similar code coverage. This doesn’t mean Dancer2’s test suite is better than Catalyst’s (Catalyst’s test suite could have higher cyclomatic complexity). But it does show that Dancer2’s simpler micro-framework approach offers benefits beyond fast startups and application development time. As a Catalyst user, I’ve found Catalyst to be plenty fast for my needs (like PerlTricks.com), however its startup time is noticeably slow and during installation tests the Catalyst test application is started and stopped multiple times. I expect that contributes more to discrepancy in installation times between Dancer2 and Catalyst than anything else.

Enjoyed this article? Help us out and tweet about it!

This article was originally posted on PerlTricks.com.