Automated Testing in Sitecore – Get the Most Out of Your Tests

Wednesday, August 07, 2013 @ 03:37

By: Dusty Eves, Senior Developer

Introduction
The requirements document are laid out on the table before you, the vision of your pending Sitecore site is pictured firmly in your head.  Soon the rubber will meet the road and your developers will begin transforming that vision into a reality.  You’ve got a team of brilliant coders, meticulous planning, and daily meetings as the tools you've gathered to protect the integrity of that vision.  One more tool from your proverbial bag of tricks is automated testing.  Sitecore as a platform is something of a different animal, not easily testable in and of itself. To make things worse, it does not match the paradigms used for most automated testing philosophies.  So how do you make automated testing with Sitecore work for you?

What not to test
While it obviously makes no sense to write tests for core Sitecore functionality, it can be hard to tell where Sitecore ends and your custom implementation begins.  A good rule of thumb: Don't write tests that are or can be made erroneous by proper Sitecore structure and validation.  For example, if you have a page template and you wanted to ensure that all pages had meta tags, any automated test would made be redundant by proper required field validation.

How not to test
The biggest rabbit hole to be avoided is over-abstraction.  Proponents of traditional TDD wisdom may tell you to abstract everything and test it all in isolation. In this case, that would suggests you abstract out Sitecore.  I can't stress enough not to do this for two reasons.  Firstly, it is a lot of work to do.  The second reason is there's no logical reason to do so.  Sitecore is the platform on which you're building your site.  Abstracting out Sitecore for your tests makes no more sense than abstracting out the .NET runtime from any other .NET project. 

What to test
Your highest value tests in Sitecore usually fall into one of two categories: extension points and integration points.  Site search is an excellent example for a good extension point test case.  Need to validate that your indexes are working correctly?  Grab a given Sitecore item, search against it, and see if you have it in your results.  Have a custom coded workflow step for new content?  Create an item through the API and see if it fires. 

Integration test points are the other high return automated tests and are most often where data is coming in from an external system.  With non-persistent data (e.g. a stock ticker displayed on a page), the test case is simple and doesn't involve the Sitecore API at all.  In the case of data coming in that will be persisted, the tests are a bit more complicated.  While it can be tempting to abstract or mock Sitecore in testing import processes, as already mentioned you'll quickly find it to be more trouble than it is worth.  The best way to test an import process is to actually perform the item creations in your import processes, track them during your tests and delete them on teardown. 

Data import, consumption of external content, and serving content through a web service; these are your quintessential high value testing facets in Sitecore.

How to test
My personal number one rule of testing in Sitecore:  Sitecore tests should be driven by data.   If you're new to data-driven automated tests, NUnit has a number of constructs to help construct your tests.

TestCaseSource Attributes
A method level attribute to which you provide a method name. That method return an object or a number of object that defines your tests.
Sequential
A parameter attribute which allows you to provide a series of values by which your tests will be executed.  The values are consumed in sequence, executing with test with all parameters at the first index, then second, etc.
Combinatorial
A parameter attribute that allows you to provide a series of values by which your test will be executed.  Much like sequential, but rather than executing the test with first, second, third etc. value of each parameter the tests are executed with all combinations of all parameters.
Random
A parameter attribute generating a random value.

For more information on how to drive your tests with data see the NUnit documentation.