There are times when you need to test every page in an Umbraco installation for errors - for example, after performing a major upgrade, or installing new components.
Working on such a site recently, which needed upgrading from Umbraco 6.1.6 to 6.2.5, I needed to perform such a test. Doing this manually would have been especially arduous and time-consuming on a site with several hundred pages. So the question is, how to automate this kind of testing?
If you've ever done substantial load testing on a website before, you'll probably be familiar with JMeter - an Apache Software Foundation project designed to be a useful tool in the arsenal of any web developer. JMeter allows you to simulate multiple users to your website, to allow you to test the throughput & performance of your code and analyse the results.
JMeter has a number of useful features which allow it to do more than just "slam" your website for performance benchmarking. These include control statements (such as if-else and for-each), and response parsing tools.
By manipulating these, you can set up a basic (single-threaded, sadly) crawler that will parse your entire site, looking for a given text expression in the output. Umbraco will, when a macro fails to execute correctly, output the string "Error loading macroengine script" followed by the filename of the script that failed to run. We can pick up this string (plus any ASP.NET error 500's thrown for more traditional MVC errors).
Firstly, we set up JMeter to crawl a sitemap.xml:
We then add an XPath Extractor to select the
/urlset/url/loc and each value is outputted into a
We then proceed to set up a for-each loop, iterating over every value of
testUrl, outputting it to a local variable called
Finally, we pass the
url variable to a HTTP Request action, which fetches the page in question. Note the
Lastly, we set up a listener, to catch any incidences of the required text string "Error loading macroscript". Note the "Not" checkbox - checking this means pages that fail to contain our string will "pass", whereas pages that do contain the string will "fail" and therefore be flagged. Any pages that fail to return a 2xx or 3xx code will also "fail" in this test.
Let's run it!
Eesh, that's not pretty. I need to fix a few URL rewrites!
The JMeter test plan can be downloaded here. Save the file with a .jmx extension, open it in JMeter, adjust the URLs accordingly, and let it loose on your sites. Let me know if you find it useful!