Thursday, May 05, 2011

Web Test Frameworks

I've spent some time looking around for some decent test frameworks for the CRM User Interface.  I want to test that certain things change, or update, as a result of workflow, security rights or javascript.

Ideally I would like to:
- Record the test in IE from  my CRM 2011 dev environment 64bit operating system
- Export the test to C#
- Be able to run the tests from Visual Studio 2010 via the Reshaper Test Runner

I've not been able to find a suitable fit for free, but here's some note worthy frameworks I found at the time of writing.

Selenium 

Selenium seems to be one of the better frameworks out there, it has many plug-ins, tests can be recorded (in FireFox only) and scripted to a lot of languages, and can run the tests across many different browsers.  Unfortunately not being able to record the tests in IE means that this is not ideal for my requirements.

WatiN  and Test Record

WatiN seems to be the next best thing if you need to record tests using IE, there's a Watin Test Recorder that allows you to record and export the tests in to C# and other languages, this sounds great but unfortunately it's not compatible with 64bit operating systems which means it can't be used for CRM 2011.  I did post a comment to a feature request to get it working with 64bit operating systems so there's hope yet.

Robot Framework 

The Robot Framework with an add-in called the SeleniumLibrary also looks promising. Robot framework is a keyword-based testing framework that is written in python. It allows you to write test cases in plain text or as html tables. SeleniumLibrary is an extension that lets you script web pages with keywords like Open Browser, Click Link, Checkbox Should Be Selected, etc.  The downside to this is that you can't record the tests and then edit them.


So with no perfect fit I'm going to have to change my requirements
. I suspect that I will find that I will get more value and more robust tests if you code them myself.

Both the Robot Framework, and Selenium seem to be good bets, both are large projects with a lot of support and activity on them.   Unfortunately test recording and VS test running don't seem to be options with the frameworks.  Long term my money is on the Robot Framework but today its .Net integration is not really there.  I came across a project called Selenium Toolkit for .NET
 that simplifies the setup and use of the functional testing framework Selenium, while I'm unable to get the tests to run from Visual Studio/Reshaper at this point, I'm sure it's possible.

For a comprehensive list of open source testing frameworks look here

0 comments: