Testing your web UI using SWExplorerAutomation and NUnit

[textile]In my "previous posts about SWExplorerAutomation (SWEA)":http://www.testingreflections.com/node/view/2592, I addressed a couple of initial problems with getting "SWEA":http://home.comcast.net/~furmana/SWIEAutomation.htm to work on a specific configuration of machine.

Having got past that, I moved on to more interesting things... Specifically, using SWEA under NUnit.

I won't go into the detail of any of the SWEA main features since "Alex Furman":http://www.alexfurman.net does that better than me in his "Flash Demo":http://home.comcast.net/~furmana/SWExplorerAutomation.html and developer documentation available on the "SWEA home page":http://home.comcast.net/~furmana/SWIEAutomation.htm

When should you use SWEA? Well, there are so many different circumstances where it would be appropriate and many where it wouldn't... rather than guess your circumstances, I'll tell you about mine:

* Intranet Application - everyone in this organisation has to use Internet Explorer - so testing compatibility with other browsers isn't an issue;

* Client-side UI features are relatively rich, with various helpers for selecting items and populating fields. These UI features can't be exercised unless executed in a browser. These are the only things we are trying to test via the browser. For example, problem-domain tests are pointed at the 'Model' layer of the application (see Model View Controller pattern);

* We currently have no test-coverage of these UI features, which in past months have been known to break without us noticing until the end of the iteration;

* We are working in a Microsoft-centric organisation with Windows PCs and servers, an application written in C#;

* We are employing TDD and Continuous Integration using Nunit + Nant + CruiseControl for other tests and it would be of value to have all test-results aggregated in the CruiseControl build reports;

One thing we aren't trying to do is run these tests with every build. As test-coverage grows, the duration of each build will become unwieldy so the plan is to comment them out in the build script and have a scheduled task that will uncomment them at night and a force a build.

Enough of the context...

h2. SWEA Scenes

A scene is essentially an abstraction of the web-page, decoupling your test-code from the layout of the page and even the names/ids of elements. It is much like an 'object map' or GUI map' as used by many other GUI test tools. Elements on the page (i.e. in the scene) are called 'Controls'. If an input field (HtmlInput Control) is given a new ID or you change the position of the control in the DOM, the scenes allow you to change the element's definition in one central place - the SWEA Project File. You can further desensitise your tests from UI changes since scenes support identifying controls by ID, Name and SWEA also supports using Xpath to find an element in the page, e.g.:

%{color:blue;padding:1cm;}@//a[attribute::class="search_button"]@%

This finds an anchor with the class attribute of "search_button". Ideally, each element in the page will have a unique ID. This will make automating your tests that much easier.

It is up to you to define these scenes using the SWEA Designer which is a rich-client gui that facilitates the process.

!http://www.testingreflections.com/files/usrimages/SWEADesSmall.jpg!:http://home.comcast.net/~furmana/images/SWDesigner.png

Run-time identification of what 'scene' you are in is automatic, although you have programmatic control over this too. In your code, you can interact with scene controls like this:

%{padding:1cm;color:brown}@((HtmlInputText)(scene["QueryText"])).Value = "Software Testing";@%

h2. An NUnit Test Fixture using SWEA

How have I structured the Test Fixtures...

I thought I'd illustrate with an example using the "MSN Search":http://search.msn.com Search Builder (link underneath the text-box)... I have created two simple tests to illustrate the example, focusing on the Search Terms feature.

Firstly, I'll need a few things:

%{padding:1cm;color:brown}@using System;@%
%{padding:1cm;color:brown}@using System.Diagnostics;@%
%{padding:1cm;color:brown}@using NUnit.Framework;@%
%{padding:1cm;color:brown}@using SWExplorerAutomation.Client;@%
%{padding:1cm;color:brown}@using SWExplorerAutomation.Client.Controls;@%
%{padding:1cm;color:brown}@using SWExplorerAutomation.Client.DialogControls;@%

An important class is the ExplorerManager. As its name implies, it manages the Internet Explorer (IE) instance. To minimise test execution time, it is best to define this as a class-level object otherwise, with each test you will have to create a new explorerManager adding an unnecessary overhead to the test. Additionally, only DisconnectAndClose when you have finished with Internet Explorer. One of the slowest parts of the test can be loading IE for the first time so, when we load it, we want it to be available to all of the tests without having to start a new IE instance or find an existing instance to attach to... (normally, I'd prefix private or protected members with an underscore "_" but I haven't in this example):

%{padding:1cm;color:brown}@[TestFixture]@%
%{padding:1cm;color:brown}@public class QueryBuilderSearchTerms@%
%{padding:1cm;color:brown}{%

%{padding:2cm;color:brown}@private ExplorerManager explorerManager;@%

I like to assign the Controls (Html elements in the scene) to names that are more easily readable in the tests so I create these once at the class level:

%{padding:2cm;color:brown}@private HtmlInputText queryBuilderQueryText;@%
%{padding:2cm;color:brown}@private HtmlSelect queryBuilderSearchType;@%
%{padding:2cm;color:brown}@private HtmlInputButton queryBuilderAddToSearch;@%
%{padding:2cm;color:brown}@private HtmlInputText queryText;@%

There are some event handlers that come in handy. I'll talk about these more in a later post so, although I have included them in this example, I've left them empty for now):

%{padding:2cm;color:brown}@private static void explorerManager_DialogActivated(object sender, DialogScene dialogScene)@%
%{padding:2cm;color:brown}{%
%{padding:2cm;color:brown}}%

%{padding:2cm;color:brown}@private static void explorerManager_DialogDeactivated(object sender, DialogScene dialogScene)@%
%{padding:2cm;color:brown}{%
%{padding:2cm;color:brown}}%

%{padding:2cm;color:brown}@private static void explorerManager_Error(object sender, SWException errorException)@%
%{padding:2cm;color:brown}{%
%{padding:2cm;color:brown}}%

h3. TestFixtureSetup

Let's move onto the [TestFixtureSetup]. If you aren't familiar with Nunit, this attribute causes the associated method to be executed once at the start of the run of tests in the fixture (as opposed to [SetUp] which runs before each test method). Here, I'll create an instance of the explorerManager:

%{padding:2cm;color:brown}@[TestFixtureSetUp]@%
%{padding:2cm;color:brown}@public void Init()@%
%{padding:2cm;color:brown}{%
%{padding:3cm;color:brown}@explorerManager = new SWExplorerAutomation.Client.ExplorerManager();@%

I might need those event handlers (like I said, I'll come back to these in another post):

%{padding:3cm;color:brown}@explorerManager.DialogActivated += new DialogActivatedEventHandler(explorerManager_DialogActivated);@%
%{padding:3cm;color:brown}@explorerManager.DialogDeactivated += new DialogDeactivatedEventHandler(explorerManager_DialogDeactivated);@%
%{padding:3cm;color:brown}@explorerManager.Error += new ServerErrorEventHandler(explorerManager_Error);@%

Now, I want to start Internet Explorer:

%{padding:3cm;color:brown}@explorerManager.Connect(-1,ProcessWindowStyle.Hidden);@%

The parameter -1 tells the explorerManager to create a new IE instance and the second "ProcessWindowStyle.Hidden" (in System.Diagnostics) tells it to hide the window, so I don't have to watch the test each time it runs. While developing the test, you will want to see what happens so instead, you can use "ProcessWindowStyle.Normal".

I'll need my scenes (as described above) and I want to work only in one of them so I'll 'set the scene' as part of the fixture set-up, navigate to it (the URL is defined in the SWEA project file as part of the scene properties) and wait for the page to load:

%{padding:3cm;color:brown}@explorerManager.LoadProject("MSNSearch.htp");@%
%{padding:3cm;color:brown}@Scene scene = explorerManager["SearchWithSearchBuilder"];@%
%{padding:3cm;color:brown}@explorerManager.Navigate(scene);@%
%{padding:3cm;color:brown}@scene.WaitForActive(30000);@%

Notice that I have navigated straight to the view that has the Query Builder and not via the home page. This has trimmed several seconds off the test since I only load one page.

I also find the code easier on the eye if I assign more readable names to the Controls (as mentioned earlier):

%{padding:3cm;color:brown}@queryBuilderQueryText = ((HtmlInputText)(scene["QueryBuilderQueryText"]));@%
%{padding:3cm;color:brown}@queryBuilderSearchType = ((HtmlSelect)(scene["QueryBuilderSearchType"]));@%
%{padding:3cm;color:brown}@queryBuilderAddToSearch = ((HtmlInputButton)(scene["AddToSearchButton"]));@%
%{padding:3cm;color:brown}@queryText = ((HtmlInputText)(scene["QueryText"]));@%
%{padding:2cm;color:brown}}% %{color:green}@//End of [TestFixtureSetUp]@%

h3. Setup and TearDown

In these tests, I don't want a SetUp method but I have decided to have a TearDown method...

After each test, I want the query text value to be reset. I could do this by navigating to the scene but, in this case, I have decided that it will be faster if I just reset the Controls to their initial state. There is a more advanced approach to this by capturing the Controls' initial state and then using a Reset() method to restore all Controls but in this case, I'll simply reset the queryTextValue:

%{padding:2cm;color:brown}@[TearDown]@%
%{padding:2cm;color:brown}@public void Reset()@%
%{padding:2cm;color:brown}{%
%{padding:3cm;color:brown}@queryText.Value = "";@%
%{padding:2cm;color:brown}}%

You may prefer to take the hit and reload the page - it depends on what you are trying to prove with your tests. For maximum performance, you will attach to an existing instance. This will save loading and unloading IE with the execution of each Fixture... I can come back to that another time since there are other things to consider when taking that approach.

At the end of all of the tests, I want to unload IE (especially important if the test is run with IE being Hidden):

%{padding:2cm;color:brown}@[TestFixtureTearDown]@%
%{padding:2cm;color:brown}@public void Dispose()@%
%{padding:2cm;color:brown}{%
%{padding:3cm;color:brown}@explorerManager.DisconnectAndClose();@%

%{padding:2cm;color:brown}}%

h3. Tests

Tests can remain relatively simple. For example, if I want to verify that entering a value into the Query Builder's Query Text field and clicking the submit button 'moves' the text from the Query Builder Search Terms to the main Search Terms text-box, I might do this:

%{padding:2cm;color:brown}@[Test]@%
%{padding:2cm;color:brown}@public void TestQBTermsMovedToMainQueryText()@%
%{padding:2cm;color:brown}{%
%{padding:3cm;color:brown}@string textValue = "Software Testing";@%
%{padding:3cm;color:brown}@queryBuilderQueryText.Value = textValue;@%
%{padding:3cm;color:brown}@queryBuilderAddToSearch.Click();@%

%{padding:3cm;color:brown}@Assert.AreEqual(textValue,queryText.Value);@%
%{padding:3cm;color:brown}@Assert.IsNull(queryBuilderQueryText.Value);@%


%{padding:2cm;color:brown}}%

Or, to test that Query Builder Search Terms, once moved, are appended to the main Search Terms with a space between them:


%{padding:2cm;color:brown}@[Test]@%
%{padding:2cm;color:brown}@public void TestQBTermsAppendedToMainQueryText()@%
%{padding:2cm;color:brown}{%
%{padding:3cm;color:brown}@string textValue = "Software Testing";@%

%{padding:3cm;color:brown}@queryBuilderQueryText.Value = textValue;@%
%{padding:3cm;color:brown}@queryBuilderAddToSearch.Click();@%
%{padding:3cm;color:brown}@queryBuilderQueryText.Value = textValue;@%
%{padding:3cm;color:brown}@queryBuilderAddToSearch.Click();@%

%{padding:3cm;color:brown}@string expectedText = textValue+" "+textValue;@%
%{padding:3cm;color:brown}@Assert.AreEqual(expectedText,queryText.Value);@%

%{padding:2cm;color:brown}}%

I think you get the picture.

h2. In closing

Before using SWEA, however, there are at least two things to consider...

* Alex plans to make it shareware at some future date so there will be licensing fees, although he assures me that the fee will be 'reasonable'

* IE7 is on the horizon and I don't know much work Alex will need to do to ensure that SWEA continues to be a viable choice

So far, SWEA gets the thumbs up from me. It is clearly a well thought out approach and, if you are working in a Microsoft-oriented organisation or project then it seems ideal. No need to "bridge .NET and WATIR":http://www.hanselman.com/blog/CommentView,guid,ed8a9800-0bf5-44e8-9442-cef92ae39a48.aspx to test your web-based UI.

I have noticed a few stability related issues with the designer and will advise Alex of them as and when I find them... "Alex has been increadibly responsive so far":http://www.testingreflections.com/node/view/2719#comment-984.

On the horizon, Alex plans to include support for showModalDialog(), treating it in much the same way as a Dialog, which for many will prove to be very useful - an affordable (we hope) UI test tool with an Object/GUI map and support for showModalDialog()...

Oh... and if you are worried about Visual Studio licenses... you don't need it (although it helps).

To try out SWEA using C#, all you need is notepad and the .NET SDK. If you still want some IDE support, try "SharpDevelop":http://www.icsharpcode.net/OpenSource/SD/Default.aspx it's Open Source and suitable for just this sort of thing... especially with its built in NUnit Test Runner (although I still prefer using the NUnit GUI)...

In later posts, I intend to discuss other features that Alex has given us such as handling javascript alerts and more. Then I intend to focus on maximising test performance, including a look into using "Cassini":http://msdn.microsoft.com/msdnmag/issues/03/01/CuttingEdge/default.aspx so that the tests can be completely self contained and provide access to mocked system components.

You can download SWEA from "http://www.AlexFurman.net":www.AlexFurman.net along with a comprehensive API reference and an easy to use quick-start guide.

%{color:grey}This is my personal experience of SWEA. Use it at your own risk. You should always perform your own evaluation before adopting a new tool or framework.%