TripleA (Application Automation Api) provides building blocks for easily creating developer owned system & deployment smoke tests.
Use these components directly or build your own business aware components to provide system level actions to use with your favourite test framework.
Binaries available only
as a set of NuGet packages
. Full source code available from the "Source Code" tab (HG repo).
- TripleA package contains the core components and can be used for all test frameworks
- TripleA.StoryQ package contains components that make it easier to use with the StoryQ framework.
- This will also install the TripleA and StoryQ packages
A large number of "bugs" & issues occur when software transitions from one environment to another, infrastructure, configuration, deployment and state problems can cause failures and inconsistent test results. Outside the comfort of a unit test there are many
more "variables" that come into play.
Exercising your software at an integration or system level proves that the software actually performs and delivers as expected...however, once you increase the scope of a test and multiple components or sub-systems are in the mix you need to ensure that each has the expected state prior to the test otherwise all you achieve is brittle tests that produce inconsistent results.
There are a number of test frameworks and applications that can interact with software at a system level, eg: WatiN, Selenium etc but my experience with these tools is that they are often set up by skilled test engineers and then fall into disrepair as often the skills to maintain it are lost to the development team - not all testers are created equal and good test automation engineers are hard to come by!
TripleA tries to bridge the gap to allow developers
to easily write system smoke tests and deployment verification tests that "prove" the system is a "good build" and is operating sufficiently well that you can commit your test team to further in-depth testing - a very good summary can be found here
about how valuable rapid fail and feedback is.
- Use your existing test framework (NUnit, xUnit, MSpec, StoryQ etc) & test runner .exe.
- Use TripleA components to Arrange, Act and Assert at a system level.
- Write coarse grained tests that prove the system is "working" at a basic level - eg: pages can be called, a web service responds, information can be read/written to the database.
- Assert that the infrastructure is configured correctly, that databases have the correct permissions set for your application.
To be clear, TripleA is not a new test framework
- it just allows you to create reusable components for use in your existing framework
that can manipulate your system state (arrange) and assert things about it.
This screenshot shows some real deployment tests in action. Here I'm using plain old NUnit tests in a "Context/Specification
The tests cover two sub-systems, a web application and a windows service. Each has separate sub-components that we need to verify has been deployed correctly including database, file system permissions and service identity. As you can see, "context/specification" makes the test results layout easy to read and navigate.
As I've previously discussed
- this approach offers some really interesting benefits...
- It makes it very clear what is failing - you know immediately where to start investigating
- You can write these tests up front - TDD = Test Driven Deployment! As the developer you know what needs deploying so encapsulate this in a test, when it goes green you know this task has been accomplished!
- Using a test to declare the deployment requirements also enables you to hand off the implementation of the deployment to the best suited team member/group.
One of the problems that TripleA sets out to solve is to reduce the complexity and code required to quickly arrange, act and assert at a "system" level; the challenge is to wrap up quite complex arrange, act, assert blocks of code into simple one liners that can be used within your test framework. For instance the amount of code to call a web page, parse the html and assert something within it is not trivial - you want this wrapped up into the smallest repeatable parameterised statement possible...this is what TripleA delivers.
In the above tests, the code to call the web application home page and assert it returns 200 OK...
cfg.Credentials = CredentialCache.DefaultCredentials;
cfg.Uri = "http://webserver/mrm";
cfg.ExpectedStatus = HttpStatusCode.OK;
...to check the database has the right user is simply...
...to check the windows service account has the correct folder permissions...
_tripleA.FolderPermissions(@"\\FileServer\AppShare", @"SomeDomain\SomeUser", AccessControlType.Allow, FileSystemRights.ListDirectory | FileSystemRights.Read);
The real power of TripleA is that it can be fully customised to create arrange, act or assert extensions that encompass your
domain. TripleA is designed around using extension methods so you can continue to use it as your system grows. You want your dev team to be able to roll new tests quickly so start building reusable components that are meaningful to your system, eg: "add a new customer". TripleA provides some basic extensions but these can be wrapped up and parameterised so that they can be dropped into your tests to quickly arrange or assert something meaningful to your system.
TripleA comes with a bunch of pre-installed extensions...the full list (and details on each one) can be found here
Want to know how you write your own extensions? This page
has all the info you need.
page contains the full list of help pages to get you started with TripleA.