UI Testing Checklist

Posted on June 2, 2014 by Andrew Schiano

When testing a UI, it’s important to not only validate each input field, but to do so using interesting data. There are plenty of techniques for doing so, such as boundary value analysis, decision tables, state-transition diagrams, and combinatorial testing. Since you’re reading a testing blog, you’re probably already familiar with these. Still, it’s still nice to have a short, bulleted checklist of all the tools at your disposal. When I recently tested a new web-based UI, I took the opportunity to create one.

One of the more interesting tests I ran was a successful HTML injection attack. In an input field that accepted a string, I entered: <input type=”button” onclick=”alert(‘hi’)” value=”click me”>. When I navigated to the web page that should have displayed this string, I instead saw a button labeled “click me”. Clicking on it produced a pop-up with the message “hi”.  The web page was rendering all HTML and JavaScript I entered. Although my popup was fairly harmless, a malicious user could have used this same technique to be, well, malicious.

inject

 

 

 

 

 

 

Another interesting test was running the UI in non-English languages. Individually, each screen looked fine. But when I compared similar functionality on different screens, I noticed some dates were formatted mm/dd/yyyy and others dd/mm/yyyy. In fact, the most common bug type I found was inconsistencies between screens. The heading on some pages were name-cased, while others were lower-cased. Some headings were butted against the left size of the screen, and others had a small margin. Different fonts were used for similar purposes.

Let’s get back to boundary value analysis for a minute. Assume you’re testing an input field that accepts a value from 1 to 100. The obvious boundary tests are 0, 1, 100, and 101. However, there’s another, less obvious, boundary test. Since this value may be stored internally as an integer, a good boundary test is a number too large to be stored as an int.

My UI Testing Checklist has all these ideas, plus plenty more: accented characters, GB18030 characters, different date formats, leap days, etc.. It’s by no means complete, so please leave a comment with anything you would like added. I can (almost) guarantee it’ll lead you to uncover at least one new bug in your project.

 

Click this images to download the UI Testing Checklist

Click this image to download the UI Testing Checklist

Continue reading →

Filed under: Security, Test Planning, Testing Tools | Tagged: , , , , , , | 2 Comments »

Security Automation: How Can it Help Your Team?

Posted on February 28, 2012 by kristianjaeger

I have been working as a Software Design Engineer in Test for a little over six years now and I wanted to share some of my experiences with security automation.  I have worked on several different on-premise and cloud-based products, the most recent being a ratings & recommendations service for the Sky Player on Xbox, Xbox.com, and the service for the Xbox Companion application.

Security automation is not all that different from other forms of test automation.  It can be used on code that is still under development as well as existing code that may have changed.

The Microsoft Security Developer center is a great resource for testers, developers, and project managers.  It has getting started guides, tools, articles, tips, and more.  Please check it out.  The Trustworthy Computing site is a good resource as well.

For security automation, feel free to start automating what you can and then iterate and improve from there.  I think that you will find that once the automation is written that it will save your team time and help find bugs.

Here are some ideas for getting started.

  • CAT.NET: CAT.NET is a static analysis tool that helps find Cross-Site Scripting issues, amongst others.  It can be run from a cmd window.  So, one approach to automating this process would be to do the following: 1) create C# console project, 2) have that code accept a drop path as an argument to the tool, 3) iterate through each .dll and/or .exe file of interest, 4) save the files in a directory of your choosing.  One could probably use PowerShell or other tools as well.  You could also consider having this run as a post build step.
  • FxCop: FxCop can also help pinpoint security issues, using its security rules.  It also has a command-line option called FxCopCmd.  So, using a batch file, PowerShell, or other means one could automate this as well.  This could be a good candidate for running as a post build step also.  It is also possible to have FxCop run as a pre-checkin step.
  • BinScope: This is another useful security tool that analyzes binaries for SDL compliance.  It can be launched from a cmd window and the results can be piped to an XML file.  Again, a batch file or PowerShell script could be used to automate this and then also be added to a post build step set.

Using this sort of automation has saved me a considerable amount of time in the past when I was acting as a “security champion” for my team, as opposed to running the tools manually.  Currently, the Xbox.com team has CAT.NET and FxCop setup as part of the build process.  By doing this they are able to catch issues early that were caused either by code changes or additions.  It takes some time to set it up though as sometimes there are false positives or known warnings that can be suppressed if appropriate.  Also, in my current role I have developed internal security automation that is relevant and helpful to my team.

Security automation has a lot of possibilities.  I hope that you find it a helpful addition to your testing toolbox.  Cheers.

Filed under: Security | Tagged: , | Leave a comment »