Expert Testers

When I’m planning upcoming tasks for performance, scalability, or reliability testing, the first thing I do is learn the architecture of the system I’ll be working on. This helps me figure out the areas of the system that are most likely to fail.

How do I learn system architecture? I follow the data. Data has three states: it’s either at rest, in use, or in motion. Data at rest is stored in a database or on a file system and is infrequently used; data in use is stored in a database or on a file system and is frequently used; and data in motion is being transmitted between systems or stored in physical memory for reading and updating.

Here are some examples:

• Data In Motion
  • A client application calling a web service.
  • A client mail transfer agent (MTA) sending an email message to a server MTA via the SMTP protocol.
  • One process calling a COM object in another process.
  • An email message being stored in RAM, so it can be scanned for viruses or spam.
  • A log being stored in RAM, so that it can be parsed.
  • Customer data being queried from a database and aggregated for presentation to the user.
• Data In Use
  • Customer transactions stored in a database.
  • Program logs stored on disk.
• Data At Rest
  • Archived databases

So how does this help in planning a testing effort? Usually after I learn and document a system architecture, the obvious weak areas identify themselves. Here are some example epiphanies:

• “There will be 100 clients calling into that server’s web service…I wonder what the performance of that server will be? And I wonder what would happen if the service were unavailable?”
• “That data is being stored in RAM during the transaction. How big can that data get? Will it exhaust the machine’s physical memory?”
• “That data in RAM will be processed N times…how much CPU will that transaction take?”
• “Those logs will be archived to the file share daily. How much data will be produced each day? Does that exceed the size of the file share?”

Following the data helps me quickly learn the architecture and plan the testing effort. What things do you do in order to learn system architecture?

Filed under: Performance | Tagged: Performance, Reliability, Scalability, Test Planning | 1 Comment »