At EK, we help organizations solve their Knowledge Management (KM) problems. Typically, part of the solution is an integrated set of tools that improve the way that content and information is shared. These solutions only work when they are widely adopted, and one of the biggest hurdles to adoption is slow performance. Modern professionals are accustomed to Google-like performance, and have little patience for slow performing applications. As a result, it is critical that our applications perform as quickly as possible.

Though just one part of our comprehensive approach, performance testing is particularly important because it allows us to make sure that our applications are tuned to respond as quickly as possible. It is a critical part of every one of our application development projects. It is also one of the most misunderstood things we do for our clients. When we start performance tuning, I regularly hear the following statements.

• “Run a load test so that I can find out how many concurrent users the application will support.”
• “Make sure every page loads in less than 3 seconds.”
• “We will do performance testing the week before we launch to make sure we are good to go.”
• “Why do I need a separate environment for performance testing? Can’t I just use production before we launch?”

If any of these statements resonate with you, please read the rest of this blog post to understand how performance testing really works. The first and most important thing to understand is that there are different types of performance tests, and each of them meets a specific purpose. The most common types of performance tests are:

• Load Testing;
• Stress Testing;
• Endurance Testing; and
• Spike Testing.

Load Testing

A load test is executed to see how a system performs under a pre-specified load. The tester specifies the number of concurrent users and the number of transactions (ie. web hits) that occur as part of the test. Keep in mind that the most important aspect of any load test is the number of transactions that occur, and not the number of concurrent users. This is one of the most misunderstood aspects of load testing. Let me explain:

User A clicks three times to find the page they are looking for and then spends 3 minutes reading it. They then click to another page and spend another 5 minutes reading that page. At the end of 7 minutes User A has clicked on 5 pages.

User B is more of a browser and clicks on pages every 15-20 seconds. Over that same 7 minute time User B has looked at 21 different pages.

As you can imagine, User B puts a significantly heavier load on the server than User A. Now imagine if we talk about 100 concurrent users. If they are all like User A, then the server load will be 500 pages in 7 minutes (relatively small). If they are all like User B, then the server load will be 2,100 pages in 7 minutes. As you can see, the number of users is much less important than the number of transactions they create. All of this gets even more complex when you consider that different types of transactions create different loads on a server. For example, a search is much more work than loading a simple web page.

When defining your load test, use analytics to understand the number and types of transactions that occur on your current site. Develop a load test that mimics those figures with a slight increase and then look at page load time. This more well-defined load test will allow you to understand how your system performs under normal or slightly elevated activities.

Load tests should be run each time new code is added to the system that could reasonably impact performance.

Stress Test

Once you understand how your system performs, you may look for ways to improve performance. Stress tests are great ways to identify the bottlenecks that slow down performance. A stress test is a load test that adds users slowly over a period of time to understand when the system will fail. During a stress test it is important to monitor all of the systems that power the website (servers, database, web server, and applications.) As the system begins to fail to perform, you should be able to see which systems are unable to keep up. Once this is understood, you can focus your tuning activities on the slowest system.

To do correctly, stress tests can take quite some time, as once a problem system is identified and tuned, the test needs to be re-run to see if a separate system creates a new bottleneck. An experienced team will run stress tests on a periodic basis to make sure there is no degradation over time.

Endurance Testing

The primary purpose of endurance testing is to identify memory leaks that could cause system failures over time. An endurance test is a load test that is run over a long period of time to make sure the system does not degrade over time. The load should be at an average or below average amount. Endurance tests can be run for one or more days to make sure the system performance does not degrade or fail over time.

Endurance tests are time-consuming and as a result they are not often used. We recommend that our clients run endurance tests before the launch of a major new application and when memory or degradation issues are found.

Spike Testing

A spike test is a test in which the load on a server escalates very rapidly so that the developers can see what happens when the system is hit by a sudden load. Spike tests are rarely used on internal sites as the loads do not typically go through drastic changes. We recommend spike testing on public sites that may have their load impacted by external influences. Examples include shopping sites with a hot new product or news sites impacted by sudden events.

As you can see there is a lot to performance testing. It is something that requires advance planning and it does not provide easy answers as to how many users your site can handle. It is, however, an important tool to help make your KM system performs the way it should.

If you are looking for help implementing a new KM system or fixing a broken tool, email us at [email protected].