Now that we know the basics of software services and their basic performance considerations its time we get more familiar with the performance of software services. We will start this post by quick review of what I covered in the last post. I wrote about how software services have some traits in common (like alwyas ON etc) and how those traits ranslate to certain performance criteria that the services should meet (e.g. memory leaks, maintaining connections to resources etc). Now lets get a little further down this road....
When measuring the performance of software services we need to be cognizant of the following (this can also be treated as a glossary of terms used in this domain)
Production Workload:
This is the load on the service that is expected once the service is released to the world (production). This is generally easy to define, for example the production workload on something like google transaltor service can be a few hundred (700 , i am making a wild guess here) translation requests per second. It is important to keep in mind that the production workload for new services should be the expected traffic to the service 6 months or so into production. Since the traffic or load at the begining of the release might not be a right candidate for production workload.
Stability: The stability of the service is measured by checking for process crashes or hangs while the service is under production workload. Note, while measuring stability we are not looking for performance metrics (for e.g. if under production load a service stays in memory , does not hang or crash, for a gooad amount of time , ususally 24 hours then it is stable even if the response times are higher than expected or the resource consumption is higher than expected.). All we are interested in, when testing for stability, is whether the service functions without a hang or a crash under production workload for a long enough period (usually 24 hours).
Availability: ist the a measure of the the availability of the service to client requests. It does not gurantee that the service would respond to the client's request with the right response, but it does guarantee that the service will be available for a client to make requests against. For example, if a client tries to make a request to the bing maps service and can not reach the service(network timeout or web server time out ) then it should be counted against the availability of the service. However if the service returns a bad response that is not counted against its availability.
Reliability: The measure of the service to respond with the expected responses to the client requests. In the example above, incorrect responses count against the reliability of the service. Normally reliability can be measured by using a generic formula. like this , Reliability = 1- (number of response errors/total number of responses) * 100. Most services on the web have a reliability around 99.99 %.
Scalability: Since services are supposed to live forever in the wild, (read in the real world) and since the load or the number of client requests can increase at any time without an advanced notice, it is desirable that a software service be scalable. In other words, a software service should be able to handle more load by adding more hardware resources to it (if needed). For example, the number of subscribers to facebook now (2011) is around 600 million but it sure did not start this way, it started a little known service in Harward with only a few thousand subscribers. This enormous jump in the facebook service's subscribers is made possible by the scalability in the design of the service. In other words to handle the increased number of users all that needs to be done is to increase the hardware (or the number of servers hosting the web service). Scalability is one of the most important features for a service to have. Without the ability to handle large number of users your service will have a real hard time making money. And if you think that all software is scalable...you can make any software work faster by adding more resources (hardware) then you need to look up bottlenecks in software and Amdahl's law.
This brings us to the last feature on our list of software service features....
Cost to Serve: Cost to serve is a measure of the investment (money i.e. hardware costs, operations cost) that goes into supporting a service Vs The money the software service makes for you. For example, if it takes 500 servers to keep the gmail service running and takes 3 person staff working in shifts to maintain it to have the gmail service support 5million users,(to which 10 million ads are shown every week, and 1 % of those ads are clicked, and google makes 35 cents on each click) then the cost to serve
(3 person salary + cost of servers (monthly installment))/ (Total clicks*.35). Ideally , a software service should have a low cost to serve, decrease numerator and try and increase denominator.
Note: There are no straight forward methods to bring the cost to serve down...this is a complicated measure specific to your service...but a very important one ..ask anyone who pays the bills in your place of work.
I hope the last two posts can explain the performance considerations to keep in mind when thinking about software services....
Happy New Year everyone.
Friday, December 31, 2010
Wednesday, December 29, 2010
Peformance Testing Software Services-I
I think it is important to discuss how software services are different from software programs and in particular map the differences between services and programs in the domain of software performance. So let's begin....
What is a software service? (How is it different from a software program e.g. Microsoft Excel)
Examples of software services range from google's translator to XBOX live from Microsoft, from google maps to Yahoo answers. Given that this is a diverse group, it is still not difficult to outline the properties that all of the above have in common :
For a service to be always ON , or always available it has to reside in memory of a computer(multiple computers) all the time, it(some instruction inside it) should always be in the ready state to be picked up by the processor, it should have the connection to all the resources (database, file shares etc) open at all times (this is not a requirement but it is an optimization that can be assumed for most services). This has some very important connotations specially with regard to physical memory. The memory used by the service should not keep increasing , in other words new memory should be used and then returned back to the OS after some time. Let me explain this a bit...if the memory usage of the service keeps going up with time it will eventually exhaust the physical memory limit for a process. (Remember memory management in OS, see windows internals by Microsoft press for a refresher). This, what i just desrcibed, is a common issue plaguing software services and it is called a Memory Leak. A memory leak is bad for any kind of software but especially services, since they are always ON ( I can't emphasize that enough)
It's not just memory leaks, imagine that we are designing a weather web service , which is accessible through a web browser. The weather service gives you the average temprature at your location by deducing your location based on your IP address (this is called reverse IP and most web portals use it). Let's say this IP-geo location map is stored in the database. Every time a user accesses the web service, the users IP address is looked up in the database and an associated location is retrieved. Since this is a service and is always ON, we would need to always have an open connection to the database (establishing data base connections is expensive).
I hope this makes you appreciate the performance traits of software services. In the next post I will talk more about the performance of software services.... so long till then.
What is a software service? (How is it different from a software program e.g. Microsoft Excel)
Examples of software services range from google's translator to XBOX live from Microsoft, from google maps to Yahoo answers. Given that this is a diverse group, it is still not difficult to outline the properties that all of the above have in common :
- They are always ON (always there), in other words there is no down time
- They are accessible from everywhere (most respectable services are available globally, but even if you think about the US, most of these services are as available in washington state as they are in florida)
- They can support multiple users at the same time, (most of these services have thousands of users active at any given unit of time)
To summarize , a software service is a piece of software that is always available(always in memory of some coputer server somewhere) and is accessible to all its clients (via some agreed upon protocol) and can support multiple users at the same time.
Performance of software services: Now that we know what traits software services have in common, lets ask ourselves what that means for the performance of such services.
For a service to be always ON , or always available it has to reside in memory of a computer(multiple computers) all the time, it(some instruction inside it) should always be in the ready state to be picked up by the processor, it should have the connection to all the resources (database, file shares etc) open at all times (this is not a requirement but it is an optimization that can be assumed for most services). This has some very important connotations specially with regard to physical memory. The memory used by the service should not keep increasing , in other words new memory should be used and then returned back to the OS after some time. Let me explain this a bit...if the memory usage of the service keeps going up with time it will eventually exhaust the physical memory limit for a process. (Remember memory management in OS, see windows internals by Microsoft press for a refresher). This, what i just desrcibed, is a common issue plaguing software services and it is called a Memory Leak. A memory leak is bad for any kind of software but especially services, since they are always ON ( I can't emphasize that enough)
It's not just memory leaks, imagine that we are designing a weather web service , which is accessible through a web browser. The weather service gives you the average temprature at your location by deducing your location based on your IP address (this is called reverse IP and most web portals use it). Let's say this IP-geo location map is stored in the database. Every time a user accesses the web service, the users IP address is looked up in the database and an associated location is retrieved. Since this is a service and is always ON, we would need to always have an open connection to the database (establishing data base connections is expensive).
I hope this makes you appreciate the performance traits of software services. In the next post I will talk more about the performance of software services.... so long till then.
Tuesday, December 14, 2010
Basics , definitions and directions
What is software performance?Software performance is a measure of three kinds of metrics
Latency/Response Time/ Round trip Time etc. This basically means how fast is your software, for example, how fast does your website's home page load on a browser connected to broadband network, or how quickly can your software service complete an operation (for example how quickly does your blog post get published once you click the publish button).
Throughput/Bandwidth: How many concurrent operations can your service (applied more to services) process at the same time. Example how many users can be active on facebook at the same time, or how many users can send an email at the same second in yahoo mail or how many people can search for some reality show star on google at the same time... (this limit is usually crossed by DOS attack).
Efficiency: How efficiently does the software use the resources available to it while working at its peak load. Resources available to a software include local machine resources ( the machine on which the software is hosted) or resources available over a network. Example how much disk space is used by youtube.com to host the latest cute kitty video. Or how much memory is used by the Browser favuorites service to serve 250 million windows live users.
Why is software performance such a big deal?
Firstly and obviously users (= money) like fast software, whether it be a web service or spread sheet on the desktop.
Secondly, and this is not one that everyone thinks about, better performance often means more optimized resourse usage, hence it lowers the cost to serve for software services.Example, facebook.com would make lot more money if they could reduce the number of servers hosting the facebook service by 5%. Remember, cost to serve is a very important to managers, and you can use this to make them see why software performance is such a big deal.
What is performance testing?
Performance testing is the process of measuring software performance, it involves the following:
1. Very Important: Having performance goals.
2. Simulating the correct conditions under which to test software, this includes loading the right amount of data, having the correct amount of load, and the correct conditions (e.g. network latencies , important for web site performance).
3. Collecting performance information (like logs, ETW traces, profiler traces etc)
4. Analyzing collected information and validating the performance against goals.
There are a lot of terms thrown around in the performance testing field and before I talk further it would make sense to try and define a few most common ones.
Definitions:
Stress Test: Testing a piece of software by incresing the load on it (example number of concurrent users) till the software starts failing.
Stability Test: A performance test that validates how long can a service run without crashing under normal load expected in the real world.
Availability: is defined as a measure of how available a software service is.
Reliability: is defined as a measure of how reliable a software service is.
The difference is a service can be available 100% of the time but might only return a valid response 50% of the time. Think of availability as a measure of 404 error in http, where as reliability a measure of 500 erroc code in http.
Resource Leak: This happens when resources used by the software are held on to even when they are no longer being used. Examples include memory leaks, thread leaks etc.
Directions:
The rest of this blog is going to talk about specific topics on performance testing, some basic and some advanced but all interesting.
Latency/Response Time/ Round trip Time etc. This basically means how fast is your software, for example, how fast does your website's home page load on a browser connected to broadband network, or how quickly can your software service complete an operation (for example how quickly does your blog post get published once you click the publish button).
Throughput/Bandwidth: How many concurrent operations can your service (applied more to services) process at the same time. Example how many users can be active on facebook at the same time, or how many users can send an email at the same second in yahoo mail or how many people can search for some reality show star on google at the same time... (this limit is usually crossed by DOS attack).
Efficiency: How efficiently does the software use the resources available to it while working at its peak load. Resources available to a software include local machine resources ( the machine on which the software is hosted) or resources available over a network. Example how much disk space is used by youtube.com to host the latest cute kitty video. Or how much memory is used by the Browser favuorites service to serve 250 million windows live users.
Why is software performance such a big deal?
Firstly and obviously users (= money) like fast software, whether it be a web service or spread sheet on the desktop.
Secondly, and this is not one that everyone thinks about, better performance often means more optimized resourse usage, hence it lowers the cost to serve for software services.Example, facebook.com would make lot more money if they could reduce the number of servers hosting the facebook service by 5%. Remember, cost to serve is a very important to managers, and you can use this to make them see why software performance is such a big deal.
What is performance testing?
Performance testing is the process of measuring software performance, it involves the following:
1. Very Important: Having performance goals.
2. Simulating the correct conditions under which to test software, this includes loading the right amount of data, having the correct amount of load, and the correct conditions (e.g. network latencies , important for web site performance).
3. Collecting performance information (like logs, ETW traces, profiler traces etc)
4. Analyzing collected information and validating the performance against goals.
There are a lot of terms thrown around in the performance testing field and before I talk further it would make sense to try and define a few most common ones.
Definitions:
Stress Test: Testing a piece of software by incresing the load on it (example number of concurrent users) till the software starts failing.
Stability Test: A performance test that validates how long can a service run without crashing under normal load expected in the real world.
Availability: is defined as a measure of how available a software service is.
Reliability: is defined as a measure of how reliable a software service is.
The difference is a service can be available 100% of the time but might only return a valid response 50% of the time. Think of availability as a measure of 404 error in http, where as reliability a measure of 500 erroc code in http.
Resource Leak: This happens when resources used by the software are held on to even when they are no longer being used. Examples include memory leaks, thread leaks etc.
Directions:
The rest of this blog is going to talk about specific topics on performance testing, some basic and some advanced but all interesting.
Subscribe to:
Posts (Atom)