Now that we know the basics of software services and their basic performance considerations its time we get more familiar with the performance of software services. We will start this post by quick review of what I covered in the last post. I wrote about how software services have some traits in common (like alwyas ON etc) and how those traits ranslate to certain performance criteria that the services should meet (e.g. memory leaks, maintaining connections to resources etc). Now lets get a little further down this road....
When measuring the performance of software services we need to be cognizant of the following (this can also be treated as a glossary of terms used in this domain)
Production Workload:
This is the load on the service that is expected once the service is released to the world (production). This is generally easy to define, for example the production workload on something like google transaltor service can be a few hundred (700 , i am making a wild guess here) translation requests per second. It is important to keep in mind that the production workload for new services should be the expected traffic to the service 6 months or so into production. Since the traffic or load at the begining of the release might not be a right candidate for production workload.
Stability: The stability of the service is measured by checking for process crashes or hangs while the service is under production workload. Note, while measuring stability we are not looking for performance metrics (for e.g. if under production load a service stays in memory , does not hang or crash, for a gooad amount of time , ususally 24 hours then it is stable even if the response times are higher than expected or the resource consumption is higher than expected.). All we are interested in, when testing for stability, is whether the service functions without a hang or a crash under production workload for a long enough period (usually 24 hours).
Availability: ist the a measure of the the availability of the service to client requests. It does not gurantee that the service would respond to the client's request with the right response, but it does guarantee that the service will be available for a client to make requests against. For example, if a client tries to make a request to the bing maps service and can not reach the service(network timeout or web server time out ) then it should be counted against the availability of the service. However if the service returns a bad response that is not counted against its availability.
Reliability: The measure of the service to respond with the expected responses to the client requests. In the example above, incorrect responses count against the reliability of the service. Normally reliability can be measured by using a generic formula. like this , Reliability = 1- (number of response errors/total number of responses) * 100. Most services on the web have a reliability around 99.99 %.
Scalability: Since services are supposed to live forever in the wild, (read in the real world) and since the load or the number of client requests can increase at any time without an advanced notice, it is desirable that a software service be scalable. In other words, a software service should be able to handle more load by adding more hardware resources to it (if needed). For example, the number of subscribers to facebook now (2011) is around 600 million but it sure did not start this way, it started a little known service in Harward with only a few thousand subscribers. This enormous jump in the facebook service's subscribers is made possible by the scalability in the design of the service. In other words to handle the increased number of users all that needs to be done is to increase the hardware (or the number of servers hosting the web service). Scalability is one of the most important features for a service to have. Without the ability to handle large number of users your service will have a real hard time making money. And if you think that all software is scalable...you can make any software work faster by adding more resources (hardware) then you need to look up bottlenecks in software and Amdahl's law.
This brings us to the last feature on our list of software service features....
Cost to Serve: Cost to serve is a measure of the investment (money i.e. hardware costs, operations cost) that goes into supporting a service Vs The money the software service makes for you. For example, if it takes 500 servers to keep the gmail service running and takes 3 person staff working in shifts to maintain it to have the gmail service support 5million users,(to which 10 million ads are shown every week, and 1 % of those ads are clicked, and google makes 35 cents on each click) then the cost to serve
(3 person salary + cost of servers (monthly installment))/ (Total clicks*.35). Ideally , a software service should have a low cost to serve, decrease numerator and try and increase denominator.
Note: There are no straight forward methods to bring the cost to serve down...this is a complicated measure specific to your service...but a very important one ..ask anyone who pays the bills in your place of work.
I hope the last two posts can explain the performance considerations to keep in mind when thinking about software services....
Happy New Year everyone.
No comments:
Post a Comment