Performance - Part 1
December 22, 2008 · by Kevin Runde
Performance Tuning
For years now, I have ended up doing performance tuning. Well, I really call it performance triage. The first one was a Java Web Crawler that was a beautiful OO creation. I mean this looked wonderful on paper and in code. We even had an independent analyst come in and examine the entire project. He looked through the code and the db for over a week. His final report came out and all he could say is we might have problems with too many method invocations. Later, we turned the system on and it blazed across the Internet at 0.2 pages per second. HUH? Yeah, 0.2 pages per second. Eventually we were able to tune the crawler and got it going well over 100 pages per second. Yep, 0.2 to 100 pages per second. Oh, and just in case you are wondering, method invocations were never a performance problem. ;-) Since then, I have been on other projects and the same sort of things happen over and over. Big companies, small companies, medium sized companies, big project, small projects, critical projects, and unimportant projects. They all were the same and they all were solvable. Over the next several blogs I will tell you my tips and tricks for performance tuning.
To start with, though, you need to understand something. This is probably the most important TRUTH I know.
“You don’t know what you don’t know”
Yeah, I know what you are probably thinking. This guy is an idiot. Of course I don’t know what I don’t know.BUT do you really realize it? Truly learning this lesson is a very humbling experience. You have to question every “fact” you use in your thought process and ask yourself how do I know this. What proof do I have? How reliable is this proof? For example, one data point is meaningless; it is just a dot on a graph. I know, “but Kevin, two data points is a line.” So, most people do a test twice and draw their line. I say: “But three tells you if that line is really a curve!” So, the more times you can reproduce a result, the more reliable the “fact” is. Also, in order for anything to even be a “fact,” I say it needs to be reproduced at least 3 times. Also, who did the test? What recorded metric data? Was the test done in the same way and with the same data as last time? What about the code? Is it exactly the same and if so, how do YOU know? Do you know if the network and performance testing systems can really handle the load you are putting on them? Are you sure there isn’t a switch some place that only allows 10 Mbps, but you are trying to generate over 45 Mbps? Are both those speeds Mega Bits Per Second or Mega Bytes Per Second? How many objects are the JSP pages really creating? Does our response time include the time to transfer the data to the client or just how long it takes you to generate the content to send to the user? How often is the Garbage Collector running and for how long? What are the CPU loads on every machine involved (the servers running the application, the database servers, the server recording the metrics, the server running the tests, and each and every box used to play the test)? How much can you really trust your team members? If you don’t know the answers to these questions, then you don’t know what you don’t know. The next posts are going to be about learning what you don’t know. Because once you know what you don’t know, you can come up with a game plan to find the problem. Once you know the problem, it is trivial to create solutions. Yes, as in multiple solutions. Implementing them may not be so easy, but that’s not part of performance tuning.
-Kevin
Filed in: Team Member Blog Comments (0)
Comments
There are no comments for this entry.