Friday, February 2, 2007

Watching the watchmen

Talk of monitoring tools today got me to thinking.

Back in my C/C++ days, efficiency was my main concern. I wanted things to go as fast as possible. So I couldn't be spending instructions to do things like logging, monitoring my own performance, etc. And my code isn't going to crash anyway, so what's the point, right? Wrong. Of course my system was perfect :), but it talked to other systems that weren't. This meant my programs just didn't seem to do anything. We were completely flying blind since there was no indication of any sort of what it was doing. That was always the main push of my manager at the team (to all of us, not just me): make the systems better at monitoring themselves.

Fast forward a few years and I am in a meeting hearing about monitoring tools that will be able to tell us things we need to know this time. This got me thinking about the nature of this beast. When you write a tool, you are putting in extra code to expose a part of your system. A different interface, or view :) then the normal. You have to, first, guess what might be a problem area so that you can write the interfaces to show it. But, as was implicitly mentioned in this meeting, you are often wrong in your guess about this. Or at least you don't expose enough.

Our systems use the popular *4log libraries to handle logging via configuration. What most programmers do with this is put a log message at the start and end of every (!!!) function. So if you turn on debugging you will literally see every single call the system makes. Is there no better way to do this?

I think there is. In smalltalk, a live system, I can view any part of the code any time I want. This is a good thing, but it was the first thing I saw as a negative when I started using smalltalk. So inefficient with so much of the system exposed! But is it? It has reflection, a way for us to use the system to ask the system things about itself. It's not some hack and it's not something you can "turn off" for production. It is part of it from top to bottom. It also has viewers that are designed to use this reflection to show us parts of the system. While it's running.

So if you think about it, they did the same thing my manager ask me to do years ago, and the same thing the developers at my company are trying to do now: they are giving ways to monitor the system. But the difference is, instead of guessing which parts of the system might have problems that we need to look at, they just gave us the whole thing.

As far as efficiency, sure, raw C++ (and even Java) may be more performant then raw smalltalk. But if end the end you wind up doing things like calling two logging functions per normal function, every time, then how much more efficient is your code? After all, Smalltalk isn't calling into the viewers for every function. Only if you use them. And you never have to go back and add new monitoring tools because you guessed wrong.

No comments: