This article is a description of some of the technical meltdowns I had to deal with and their effects on those companies. Then I outline the probable effects to come from technical financial meltdowns on all of us...

When I worked for Oracle Corp. as a systems engineer, my job was to save companies that had damaged their databases, and of course a myriad of technical sleuthing in general. I sort of figured out what a "systems engineer" meant as a title by all that diverse work...

One thing which impressed me from this work was, above all: The VOLATILITY of essential company information upon one stupid mistake by a database administrator. It was always the dumb human error that turned out to be a gotcha!...or, your "going out of business in one day if you don't get this up!".. stuff.

It was the dummies that were fatal to all of them and their customers. It's not exactly the best way to get to the bottom of the emergency when you NEED those dummies to reveal their fatal error. One time, I had to review the system log at the latest crisis I was dispatched to, and found it had inexplicably been erased, a thing impossible unless ordered... well, I mentioned the erasure in the after action report; who knows if the powers that be there did anything about it. Usually I had to be oblique about it, for political reasons... why such critical realities are subject to internal suppression is frankly beyond my understanding.

I would go into these emergencies and have the database administrator, the systems administrator, the technical VP, the general VP and so on and usually about ten panicky technical people looking over my shoulders as I typed at the keyboard... all of them in a friggin panic, trying to hang back but having a hard time of it. I guess I understand that but, it sure didn't help me! Such situations are exactly like flying a jet fighter, a PURE ADRENALIN RUSH.

There was this internet company that had a feature on very large internet company's web site. Sort of a co-service.. Well, they encountered a problem with their system not shutting down so the guy issued the command to shut down Oracle. That overrode the first problem and was not a biggie. But then he had to restore a backup of his data, which is normal. One small problem. All of their back up sets were corrupted because he had not done them with one simple normal procedure. So, he made a dumb mistake and he's calling me, literally panting on the phone, and 'very large internet company' is on his case cause the button isn't live anymore at their web site.

"I have to get this up today or we are out of business!"

Well, to make a long story short, I had to dispatch another engineer to some place back east in a jiffy and get a hold of that engineer because Oracle's offices were closing, and I had to page him... and I had to convince the victim's company on conference call to shell out $10,000 per day for an emergency database rebuild with no guarantees because they didn't do the backups right.

Information volatility... Information time dependence... ie if its not alive when its needed then forget about it anyway. And so on. This translates simply into MONEY volatility, volatility not being VOLUME, rather being DESTRUCTION BY FIRE. This is a reason why I believe that electronic deflation will erase trillions in a day.

Now, I remember other situations which I'm not going to pontificate about, perhaps because of security reasons.... or just my better judgment. But I'm using this internet company example for illustration. But, I will say that the defense industry is definitely capable of this kind of stupid error.

What about the Banking Industry? same issues... ones I personally know about. On and on and on.

Now, Oracle is very robust, and safe and also relatively simple to run reliably. Actually, the algorithms for handling data and getting it saved redundantly are quite clever, even ingenious and safe. But what do you do about a simple human error? I've dealt with situations where the involved organization would go out of operation in a matter of hours if it wasn't fixed...

Fast forward to our economy. I wrote several GOLD-EAGLE.com editorials about this information phenomena. One was Brave New World. One was the "Time Value of Money," - (www.gold-eagle.com/editorials_04/laird103104.html) that one is probably worth a book, and a thesis. Basically, this is the issue: MONEY TODAY IS VOLATILE. Our financial world is designed to be robust, but is subject to two problems that are not addressable right now: one, human error. Two, complexity.

The first I have described, is human error, and frankly there is no answer for this. In reality, this means that human error WILL occur, and randomly, even in the most secure and essential information and database installations. As I said, the human errors are usually the worst, and act like an asteroid hitting an organization in a fraction of a second. One minute your all having a party, and drinking champagne, successful. The next, OUT OF BUSINESS!. Just like an asteroid from space hitting the operation. Believe it or not, Oracle has best practices that even address the real asteroid eventuality! But human error? there are often no recoveries from them.

Now then, what does all this mean? Well, for one thing it means our little smug life can turn on a dime, especially if its dependent on the transfer of massive amounts of TIMELY INFORMATION. So what does NOT fall into this category today? With JUST IN TIME MANUFACTURING being the way we get all our goods, and SERVICES the same today in the US? Um, where would you be without your services right now? Your banking, your online accounts, and on their side as well.

Add to this all the monstrous COMPLEXITY of the electronic world, and economy. Complexity has all its own associated issues. There is a fact called system panics. Basically a system panic is where, in the case of a large scale operating system, a chaotic threshold is reached on allocating system resources, like memory reads, etc. The system goes from normal to panic in seconds... The phenomena has been shown to have mathematical chaotic thresholds.

Well, the system administrators try to keep things with in the known thresholds so there is no panic state. But what if there are cascading losses of information in a banking crisis? Do you think the system administration will avert one catastrophe from cascading into system panics, if the NORMAL OPERATING LOAD of the financial clearing systems clearing trillions of dollars in currencies each day at the BIS spirals up exponentially in a financial world panic?

Hey, I'm saying here that even if all the transactions did clear without default problems, the mere exponential increase from a panic in transactions will cause some system panics and just add to the chaos. All of a sudden, bank 'A' cannot clear ANY transactions. The systems don't allow a buildup of failed transactions, they shut down in such cases, and hopefully in an orderly manner.

So I guess I'm saying that in very stressful economic markets, like crashes or a real currency crisis we've not yet seen on this scale today, things will deteriorate very quickly, like in maybe an hour. I think this situation is not just a possibility, but is likely, and it will manifest itself in the news as, "financial crisis". They will definitely not tell you all this stuff. They will be too busy. So I'm telling you in advance.

If this comes down like this, the implications to the world economy will be devastating losses of timed financial information, IE irrecoverable losses that will be realized in the sense at first of a system failure, then a cascade of say, derivatives losses... then the FED jumps in and does what, sends out 200000 systems engineers with the checks to clear the mess? NOPE.

By the way, I've had some readers ask me about my background... Ok here are some things: I come from a family of writers. But I have focused my energy on getting a mathematics degree, teaching math, then moving into the computer field and becoming a systems engineer for Oracle Corp for two years. I'm an Oracle certified database administrator. My Grand mother, Alice Widener, published USA magazine (not to be confused with USA TODAY), and was talking about many of the topics we see here at GE in the 60's and 70's. And my father is Jere Laird, a relatively famous financial editor of KNX news in Los Angeles. So, I have grown up basically immersed in financial issues, but I have recently started to work on writing about it. This is a synopsis. I have always had a fascination with financial matters, and it started when I was about 24, and read A. Fekete's monograph entitled "Borrowing Short and Lending Long, Illiquidity and Credit Collapse" - (www.gold-eagle.com/gold_digest_02/fekete100702.html)in 1984. So I'm just starting my writing career on financial topics.

But I'm not some financial guru and I don't give financial advice. I consider myself a commentator. Basically, if I was to ask you to make a value judgment on me, make it on my GOLD-EAGLE.com editorials.

December 29, 2004

Chris Laird
tec_10000@yahoo.com