When I was a young man I worked as a consultant for the US Navy. One of my first programming assignments was to produce a printed report of some shipboard equipment in order of it’s cost-effectiveness for “the Admiral”. “OK, but I’ll need the algorithm for computing the cost-effectiveness first.” I responded. “Oh, don’t worry about that --- we’ll give you those figures before you write the program.” “So,” I replied, “you want to use the computer as a printing press --- why not just have our secretary type up the list --- it’d be a lot cheaper.” (Computer time was 360 dollars per hour back in the mid-sixties). “No, no , no,” was the response, “This report will have much much more credibility if it comes off the computer.”
Unfortunately, this attitude has not changed much in 50 years. There is an integrity and a legitimacy about the printed page that defies reason, especially if it’s printed by an esoteric technology like a computer --- it’s as if our secondary national motto has become, “In Technology We Trust.” The computer in particular seems to carry an authority on a par with judges, ministers, scientists and government officials. I was reminded of this interesting flaw in human nature by the Nov 19 PR headline, “Government insists full-body scanners are safe”. The gist of the article is that some of us worry that the X-rays used by the full-body scanners at airports are not without risk and airline pilots in particular are concerned about the cancer risks. In this instance the technology in question is an X-ray device and not a computing machine but the same sort of naivety is at work here --- we’re supposed to trust the pronouncements of the appropriate authorities that these machines are safe. However, the article goes on to quote a physics professor as saying, “The thing that worries me the most is what happens if the thing fails in some way and emits too much radiation.” And large doses of radiation can cause cancer. What are the odds of this happening? It’s already happened.
In the early 1980s, several hospitals used a radiation machine called the Therac25 to provide therapy for cancer patients. Instead of helping them, it caused several deaths and burn injuries due to a software bug. There is not enough space here to describe scenario, who the stakeholders were and who was blameworthy but the following three links will be helpful: the first describes the Therac briefly in a list of other software disasters (including a shockingly similar one 15 years later), the second is a more detailed description in Wikipedia, and the third is a link to a list of other links from a Google search.
So, I remain wary of “official” reports stressing the safety of any potentially dangerous technology not only because of documented cases like Therac but from personal experience. As I mentioned above, at an earlier juncture in my career I worked for the Navy as a civilian. In a section of Engineers and Physicists I was the only Mathematician and so it fell to me to write programs to process the effects of the shock wave resulting from underwater explosions on critical shipboard systems .The most exciting part of the job was crawling around in the bilges of destroyers and installing the instrumentation that would gather the shock wave data. Later I would digitize and process it with my programs.
Some of the ships carried nuclear material and so we were all issued special badges to record the amount of radiation we were exposed to on each visit as a safety precaution. So far, so good. At that time, we were told that was radiation was not cumulative over long spans of time so we got fresh badges each time we went to work. As we now know, it is cumulative and maybe the radiation I accumulated made me smarter, stronger and more handsome but I doubt it. The take-away is that the government is not evil but it can be ignorant.
So, while it is true that we can never know the long term effects of any technology with perfect certainty, we can still demand the best efforts from our scientists and technologists to investigate potential safety issues before they unleash it on the public. Have we made any progress in the last 50 years? Some perhaps but, as one of my colleagues used to say when we were working together way back then, “If your car acted like your computer does, you’d sell the *%#!! thing in a minute! “. As our French friends say, “Plus ça change, plus c'est la même chose.”
No comments:
Post a Comment