TROST: Open-System Trustworthiness

i051002 TROST InfoNote
 What Computers Know
But What's the Use?

TROSTing>info>
2005>10>

i051002c>
0.25 2005-11-09 -22:20 -0700


Bill Anderson and I have a running conversation on the difference between what computers are and the use we make of them.  We've explored that question for a long time, almost the full 30 years we've known each other.  In recent buddy calls between Seattle and Rye, New York, it seems to us that much is starting to crystallize in our thinking.  We want to expand this conversation onto our blogs and engage with others, out loud and on the record.  My kick-off contribution is to draw together some of the themes that solidified as I researched the TROST project.  Here's the sketch.

1. Computers and Embodied Knowledge
    1.1 The Knowing Computer
    1.2 Dumb Enough to Follow Instructions
2. Acting Like a Computer
    2.1 A Digital-Human Program
    2.2 Computer and Non-Computer Rôles
    2.3 Instructing the Digital Human
3. What It Is What It's For
4. Answers Good and Not-So-Good: Calculation vs. Reality

1. Computers and Embodied Knowledge

1.1 The Knowing Computer

1.1.1 I want to inquire into what it is that a computer can be said to know. 

1.1.2 I'm not considering that a computer has any form of sentience.  I am using "knowing" in the sense of embodied knowledge, in the way that one speaks of embodied action (Denning 2002; Gladwell 2005).  It's akin to our talking about what your wristwatch says the time is, as if the watch knows how to tell time, as if it knows the time.

1.1.3 I am using "knowing" in the sense that our male cat, Teh Amor, knows how to spring five feet in the air from a motionless crouch on the floor and straddle my shoulders, landing perfectly lightly with no harm to either of us (Devlin 2005).  It is in the sense that we can imagine the cat knowing (in the sense of embodying) ballistics and calculus that I want to speak of a computer knowing something.

1.1.4 In particular, if the computer has some innate knowledge, in the way that Teh knows how to leap onto a high perch, what would it be?

1.2 Dumb Enough to Follow Instructions

1.2.1 Computers are commonly spoken of as dumb (that is, stupid) and only able to do what they are told.  This reassuring platitude, not heard so much lately, is both accurate and also quite marvelous.  We've chosen a different avenue so that the marvelous part is not overlooked.

1.2.2 Let's look at the kind of thing a computer is doing when it is doing its thing.  We're not so concerned with how it does it, certainly not why it does it, but simply what it is doing.

1.2.3 This may give us a greater appreciation for just what it is that a computer is told that it carries out in its stupid, unerring way.

2. Acting Like a Computer

2.1 A Digital-Human Program

2.1.1 Every month my bank sends me a digital-human program along with some data.  Let's take a look at that program to develop our sense of what it's like to "merely follow instructions," and exactly what kind of instructions those are.Checkbook Reconciliation Worksheet (click for larger image)

2.1.2 Back when I maintained my checkbook by hand, I would follow a procedure similar to the one shown on the right.  I would carry it out and arrive at a number.  Sometimes I would achieve agreement.  Other times there would be a discrepancy.  When electronic calculators became commonplace my error rate decreased and the overall process went more quickly.  Now I let Microsoft Money figure it out from downloaded bank statements.

2.1.3 When performing manual reconciliations, I developed alternative ways of repeating the procedure when it didn't balance the first time: adding the list from bottom to top instead of top to bottom, adding up blocks of numbers then adding all of the block totals, changing the order in which I reviewed my check book, and other variations such that if I had made an arithmetic error, I would be less likely to make it the same way again.  I also verified all of the numbers once again, both on the bank statement and in my checkbook.  Sometimes I had read the checkbook entry incorrectly.  Also, the amount of the discrepancy often provided a clue for what to look for, such as an outstanding check that I failed to list.

2.1.4 It was rare to find a genuine discrepancy and it could be maddening when one did occur.  I've never forgotten the occasion, over 35 years ago, when I failed to notice that the starting balance of the current month was different than the ending balance on the previous month's statement.  A deposit had been credited to my account, but it wasn't accounted for by a line-item, and I couldn't figure out why I had too much money in the bank when I carried out the reconciliation procedure.  I finally noticed the statement discrepancy.  I was extra-watchful in the next few months to see if it happened again or if the deposit was finally acknowledged.   There were no more hiccups and the unreported deposit never did appear on any statement, though it was reflected in the statement balance.

2.2 Computer and Non-Computer Rôles

2.2.1 Look at the reconciliation worksheet and notice its miniscule rôle in the drama narrated in the preceding section.

2.2.2 It is important to understand that the digital-human part, the part specified in the worksheet instructions, is only a small part of what I as the user of the procedure am engaged in.  In particular, the worksheet instructions say nothing whatsoever about what to do if the calculated ending balance disagrees with my check register balance.

2.2.3 The digital-human part is rote and mechanical (although designed to expose mechanical errors because the procedure is indeed being performed by a human being).  It depends on the correct entry and manipulation of numbers and that is all.  It does not depend on facts other than that.

2.2.4 When there is a discrepancy, the procedure cannot provide advice because there are no facts in evidence on which to base any recommendation.

2.2.5 As the human observer of my digital-human activity, I can verify the data and the calculation.  If discrepancy persists and is consistent, I must look elsewhere for the source of error.  It is fruitless for the author of the procedure to presume where that might be.  (This does not deter software producers from taking liberties in this regard and presenting speculations as designed-in facts, but that's another story.)

2.2.6 Because a person is playing computer and is also the one having a purpose for carrying out the procedure, it may be difficult to separate out the rote procedural part and recognize how purposeless it is when taken by itself.   This is important to appreciate, because it is part of the marvel and mystery of computation.

2.3 Instructing the Digital Human

2.3.1 To understand better what little a computer might know, we are going to take away everything a computer cannot know and see what's left.  It will be set up as the story of a prisoner in a locked room.



Figure 2Setup for Playing Imprisoned Digital Human

2.3.2 Playing the Digital-Human Game.  There is a simple procedure that is followed every time (Table 1). 

Table 1. Digital-Human Play Action


1
. Suddenly, the Green light is flashing and a bell is ringing.


2 In front of you is a list of instructions in a fixed code,
like a secret language.


3. You progress through the instructions and carry out the actions specified in the code.  You know exactly what to do.  You've always known.


4
. The only actions that the instructions can specify involve elements just like the listed instructions that  
a. you can move from the in-slot to a scratch-pad square
b. you can move from a scratch-pad square to the out-slot
c. you can transform in simple ways into derived scratch-pad squares
d. you can inspect for conditions that determine the next instruction to follow
e. you can take as instructions


5. Suddenly, the Red light is flashing and a bell is ringing.  Everything stops.  Nothingness.


6
. Suddenly, the Green light is flashing and a bell is ringing ...

2.3.3 This mind-numbing experience is no problem for the digital-human.  As a computer, it is all there is.  There is nothing else to be concerned about.  That there could even be anything else is beyond its ken.

2.3.4 The challenge of instruction.  As the instructor of the computer, all you can do is signal start and provide coded elements at the input slot and record the elements that appear at the output slot.  You can stop the computer.  These are the only actions available to you.  Your interaction with the computer is extremely limited (Table 2).

Table 2. Restrictive Interaction with the Computer


1
. You Can't Tell It What You Want.


2 You Can Only Tell It What To Do.


3. You Can Only Use
What It Can Already Do.


4
. You Must Communicate in
a. Elements of a Fixed Code That's
b. All It Can Accept,
c. All It Can Act Upon,
d. and All It Can Report


5. There's Nothing That It Can Do
In Its World
That Has Any Bearing On Your World
Except By Coincidence


6. Why You're Having It Do What It Does
Is Unknowable to It

2.3.5 This is the fundamental relationship between computers and those who instruct them or employ them as instruments in something else (such as reconciliation of a bank statement).

2.3.6 This is enough, along with astounding progress in electronics, to accomplish everything that we have achieved since the introduction of the modern digital computer barely more than 60 years ago.  Nothing we know to do has altered that basic arrangement.

3. What It Is What It's For

{AuthorNote: The remaining material is rougher than everything above this point.  It will be completed while the story so far is under review in rough draft.}

affordance, what the program affords.  This quite alien thing. 

In this characterization, the only "meaning" to the computation, in the context of the computer, is simply what it does.  Not what it's for.  Here the computer is a pure "doer," a machine.

We are now close to being able to consider what a computer knows.  We'll look at what a computer (program) can't know, and we will

The digital-human need have no concern for, nor even knowledge of, the intended purpose served by the calculation in order to carry it out.  Let's look at that.  The digital-human, under the conditions described, has no basis for inferring what the procedure is for.

 

4. Answers Good and Not-So-Good: Calculation vs. Reality

 When the procedure is being applied in reconciling a bank statement and a checkbook, uncovering a discrepancy is important.  When there is no discrepancy, does that mean all is right with the world?  Not necessarily.

 First, with regard to the procedure itself, there is the possibility of compensating errors that have the calculated ending balance agree with the checkbook, or be very close.  I have had paired erroneous entries that balanced each other out.  I only noticed because I was looking for a penny and I found a bigger discrepancy.   The compensating error was not dangerous, because ultimately the status of funds in the bank was not impacted.

 How about as a picture of reality?  Does successful reconciliation mean that everything is in order and there will be no problems?  Not necessarily.

 Although the procedure predicts the ending-status of the account when all outstanding transactions are completed, it does not indicate whether an overdraft condition may occur -- may have already occurred -- depending on the order in which deposits and checks clear the bank.   The procedure does not provide any assurance in the matter.  It does not deal with cash flow in any way.  These days, checks are cleared (or their amounts reserved) very quickly, and it is dangerous to count on a deposit to arrive like the cavalry in time to save the day.  These days, it is also possible to review account activity on-line and by telephone to automated response systems.  People I know check their accounts regularly when the balance is dangerously close to zero.

 There are other real-life factors that the procedure does not address.  Any outstanding deposits consisting of payments by check may be returned from the payer's bank because of insufficient funds or any other reason.  Suddenly, not only is the presumed state of the account incorrect, there will be unexpected charges if the bank imposes a fee for failed deposits, not just for your own overdrafts.  And that means I may be bouncing one or more checks and I will be assessed the even-larger overdraft fees.  The spiral continues until I get wind of the situation and find a way to prevent further overdrafts.  Banks in the United States offer special protection against this situation, but having gone decades between overdrafts, I never think to arrange it.

The point is that the procedure provides only a hypothetical prediction of the account's status, even when there are no errors of calculation and the reconciliation is successful.   What the computer "knows" is how to carry out the calculation it is instructed to perform.  There is no guarantee that the theoretically-justified result will have anything to do with the actual course of events, despite the fact that the result is dependable most of the time.

 

5. Thinking It Over: Something to Notice

{AuthorNote: Don't know if I'll bother with this here, in the first published version.  Might add it as a separate commentary page too.}

Could we put layers on top of layers and

The simplifications made for focus and to have the illustrations stand out. 

Almost-arithmetic. 

Could something else emerge.  Perhaps. 

But for now, it is not clear how a computer complex will have enough presence and awareness at the scale of human existence, to interact with the world as something experienced.

Until that interesting event, let's stick with the notion that computers will continue to be instructed by programs in machine language and that these programs, at no matter how many levels of remove, are written by people in a way that what the computer does and what the program is serves a human purpose that is external to and afforded with that.

6. Resources and References

Cunningham, Ward (2004).
Ward CunninghamIs There a Revolution Coming in the Way People Communicate?  Interview, The Videos, Channel 9 Forums, MSDN, updated 2005-08-18 at <http://channel9.msdn.com/Showpost.aspx?postid=8864#8864>.
    
Denning, Peter J. (2002)
Career Redux.  The Profession of IT, column, Comm. ACM 45, 9 (September), 21-26.   Available at <http://doi.acm.org/10.1145/567498.567516>.  Also at <http://www.cne.gmu.edu/pjd/PUBS/CACMcols/>.
     
Devlin, Keith (2005).
The Math Instinct: Why You're a Mathematical Genius (along with Lobsters, Birds, Cats, and Dogs).  Thunder's Mouth Press, New York.  ISBN 1-56025-672-9.
    
Gladwell, Malcolm (2005).
Blink: The Power of Thinking Without Thinking.  Little, Brown and Company, New York.  ISBN 0-316-17232-4.
    
Hamilton, Dennis E. (2004)
Ward Cunningham - Revolution in Communication.  Orcmid's Lair (web log), 2004-05-28.  Available at <http://orcmid.com/blog/2004/5/ward-cunningham-revolution-in.asp>
    
MacKenzie, Donald (2001).
Mechanizing Proof: Computing, Risk, and Trust.  MIT Press, Cambridge MA.  ISBN 0-262-13393-8.
    
Norman, Donald A. (1998)
The Invisible Computer: Why Good Products Can Fail, the Personal Computer Is So Complex, and Information Appliances Are the Solution.  MIT Press, Cambridge, MA.  ISBN 0-262-14065-9.

Revision History:
0.25 2005-10-14-11:50 Provide Rough Draft
There is enough here to invite review from a few people while I polish off sections 3-4.  The resulting update will be publicized more widely by a blog entry, so this page will be frozen on that account and anything else arriving in a new version.  But now, just the rough draft for review.
0.00 2005-10-12-12:32 Create Initial Placeholder
Introduce an initial placeholder that serve as a target of links and include-page components until more content is developed.  This page is a customization of the InfoNote Bootstrap Template 0.20 Material [placeholder] template.  A version from Develop InfoNote Bootstrap Template (0.20) Material was used.

Construction Zone (Hard Hat Area)
Creative Commons License You are navigating TROSTing.org.
This work is licensed under a
Creative Commons License.

created 2005-10-12-05:24 -0700 (pdt) by orcmid
$$Author: Orcmid $
$$Date: 06-08-08 22:20 $
$$Revision: 96 $

Home