Small step for a man: Defining and Evaluating "Usability" of Digital Libraries: Ferreira and Jeng

I read in the Christian Science Monitor http://www.csmonitor.com/2006/1002/p03s03-nbgn.html that new digital audio technology had been used to "listen again" to Neil Armstrong's famous moon-landing speech. Apparently, Armstrong has long claimed to have said the landing was "a small step for A man..." and that the "A" has been left out of the quote since it happened. According to the article, and Australian computer programmer used sensitive digital sound equipment to enable listeners to hear the omitted "A" in the quote for the first time ever. This "discovery" made me wonder about the effects of technology on our perception....What else have we been missing? What does it mean to hear and see what can't be experienced with human senses? As we get closer to "ubicomp" will there be more and more augmentation to our senses in everyday life?

This is only tangentially related to these articles, but I was thinking about it at the same time, so, oh well. Want to see me try to connect this story to digital library evaluations? Here goes: The moon landing constituted a literal and metaphoric "new frontier" for science and exploration, reviewing (or rather, re-hearing) the seminal moment of the moon landing using technology developed so many years after the event shows us how our desire to explore and discover often out paces our ability to evaluate and understand what we are experiencing in our discoveries. I will argue that digital libraries are an example of this kind of "frontiering" that is happening so fast and changing our lives so dramatically, that we aren't prepared to evaluate their impact and effectiveness. The evaluation tools and criteria we use for print libraries aren't adequate to usefully analyze digital libraries, just as the sound equipment available at the time of the moon landing wasn't adequate to really hear what was going on...Hm. A ridiculous stretch but I'm kind of proud of it anyway...

The Ferreira and Jeng articles both attempt to define digital libraries and usability and formulate useful criteria for the evaluation of usability in the context of digital libraries. Both articles report research conducted to determine what aspects of a user's experience determine their experience of a "usable" site, and how this quality of usability can be measured. The Ferreira study looks for evidence of synergy between human computer interaction (HCI) theory and information science (IS) theory. This study uses the following criteria for evaluating the usability of digital libraries: learnability, efficiency and effectiveness of the digital library, management of errors, memorability, and user satisfaction. The Jeng article reports similar research evaluating digital libraries on the basis of the relationship between effectiveness, efficiency, satisfaction and learnability. The Jeng study was of particular interest to me because of the inclusion of the issues of user "lostness" and "navigation distorientation" in digital libraries. Digital libraries are new territory, and the maps we devise to navigate and evaluate them must take into account the experiences of users.