«Running Head: Analyzing A Novel Expertise Analyzing a Novel Expertise: An Unmarked Road Wayne D. Gray George Mason University & Susan S. Kirschenbaum ...»
Conclusions The current chapter has concentrated on the difficulties of doing a deep-level cognitive task analysis of a novel expertise. The difficulties are all the more notable in that our team of researchers brought to the study considerable expertise in cognitive theory and in applying cognitive theory to real-world tasks. Prior to working on Project Nemo, the co-authors of this
small-unit tactical team training, phone company operators, as well as more traditional decisionmaking studies of submariners and school children.
Although the road was difficult and unmarked, we have arrived at our destination. Our current characterization of the AOs’ expertise – schema-directed problem solving with shallow and adaptive subgoaling – is both simpler and more profound than what we had envisioned when we began our journey. As far as we can tell, this characterization is unlike any that appears in the literature on expert performance. As such, it is important that those who are designing the command workstation understand this characterization of the AOs’ expertise rather than designing an interface that will support the consideration of multiple hypotheses (as in medical diagnoses) or the in-depth exploration of several alternative courses of action (as in chess playing).
Instead of telling stories about how difficult our trip was, we would rather give the reader a sure-fire guide to plotting a safe path to any destination, on any road, marked or unmarked. We do not know if such a guide can be written. However, we are sure that we cannot write one.
Unfortunately, the truth remains that, whatever may be done differently, the task of understanding a hitherto unstudied expertise will never be quick or easy. The problems discussed in this chapter can be anticipated but not avoided.
Notes Acknowledgment More than usual, we thank our agency sponsors and our scientific officer for understanding
results justify their long-term support of this effort. However, we also understand that there were times when they might have thought that a successful outcome was unlikely. We thank Brian D. Ehret who joined our project about two thirds of the way through the events recounted here. Brian was the third encoder on each of the final encodings of the transcripts. His diagnoses have guided the current phase of data collection. The work on this project at George Mason University was supported by a grant from the Office of Naval Research (#N00014-95-1-0175) to Wayne D. Gray. Susan S. Kirschenbaum’s work has been jointly sponsored by Office of Naval Research (ONR) (Program element 61153N) and by Naval Undersea Warfare Center's Independent Research Program as Project A10328.
Authors present address Wayne D. Gray can be reached at Human Factors and Applied Cognitive Program, George Mason University, MSN 3f5, Fairfax, VA 22030, USA. E-Mail: firstname.lastname@example.org. Susan S.
Kirschenbaum can be reached at Naval Undersea Warfare Center Division Newport, 1176 Howell
St., Code 2211, Building 1171/1, Newport, RI 02841-1708, USA. Email:
End Notes The initial task analysis can be as informal as it has to be, but should be as formal as possible. For Project Nemo, the initial task analysis was based on the published literature (Kirschenbaum, 1990; 1992; 1994).
2 After a target is detected, it must be localized. Detection tells the AO that a target is out
speed. Due, in part, to the physics of sound transmission underwater and the need to remain covert, localizing a target is a mathematically underconstrained problem. Passive sonar is the only tool available to the AO. From passive sonar, the AO can directly compute the bearing of the target. Computing the target’s range, course, and speed is a difficult process.
As we think of the schema in terms of ACT-R mechanisms, the schema would be a body of task-specific, declarative memory elements and productions. Any given declarative memory element is relatively small and limited. However, the set of task-relevant, declarative memory elements have high interitem association values (see Anderson & Lebiére, 1998).
As described by Lovett (1998), this adaptive subgoaling can be modeled in ACT-R 4.0 as the temporary depression and recovery of the expected value of a goal.
The simulation required extensive training to operate. Rather than teaching AOs this esoteric task, the simulation was run by an experimenter in the role of own ship operator. This arrangement mimicked procedures onboard submarines and was acceptable to all of the AOs.
Anderson, J. R., & Lebiére, C. (Eds.). (1998). Atomic components of thought. Hillsdale, NJ:
Annett, J., & Stanton, N. (1998). Introduction to this Special Issue on Task Analysis.
Ergonomics, 41(11), 1529-1536.
Ehret, B. D., Gray, W. D., & Kirschenbaum, S. S. (1999). Scaled worlds and computational cognitive models as tools to increase the usefulness of cognitive task analysis. manuscript
Ehret, B. D., Kirschenbaum, S. S., & Gray, W. D. (1998). Contending with complexity: the development and use of scaled worlds as research tools, Human Factors and Ergonomics Society 42th Annual Meeting. Santa Monica, CA: Human Factors and Ergonomics Society.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. (Revised ed.).
Cambridge, MA: The MIT Press.
Gray, W. D., John, B. E., & Atwood, M. E. (1993). Project Ernestine: Validating a GOMS analysis for predicting and explaining real-world performance. Human-Computer Interaction, 8(3), 237-309.
Gray, W. D., Kirschenbaum, S. S., & Ehret, B. D. (1997a). The précis of Project Nemo, phase 1:
Subgoaling and subschemas for submariners, Nineteenth Annual Conference of the Cognitive Science Society (pp. 283-288). Hillsdale, NJ: Erlbaum.
Gray, W. D., Young, R. M., & Kirschenbaum, S. S. (1997b). Introduction to this Special Issue on Cognitive Architectures and Human-Computer Interaction. Human-Computer Interaction, 12(4), 301-309.
John, B. E., & Kieras, D. E. (1996a). The GOMS family of user interface analysis techniques:
Comparison and contrast. ACM Transactions on Computer-Human Interaction, 3(4), 320John, B. E., & Kieras, D. E. (1996b). Using GOMS for user interface design and evaluation:
Which technique? ACM Transactions on Computer-Human Interaction, 3(4), 287-319.
902149): Naval Underwater Systems Center.
Kirschenbaum, S. S. (1992). The effect of level of experience on information gathering strategies.
Journal of Applied Psychology, 77, 343-352.
Kirschenbaum, S. S. (1994). Command Decision-Making: A Simulation Study (Technical Report 10,350): Naval Undersea Warfare Center Division Newport.
Kirschenbaum, S. S., Gray, W. D., & Ehret, B. D. (1997). Subgoaling and subschemas for submariners: Cognitive models of situation assessment (Technical Report 10,764-1).
Newport, RI: NUWC-NPT.
Kirwan, B., & Ainsworth, L. K. (Eds.). (1992). A guide to task analysis. Washington, DC: Taylor
Lovett, M. (1998). Choice. In J. R. Anderson & C. Lebiére (Eds.), Atomic components of thought (pp. 255-296). Hillsdale, NJ: Erlbaum.
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice
Norman, D. A. (1989). The design of everyday things. New York: DoubleDay.
VanLehn, K. (1989). Problem solving and cognitive skill acquisition. In M. I. Posner (Ed.),