4.10 Design Seed 7: Using Context-Specific Models to "Seed" Themes

Our "diagnosis" of data overload (Woods et al., 1998) found that an extremely
difficult challenge when addressing the data overload problem is that data is
only informative given a particular context. Context sensitivity plays a central
role in many of our design seeds to address data overload, and makes many of
our ideas distinct from current design directions. We believe that using the basic
human competence for finding what is informative in natural perceptual fields
despite context sensitivity is our guide for innovation. With this approach, our
goals are to use the power of technology:

• to enhance observability,
• to take into account context sensitivity, and
• to build conceptual spaces.

One way to take into account context sensitivity is to use the semantics of
underlying processes or field of activity to help define the relationships that give
data meaning (Vicente and Rasmussen, 1992). For different analytic scenarios,
there will be multiple organizing themes, each of which defines a perspective on
the data field. In the Ariane 501 scenario, there were many potential models at
different levels of abstraction (Figure 29) that could be leveraged in a contextsensitive
approach, including models independent of both the satellite industry
and the specific scenario that could be used in a variety of analysis tasks.



There are a variety of ways that machine intelligence could take advantage of
models such as these to aid analysts during the analysis process. One strategy
would be for the computer to "seed" a display with initial themes to consider
pursuing in the analysis. These themes could come be available to the machine
from a variety of sources:

• a "modelbase" designed into the software
• themes from past analyses that are recognized as similar such as with
case-based recognition algorithms
• Natural Language Understanding (NLU) processing of a written or
verbal question
• A tailored "modelbase" created from a machine synthesis of past analyses
or crafted by an individual or team of analysts.

In addition to seeding potential themes, machine intelligence could be used to
"know" something about themes that helps with their management. For
example, themes can be suggested to be added or removed based on particular
heuristics. Similarly, labels for themes can be suggested from analysis of
documents that are "attached" to themes. Additionally, the machine could be
asked to organize themes based on heuristics about how certain themes relate to
each other (e.g., "background" comes first and "impacts" goes last).

In summary, this design seed has characteristics of:
• Using context-specific knowledge about a domain and/or scenario to
guide searches for information and organization of data
• Encouraging a meta-analysis of themes to include in an analytic product

Benefits to this design seed on performance would hopefully include
encouraging a "meta-analysis" of what themes to include in an analysis task
before beginning. These themes can then be used to better guide the search for
information and the determination of when an analysis should stop. By
determining what to look for in advance, hopefully it would be more difficult for
an analyst to get sidetracked onto tangential themes or to let personal
preferences about interesting topics dominate an analytic product.

As an illustration of how the approach of instantiating design seeds in Animocks
can lead to discussions at the "usefulness" level of design as opposed to
"usability" comments, such as feedback about color choices on a display,
consider the reaction of an expert analyst:

"So, the problem, as I see it here is finding a way to identify a
"theme" and identifying what is significant to that theme and
somehow associating it with the theme. Now, if I have 100,000
documents and identify 100 documents in some theme process; I
notice that I have only got 0.1% of the pile. This may be good; or
this may be bad. I can probably work with only 100 documents.
But the question remains: what is still in the other 99.9%? Is there
anything there that has a bearing on what I know from the 0.1%?
How can I best satisfy myself that the rest is all trash for this
exercise (I might hurriedly do a list of titles and scan them for
whether something will catch my eye or not)?"

This feedback is very important because we can then add another desirable
characteristic of this design seed: that a dataset be characterized by themes so
that the analyst can verify that (s)he has a sense of what themes are available and
what the possibilities are that (s)he has missed a critical theme. As further
evidence that we are engaging in a fruitful discussion of what would be useful to
design in order to reduce the risk of designing a system that will be later
underutilized or rejected by end users, the analyst further notes that "there may
be other ways to achieve [this goal]" after describing a particular approach that
could be taken:

"Another approach to the above would be to have a large set of
"canned" themes and have them run against the same set of data
simultaneously... if the display shows a fabric representing the
100K documents with a bunch of "theme" peaks all over the
topography that I can access to see what they represent. Then I feel
pretty good when I find only one peak on my primary interest and
all the others are so far removed from my interest that I can
disregard them."

Note that any model-based method to depict more than base data is subject to
the "right" model catch -- how do you know the model that specifies how data is
informative is appropriate for the task or situation? Also, as was pointed out by
the expert analyst, "The "canned theme" might have to be regularly
updated to meet changing times (that could be a "downer")." Although we do
not have any completely satisfying solution to this catch, we advise that the
human user always to be allowed to override the suggestions of a machine
processor as well as update or otherwise alter the available "modelbase." In
order to make a useful and usable design concept based on this design seed, we
would want to explore further how to address these concerns. For, as the analyst
notes, "The problem is immensely complex."



TABLE OF CONTENTS