Does Irrelevant Information Play a Role in Judgment?
Boicho Kokinov (email@example.com)12
Penka Hristova (firstname.lastname@example.org)1
Georgi Petkov (email@example.com)1
1Central and East European Center for Cognitive Science, Department of Cognitive Science and Psychology,
New Bulgarian University, 21 Montevideo Street
2Institute of Mathematics and Informatics, Bulgarian Academy of Sciences, Acad. G. Bonchev Street, bl.8
category. He believes individuals have their “ideal points”and therefore judging a stimulus can be described as
This paper presents an unusual prediction made by the
comparing it to this standard and measuring the distance
DUAL-based model of judgment JUDGEMAP and its
toward it. The Adaptation Level Theory (Helson, 1964) falls
verification. The model is shortly presented as well as the
into the same category, however, here the standard
simulation data obtained with it. These data predict that
(adaptation level) is changed depending on context. Finally,
people will use the information on an irrelevant dimension
the Norm Theory (Kahneman & Miller, 1986) follows a
when judging another dimension. This prediction is thentested in a psychological experiment and confirmed.
similar approach, however, the standard here is called“norm” and what is more important is that this norm is
constructed on the spot rather than retrieved from long-termmemory. A comparison set is constructed in working
Suppose that you are judging how tall a person is. Do you
memory consisting of known exemplars and its norm is
expect that the color of his or her eyes will play a role in
computed. Thus all three theories can be described as
that process? Or suppose you are judging the quantity of oil
relying on comparison of the target stimulus with a standard
in the bottle you are buying, do you expect that the font
(Figure 1), but they differ in the degree to which they
used on its label will have an effect? Finally, suppose you
subscribe to the constructivist approach toward this
are judging the length of a given line segment. Do you
expect that the color of the line will make a difference?
Both our intuition and the theories of judgment would
answer these questions negatively. Basically they wouldassume that when judging length we ignore all irrelevantfeatures (including color) and only physical length plays arole. Of course, many other factors, like order of
presentation and context, may play a role, but only thelength of the lines will take part in the judgment.
This paper is challenging this assumption of standard
theories of judgment and is trying to answer the aboveseemingly
surprisingly to show that all features (including theirrelevant ones) do matter or more precisely they may matter
Figure 1. Judgment as comparison with a standard.
Judgment as classification task.
Within this approach
Approaches to Judgment
the comparison set is subdivided into subcategories each of
There are a number of theories of judgment and a few
them corresponding to a judgment label (or scale element)
running models. Most of the theories originate from
and the target stimulus is classified within one of these
psychophysics and are mathematical in their nature; they do
subcategories. The Range-Frequency Theory (Parduci, 1965,
not describe the process of judgment, but only characterize
1974) postulates the constraints which should be met by
the end result. Since we are interested in describing the
such category subdivision: the range of value variation
process of judgment we will briefly outline only the main
within all subcategories should be about the same, and the
approaches proposed so far in that direction.
number of examples in all subcategories should be about the
Judgment as measuring similarity/dissimilarity with a
same. The Theory of Criterion Setting (Treisman &
The classical ideal point approach proposed by
Williams, 1984, Treisman, 1985) is a process model that
Coombs (1964, Wedell & Pettibone, 1999) falls into this
explains how dynamically we change the boundaries of thesubcategories. Finally, the ANCHOR model (Petrov &
Anderson, 2000, in press) describes the process of learning
DUAL-Based Model of Judgment
of these subcategories and solves the classification task bycomparing the target stimulus to the prototypes of each
The current model – JUDGEMAP (Judgment as Mapping) –
subcategory, these prototypes are supposed to be hold in
is based on a general cognitive architecture – DUAL
long-term memory and are called anchors (Figure 2). The
(Kokinov, 1994b, 1994c). This architecture is a hybrid
comparison set represented by the set of anchors is
(symbolic/connectionist) one and is explicitly designed to
model context-sensitivity of human cognition. It is basedon decentralized representations of concepts, objects, andepisodes and parallel emergent computations.
The AMBR1 (Kokinov, 1988, 1994a) and AMBR2
(Kokinov, 1998, Kokinov & Petrov, 2001) models are built
on DUAL and integrate memory and analogy-making. Sincethe process of judgment, as described above, involves
memory (construction of the comparison set in working
memory) and mapping (which is a central mechanism inanalogy-making) the JUDGEMAP model is most naturally
integrated in DUAL and borrows many of the mechanismsdeveloped for analogy-making in AMBR. Because of the
lack of space the model is described only in broad strokes.
Interested readers are invited to consult the literature on
Figure 2. Judgment as classification task. Comparing the
target to the standard of each of the subcategories.
Construction of the comparison set.
The comparison set
is formed from perception (the target as well as potential
Judgment as a mapping task.
The DUAL-based model
context stimuli) and from long-term memory (familiar or
of judgment discussed in this paper follows a third
recently presented exemplars as well as generalized
approach: The target stimulus is not compared to the
prototypes, if such exist in LTM). The mechanism
comparison set, but is rather included in it and then a
responsible for that construction is spreading activation. The
mapping is established between the elements of the
sources of activation are the INPUT and GOAL nodes, i.e.
comparison set and the set of rating labels (or scale
the perceived target (and possibly context) stimuli and the
elements). This mapping should be as close as possible to a
goal to judge the stimuli on a scale predefined in the
homomorphism, i.e. the relations among the elements of
instruction (e.g. a scale from 1 to 7). Thus the
the comparison sets should be kept among their
representations of the target and the scale elements become
corresponding rating labels. Thus the process of judgment
sources of activation which is then spread through the
involves construction of the comparison set, joining the
network of micro-agents. Naturally, concepts related to the
target to it, and mapping between the comparison set and
representation of the target become active, e.g. various
features of the target – these include both relevant andirrelevant features (of course, relevant features receive more
activation than irrelevant ones). The activation spreads
further from the general concepts (like RED, GREEN, etc.)towards specific examples of the concepts (other red or greenobjects). However, there are only a few links from thegeneral concepts to their exemplars – only to the most
familiar (typical) exemplars or to recently experienced ones.
Thus gradually a number of exemplars (and possiblyprototypes) are activated and become part of workingmemory – all these form the comparison set (Figure 4).
Figure 3. Judgment as mapping in the DUAL-based model.
Figure 4. Formation of the comparison set in WM by the
spreading activation mechanism of DUAL.
Mapping of the comparison set onto the scale elements.
of the comparison set (Figure 7). Now, if it happens that the
We can now consider the comparison set as a retrieved base
known red lines are longer than the known green lines, then
and map it onto the scale elements which are the target. The
the two target stimuli (differing only in color) will be
mapping process should preserve the relations among the
included in different comparison sets and thus judged
elements of the comparison set among their images on the
differently and there will be a shift in favor of the green
scale. The mapping should also follow the range-frequency
target. Therefore the speculative prediction of JUDGEMAP
principle described in the previous section. How is the
will be that even such irrelevant feature of the line like its
mapping achieved in JUDGEMAP? Similarly to AMBR, a
color will play a role in the judgment process. This
constraint-satisfaction network is constructed by the marker-
prediction is in sharp contrast to all theories and models
passing and structure-correspondence mechanisms. This
described in the first section, which assume that only the
network consists of temporal agent-hypotheses representing
possible correspondences between members of
comparison set and elements of the scale. These initial
hypotheses are formed according to the range principle.
Excitatory and inhibitory links are constructed among the
hypotheses and the spreading activation mechanism selectsthe winning hypotheses which form the mapping (Figure 5).
The competition among the hypotheses implements thefrequency principle. As result of this process not only the
target stimulus but also each element of the comparison setreceives a judgment. This does not mean that people would
be aware of all these judgments – most or even all of them
Figure 6. The target stimulus is red and therefore we expect
more red exemplars in the comparison set. They happened
to be larger in size and thus they compete for the upper part
of the scale. In this case the target stimulus (of the same
size as in Figure 7) will compete with them and will be
Figure 5. The process of mapping accomplished by the
constraint satisfaction mechanism. The winning hypotheses
Since the activation spreads from
the target stimulus (represented in a decentralized way bymany agents), exemplars, similar in some respect to it
(sharing some feature with the target), can be potentiallyactivated and thus become members of the comparison set
Figure 7. The target stimulus is green and therefore we
in working memory. This means that in addition to
expect more green exemplars in the comparison set. They
currently perceived stimuli, to recently activated exemplars,
happened to be smaller in size and thus they compete for the
and to highly familiar (typical) exemplars, exemplars which
lower part of the scale. In this case the target stimulus (of
are simply similar to the target will also participate in the
the same size as in Figure 6) will compete with them and
comparison set. Moreover, these exemplars might be similar
eventually will be mapped onto 5. In this way we receive an
along the relevant (judged) dimension or along an irrelevant
Let us consider the following example. Suppose we are
judging the length of line segments but the lines arecolored. Let the target stimulus be a red line of certain
Thus we will first describe a simulation experiment with
length. In this case we may expect that there will be more
JUDGEMAP that tests in practice this speculation and will
red lines in the comparison set (Figure 6) – they will be
also give us a rough estimation of the order of this color
activated through the RED concept which is shared with the
effect (if any). If we are successful, we will run a
target. On the other hand, if the target stimulus is a green
psychological experiment to text the model’s prediction and
line of the same length, more green lines will become part
Thus the mean of the mean ratings of all red categories is4.012, while the mean of the mean ratings of all green
In this simulation experiment we use a stimulus set of 56
categories is 4.065, which makes a difference of 0.053
lines. They are all in the long-term memory of the model.
which turns out to be almost significant tested with repeated
The lines differ in length and color. There are 7 different
measurements analysis (F(1,41)=3.917, p=0.055). The data
sizes (from 10 units of length to 34 unit with increment of
show that the possible size of the color main effect is very
4 units) and two different colors (red and green). Thus in
small, but may still be significant. This prediction makes
each size group there are 8 lines. The frequency of the red
sense: on one hand it is small enough, so that we can ignore
(respectively green) lines varies across the size groups. In
it in everyday life and this explains why our intuition says
size group one (the shortest lines - length 10 units) there are
that irrelevant information does not play a role in judgment.
7 green and 1 red line, in the second shortest group (length
On the other hand, the simulation predicts that the irrelevant
14 units) there are 6 green and 2 red lines, etc. In the largest
information does play a role and shifts a bit the evaluation.
group size (length of 34 units) there are 7 red lines and one
This means that under specific circumstances this shift
green line. Thus we have positively skewed distribution of
might be larger and become significant.
the green lines and negatively skewed distribution for thered lines.
The experiment described below is designed to test this
Each line is represented by a coalition of 5 agents
prediction of the model. Basically it replicates the
standing for the line itself, for its color, for its length, and
simulation experiment with a larger number of lines.
for the two relations (color_of and length_of). In additionthere are agents standing for the numbers from 0 to 8, but
only the agents standing for 1 to 7 are instances of “scale
In this experiment human participants rate the length of red
and green lines of various sizes. The interesting question is
On each run of the program we connect one of these lines
whether we will obtain a main effect of color, i.e. whether
to the input list thus simulating the perception of the target
there will be a difference between the ratings of the red and
“scale_from_1_to_7” to the goal node thus simulating theinstruction for rating on a 7 point scale.
We have produced 42 variations of the knowledge base of
the system thus simulating 42 different participants in the
experiment. The knowledge bases differ mainly in the
The experiment has a 14x2 within-subject factorial design.
associative and instance links among the agents, thus
The independent variables are length (varying at 14 levels)
although all our “artificial participants” will know the same
and color (varying at 2 levels: green and red) of the lines.
lines and the same concepts, they will activate different
The dependent variable is the rating of the length of the
lines on a 7-point-scale. The experimental question is
For each of these knowledge bases we have run two
whether there will be a main effect of color, which is
judgment trials: one for a red line of size 22 and one for a
supposedly an irrelevant factor in judging length.
The results from the simulations are presented in Figure 8.
A set of 14 color lines has been presented horizontally
As we can see the mean rating of the green lines are in most
against a gray background on a 17-inch monitor. The
cases slightly higher than the mean rating of the red lines
shortest line is 12 pixels, the longest one is 727 pixels and
the increment is 55 pixels. Each particular line length hasbeen shown eight times in red or green color. The shortlines were predominantly green while the long ones were
predominantly red. The color distribution within the set of
all 112 lines (14 lengths x 8 times) is presented in Table 1.
The frequency of the stimuli was calculated in order to
receive a positively skewed distribution for the green color
and a negatively skewed one for the red lines.
Figure 8. Simulation data. The mean rating of each line
with a certain length (1-7) and color (green and red) obtained
Table 1. Frequency of the presented stimulus lines (where 1
represents stimulus length 12 pixels, 2-67 pixels and so
number of the
number of the red
Figure 9. The mean rating of each line with a certain length
(1-14) and color (green and red) obtained from all subjects.
The JUDGEMAP model of human judgment has been
presented. This model is based on a general cognitive
The participants were tested individually in front of a
architecture (DUAL) and is thus integrated with the memory
computer screen where all 112 stimuli were shown
and analogy-making model AMBR. Moreover, this model
sequentially and in random order. They were instructed to
inherits the underlying assumptions of DUAL and AMBR:
judge the length of each line presented on the screen on a
human cognition is context-sensitive (Kokinov, 1994c),
seven point scale: 1-“it is not long at all”, …, 7-“it is very
judgment included; human memory is constructive
long”. No feedback was provided to the participants and no
(Kokinov & Hirst, 2003), analogy-making is at the core of
time restrictions have been imposed on them. The whole
human cognition (Gentner, Holyoak & Kokinov, 2001) and
experiment typically lasted about 15 minutes.
its mapping mechanisms may be used in judgment.
The JUDGEMAP model is similar to the Norm theory
The participants were 18 undergraduate students (9 men and
and the ANCHOR model with respect to the constructive
9 women none of whom was color-blind) from the
approach to the formation of the comparison set. However,
introductory classes in psychology at New Bulgarian
unlike all the models described in the first section judgment
University who participated in order to satisfy a course
in JUDGEMAP is not based on comparison of the target
with some aspect of the comparison set, but rather the targetstimulus is included in the comparison set and it receives a
Results and Discussion
rating along with all other members of this set. This rating
We had 14x2=28 data points for each participant. The
process is based on establishing a mapping between the
results averaged over subjects are shown in Figure 9. Each
comparison set and the set of scale elements which mapping
bar stands for the mean rating that a line of the
corresponding size and color has received during the
Unlike all other models JUDGEMAP does not ignore the
experiment. The repeated measurements analysis showed
irrelevant features of the to be judged targets, moreover
that the difference (0.046) between the mean judgment of
these irrelevant features play a role in the construction of the
the green lines (4.239) and the mean judgment of the red
comparison set (retrieving similar objects according to these
lines (4.193) is significant (F(1, 17)=5.966, p=0.026).
irrelevant dimensions). The model makes a strange
Surprisingly enough we obtained a difference (0.046) that
prediction that the color of the target line may play a role in
is almost the same as the difference we obtained in the
the rating of its length and thus predicts a shift of the mean
simulation (0.053). No tuning of the model was possible
rating (although a small one) with the change of color. This
since we did not have the experimental data in advance.
prediction has been tested in a psychological experiment and
Thus the prediction of the JUDGEMAP model has been
The size of this color effect is very small, but the stimuli
have been very simple and the features unremarkable. It isdifficult to imagine that the green color reminds us of aparticular green line. That is why we plan to repeat theexperiment with more complex stimuli (human figures andclothes) and more memorable features (human faces). It ispossible the size of the effect in this case to become larger.
Petrov, A. & Anderson, J. (submitted). The Dynamics of
Scaling: A Memory-Based Anchor Model of Category
We would like to thank the AMBR research team for their
continuous support and stimulating environment as well as
Sarris, V., Parducci, A. (1978) Multiple Anchoring of
Stefan Mateeff and Maurice Grinberg for the valuable
Category Rating Scales. Perception and Psychophysics
Thurstone, L. (1927). A Law of Comparative Judgments.
, 34, 273-286.
Treisman, M. (1985). The Magical Number Seven and
Coombs, C. (1964). A Theory of Data. NY: Wiley.
Some Other Features of Category Scaling: Properties of a
Gentner, D., Holyoak, K. & Kokinov, B. (2001). The
Model for Absolute Judgment. Journal of Mathematical
Analogical Mind. Cambridge, MA: MIT Press.
Treisman, M. & Williams, T. (1984). A Theory of
Experimental and Systematic Approach to Behavior. NY:
Criterion Setting with an Application to Sequential
Dependences. Psychological Review, 91(1), 68-111.
Kahneman, D., Miller, D. (1986). Norm Theory:
Wedell, D., Pettibone, J. (1999) Preference and the
Comparing Reality to Its Alternatives. Psychological
Contextual Basis of Ideals in Judgment and Choice
, vol. 93 (2), pp 136-153.
Journal of Experimental Psychology: General, vol. 128,
Kokinov, B. (1988). Associative memory-based reasoning:
How to represent and retrieve cases. In T. O'Shea and V.
Sgurev (Eds.), Artificial intelligence III: Methodology,systems, applications
. Amsterdam: Elsevier.
Kokinov, B. (1994a). A hybrid model of reasoning by
analogy. In K. Holyoak & J. Barnden (Eds.), Advancesin connectionist and neural computation theory: Vol. 2.
(pp. 247-318). Norwood, NJ:Ablex
Kokinov, B. (1994b). The DUAL cognitive architecture: A
hybrid multi-agent approach. Proceedings of the EleventhEuropean Conference of Artificial Intelligence
. London:John Wiley & Sons, Ltd.
Kokinov, B. (1994c). The context-sensitive cognitive archi-
tecture DUAL. Proceedings of the Sixteenth AnnualConference of the Cognitive Science Society
. Hillsdale,NJ: Lawrence Erlbaum Associates.
Kokinov, B. (1998). Analogy is like cognition: Dynamic,
emergent, and context-sensitive. In K. Holyoak, D.
Gentner, & B. Kokinov (Eds.), Advances in analogyresearch.
Sofia, Bulgaria: NBU Press.
Kokinov, B. & Hirst, W. (2003). Constructive Memory.
Kokinov, B. & Petrov, A. (2001). Integration of Memory
and Reasoning in Analogy-Making: The AMBR Model.
In: Gentner, D., Holyoak, K., Kokinov, B. (eds.) TheAnalogical Mind: Perspectives from Cognitive Science
Luce, D. (1959). On the Possible Psychophysical Laws.
, 66(2), 81-95.
Parducci, A. (1965). Category Judgment: A Range-
Frequency Model. Psychological Review
, vol. 72 (6), pp407-418.
Parducci, A. (1974) Contextual Effects: A Range-Frequency
In: Carterette, E., Friedman, M. (eds)Handbook of Perception.
vol. II. PsychophysicalJudgment and Measurement. NY:Academic Press, pp127-141.
Petrov, A. & Anderson, J. (2000). ANCHOR: A Memory-
Based Model of Category Rating. In: L. Gleitman & A.
Joshi (eds.) Proceedings of the 22nd Annual Conferenceof the Cognitive Science Society. Hillsdale, NJ: Erlbaum.
Das ovarielle Überstimulationssyndrom (OHSS) Kurzfassung: Das ovarielle Überstimulations- Erscheinungsbild, Präventionsmaßnahmen unddothelial growth factor seems to be the media-syndrom (OHSS) ist eine relativ häufige Kompli-Therapie werden mit dem Ziel erläutert, dietor followed by a fluidshift into the third space. kation bei der künstlichen Befruchtung bzw. beiKomplikationsra
Global Programme against Corruption: an outline for action Centre for International Crime Prevention Office for Drug Control and Crime Prevention United Nations Interregional Crime and Justice Research Institute The Global Programme against Corruption has been drawn up by the Centre forInternational Crime Prevention of the Office for Drug Control and Crime Prevention of theUnite