The measure of an illusion or the illusion of a measure?
Contributed by Bruno Rossion, May 2013 PDF of the text below
Comparing the matching of ‘same’ top face halves when they are aligned vs. misaligned with different bottom halves typically derives a measure of holistic face perception. In one case, aligned face halves, there is a powerful illusion of seeing the two face halves as being different. This illusion vanishes if the face halves are misaligned. This is the (standard) composite face paradigm, the better performance for misaligned compared to aligned trials giving rise to the composite face effect (CFE).
ALIGNED MISALIGNED
Some authors attempted to incorporate this paradigm into the general framework of congruency/interference between sensory dimensions, an approach introduced by Stroop (1935) in experimental Psychology. In a Stroop-like design, each dimension is associated with a behavioral response that either agrees with the behavioral response of the other dimension (“congruent trial”) or enters in conflict with it (“incongruent trials”) (MacLeod, 1991).
Here the goal is to test if these two dimensions, for instance a color word and its congruent/incongruent printed colour, or a central arrow and congruent/incongruent lateral flankers (Eriksen, 1995), are processed independently, or if they interact at some processing stage in the system.
Reinterpreting the composite face paradigm within this framework (e.g., Richler et al., 2008) means considering the two halves of the face as the two dimensions that potentially interfere in the system.
CONGRUENT INCONGRUENT
This congruency/interference approach has proved highly valuable in several areas of research, essentially to characterize attentional and response decision conflicts (MacLeod, 1991). However, a reinterpretation of the composite face paradigm within this approach is misleading. This is because the goals of the two approaches are different: researchers inspired by the visual composite illusion do not want to know if the two face halves interfere somewhere in the system. They are interested in capturing an objective signature – behavioural or neural – of a visual illusion. That is, the composite face paradigm measures perceptual integration, or grouping, of two parts into a whole face. Two new whole face identities are created in the aligned version, and they have to be compared to make a ‘same’ judgment. Hence, in this particular paradigm, performance drops in the condition associated with holistic face perception (aligned) as compared to the condition that is not associated with holistic face perception (misaligned). However, despite this negative effect of holistic face perception, it is not a paradigm that aims at measuring interference, or a negative influence of one face part (the bottom part) on another part (the top part). If one refers to interference in the paradigm, it is because perceptual integration (of the target and distracter halves) interferes with performance on the target half. Thus, the interference comes from the comparison of the two faces and the nature of the task, not from the intrinsic relationship between the two face halves.
Crucially, in the standard composite face paradigm, everything is incongruent: the top and bottom halves are associated with distinct responses both for aligned and misaligned trials. However, in a congruency/interference paradigm, two face halves are associated with an identical response in one condition (‘congruent’), and with a conflicting response in the other condition (‘incongruent’). Hence, an effect of congruency does not mean that the two face halves are integrated perceptually: the two face halves may well be perceived in parallel, but interference may arise purely at the response level. As a matter of fact, the interference between the color of a word and its lexical representation is not interpreted in terms of a holistic visual representation but rather, usually, in terms of response selection conflicts (MacLeod, 1991). Thus, a congruency/interference effect may have nothing to do with visual perception, and it is not surprising that this approach has led to all sorts of never-ending debates about whether the effect is perceptual or decisional (Richler et al., 2008). The effect of congruency may indeed be carried out by decisional factors. But it does not mean that holistic (face) processing is decisional !
A congruency effect on different trials reflects part-based processing
In addition to this fundamental problem, a congruency effect is not based on ‘same’ trials only. Congruency is manipulated in trials for which a ‘different’ response is the correct response (the two top halves being physically different).
Whether an overall measure of performance is used, or a d’ index, the ‘same’ and ‘different’ trials are given the same weight in the congruency effect. This is highly problematic because ‘different’ trials are unlikely to provide an effect of congruency: two different top halves will not look suddenly identical if they are aligned with identical bottom halves. In other words, the composite face illusion concerns only the erroneous perception of identity between halves, not the erroneous perception of actual differences. Moreover, if there is a small drop of performance in the incongruent ‘different’ trials, it is likely to reflect a part-based judgment: judging correctly the face as a whole should not lead to a ‘identical’ response. In summary, the source of a congruency effect obtained with facial halves is ambiguous and cannot be directly related to holistic face perception.
The fool design
In an attempt to salvage the approach, and merge it further with the standard composite face paradigm, some studies have incorporated alignment as a factor in the congruency paradigm (Richler et al., 2011; see Richler et al., 2009 for a direct interpretation of an effect of congruency, without alignment). This has led to a paradigm that has 4 conditions, or 16 kinds of trials. Beyond a main effect of congruency, which is (incorrectly) interpreted as reflecting holistic processing, an interaction between alignment and congruency is interpreted as further evidence of holistic processing in these studies. According to this approach, the congruency design with alignment would be a “full design” while the standard composite face paradigm is labeled as a “partial” design (Gauthier & Bukach, 1997). In reality, the “full design” approach lacks parsimony – to say the least – and is doomed from the start: an interaction between alignment and congruency could be due to part-based ‘different’ trials, or to an interaction between alignment and response conflicts. Thus, unsurprisingly, such effects have been observed with many items like letters, novel shapes, musical notations and even inverted faces. What these effects truly mean remains unclear.
Using the terminology of a “full design” as opposed to a “partial” design is also dismissive for the composite face paradigm, and is misleading: the standard composite paradigm is complete insofar as it has one experimental and one control condition that differ only by one factor (alignment). Because it is so complex and lead researchers in the wrong direction, the so-called “full design” could be renamed as the overextended design or, jokingly, as the “fool” design.
In conclusion, while the composite face paradigm measures an illusion, the congruency paradigm provides the illusion of a measure.
For a full discussion of this issue, please see the following paper:
Rossion, B. (2013). The composite face illusion: a window to our understanding of holistic face perception. Visual Cognition. PDF
Other references
Eriksen, C.W. (1995). The Flankers Task and Response Competition: A Useful Tool for Investigating a Variety of Cognitive Problems. Visual Cognition, 2, 101-118.
MacLeod, C.M. (1991). Half a century of research on the Stroop effect: an integrative review. Psychological bulletin 109, 163–203.
Richler, J. J., Gauthier, I., Wenger, M. J., & Palmeri, T. J. (2008). Holistic processing of faces: Perceptual and decisional components. Journal of Experimental Psychology: Learning, Memory & Cognition, 34, 328–342.
Richler, J.J., Mack, M.L., Gauthier, I., Palmeri. T.J. (2009). Holistic processing happens at a glance. Vision Research, 49, 2856-2861.
Richler, J.J., Mack, M.L., Palmeri, T.J., & Gauthier, I. (2011). Inverted faces are (eventually) processed holistically. Vision Research, 51, 333–342.
Stroop, J. R. (1935). Studies of interference in serial verbal reactions. Journal of Experimental Psychology,18, 643-662.