David S. Stodolsky
Entropy is on the rise.

About

Members Join Now Login

Self-management of Criticism in Dialogue

Stodolsky, D. (1988, September). Self-management of criticism in dialogue. Fourth European Conference on Cognitive Ergonomics, Cambridge, UK: European Society for Cognitive Ergonomics.

Abstract

Dialogue plays a crucial role in decision making, and its management is an important function for a group decision-support system. Effective dialogue management requires that levels of criticism are regulated. A signal detection model of individual performance can be used by an automatic mediator to facilitate self-management of criticism. The mediator receives confidential offers of criticism from individuals and combines them using model parameters in order to predict whether a given criticism will be sustained by group consensus. The accuracy of these predictions controls the levels of reward offered for successful criticisms. Inadequate reward tends towards a lack of criticism, while excessive reward tends towards every statement being disputed. Dynamic regulation of reward levels guides the group between these two extremes, thereby facilitating adequate criticism and improved group performance.

The notion of an ideal speech situation has a long history. Both Plato in the Phaedrus and Habermas argue that if dialogue is to lead to truth, an ideal speech situation is needed, which in turn presupposes an ideal social environment. Additional key ideas are priority for criticism and a discursively achieved consensus (Bernstein, 1978:262; McCarthy, 1978). Recent theoretical and methodological advances in social psychology allow a more rigorous treatment of this notion. Computer mediated conferencing, for example, permits the construction of highly egalitarian social environments within which speech can be structured (Stodolsky, 1981a, 1981b).

Specification of an ideal social environment is based here upon a social psychological theory of self-management. Self-management can be characterized as maximally decentralized democracy. In such an environment individuals learn to manage their own behavior effectively. Basic theoretical questions are how to protect and promote expression, how to organize participation, how to give meaningful feedback, and how to resolve conflict (Stodolsky, 1985a, 1985b). Progress in this research results from better models of the person and of the environment. The fundamental limits to decentralization are explored by examining the psychology of perception and stress (Stodolsky, 1984). The social and political dynamics of the decision making environment control the degree to which these limits can be achieved. Thus, the design of decision making environments can play a key role in theory testing. Such environments can be considered operational models of a theory. An operational model is defined here then as an interactive decision aid or decision support system used by the group.

Background

One on-line model that offers a powerful protective mechanism, but limited adaptability has been experimentally tested (Stodolsky, 1976; Stodolsky, 1981a). It was tested in laboratory simulations of telephone meetings. The experiments included a new method for the regulation of speaking order. One way to understand the operation of the model is to think of each speaker having a computer that in certain conflicts acts as a confidential agent. When many persons wish to talk at the same time, these agents negotiate to determine which person will speak first. In the simplest case, the agents compare their clocks that show how long each person has thus far spoken. The person who will be permitted to speak has the least time on their clock, therefore the one who has thus far spoken the least. This is called the equal-time resolution rule.

The transition from speaker to speaker is protected in these cases, since the responsibility for either cutting off an over-time speaker or rejecting a pending request is shared by the group, as opposed to being seen as the sole responsibility of the succeeding speaker, which is the case in unstructured dialogue (Stodolsky, 1981b). This is true for two reasons, first the group has agreed to use the program to select among conflicting requests. Second, at any contested transition, between one and all of the listeners have requests pending, thus indicating a desire to terminate the turn of the current speaker. Neither the presence of, nor the source of a request is revealed if it is rejected. An accepted request is revealed only when the name of the new speaker appears. The protective mechanism is similar to that used by scientific journals that have blind review (author anonymity during the review process). That is, contributor's names are only revealed if they are selected to present their message.

Persons in groups that used the equal-time resolution rule showed superior task performance and experienced less frustration as compared to persons in groups using a first-in first-out resolution rule. Also, in groups using the equal-time resolution rule, persons who were fearful of speaking in groups were felt to be group leaders just as often as persons who were not fearful (Zimbardo, Linsenmeier, Kabat & Smith, 1981). Also, teams working in the telecommunication settings made better decisions, enjoyed themselves more, experienced less frustration, and viewed their teams' decisions more favorably as compared to face-to-face groups (Linsenmeier & Zimbardo, 1982).

These results are unusual when compared with other experimental work on telephone meetings, that often shows greater dissatisfaction and inferior performance as compared to face-to-face meetings (Weston & Kristen, 1973; Weston, Kristen & O'Connor, 1975). The dissatisfaction and inferior performance is understandable, however, if one analyzes experimental comparisons of face-to-face and mediated communication. In a review, Williams (1977) noted that mediated communication inhibits role differentiation. The experimental results, however, could not distinguish between greater equality in groups and simply greater disorganization. Meetings of an egalitarian type require substantial interactional work. In telephone meetings, the visual channels for regulating turn taking are blocked, thus an extra load is imposed upon the voice channel, which must also carry substantive information. Thus, disorganization leading to inferior performance can easily occur and with it dissatisfaction. One way to summarize this result is that the problem situation needs greater bandwidth than available with only a audio channel.

Another factor of importance is the match between the problem- structure communication needs and the social structure in the group. While there is no reduction in bandwidth in face-to-face groups, role formation proceeds normally and dominance relations can block information exchange. Huber (1982) observes that performance in face-to-face group decision making is impeded by excessive dominance. Janis (1972) has also observed certain pathological results of group decision making as a result of such relationships and the lack of structures for adequate criticism. Strict hierarchical relationships were apparently not suitable for the types of decision making examined.

Even in cooperative work groups, such as scientific teams, where both the desire and need for openness and criticism is high, status related impediments to free communication often occur. This is even true in situations where the entire team is focused on a single common goal, such as in the Manhattan Project (Feynman, 1984). If status differences can impede effective communication in such extreme cases, where success of a joint effort is felt by all to be essential to survival, then powerful structuring techniques that include protected response capabilities can be seen to offer a significant advantages in group decision making.

With automatic mediation the protected request channels increases bandwidth and the resolution rules take up the extra interactional work load in telephone meetings, thus contributing to enhanced performance and reduced frustration. The inhibited role differentiation and associated dominance relations are restructured toward more equality by the protection mechanism and the equal-time resolution rule (Stodolsky, 1981b; Linsenmeier & Zimbardo, 1982). Since egalitarian relationships are needed for effective exchange of information in the type of group decision making studied, superior performance resulted. The inhibition of the dominance relations in the group also led to greater satisfaction, apparently since instead of one person having the leadership role and dominating others, each person could make a leadership contribution, or at least avoid being dominated for the entire time. The average participant, typically in a dominated position, experienced a more favorable emotional climate. Thus, the structure for dialogue management can play a crucial role in decision making.

Self-management of criticism in dialogue, implies that the group adjusts the level and type of criticism according to its own needs. This means both that people offer responsible criticism when appropriate and that they are as a group able to judge the relative competence of their members. An optimal level of criticism permits people to make such judgements. An adquate control structure stabilizes a group at this level of criticism. Such self stabilization requires adequate input and output interfaces, a clear understanding of the rule structure, and adequate time for various learning processes to occur.

A Theory of Criticism

Optimal criticism is a multi-dimensional concept, which is based upon categories of criticism and the priorities among them. The following review will first address the dimensions or categories for criticism and the priority relations among them. Next, the optimization of criticism on one dimension, truth, will be treated in detail.

Priority Structures in Dialogue

Formal procedures for dialogue management, such as parliamentary procedure, have structures that give priority to statements that deal with the management of meetings (privileged motions), and secondarily to questions arising out of procedural considerations (incidental motions). These motions take precedence over, and suspend the discussion of substantive questions. They often question the validity of certain statements. These formal procedures and other less formal methods of policy argumentation function by questioning the assumptions, most often implicit, upon which the dialogue is based.

While the management of meetings (setting of time limits, entry and exit rules, and rule changes) is beyond the scope of this paper, the ability to reprogram a system as part of the meeting process would be desirable. Stodolsky (1984) suggested a model for structuring dialogue based upon categories of criticism related to validity claims (C - F). Even cooperative responses (A & B) to statements can be critical of the manner of presentation (Grice, 1975). Priority to cooperative responses gives the following ranking:

(A) Addition or correction offered; (B) Additional information needed; (C) Do not understand; (D) Not sincere; (E) False; (F) Disagree.

In the following section we examine how truth maintenance can be ensured by encouraging an appropriate level of negative assertion. Participants assert this claim by pressing a button labelled False.

Responsible Criticism

Often in social interaction, accountability, or more precisely its expectation, precedes responsible action. In automatically mediated dialogue, one mechanism of accountability is a speaker selection procedure. Persons indicating that an error in reasoning has been made, by offering a criticism, increase their probability of being selected to speak and being required to show a flaw in the previous statement. If they cannot explain the flaw to the satisfaction of the group, then their estimated probability of giving a correct criticism is reduced, and they might loose later speaking opportunities (Table 1). This could dampen tendencies toward being over critical.

Table 1
Payoff matrix for truth maintenance


Digits indicate the number of minutes credited to speaking time.


                                Behavior


                      Pressing      | Not pressing
                      button marked | button marked
                      False         | False
                     +-----------------------------+
                     |   Hit        | Miss         |
           Erroneous |          1   |        -1    |
Utterance            |--------------+--------------+
           Correct   | False        | No           |
                     | Alarm   -2   | Error   0    |
                     +-----------------------------+

This accountability structure requires the identification and verification of errors in utterances. Figure 1 gives examples of how utterances can be characterized. Case 0 indicates the notation for statements accepted as correct. The simplest identification-verification sequence (Case 1) occurs when a statement by person A triggers a False indication (offer of criticism) by person B. B is then selected to speak, criticizes A, and the criticism is verified (accepted by all), that is, none press their False buttons. In this case A is charged for an incorrect statement (S-) and B is credited a correct criticism (C+). In general, statements accepted as correct (S+) precede and terminate error processing sequences.

Figure 1
Possible categorizations of verbalizations


     Participants = A, B, C, D (public responses)
                  = a, b, c, d (anonymous responses)


     Time ------->



Case 0
Utterance of :    A  B  C
False signal by :
Category score :  S+ S+ S+


Case 1
Utterance of :    A  B  C
False signal by :   b
Category score :  S- C+ S+


Case 2
Utterance of :    A   B  C  D
False signal by :   bc c
Category score :  S0  C- C+ S+


Case 4
Utterance of :    A  B  A  B  A  C
False signal by :   b  a  b  a  b
Category score :  S0 C0 C0 C0 C0 S+



Key :
      S+     verified correct statement
      S0     unverified statement
      S-     verified incorrect statement
      C+     verified correct criticism (Hit)
      C0     unverified criticism
      C-     verified incorrect criticism (False alarm)

A double-error sequence (Case 2) occurs when B's criticism of A is not accepted by C who accurately criticizes B. A simple error processing algorithm would charge B with an erroneous criticism (C-) and credit C with a correct criticism (C+). A more sophisticated error processing scheme would then permit C to criticize A's statement. This could be achieved with a pushdown stack that recalls previous utterances after later ones are scored.

A possible problem in processing errors is the sequence in Case 4. Here A and B speak repeatedly and each indicates the other's criticisms are in error. The rule indicated would terminate the sequence, after giving each speaker two criticisms, by selecting a new speaker, in this case C.

A simple approach to modeling participants within this framework is to compute the contribution of each statement to the probability of making a correct statement, where correct is defined as generating no False indications. Thus S+ is assigned the value of one and S- is assigned the value of zero, for each of N statements made by a given person. If we consider only verified statements, a person's estimated probability of making a correct statement is calculated: P(S+) = sum S+/N. A similar score is derived for criticisms.

Now consider the four outcomes possible with each statement (Table 1). The No Error condition is in this case represented by Case 0 (Figure 1); all participants are credited uniformly where everyone agrees statements are correct. The Hit is illustrated in Case 1 where B correctly criticizes A. Since other persons did not offer critical comment, they are charged with a Miss. A person's verbalizations and button responses can be scored independently with this approach. Each participant can then be modeled with three probabilities. The probability of making a correct statement (P(S+)), the probability of accepting true statements (No Error - not indicating False to a correct statement), and the probability of responsible criticism (Hit - indicating False to an erroneous statement). Then statements can be estimated as either correct or erroneous by combining the speaker's probability of making a correct statement and the False indications which indicate offers of criticism. If we know the probability of a pattern of False indications occurring given that a statement is true, then by Bayes' Theorem we can compute the probability that the statement is true given the pattern.

Dynamic Regulation of Level of Criticism

The scores computed by Bayesian estimation are available after a statement has been made and criticism has been offered, but before any criticism has been voiced. Thus, the computed score is a prediction of whether the group will accept the coming criticism. These predictions show the degree of knowledge the group has of its self.

In table 1, each event has associated with it a payoff in minutes of speaking time. Thus if a person indicates False to a statement and the criticism is accepted by the group, a Hit is credited. With the nominal payoffs shown, a person would get 1 minute for each Hit. If False were not indicated by someone else and a statement was shown to be incorrect, then 1 minute would be charged against their account. This is called a Miss. Thus a person not attending could easily lose the opportunity to speak in future competitions with others. If False was indicated, but the criticism was accepted as incorrect, the person would be charged 2 minutes for a False Alarm. These adjustments allow individual difference in competence to be taken into account.

By adjusting the payoff for a Hit, the level of criticism can be regulated. The payoff is increased to generate more criticism. When a statement is challenged, the model predicts whether the group will accept the criticism. If it is correct then the payoff can be increased. At least two situations could result in incorrect predictions. The first is a change in the topic of discussion that alters the relative balance of competence in the group. Under this condition, more verified statements are needed so the model can be recalibrated. Lowering the payoff for Hits has the effect of increasing the fraction of verified statements, because less criticisms are motivated and the group is likely to come to a consensus more quickly. The second situation relates to the same basic assumption of the model. The probabilities are assumed to change slowly. If, for instance, a person attempts to game the situation by saying in effect, "I know this statement is wrong, but I will say it anyway, because no one will know it, and I can escape criticism," this assumption of the model is violated. The resulting prediction errors would reduce the payoff for Hits. This reduction would move the group toward greater agreement, which tends to increase group cohesiveness at the risk of groupthink (Janis, 1972). The increased solidarity, mutual liking, and positive feelings about carrying out group work would be a powerful antidote to gaming. While the model cannot cope with gaming, it can detect when it might be occurring and guide the group to a state that discourages it. Stodolsky (1984) further discusses how signal detection theory can be used for analysis of prejudice and stability in group decision making.

References

Bernstein, R. J. The reconstruction of social and political theory. Philadelphia: University of Pennsylvania Press, 1978.

Feynman, R. Los Alamos from below. Science 84, 5(12), (December) 1984, 63-69.

Grice, H. P. Logic and Conversation. In Cole, P. & Moran, J. L. (eds.), Syntax and Semantics (vol. 3), NY: Academic, 1975.

Huber, G. P. "Group decision support systems as aids in the use of structured group management techniques." In G. Dickson (Ed.), DSS-82, Second international conference on Decision Support Systems transactions, 1982.

Janis, I. Victims of groupthink. Boston: Houghton Mifflin, 1972.

Linsenmeier, J. A. & Zimbardo, P. G. Effects of system and social variables on gender differences in communication and team decision making (Office of Naval Research Technical Report Z-82-02). Stanford, CA: Department of Psychology, Stanford University, December 1982. (National Technical Information Service No. AD-A125 006)

McCarthy, T. The critical theory of Jurgen Habermas. London: Hutchinson, 1978.

Stodolsky, D. Machine-mediated group problem-solving: Therapy, learning, performance (Doctoral dissertation, University of California, Irvine, 1976). Dissertation Abstracts International, 1976, 37, 1949B. (University Microfilms, No. 76-19, 633)

Stodolsky, D. Automatic mediation in group problem solving. Behavior Research Methods and Instrumentation, 1981a, 13, 235-242.

Stodolsky, D. Protected actions in dialog. In W. J. Reckmeyer (Ed.), Proceedings of the Twenty-fifth Annual North American Meeting of the Society for General Systems Research. Louisville, KY: Society of General Systems Research, 1981b.

Stodolsky, D. Self-management of criticism in dialog: Dynamic regulation through automatic mediation. Presented at the symposium Communication and Contracts between People in the Computerized Society, University of Gothenburg, Gothenburg, December 1984.

Stodolsky, D. Information systems for self-management. Human Systems Management, 5, 1985a, 39-45.

Stodolsky, D. The complete self-management information system [Letter]. Human Systems Management, 5, 1985b, 261-262.

Weston, J. R. and Kristen, C. Teleconferencing: A comparison of attitudes, uncertainty and interpersonal atmospheres in mediated and face-to-face group interaction. Ottawa, Canada: The Social Policy and Programs Branch, Department of Communications, December, 1973.

Weston, J. R., Kristen, C. and O'Connor, S. Teleconferencing: A comparison of group performance profiles in mediated and face-to-face interaction. Ottawa, Canada: The Social Policy and Programs Branch, Department of Communications, April, 1975.

Williams, Ederlyn. Experimental comparisons of face-to-face and mediated communication. Psychological Bulletin, 84(5), 1977.

Zimbardo, P. G., Linsenmeier, J., Kabat, L. & Smith, P. Improving team performance and participation via computer-mediated turn taking and informational prompts (Office of Naval Research Technical Report Z-81-01). Stanford, Ca.: Department of Psychology, Stanford University, March 1981. (National Technical Information Service No. AD-A097 028)