But assessment strikes fear in most people’s hearts. The word itself draws forth a strong, gut-level memory of being evaluated and measured, whether through grades in school, ranking in competitions, or promotions on the job. As writer Sue Miller Hurst has pointed out, most people have an intrinsic ability to judge their progress. But schools and workplaces subjugate that natural assessment to the judgment of teachers, supervisors, and other “experts,” whose appraisals determine promotions, wealth, status, and, ultimately, self-esteem.

Assessing Learning

Is it possible to use assessment in the service of learning? Can assessment be used to provide guidance and support for improving performance, rather than elicit fear, resentment, and resignation? This has been a guiding question at the MIT Center for Organizational Learning for several years, as we have struggled to find a reasonable way to assess learning efforts. The motivations are essentially pragmatic: our corporate affiliates need some idea of the return on their investments, and we as researchers need a better understanding of our work.

To create a new system of assessment, we started by going back to the source — to the people who initiate and implement systems work, learning laboratories, or other pilot projects in large organizations. We then tried to capture and convey the experiences and understandings of these groups of people. The result is a much-needed document that moves beyond strict assessment into the realm of institutional memory. We call it a “learning history.”

The Roots of a New Storytelling

A learning history is a written document or series of documents that is disseminated to help an organization become better aware of its own learning efforts. The history includes not just reports of action and results, but also the underlying assumptions and reactions of a variety of people (including people who did not support the learning effort). No one individual view, not even that of senior managers, can encompass more than a fraction of what actually goes on in a complex project — and this reality is reflected in the learning history. All participants reading the history should feel that their own points of view were treated fairly and that they understand many other people’s perspectives.

A learning history draws upon theory and techniques from ethnography, journalism, action research, oral history, and theater. Ethnography provides the science and art of cultural investigation — primarily the systematic approach of participant observation, interviewing, and archival research. From journalism come the skills of getting to the heart of a story and presenting it in a way that draws people in. Action research brings to the learning history effective methods for developing the capacities of learners to reflect upon and assess the results of their efforts. Finally, the tradition of oral historians offers a data collection method for providing rich, natural descriptions of complex events, using the voice of a narrator who took part in the events. All of these techniques help the readers of a learning history understand how participants attributed meaning to their experience.

Each part of the learning history process — interviews, analysis, editing, circulating drafts, and follow-up — is intended to broaden and deepen learning throughout the organization by providing a forum for reflecting on the process and substantiating the results. This process can be beneficial not only for the original participants, but also for researchers and consultants who advised them — and ultimately for anyone in the organization who is interested in the organization’s learning process.

Insiders versus Outsiders

One goal of the learning history work is to develop managers’ abilities to reflect upon, articulate, and understand complex issues. The process helps people to hone their assessments more sharply by communicating them to others. And because a learning history forces people to include and analyze highly complex, dynamic interdependencies in their stories, people understand those interdependencies more clearly.

In addition, the approach of a learning history is different from that of traditional ethnographic research. While ethnographers define themselves as “outsiders” observing how those inside the cultural system make sense of their world, a learning history includes both an insider’s understanding and an outsider’s perspective.

Having an outside, “objective” observer is an essential element of the learning history. In any successful learning effort, people undergo a transformation. As they develop capabilities together, gain insights, and shift their shared mental models, they change their assumptions about work and interrelationships. This collective shift reorients them so that they see history differently. They can then find it difficult to communicate their learning to others who still hold the old frame of reference. An outside observer can help bridge this gap by adding comments in the history such as, “This situation is typical of many pilot projects,” or by asking questions such as, “How could the pilot team, given their enthusiasm, have prevented the rest of the organization from seeing them as some sort of cult?”

Similarly, retaining the subjective stance of the internal managers is important for making the learning history relevant to the organization. In most assessments, experts offer their judgment and the company managers receive it without gaining any ability to reflect and assess their own efforts. The stance of a learning history, on the other hand, borrows from the concept of the “jointly told tale,” a device used by a number of ethnographers in which the story is “told” not by the external anthropologist or the “naive” native being studied, but by both together. For these reasons, the most successful learning history projects to date seem to involve teams of insiders (managers assigned to produce and facilitate the learning history) working closely with “outside” writers and researchers hired on a contractual basis.

Results versus Experience and Skills

Companies today don’t have a lot of slack resources or extra cash. Thus, in every learning effort, managers feel pressured to justify the expense and time of the effort by proving it led to concrete results. But a viable learning effort may not produce tangible results for several years, and the most important results may include new ways of thinking and behaving that appear dysfunctional at first to the rest of the organization. (More than one leader of a successful learning effort has been reprimanded for being “out of control.”) In today’s company environment of downsizing and re-engineering, this pressure for results undermines the essence of what a learning organization effort tries to achieve.

One goal of the learning history work is to develop managers’ abilities to reflect upon, articulate, and understand complex issues.

Yet incorporating results into the history is vital. How else can we think competently about the value of a learning effort? We might trace examples where a company took dramatically different actions because of its learning organization efforts, but it is difficult to construct rigorous data to show that an isolated example is typical. Alternatively, we might merely assess skills and experience. A learning historian might be satisfied, for instance, with saying, “The team now communicates much more effectively, and people can understand complex systems.” But that will be unpersuasive — indeed, almost meaningless — to outsiders.

In this context, assessment means listening to what people have to say, asking critical questions, and engaging people in their own inquiries: “How do we know we achieved something of value here? How much of that new innovation can we honestly link to the learning effort?” Different people often bring different perceptions of a “notable result” and its causes, and bringing those perceptions together leads to a common understanding with intrinsic validity.

For example, one corporation’s learning history described a new manufacturing prototype that was developed by the team. On the surface, this achievement was a matter of pure engineering, but it would not have been possible without the learning effort. Some team members had learned new skills to communicate effectively with outside contractors (who were key architects of the prototype), while others had gained the confidence to propose the prototype’s budget. Still others had learned to engage with each other across functional boundaries to make the prototype work. Until the stories of these half-dozen people were brought together, they were not aware of the common causes of each other’s contributions, and others in the company were unaware of the entire process. The learning history thus included a measurable “result” — the new prototype saved millions of dollars in rework costs — but simply reporting a recipe for constructing new prototypes would be of limited value. At best, it would help other teams mimic the original team, but it wouldn’t help them learn to create their own innovations. Only stories, which deal with intangibles such as creating an atmosphere of open inquiry, can convey the necessary knowledge to get the next team started on its own learning cycle.

The Strength of the Story

Some learning histories have been created after a project is over. Participants are interviewed retrospectively, and the results of the pilot project are more-or-less known and accepted. Other histories are researched while the story unfolds, and the learning historian sits in on key meetings and interviews people about events that may have taken place the day before. “Mini-histories” may be produced from these interviews, so that the team members can reflect on their own efforts as they go along and improve the learning effort while it is still underway. But such reflection carries a burden of added discipline: it adds to the pressure on the learning historian to “prove results” on the spot, to serve a political agenda, or to justify having a learning history in the first place.

To create an ongoing learning history, an organization must embrace a transformational approach to learning. Instead of simply learning to “do what we have always done a little bit better,” transformational learning involves re-examining everything we do—including how we think and see the world, and our role in it. This often means letting go of our existing knowledge and competencies, recognizing that they may prevent us from learning new things. This is a challenging and painful endeavor, and learning histories bring us face to face with it. When the learning history is being compiled simultaneously with the learning effort, then the challenge and pain of examining existing frameworks is continuous. But to make the best of a “real-time” learning history, admitting and publicizing mistakes must be seen as a sign of strength. Uncertainty can no longer be a sign of indecisiveness, because reflecting on a learning effort inevitably leads people to think about muddled, self-contradictory situations. Much work still needs to be done on setting the organizational context for an ongoing learning history so that it doesn’t set off flames that burn up the organization’s good will and resources.

Currently, there are almost a dozen learning history projects in progress at the Learning Center. In pursuing this work, we no longer talk about “assessing” our work. Instead, we talk about capturing the history of the learning process. It is amazing how this approach and new language changes the tenor of the project. People want to share what they have learned. They want others to know what they have done — not in a self-serving fashion, but so others know what worked and what didn’t work. They don’t want to be assessed. They want their story told.

George Roth is an organizational researcher with the MIT Center for Organizational Learning and a consultant active in the study of organizational culture, change, and new technology introduction.

Art Kleiner is co-author and editorial director of The Fifth Discipline Fieldbook, and author of the forthcoming The Age of Heretics, a history of the social movement to change large corporations for the better.

The post Learning Histories: “Assessing” the Learning Organization appeared first on The Systems Thinker.

First-Person Research

Reason and Torbert (2001) describe first-person research as encompassing those “skills and methods [that] address the ability of the researcher to foster an inquiring approach to his or her own life, to act awaredly and choicefully, and to assess effects in the outside world while acting.” As such, first-person research is research that we do by ourselves on ourselves.

The first step is to study ourselves by “dividing or stretching” our attention in a variety of ways to increase our awareness of the present moment. In a conversation, for example, instead of concentrating on our thoughts, we may also observe the nonverbal cues that indicate how others are responding, listen to how we are speaking, and recognize the attributions and mental models that are guiding us. By doing so, we can assess the effects of our actions and make choices about how to act in the future.

A variety of practices can build our capacity for focusing on the present moment, such as meditating, praying, painting, keeping a journal, and

By becoming “action researchers” of our own experience, we can go beyond our deeply ingrained defense mechanisms to create new possibilities for action.

writing autobiographical material. The key in these practices is to reflect critically on issues that make us vulnerable and reconsider the ways in which we live our lives. The long-term result is increased creativity and a wider range of alternatives for action.

Another good way of practicing first-person research is to explore how we might improve our effectiveness in dealing with others in the workplace. At Boston College, MBA students in the Leadership Workshop and executives in the Leadership for Change program engage in first-person research practices at work, write about the experience in journals, and draw lessons from their findings. These might include simple things such as trying to speak up more in meetings or taking five-minute walks through the office while paying attention to features and people that don’t usually attract attention. Over several weeks, such small experiments may begin to reveal new opportunities for action.

Second-Person Research

Second-person research involves creating communities of inquiry with others in which we are willing to explore the possible incongruities between what we say and what we do (or, put more formally, between our mission, strategy, performance, and outcomes). It occurs when we inquire with others in a face-to-face group about our shared mission, our norms, and the quality of our individual performances on behalf of the team’s mandate. Simply put, second-person research includes conversations in which participants share an intention to learn about themselves, the others present, and the team culture, as well as to delve into how to generate mutual transformation, if it is warranted. This process takes place in a supportive, self-disclosing, and open way.

The U. S. Army’s after-action review process, action inquiry groups, and a theatrical company in rehearsal are examples of this kind of research. In these groups, participants seek to learn from each other, become skillful in giving and receiving feedback, and become increasingly flexible in experimenting with new behavior from moment to moment. In action inquiry groups, participants commit to publicly testing their assumptions and to slowing down their thinking process and making it explicit so that they do not readily jump to ineffective conclusions when relating with others. Participants help each other in reducing their defensiveness and increasing the mutuality of their relationships.

Third-Person Research

In third-person research, people come together to create an organization that provides the necessary conditions for people to engage in first and second-person research. In these organizations, processes, procedures, and assessment mechanisms incorporate incentives for people to exercise critical thinking and institute change, thus opening the system to evaluation and restructuring. A good example of third-person research/practice is the idea of enacting learning organizations that are open to practicing inquiry, learning from mistakes, questioning their assumptions, and reducing defensiveness.

Some organizational development efforts, such as Open Space or Appreciative Inquiry initiatives, are both second-and third-person research/practices. They are second-person in that they provide tools for organizational members to learn about their mutual hopes and fears. They are third-person in that they aim at generating new ideas that redefine the whole organization and the way people work together.

The Society for Organizational Learning Greenhouse, held at Boston College in January 2004 for some 160 practitioners, consultants, and researchers, was a form of third-person research/practice. The organizers used questionnaires to solicit daily feedback in an effort to test the event’s ongoing efficacy. Participants took part in various forms of first-person research through practices such as yoga, tai chi, and silent meditation. They participated in several second-person research projects, including an ongoing consortium on sustainability in which participants engage together in action research projects directed toward changing their companies’ practices.

The event also explored the continual interweaving of first-, second-, and third-person action inquiry to achieve personal integrity, interpersonal mutuality, organizational profitability, societal equity, and environmental sustainability. Some half dozen new action research projects grew out of the conference itself. A keynote address by Joan Bavaria, founding president and CEO of Trillium Asset Management, on socially responsible investing showed how combining first-, second-, and third-person research and action in her work has led to an ongoing transformation in the field of investing as a whole.

Jordi Trullen (trullen@bc.edu) is a doctoral student in the Organization Studies department at the Boston College Carroll School of Management.

Bill Torbert is a professor at Boston College’s Carroll School of Management.

For Further Reading

Chandler, D. and W. Torbert., “Transforming Inquiry and Action: By Interweaving 27 Flavors of Action Research”, Journal of Action Inquiry. 1, 2 (2003).

Reason, P. and W. Torbert., “The Action Turn: Toward a Transformational Social Science: A Further Look at the Scientific Merits of Action Research”, Concepts and Transformation. 6, 1 (2001).

Torbert, W., “Transforming Social Science: Integrating Quantitative, Qualitative, and Action Research,” in Sherman, F. and W. Torbert (eds.), Transforming Social Inquiry, Transforming Social Action (Kluwer Academic Publishers, 2000).

Torbert, William and Associates. Action Inquiry: The Secret of Timely and Transforming Leadership (Berrett-Koehler, 2004).

The post First-, Second-, and Third-Person Research in Practice appeared first on The Systems Thinker.