assessment Archives - The Systems Thinker https://thesystemsthinker.com/tag/assessment/ Fri, 23 Mar 2018 18:38:08 +0000 en-US hourly 1 https://wordpress.org/?v=6.8.3 The Learning Organization Journey: Assessing and Valuing Progress https://thesystemsthinker.com/the-learning-organization-journey-assessing-and-valuing-progress/ https://thesystemsthinker.com/the-learning-organization-journey-assessing-and-valuing-progress/#respond Fri, 26 Feb 2016 14:30:14 +0000 http://systemsthinker.wpengine.com/?p=5128 uppose you have just been appointed the CKO—Chief Knowledge Officer—of your organization. You are responsible for managing the company’s knowledge capital, including how it is created, maintained, and used. You understand the principles of organizational learning and agree that effective learning is the pathway to accelerated performance improvement. Now you need to determine the right […]

The post The Learning Organization Journey: Assessing and Valuing Progress appeared first on The Systems Thinker.

]]>
Suppose you have just been appointed the CKO—Chief Knowledge Officer—of your organization. You are responsible for managing the company’s knowledge capital, including how it is created, maintained, and used. You understand the principles of organizational learning and agree that effective learning is the pathway to accelerated performance improvement. Now you need to determine the right approach for your organization, and how to get started.

It might help to think of organizational learning as an ongoing journey (see “Organizational Learning Journey: A Roadmap”). Although each company’s path will look slightly different, assessing your organization’s knowledge base and learning skills is a good place to start. This information provides a baseline against which you can measure your progress toward becoming a learning organization.

Learning Serves the Business Vision

Planning your organizational learning journey begins with knowing where you want to go. Articulating a vision for both the business and the organization will help define your destination — the higher purposes you want to serve through the specific learning initiatives. Starting with an organizational vision ensures that the learning needs of the company are driven by business and organization goals.

Organizational Learning Journey: A Roadmap

Organizational Learning Journey: A Roadmap

The stages of the organizational learning journey include an articulation of the business and organizational vision, an evaluation of the company’s learning needs. and the development of a learning strategy that will achieve the firm’s learning objectives

Different learning needs call for different learning styles and practices. For example, at Electricite de France, safety concerns do not allow nuclear power plant operators to experiment or to take risks on the job, so their learning takes the form of incremental improvements based on TQM techniques. By contrast, Kodak is experimenting with electronic imaging products that go well beyond its traditional lines of chemical-based films — a change that may require the firm to learn how to redesign key processes. Because of this need, Kodak fosters a creative and relatively high-risk environment aimed at rethinking its business over the long term. Although they differ in style, the learning practices at each company match its particular learning needs.

Once you have articulated your company’s business vision and defined its learning needs, you will want to get a better sense of the organization’s current learning condition. To assess this, there are three key areas to consider:

  • Knowledge Base: the strengths and weaknesses of your organization’s knowledge base and the knowledge areas to be reinforced
  • Learning Practices: the management and operating practices that foster or hinder learning
  • Learning Climate: the work culture and its effect on learning

Knowledge Base

In today’s economy, knowledge, not capital assets, is the primary source of wealth. While there are some exciting new methods for measuring and valuing knowledge capital, few organizations have studied how they themselves create, store, and use that capital. You can start your learning assessment by mapping your organization’s business processes in terms of knowledge generated and used.

This is exactly what one European company did five years ago. The company faced increasing pressures from global competitors in the baby-diaper business, as a recent result of product innovations. Company personnel began a Knowledge Base Assessment by defining what the company knew how to do in all its business processes, and then assessed their knowledge position versus what they viewed as world-class practices. Then they unbundled the knowledge base into precisely defined areas of know-how, determined the competitive impact of each area, rated their own performance versus that of the competition, identified gaps in the knowledge base, and designed corrective actions (see “Strategic Knowledge Map”).

The results prompted a wake-up call. Company employees recognized that they needed to reinforce the company’s knowledge base. They defined the areas of collaboration needed to fill the gaps in the base and entered several strategic alliances with other firms. The collaborative partners found the approach so helpful that they, in turn, initiated Strategic Knowledge Mapping in their own companies.

When you assess your firm’s knowledge base, it’s important to remember that knowledge comes in many forms, not just in databases and procedure manuals. Tacit knowledge — based on experience and practice — can be as important as explicit knowledge. For example, Matsushita developed a bread-making machine in the late 19:30s. When early prototypes could not replicate the art of high-quality bread-making, developers apprenticed themselves to master bread-makers to discover the tacit knowledge that these experts could not communicate explicitly. Your knowledge map should show the strategic importance of both tacit and explicit knowledge.

Learning Practices

Successful learning does not happen by accident. One hallmark of a learning organization is a purposeful learning approach designed to create knowledge and translate it into effective action. How can you create learning practices?

You can begin by looking for learning cycles. Successful learning typically follows a sequence:

  1. Shared awareness of a need for learning
  2. A common understanding of the situation
  3. Aligned actions, with measured results
  4. Joint review and communication of results
  5. Collective reflection about the learning process

(For examples of these steps in action, see “Supporting the Learning Cycle” on page 5.)

A summary way to view your company’s existing learning practices is to compile an inventory of your organization’s use of specific learning practices. This profile will provide a sense of where the organization perceives gaps between current conditions and the desired future reality, and can indicate priority areas for attention.

Learning Climate

In order to foster organizational learning, you should focus on enhancing individual and group skills, designing support structures for ongoing learning, and creating an overall organizational attitude that encourages learning. All of these aspects make up an organization’s “learning climate.”

A learning climate has both “soft” and “hard” components. On the soft side, cultural norms can either support or hinder learning. The hard side of a learning climate includes the structures and technologies that support open communication, knowledge management, and teamwork. One way to sense your organization’s learning climate is to look for evidence of the following conditions:

Curiosity. A culture that values curiosity and inquiry adopts learning behaviors naturally. Simulations and experiments follow from “what if” questions. Questions about what customers think and what competitors are doing lead to environmental scanning and targeted studies of the outside world.

For example, Sharp defines its product development vision as “optoelectronics,” a grand but undefined term. The breadth and open-endedness of the term spurs the curiosity of employees, who ask, “What does that mean? How can this term fit my work?” The creation of Sharp’s overhead projection computer display is one result of the creative tension prompted by such purposeful ambiguity.

Recognition of Conflict and Errors. Learning requires openness to new ideas, even when they generate controversy. Conflict should be welcomed as the means to develop common understanding, rather than suppressed for the sake of harmony.

Organizations that celebrate the discovery of errors, rather than search for blame, will learn from their mistakes. A good example of this comes from a team that writes documentation for electronics products. Each month the team celebrates the discovery of documentation errors with a bonfire of obsolete manuals — the bigger the better!

Leadership. The leader of a learning organization is not the traditional hero, individually responsible for tough decisions. Instead, he or she is the designer of corporate culture who accepts the uncertainty implied by experimentation. This is a very different model of leadership, and if it is embraced by top management, it is likely to be diffused to all management levels. Those being led can tell you which model of leadership is prevalent in your firm.

Staff Development. The implicit employment contract between a firm and its employees has changed. Long-term employment guarantees are being replaced by employer-supplied opportunities to maintain and expand knowledge and skills. Look for learning opportunities not just in the training department, but in job experiences that broaden responsibilities across functions.

Information and Communication Systems. Technological solutions to the challenges of creating, storing, and sharing knowledge include groupware, corporate knowledge bases, and videoconferencing. As you trace the flow of knowledge through your organization, look for how well these technologies are used.

Strategic Knowledge Map

Strategic Knowledge Map

Team-Based Work. Some work environments encourage learning efforts by single individuals, while others foster collective work. Learning organizations tend to encourage interaction and problem solving by teams. To assess whether your organization values individuals or teams, look at the recognition and reward systems. Is performance measured individually or in groups? Do rewards go to stars or to stellar team efforts? Are major initiatives personalized (as when a project takes on the name of its leader, such as the Grace Commission) or do they remain the responsibility of teams?

Some artifacts are subtle. For example, when we visited the offices of a construction equipment producer, a manager explained that coffee stations were placed in such a way that the design staff and customer service staff were forced to share stations. This set-up guarantees that design staff have at least informal opportunities to learn from the voice of the customer.

Incomplete Learning Cycles. Your search for learning practices should include problems as well as successes. One way to diagnose learning problems is to look for patterns of consistently broken learning cycles (see “Incomplete Learning Cycles”). If your organization has many stories of fact-finding and analysis, but few examples of taking action, it may be trying to learn vicariously (“analysis paralysis”). If there is lots of action but little analysis and planning, your learning may be accidental at best (the “ready, fire, aim” approach). If your firm regularly progresses to aligned action but can’t seem to learn from results, you may not have adequate measurement, review, and feedback systems in place. You “reinvent wheels” because the results from past wheel designs were never internalized.

Guidelines for Assessing Learning

Once you know what to look for as you assess your company’s learning efforts — knowledge management processes, learning practices, and the learning climate — you need to know how to find them. The following activities can provide guidelines for assessing your company’s progress:

Self-Assessment. Because learning is embedded in day-to-day activities and organizational culture, guided self-assessment can yield valid results. Train a member of each group in the principles of organizational learning and have these people lead structured interviews that identify learning practices and climate factors. This training also prepares selected process members to facilitate the learning action plan that should follow an assessment.

Group Interviews. Interviews that are intended to tease out learning practices are better done in groups rather than one-on-one. This is because learning practices at the team and company level depend on group dynamics, such as communications and coordinated efforts. Participants who describe both sides of knowledge transfers can offer more complete perspectives than those who relate to just one side.

Stories. Clinical questions about learning lead to abstract answers, bur stories and anecdotes can help people vividly recall their learning practices. Ask interviewees to remember incidents when change took place rapidly and effectively, when they mastered new processes, or when a good practice was diffused rapidly throughout the organization.

Incomplete Learning Cycles

Incomplete Learning Cycles

One way to diagnose learning problems Is to look for patterns of learning cycles that are consistently broken. These three common examples show how learning processes can become derailed.

Artifacts. Anthropologists search for artifacts that offer tangible clues of how a society behaved. What artifacts might a learning assessment find? For example, publicly displayed performance scorecards, often seen in production facilities, show a concern for measurement and feedback.

Ongoing Assessment

The assessment process can provide a wonderful opportunity to train staff members in learning principles. Scientists from Hawthorne to Heisenberg have discovered that measuring a process inevitably causes it to be altered. In the same way, when learning is measured, learning processes are altered. With this in mind, you can design assessment interviews to serve as training in the principles of organizational learning, and improve your learning state even as you measure it.

Above all, it is important to continue to measure your company’s learning activities over time. Conducting an initial learning assessment can provide a valuable baseline of learning practices against which to evaluate progress, but overall assessment should become an ongoing part of the organizational learning process. This is especially important because the learning needs of a company will change as it revises its vision and strategy. Making progress on the journey toward creating a learning organization requires a continual realignment between the goals of the company and its chosen learning path.

Edited from “Measuring Learning: Assessing and Valuing Progress.” Reprinted with permission from the Third Quarter 1995 issue of Prism. The quarterly journal for senior managers, published by Arthur D. Little. Inc.

Nils Bohlin is a vice president of Arthur D. Little International and coordinator of Its global Pharmaceutical Industry Practice, based In Stockholm. He is also the leader of Arthur D. Little’s learning organization project in Europe.

Paul Brenner Is president of Arthur D. Little Program Systems Management Company. specializing in information and program management services to government and Industry.

Supporting the Learning Cycle

The following examples for each of the stage of the learning cycle may help you identify learning practices in your organization:

Generating Shared Awareness involves continuously assimilating internal and external information about problems and opportunities.

  • Dell Computer holds regular Customer Advocate Meetings to share what support people have heard from customers with colleagues in product development, sales, and marketing.
  • NUMMI rotates shop-floor employees through the plant to build shared awareness of new processes.

Creating Common Understanding requires tools and processes for creating a common understanding of key problems and opportunities and openly discussing options for action.

  • Ford uses management simulators to experience the results of decisions without “betting the company.”
  • Royal Dutch Shell has a rich history of using scenarios of possible oil industry trends as team-based planning exercises designed to develop a common approach to strategy.
  • DuPont maintains and publishes a reference model of all business processes.

Producing Aligned Action. The purpose of learning is to enable the organization to take more effective action. Alignment refers to the match between an organization’s goals and its actions, and to the choreography of actions across divisions and over time.

  • Honda helps to ensure that customer management and engineering actions are aligned by including representatives from sales, engineering, and product development in every project team.

Performing Joint Review. It is helpful to review and measure the results of actions in an open forum. The purpose is not to assign blame or praise, but to gain insight from the complete cycle and kick off the next cycle of performance improvement.

  • Procordia, a Scandinavian consumer goods and health care group, undertook two major acquisitions simultaneously in 1990. In order to manage the integration of the two groups, it created a merger process organization that masterminded and reviewed actions. Every second Friday during the four-month process there was an all-afternoon meeting with the top management group to report on progress from the merger task forces.

Conducting Collective Reflection. In order to be purposeful about learning, it is important to reflect continuously on past and present operations and seek improvements in learning activities.

  • British Petroleum uses a five-person unit reporting to the board of directors to derive lessons learned from past major projects.
  • Boeing commissioned a group called Project Homework to dissect its past product development processes, leading to the successful development of the B757.
  • L.L. Bean has a team devoted to improving its business process improvement process.

The post The Learning Organization Journey: Assessing and Valuing Progress appeared first on The Systems Thinker.

]]>
https://thesystemsthinker.com/the-learning-organization-journey-assessing-and-valuing-progress/feed/ 0
From Students to Citizens and Workers: An Interview with Deborah Meier https://thesystemsthinker.com/from-students-to-citizens-and-workers-an-interview-with-deborah-meier/ https://thesystemsthinker.com/from-students-to-citizens-and-workers-an-interview-with-deborah-meier/#respond Wed, 13 Jan 2016 14:15:00 +0000 http://systemsthinker.wpengine.com/?p=2110 ou and some colleagues are on a retreat, discussing long-term strategies for your organization. As the hour grows late, someone brings up the issue of future capacity: “What skills are we going to need our workers to have down the line?” People toss out terms like creativity, self-motivation, technical knowledge, the ability to collaborate, flexibility, […]

The post From Students to Citizens and Workers: An Interview with Deborah Meier appeared first on The Systems Thinker.

]]>
You and some colleagues are on a retreat, discussing long-term strategies for your organization. As the hour grows late, someone brings up the issue of future capacity: “What skills are we going to need our workers to have down the line?” People toss out terms like creativity, self-motivation, technical knowledge, the ability to collaborate, flexibility, the ability to learn. Someone else leans forward and asks, “So are kids learning these things in school now?”

Acclaimed educator and writer Deborah Meier has spent more than 30 years thinking about these questions and about what it means to be an educated person in today’s society. As the founder and principal of several inner-city public elementary and secondary schools in New York and Massachusetts, she has made her career helping children in underprivileged communities build productive, meaningful lives.

To Deborah, the core mission of schools in a democracy is producing critical, thoughtful, interesting citizens and workers. From her experience, the current emphasis in the U. S. on standardized testing, as required by the 2001 “No Child Left Behind” Act, stands in the way of achieving that goal., “If Americans had an edge in the world, it was that they were presumably more ingenious, more self-initiating,” she says., “The special American genius was our inventiveness. That spirit of inventiveness is what schools don’t currently reward. It’s not what you’re supposed to be thinking of when you’re taking tests; you’re supposed to be thinking of the rules of the game, not how to break the rules or how to invent new rules.”

Dynamic Learning Communities

Deborah knows about inventing new rules. She became an educator in the 1950s, starting as a part-time substitute teacher in the Chicago public schools while her children were young. During that experience, she found that school was “for many kids irrelevant, and the extent to which it was relevant, didn’t produce lively minds. The same was true for teachers — the environment was barren and sterile. I thought it was amazing that they came to school each day.”

“If Americans had an edge in the world, it was that they were presumably more ingenious, more self-initiating. The special American genius was our inventiveness. That spirit of inventiveness is what schools don’t currently reward.”

While teaching kindergarten in Harlem in the early 1960s, Deborah began to work with education professor Lillian Weber of the City College of New York, who developed the “Open Corridor” concept. In it, three or four teachers work together to turn their hallway into a shared children’s space. By collaborating in this way, the instructors demonstrate cooperation and create an engaging and dynamic learning community.

In 1974, Deborah was recruited to apply these progressive ideas in launching the Central Park Elementary School in East Harlem, one of the poorest areas in the city. The school and three others she spearheaded became highly successful, with more than 90 percent of the students who entered the Central Park East Secondary School going on to college. More than two decades later, Deborah moved to Massachusetts to found the Mission Hill School.

Habits of Mind

The schools that Deborah has launched all share certain characteristics. They are relatively small; the Mission Hill School, with around 180 students, is about one-third the size of the average school in Boston for that age group. Classrooms look like a combination of art room, science laboratory, and library. Children from kindergarten through 8th grade study a common set of themes — American history in the first trimester, ancient history in the second, and science in the third — so that the older students can model certain “habits of mind” for the younger ones.

According to Deborah and her colleagues, these habits are crucial for exercising judgment on complicated matters. At Mission Hill School, developing such intellectual skills is a core part the educational process. They include:

  1. Evidence: How do we know what’s true and false? What evidence counts? How sure can we be? What makes it credible to us?
  2. Viewpoint: How else might this look if we stepped into other shoes? If we were looking at it from a different direction? If we had a different history or expectations?
  3. Connections/Cause and Effect: Is there a pattern? Have we seen something like this before? What are the possible consequences?
  4. Conjecture: Could it have been otherwise? Supposing that? What if?
  5. Relevance: Does it matter? Who cares?

The habits of mind are supplemented by habits of work: meeting deadlines, being on time, sticking to a task, not getting frustrated quickly, listening to what others say, and more.

Because kids learn by seeing adults practice these habits as part of a democratic community, the school operates as a staff collective, with input from a board of directors composed of five teachers, five parents, five people from outside the school, and two students. Most meetings are open to all, including students, who are encouraged to submit proposals. Children then apply these skills to making decisions within their classrooms.

Mission Hill School also brings the classroom into the larger community and the larger community into the classroom. The school has close ties with local museums, a farm, and several sports programs. Older kids participate in a “school to community” initiative, in which they spend one morning a week for 12 weeks working at a nonprofit or business. “The main point,” Deborah says, “is that it’s a place where we know there are some interesting adults doing interesting things who love what they’re doing.” In a similar way, if the students are studying ancient Greece, “we try to find people who have ancient Greek expertise, either as hobbies or professions, so our kids see that there are people who study this all the time and to whom it is a life love.”

For inner-city kids in particular, finding and cultivating a passion can be a lifesaver. According to Deborah, “Over the years, we have gathered a lot of evidence that this approach has had an impact on kids: fewer of them drop out, get in trouble, or despair of their lives. The vast majority go on to post K–12 education; they come to think that having interesting occupations is a possibility for themselves, not just for other people; they are likely to have strong hobbies; they want their kids to have an education like this too.” She adds, “The other exciting thing is how many teachers come see our schools, hear our stories, and want to start schools like it. We started with just one in NYC and now there are hundreds. The same is true with parents. It speaks to something that we’re longing for in our lives.”

Real-Life Achievement

By law, students at Mission Hill School must take standardized tests, and overall scores exceed those of many other schools in Boston. Nevertheless, the staff doesn’t let test preparation alter the curriculum or the process for evaluating student performance. As a requirement for graduating from eighth grade, pupils present portfolios of their work in different fields of study to committees of five people, including external reviewers, a member of their family, and two members of the faculty. A younger student also sits in as a learning opportunity. The centerpiece of each portfolio is a single, extended piece of work. The committees question presenters and rate the depth and breadth of their understanding of the material. “We are pushing kids to look at themselves as learners,” comments Deborah.

Deborah sees the portfolio process as a better, if somewhat more time-consuming, way of assessing kids’ competence than standardized testing. She says, “Higher test scores are supposed to be a measure of some real-life achievement and yet we have isolated them from real-life achievement.” As an example of this discrepancy, Deborah points out, “Young people who started as students in the seventies — the period in which we started concentrating on testing — are reading precipitously less well than the students who started reading in the forties, fifties, and sixties. If you ask kids, they’ll tell you, ‘When testing is over, we stop reading.’”

Awakening to the Future

So what can we do as a society to ensure that students gain the skills and knowledge they need to be the leaders of tomorrow? According to Deborah, “I think we start off by deciding what’s important to us and how we would know whether we’re achieving what we had in mind.” Another step is to create ways for parents and teachers to get to know each other, through maintaining smaller classes, keeping kids with the same teachers for several years, and scheduling additional time for them to meet. Public policy could support this process by requiring employers to give employees time for visiting their children’s schools., “We could maybe make it a duty of citizenship, like jury duty is,” Deborah comments.

With many educators, parents, and politicians beginning to raise the alarm about the downside of high-stakes testing, Deborah hopes that we’re on the cusp of an awakening that “whoops, this is not what we’ve meant to be doing to children for 20 years, this has nothing to do with what we dream about, this is not what the American future is supposed to be, this is not how to lead a competitive race with the rest of the world.” The fact that the choices we make now will affect our ability to muster an effective workforce and an engaged and thoughtful citizenry well into the 21st century is something that everyone can agree on.

Janice Molloy is content director at Pegasus Communications and managing editor of The Systems Thinker newsletter.

Resources by Deborah Meier

In Schools We Trust: Creating Communities of Learning in an Era of Testing and Standardization (Beacon Press, 2003)

The Power of Their Ideas: Lessons for America from a Small School in Harlem (Beacon Press, 2002)

Will Standards Save Public Education, series editors Joshua Cohen and Joel Rogers (Beacon Press, 2000)

The post From Students to Citizens and Workers: An Interview with Deborah Meier appeared first on The Systems Thinker.

]]>
https://thesystemsthinker.com/from-students-to-citizens-and-workers-an-interview-with-deborah-meier/feed/ 0