A choreographer, a physicist, and an evaluator walk into a theater: Mixed methods research designs for evaluating impact at the art/science/engagement interface
Diane Doberneck, Researcher, National Collaborative for the Study of University Engagement, Michigan State University [connordm@msu.edu]
Paula K. Miller, graduate researcher, Michigan State University [mille995@msu.edu]
John H. Schweitzer, Professor, Michigan State University [schweit1@msu.edu]


Keywords: Informal learning environments, art/science engagement, evaluation, mixed-methods

Track: Program evaluation and assessment

Format: Interactive workshop on research methodologies

Date & time: Thursday 2:00-3:10
Location: Salon 7

A great deal of learning takes place outside of schools and universities in informal learning environments (ILEs), including museums, libraries, science centers, virtual environments, and even in everyday conversations. While these ILEs are acknowledged and praised for their critical role in shaping public understanding of arts and sciences, critics cite the paucity of rigorous evaluation of learning outcomes as a major weakness. Assessment of ILEs poses a challenge for researchers: how do you rigorously evaluate outcomes without the research interrupting the learning activities?

Choreographed by Liz Lerman Dance Exchange, “The Matter of Origins” employs dance movements, stories, and images to explore historical perspectives and cutting-edge physics about our beginnings. In Act One, the audience experiences projected media, soundscape, and dance by a multi-generational cast in a theater. In Act Two, they adjourn to a nearby room for tea, cake, and dialogue facilitated by local scientists. Tea includes dance interruptions and additional science content to stimulate reflection and public engagement about the nature of science, limits of measurement, and meaning of movements—both big and small. This NSF-funded, multi-site, mixed-methods evaluation exemplifies how rigor and creativity are possible when choreographers, physicists, and evaluators collaborate at the art/science/engagement interface. The research design was iterative and generative, accommodating variations in venues and audiences. At all sites, qualitative and quantitative data were collected to assess the impact of the ILE on the audience.

Findings include: 1) significant changes in audience members’ attitudes, interests, knowledge, and behavior towards science; 2) significant changes in emotional engagement with the subject matter during the performance and the tea experience; 3) positive reactions to the opportunity to engage others and scientists in conversation about individual reactions to the performance and its themes; and 4) relationships between audience members’ ways of thinking and their experience at the art/science/engagement interface.

Bell, P., Lewenstein, B., Shouse, A. W., & Feder, M. A. (Eds.). (2009). Learning science in informal environments: People, place, and pursuits. Washington, DC: National Academies Press.

Brown, J., Isaacs, D., & the world café community. (2008). The world café: Shaping our future through conversations that matter. San Francisco, CA: Berrett-Koehler Publishers.

Cooperrider, D. L., & Whitney, D. (2007). Appreciative inquiry: A positive revolution in change. In P. Holman, T. Devane & S. Cady (Eds.). The change handbook: The definitive resource on today’s best methods for engaging whole systems, 2nd edition, pp.73-88. Berrett-Koehler Publishers.

Creswell, J. W., & Plano-Clark, V. L. (2010). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage Publications.

Edwards, D. (2008). Artscience: Creativity in the post-Google generation. Cambridge, MA: Harvard University Press.

Ellinor, L. & Gerard, G. (1998). Dialogue: Rediscover the transforming power of conversation. New York, NY: John Wiley and Sons.

Fenichel, M., & Schweingruber, H. A. (2010). Surrounded by science: Learning science in informal environments. Washington, DC: National Academies Press.

Friedman, A. (2008). Framework for evaluating impacts of informal science education projects.

Gardner, H. (2006). Multiple intelligences: New horizons. New York, NY: Basic Books.

Goleman, D. (2006). Working with emotional intelligence. New York, NY: Bantam Books.

Hammond, S. A., & Royal, C. (2001). Lessons from the field: Applying appreciative inquiry. Plano, TX: Thin Book Publishing Company.

Holman, P. (2009). Engaging emergence: Turning upheaval into opportunity. San Francisco, CA: Berrett-Koehler Publishers.

Lerman, L. (2011). Hiking the horizontal: Field notes from a choreographer. Middletown, CT: Wesleyan University Press.

Neal, C., & Neal, P. (2011). The art of convening authentic engagement in meetings, gatherings, and conversations. San Francisco, CA: Berrett-Koehler Publishers.

Perkins, D., & Tishman, S. (2010). Patterns of thinking. Cambridge, MA: Harvard University.

Pink, D. H. (2006). A whole new mind: Why right-brainers will rule the future. New York, NY: Penguin Group.

Robinson, K. (2011). Out of our minds: learning to be creative. West Sussex, UK: Capstone Publishing Ltd.

Runco, M. A. (2007). Creativity: theories and themes: research, development, and practice. Burlington, MA: Elsevier Academic Press.

Sternberg, R. J., Grigorenko, E. L., & Singer, J. L. (Eds.). (2004). Creativity: from potential to realization. Washington, DC: American Psychological Association.

Tharp, T. (2003) The creative habit: Learn it and use it for life. New York, NY: Simon and Schuster.

Wheatley, M. J. (2009). Turning to one another: simple conversations to restore hope to the future. San Francisco, CA: Berrett-Koehler Publishers.

Please click here to access a PDF of this page:
Doberneck_A choreographer, a physicist, and an evaluator walk into a theater-Mixed methods research designs for evaluating impact at the art-science-engagement interface.pdf

To download materials from this session, please click on the file link(s) below:

Please join the conversation about this session! To do so, please click on the "Edit" tab at the upper right, scroll back down to here, and lead the comment with your name.