Switching Modes: Images to Text
In my classes (undergraduate composition courses, mostly) in Seoul, I am trying to expose my students as often as possible to activities where they are forced to switch between modes and compose based on those transitions. So, earlier in the semester we started with text prompts for writing essays (respond to this data or this article, etc.) and for the second half of the semester I have been attempting to start my activities from a different mode altogether. So, I prompt them with images, sets of images, and sets of sequential images. Some require the student to interpret the significance of a single image, some from a set of unaffiliated images, and some from a set of images that I have sequenced into a meaningful order.
We discuss these images, discuss their significance (however interpreted), and begin to map a coherent means of presenting that significance textually. I like these activities as they activate a few different cognitive spaces simultaneously and the composition becomes a bit more holistic. Not just a textual, linear activity of rendering A to B, but rather a slightly asymmetrical activity of rendering Image A (presented with non-linearity-it is just there all at once) into Sequence B (taking the salient bits of the image, identifying their significance, and identifying how they aggregate into something whole) and then composing in Writing C. I am still emphasizing the skill of writing textually as that is what these particular students need for their particular majors (and what my university emphasizes as an output of these courses).
The added wrinkle on all of this is that this is taking place as a second language exercise for them. The students are primarily Korean (some Chinese, Russian, Mongolian, some SE Asian, a few European students as well), but these classes are all conducted in English. Students are required to demonstrate some cultural agility here as well in that they need to interpret images not only from visual to text, but also they need to disentangle a Korean interpretation (perfectly valid) based on the Korean understanding of colors (and their related significance), composition, etc. from the culture being represented or manifested in the image. They need to become aware of the artifacts of cultural understanding.
Without overtly saying so, I am attempting to increase their capacity for multimodal literacy. More importantly, to translate one from mode to the other and back again, across several languages and several cultural foundations of interpretation. This is an increasingly growing need for several of these students (particularly those majoring in Translation and Interpretation Studies, the speciality of my university). The Journal of Specialized Translation has devoted an entire issue to the particulars of multimodal translation across languages. For everyone who has read Kress, Bezemer, Jewitt, et al, imagine taking that entire process of multimodal interpretation across languages as well.
Pinterest as Tool for Writing Across Modes
That is why I do it, but what do I do. To ease them into it, I start by providing a single image or a linear sequence of images designed to advance a narrative. I generally do this via my slides or occasionally through some other application. I make these slides available for my students as well before the class. After we ease into writing in this way (see image, interpret image, identify parts and aggregated whole, write), we move on to more challenging exercises. I present a series of seemingly disparate images without any evident overriding narrative. I ask them to interpret those (generally in groups), identify the elements/artifacts of the compositions (colors, layout, symbols, gaze, etc.) and then construct narratives to cohesively link them.
The first time, I present my disparate images to them via a Pinterest board I have created. The second exercise involves having them create their own boards for their classmates to interpret. Each individual identifies ten images that have some loose connection (either anecdotally through the composer’s past experience or topically or even artistically; I leave it open), puts them on a Pinterest board with as little accompanying text as possible, and shares that link with another classmate. The second classmate proceeds to analyze, interpret, and compose a response to these collection of images (generally through a textual essay-I can’t divorce myself too far from my university’s guidelines of what the students are to be learning). These responses are then returned to the person who crafted the original board and that author composes a response to the responder. Then, both parties come together to create a board together based on their collaboration, clarification, and feedback processes. This all takes place across languages (from their native language to English, repeatedly) and across modes (from images to spoken discussion to textual composition and back again). It has generated some good results and more importantly it is generating a marked improvement in their writing.
I have theories as to why (beyond those cognitive theories that would be most obvious!). Part of this has to do with plateauing on a particular mode. Text to text interpretation, analysis, and writing is important, but occasionally the linearity of the presentations effects/stunts the output. Students go from A to A. Switching modes (and languages) broadens the interpretative aspects of the writing, forcing a rethink on the processes the students have been using until now.
This is not a particularly original idea, but I have found it quite useful for invigorating some creativity into writing (itself an act of translation and interpretation). If interested in using Pinterest, some good resources here (some information on how universities are using it as well).