Making engagement observable: developing mLearning methodology (Part 1)

I am in the process of writing my methodology chapter for my thesis and wanted to share some of that here as methodology is another area where some of the complex permutations of mLearning rear their heads. Please note that this is a draft, it is more than likely rife with some typos, and it is incomplete. That is what I have now is not what will be here towards the end of this process. Some nuances or blatant contradictions will emerge in the pilot forcing a redesign of this methodology or even a complete reworking of the research questions. Which is probably where I should start:

1. How do faculty and graduate students in higher education in the Humanities in South Korea use mobile technology to support their learning practices?
2. What work is being produced in mobile technology in Korean higher education in the Humanities?
3. What is the nature of participation for graduate students in the Humanities in higher education in Korea?

I have about 85 more things I would like to know, ie other research questions, but my supervisors rightly drilled me down to these potentially answerable questions. In further evidence of why they are supervisors and I am a student, they also advised me to zero in on the research questions and then work the questions both ways, one way towards theory and the other way towards methodology. So something that looks like this:

After months of floundering about in the heady ether of my theory chapter, trying vainly to articulate a position amidst the tenets and twists and turns of Community of Practice and Multimodality, his sound advice was simply to stop that, look at your research questions, and to think logistically about what data could reasonably be collected to answer those. Then draft a methodology and constantly work back towards the research questions. Occasionally, stop that and work the theory back towards the research questions as well. And then everything made sense. I am bound by the logistics of data collection and shouldn’t shy away from that.

But then it left with me with the complexities of methodologies for mobile activity. A set of methods and a means to analyze the data collected from those methods. So this is what we have here. My first pass on this. Some of it is very orthodox (narrative interviews), some slightly less so (mobile prompts, media artifacts), but all of it, I hope, helps triangulate the findings. I am putting the sources up front here to save anyone from having to read through this whole thing, but I thought it might be useful for other practitioners or fellow students out there. I will post this in a few parts to break up the reading with the next part being some specific mobile media models.

References

• Bird, P., & Soreze, F. (2009). Methodological Issues in a study of Mobile Learning as a Disruptive Innovation.
• Frohberg, D.; Goth, C. & Schwabe, G. (2009). Mobile learning projects- a critical analysis of the state of the art. Journal of Computer Assisted Learning, 25: 307-331.
• Pachler, N. (2010). Mobile learning: structures, agency, practices. Springerverlag Us.
• Pachler, N. (2007). Mobile learning: towards a research agenda. Retrieved April 10, 2013 from http://eprints.ioe.ac.uk/5402/1/mobilelearning_pachler_2007.pdf.
• Traxler, J., & Bridges, N. (2005). Mobile learning–the ethical and legal challenges. Mobile learning anytime everywhere, 203.
• Vavoula, G. & Sharples, M. (2009) Meeting the Challenges in Evaluating Mobile Learning: a 3-level Evaluation Framework. International Journal of Mobile and Blended Learning, 1,2, 54-75.

Methodological Issues specific to mobile learning

The overall aim of this research is to determine the particulars of graduate student participation in the Humanities in higher education in South Korea through mobile technology, how mobile technology is used as part of a larger process of ‘coming to know’, a process whereby meaning is constructed through the use and mastery of a number of different tools, technological, intellectual, and physical (Saljo, 1999). This process of ‘coming to know’ through the mastery of tools such as mobile technology presents considerable methodological difficulties in terms of data collection, observable behavior, social interaction, and disciplinary output (composition, etc.). As such, methodologies chosen to observe, collect, and analyze data related to this process of ‘coming to know” through disciplinary process must reflect the fluidity of student engagement in terms of transactional distance (Park, 2011) and oscillations between independent and socialized activity. Further, the chosen methodologies need to span the divide between formal and informal learning, acknowledging the seamless learning that occurs when students engage with mobile technology across informal and formal strands of learning and across different disciplinary contexts (Looi et al, 2009 & Sharples, 2006) and across fluid learning environments (Taylor, 2007). This is methodologically challenging. However, the theories employed in this thesis, Community of Practice theory and Multimodality, enable a vantage point for observation and analysis where disciplinary activity mediated through mobile technology is governed by community practice and where meaning-making and related outputs are socially negotiated. It is through community practice and social negotiation of meaning that activity becomes observable and analyzable.

The methodology being presented and critiqued in this section is organized according to the needs of the research questions as well as through the investigation of mobile learning research in terms of data collection and mobile media. Whenever possible, this critique is centered specifically on the mobile learning practices emerging in the Korean context, but further methodological research is presented that demonstrates the methods emerging in the mobile learning field internationally. This research, whether Korean or not, will be analyzed according to its capacity to provide a methodology for this thesis that does the following:

• Proves consistent with the Korean cultural context of mobile technology use, i.e. is technologically localized to the Korean context of coming to know through mobile technology
• Provides evidence of informal or formal participation by graduate students in the Humanities in Korean universities, participation that oscillates between high and low transactional distances and individualized and socialized behavior (Park, 2011)
• Proves logistically feasible in terms of data collection and participant access
• Allows for the inclusion of qualitative data that is textual or multimodal

Please note that these conditions are used as guidelines to analyze the following methodological approaches. Following this critique, a review of what this thesis is hoping to both embrace and avoid in terms of methodology will be presented along with my chosen methodology and justification for its inclusion.

There are several types of methodologies that have proven useful to conducting quantitative and qualitative research in mobile environments or use. However, methodologies designed or appropriated for mobile use, henceforth known as mobile methodologies, are generally difficult to conceive and construct. The first, especially apt for the purposes of this thesis, is the general translation/transportation of mobile learning into the accepted structures and practices of higher education; this proves difficult to conceptualize and hypothesize sufficiently to cover the variables being observed (Bird, Soreze, 2009). For the purposes of this thesis, mobile learning in the universities of Korea exists in a particular friction of formal top-down driven models designed to support existing practices and informal, bottom-up, often student-centered and led mobile environments designed to provide social and peer support. It is difficult, as Bird & Soreze stated, to hypothesize sufficiently broadly to encapsulate this informal/formal spectrum and equally difficult to employ methodologies to support these hypotheses. Based on how the research questions are formulated, it does not exclude the use of either quantitative, qualitative, or a mixed method approach. In fact, some have gone so far as to state that a mixed method approach is optimal as it allows for the “capturing of different perspectives of the learning experience” as well as providing some mechanisms for validating collected data (Vavoula, Sharples, 2009).

A purely qualitative approach suffer from “the accuracy of recall” syndrome in the data collected through retrospective interviews, diaries, or attitude surveys reflecting the participant’s concern in their self-projection (2009). In short, the participant will adjust reflection and participation in keeping not with the accuracy of their responses to the questions posed, but rather to their own sense of self-identity or in relation to the researcher. The accuracy of the responses will prove revealing in terms of self-projection and motivation for participation, but not always so for the questions asked directly. This can be mitigated through the collection of supplementary qualitative data, such as recorded video, audio, observation notes, and other artifacts, designed to contextualize the mobile learning in a larger context of interaction, or even within a Community of Practice. This thesis employs several of these methods in the understanding that the existing literature in the Literature Review is insufficient for revealing the complexity of participation for graduate students in the Humanities in Korean universities. Qualitative methods provide a safeguard for ensuring that themes and practices can emerge from the data collected that might not emerge from more strictly controlled methodological approaches.

Other purely quantitative approaches, some facilitated through technological data collection methods, often suffer from a lack of social, community, or motivational evidence for participation. These quantitative driven approaches, some of which are outlined below, might include technological solutions such as “mobile eye tracking or wearable interaction capture kits” or the more traditional means of collecting mobile technology use data and offering subsequent analysis based on specifically defined and controlled observation points (2009). There are several challenges involved in evaluating mobile technology use through a quantitative approach, most notably in those that oscillate between formal and informal learning (McAndrew, Taylor, Clow, 2010). The approach that McAndrew, Taylor, & Clow (2010) put forth for evaluating mobile learning in terms of both the quality of learning in technology and the nature of interaction with that technology provides evidence that a hybrid methodology is appropriate for observing such complex behavior. It indicates that methods can be appropriated and fused to observe complex phenomena. For this thesis, although useful in terms of providing evidence of a hybrid approach, it proves less applicable as this thesis is concerned with existing practices, existing participation in formal Communities of Practice, and how mobile technology provides mechanisms for new practices to emerge. We are not so much concerned with the quality of learning, but rather than the transformation of scholarly practice as a result of mobile technology. Yet, McAndrew, Taylor, & Clow provide a convincing approach that hybrid approaches are more appropriate to the mobile medium.

For mobile learning, these quantitative methods are surprisingly complementary to the qualitative ones as they provide those different perspectives on learning in mobile technology that Vavoula & Sharples (2009) deemed as optimal. As such, the methodological structure of this thesis will be evaluated upon completion of the pilot study to ascertain whether the inclusion of some quantitative methods might further contextualize participation in mobile environments for graduate students. As it stands, this thesis employs a more qualitative approach as it ‘is much better suited that a quantitative one to the task of understanding how complex, highly context-sensitive processes unfold in organizations and how they impact on those involved’ (King, 2000, p.590). The organization under observation, the university, is materialized through graduate student participation in the complex, highly sensitive process of coming to know in a disciplinary context.

Frohberg, Goth, & Schwabe (2009) outline several methodological approaches for use with mobile learning, include the Review as a research method unto itself. The review involves the analysis of work done in the field to date in an attempt to discover “patterns and gaps in the research field” (2009). A review approach, while outside the scope of this thesis, is beneficial to the mobile learning research community as it provides benchmarks of activity and progress from which prospective researchers can gauge the relevancy and redundancy of their research questions. The review approach, albeit with limited scope and cursory application made evident through a Literature Review, was applied in this thesis specifically in the Korean context in an attempt to determine what, if any, mobile learning activity was taking place in higher education in the Humanities amongst graduate students, to subsequently analyze that activity, and to identify gaps in the research to determine if the research questions presented in this thesis were relevant, redundant, or otherwise. What the Literature Review made visible was the need for a methodology that provides a means of observing and analyzing behavior in mobile technology in one of those “gaps in the research field”, namely how mobile technology is being used to make meaning by graduate students in the Humanities in Korea. Seipold, Pachler, & Cook (2009, March) provide an outline of how this might be accomplished methodologically by stressing the potential methodological focus of observation on the activities of learners in the context of university and their life worlds in mobile settings. This focus on activities of learners across university and life world settings correlates adequately with the research questions’ focus on graduate student participation in the Humanities across informal/formal and individualized and socialized settings. Furthering Seipold, Pachler, & Cook’s outline, further evidence can be drawn from the resources which learners are using (“in terms of agentive and meaningful activities”); in the case of this research, this could include mobile applications specifically designed for formal university application, informal mobile applications designed for and by students, and informal media manipulation tools and social media channels. For the purposes of the chosen methodology for this research, there is an appropriation of the term potential in the phrase “potential inherent in these resources and activities” (2009, March) as meaning the potential of mobile use to transform practice and allow for meaning-making rather than a focus on the potential for structured output or formalized assessment. In short, this research is focused on practice rather than outcomes and the term potential is defined as such.

Seipold, Pachler, & Cook (2009, March) outline potential qualitative methodologies as grounded theory, individual case studies, discourse analysis, and others. These methodologies and means of analysis are useful for this research as they stress the emergent properties of learning in these mobile contexts, that is much of the subsequent framework from analysis will emerge from the data collected. This research will employ a method that adopts elements of grounded theory, but does not ascribe to a grounded approach in its entirety. Data will be collected according to its relevance to the research questions that have guided the theoretical and methodological positions of this thesis. Yet, emergent themes and properties that emerge from the pilot will evolve the research questions and methodological approach of the larger research study. Case studies are valuable for this thesis as they provide contextual evidence of mobile use for meaning-making in the Humanities; a hybrid case study approach will be adopted for the purposes of this research that focuses on how individual graduate students in the Humanities in several Korean universities use mobile technology to participate in disciplinary practice and make meaning. The case in this case study will not be one geographical location, but rather across multiple universities. The case is the larger Community of Practice that might exist that governs, directly or indirectly, graduate students’ participation in the Humanities and how their use of mobile technology influences that participation. As such, there is less focus on the dichotomies of “in school” and “outside school” that are positioned to analyze the potential of mobile learning (2009). This research assumes that the oscillation between informal and formal practice is a constant oscillation and that categorizing mobile activity according to one compartment or the other severs that mobile activity from the larger process of coming to know in the graduate student in the Humanities. It is critical for this research to position learning at the individual level within a larger disciplinary and organizational community, rather than at an organizational level with different actors in an activity system.