So this is the third part of this methodological exploration of, well, my methodology from my thesis. I thought at least some of this would be useful for any aspiring or current PhD students out there. Or anyone trying to conduct a large-scale (kind of) qualitative study.
Please keep in mind this highly subject to change. I suspect I will be changing it considerably after a few more meetings with my supervisors and, most especially, afte the pilot which I hope to do this summer. I am sticking with guidelines I have crafted for myself in terms of what my methods and overall methodology should provide me. In short it should do the following:
- Prove consistent with the Korean cultural context of mobile technology use, i.e. is technologically localized to the Korean context of coming to know through mobile technology
- Provide evidence of informal or formal participation by graduate students in the Humanities in Korean universities, participation that oscillates between high and low transactional distances and individualized and socialized behavior (Park, 2011)
- Prove logistically feasible in terms of data collection and participant access
- Allow for the inclusion of qualitative data that is textual or multimodal
Methodological Pitfalls: Avoiding Technology Acceptance Models
Pitfalls is a strong word here, but I wanted to steer clear of methodologies that focused on technology acceptance. Mobile is a thing in Korea, an already established thing, so I don’t need to spend time (I think) on whether or not they have accepted something I know they have.
In terms of methodological characteristics to avoid, these include an avoidance of any sort of technology-driven or technologically deterministic model of data collection and analysis. As such, the methodology use for this research is willing to follow the graduate student through their process of meaning-making in a disciplinary context, whether that meaning-making occurs in mobile technology or another form of ICT. Building on this desire to avoid a technologically-driven model, this research is not concerned with a Technology Acceptance Model (TAM) or any such model for measuring acceptance or quality of use of mobile environments, such as those presented in Kim, Fisher, & Fraser (1999) & Lee, Yoon, & Lee (2009). This research assumes current use of mobile technology and hopes to capitalize on existing formal and informal practices for making meaning in mobile environments. However, TAM and other acceptance models offer a potential point of later analysis, namely trying to ascertain whether graduate students would be willing to use mobile technology for ‘new’ formal practices, such as those introduced in a participatory design intervention. For the purposes of data collection itself, namely the interview schedule, it offers a cautionary tale on carefully phrasing questions to avoid making participants associate new practices strictly with new technology, or new uses of existing technology. The focus should be on the meaning being made in these environments towards disciplinary participation and the interview schedule should be structured as such. Questions from the interview schedule designed to answer research questions are structured to avoid this causation whereby new practices are linked exclusively to new technology.
At this stage of understanding meaning making mediated through mobile technology in a disciplinary context, it isn’t necessary to employ a design driven methodology for structuring this research as it assumes a fully understood, stable environment requiring an intervention, technological or otherwise. Design methodologies might be useful at later stages of this research once the community of practice and the graduate students interaction with mobile technology to participate in disciplinary activity is fully understood. If interventions designed to foster further interaction or accelerate meaning making in a mobile context are presented as warranted, then a design methodology will be evaluated for its capacity to enact such outcomes. Building on this, at this stage this research is not concerned with replicability and transferability of findings to other contexts unless the data and findings suggest such applicability to other contexts. It is assumed, in parallel with a case study approach, that the context of learning being analyzed in this research is unique to the community, practices, domains, and artifacts presented here. As such, findings will be methodologically collected, critiqued, and presented as relevant to this specific Korean context.
While not specifically a design model, the methodology chosen for this research will avoid the Task Activity Model (Frohberg, Goth, Schwabe, 2009) and instead focus observations towards context and communication (less on control). This avoidance of the Task Activity Model is not due to any methodological shortcomings of this approach, but rather its working assumption that tasks, activities, processes or workflows have been identified in the context of the Korean Humanities graduate student environment. In keeping with, but not adhering to, the emergence of grounded theory, the data collection techniques employed for this research allow for the emergence of informal and formal practices that collectively identify graduate student participation in the Humanities. Building on this focus on context, this research is less concerned with assessment and outcomes than practice and process. The research questions are designed to ascertain what graduate student participation looks like in the Humanities when mediated through mobile technology; media and other compositions will be collected as evidence of meaning making, but not specifically how the Humanities community might assess these compositions. As such, there is no methodological technique employed to look at alignment of these compositions with formalized curricula.
Methodological Adherence
The methodological techniques used for this research embrace collecting qualitative data demonstrating the following:
- Engagement through mobile technology with the graduate students’ discipline, informally or formally
- Graduate student participation across formal and informal, individualized and socialized activities of meaning making, paying particular attention to the oscillation between different states of being consistent with seamless learning. Mobility in this context, “is not an exclusive property of the technology, it also resides in the lifestyle of the learner, who in the course of everyday life moves from one context to another, switching locations, social groups, technologies and topics” (Vavoula & Sharples, 2009)
- The creation and use or potential use of multimodal artifacts for disciplinary participation
As this research is less concerned with assessment, but rather process and outcomes, it will be drawing on Vavoula & Sharples (2009) considerations for evaluating mobile learning, namely how to capture and analyze learning in context, with consideration for learner privacy. This context will be established methodologically in this research by collecting narratives of participation in a disciplinary context, as well as collecting created media artifacts for analysis. Further, this research is attempting to “look beyond measurable cognitive gains into changes in the learning process and practice” (2009). The chosen methodologies attempt to identify and theorize on these changing practices by following the graduate student across informal and formal channels as they create meaning in a disciplinary context.
Research Design and Methodology
The methods chosen for this research support a qualitative methodology for the purposes of answering the research questions on mobile technology use and graduate student participation in the Humanities. The methods for data collection are presented below, with justification as to the methods of their selection.
Narrative Interviews
Since the research questions focus on graduate student participation in the Humanities, formally or informally, it will be important to construct the narrative of what that participation looks like and the oscillations that occur within this context between formal and informal practices, and individualized and socialized activities with high and low transactional distances (Park, 2011). To coherently gauge this participation and these oscillations, it is important to provide the students a voice for establishing their identity in these movements and activities. This research does not presuppose a particular a level of receptiveness to or use of mobile technology, but rather attempts to gauge that receptiveness based on the individual transcripts and subsequent narrative analysis (Robson, 2002). The narrative interviews are designed to identify and map the processes, activities, and reflections that graduate students encounter as they move through their graduate education. They also are designed to let data emerge from the transcripts that might demonstrate the mitigating circumstances that affect participation in the Humanities, such as personal or family commitments.
Participation Self-Reflection
Further to these narrative interviews will be reflections conducted at intervals during the research process by the participating graduate students. These reflections will be requested from the participating graduate students in whatever medium is convenient to their current practices (social media, textual, or otherwise) and will attempt to gauge graduate engagement in the Humanities and the use of mobile technology to mediate that engagements. This secondary data stream will be analyzed in conjunction with the narrative interview transcripts in an attempt to extract answers to the research questions of how mobile technology is being used by graduate students in the Humanities, what is being produced there, and what participation in the Humanities looks like across all states of activity (informal, formal, individualized, etc.). For the purposes of the pilot, this will involve two reflections to triangulate the findings from the one narrative interview. These self-reflections will be paired with a further reflection which will take place exclusively through mobile. One question will be delivered to the participants asking them to reflect on one granular aspect of disciplinary practice or mobile use through their mobile technology; this question is designed to provide a less rigorous means of participation and will be employed in the pilot study to determine its use in answering the research questions. If considered useful, it will be employed at intervals throughout the larger research study.
Artifact Analysis
Participants will be asked to make available an artifact (media or otherwise) created in mobile technology to support disciplinary participation. This artifact can be mobile media, a textual composition, a dialogue-based activity, a collaboration, or otherwise. These artifacts will be collected and analyzed to answer the research question on what is being produced in mobile technology in the Humanities and what these compositions/productions might be classified as (informal, formal, individualized, socialized, high/low transactional distance evoking Park, 2011). These artifacts will be analyzed to indicate what practices are made visible through their production; further, the Task Model will reviewed to see if these artifacts represents tasks and whether this Task Model approach has merit for revealing disciplinary practices. For the purposes of the pilot, participants will be asked to make available one artifact for analysis.
These three data collection methods are designed to triangulate the findings emerging from the narrative interviews on what graduate participation looks like in the Humanities and how meaning is made there when mediated through mobile technology. Jointly, they constitute a case study approach (sort of), one scattered across multiple institutions but which encapsulates a Community of Practice (maybe). They also employ methods in keeping with an ethnographic approach, but remain sensitive to the particular Korean context in which embedding the researcher as member of the community would be logistically problematic and might skew the data collected there.
Full Disclosure
I don’t know at this stage whether these data points will provide anything that will get me closer to answering my research questions, although I suspect they will. The methods here are consistent with qualitative approaches being used by the research community I am drawing from (studying mobile practices in Korea/Asia) and so I feel confident that at least one of these methods, but maybe not all three, is the way to go.
Ultimately though, I only know that I need a methodology to enact the pilot which let me gather data on the effectiveness of these methods. I am not married to any of it except the research questions, so I am perfectly happy to change.