The open course on MobiMOOC 2012 has kicked off and conversations are proceeding in that chaotic, highly energetic way that they seems about right for such a lively mix of people. I love the energy and the experimentation that I see there. Some of the more accessible discussions seem to be around assessment and so I thought I would post my reflections on that topic here and marvel at a few of the other threads that remind me of how diverse MOOCs can actually be.
Open Badges and Experimenting with Assessment
The most interactive thread so far seems to be around assessment of MOOCs (which is essentially a meta-discussion on the structure of the course itself rather than specifically geared towards mobile learning). There is a healthy debate going on discussing the role of assessment in open learning and in MOOCs in particular, whether assessment is needed at all, what the practical applications of assessment are, etc. A healthy and encouraging dialogue with people from all walks of life coming together around a shared curiosity or affinity (mobile learning). The thread itself can be found here. Be sure to post your thoughts if you find it interesting.
Some of the more notable bits (and I am extracting these out of context) are as follows:
I agree with the participants that there is a problem of the recognition of the results of MOOC by employers. I am not referring to the posters by name as they might have wanted to remain somewhat anonymous.
In my view, the recognition of non-formal training will be a commonplace.
However, as part of the cMOOC will do it hard, because in cMOOC everyone puts their goal and aims to achieve it. But in fact there is no mechanism to measure the achievements of the participants. Questionnaires and surveys do not count. That is, it is not clear whether the student has achieved their goal or not.
In xMOOS it easier to do – there is a clear agenda, clear targets and deadlines. In Colorado, as far as I know, has set off credits of Udacity.
These are valid distinctions between the x and cMoocs and I am glad to see these distinctions starting to appear. When structure is at a premium, a structure that adheres to a predetermined set of outcomes (outcomes aligned with institutional missions), then the issue of assessment seems less chaotic. By aligning outcomes with institutional missions, it becomes a bit of a self-fulfilling assessment prophecy. We said we are going to do this, we did this, and so we were successful at doing this. Nothing wrong with this at all; it resembles existing assessment mechanisms.
For the cMOOCs, this becomes more problematic as the structure is less rigid and outcomes aren’t necessarily tied to any particular set of institutional objectives. They are (ideally) tied to pedagogical assumptions grounded in learning theory (constructivism, connectivism, etc.) and are therefore valid if not quantifiable. Perhaps our modes of assessment for MOOCs can take a nod from the Humanities where knowledge construction (even validity itself) is constantly challenged, debated, and evaluated. What this means to the average participant in a cMOOC is that this is a fluid assessment mechanism, one that has to be constantly appraised much as one would constantly reflect on their learning. This isn’t the major thrust of that quote above, but the distinction between c and xMOOCs is at the heart of this discussion. Now the attention shifts to Open Badges.
Sometimes giving people awards for participation can distort the conversation. Posting to learn may not be the same as posting for the badge. I have no problem with badges if people need them, personally as an employer however if people give me this level of detail in a CV it raises concerns for me about their capacity to focus, and present themselves appropriately.
I cannot disagree with a bit of this above statement as assessment inherently alters structure, distorts conversation. In fact, I might argue that it is intentionally designed to do so by making outcomes predictable and replicable. To be repeated, one might argue that structure needs to be formalized to a degree. This formalization is a natural process, one that I suspect the cMOOC crowd (myself included) will naturally resist as it is that chaos of novelty/autonomy that provides so much energy to cMOOCs. But I cannot deny it. It is a bit of a conundrum. How do we assess learning in a format with minimal structure and a great degree of learner autonomy? We can make the distinction between formative and summative assessment here (as we should), but I am not convinced that will overcome this assessment of open learning. I think we should be tailoring these informal assessment mechanisms (which in this case for MobiMOOC is the Open Badge framework) to a formative approach. We should acknowledge the active/aggressiveness of participation (I mean that in a positive way), but when open learning is at such a scale I am skeptical that qualitative assessment can occur with any regularity.
So if qualitative assessment is suited to open learning (MOOCs in particular) then it stands to reason (being constructivist in nature) that formative assessment would be collaborative and peer-oriented. Individuals or groups of individuals with shared affinities regulating learning outcomes based on the needs of their communities. I think this is a model that MobiMOOC will pursue indirectly. Somehow trying to qualify the ability of the individual to navigate, participate, create, and reflect within a large, unregulated space. That sounds like a skill worth having to me. But is it an employable skill? (and does that matter?). I believe it is, but it would be hard to couch in ways that would be convincing to a potential employer.
Role Play: CEO Gallagher
I am Michael Gallagher. I run an organization. I exist in a creative oriented competitive environment. Will Open Badges mean something to me? My answer is a qualified yes. If they are presented in a scattershot manner (indicating a lack of the skill of cohesion and comprehensive presentation), then no. It will look undisciplined, raw even. But if I see a hunger to push beyond limitations (socioeconomic or otherwise), a desire to perpetually cultivate skills and abilities, then I will consider this individual better qualified than those candidates with a more structure (less chaotic) path. I want the person most capable of working efficiently in challenging, evolving, chaotic environments (ie, new fields and innovative environments) and Mr./Mrs. Informal Learner, the one with a series of badges and the one indicating that they are perpetually cultivating themselves with whatever tools are presented to them, is the one I want.
Informal assessment of the Open Badges sort will reap rewards for some and for some it won’t matter. Some learn just to learn, some learn to economically better themselves, but we shouldn’t forget that learning is empowerment. Empowerment can take many different forms, certainly, but I do believe that career opportunities is one of those.
All that being said, I could be completely wrong. Happy to discuss.
I am not particularly interested in badges, possibly a function of age and no longer need to worry about the cv-ability of my life. Still the question is an interesting one. Whether or not Badges could have value would seem highly subjective, totally depending on who was hiring and for what. Actually, I once worked for an eccentric engineer /inventor for whom a mooc outlook, skills and ability to navigate chaos would have been excellent preparation. Perhaps that job prepared me for moocs.
I also recall the English dept chair telling graduate students to include ongoing PD including workshops and training on CV’s. That may have been to keep them from looking too naked but also, as she explained, to show taking initiative and interest in on ongoing professional development.
Good points all around, Vanessa! Thanks for this as it does help me draw distinctions between the potential reach of Open Badges. Maybe, as you allude to, the Badges are better served in a portfolio scenario rather than on a CV/resume. A portfolio of project work, multimedia, etc. along with a series of badges indicating a consistent efforts towards professional and personal development would be a nice counterbalance to the CV?
I suppose I am eager to give those without access to formal schooling (many in my field) the chance to cobble together an ‘equivalent’ education, essentially a series of open courses that approximate a university degree. I think this process, if possible and/or recommended, is healthy as it would expose an additional marginalized section of the community to professional life (and economic independence) and expose a structured professional community to an energetic, active, innovative set of workers who hadn’t been heard from before. So that is my interest, more or less. But I am thinking that a portfolio might be a better approach than a CV based on your comments.
Thanks again, Vanessa!
I’d been thinking about portfolios too. Do you recall Carol Yeager from Change 11? She is at SUNY Empire College, which describes itself as Open and experimental and, I gather does a lot of independent studies and portfolio review for credit
Hi Michael, thanks for making me aware of the conversation in the MobiMOOC group. Interesting stuff. 🙂
Two things popped into my head whilst reading your post:
1. “Sometimes giving people awards for participation can distort the conversation.” Absolutely. In fact there’s evidence to suggest that the construct we call ‘extrinsic motivation’ can overwhelm and crowd-out the construct we call ‘intrinsic motivation’. That’s a feature of *badge design* however, not the badges themselves. 🙂
2. What I find really interesting is the way that people project their worldview into what is, in fact, an open and emergent ecosystem. Including me, of course! I think you might find it Alex Halavais’ work interesting: he gets his students to *pledge* for badges – something I think that could work well in cMOOCs. 🙂
http://alex.halavais.net/
Thanks for the feedback, Doug; great stuff. Continuing the thread above:
1. Re: badge design influencing extrinsic motivation (and the course itself). Agreed, that this is a facet of design and not the ‘thing’ itself. No argument there. I think the open standards of Open Badges will allow for flexibility and alterations based on experimentation. So the flexibility creates a pliable, resilient assessment/acknowledgment tool. Kudos to you all for working that into the design itself. I think I am most curious to see the range of potential badges that begin to emerge and begin seeing how multiple badges interact and influence course design. Badges for formative and summative assessment structures. But keeping it open and transparent keeps the experimentation flowing, so well done.
2. Agreed about projecting worldviews onto (eco)systems. There is no doubt about that. That is a predictable starting point, in my opinion, for an individual’s response to a new system or process. First, what can it do for me? How can it advance my particular field? These are all the precursors to the real transformation, but if the system fails to address these early (occasionally superficial) questions, then it fails out of the gate. However, if it acts on these questions and addresses them to some satisfaction, then we see the transformation take place. Not only what can the system do for me, but also allowing for the system to transform me (worldview, fields, even outlook). To reimagine the scope of what is possible. This is what Open Badges (conceptually) did for me. It imagined an environment where alternatives existed to formalized structures of education (and accreditation). Many many details to be worked out, but that imagination stokes creativity leading to experimentation.
I suppose what I wanted to say there is that projecting a worldview onto a system (however limited and occasionally just plain wrong) is a natural process of discernment, a litmus test for immediate gain. If it passes, then it establishes utility and a bit of momentum towards the larger transformational stuff.