Psychometrics and Crowd Wisdom

We hosted Preetha Ram, co-Founder of Open Study for sessions that we split into two audiences – a core academic group on our main campus and an instructional design / enrolment / student services group at our millyard COCE campus.

One good indicator of a solid product is when distinct audiences and non-believers get enthralled. The sign of a spectacular product is when someone who has seen this show before, (me) and already was a fan, got to see the continued/continuing evolution of both the product and its potential.

Open Study people are working on the back end to review what might be gleaned from a working group of 100,000+ crowd wisdom generators. This is taking them beyond what they’ve had for a while: 24/7 student support, community and game intrinsic motivation (“stickiness”), to learning analytics and demonstration of competencies among their user group.

All competency-based education systems (WGU, P2PU, Excelsior, MITx) need to continually focus on the importance of the “how do we know they have learned?” question. Open Study participants who are answering hundreds (or thousands) of math questions in supportive and constructive ways are not just displaying math ability. They are demonstrating effective non-cognitive skills in tandem with cognitive; domain expertise in conjunction with tangible skills. Analysis and demonstration of specific user correlations as Preetha described might just add up to successful psychometric testing.

With the data they have at their fingertips, Open Study has the potential to track not only teamwork, helpfulness and engagement, as they do now, but also to extend to LEAP / Institute for the Future key skills like problem solving, critical thinking and creativity. I was delighted to witness the immediate engagement of many in the academic session (esp. Kim Bogle – chair of the assessment committee at SNHU, and Mark McQuillan –wonderful – new Dean of School of Ed) discussing how data and metrics might be mined to demonstrate competencies.

It was exciting to sample this “esprit de corps” among true educational entrepreneurs, eager to respond to the genuine needs of students.
Thanks to all who participated.


New (Disrupted) Faculty Roles

Rather than the Sage on the Stage or Guide on the Side, you’re going to see a growing embrace of the Sage on the Side model. The need for an instructor with high-quality, in-depth domain knowledge (The Sage) will never go away. But, in an age of ubiquitous information, he just doesn’t get a stage anymore. However, an age of ubiquitous information also means a lot of that information is going to be crap. An education Sherpa is needed to help students develop information literacy so they can sort the good from the bad.

The above quote comes from a Campus Technology (December 2011) report entitled What’s Hot, What’s Not 2012

As we at the Innovation Lab are looking at alternative models for T+L, we are enthused by initial conversations with Open Study and LOVE Philipp Schmidt and the guys at P2PU. That seems to make us threatening to some of our traditional faculty colleagues who see us as part of the conspiracy to replace them with robots, peer-to-peer non-experts or, to put it another way, to “SurowiekiTM” them out of existence.
We are hoping to work with Open Study on a research project as to how implementation of an alternative means of student support affects a classroom (online or face to face) community and the T+L experience.
Here’s what I think, or hope, or hope I think… It sort of builds on the above CT quote:

  • 80% of questions asked in an online class environment do not require a PhD to answer (that’s a near quote from Carol Twigg – hybrid teaching guru)
  • 80% of questions asked in a class were asked last year (Wayne Mackintosh – OERu guru)

Funny that they both landed on 80% – as the percentage of background, less-than-challenging questions that (perhaps) technology, or someone other than a Full Professor can help with.
Now if I were a FT professor, I could take this one of two ways – and I understand both perspectives:

  1. OH NO – My job is being taken away, how dare “they” how can computers and T.As or mentors replace me with my experience, passion and qualifications OR
  2. GREAT – now I don’t have to answer all those mind-numbingly dull, repetitive questions that I didn’t spend 5,6,7 years getting my PhD for – the ONLY time I will have to step up is when a question merits my expertise!

When answering only the 20%’s (the subject-matter-specific, *interesting* questions)– the faculty member who gets it can now oversee 5 x as many classes and will feel challenged and stretched in her/his discipline rather than in the basics that a computer, or a good generalist T.A / mentor could easily address. If this is built correctly, we enhance and honor the expertise of the expert and free them up to stay at their cutting edge of knowledge, rather than dealing with the “I uploaded the doc to Blackboard but it failed” / “I lost my password” / “my textbook hasn’t arrived yet”… etc.  That an FAQ (per: Wayne) or a “mentor” (per: Carol) covers for them.

One obvious flaw in this argument that could push faculty back to a fear and distrust would be if there weren’t 5 classes for them to oversee – and their load got cut. To that I would say that if we do embrace these new (disruptive) models then we have a shot at engaging the extra millions who need the education but can’t get access due to outdated models and non-scalable costs. No evolution / disaggregation of roles and I feel that traditional faculty roles will be threatened.  Better efficiencies, everyone playing to real strengths and I think we’ll get there.

“Sage on the side” / “Sherpa” – great terms – I wish I’d said that –  although as the saying goes:  Talent borrows, genius steals – so likely I will

Why free online resources will destroy universities

Nothing super new, but a succinct review of why “we” really must keep moving to look into new, disruptive models by Adrian Hon in today’s Telegraph (UK)

“We have a romantic ideal of universities being places of higher education where students absorb knowledge, skills and critical thinking – an ideal modelled over centuries on universities like Oxford and Heidelberg. Since they used a multi-year, highly structured residential course of lectures, tutorials, and exams to produce smart graduates, we now believe that this same model ought to work for the majority of the adult population.

We’re wrong. The simple fact is that university lectures never worked that well in the first place – it’s just that for centuries, we didn’t have any better option for transmitting information. In fact, the success of top universities, both now and historically, is in spite of lectures, not because of it.” …


NOTE – one of the funniest comments and perhaps the reason there will always be a role for the traditional college format is posted by pewkatchoo:
“Please, no, no, no. University is a break for the hard pressed parents as much as it is about educating our young. After 18-19 years of putting up with the growing pains (particularly the post 16s) of our truculent offspring, paying the cost of getting rid of them for 3-4 years is a positive relief.


Game Theory applied to Online

I’ve been reviewing for a few weeks now, our online (Blackboard) courses at SNHU – and thinking of ways to make participation more meaningful. I hadn’t managed to get further than describing it as an Angry Birds model – where increasingly difficult screens become available and you spend longer and longer (being more engaged), as you progress and get “into it.”

I found that by editing a GC magazine article, I could illustrate where I think we should be looking for better engagement with course materials and content:

GC December 2010 – “Now that the social layer has been built, some people say, the next layer will be the game layer….
EDITING this slightly allows me to present:  GAME THEORY APPLIED TO EDUCATION – TEACHING AND LEARNING: my italics added…

“Now that the social layer has been layered into education, some people say, the next layer will be the game layer….
With the theory of game design applied to an online course, you want a curve like this: increasingly large payoffs (points towards grades or perhaps initially just appreciation and comment from instructor / peers) at random but increasingly spaced intervals. So the first payoff is very small (great post Johnnie – you have really grasped this concept), and the next payoff is a little bigger, (10/10 for that short paper Wendy) and the next one … To begin with you get a payoff for one out of five actions, then it’s one out of twenty, then it’s one out of fifty (Final paper was a bear but I “ACHIEVED IT” / “cleared that level!”) – but those intervals have to be random. That is the key to human addiction.
Game mechanics mostly means overtly turning thingies into games. Even without overt game mechanics, the gamelike nature of social interaction is why we’re addicted to social media in general. It’s why I’m addicted to email (posting to the discussion boards / interacting with the class). Most emails Many posts I get (read) are bullshit (not hugely helpful or insightful). But every once in a while, I get an email read a post that feels affirming in some special way – etc etc etc…  Ergo I check my email the discussion boards / work on my classwork about every twenty-seven seconds. You can’t blame the Internet for the flaw that we’re basically crackheads. Our bodies are designed with the flaw of wanting to be crackheads. And manipulating that flaw is partly how you get 500 million users for your thingy a hugely engaged and engaging learning experience….

Too simplistic??