Learning to Adapt #2

A follow-up to my recent L-to-A #1 post:

Part II

In a follow-up to my recent post, I want to discuss an often neglected element of online instruction; likely the most critical.

Let’s be honest, in the debate as to whether “online works,” MOOCs confused people. For those new to online, those (faculty) considering for the first time now that someone other than (for-profit/insignificant/not proper school) was doing it, they were the easiest online classes to access, and initially focused almost exclusively on slick content presentation. When people assessed MOOCs and then came back to assess (normal? traditional? retention-emphasized? credit-bearing) online courses they tend to ask for access to the course materials sans instructor or student activity. I’m yet to determine if this is merely an oversight, deference to the beast that is FERPA, or collegial respect for the sanctity of the classroom…

My team at Northeastern has experience gathered over more than 150 years working in and around online courses (there are 14 of us). The range of experience includes time at most of the leading online educators over the last one and a half decades. Even a quick sweep from my mental inventory brings up Drexel, Kaplan, UMass, SNHU, UNH, Capella, Boston University, Harvard / Harvard Extension and RISD. During our time together at NU, we have implemented the work of leading theorists in learning, and cognitive science, assessment, usability, self-efficacy, information architecture, multimedia design, graphics and as I write / you read, are experimenting with gamification and the challenge of intrinsically motivating underserved populations.

The work of people like Dick Clark (USC) and Bror Saxburg (Kaplan) has provided us with depth and detail on the evidence-based learning science behind a lot of our development work. We feel that we have clarity on what works in terms of content formatting (length of chunks, organization of media elements etc.) and of some behavioral elements – the importance of frequent, timely corrective feedback, the level of challenge required to keep students engaged. My team works directly with the full-time faculty trying to build in elements that approximate the type of learning that we have called Online Experiential Learning. Authentic, tangible experiences with opportunities to spiral back, reflect, abstract and re-apply case-based learning in a wider, retained for life context. We focus on, and sweat over, materials, formats, fonts, activities and assessments.

The feedback we receive includes: “I don’t care for the fonts” “or “I took a MOOC last week and their videos were really cool… can’t we do that?”
Colleagues who worked with me at smaller institutions (with even smaller budgets) will snigger at this but my budget is less than a tenth of what many MOOC providers have quoted us for production costs. That in itself may be misleading and counter to my point here (yes, I do have one…)… I have reviewed and developed courses that were superb, and superbly appreciated by demanding students who called them life altering. The reactions or conversely complaints rarely if ever center on the content. At one of my former institutions where we tracked student grievances we registered 4% of student complaints that were content-related; 96% focused on non-content concerns (read on…)

At a recent Bill and Melinda-Gates Institute hosted event I heard students respond to the ever-present question “Does online work?” or “Did it work for you?” Some replied enthusiastically and positively, others with quite definitive “No”s / “It was terrible”-type comments. This begs the key question: “What exactly did they hate?” Can you guess???? – the materials? the fonts? the quality of the videos? No, no and no. The comments fell into three consistent buckets:

“The online class was terrible because I got no feedback on my work”
“I didn’t ever really know how I was doing”
“The instructor was M.I.A.”

In other words, and either depressingly or reassuringly, depending on your perspective the juxtaposition of images and text as advocated by the learning science was not pivotal??? Hmm, OK – so here’s a question that I would pose to would-be online students: You have the choice of a great teacher with crappy materials or an absent / crappy teacher with great materials – which would you choose?

Students who self-select for an online class are in my experience, tolerant of technical glitches and they don’t really care if a video has the instructor in pajamas in front of family pictures in a poorly lit room. A responsive, attentive, responsive, empathetic, responsive, caring but challenging, responsive instructor more than offsets the fact that the video is not green screened so that (s)he appears to be in front of the Hanging Gardens of Babylon, nor because Krakatoa’s not erupting right now* Their attention has gone after 6 minutes anyway, they don’t care whether “herds of Wildebeest are sweeping majestically across the plain”….

So, to summarize, we assess our collective sweat and tears in the realms of Instructional Design, Learning Science, Materials development, web usability, information architecture, Cognitive Science, Assessment specialists and Graphic art and are then judged by whether the instructor shows up for class or not? – Really?!

Imagine the same criteria for traditional classes: –

Q – “How was your face to face class today?”
A – “Well the instructor didn’t show so we all kind of read the books without direction and chatted amongst ourselves, but that’s fine, it was a great class. I love face-to-face classes.

Here’s my money-shot statement:

A class where the instructor does not devote energy and attention providing presence and guidance is not a failed class, it is a failed instructor.
It is not a rationale for concluding that a means of reaching hundreds of thousands of learners for whom face to face is not an option just doesn’t work.

To be clear, my intention in penning this is not at all to diss overstretched instructors who have themselves not self-selected to teach (see earlier comment on self-selected students). Teaching online is extremely different, not suited to all, can be learned but needs to be embraced (or at least a little bit hugged).

Who make the best online instructors?
Here’s a low-tech answer first:
• Jigsaw puzzlers who want to do 5-6 pieces at breakfast, coffee break, lunch, tea break and an hour or so before bed.
• Slightly obsessive gardeners who feel the need to check in on their tomato plants 4 times a day sometimes just to say “hi!”
A higher tech answer for 2014
• Committed e-bay-ers,
• Social media users (even moderate – parents / grandparents accessing Facebook to see progeny pics),
• Anyone who has ever felt the need to Advise TripAdvisor, then gone back to see if others rate their comments.
• Someone incentivized, by some inner passion, who gets a kick out of nudging things along incrementally. Someone who is a little compulsive and doesn’t like to think of a book misaligned on a shelf. Someone who has bought into the idea that they can have influence (on tomatoes or on travelers). Someone who gets a teeny bit jazzed at the thought that they could be just helping make the world a slightly more informed place, affecting or maybe even changing lives – sounds like a heck of a lot of the teachers I know.

It IS a transition though; teaching and changing lives in fifteen-minute increments, rather than through three-hour classroom performances between grading marathons.

Extending my not-great metaphors even further; does anyone garden in a half-assed manner so that they can prove that gardening doesn’t work? Does anyone eBay because they hate the whole system, which is an online manifestation of the capitalist marketplace, and vent when they sell (or buy) things?

I quite miss writing postcards when I travel, I was quite known for them back in the day (OK, I am old), but it is kind of cool now that I can let 20 times as many people know that I’m in a very cool (or hot) spot and also that they know before the week’s out. Change makes things different. If anyone is so wedded to the traditional that they can’t move – that’s fine. I remember hearing of an instructor at another of my former colleges who eschewed the phone as it was too new-fangled and he needed to see whites of eyes.
I get that. I miss things too. Instructors who dislike or distrust (whatever the rationale) “class” too much to show up should not be given online teaching assignments (surely). Those who are a fit and who get it in its slightly compulsive glory (eBayers, Facebookers, TripAdvisors, / Jigsawers, Tomato-growers or book-aligners) should be supported and cherished. Not every personality makes a good face-to-face instructor, not every personality makes a good online instructor. There is the choice; change, adapt, give it a genuine try, or (simply) don’t take the assignment

My job and the job of the Instructional Designers I work with should be supporting great instruction and genuine effort with appropriate spaces and backdrops for learning to happen. My job can’t be developing materials that substitute for instructors who don’t want to be there and don’t show up for class.

I know that academia is a big ship to turnaround but I wish there was a way to convince instructors that the most important thing in an online class IS STILL YOU. If people like me do our jobs well, we can automate some parts – but PLEASE work with us – we might even be able to take away the boring, dull parts that you don’t like doing. For example:
• Answering questions that you have answered 5000 times before (boom! – a FAQ),
• Reminding people that assignments are due (boom ! (again) – Calendar),
• Developing a working understanding of basic, underpinning knowledge (chunked content and Check Your Knowledge self-checks)
• Being there 24/7, answering every question (let us show you scaffolded, supported peer-to-peer interaction).
I now pronounce you FREE to only have to focus on questions that are stimulating, that allow you to demonstrate and indulge your passion for your subject and engage (disproportionately) through interactions that are significant and (could) change lives. I wrote an earlier blog post on this two years ago titled Disrupted Faculty Roles

In summary:
An online class that is poor because the instructor didn’t show up is

    a poor class led by a delinquent instructor.

If an institution does nothing about it, turns away from the data that demonstrates it or deflects blame towards materials that aren’t as cool as the latest MOOC, shame on it/them/us.

Online works for a lot of people when planets align and people work together. The instructor’s responsiveness should be close to the top, rather than near the bottom of the list of requirements.

As I said, work with us, then show up for class… in pajamas, eating tomatoes.

  • Note    * obscure scenic references courtesy of classic John Cleese – Fawlty Towers BBC


A colleague of mine sent the following Inside Higher Ed article this morning,entitled Without Credit it speaks to the search for a viable model to generate revenue out of MOOCs

In response I mentioned that I’m claim the phrase “MOOC-washing” (for disruptive Education wannabes) which so reminds me of Greenwashing in the sustainability movement 10-15 years ago..

– The article merely demonstrates that there IS no real game changer until someone works out a revenue model that is neither (100,0000 enrollments  x free) nor this model of  an “Enhanced MOOC” – Emperor’s new clothes anyone – guess what ?- It’s an online class  for credit costing $300-$400-$500 per credit. Known in some circles as traditional online.

The entrenched, perceived value of the Credit as THE proxy for learning is the real brake on much of this innovation.

There are two key possibilities regarding the CREDIT and it’s centrality to all things.

  • Option 1 – Alternative to credits (Certificates / Badges etc) – these will only succeed if there is some recognized norming or development of an Industry Standard. Something tangible that employers will recognize as currency – this is a long way off in my opinion
  • Option 2 – completely decouple competencies from credits and have all students forced to “show and tell” competencies in a very Open format that proves to employers (undeniably) their employ-ability – this would likely be portfolio or third party standard testing (or a blend)

Given that Option 1 is glacial and outside of anyone’s clear control, I vote Option 2 as the viable game-changer within the next 18 months or so. There will be issues – lingering vestiges of “seat-time” although most people seem to be beyond that now, and the need for collaboration between faculty / departments to get them to agree on what ARE core competencies and how they can be demonstrated.

This model may work better at lower levels (associates rather than Graduate), but I believe that with fresh set of eyes and open rather than turf-war mindsets, we could really produce something innovative and truly disruptive. I LOVE MOOCs but they will not transform Higher Ed. “Enhanced MOOCs” sounds like an attempt to be “down with the kids” without actually doing much of anything innovative at all.

Let’s think outside the box, blow it all up and start again – just pretend you’ve never heard of CREDITS…

Who’s MOOC-ing?

One of my new colleagues here at Northeastern (1 month in – time to blog!), sent me the following article on MOOC participation in Inside Higher Ed – Who Takes Moocs?

Here’s my interpretation (for what it’s worth):

I think (personally) that these MOOCs etc are hitting a really different demographic: the intellectually curious rather than the under served / locked-out-of-higher-ed that they aspire to

My synopsis would be

  • 10-15% complete the course or are engaged enough to complete surveys – that seems to be a developing standard
    (let’s call it 15% to be generous, of that 15%:)
  • (20%)     2-3% of the total are Grads,
  • (10%)      1-2% are Undergrads
  • (not quite) 2-3% are taking it for concrete career reasons (to get a better job) – the “keep skills sharp group I feel may just be taking a long lunch!)

In over-stressing the “employment skills / career development” aspect to 75% of participants (Stanford) – the self-justification seems palpable
This field (MOOCs) is extremely instructive, and the analytics it generates could be a huge boon to Academia, but the bringing education to the masses / opening up the marbled halls aspect seems over-stated.

The Golden bullet still looks to me to be: open course-ware, open platform, faculty (strategically) answering very high level expert questions  only, comprehensive student support, very LOW price point with TONS of intrinsic motivation – (game-theory-based?)

MOOCS – get 3 of the six 6 but illustrate very clearly to me that without at least a couple more of these important elements – support, strategic faculty and motivation, they will remain an interesting, academic, exercise.

Angry Birds – again

Clearly I am connecting with Mindflash today on many levels. David Kelly has a great post on What Angry Birds can tell us about Instructional Design. If you only have 1 minute to skim his paragraph headings do so – I agree 100% – as evidence see my many posts and the fact that many colleagues roll their eyes as I have discussed gamification (game principles rather than simulations) one too many times over the last couple of years…

My earlier posts on this subject:
January 2011 –  Game theory applied to online
Gamification January 2012


BTW – assuming he’s not THIS David Kelly – although that would be awesome !

The instructor role in a FLIPPED class

I blogged about this a while back – how the instructor role can be reinvented, rather than threatened by effective application of technology and process.
I was, obviously, more focused on an online or hybrid class environment, but I believe the same applies to instructors who embrace, rather than run from, elements that allow them to focus on the “what’s really cool about teaching (and learning)…. insert discipline here
Ian Stewart’s post today references instructors who value class time so highly that they would not dream of doing drill and kill or lecturing to a passive audience.
He writes: “By focusing that valuable face-to-face classroom time on exercises that put the lessons learned during lectures into actual practice (doing the homework at school), instructors are supporting the part of the learning process (the “doing”) that students really retain. That is, since students learn the most by implementing theories they’ve learned into real-life work, it makes sense to use as much of your 50-minute in-person session on that as possible.”

My recent work at SNHU has been focused on what we’ve called transitional text that is going to help provide students with the guidance and hopefully some of the impetus to get them from OER resource #1 to #2 and #27, then back to take a self-check quiz or post an assignment. Interesting (flow-inducing) resources and the students’ own intrinsic motivation may get some of them down that path to success, but the main? challenge for online can, and indeed should, be how to convey that passion that a true scholar in the field has developed – how to ignite the fire, illustrate the end point, and viscerally engage the drowsing? student…

Stewart, in what is looking like a pact among Scottish-sounding ed-theorists, references Andrew Miller, who writes that a “flipped” classroom still requires instructors to demonstrate the value of their content, whether online or offline. “Just because I record something, or use a recorded material, does not mean that my students will want to watch, nor see the relevance in watching it,” he writes. ” … If the flipped classroom is truly to become innovative, then it must be paired with transparent and/or embedded reason[s] to know the content.”

Both these posts confirm for me that there is a great opportunity in “unbundling” the instructor role – allowing technology and online (peer-to-peer) communities to deal with some of the things that they can take off an instructor’s plate, allowing her/him the chance to get back to inspiring, motivating and Captain-my-captain-ing.

Stewart’s post here: In ‘Flipped’ Classroom, the Emphasis must be on support, not video”

The Primary Challenge for the OER Movement

David Wiley recently posted an article on the challenge of assessment in the OER world. http://opencontent.org/blog/archives/2042
It certainly does seem to be a challenge – we (SNHU Innovation team) spent time with ETS at their Higher Ed Advisory Council last week in San Diego where we had some great break-out discussions around standardized testing. It was a great session; they have some VERY smart employees in Princeton (special mentions for Ross, Patrick, Kate and David) and they convened a very interesting group of academics.

The current assessment choice for those of us working in the OER space seems to be:

  • on one hand, multiple choice / self-checks – with no concrete feedback from humans (many OER courses have these included)
  • on the other –  blog / journal reviews which are time-consuming (hence questionable given scaling aspirations),  subjective, organization-specific, open to inflation, bias and inconsistent leveling.

I appreciated the Vision Project’s* working group March 2011 report on Student Learning Outcomes and Assessment which I think frames the issue very clearly (the bold is my highlight)

If colleges and universities … are to devise a system for assessing the learning outcomes of their undergraduates in a way that allows for comparability, transparency, and accountability, we must agree on some of the qualities of an undergraduate education that we all expect our students to possess. At the same time, those qualities we agree on must allow us to preserve the unique missions of individual colleges, appropriate measures of privacy around assessment work, and an ability to actually improve teaching and learning with the results that we find.
Research and literature on sound assessment practice is clear that no single instrument or approach to assessing learning can meet all of the challenges and notes that the most effective systemic models of assessment offer multiple measures of student learning in a triangulation approach. The most effective systemic models of assessment offer multiple measures of student learning in a “triangulation” approach that includes indirect assessments such as surveys, direct assessments like tests, and embedded assessments such as classroom assignments.

This notion of triangulation seems viable – mixing in institutional (mission-related) emphases, with quick turnaround self-checks. The missing (third) element is the industry-standard independent test. In some disciplines – Project Management (PMI), IT (Microsoft), HR (PHR, SPHR) there are clear standards that can be applied. There is certainly a window of opportunity for someone like ETS to take a lead on this, if they can develop the adaptability of development and pricing that we, and other college partners, would likely need. I hope that as colleges free up Instructional Design time that they would typically have been spending making *another* version of PSYCH101 content (which is freely available and wonderful at Saylor.org), they spend more time on developing key benchmarks for assessment that can become more widely disseminated.
Assessment is indeed the golden bullet for this work. Ideally it’s fun too.

* The Working Group on Student Learning Outcomes and Assessment (WGSLOA) was established by Richard Freeland, Commissioner of Higher Education, in late fall 2009 in anticipation of the Vision Project, a bold effort to set a public agenda for higher education and commit public campuses and the Department/Board of Higher Education to producing nationally leading educational results at a time when the need for well-educated citizens and a well- prepared workforce is critical for the future of the Commonwealth.

Psychometrics and Crowd Wisdom

We hosted Preetha Ram, co-Founder of Open Study for sessions that we split into two audiences – a core academic group on our main campus and an instructional design / enrolment / student services group at our millyard COCE campus.

One good indicator of a solid product is when distinct audiences and non-believers get enthralled. The sign of a spectacular product is when someone who has seen this show before, (me) and already was a fan, got to see the continued/continuing evolution of both the product and its potential.

Open Study people are working on the back end to review what might be gleaned from a working group of 100,000+ crowd wisdom generators. This is taking them beyond what they’ve had for a while: 24/7 student support, community and game intrinsic motivation (“stickiness”), to learning analytics and demonstration of competencies among their user group.

All competency-based education systems (WGU, P2PU, Excelsior, MITx) need to continually focus on the importance of the “how do we know they have learned?” question. Open Study participants who are answering hundreds (or thousands) of math questions in supportive and constructive ways are not just displaying math ability. They are demonstrating effective non-cognitive skills in tandem with cognitive; domain expertise in conjunction with tangible skills. Analysis and demonstration of specific user correlations as Preetha described might just add up to successful psychometric testing.

With the data they have at their fingertips, Open Study has the potential to track not only teamwork, helpfulness and engagement, as they do now, but also to extend to LEAP / Institute for the Future key skills like problem solving, critical thinking and creativity. I was delighted to witness the immediate engagement of many in the academic session (esp. Kim Bogle – chair of the assessment committee at SNHU, and Mark McQuillan –wonderful – new Dean of School of Ed) discussing how data and metrics might be mined to demonstrate competencies.

It was exciting to sample this “esprit de corps” among true educational entrepreneurs, eager to respond to the genuine needs of students.
Thanks to all who participated.