by Ted Curran

@tedcurran having trouble commenting on great post; what is your take on mastery v competency per https://t.co/jCgIbZSoVj ? #blogrequest

— Phil Hill (@PhilOnEdTech) May 10, 2015

Hi Phil, Thanks for your kind invitation to think this out.

I think we all need to be very explicit about what we mean when we use terms like “competency”, “mastery”, “outcomes”, and “learning objectives”. Either I’m showing my ignorance of the “true” definition of these terms, or they are used interchangeably and erratically by different speakers to mean different things entirely. I wish Ebersole‘s call for clarity on these terms came with an explicit definition section!

He appears to be saying that some programs sold as “Competency-Based” are actually measuring “mastery of content” — that is, students’ ability to read, memorize, and reproduce factual information, similar to the “tell and test” modality found in traditional face-to-face lecture courses.

He uses the example of “instructors who had complete command of their subjects, but who could not effectively present to their students. The mastery of content did not extend to their being competent as teachers.” (2014) The distinction here is between knowing a lot and being able to apply that knowledge to solve real-world problems in novel contexts.

We Don’t Know That We Don’t Know We’re Not Assessing Critical Thinking

Flores (2010) is a blistering indictment of a higher ed system that fails to accurately measure (or teach) critical thinking, but rather focuses almost entirely on low-level recall and understanding of factual information. The article is full of quotable disasters, but one stands out:

“The general consensus is that the educational system has not performed well in consistently producing critical thinkers… Paul (2005) offers three reasons for this discrepancy. He believes the majority of teachers do not understand the concept of critical thinking. One cannot effectively teach what one does not understand. He further posits that most teachers do not realize they operate from this deficit position. Lastly, he blames the continued use of traditional teaching techniques such as memorization and lectures.” (Flores, 2010)

My fear (and apparently Ebersole’s) is that many of the same people tasked with developing and implementing CBE programs don’t have a clear idea of how they should differ from traditional content delivery models. I agree that we must be careful not to reproduce the failings of the current system as we move to the next chapter of online learning, and I believe that verifying mastery through performance assessment is the right way to get there.

True Competency is About Doing, Not Just Knowing

I’d be interested to know which specific CBE programs he’s referring to, though, because while most still use more of this didactic learning than I’d like to see, most also make use of true performance assessment for key course grades. Maybe he’s cautioning us that, in the breathless gold-rush to develop CBE programs, we can’t just blindly replicate the MOOC Mania of 2013, but rather to be thinking about how to improve quality.

I agree with Ebersole that the best way to verify that students will be able to perform high-quality thinking and work tasks is to have them perform those tasks and assess them on explicit criteria.

The best definition of competency I’ve ever seen is (Knowledge + Skills + Attitudes) + Performance = Competency

A good definition of a Competency = (Knowledge + Skills + Attitudes) + Performance #CompetencyEd #WCET14

— Ted Curran, M. Ed (e, em) (@tedcurran) November 21, 2014

The essential difference (whatever you call it) is that students need to be able to perform real-world work tasks to demonstrate competency, not just to show that they can recall and regurgitate knowledge.

Think of the “star worker” on your team at your job. The person that, if you had ten of them, it could take your department to the next level. What is it that that person does that adds so much value to your team? What are the tasks, skills, and habits that make this person different from other workers on your team? The projects students create should look very much like the unique work products created by your star employee. Students who can perform at that level pass the competency, and students who don’t get it on the first try get more opportunities to revise, practice, and work with instructors until they demonstrate mastery.

Unlike the helpless academic in Ebersole’s example, what we end up with at the end of this process is more copies of the star worker. We trade a little of the arcane book knowledge our professor has memorized for a bag of functional tricks that workers can use to be successful in real-world situations.

I’m working on a new high-touch model for CBE that emphasizes performance-based assessment and mastery learning. It’s a model that relies on human faculty — not replacing them. It uses their humanity as a key asset for reaching under-prepared, non-traditional students and mentoring them up to mastery through a rigorous revision process. Though mastering content knowledge is a focus, it is just the first step towards performing higher order thinking tasks.

This process is largely based on my experience teaching in the Envision Schools model, now in book form. That model is the best I’ve seen in actually helping under-prepared students rise to grade level and beyond through high standards, strong relationships, and supported mentoring to mastery.

Putting it All Together

Of course, Ebersole raises many questions about how different institutions will implement CBE programs (excerpted below, please read the original article for context). These are all questions that different institutions answer in their own ways, and which will ultimately determine the quality of resulting programs. While many different types of CBE solutions will technically fit into Malan (2000)’s definition, the ones that succeed will be the ones that offer a genuine improvement over traditional education, especially for under-served populations.

Who specifies the elements to be addressed in a competency determination?
Who does the assessing, and what criteria must be met to be seen as a qualified assessor of someone’s competency?
How will competence be assessed, and is the process scalable?
Who is to be served by the growing number of CBE programs?

Do prospective students want this type of learning/validation?
Will employers and graduate schools embrace those with credentials earned in a CBE program?