LMS Evaluation: Which Tools do Faculty Really Use? (Updated)

During the process of evaluating Blackboard as our Learning Management System (LMS), we came to a basic and fundamental question:

“Which tools do faculty really use?”

Although most of the core functions of an online course can be managed with a blog, Learning Management systems come stuffed with loads of bells and whistles that promise to engage, inform, and assess students like nothing else can. That’s why I was so surprised to find that Blackboard provides no way of knowing which tools faculty are actually using in the LMS. Could it be that they don’t want us to know how few of these expensive bells and whistles are actually being used?

After seeing an excellent talk by Jon Mott on the future of the LMS, I was amazed that he had the kind of data we were looking for. He had surveyed his faculty to find out (AKA low-tech analytics), and graciously agreed to share his survey instrument with us.

With that great input, I created this simple survey to understand faculty LMS usage:

(Please feel free to view our survey instrument and make your own copy to support your own LMS self-study).

Update- Data Collected

As part of our effort to understand which LMS features our faculty depend upon from Blackboard 8, we polled them (84 responded) and we created this breakdown of how they are using our current LMS:

• Posting Documents, Powerpoints, and PDFs: 90.3%
• Posting Course Announcements: 84.7%
• Posting and Calculating Grades: 77.8 %
• Emailing Students and Colleagues: 76.4%
• Sharing links to outside websites: 59.7%
• Online Discussion Board: 50%
• Delivering Online Exams, Quizzes, and Surveys: 40.3%
• The “Assignment” tool for assigning, collecting, and grading student work: 38.9%
• Blackboard Groups (group collaboration spaces): 27.8%
• Web 2.0 Tools (Blogs, Wikis, Podcasts, Private Journals): 25%
• Blackboard Calendar: 20.8%
• Delivering SECURE online exams (with Respondus Lockdown Browser): 13.9%
• Blackboard Chat: 12.5%
• Creating TurnItIn Assignments: 11.1%
• Blackboard Virtual Classroom 1.4%

This data served as a baseline comparison for all of the LMS candidates we looked at. We assumed that any “future LMS” we looked at needed to contain all of these functions that faculty are already depending on to run their courses. Once we had this, we evaluated the candidates against this baseline standard.
We also polled our faculty to ask which features, NOT currently in our Blackboard instance, would they like to see implemented as part of our “future LMS”. We wanted to understand what kinds of features they were excited about integrating in their teaching practice. This is the data generated:

• The ability to submit grades to the Registrar from inside your gradebook: 81.9%
• The ability to upload video & multimedia content for my students to watch within the course space: 75%
• Tools to easily add web content (such as YouTube videos, Flickr photos, news articles, etc.) into your online course spaces without leaving the course page: 73.6%
• The ability to upload video & multimedia content (including lectures) for students to download to their personal media players for later playback: 66.7%
• A course calendar that can appear alongside your personal calendar and/or university-wide calendar: 65.3%
• The ability for students OR faculty to set up their own ad-hoc study & collaboration groups: 52.8%
• Private Reflection Journals for students to maintain confidential reflection notes on course material: 52.8%
• A course calendar which syncs to your smartphone and desktop computer: 51.4%
• Wiki-style documents spaces so students and faculty can collaboratively edit the same online document: 48.6%
• A tool to build, share, and assess ePortfolios of students’ best cumulative work: 45.8%
• Outcomes-based grading tools so you can grade using SMU’s Core Learning Competency Rubrics in addition to/instead of the standard ABCDF scale: 44.4%
• The ability to video chat with students or faculty in real time from within your course: 38.9%
• The ability to maintain faculty and/or student blogs for timely reflections and community discussion: 38.9%
• To access the Learning Management System via a dedicated smartphone “app” or mobile web interface: 33.3%
• The ability to selectively receive notifications about course events via email, text messages (SMS), Facebook, Twitter, or other social media: 31.9%
• The ability to read course content in an RSS reader like Google Reader: 23.6%
• The ability to create a personal home page with contact information, recent activity, and other customizable content (like a Facebook or LinkedIn profile): 20.8%
• To have the option to post coursework or announcements to “the outside world” (e.g. give it a public URL that is visible to non-SMU users): 16.7%

We used this data to evaluate our LMS candidates (Blackboard 9, Moodlerooms Joule, and Instructure Canvas) and having this data was critical in determining which system would meet our needs. Ultimately, we adopted Canvas because it provided all of the top 5 most requested features and all of the baseline features for about 1/3 of the cost of Bb 9’s most limited package. There was considerable resistance to moving off Blackboard among the university community, and having this data was extremely useful in clarifying why Canvas was the right choice.

I hope this article is helpful in your process of evaluating your university’s technology choices.

• NetDimensions’ ‘Is the LMS Dead?’ Webinar Answers Why Learning Management Systems Need to Evolve (prnewswire.com)
• Strategies for renewing eLearning environments – Melbourne University (slideshare.net)
• If We Ditch The LMS, What Else Could We Re-think? (downes.ca)
• There are alternatives to Blackboard and Moodle: Instructure Canvas goes open source (zdnet.com)
• A Fiery New Competitor (fool.com)

Related articles

• Florida State College at Jacksonville Selects Blackboard Following Faculty-Wide Vote
• CMS/LMS Review -Blackboard
• Lynn University to drop Blackboard Learn in favor of iTunes U | Inside Higher Ed
• Migration Debrief
• What Do You Look for in an LMS?: A Tale of Unexpected Findings

Liked this post? Follow this blog to get more.