Jon Mott: Bridging the Gap Between the PLE and the LMS

Back in August 2009, Jon Mott gave this presentation at OpenEd 2009. He reports on their work at BYU to create a new learning environment based on PLE principles.

I strongly encourage anyone interested in the future of the CMS/LMS to watch this presentation. I took away the following points:

1. PLE values (student-centric, openness) more closely match the values of the institution than do the values of the CMS

2. Lack of continuity for the learners from semester to semester exists in the current CMS. We typically turn off access to course sites at the end of the course. It’s the “chief demerit of the course management system” Mott says.

3. Stand-alone gradebook. This is a highly important progression in the future of the course management system. Integration with various systems will be key, but de-coupling this critical tool is the first step.

4. Get rid of the dropbox? He says that instructors can now tell students, “Don’t upload your paper, just give me the URL for where your paper lives on the web”. Furthermore, he says, "Maybe one of the requirements for being digitally literate in 2009 is being able to publish a web viewable document”. Bingo. How many institutions offer easy ways for students to publish on the web? However, I do know that many instructors value the ability to zip and download all the submitted papers for an assignment in order to more efficiently read and grade (likely after printing!) — something the digital dropbox currently offers.

Good stuff! I’ll be following Jon Mott and his work at BYU more closely these days. Some of my colleagues saw him speak at ELI 2010 recently and posted their summary and comments here.

Choose a new LMS using integrated multi-criteria decision making approach

Imagine you're tasked with selecting a new course/learning management system for your institution. Obviously you will want to involve the various stakeholders (a group that is growing in diversity each year, by the way) and identify a laundry list of criteria. But how do you develop an objective method for evaluating various systems?

I just came across the following paper that describes "an integrated multiple criteria decision making approach, which combines the analytic hierarchy process (AHP) and quality function deployment (QFD), to evaluate and select the best system. The evaluating criteria are derived from the requirements of those who use the system."

Measuring performance of virtual learning environment system in higher education
William Ho, Helen E. Higson, Prasanta K. Dey, Xiaowei Xu, Rami Bahsoon
Quality Assurance in Education, Vol. 17 No. 1, 2009 pp. 6-29

Article URL for subscribers:
http://www.emeraldinsight.com/10.1108/09684880910929908

I admit that I had no idea what AHP meant. (According to the QFD Institute: The Analytic Hierarchy Process (AHP) is a mathematically rigorous, proven process for prioritization and decision-making. By reducing complex decisions to a series of pair-wise comparisons, then synthesizing the results, decision-makers arrive at the best decision with a clear rationale for that decision.) OK...that sounds useful and relevant to the process of selecting a C/LMS in my mind.

Oh, "QFD is a comprehensive quality system that systematically links the needs of the customer with various business functions and organizational processes", according to the QFD Institute.

The article elaborates on this analytical process. Skimming to the conclusion, I find that the authors explain the value in this approach for institutions with existing VLEs:

"...the proposed approach can support the decision makers of universities in reviewing their existing VLE systems and determining whether it is necessary to replace the existing systems by a better one...[T]he proposed approach can [also] support the VLE system developers in analyzing their strengths and weaknesses, and also identifying the opportunities and threats against the competing systems."

Unfortunately, I don't have time now to dive deeper into this topic or mentally apply it to my own institution's environment. (It does make me wonder what analytical process was used 6+ years ago when UW switched from WebCT to Desire2Learn.)

I will, however, be saving this article for future reference.

Desire2Learn usage at UW-Madison

Now that the fall semester has ended (yikes, January and the Super Bowl have passed too!), I checked in on the data regarding D2L (Learn@UW) use here at UW. My hunch is that the trends we are seeing here reflect trends in other North American universities.

Active Course Sites
This is the stat most commonly referenced on campus, probably because it is the easiest to understand...at first. This is the number of course sites that were activated by instructors over the course of a semester. This data includes "non-timetable" courses such as used for non-credit outreach courses and internal administration. The trend shows a general increase but not at as dramatic a pace as in previous years. Obviously, there is a natural limit that will be reached for official timetable courses. The last time I compared this data with total number of courses offered, we were at approximately 30%.



Unique Users in Activated Course Sites
How many different people are we serving through our central course management system? Again, this measure is trending upward towards a logical upper limit. I am amazed that we are serving over 30,000 different people each semester. At UW, there are approximately 42,000 students and 2,000 faculty (not including the plethora of TA's and lecturers who are big users of the CMS).



Total Enrollments in Active Course Sites
This is where, I think, things get interesting. This is one stat that seems to still show linear upward growth. "Total Enrollments" is the sum of every course's total number of members, instructors + students. This data shows that the typical user has more courses in Learn@UW now than in the past. I wonder when we will hit the point where courses that do not have an online component are in the minority.



Help Desk Cases
Although not yet updated with Fall 2008 data, this too is quite interesting and satisfying. Although we are seeing a general increase in use of Learn@UW, use of the Help Desk for Learn@UW is not increasing proportionally. In fact, there's evidence that it might be decreasing.

This data does not show the total amount of support provided for Learn@UW across campus, just those official cases coming through the campus Help Desk. Hopefully, this data shows that more people are using Learn@UW without the need for support. Fewer bugs exist in the software than ever before. Performance and up-time have been outstanding for the past 6 months, at least. And indeed, there is something to be said for users' familiarity with a complex system.



Obviously, numbers are only part of the story. The real story lies within each course and in the value derived by the instructors and students. That's why I still love to be involved with faculty at workshops, brown bags, and 1-1 consultations. I enjoy hearing directly from them what is working, what needs improving, and ideas on how we can help.