To Use Technology Well, Measure the Results

E-textbooks. Hybrid and online courses. Using learner analytics to assess experiments that involve academic technology. The future of SmartSite.

These are a few of the areas that David Levin intends to discuss, and address, as director of Academic Technology Services for Information and Educational Technology. Below, you’ll find an excerpt from an interview with Levin. You can also read the full text.

 

How do you make sure that technology supports good education, as opposed to intruding on education?

Great question. We want to understand how we can we assess the effect of any change.

As we build our tools to help faculty, we also want to build up what I call ‘learner analytics.’ As we develop our online courses, introduce a new technology, or make changes to SmartSite, what data can we collect to see the effect in teaching and learning?

I’ll give an example. We’re beginning to test a set of lecture-capture technologies. We have long offered services to record faculty when they present their lectures. This is not new. When I was a student, we individually brought devices to record lectures. But now faculty members are more likely to record their content, and then put it online.

What’s the effect? There are many, but many of us hope that if we introduce various lecture-capture techniques, we might see that students will spend less time in class trying to write down what the instructor is saying, and spend more time engaged in real listening and participation in the class.

The data you collect would tell you if the technology is helping or not.

We want to examine at least two or three things. One, are students more engaged—and then we have to figure out how to measure engagement. Second, faculty members are often worried that if we introduce lecture-capture, then students won’t show up. Why show up if I can get to it all later? Can we measure those things?

Third, what’s the effect on learning outcomes?

Do you have data sets that would tell you?

We don’t now, because we haven’t been doing this enough, but we want to. The data would be student performance. In an ideal situation, we might look at students who:

• Come to class, but do not have access to, or use, the recorded lecture.
• Don’t come to class, but listen to the recorded lecture.
• Come to class, and listen to the recorded lecture.

We’ve got three different sets of students. Which perform better? Is there any difference?

The same kind of measures ought to go on with online learning. In our engagements with faculty who are developing online courses, we are building in measures of student success in different modes of learning.

Ideally, one would want to look at a fully on-campus student experience, a hybrid student experience, and a fully online student experience. What can we do to collect data on student learning outcomes in these different modes of learning?

I imagine you welcome faculty who want to talk with you about all this?

Absolutely.

What’s the best way for them to contact you?

Send me an email (dslevin@ucdavis.edu) or call me (530-752-2133). Even better, as much as I live in the virtual world, I want to get together and talk with you. So, set up an appointment. Drop by.