EdTech for higher ed

A few colleagues from our faculty have put together a workshop this week on using technology in your classroom. They want info and demonstrations of things that people are doing right now. I volunteered to talk a bit about the cloud services that I use, Dropbox and Evernote, and how I use the iPad app Explain Everything to make little voice-over-PowerPoint video's for the "flipped classroom". Here is my video of the story:

Some education thoughts of the week

It was a busy week and today I was in three different meetings about teaching-learning. The first was about electronic study guides and we discussed a number of issues:

• pdf-paper-behind-glass type electronic guides vs. full-scale interactive electronic guides, 
• what do students need, 
• what can lecturers provide, 
• it may help to go green and save paper, but will it save money, and
• do we have the IT infrastructure? 

All of these are relevant issues, but for me the key is to get more lecturers to buy into the blended-learning model. If you are not flipping your classroom, an electronic guide will just be something that you are forced to developed (hopefully using some template!) and it won't be used properly for learning.

In the School we had a nice session on assessment, talking specifically about essay questions and papers and the memo's and marking rubrics for those. I learned that I have to go and have a look at Bloom's taxonomy and the action verbs again. A big word of thanks to Dr Inge Venter of Academic Support Services for her time.

Finally I also started playing with Quizlet. I hope that the first years will start using it and create a resource for studying the basic concepts of introductory Economics. Have a look at these (in Afrikaans):

Flipping ECON, chapter 1

This afternoon was the first of my flipped ECON111 lectures and I though it went quite well. The video was uploaded unto eFundi two days before and viewed 139 times by lunch time. The English version on YouTube had 2 views.



To start off the class I showed the video and then we worked through a few multiple choice questions. Hopefully I will have those on the Quiz system next week and then students can answer them before class on their mobile phones. Today we used the old-school show of hands. In cases where every option had some votes I first gave them the opportunity to explain their answer to a neighbour and then vote again. This was followed with some explanation from my side.

I still want us to spend some time making outlines or mind maps of a chapter's content in class, but I first have to book a document camera for our room. Depending on the topic, I would like to add some group work and a debate to the MCQ-Q&A part.

It is way to early to judge success, but I enjoyed it.

I would love to hear your thoughts and comments.

Advice for post-grads

This week we are receiving the Master's degree students of 2013 in the School and I am busy updating a few presentations. Along with the usual advice on how to approach the research proposal and tips and trick for academic writing, I also want to share my thoughts on how they can add value to the post-grad experience. This goes along with a post from last year about thinking like a researcher.

So I make a few PowerPoint slides and would love to get more input:

A few first thoughts on games and badges

At the start of the new academic year I am reading and thinking about the so-called gamification of education. I have not read enough for a proper overview, but want to get some thoughts out there. Put very basically, the idea is that learning can occur through play. That does not mean that there has to be some kind of simulation, on-line or in-class game, rather it is about a way of structuring the learning experience. The ideas behind game-based learning are:

• learning-by-doing,
• focus on problem solving, and
• engagement of students.

A nice post on the Inside HigherEd blog explains how something like the Fitbit brings gaming elements to fitness and what that may mean for learning. (Fitbit is an activity and sleep tracker, similar to the Nike Fuel Band). You can set your objectives, in the fitness case, in terms of calories or steps. Measuring steps taken or floors climbed adds motivation. The data can be tracked and analysed, but there is also immediate feedback and recognition of milestones reached. There is a social element and you can share the process and the results. It is of course all very much mobile.

For teaching an Introductory Economics course this could mean:

• finding a way for students to set their objectives, 
• regular measurement of knowledge, skills and competencies gained, with quick feedback,
• adding elements of sharing, collaboration and competition through badges and leaderboards, and
• ideally it should all be easy to access - electronic and mobile.

I don't know how to tackle all this yet, but it is being done. HBR blog network reports how Deloitte made learning in their leadership programme a game with great success.

I'll keep everyone posted on the progress.

More on lecturer evaluation

In August I wrote a post on lecturer evaluation, having used factor analysis to narrow down the characteristics that our students think make for a good lecturer. The questionnaire has 25 items that students score on a 4-point scale from strongly disagree to strongly agree. The items include things like: The lecturer plans thoroughly and prepares thoroughly for contact sessions. Or, the lecturer makes use of multimedia in support of learning. The lecturer may also explain the relevance of concepts and theories, or explain the relationship between theory and practice. The result was the following:

Component 1 seemed like the baseline characteristics of a good lecturer – being friendly, accessible, fair, offering support, an all-round professional that can explain the work. It accounted for 19% of the variance in the data. Component 2 looked like the things that you have to learn to become a good lecturer. It is not only about knowing your field, you also need to know a bit about formulating outcomes, action verbs, putting it into a study guide and using it. This explained 15% of the variance. Component 3 looked like it was about the enthusiasm and adding value – engaging with the students. And it explained another 15% of the variance. The other 50% of the variance was not explained by these three constructs.

Now, students don't really like doing these lecturer evaluations for all their courses. After the second or third one they realise that it takes up valuable beer-drinking time. And as with many of these "customer satisfaction" type surveys, they don't really see their inputs making a difference. Unless you are repeating the course next year, you would not know if the lecturer took the low score for providing quick feedback to heart.

On the lecturers' side, the process does not seem to provide that much useful information or action. If it is going badly in a course, the School director knows about it long before the lecturer evaluation is done. Low scores on a few items might get a mention at the meeting where next year's teaching load is assigned. If you are really keen, scoring low on "basing assesment on learning outcomes" might get you to enroll in the next assessment training course on campus, but then you also do have some articles to write. At best (and worst) the aggregate lecturer evaluation score matters for the top-20% bonus at the end of the year.

Which brings me to my point, if we are capturing half of what makes for a good lecturer and the result is not particularly useful to students or lecturers, shouldn't we replace the whole system with a simple vote by text message: rate your lecturer with score out of 100.

This has some support in what has been written about so-called "thin slicing": In his book Blink, Malcolm Gladwell writes about people’s ability to 'thin slice' – to judge what is good or important from a narrow period of experience. He goes on to argue that having too much information can interfere with the accuracy of a judgement.

To get an idea of the difference between our questionnaire score and a 'thin-sliced' score we asked our students for both this semester and the results are as follows. For the analysis I have 869 usable responses from undergraduates in 5 modules for 9 lecturers.

The average score from the questionnaire was 85.7% and the standard deviation 13.3. The figure shows the distribution between the courses.

The average of the 'thin slice' scores out of 100 was 82% with a standard deviation of 17. The correlation coefficient between the two sets of scores was .741 and it was significant at the 5% level. Controlling for the different modules the partial correlation is slightly lower at .731. A paired samplest-test showed that the differences between the means is statistically significant.

It seems that the questionnaire is useful to the extent that it ammeliorates a general dislike of the lecturer. For 'thin slice' scores between 0 and 50%, the average questionnaire score was 63%. Having considered all 25 elements of the evaluation it turns out the that lecturer is not so bad as the initial 'thin slice' score out of 100 indicated. This effect gets smaller at higher scores: For 'thin slice' scores between 71 and 80%, the average questionnaire score was 83%.

Thus, for all its shortcomings, it does seem that the questionnaire serves a purpose. Next we'll have to try a five-point scale and see if we can measure more accurately!