On lecturer evaluation

Moving offices and attending meetings means that I am writing fewer posts than I would like to. I really wanted to do one about the stuff I discovered when pack up the old office - notes from my PhD, the handbook for new lecturers 1999 etc. But leaving nostalgia aside, once in a while I do management work that also makes for good blog content. Here's a re-post of the one I wrote for the School blog on lecturer evaluation:

In his book Blink, Malcolm Gladwell writes about people’s ability to “thin slice’ – to judge what is good or important from a narrow period of experience. He goes on to argue that having too much information can interfere with the accuracy of a judgement.

Putting all that to one side, the NWU-Puk has for some time been asking students what makes for a good lecturer? Formally these are known as lecturers’ evaluation and normally it takes place towards the end of a semester, ideally in all courses. We don’t ask students for their gut feel on who are the good lecturers, we ask them about the characteristics of good lecturers using a questionnaire. Is there a case for thin slicing, or does our analysis approach yield results?

Now, I have been to a faculty council meeting where the questionnaire has been pulled apart and people congratulated on their results in almost the same breath. Some Schools use the “old” ones, Institutional academic support services have developed one and then there is the “new” version. A number of Faculties use their own questionnaires because they are different.

Being Economists, our School decided to bring some rigour to the debate. In the first semester of 2012 we used the e-learning platform to administer the “new” questionnaire and with that data, we now have some clues as to what students think makes for a good lecturer.The questionnaire has 25 items that students score on a 4-point scale from strongly disagree to strongly agree. The items include things like: The lecturer plans thoroughly and prepares thoroughly for contact sessions. Or, the lecturer makes use of multimedia in support of learning. The lecturer may also explain the relevance of concepts and theories, or explain the relationship between theory and practice. So what is it that the high-scoring lecturers do? I have always argued that being well-prepared and enthusiastic is more than half the battle. To see what different constructs make up that elusive “good/great” lecturer we used factor analysis to analyse our data.

We have responses from seven undergraduate courses in Economics, International Trade and Risk Management, ranging from first years through to third years, and a total of 1962 observations. There are 12 lecturers involved.

The KMO measure of sampling adequacy was .975, which shows that the data are suitable for factor analysis and the principle component analysis extracted three factors, explaining 50% of the variation in the construct “good/great” lecturer. After a Varimax rotation the items are grouped as follows:

Component 1	Component 2	Component 3
The lecturer is on time for classes	The lecturer makes use of the study guide during contact sessions	The lecturer encourages students to attend learning support on campus when their performance is weak
The lecturer makes use of multimedia in support of learning / makes effective use of visual aids	The lecturer bases assessments on learning outcomes as stated in the study guide	The lecturer encourages students to work together during contact sessions
The lecturer plans thoroughly and prepares thoroughly for contact sessions	The lecturer explains how outcomes will be assessed	The lecturer encourages students to participate in the class discussions
The lecturer is friendly towards students	The lecturer presents study material in an organised manner as set out in the study guide	The lecturer encourages students to make a greater effort in their studies
The lecturer offers support and assistance when requested to do so	The lecturer states learning outcomes I have to master for every contact session	The lecturer presents contact sessions that are valuable learning opportunities for me
The lecturer promotes an atmosphere of mutual respect	The lecturer explains the relationship between study units	The lecturer explains the relationship between theory and practice
The lecturer uses a level of language that I can understand		The lecturer refers to relevant and recent developments in the subject
The lecturer assesses assignments and projects fairly and transparently
The lecturer gives feedback on tests and assignments within a reasonable time
The lecturer explains the relevance of concepts and theories

Component 1 seems like the baseline characteristics of a good lecturer – being friendly, accessible, fair, offering support, an all-round professional that can explain the work. It accounts for 19% of the variance in the data. Component 2 looks like the things that you have to learn to become a good lecturer. It is not only about knowing your field, you also need to know a bit about formulating outcomes, action verbs, putting it into a study guide and using it. This explains 15% of the variance. Component 3 looks like it is about the enthusiasm and adding value – engaging with the students. And it explains another 15% of the variance. The other 50% of the variance is not explained by these three constructs. What describes the other half of being a great lecturer? I would love to hear how students/ lecturers think we can measure that magic. Or should we just thin slice and ask the students to score the lecturer out of a 100?

Finally, I am glad to say that my colleagues in the School did really well in these evaluations and should take a bow.

More on growth from ECON621

A post on growth slowdowns and growth plans

I wrote this post for the School's blog but also want to post it here:

With economic growth slowing down, economists everywhere are offering advice on what can and should be done. There is advice on how the ECB can stave off crisis and restore confidence, blogs argue that the Fed should be doing more and there are views on how China should respond to a global slowdown.

In South Africa we are never short of plans and strategies and recent the Democratic Alliance added their Plan for Growth and Jobs to the proposals of government’s New Growth Path and National Development Plan.

And in the academic background the Institutions versus Geography debate rages.

To provide another perspective on this story it may be useful to distinguish between the drivers of growth over the next 6 to 12 months and those of catch-up growth and deep development.

To my mind, there is little sense in talking about drivers of growth over the next few years. RMB economist Ettienne le Roux shows that the real growth rates expected in the EU are -0.5% in 2013, 1% growth in 2014 and 1.5% in 2015. The private sector, governments and banks need to deleverage. The slowdown in Europe will have an important contagion effect through channels like trade, investment and labour mobility. Developed economies look set to “muddle through”.

The prospects for the other 50% of the world are better. They are under-leveraged, have smaller deficits to GDP and more open credit channels, which provides for policy flexibility. Average growth rates of 5-6% per annum are forecast.

The South African economy grew by only 5.8% in total from the third quarter of 2008 up to the first quarter of 2012. The drivers of growth have been households, government consumption spending and investment by State-owned Enterprises. Exports, private fixed investment and government fixed investment have been slow to catch up or contribute to growth. Most of the work created since 2008 has been in the public sector.

Given the global outlook and the limited fiscal and monetary policy room available, one can hardly expect South African policymakers to drive growth over the next few years.

It is when one considers the drivers of catch-up growth and deep development that the story becomes interesting. There are probably as many views on what matters for growth as there are economists. “Everyone knows” that human capital, innovation and infrastructure are important. Some say these drivers are fostered by openness, agglomeration or a developmental state.

Years ago Rudolf Gouws presented two summary slides at a BER policy conference and today we can still plot most of our plans and strategies against these drivers of growth. The DA’s plan, for example, emphasises, education and training, a Youth Wage Subsidy, Job Zones, reducing the cost of doing business and encouraging entrepreneurs, and building infrastructure that crowds-in investment.

These are all sensible ideas, but unlikely to deliver a knock-out blow to a growth ceiling or to structural unemployment – least so in the next electoral cycle. One private sector economist opined: "Eight per cent is achievable, but not with this plan. They are making the right noises and politicking very well, but it’s not offering anything inherently different…”

Which brings me to the point of this post: there is nothing else. No single plan or idea will be able to deliver catch-up growth of 6% or 8% per annum. Growth is a many splendored thing. We have to get education, infrastructure and innovation right. We need institutions that nurture investment and protect the vulnerable. Most importantly, we need to get away from the idea that if we were to do only that one thing: weaken the rand, support SMMEs, invest in railways, improve competition develop tourism, curb corruption, or whatever, then we would be on our way. In fact, we need to do all that and more.

And that makes it so difficult. Politicians, bureaucrats, the public, all love a big shiny project that will change everything. Whereas we need to do many different things right and we need to start now.

CV and job interview advice for students

Toga week 2012 cvs interviews from School of Economics, North-West University.