Tuesday, September 14, 2010

Perverse incentives

I didn't think I'd miss the previous govt. It was well past its use-by date, and for all that National indulges in a lot of nanny state enthusiasms, it has nothing on Labour when it comes to stuff like considering yourself in loco parentis over the grown-ups who elected you.

That said, there was one respect in which the last Labour administration had it all over National - it knew what to do with the tertiary education sector (unsurprising, given the backgrounds of much of its Cabinet). National, on the other hand, takes an approach to tertiary education that involves the application of Perverse Incentives, or as they prefer to call it, Performance Measurement.

It sounds reasonable enough - tertiary ed soaks up a huge wodge of taxpayer cash, and if you don't measure performance, how do you know whether you're getting value for money or not? Well, yes. Except that measuring universities' and polytechs' performance is a bit harder than establishing which salesman shifted the most appliances last year. With higher ed, you really need to have a good think about what behaviour you're going to encourage by rewarding it financially. So, an incoming National govt will naturally get experts in the field to advise them on appropriate performance measurement for it, right? Er, wrong. You see, people who have some knowledge of the sector are what right-wingers call "vested interests." Instead, it's apparently much more sensible to take advice on this from Treasury staff who know little of the sector other than that they went to university once, but who have full confidence that their ideology constitutes a magic toolbox equipped to offer sound advice on any public service.

So the last time National was in power, Treasury had a quick squizz at tertiary ed, decided that universities and polytechs are selling a product (education) to consumers (students), and therefore correct performance measurement = how many student consumers purchase your education product. The govt accordingly based funding on the ability to attract student enrolments.

This Perverse Incentive became known as the "bums on seats" model and had some predictable consequences: it became important that courses were popular and that they were easy enough that students didn't fail them; it was important to encroach on competitors' territories and try to lure their customers away; it was important for universities to diversify into polytechs' more popular fields of study and vice versa; it was important to direct taxpayers' money into TV ad campaigns aimed at bringing in more taxpayers' money; in fact, the only thing seemingly lacking in importance under this performance measurement regime was that students should receive a good education. I began working in the sector at the beginning of 1999 and could barely believe that a govt would pursue such lunacy. It was a positive relief when Labour was elected at the end of the year and promptly put a stop to my employer's next takeover bid.

So much for the unlamented 1990s. Few in National now would want to go back to the idiocy of bums-on-seats performance measurement, they have at least learnt that much. Unfortunately, they don't seem to have learned not to ask Treasury for advice on tertiary ed policy.

The latest performance measures have just been announced by TEC (Tertiary Education Commission). It's announced as though it's TEC's system, but in fact it's National's system, as suggested by the Treasury. TEC, being thoroughly knowledgable about the sector, counts as a "vested interest" and is therefore unwanted in the development of policy for it.

Tertiary Education Minister Stephen Joyce (the govt accepted a while back that the sector wouldn't take Anne Tolley seriously) is now measuring performance on completion rates rather than enrolments. Well, it is a better system - if only in the sense that breaking your leg is "better" than breaking your pelvis.

The idea behind this measurement is that the taxpayer is best served by tertiary ed institutions concentrating on turning school leavers into people with job qualifications. This means that institutions with young, internal, full-time students succeeding in courses that involve training for a particular career will be counted as a success. On the face of it, a laudable and sensible performance measure that few could describe as a Perverse Incentive, surely? Well, no. Because the flip side of it is that those institutions that indulge students in part-time study or extramural study, institutions that welcome mature students, re-trainers, second-chancers, people trying to fit study in around work or child-rearing, beneficiaries or prisoners trying to improve themselves and suchlike, will not only be appropriately financially punished for their contradiction of govt intent, but will be declared poor performers on a publicly available league table.

You can tell from the fact I'm grumpy about it that my institution is not looking any too flash on that league table. We're the national flagship for extramural study, which basically means we're in the shit. The Perverse Incentive is about to go into action - next year, the govt's view that extramural or part-time study is a Bad Thing will take tangible form in a serious cut in funding for my institution, and it will begin trying to divest itself of these Jonahs and replace them with full-time internal students fresh out of school. The president of our Extramural Students Society, Ralph Springett, summed it up thus: "It is ridiculous that students who avoid taking a student loan and work productively are the ones singled out as non-performers." Yes it is ridiculous, but that's Perverse Incentives - sorry, Performance Measurement for you. Welcome to another 9 years of tertiary ed buffoonery.


Anonymous said...

Why don't they just measure the quality of the education on offer? It isn't like we don't know how to come up with ways of measuring quality.

Oh. Thats right. Measuring quality would expose the massive underfunding of teriary education.

Flashman said...

A complete rationalisation of the NZ tertiary sector is long overdue. 8 universities and 20-odd polytechnics for a tiny population of 4 million is an absurd waste of taxpayers' money: duplication of courses, capex and, most important, management and administrative overhead. For example, Christchurch [a small regional city by generous global comparison] has no less than THREE tertiary providers in direct competition with one another. Madness!

The Aussies bit this bullet 15-odd years ago and did away with the 19th Century transportation legacy whereby each isolated town had its own tertiary provider.

It's only vested interests keeping these fixed-cost clunkers on life support.

So come on Treasury! Crunch the numbers, put on the record and let's see who is sitting in who's lap when you stop the music.

Anonymous said...

The latest figures for ranking of universities has NZ falling woefully behind the Aussies. Their universities are on an upswing, and indeed many rank in the top 100, with a few in the top 50. Perhaps its time to look at their model and adjust ours accordingly.

OECD rank 22 kiwi said...

Looks like the Government will have to shut down Massey University.

Wonder where the the Vet school will be moved to? That will affect like hundreds of people.

Psycho Milt said...

8 universities and 20-odd polytechnics for a tiny population of 4 million is an absurd waste of taxpayers' money...

Don't know about polytechs, but our university count is about the same as Aus - approx 2 per million pop. The situation of multiple universities represented in one city (eg, mine has a campus in Wellington competing directly with Victoria) is a direct consequence of the bums-on-seats Perverse Incentive, and yes it probably should be rationalised.

Perhaps its time to look at their model and adjust ours accordingly.

Their model is that the federal and state govts put in shitloads of cash in the form of research grants and so on - it's not really a model we can emulate.

Michael Wynd said...

As a current post-grad student at the same institution, I'm not suprised at the outcomes of the measurement. Any unit or form of measuring that doesn't take into account ages and types of courses is meaningless.

The only thing this shows me is that we need to return to the University of New Zealand model we so foolishly threw away in the 1960s.

OECD rank 22 kiwi said...

it's not really a model we can emulate

It is if we join the Australian Federation.

MacDoctor said...

I don't disagree with your post, PM, but we do need some form of performance measure. You can't keep slinging millions of dollars into educational institutions without knowing that the money is being used well. What sort of measure would you propose?

Flashman said...

Here's an example of how a real-life NZ tertiary education institution operates:

Spreadsheet one - fact based evidence of the resources required to deliver a course.

Spreadsheet two - a fictional statement of the resources required presented to top management in order to make the head of school look good.

Spreadsheet three - a fictional statement etc in order to allocate tutor time.

Spreadsheet four - a fact based statement of the actual resources consumed to deliver a course.

And the taxpayer foots the bill for this incompetence and lunacy!

The Veteran said...

PM ... perhaps you might care to comment on the fact that, in my area at least, the % of extra mural students who front up for exams is around the 60% mark.

And the 40% who don't bother????

You might infer from that there is something not quite kosher with the system.

Psycho Milt said...

Macdoctor: I'm no expert on the tertiary sector myself, but it seems to me the new performance measurement criteria (completion rates and progression to higher ed) wouldn't be too bad if they weren't being used as a one-size-fits-all blunt instrument for the purpose of constructing a league table. They probably only need a bit more granularity in there - eg, how are you performing on completion rates for internal, full-time students? OK, how are your part-timers doing? How about your mature students? and so on. That would at least provide fewer perverse incentives.

Veteran: extramural's a tough area. The students are more likely to be trying to juggle study with other commitments and at the same time can't be given the same level of support as internal students (for entirely physical, practical reasons). It inevitably suffers a higher dropout/failure rate than internal study - which isn't to say that extramural providers shouldn't be trying to minimise that problem, or should be given carte blanche to fling taxpayers' money at people who really shouldn't be attempting university-level study, but is to say that that inevitably-higher failure rate needs to be taken into account when constructing performance measures.

The Veteran said...

PM ... thank you for your considered reply to my question.

But I continue to wonder why MU continues to accept some students when their level of comprehension of examination questions and their ability to write coherently I would put at the Year 9 level.

I can pick with about a 80% degree of certainty the names of sudents who who fail to front come examination time.

Bums on seats has been alive and kicking over the last seven years that I have been involved.

Psycho Milt said...

...their level of comprehension of examination questions and their ability to write coherently I would put at the Year 9 level.

Oh, believe me, this is by no means restricted to extramural students...