The Dashboard Turns Around
Evaluating the managerial university, part one
On a Wednesday afternoon in late October the school meeting assembles in a teaching room that still smells of the morning's tutorial. Twenty-two academics sit in chairs bolted to writing tablets designed for the forearm of a twelve-year-old. At two minutes past the hour the Associate Director of Student Recruitment, Growth and Pipeline (Domestic and International) arrives with a laptop, a colleague whose title nobody catches, and a slide deck in the university's approved shade of teal. The first slide says Enrolment Insights. The second shows the Bachelor of Arts in Ethnic Studies down eighteen percent on last year, a red arrow pointing at the floor with the grim confidence of a dowsing rod.
The explanation takes forty minutes. Demand has softened. The market has shifted. Gen Z is choosing differently. Ethnic studies, the Associate Director suggests with the sympathetic tilt of the head reserved for a relative's unsuccessful business, may be a niche proposition. Then the room is asked to own the narrative. Could the academics staff the Saturday open days? Could someone make short videos for the social channels, ideally with energy? Could the degree be refreshed, perhaps rebadged as something with the word global in it, to lift conversion? The academics write these requests down. Nobody writes anything down for the recruitment team, because nobody in the room has the authority to ask it anything. The meeting ends with nine action items, all assigned downward.
Eleven months later the programme appears on a list titled Portfolio Sustainability Review. Three months after that, in an email sent at 4.52 on a Friday, the university announces that the Bachelor of Arts in Ethnic Studies will close to new enrolments. The email uses the word regret once and the word sustainable four times. Students who came to the university because the degree existed are told they will be supported to complete. Two academic positions are disestablished. The recruitment vertical's budget for the following year rises by six percent. The Associate Director is promoted to Director.
The demand they could not find
Here is the part the slide left out. The appetite for ethnic studies among young people is one of the better-documented facts in contemporary education. The field was born of it. The California State University system built the first ethnic studies courses, programmes, departments and a whole College of Ethnic Studies in the late 1960s in response to student demand, and half a century later California law made a three-unit ethnic studies course a graduation requirement for CSU undergraduates. The movement runs down into the schools too. Los Angeles Unified, the second largest school district in the United States, now requires every high school student to complete a one-semester ethnic studies course to graduate. Where the courses are missing, students ask for them: a youth participatory action research project in San Antonio interviewed students who were not enrolled in Mexican American studies and found they wanted to learn more and take the courses.
The evidence on what these courses do for students is just as strong, and it speaks directly to the numbers any recruitment and retention office claims to care about. A preregistered study of San Francisco ninth graders found that being assigned to ethnic studies substantially increased high school graduation, attendance, and the likelihood of enrolling in college. A newer study tracking more than 24,000 San Francisco students across more than a decade found that ethnic studies enrolment raised grade point averages and cut course failure rates, and produced a 15 percent increase in students meeting the GPA threshold for admission to the University of California.
Read that as an evaluator would. Here is a field with a long record of student demand, a growing pipeline of school leavers who have already studied it, and causal evidence that it keeps students engaged and lifts them into higher education. A competent recruitment operation would treat such a degree as a gift. It would find the schools and communities where the demand sits, speak to young people from migrant, Pasifika and Māori whānau who want to study their own histories and the structures that shaped them, and build campaigns around a subject that retains the students it recruits. When enrolments in a programme like this fall, the first question an evaluator asks is what the recruitment function did, where it went, what it spent, and whom it spoke to. At this university nobody asked, because nobody was required to.
The borrowed instrument
The managerial class that has spread through the university over three decades speaks the language of measurement fluently. It talks of key performance indicators, dashboards, insights, data-driven decisions. It has built an apparatus that counts the academic in every direction: publications, citations, grant income, teaching evaluations, workload hours in fifteen-minute blocks. What it has never done is turn that apparatus on itself. The dashboard points one way, down the stairs.
This matters to me professionally. I have spent more than two decades designing and evaluating communication and health interventions with communities at the margins, across more than fifty sites in seventeen countries. Evaluation science is a demanding discipline, far older and more exacting than the corporate dashboard. A community health programme seeking funding has to produce a logic model: what resources go in, what activities follow, what outputs those activities produce, what outcomes the outputs are expected to change, and over what time. It has to state its indicators in advance, establish a baseline, name its targets, and report against them to the people it serves and the people who fund it. If it misses, the evaluation records the miss, and the programme explains itself. A women's collective running a food cooperative in a village in Odisha is held to this standard. A university recruitment vertical of forty staff, spending millions of dollars of public and student money each year, is held to none of it that anyone outside the executive floor has seen.
Donald Campbell, one of the founders of modern evaluation, warned that the more any quantitative indicator is used for social decision-making, the more it will be corrupted and the more it will distort the processes it was meant to monitor. The managerial university proves him right every semester, but only on one side of the ledger. The academic's indicators are sharpened into instruments of restructure. The manager's indicators, when they exist at all, are reset each year to whatever happened.
The framework I am proposing takes the tools the managerial class claims to love and applies them where they have never been applied.
Three instruments and one audience
Every managerial function in the university, every vertical, portfolio, directorate and office, should be evaluated through three instruments working together.
The first is performance measurement. This asks the simplest question in evaluation: what does this unit exist to do, and is it doing it? The answer has to be written down before the year begins, as a small number of indicators tied to the unit's stated purpose, each with a baseline and a target. A student recruitment office exists to bring students into programmes. Its indicators follow from that and nothing else. Performance is monitored continuously and reported each quarter, in plain numbers, with no teal.
The second is measurement against established metrics. A unit's performance means little in isolation, so it has to be set against standards agreed in advance: sector benchmarks, the performance of comparable offices at comparable institutions, the unit's own record over five years. For recruitment this includes the cost of each enrolled student, the rate at which enquiries become applications and applications become enrolments, the time taken to answer an applicant, the time taken to issue an offer, and the accuracy of the unit's own forecasts. The word established carries the weight here. The metrics are fixed before the results arrive, and they cannot be quietly rewritten afterwards to match them.
The third is deliverable-based evaluation. Managerial units make promises constantly: a campaign, a partnership, a new market, a system, a strategy with a colon in its name. Each promise is a deliverable, and each deliverable can be specified in advance by what it will produce, when, at what cost, and with what effect. The evaluation then asks whether it arrived, whether it arrived on time and on budget, and whether the effect claimed for it can be seen in the numbers. A memorandum of understanding signed in a hotel in Kuala Lumpur is an output. Forty students enrolled through it in the following two intakes is an outcome. Evaluation keeps the two apart, which is precisely why the managerial class avoids it.
These three instruments only change anything if they are answered to the right people. Here the culture-centred approach does its work. In every evaluation I have run with communities, the central question is voice: who defines the problem, who sets the indicators, who reads the report, and who can act on it. In the university, the community most affected by the performance of a managerial function is the academics whose programmes, students and research depend on it, together with the bodies of academic governance, the academic board and its committees, and in Aotearoa the Māori academic leadership who carry the university's obligations under Te Tiriti o Waitangi. The recruitment vertical should report to the academics in the area it recruits for. The research office should report to the researchers whose applications it handles. The evaluation should sit with academic governance, and it should carry consequences for budgets, staffing and the renewal of senior contracts.
The ethnic studies ledger
Take the Wednesday meeting and run it again under the framework.
The number of students recruited into the Bachelor of Arts in Ethnic Studies is a marketing and recruitment function. The academics teach the degree, design it, and keep it alive with their research and their relationships with communities, and they answer for that through the long list of measures already bearing down on them. Filling the seats belongs to the people paid to fill them. So at the start of the year the recruitment vertical presents its plan for ethnic studies to the academics in the area: target enrolments by market, the schools and community organisations it will visit, the campaigns it will run and in which languages, the budget for each, and the forecast yield of each. The academics, who know exactly where the interested students are because many of them came from those communities, can question the plan, add to it, and record their agreement or their reservations.
Each quarter the vertical reports back to the same room against the same indicators. Enquiries generated. Conversion at each stage. Cost per enrolled student against the sector benchmark. Average response time to applicants. Offers issued within the published turnaround. Schools and community organisations actually visited, set against those promised. When the report shows the ethnic studies campaign consisted of one page on the website and a paid post that ran for nine days in February, the academics can see it. When the vertical's travel budget took the Director to three international fairs and nobody from the office set foot in a South Auckland secondary school, the report shows that too.
At the end of the year the evaluation goes to the school and then to academic governance. If enrolments fell and the recruitment vertical met none of its own targets, the portfolio review examines the vertical before it examines the degree. The positions at risk are in the office that failed to deliver. If the vertical did its work well and enrolments still fell, the evaluation has earned its keep by showing that too, and any conversation about the programme can proceed on evidence rather than on a red arrow.
Retention sits in shared territory, and the framework handles that honestly. Teaching quality affects whether students stay, and academics already answer for it. So do enrolment advice that arrives in time, fee processing that works, visa support that answers the phone, pastoral care with enough staff to see a student within a week, and a learning platform that loads. Each of these is a service with measurable outputs, and each can be measured. When a first-year ethnic studies student withdraws in week five because her fees were misallocated and nobody replied to three emails, the withdrawal belongs in the evaluation of the finance and student services functions, not in the teaching evaluation of the lecturer who never saw her again.
The softest landing in the economy
Ask why the programme closed and the office did not, and the answer has less to do with ethnic studies than with the peculiar economics of managerial employment in the public university.
Consider the career of a certain kind of recruitment and growth executive. Several years in marketing at a telecommunications company, where the numbers went the wrong way. A stint at a consultancy, where the client did not renew. A brief tenure as head of customer experience at a retail chain that no longer exists. In the private sector each of these episodes would have followed the person like a smell. In the university it becomes a CV. Commercial experience, the appointment panel notes with approval. Brings a market lens. The for-profit world, for all its faults, has a way of noticing when a marketing department produces no customers. The university has found a way of not noticing, and so it has become the place where people who could not make the market work go to explain the market to historians.
The job itself is well appointed. An office with a door and a view of the ranges. A title with three nouns in it. A travel budget that covers the international education fairs in Delhi, Jakarta and Ho Chi Minh City, where the university's stand sits between two other universities' stands offering the same banner and the same lanyard. A calendar full of meetings about the calendar. A performance agreement that nobody outside the executive floor has read, with targets set each year at whatever last year happened to produce. When enrolments rise, the vertical takes the credit. When they fall, the market softened, the government changed a visa setting, Gen Z behaved unpredictably, or the academics failed to refresh their offering. Heads, the vertical wins; tails, the programme loses.
This is how the best protected jobs in the modern university came to be those with the least to show. An academic in ethnic studies who publishes nothing for two years will hear about it at her annual review. An associate director whose campaign produced no enrolments will hear about it nowhere, because no instrument exists that could tell him. The academic works under the most elaborate regime of measurement ever applied to a profession. The manager works under none. And when the budget has to be balanced, the institution cuts where the measurement is, which is to say it cuts the people doing the teaching, and leaves untouched the people who were paid to bring the students in.
The objections, taken in turn
The first objection comes from the seventh floor: academics are not qualified to evaluate marketing. They do not need to be. They need to know whether the students promised to their programme arrived, what that cost, and whether the unit did what it said it would. A community in a Santali village does not need a degree in epidemiology to judge whether the clinic that was promised was built and staffed. Evaluation rests on this principle. The people who live with the results of a service are competent to hold it to its commitments, and the technical work of measurement can be done transparently in front of them.
The second objection comes from colleagues who have spent years resisting the metric regime: surely more measurement is the disease, and prescribing it as the cure makes no sense. I share the suspicion, and Campbell's law cuts both ways. Any indicator will be gamed once it carries consequences. The answer lies in how the evaluation is built. Indicators must be few, set by those the function serves, tied to purpose rather than activity, and read alongside qualitative evidence: the voices of students who left, of applicants who gave up, of school students who never heard the degree existed. What makes the current regime corrosive is its asymmetry. A class that measures everyone and is measured by no one accumulates power without responsibility. Symmetry restores the relation the university was built on, in which administration exists to serve the academic mission and answers to the people who carry it.
The third objection is the one I take most seriously, because the managerial class will reach for it first. Any framework for scrutinising services can be turned into a pretext for cutting the people who deliver them: the subject librarian, the student advisor, the technician, the administrator in the school office who knows where everything is. The framework is aimed at the vertical, the layers of directors, associate directors, heads of portfolio and business partners that have multiplied above the frontline. Evaluation built on outputs tends to protect frontline staff, because their outputs are visible and countable. The student advisor saw four hundred students this semester. The Director of Strategic Enablement produced a roadmap. The evaluation will show which of these the university could least afford to lose.
Who holds the pen
Public universities in Aotearoa were built on a promise that a society made to itself, written into law as academic freedom and institutional autonomy, and carried alongside the obligations of Te Tiriti o Waitangi. Autonomy was granted to a community of scholars governing itself, and the statutory academic board exists because the people who teach and research were meant to have the decisive say over academic matters. Over thirty years that say has been hollowed out. Decisions about which programmes survive now flow from recruitment dashboards, budget models and consultants' reports, all produced by functions that answer to no academic body.
That the first programme to go should so often be one like ethnic studies is no accident. A degree that studies race, colonisation and the lives of communities at the margins is easy for a managerial class to call niche, because the managerial class does not come from those communities and has never tried to reach them. The closure then gets recorded as the market's verdict, when the market was never asked.
Evaluation returns the pen. A school that receives a quarterly report from its recruitment vertical, questions it, and records its judgement in the minutes of academic governance has recovered a piece of the self-government the law assumed it had. A Māori academic leadership that evaluates whether the university's recruitment of Māori students met the targets the university itself set, and holds the responsible function to account when it did not, turns Te Tiriti commitments from a paragraph in the strategy into a line in the ledger. This is what accountability looks like when it runs upward.
The series
This is the first in a series. Each piece will take one managerial function and apply the three instruments to it in detail, with the indicators, benchmarks and deliverables that function should answer for, and the academic body it should answer to. The next piece takes student recruitment and retention in full. After that come technology and the platforms that spin; professional services and the long chain of approvals between a request and a result; the research grants office, which should be measured on turnaround, errors and the success rates of the applications it touched; and finally management itself, the executive layer that evaluates everyone and has so far escaped every evaluation.
Somewhere on the seventh floor, a template is about to arrive. It asks six questions: purpose, indicators, baseline, target, deliverables, cost. It has come from the academic board, and at the top it names the Bachelor of Arts in Ethnic Studies. The Director of Student Recruitment, Growth and Pipeline reads it twice, then forwards it to a business partner with a note asking whether it could be reformatted as a dashboard, and whether anyone knows if the degree is still running.