Dr Gillian Houston on doctoral examination — Part 1: Understanding the viva

In this series, Dr Gillian Houston, former chair of UKCGE, discusses her recent publication The final examination of the UK PhD: fit for purpose?, written with Ingrid Lunt and published in July 2024. Gill is currently an independent researcher, having completed her own PhD in 2018 focused on the processes, attributes, and outcomes of the PhD examination in the UK. Gill is a long-standing member of the UK Council for Graduate Education’s governing body, latterly as chair and vice-chair, completing her term of office in July 2021. Her research work focuses on the topic of doctoral education and assessment.

In part one of this interview-based series, Gill discusses the processes of the viva, discussing the challenges and positives of the final PhD examination.

Could you explain what led to you looking into the viva experience? 

Yes, I’m very happy to do that. To begin with, though, I’d like to start with something that is common in anybody who asked me about this piece of research: it’s about the doctoral examination as a whole. The two parts. It’s about the thesis and the viva. And I wanted to make that point, because so many people say to me, ​“oh, your research about the viva.” 

Well, yes, it is. Because the data that emerged from my study about the viva was fascinating, and it turned out to feature more prominently than I expected. But if you look at the first findings chapter in my thesis¹, it’s about the process and the continuum of judgment that occurs across the thesis and the viva. 

I think people want to focus on the viva because it’s thought of as mysterious and an unusual form of assessment. But, actually, what I found in my study was that the thesis quality is principally what the examiners make their judgment on. The viva is really important — I don’t want to play that down. But it would rarely rescue something that was going to fail. 

One of my respondents said, if ​“you’ve got a wobbly thesis, the viva becomes much more important.” I had three examples of my 10 cases in the research where the viva changed the outcome: twice for the better and once for the worse. The examiners I interviewed made that clear — it’s the thesis that most of their judgments are based on. 

I wanted to do this because, in my professional life, I was aware the doctoral examination was an interesting, unique example of assessment in higher education. It’s the only degree that confers the status of independent researcher. And, therefore, the examination needs to be a really rigorous test of academic, professional, and personal attributes. I was very interested in how that was done. 

I’d also noticed that social sciences researchers were particularly critical of the two-part examination, in particular, the viva. This is across the globe, really. I was curious to find out why they judged that it didn’t fit with the more recent expectations of assessment practice in UK higher education. 

Could you briefly describe the methodology you used for this project? 

The PhD study meant that I attended 10 PhD vivas — not other [types], although one of them was a practice-based PhD and one was a clinical PhD. So, I did have some variety across a range of disciplines in six universities, and I observed and took notes in the vivas, but I didn’t participate in any way. 

On page 100 of my thesis, there’s table 4.4 that contains further details of the interviews, like the split between STEM and Arts, Humanities and Social Sciences, etc., and how many were in each category. 

One of the main challenges for the study was obtaining access because one reason why very little research has been done on the PhD examination is because of its privacy, its confidentiality, and mainly, I think, the access problem. I got full ethical permission from my own university. The six participating universities gave me approval to contact the Pro Vice-Chancellor for Research, and then they got in touch with departments. All this resulted in opportunistic sampling. It was very good that I happened to get equal amounts of STEM, Arts, Humanities, and Social Sciences. To extend the disciplinary range, I filled in with interviewing experienced examiners in other subjects, like Earth Sciences, Engineering, English, or Philosophy. Agreement to their involvement was obtained by the staff in the university. The supervisors were written to [with letters I provided], the examiners were written to, and then the candidates. 

My ethical approval required agreement of all the actors, so I had many false starts. Some supervisors said, ​“no, I don’t want this.” In most universities, supervisors are allowed to be the vivas, but not to participate. Some didn’t want [to participate] because they were worried about their candidate or the examiners said, ​“oh no, I don’t want an observer.” The candidates were much more up for it than the staff. There were delays, so instead of taking about six to nine months, the data collection took more than two years. That was a setback, but it was worth waiting. 

Then I combined the observations and all the interviews with candidates and supervisors with the interviews of seven experienced examiners and one university-wide docus group, making a total of 43 interviews. It was all really useful to have them as like a control, in a way. I took the interviews as an opportunity to ask people about the wider picture, not just the viva, but about the thesis and pass-fail rates, and that was very useful. 

I think the one criticism of my method could be that because I was there, people behaved differently. I’m sure you know that in some subjects, there have been these horror stories of how people fared in their viva. I think that’s a combination of some examiners’ rigorous questioning being taken as bullying of the candidates, so [the examiners] would need some training on that. A combination of that, and the candidates being slightly unprepared about what they would face in the viva. But I didn’t see any of that. People said, ​“well, you wouldn’t because people knew you were there.” But, when I went into the room, I made every effort. I never sat where the candidate could see me, and I tried to sit where the examiners couldn’t see me either. After the first few minutes, I’m sure they forgot I was there, because they were immersed in the process. 

Given that this project was centred around understanding if the viva is fit for purpose as a whole process, can you describe the responses from your interlocutors? What were the major, if any, differences in the interpretation of the viva from the different parties you interviewed? 

There were differences among disciplines in the way examiners approached the viva, and I found that very interesting. For example, different methodological questions arose that were dependent on the subject. 

In STEM, data generated by the candidate was often quantitative. Whereas in Arts, Humanities, and Social Sciences, more likely, but not always, it was going to be qualitative data. So that influenced the questions. Examiners in STEM fields tended to focus on technical and practical elements of the research, including any statistical analysis and its relationship to the candidate’s findings. [For example] was the statistical method they’d used appropriate to the findings? Did it help them to display the findings in the best way? Had the data been interpreted correctly? Could they have used a different statistical method? This incorporated the design of experiments and the integrity of the data. Is the way you’ve described this data and are the statistics you used appropriate? Or would it have been better to do something else? 

STEM examiners often use terms like methods, technical, statistics, statistical, data, and experiment. Whereas Arts, Humanities, and Social Sciences referred to methodology, methodological choices, qualitative or quantitative — the latter for Social Sciences where a couple of them had more quantitative data. 

In some Arts, Humanities, and Social Sciences fields, the choice of methodological options is not straightforward. Some candidates experimented with different methods — which was seen as a strength by the examiners — before deciding that what they finally chose was the best way of answering their research question. 

The other thing I noticed was that while, AHSS examiners tended to look quite broadly at the thesis and its outcomes, the STEM examiners tended to go to exact pages. I would advise candidates to take their thesis with them, and I’m sure all supervisors do that. With STEM, it was essential, because the examiners would say, ​“well, can we turn to the analysis on page X.” There’d be this very complicated statistical table and they wanted to explore it. 

All the examiners were really assiduous in the way they approached the thesis. They really, really knew it. They’d obviously prepared their questions in advance and decided which of them would ask which question. 

I also just wanted to say that mainly there was a huge amount of agreement among examiners of all disciplines about the core attributes that they were seeking in candidates, and there’s a table on page 188 of [my] thesis, table 6.2, where I did a typology of the attributes. [This typology] aligned quite well with some previous research from the States, by Jody Nyquist (2000)² and Barbara Lovitts (2007)³. It was very good to find that alignment and to be able to do this typology of four groups of attributes that all the examiners were interested in. 

Finally, I think the striking point about candidates’ responses — especially to the question about what they thought examiners were looking for — did not mention professional or personal attributes. It only mentioned the significant role of their research. I was a bit disappointed in that because I thought perhaps nobody said to them that other attributes really matter. 

What would you say the major challenges of the viva currently are?

In Australia, there has traditionally not been a viva oral defence. The history of that is the distances involved in Australia. But one of my examiners was Australian and, at the end said, ​“you’ve persuaded me of the importance of the viva.” I think Australia is now much more inclined to have a viva if anybody is concerned that the candidate is going to have trouble defending the thesis – and following the pandemic, virtual vivas are used much more. 

I know quite a lot of colleagues who have been examiners of Australian theses, and they say it’s very unsatisfying. So, although in Australia there are normally three external examiners, they never meet, and they don’t know what the outcome is; it’s not very satisfying to the examiner. 

I also wanted to mention the introduction of independent chairs or convenors in vivas, and that helps in those situations that I mentioned at the beginning where candidates have had an unpleasant experience. They should enjoy the viva; it should be the point in their career where they become a peer with the people who are examining them. I saw several examples of that, [where] the examiners were so fascinated by the research that all they wanted to do was talk about it more. That’s how everybody should feel, and luckily, all the candidates I saw enjoyed discussing their work — some more than others. 

I think the use of an independent convener is very good. It’s growing — I would say possibly about half candidates had an independent person at their viva who is an examiner, but in a cognate subject. 

Colleagues have pointed out to me that the difficulties of that can be that these conveners either don’t read the guidelines they were given, or they haven’t been trained properly, because they try and intervene in the exam. They’re only supposed to intervene if they see the candidate is very uncomfortable, either physically or mentally, or if one of the examiners has a problem, if there’s illness, or if somebody asks them a question about the procedure. Those are generally the roles of the independent convener. Some of them feel that they want to take a different role. 

An alternative to that is to record the viva. I have a colleague who does that routinely in a social sciences subject. Mostly those recordings are just stored electronically, but then if a problem emerges post-viva, they can listen to them and hear what went on. I think that’s quite a good safety net. I think those two [procedures] can overcome some of the behavioural problems in the viva. 

The second [challenge] is candidate preparedness for the viva. Many good supervisors enable candidates to have a mock viva. I would say that’s brilliant preparation, as long as the candidate knows that, apart from some basic questions, the real examiners are not going to ask the same questions as the mock examiners. Most often supervisors will do the mock viva, and of course they know much more about the candidate’s research than the examiners are going to. They know all the peaks and troughs that happened along the way. And everyone who reads a thesis is going to come up with different questions. 

The final thing I’d like to say here, which, I think we’ve made something of in the paper, is that the whole issue of candidates knowing how they’re being assessed, even if implicitly on their professional and personal skills, needs to be brought up. So those are the challenges, according to me. 

It’s quite interesting to hear that a lot of the candidates didn’t necessarily realise that they weren’t just being examined on their research work. It seems to present a unique challenge of how we communicate that facet of the examination better to them, which could also deal with some of the nerves. 

Yeah, and in some subjects, candidates are asked to do a 10-minute presentation at the beginning of the viva. I think that’s a really good way of settling somebody down because by the time you get to the viva, you should be the world’s expert in your topic. Therefore, what can be nicer than to talk about it for 10 minutes with eminent people in your field? You want to impress. At that stage, you are supposed to be a mature researcher, so small challenges should not faze you. I know it’s really hard and people are terribly nervous. I understand that, but they do need to have a bit of steel when they go into the exam. I don’t mean they should be defensive or argue with the examiners. Take graciously the advice they give you, but don’t just, cave in when they challenge you. You’ve got to show them that you can argue your case. That’s really important. 

On [the UKCGE] website, there’s a document that Ingrid Lunt and I put up, which is advice for candidates about preparing for the viva. She and I used to do sessions for candidates with vivas upcoming. We always said, do argue in a gracious way. Don’t be defensive but defend your decisions. [However], the best preparation for a good viva is a high-quality thesis. 

Can you discuss the positives that your interview participants identified in the viva? 

I found the candidates I interviewed to be so impressive, and of course, they are. Their insight into why there needed to be a viva was really spot on. Most of the points I’m going to make have come from interviewing the candidates but aligned with the examiners’ views. 

There wasn’t any disagreement about the positives of the viva among any of my interviewees. In my thesis, I’ve got quotes from candidates, quotes from examiners and supervisors, and you can see they put it differently, but basically, they’re saying the same thing – that the viva was a positive experience and a necessary component of the assessment. 

So I’m going to start with the thesis defence; the defence of the thesis is a positive purpose. It’s an opportunity for the candidates to explain elements of the thesis: what went well, what went not so well, what they wish they’d done, what they wish they had done differently. A couple of the candidates said, ​“it’s impossible to put everything in the thesis.” Of course it is, you can’t even put all your data in the thesis. [That’s] very frustrating as far as I was concerned. The thesis may not be of a similarly high quality all the way through because you may have had some things that were difficult. But one thing I would say is that the analytical element of the thesis is very important and if, when the candidate got to the viva, they had not been sufficiently analytical about something in the thesis, the examiners would pick up on that and ask them to go into it in more detail. That did the candidates a favour because there always was more detail. They maybe left something out because they thought, ​“oh my word limit’s coming up or whatever.” I think meeting somebody and enabling them to explain stuff avoids misunderstandings by the examiners. It’s much fairer to the candidate I would say so that’s a that’s a big positive: fairness. 

Second thing is [it is] an opportunity for the examiners to confirm their thesis judgments and deepen their understanding. Really, in a way, that’s an extension of the same point. Meeting somebody, being able to explore with them what they’ve done, whether the examiners can say to themselves, ​“okay well that’s what I thought before but now I’ve spoken to them and my view about that is slightly different.” I saw that happening. 

Certainly, particularly some of the science ones deepened their understanding of the candidate’s topic because they were very focused on exploring the thesis in detail with the candidate. That was quite important because they discovered that the candidate had a huge amount of knowledge underlying what was in the thesis. [That is] very helpful. 

Affirmation of the candidate’s academic achievements, [as well]. The candidate saw it as a rite of passage… an important rite of passage. What I was saying before about an opportunity to discuss with your peers what you’ve done, that’s how they saw it and they thought the rite of passage was a good thing. They became equal during their viva. I thought that was a positive. 

The next thing, which is not relevant to all candidates because in some cases it’s absolutely obvious that the candidate’s written the thesis or done the research, is to confirm authorship and authenticity of the thesis. That also enables the candidate to take ownership of their research and to show research leadership which is quality examiners are very hot on; they really want to see that being displayed. 

Finally, it provides the candidate with feedback on their research. In the UK, the fact that the candidate does not know the outcome of the exam before they do the viva enables them to get feedback on their research and to improve the quality of the thesis for posterity. It’s there then and you move on, and you do more work and, [especially] with younger people, they go on to do greater things. It’s not, as Mullins and Kiley (2002)⁴ said, a Nobel Prize. It’s something you do early in your career usually. I think it’s important to get as high quality a theses as possible because most candidates are directed to read other people’s thesis in their field before they start writing theirs.