Dr Gillian Houston on doctoral examination — Part 2: The viva in context

Dr Gillian Houston

Independent Researcher

In part two of this series based on a UKCGE interview with Dr Gillian Houston, former chair and vice-chair of UKCGE, Gill discusses the viva within its contextual bounds – analysing the different modes of examination that other countries enact, how the viva fits into different degree forms, attributes examiners look for while carrying out the viva, and how PGRs can better prepare themselves of the modern day viva. 

This resource is a continuation of part one of the discussion with Gill, focused on understanding the process, positives, and challenges of the viva. This dialogue comes as a discussion of Gill and Ingrid Lunt’s recent article publication titled, The final examination of the UK PhD: fit for purpose?

How can we reconcile the traditional viva experience with the unique requirements of new degree types like collaborative, professional, and practice-based degrees? Especially given that a lot of them have potentially different forms of theses or artifacts attached that the viva is being examined in conjunction with? 

I like how you asked that because I think the question applies to the thesis and the viva.

So, let’s just touch on thesis format and models of theses. There are still some subjects where all you can do is write a monograph. My subject was like that, but I published a paper with my supervisor before I did mine about halfway through mine. It was important to me that I got published but it wasn’t the norm. I could refer to the publication, but I couldn’t put it in my thesis. Whereas in some subjects, for example economics, biological sciences, it’s the norm for a thesis to be made up of two or three chapters or, sometimes only one, of a publication. I think that detracts from the skills you gain in writing a monograph because your brain has to be very organized. My main supervisor used to say, a golden thread has to run through it,” and I think it’s more difficult if you’ve got like two papers published. You have to top and tail them; you still have to do something about the methodology; you have to tweak that a bit; and your conclusions will be different because you’ve done more than one piece of research. I think that’s probably a good exercise in different ways in showing how you can draw together your body of work and situate it in the field. I think it’s fine for different disciplines to have different thesis formats. 

But when you come to the examination, it can be quite tricky. Let’s start with interdisciplinary degrees. I’ve heard, anecdotally — I haven’t done any research into this — that sometimes it’s quite difficult for examiners in different disciplines to align their judgements because of different subject requirments. 

Let’s take chemistry and archaeology: it’s possible that you’d have somebody who was doing research that would involve some chemical stuff, or even anthropology. And then you’ve got the traditions of archaeology and all the subjects have to be aligned in the assessment. 

During the pandemic, examiners had to find different ways of interrogating things like artefacts or plays or music. Some of that could be done through recording, some of it could be done through video. So, I think there are ways, it’s just people have to get used to thinking about [it differently]. And I think those ways are probably integrated already in examination processes. People may already submit videos, for example, and the practice-based PhD that I observed, the person [was] being examined on two of the films in their professional career. They were a mature candidate. One of the examiners said that the films were brilliant… they used that word. Unfortunately, though, because the candidate was working and had only had just over a year’s registration for the doctorate, it was quite hard for them to make sure they’d done all the necessary literature reviewing and that and immersion in the field. They passed. The examiners were totally convinced of their ability to do independent research — the usual stated criterion for doctoral exams — but the person had to do some corrections to include more literature and to situate their work in the field. So yeah, I don’t know whether that’s new. I suppose it is slightly. 

The clinical one I did, that had its own challenges because, again the person was working. So, I think it’s not necessarily the type of degree, it’s the field. I would say this quite often with doctorates. These people had day jobs. One had a clinical job, the other had a professional job. Accessing the adequate amount of support for doing their thesis was quite challenging for them. So sometimes it’s not the type of degree, it’s the context and the field. 

One of the key takeaways from your project is understanding how examiners aimed to assess not just academic prowess, but also personal and professional skills during the viva. What key attributes did you observe examiners looking for? 

There’s a big section on this in my thesis. The table that I didn’t have room for in the article1 has been published in two other places. If you look at the references at the end of the article, it’s in a book edited by Lee and Bongardt (2021)2. In a way, that table is better developed than the one in my thesis, because I’d obviously have more time to think about it. 

I think that three important conclusions can be drawn from the attributes that I was able to do a typology of in my thesis. First thing, the originality or a contribution to knowledge — I don’t conflate them. I never say original contribution to knowledge, because what I discovered in the paper that I did with my supervisor halfway through my doctoral research was that in different subjects, people reverse the meaning of originality and contribution to knowledge. It’s a very kind of technical paper, but it was really interesting. So, originality or a contribution to knowledge is a necessary but not sufficient attribute for the award of a doctorate. It’s absolutely necessary: the underlying basis for doctoral assessment that differentiates it from taught degrees. 

The second thing is that while some attributes are research specific and others professional or personal, they might be so amalgamated in the individual that it’s quite difficult to separate them. That to me is key when we get to the bit about how can we better prepare doctoral researchers for the examination. 

And the third point is both the thesis and the viva are significant in examiners evolving judgments. So I don’t think an exam without a viva in the UK would be fair. I think the attributes that are demonstrated in the viva really support that. 

What I found with attributes was that, although examiners give prominence to the thesis in their judgments, it was really striking that many of their responses implied a really close relationship between the thesis and the viva. I also noted that some attributes could only be assessed in the viva. 

A range of essential attributes were common to all examiners’ judgements, e.g. intellectual rigor, research integrity. Those exemplified a mix of research specific and personal qualities that would be evident in those successful candidates. 

In the interview responses, the omissions were as significant as the inclusions. Very few referred explicitly to what we might term transferable skills. And in a way that was really useful because that’s much too generic a description of what they were looking for. So I have tried to refine them. 

The four groups in the typology are: originality or a contribution to knowledge; publishability; research competence and integrity; and intellectual rigour. 

Publishability. Of course, that’s very discipline specific because some people have proved their publishability already. And one examiner said to me, I believe that makes it easier for examiners because they know that they’ve been peer reviewed before.” In some subjects, that’s seen as an indicator of somebody exceptional because they have published, whereas it might not be common in their area. 

Research competence and integrity is the third. This is where I think there’s a crossover with research, professional and personal skills: research competence, integrity, research methodology and methods, data analysis, coherence — including the thesis structure, ability to situate the research in the wider field, avoiding plagiarism, acknowledging contributions from others. 

And the final one is — this is the kind of catch all for the professional and personal — intellectual rigor: critical, analytical, and reflective thinking, problem solving and logic, independent thought and research leadership. And again, they’re general, but they’re much more than transferable skills. 

There were some additional core competences that Nyquist3 identified but the contributors to my study did not prioritize them, though. So they didn’t get into my thesis, but I would say they’re still complementary: commitment to an informed career choice based on exposure to a broad array of opportunities and paths; teaching competence broadly considered in one-to-one interactions in the classroom; preparedness to be a faculty member, project manager, motivator, evaluator of others’ learning; understanding of the diversity of present and future students and workforces; to have a global perspective; Importance of doctoral work in relation to a global economy; sensitivity to cultural differences; ability to see oneself as a scholar citizen who will connect his or her expertise to the needs of society; and finally, ability to communicate and work in teams and explain work to public audiences and to those who set policies. 

Tying into that, you discuss in your paper other forms of final examination such as the practices in Europe and in the United States system — were there any proposals for new forms of examination that you heard from interviewees that did not make it into your final paper? 

I did, and I’ve outlined them in a bit more detail in [my] thesis4. I want to mention two forms that people were positive about, but not universally positive. Most people thought, broadly, that the UK system is pretty good. They like the combination of thesis and viva. They think it’s fair. And international researchers have also found that the general process around the PhD or the doctoral exam is pretty good, notwithstanding some of the things I’ve said earlier about the criticisms. 

Sometimes, people talk about the European system.” Well, there isn’t one. In fact, each country and each discipline has its system. And you could say, well, that’s very unscientific. It should all be the same.” Well, it can’t be. And there are reasons why these processes of examination have developed. But I would say, broadly speaking, in Europe, there are multiple models of doctoral examination and they depend on conventions in both the country and the discipline. 

Mostly in European countries, whatever your discipline, you will have a public viva. Your friends and family can come, which I would have loved. And anybody in the university can come or from another university. So it’s not the closed second part of the exam that we have in the UK. 

However, one of the examiners I interviewed said, okay, I see the benefits of a public viva. However, I would not feel comfortable asking such difficult questions of the candidate in a public setting.” Actually, that examiner said afterwards, it would affect the best candidates because you want to ask them the most challenging questions.” And they said, you can get much more from the candidate and it’s much fairer if you can ask them these more challenging questions and I wouldn’t feel comfortable doing that in front of an audience.” I thought that was a very good response and interesting enough to mention. 

In some European countries, they also have something like a challenger or opposer. The purpose is you’re on the stage with the candidate and you challenge them. You’re there to probe them, if you like, to really argue their case. I’m not sure people in the UK would be that comfortable with that. 

The third thing, which is — it can go across the other two models that I’ve already described — that in some cases in Europe, discipline and or country, the candidate knows they’ve passed before they performed the viva. And for me, that’s not good because you don’t get one of the huge benefits of our system, which is giving the candidate feedback that they can use further on in their career. It doesn’t improve the quality of the thesis either because it’s already kind of banked by them. Those are interesting models, and they have their merits. 

Going back to your question about interdisciplinary, practice-based [degrees], and all that, I’m going to turn to the US model because I think some elements of it are really good and others aren’t. I really like the thesis panel approach, especially before the defence in the States. The thesis panel can work with the candidate, and you get that improvement in quality because the candidate gets advice from a wider range of people than just two supervisors or sometimes three in the UK. I think that’s likely to result in a better thesis because the more people you have, the more perspectives you will have on the research that’s being done. Also, the candidate’s getting advice early on. The earlier you get advice about stuff, the better the end product’s going to be. It’s a great idea to have a thesis panel. 

But I think, possibly, the flaw is that those people then examine the candidate. That’s my understanding. They have, in a way, too much knowledge, whereas in the UK, you cannot know your examiners. You might have used their work, but if you’ve met them, it could only be at a conference where you’ve heard them speak. You do not have a close relationship with the examiner. To me, that would be inevitable if they were advising you about your thesis. 

I believe that in the US the supervisor is a member of the thesis panel. And I don’t really agree with that. Whilst I think it’s good for supervisors to be at the viva if the university allows it. I think [that’s] very important, because the supervisor’s there to take notes for the candidate for when they have to revise the thesis or the artifact or whatever it is. But they must not say anything. They must not contribute to the assessment. I think, again, that that space between the exam and what’s gone on before is really important. 

Finally, how can supervisors better prepare PGRs for their vivas and for their examination as a whole? How can they better align new forms of Postgraduate education — in which collaborative, cohort-based, professional degrees, and practice-based methods are becoming more common? 

There’s quite a bit about this in the conclusions to the paper. The viva is essential to the examination as a whole. It plays a crucial role. 

But I think there [could be] some tweaks to the examination, as well as more candidate preparedness. I think it should be essential that the final examination requires examiners to show in some way — we put [this] quite explicitly in paper — that they should comment on [these professional and personal skills that are so implicit] in their reports. I don’t want to make life more difficult for examiners because it’s a difficult job and they’re paid very little. They do it for love of the subject, really. Non-research skills aren’t explicitly mentioned in the viva, but because they have such a significant bearing on the exam outcomes, I think that more emphasis on the way that they are developed alongside research capability needs to be acknowledged by examiners somehow in the final assessment. 

They may not want to put it in their reports. But there may be some way of doing that. It wouldn’t require much. But also I wouldn’t want to disadvantage candidates who haven’t displayed a huge range of those. It’s a difficult judgment call. But somehow, we need to find a way of examiners being able to say, yes, this is a rounded doctoral candidate, and now independent researcher.” 

In Australia, there was a study that addressed the perceived lack of employability of doctoral graduates. The UK and Australia are the two places where people criticise doctoral employability. And of course, very few doctoral graduates in the UK, at least, and probably in Australia, go into academic jobs. So, the employability aspect is really important. Chen et al. recommend improving stakeholder understandings of the value of doctorates. That means helping employers understand what they’re getting, as well as the candidates and the researchers. 

There was another study about doctoral graduates in their 30s and 40s. The authors concluded that the life skills they’ve acquired through personal experience had helped them more than their degrees to cope with demands in their life. 

In the recent Assessment and Evaluation article, we suggested helping doctoral candidates to think about their experience holistically and all those professional and personal attributes they’ve acquired during their degree. I think that would help prepare them for the exam as well, because they’d have a stronger sense of the value of what they can do. I think it’s very important that starts quite early in the degree — not to interrupt what they’re doing in their research area — but to just help them reflect that, okay, if I wasn’t diligent, if I wasn’t determined to follow through on this research, I wouldn’t be persuasive, and my results wouldn’t be so good.” We always have progress files in universities, where the individual records what they learned at a different stage, and I think that needs to come back for doctoral candidates. 

In Europe, in some areas, candidates have got to put in an appendix to their thesis the range of attributes that have been important to them in developing as a researcher. I think that would be a great thing to do in the UK. 

Basically, the supervisors just need to be aware of it all the way through the degree. And that’s a big ask, because supervisors are very busy people, they’re put upon in universities, but they always have the candidate’s interests at heart. They do an excellent job, but it’s adding something to their burden as well. We need to think creatively about how to do that. The aim would be to instil greater confidence in graduates of their employability, particularly. But also, their approach to their lives. You gain so much from having to organise a research project and then write about it. 

I don’t think, at the moment — and we didn’t think when we did the paper — that the existing outcomes of the final examination are explicit enough to confirm candidates’ readiness for a huge range of employment which they go into or to provide evidence of their ability in non-academic roles. I don’t think we do that well enough just now. I think we have to leave it to universities and to examiners and supervisors to think how to do this better. 

I would love to do some more work on this and to interview candidates about what they think they’ve gained, so they can articulate them in the viva and in the thesis as well. The examiners have much more time to concentrate on the thesis than they do in the viva. 

I think improvements can be made to the preparation of candidates in advance and to the examination itself. 

And I think that’s a brilliant statement and point to end on; prepare people at the very beginning of their journey for what this research means for them — not just in this degree and not just in this moment in their life — but what it can mean for them forever. 

That’s a lovely way of putting it. It’s our responsibility to do that. 

Footnotes

  1. 1 Houston, Gillian, and Ingrid Lunt. The Final Examination of the UK PhD: Fit for Purpose? Assessment &amp; Evaluation in Higher Education 49, no. 8 (16 November 2024): 1182–95. <a href=“https://doi.org/10.1080/02602938.2024.2364026”>https://doi.org/10.1080/026029…</a>.
  2. 2 Houston G (2021). Doctoral Examiners’ Judgements: Do examiners agree on doctoral attributes and how important are professional and personal characteristics?. In A Lee and R Bongaardt (Eds), The Future of Doctoral Research. Oxon, Routledge. 
  3. 3 Nyquist, Jody D. The PhD: A Tapestry of Change for the 21st Century. Change 34, no. 6 (2002): 12–20.
  4. 4 Houston, G. A Study of the PhD Examination: Process, Attributes and Outcomes. University of Oxford, 2018. https://ora.ox.ac.uk/objects/uuid:07291f0e-e80b-4b06-a6af-b3ac8b90a00e