Outline and content of the book
Chapter 1 begins with a discussion of the ethical case for evidence-based school leadership. This is followed by an examination of a number of additional arguments in support of evidence-based school leadership, which include: the maintaining and upholding of professional standards; reducing occurrences of unnecessary ineptitude by school leaders; making schools less vulnerable to educational fads, fashions, ‘rusty’ evidence and bullshit. The chapter goes on to consider the impact cognitive biases have on thedecision-making process and how evidence-based practice may act as a partial counterweight to such biases. Finally, given changes in the context in which schools operate, be it increased levels of autonomy or changes in the demands for accountability, the chapter discusses why evidence-based school leadership is particularly timely.
Chapter 2 provides a comprehensive definition of evidence-based practice. This leads on to a discussion of some of the commonly held misconceptions about evidence-based practice. Next, there is an examination of some of the key terms, for example, conscientious, judicious and explicit, which are embedded in the working definition of evidence-based school leadership. The chapter concludes with a discussion of the limitations of evidence-based practice.
Chapter 3 looks at the scope of evidence-based school leadership and management. This leads on to an analysis of: what is meant by the term ‘problem of practice’; how these problems of practice can be identified and prioritised; and which problems of practice are best suited to an evidence-based approach. This is followed by discussion of a number of techniques, drawn from medicine and organisational science, to help evidence-based school leaders frame and develop well-formulated questions. The chapter then looks at techniques associated with the formulation of answerable questions, which can be integrated into the day-to-day work of the school.
Chapter 4 examines the nature of various sources of scientific research, school data, practitioner expertise and stakeholder evidence, and also includes a discussion of the difference between tacit and explicit knowledge. This is followed by a review of the challenges associated with the use of the best available evidence and what criteria might be used to help identify that evidence. Subsequently guidance will be provided on how to develop an evidence search strategy for all four sources of evidence and is accompanied by a brief examination of how to distinguish between ‘good, bad and ugly evidence’ found on the Internet. Attention will then be given to understanding the components of both practitioners and stakeholders of theories of action. The chapter will then consider some of the internal and external data and evidence that is available about most schools. Finally, the chapter will look at a technique known as ‘open to learning’ conversations as a mechanism for acquiring stakeholder evidence.
Chapter 5 sets out to try to help evidence-based practitioners know when to trust expert advice, and will make use of a framework developed by Willingham (2012). This is followed by a discussion of thenotion of a hierarchy of evidence, and leads on to a review of the nature of both systematic reviews and randomised controlled trials. The chapter then draws upon the work of Wallace and Wray (2016) to examine components of an argument – the claim and associated warrant – and how these can be used to appraise the quality of research. Consideration is then given to how to make the most of research, the use of abstracts and the writing of critical appraisals of research. The chapter concludes by presenting a comprehensive framework to help evidence-based school leaders judge the usefulness of research evidence.
Chapter 6 provides a brief introduction into becoming a critical reader of quantitative educational research evidence and covers topics such as: what is meant by an ‘effect size’; how effect sizes are calculated; how to interpret the ‘size’ of an effect size; meta-analyses and their limitations. The chapter then looks at correlation and how this is often misunderstood. Next, the chapter briefly explains the terms: p-values, confidence intervals, statistical significance and highlights some of the challenges in ensuring they are interpreted correctly. The chapter concludes with a checklist of questions to be asked when seeking to interpret the quality of quantitative educational research findings.
Chapter 7 moves on to consider ways of appraising different sources of evidence. In particular the chapter looks at the challenge for evidence-based school leaders associated with using ‘organisational/school facts’ to inform decision-making. Discussion then focuses on problems associated with appraising stakeholder feedback and in particular how to know when you are receiving good advice. The chapters also looks at what to do when you, as you no doubt often will, disagree with others. The chapter ends with a discussion of experience and intuition, and explores the limits of expertise, and whether intuition can be used as a trusted source of evidence.
Chapter 8 looks the challenge of aggregating different sources of evidence. Initially, simple tables are used to aggregate research evidence, followed by more sophisticated examples found in systematic reviews. This leads on to a discussion of how the different sources of evidence – be it school data, stakeholders’ views, practitioner expertise and research – can be combined. This is followed by a discussion of how logic models can help classify and organise the available evidence. The chapter concludes with a review of several approaches the evidence-based school leader can use when judging both the quality of the aggregation and the synthesis of the evidence.
Chapter 9 examines the challenges of integrating evidence into the decision-making process. Drawing upon the work of Peter Drucker, the chapter will consider a simple rule of thumb to be used when making a decision on whether to act or not. In doing so, this will lead to a brief discussion of the costs and benefits associated with educational intervention. Following on from this is an examination of some the key issues associated with the decision-making, for example, the strength or otherwise of recommendations. Next, the chapter considers the impact of cognitive biases on the decision-making and strategies, and identifies strategies, such as the ‘premortem’, that could be adopted to minimise their impact. The last part of the chapter will begin to explore the issues associated with implementing the decision – such as the scale of implementation – and the Plan–Do–Study–Act (PDSA) cycle. Finally, attention will then focus on some of the many challenges associated with trying to implement successfully an intervention or innovation on a large scale, which will be particularly relevant to multi-academy trusts.
Chapter 10 explores the issues associated with trying to evaluate the outcome(s) of the decision that has been taken. In doing this, the chapter looks at the distinction between merit (does it work?) and worth (even if it worked was it worth it?). The chapter then looks at a technique known as After-Action-Review, to help the busy
evidence-based school leader learn from outcomes of decisions. The chapter also explores issues associated with learning from failure and how best a school can do that. There is also a brief discussion of contribution analysis and how that can be used to help evaluate decision outcomes. Finally, a checklist will be provided to help school leaders assess their performance as evidence-based practitioners.
Chapter 11 explores the difference between the rhetoric and the reality of leading the research- and evidence-based school (Coldwell et al., 2017). Next the chapter looks at a number of mechanisms that can be used to promote the capability, opportunity and motivation to use evidence. The chapter then makes extensive use of the work of Brown (2015) to help draw up a checklist that school leaders can take to help create a research and evidence-based culture within their school, with various elements being explored in some detail. Finally, the question is posed – is your school ready for evidence-based practice?
Chapter 12 brings the book to a close with a number of observations, suggestions and recommendations for individuals and organisations involved in evidence-based education – suggestions, observations and recommendations which hopefully will help close the gap between the rhetoric of evidence-based education and the reality of day-to-day life in schools.
How to read this book
The chapters in this book can either be read consecutively or as stand-alone chapters, with this depending upon your existing knowledge and understanding of evidence-based practice. Whatever way you choose to read this book, each chapter follows the same basic structure.
1. All chapters begin with a basic overview as to what is to be covered in that chapter.
2. Where at all possible, worked examples have been provided to show you how to apply some of the techniques associated with evidence-based school leadership.
3. The end of each chapter has a list of key points which summarise the contents of the chapter.
Finally, by no means should this book be seen as a definitive statement as to how to go about evidence-based school leadership. In the context of education there are still many questions to be addressed and which were highlighted in a recent UCL Institute of Education debate on evidence-informed practice (IOE, 2018). Some of the questions raised in the debate include:
Do we really know ‘what works’ in education?
Can we ever really find out ‘what works’ in education?
What is the role of values in determining ‘what works’?
How do we, or indeed should we, maintain a ‘broad church’ of what is meant by reliable and valid educational research?
How can educators determine whether ‘what works’ is something that is worth doing in the first place?
This book does not answer these questions, but should be seen as an attempt to bring together a diverse range of influences and resources to help school leaders make more evidence-based decisions. If, in reading this book, you are able to find some small insight or technique into how to make decisions based on evidence rather than prejudice, fact rather than bullshit, knowledge rather than ignorance, then it will have done its job.