CASE STUDY: ASSESSMENT USING ELECTRONIC LEARNING JOURNALS

Dr Madeleine Davies, Department of English Literature

 

Objectives

  • To use an electronic Learning Journal to improve attendance, engagement and attainment on a Part 3 module I convene, ‘Virginia Woolf and Bloomsbury’
  • To determine whether a Learning Journal + assessed essay assessment pattern offers a viable alternative to the ‘assessed essay + exam’ model favoured by the Department of English Literature (this in conversation with the ‘Diversifying Assessments’ TLDF project I co-lead in DEL see http://blogs.reading.ac.uk/t-and-l-exchange/connecting-with-the-curriculum-framework-using-focus-groups-to-diversify-assessment/)
  • To improve my ‘return of feedback’ scores on my modules; hard copy marking has always been returned to my students within 10 days yet students select ‘3’ or ‘4’ for the ‘speed of feedback’ question in their module responses. I wanted to see whether online return of marked work within the same period ‘felt’ like ‘5’ to my students more than hard copy return did.

 

Context

The pedagogic aims of my Part 3 module, ‘Virginia Woolf and Bloomsbury’ can be summarised as follows:

  • To gradually construct, over 11 weeks, a detailed and advanced knowledge of Virginia Woolf’s often complex texts and ideas.
  • To develop student’s understanding of the socio-cultural, political and literary contexts of the inter-war period.
  • To enhance skills of close reading and critical knowledge.

 

These are challenges because Woolf’s ideas connect with theoretical models including feminism, structuralism and postmodernism. In addition, the important contexts of literary modernism and of post-impressionist art have to be taught in accessible ways so that they can be understood to an advanced level. There is a great deal to learn and only thirty teaching hours available in which to develop the level of required knowledge.

 

Before I introduced technology-enhanced assessment to the module, the assessment pattern involved the following stages:

 

  • one 1500-word formative essay in Week 5 – the instruction was, ‘answer on one text’. Rushed, late, or missing essays characterised this stage.
  • one 2500-word assessed essay in Week 11 – the instruction was, ‘demonstrate substantial knowledge of at least two texts’, one of which may be the formative assignment text.
  • a summer term exam – instruction, ‘answer on two texts, avoiding the texts used for the assessed essay.

 

Not only did this model create significant question-setting work, administrative time, frustration, and paper, but it also inadvertently facilitated inconsistent attendance. Students disappeared from classes in Weeks 9 and 10 as assessed essay deadlines approached, or as they calculated that they only ‘needed’ 4 texts for assessment; when those has been selected and stored under their belts, students disappeared. Tougher material was avoided altogether because the assessment pattern meant that it did not have to engaged with. The old system also caused essay-writing panic towards the end of the module, then exams-related stress, and both triggered the inevitable chain of ECF requests.

 

None of this was conducive to consistent, productive learning and to strong attainment. In addition, the old assessment system rewarded the best writers who were able to gloss ‘shallow’ knowledge effectively: these tended to be students from more traditional educational backgrounds so the assessment model was not heeding inclusivity guidelines because it only ‘recognised’ and rewarded one type of attainment and engagement.

 

Implementation

 

Increasingly dissatisfied with the assessment model, but remaining committed to the teaching and learning aims of the module, I switched to a Blackboard Learning Journal because the pedagogic principles could, I felt, be best achieved (perhaps could only be achieved) using technology.

 

The instruction given to students about the function of the Learning Journal is as follows:

‘The use of a Learning Journal as part of the assessment

on this module is designed to encourage and reward

consistent attention throughout the course, development

in your understanding, and thoughtful reflection on your

own learning. It should support you to identify and seek

solutions to any problems you encounter in your studies.

It also requires you to organise your time carefully in order

to make regular submissions, which is a vital skill in the

world of work.’

 

This instruction emphasizes ‘understanding’ (‘Mastery of the Discipline’ in the Curriculum Framework), self-motivated problem solving, and time-management (‘Graduate Skills’ in the Curriculum Framework). From a pedagogic point of view, ‘thoughtful reflection’ is being implicitly framed within the structure of continuous engagement, and this itself is understood within the language of ‘encouragement’ and ‘reward’.

 

The online Learning Journal requires students to submit 500 words every week, reflecting on the week’s teaching and textual material; after 5 weeks, two entries from the online Journal are assessed and feedback is given (this is the formative stage – no essay questions are necessary). The 10-week Journal concludes in a retrospective entry in Week 11 and there is an assessed essay due for submission 4 weeks later. There is no longer a summer term exam. The Journal is marked online and the mark for the journal is generated by consistent completion of every entry and by the quality of entry 10 plus 4 other entries selected by each student.

 

Students know that if they miss lectures and seminars they will struggle to complete the Journal so attendance is greatly enhanced: an average module attendance rate of 72% (2016-17) has leapt to 86% (2017-18) since Journal assessment has been implemented. The high level of attendance allowed me to deliver the teaching that I know works most effectively on this module because I can rely on various connections between ideas being understood. Further, because of attendance, students are in a far stronger position when they prepare to write their assessed essays so their anxiety is much reduced and they are able to submit their best work. It was notable that no ECFs were requested for extensions on this module in 2017-18 (18 students were enrolled) where 3 were submitted the previous year as the week 11 assessed essay deadline loomed into view.

There is no exam so my marking is reduced and a redundant element of assessment is removed.

 

Impact

 

The Learning Journal initially produced some anxiety amongst students because DEL does not use Learning Journals at Part 2 so this was the first time these students were managing them. At least 5 minutes at the beginning of several seminars had to be reserved for providing students with repeated information and reassuring them that, even though the different format and requirements of the Learning Journal felt unfamiliar and even ‘wrong’, they were following the remit correctly.

 

The Learning Journal information was placed on Bb but most of the 18-strong group did not read the materials on this site; this revealed our students’ resistance to consulting Bb. DEL students seem only to recognise information when it is presented in hard copy, so I had to declared surrender and circulated the Learning Journal Guidelines to students in this form.

 

The majority of students managed to submit weekly work without difficulty and on time. Some students were worried that the Learning Journal format did not seem to adequately prepare them for the more formal writing of the assessed essay. However, by Week 8, the majority of the students expressed their growing engagement with their Journals and, through them, with the module. I also found it interesting that students were more able than usual to forge connections between texts and ideas and I wondered whether this was because the weekly Journal entries cemented the reading and seminar discussions more securely.

 

As for the feedback sheets, this module was not scheduled for assessment in 2017-18. To gather informal feedback, I asked some of the students in the group to write down (anonymously) how they rated speed of feedback: ‘5’ was registered by every student who responded. I have no idea why precisely the same time period would be viewed as ‘3’ or ‘4’ when hard copy was used and as ‘5’ when electronic feedback was used, but the implications for student satisfaction scores are clear.

 

Connecting with the SLL ‘Diversifying Assessments’ project, it is clear that Learning Journals are an increasingly popular method of assessment in DEL. The results of a 2017 Survey Monkey poll in DEL (June 2017 – see http://blogs.reading.ac.uk/t-and-l-exchange/connecting-with-the-curriculum-framework-using-focus-groups-to-diversify-assessment-part-2/) suggest that Student Focus Groups had correctly identified that this form of assessment was capable of challenging the traditional essay in terms of student choice:

 

Reflections

  • In the example of EN3VW (‘Virginia Woolf and Bloomsbury’), technology has allowed me to employ a pedagogic model that was always perfectly suited to the module but that was not always enabling success because students’ engagement was a desired outcome rather than a clear requirement. With the Learning Journal, the pedagogy underpinning the module works effectively for the first time.

 

  • It is clear, however, that students require a great deal of guidance when they initially use a Learning Journal, and colleagues need to be aware that increasing a student’s freedom to write in less structured forms also increases their anxiety. Time has to be reserved for writing advice and this can dent seminar time. The time investment is, however, worth it because the quality of work presented in the Journals was of a very high standard.

RESILIENCE: THE ROUTE TO SUCCESS by Dr Madeleine Davies

A Collaborative Initiative between Dr Madeleine Davies (Department of English Literature), Dr Ute Wolfel (Department of Modern European Languages), Dr Tony Capstick (Department of English Language and Linguistics) and Dr Alicia Pena Bizama (Counselling and Wellbeing)

PROJECT OUTLINE

In December 2016 the three Senior Tutors of the School of Language and Literature (SLL) met to discuss the growing problem of student wellbeing following a steep rise in ECF submissions for MH difficulties. We were concerned not only for the wellbeing of our students but also for their academic development and success, and for their ability to manage their professional futures. We decided that there was more that we could do to prevent student distress, build resilience, and thus support effective teaching and learning so we decided to develop and deliver two-hour, interactive Masterclass titled, ‘Resilience: The Route to Success’.

We consulted Dr Alicia Pena Bizama (Head of Counselling) to help us design a programme that would respond to the specific problems that we had noted in our SLL students. We scheduled three lengthy planning meetings, pooling our ideas and our knowledge: Dr Pena Bizama brought her extensive research into Psychology, and the three Senior Tutors brought knowledge of their individual cohorts and years of experience managing student problems. Together, we designed and produced promotional posters, disseminated the plans to all SLL colleagues, and advertised the Masterclass on Blackboard and Me@Reading. We also sent individual emails to SLL students with a link to a Doodle Poll through which students could reserve a place at the event. This would have been unmanageable for a single colleague in the middle of a busy term, but we spread the load and the materials we produced benefited greatly from the time and input of four colleagues with different research backgrounds and pedagogic expertise.

The planning group decided to intersect with associated key SLL and UofR initiatives: attendance at the Masterclass counted as credit for the Professional Track Programme (SLL) and as credit for the “Life Skills’ initiative, and it connected with the University’s emphasis on student resilience and employability.

DESIGN OF THE PROGRAMME

The Masterclass was delivered in Week 5 of the Spring Term. The 2-hour session focused on building students’ confidence in their ability to manage stress and anxiety and on equipping our students with techniques that could enhance their learning potential. The design of the session was as follows:

Dealing with Academic Pressure:

Introduction – Dr Madeleine Davies: outlining the aims of the Masterclass: managing academic pressure and facilitating success; the connection between academic and professional resilience; the roots of ‘performance anxiety’; redefining (but not denying) ‘stress’ and ‘pressure’.

Dr Pena-Bizama’s presentation (including the difference between MH problems and routine anxiety)

Feedback and How to Use it Constructively:

Introduction – Dr Tony Capstick: how feedback is often interpreted as a personal attack; what its intentions are; how it is a positive learning tool; connection with the professional workplace; how to USE feedback.

Dr Pena-Bizama’s presentation

Motivation, Perfectionism and Procrastination

Introduction – Dr Ute Wolfel: motivation – reminding students of the questions, ‘What do I want to learn? What do I enjoy about these topics?; how perfectionism produces procrastination and how both can be overcome.

Dr Pena-Bizama’s presentation

Discussion period

The students were divided into groups and asked to discuss the following:

(a) What triggers/generates their anxiety most?

(b) How can the words ‘stress’ and ‘anxiety’ be reframed?

(c) What tactics and habits can help to restore perspective.

(d) How can feedback be removed from the sense of personal attack and be redefined as a constructive learning tool?

Feedback period

The students fed back the ideas generated by their groups; many students mentioned that talking about their worries to others who shared precisely the same concerns was of great help. Students also very usefully identified the source of their anxiety and others suggested ways of tackling it. The discussion was lively, collaborative and fully supportive: it was a credit to our students.

Dr Madeleine Davies concluded the session, pointing the students towards material and University support structures that could help them in the development of productive habits and attitudes. The second, exams-focused, Masterclass was announced and written feedback on the session was collected.

STUDENT PARTICIPATION AND FEEDBACK

The Masterclass was delivered on Wednesday 8th February 2017 and 45 students attended. There was a high level of interaction throughout and student feedback was overwhelmingly positive. The form that we designed asked for feedback in the following areas:

Did you find the content reflected your concerns?

15 x ‘5’ (extremely well matched); 27 x 4 (well matched); 3 x 3 (neutral)

Do you think you will find it easier to manage academic pressure (particularly assessments) after the Master Class?

32 x ‘Yes’; 10 x ‘Maybe’; 3 x ‘No’.

Do you feel that you are better equipped to develop a more positive attitude to feedback and study following the Masterclass?

25 x ‘Yes’; 18 x ‘Maybe’; 2 x ‘No’.

Most of the forms added a comment: examples include, ‘Thank you SO MUCH’; ‘I feel much better’; ‘It’s good to know I’m not alone – loads of people here feeling the way I do’, ‘I think I can do this now!’; ‘I liked that the teachers were really honest about feeling stressed too and told us how they cope with it’; ‘Fantastic practical help – it’s what I needed’.

MOVING FORWARDS

The success of the collaboration in delivering the Resilience Masterclass initiative will be sustained going forwards. Prior to the exams period (May 2017) the same group of colleagues will collaborate on an ‘Exams Masterclass’ and in the 2017-18 session we will run three ‘Resilience’ and ‘Exams’ Master Classes in the Autumn, Spring and Summer Terms so that we can intervene early and prevent serious cases of anxiety arising (this will benefit retention). We are committed to working together as a team comprised of diverse skills to support our students in developing mental resilience to underpin academic achievement and to help them to embed the attitudes, habits and techniques that form the route to learning and professional success.