As I blogged previously, Pat and I spent some time going through the feedback I managed to collect at the seven e-Portfolio Templates trials I ran with Louise earlier this term. Please bear in mind that this is a qualitative summary — due to time constraints, we felt it would be more beneficial to (at least initially) get a general impression of the responses, taking notes of overall response and noting any exceptions/outliers or comments of particular interest. I have put some phrases in quotation marks but will try to double-check during the week that these actually correspond to what was written on the forms.
Difficulty with creation of the e-portfolio
The evaluation form asked “How easy/difficult did you find it to create your e-portfolio?”
One respondent said the creation was “difficult” but didn’t clarify why (though to be fair we didn’t ask; perhaps the question could have been phrased better). Some respondents gave much fuller answers. There were a couple of comments from different groups about the difficulty of getting into Blackboard but of course this had nothing to do with the specific operation of the widget (except that by necessity it had to be located in Blackboard). One respondent commented that it clarified “awful” Blackboard interface decisions and another said that without the tool, the process would have been much more troublesome.
Overall, the feedback said that the creation of the e-portfolio was easy. Because two of us were present for most of the duration of the workshops (me and Louise) the creation may have been made easier than it would have been without us — we got one comment on this to this effect, but as an observational point, the few latecomers we had seemed to show little difficulty in catching up, once they had the widget on the screen.
Enjoyment of creating the portfolio and any issues
The evaluation form asked “What did you like about creating the portfolio (if anything) and what issues did you have (if any)?”
One respondent in the first group said they enjoyed the self-documenting process. Another in the same group said one issue was that the prompts were too fast. Although we initially thought this was an odd comment (the prompts only change when the user has completed a step) we actually found this to be a genuine bug. For example, the prompt that advises the user what they might enter as the title of their portfolio disappears as soon as they start typing because the widget is only checking that the title field has a minimum of one character. Others claimed that they encountered bugs with template pages being duplicated though these may have been down to user error.
“Simplicity” and “clarity” were other words that came up, both apparently meant in a non-perjorative sense! One respondent in the final group said they liked the fact that it was “open-ended” — which is true; the user can move around the tool at any point and also edit the templates in any way they see fit. The widget merely provides the necessary components and guidance on how to use them.
There were apparently problems with the text editor in Blackboard. One comment suggested that a built-in spelling or grammar checker would make the tool more useful. Interestingly, it was noted that not much digital literacy was required to use the tool. A few responses noted that the templates were easy to use and another said that the widget made the templates less daunting. Although from these comments there may have been some confusion as to what the respondents were evaluating, these responses were always positive.
Usefulness of wizard
The evaluation form asked “How useful was the wizard (the prompts on the left) in guiding you through the process?”
Overall, the prompts (or “wizard”) were described as very useful. Sometimes, according to the respondents, they displayed the wrong information but this was dealt with through a quick refresh of the page. Only one comment described the prompts as “useless” and “annoying”. Another said, they were more helpful than the respondent expected them to be when the system was described at the beginning of the workshop. Another still said they didn’t notice the prompts “all the time” but that they were useful when help was needed.
Broader applicability/limitations of tool
The evaluation form asked “To what extent do you think this approach would be useful for other tools that you have used?”
The response to this question was interesting in that it was most revealing about how some students saw the tool in relation to Blackboard. Some respondents noted that the widget (when we asked “this approach” we meant “the wizard” so clearly the question could have been clearer here) would be useful in Blackboard — the implication being that, in creating their portfolios, they thought they had left Blackboard. This might be because, though launched from Blackboard, the widget actually opens in a new window. However, other related comments indicated that the change in interface (i.e. to one that was responsive and directly helpful) suggested students thought they had actually left the VLE to create their portfolios and subsequently thought this approach could be used to help with areas of Blackboard (even though, with one particular area at least, that is what the widget does).
Some students clearly knew they were still in Blackboard however and made comments regarding this — one said the “Bb Edit and Delete” buttons were too close for example. This did seem like a fair point, especially as the “Delete” button in Blackboard makes a page irretrievable!
For more general usage, Pat and I noticed in the responses collected a certain “lazy factor” (and we don’t mean that perjoratively!) — one comment said “less work is good”. They suggested the widget might be useful for “complicated systems” or for “CVs”. There were specific mentions of the help wizard being useful for databases and RISIS (Reading’s student information system, based on Tribal SITS). One more generous comment said it “would make everything better”.
Initial Conclusions and Reflection
Overall, the feedback we received seemed largely positive, with a few exceptions and some issues worth noting. This is surprising. While I expected that the tool would work and make things quicker, I did not expect that this would be recognised by the students. One of the problems of these pilots is that we were running them with a cohort who were new to the tool — that is, they had not experienced what the e-portfolio tool was like previously. As a result, I fully expected the students to have more complaints seeing as they would not recognise the improvements we had made.
What will now be interesting is to see what students make of their continued use of it, to see if any of their initial enthusiasm has faded, and to note any further issues that have emerged since the workshops. I also hope to show the widget to students who have seen the un-widgeted Blackboard e-portfolio tool (in the last two cohorts) and gather some feedback from them. This I’m going to try to arrange before Christmas if I can. This on top of our evaluation of the Feedback widget, to happen soon.