Online Presentation for JISC Activity Week

Last Friday morning, I took part in a presentation on the DEVELOP Project for JISC’s Online Activity week. It was enjoyable if slightly bewildering experience, given that it was my first experience of doing an online presentation solo. At times, it felt as though I was just sitting in a dark room talking to myself!

The other key presenters were Stephen Vickers and Judith Pickering — I came away from both their presentations wanting to find out more about their projects and what we might learn from their very different approaches to the e-portfolio question. All three presentations were recorded and can be viewed here.

I tried to give people an idea of the widgets we were developing by using screenshots in my presentation. However, I supplemented this by providing links out to silent (so attendees could continue to hear me droning on!) videos of the widgets in action. You should still be able to see them using the links below:

I don’t know if this approach worked — it’s hard to know if it was useful or merely a distraction from my continuation of the presentation with the screenshots. I had also hoped to let people actually have a go themselves with an open version of the Feedback widget but, due to problems we’d been having with our Blackboard server the previous evening, decided against it. However, I was able to give an overview of initial user feedback regarding the widgets and answer a few questions that came in afterwards.

I will post a brief summary of our initial feedback shortly.

Posted in Dissemination, e-Portfolio Feedback, e-Portfolio Templates, Uncategorized | Leave a comment

Setbacks

A lot of the blog posts I’ve been putting up recently have been on the e-portfolio widgets so I thought I should take a few moments to explain what’s happening with the other widgets. Unfortunately, the project has had a number of setbacks which meant progress hasn’t been as fast as it could have been:

  • Karsten going to teach full-time — This meant Karsten unfortunately had to leave DEVELOP. While Karsten has been around to advise us on certain issues, some of the development which we had been going to do over the summer didn’t happen.
  • Test Server down — Whilst we were in part correct when we predicted that the upgrade of our instance of Blackboard from 8 to 9.1 wouldn’t affect the project, there were related drawbacks. The upgrade itself was successful but the Test Server was down for longer than anticipated, briefly preventing continuation of some of the work we were doing.
  • Changes to annual routine — Again, whilst the upgrade didn’t directly affect the project, there was also a change in the way courses were archived this year. This decision was made at a higher level but unfortunately had an impact on the number of issues and queries that the e-Learning Team had to deal with, thus eating into project time.
  • Blackboard Administrator absence — Unfortunately our Blackboard Administrator in IT Services was away for some time due to illness. She’s better now and back at work but we really missed her while she was gone as it meant, not least, that we couldn’t do half as much as we would have otherwise!

I wish I could say that there were lessons learned from this for the Project Team but it really feels like these were circumstances largely beyond our control. Despite all these setbacks, we still had a really successful implementation of the e-Portfolio Templates widget and are expecting similar success with the Feedback widget soon.

So, apart from those:

  • The Video Widget is up and running but we are still working on advice for how this can be used by other institutions (e.g. the University of Bedfordshire).
  • Meanwhile, just today, we managed to get a prototype of the tagging/recommender widget up on our Test Server, which I’m hoping to get onto the Live Server before long so I can show it to our pilots soon.
  • And we do have the basics of a Content widget too but I think piloting of this will be limited, if there is any at all. I need to review what we have and then see if I can gather feedback from those who expressed an interest in using it.

The e-Portfolio Export widget is something I’m very much interested in exploring, even it means that I put in time beyond the project; Pat and I drew up quite a few ideas for how to move forward with this but as December nears it remains to be seen how much we can implement before next term. Pat and I both hope to post more about that in the near future.

Posted in Uncategorized | Leave a comment

Hallowe’en Whiteboard Mess

Pat and I are now in the meeting room, addressing the problem mentioned in the last post and more. We needed to wipe the board so we took a photo of the mess we created last time we were here (31st Oct) for reference and posterity…

The whiteboard mess we created a couple of weeks ago...

Posted in Uncategorized | Leave a comment

Making that “insurmountable” problem crystal clear…

Too many comments... we have been dealing with all sorts of recursion glitches lately but these have been resolved, leaving more fundamental problems.

Pat and I had another quick meeting last week to discuss how we take the development of the Feedback widget further in the short time we have left for development. In my last post, I mentioned that there might be fundamental problem with the widget that proves insurmountable. In this post, I’m going to lay out what the problem is and explain some of the work-arounds Pat and I thought might be appropriate.

 

The problem is really two-fold and has to do with the selection of the text a user makes when leaving a comment on a portfolio. Suppose that on one an imaginary user’s portfolio pages we find a note on a meeting the user attended:

At this personal tutor meeting, it was agreed that I would aim to reflect on why I received the grades I did and how I can work to improve and get one grade higher for my next assignment. I will work to improve all my grades in this way.

Then suppose that an imaginary tutor wants to use the widget to select the phrase “work to improve” and leave a comment on it. He or she might, for example, leave a comment asking what the student means by this phrase and requesting more details. Here are the two facets of the problem with the widget at the moment:

  1. As we can see the phrase “work to improve” appears twice in the text. How does our widget know, when we come back to this page, which one it should target? Or should the widget leave the tutor’s comment by both/all instances of the phrase?
  2. What happens when the user alters the page? This is especially likely given tutors are likely to use the widget suggest changes or improvements. Here, for example, the tutor’s comment could easily be interpreted as a request to remove the phrase “work to improve” and replace it with more specific information. If the student then follows this interpretation of the feedback, the widget will no longer be able attach the comment to the original phrase since it will no longer exist.

 

And when we combine those two facets we come to the “gem” of the problem:

  • The two facets should make the problem "crystal clear" (Credit: NASA / Marshall; click on image for web-source.)

    Worse still, if a page is altered by the e-portfolio, comments that were attached to phrases that are subsequently removed, might be attached to other instances of the phrase. The tutor may have had no qualms with these instances of the phrase, leaving their feedback (now erroneously attached to another part of the page) open to serious misinterpretation!

 

 

Finding some hooks will be the next problem... (Click on image to see attributions.)

In short, we could say that the problem is about “hooks” — we need specific hooks on the page that we can hang comments from. The widget cannot change the page in the e-portfolio; it can only hang things from it. So we are dependent on whatever is already there and whatever is already there may not be clearly marked (facet 1) and may be subject to change by the e-portfolio’s author (facet 2).

In the next post or two, I hope to be able to present a solution or workaround to this problem.

Posted in Uncategorized | Leave a comment

Collation of feedback from Legal Skills workshops

Pat and I collated feedback from the Legal Skills workshop today. The handwriting pictured is Pat's. Typed versions forthcoming...

Posted in e-Portfolio Templates, Pilots and Implementation | Leave a comment

Summary of recent work

Summary of activity since the last Steering Group meeting:

Clarification and further ongoing work on evaluation

The evaluation of software created using the Rapid Prototyping development method is necessarily incremental and the acceptance of the tools themselves is a significant aspect of meeting success criteria. User testing is underway and on schedule, and impact on teaching and learning – as with most impacts – take longer to assess.

Installation and testing of LT1 server

This raised a number of problems which were not completely resolved. The approach taken with the Portfolio template widget has the advantage of not requiring a separate server for widgets.

Continuing development of e-Portfolio widget & customization for Law

The development of this widget allows a platform independent version to be developed, with only a small interface layer, or configuration, for each different LMS. Further work in refactoring the code and making sure that the customisation can be done in a stand-alone file still needs to be completed, but piloting takes priority at this stage.

Upgrade of Blackboard required extra work to make video widget work again

This sort of problem is sadly all too common when service suppliers make changes to features such as input filters. Any form of customisation adds to a growing maintenance overhead whenever a new version of the underlying software in installed. The design of the system means that there are, unfortunately, a number of weak points where different systems may need to be modified whenever another technology is updated.

Portfolio widgets linked to make a usable product from the parts

These two pieces of work represent a major move forward in providing the ePortfolio functionality

Proposal for presentation at Online Educa accepted

Focussing on  UoR’s use and development of ePortfolio approaches, specifically the work done on the Develop project

Portfolio feedback widget

Still actively under development, work on this widget highlights some interesting semiotic elements of feedback and portfolios (which will be covered in a later blog post).  There are also some technical issues which still need to be resolved, but the widget as it is now will be given a good work out by a group of IT students in the near future.

Portfolio template piloting

The pilots (still awaiting full analysis) appear to show a marked benefit from the availability of the template wizard Guy has created.  Initial analysis of the feedback from the first 8 pilot sessions shows students found the template extremely useful, and allowed them extra time to reflect on the actual purpose of using portfolios, rather than focusing on the technical issues of how to create and update them.

Portfolio export widget

Various ideas are being explored, and the most promising candidate is based on the same framework as the feedback and templating widgets. The main issue here is that the export widget needs to have some semantic knowledge about the content of the portfolio. This can be achieved by having a template which the learner has to fill in, which can work in some circumstances, but lacks flexibility and does not allow for easy customisation. It can also be achieved by using a wizard coupled with a mark-up tool (such as the feedback widget) to annotate information in a portfolio whether it is fixed format or free-text . The latter would seem to be the better approach, and will be explored further when the feedback widget is complete.

Posted in Dissemination, e-Portfolio Feedback, e-Portfolio Templates, Evaluation, Pilots and Implementation, Project Management | Leave a comment

A day of “extreme programming” on the Feedback Widget

On Monday this week, Pat and I booked out a meeting room and set up our laptops side-by-side so that we could begin making progress on the Portfolios Feedback widget. I’ve been conscious that development of this widget has slipped since Karsten left and since the beginning of term, so it was good to set aside a solid block of time to focus on it.

In this post, I’m going to try and explain in the simplest possible terms some of what we’ve managed to achieve. Although I have included some examples of code in the post (JavaScript), hopefully the explanation of what the code does will be clear enough for non-coders to be able to interpret it and perhaps even learn something about JavaScript syntax.

Let’s start by restating the purpose of this development: The Feedback widget should allow a user to look at a e-portfolio that they have access to, highlight nearly any part of a page within it and leave a comment on that part. It’s like an annotation tool for leaving targetted feedback. The owner of the e-portfolio or anyone else seeing it should then be able to see the comments left by other users if they have access to the widget too. As a result of this aim, there are some issues around privacy and confidentiality but these will have to wait for another post.

I had made some progress with the Feedback widget previously by creating a frameset page that the user could launch from the e-Portfolio Templates widget (explained here and here). To summarise, the frameset contains three frames, arranged as rows:

  • Frame 1: toolbar frame (where we hope to put buttons/controls)
  • Frame 2: e-portfolio frame (where the user’s e-portfolio appears)
  • Frame 3: hidden frame (this is where we load the place in which we want to store our comments — in this case, the comments page within the user’s e-portfolio)

This is roughly what it looks like from our development point of view…

A crude rendering of the Portfolio Feedback widget (click to enlarge)

… but all the user should see in the final interface is their e-portfolio with a simple toolbar above it.

By its definition, a portfolio (e- or otherwise) can consist of one or more pages. Arguably, a one-page portfolio isn’t much of a portfolio at all, so let’s say that a portfolio should consist of more than one page, even though it can consist of just one.

The Feedback widget we’re developing works by attaching an “event listener” to the page the user is looking at within their e-portfolio. An event listener is a property one can use with elements on a webpage so that a piece of code is executed when that event occurs. In this case, the event listener is “onmouseup” — so whenever the user lifts their mouse button on the portfolio page, the widget looks at what the user has selected. The “onmouseup” event seems like the best one to listen for since it means that the user can push their mouse button down, drag the cursor over a piece of text in order to select it (or highlight it), and then when they let go over the mouse button the widget checks to see what their selection is.

Now consider that the user might, at any point, change the page they are looking at within their portfolio. It’s not enough for us to attach the “onmouseup” event to the page the portfolio happens to open on. So one of the first things that Pat and I looked at was how to attach this event to whichever page the user decides to load.

Normally the “onload” event listener would be ideal for dealing with this. The “onload” event can be used in conjunction with a window or frame to trigger a piece of code whenever a page or document within that window or frame is fully loaded — and it therefore triggers every time a new page is loaded. This would be ideal for attaching our “onmouseup” event listener to the page/document since we would know when the page had changed and then we could add the onmouseup event listener to the new page.

This is just an example of that might work:

window.frames[“portfolio_frame”].window.frames[“Portfolio_RightFrame”].onload=
“document.body.onmouseup=’thisFeedbackSession.getActiveText()'”;

Unfortunately, for us this was harder than it sounds for a couple of related reasons:

  1. The portfolio itself consists of two frames, but arranged in columns. On the left, you have the menu and on the right the portfolio page. But these two frames are written by Blackboard. Unlike the aforementioned three frames that we control, we’re not able to put an inline “onload” event into either.
  2. On top of this, it is not possible in JavaScript, so far as I can tell, to dynamically alter the onload property of a frame. Otherwise, we could simply put some code in to onload so that whenever the user clicks on a different page, the onmouseup event is added.

My initial solution to this problem was to write a piece of code that was timed so that at an interval of every 100ms the address of the page would be checked against the last one noted and if they differed, we would wait until that page had fully loaded and then add our onmouseup event. This is approximately what the code looked like:

this.interCheck=window.setInterval(function() { thisFeedbackSession.checkPage() },100);

this.checkPage=function() {
if (this.portfolioPageFrame.location != this.portfolioPageLocation) {
this.portfolioPageLocation=this.portfolioPageFrame.location;
this.tempInt=window.setInterval(function() { whenPageReady(); },100);
}
}

this.whenPageReady=function() {
if (this.portfolioPageFrame.document.readyState == “complete”) {
this.addMouseUpEventListener();
window.clearInterval(this.tempInt);
}
}

Unfortunately, this caused quite a few problems — we managed to get thousands of errors clocking up in our consoles and after much head-scratching and drawing on the board, we could see that this was no good because we’d set up what looked like two mutually recursive loops. Errors aside, it wasn’t good enough because it only checked to see if a page had changed. What if the user decided to refresh the page they were looking at? The page would reload but there would be no difference in the address so onmouseup would never be added and the highlighting facility would not work.

We now have a much more elegant code that regularly checks (i.e., every 100ms) to see whether the portfolio page’s readystate is complete (i.e., whether the document has fully loaded) regardless of which page it is and then simply checks the document to see if the onmouseup event listener is there or not. If the event listener isn’t attached, then we attach it!

Here’s what it looks like now (much nicer, I think):

this.checkPage=function() {
if (this.portfolioPageFrame.document.readyState == “complete”) {
if (this.portfolioPageFrame.document.body.onmouseup == null) {
this.addMouseUpEventListener();
this.highlightComments();
}
}
}

You might also notice there’s an extra unexplained line in there — namely “this.highlightComments()”. Once Pat and I had solved the problem of the user’s being able to change pages, we were also able to address one of the key features that the widget is supposed to provide — how to show what existing comments were there on the page, comments which need reloading every time the page is.

Although we made quite a bit of progress with this, we still have a fundamental problem with the way the widget works, one that may be insurmountable. But that’s for another post!

Posted in e-Portfolio Feedback, Plugin development | Leave a comment

Presenting for JISC Activity Week

On the 18th November, I will be presenting along with Sheila MacNeill (JISC), Judith Pickering (Open University), and Stephen Vickers (Edinburgh University) on e-portfolios as part of JISC’s online Activity Week. The title is “Enhancing and creating student centred portfolios” and the short description says “Demonstrating a number of portfolio centric integrations and widgets being developed as part of the JISC DVLE Programme.”

The full session proposal which should give you more of an idea of what the presentation will be about (which Sheila finalised after each of Judith and I had written and provided our parts) went as follows:

This synchronous session from the Open University and the University of Reading will demonstrate a number of portfolio centric widgets being developed as part of the JISC DVLE Programme.

Portfolio redevelopment at the Open University has focussed on incorporating some of the enhanced functionality available within Google, e.g., a document repository, facilities for sharing, collaboration and reflection.  The DOULS project (Distributed Open University Learning Systems) has been tasked with delivering integration between the Moodle learning environment and Google.   The presentation by the Open University will focus on these integrations and what it means for the student experience.

Part of the University of Reading’s DEVELOP Project has been to look at e-portfolio provision and use, and to develop three widgets to assist staff and students in the creation and maintenance of e-portfolios. The University uses Blackboard as its VLE and widgets have been designed using HTML/JavaScript to interact, in pilot studies, with Blackboard’s very basic e-portfolio tool. One widget, now fully developed for Blackboard, enables students to build a portfolio with all the pages as specified by their tutors/lecturers. This widget also guides the user through the various steps needed to share and maintain their portfolio. A feedback widget is at this moment being developed to allow tutors to provide feedback on specific parts of the students’ portfolios while an export widget is planned to allow students to download their portfolio in a standards-compliant form.

Steve joined the bill later when the organiser in JISC identified a potential synergy between our proposed presentation and his work. So, there will be yet more covered than what is outlined here. It should be an interesting talk!

Posted in Dissemination, e-Portfolio Export, e-Portfolio Feedback, e-Portfolio Templates, Uncategorized | Leave a comment

DEVELOP at the JISC Online Conference

We are doing a joint session on “Enhancing and creating student centred portfolios in VLEs” with the OU and Edinburgh at the pre-activity week of the JISC online conference, at 11am on November 18th. More information at http://www.jisc.ac.uk/whatwedo/programmes/elearningpedagogy/elpconference11/Activity%20Week.aspx.

Posted in Uncategorized | Leave a comment

Legal Skills workshops — first impressions

I’ve been meaning, for some time, to post about the pilots for the DEVELOP Project but various other deadlines and duties have prevented me from doing so.

On Thursday and Friday of last week, I ran a series of workshops for new students with Louise Hague from the School of Law. The workshops run in the first week of term every year and this year we were able to use the Portfolio Templates widget to improve the experience. The widget was used by almost 200 students and enabled Louise to concentrate more on talking about the purposes of each e-portfolio template than on explaining how to set the e-portfolios up.

I managed to collect completed paper-based feedback forms from everyone who attended but now have the lengthy task of typing up and interpreting the answers. In terms of the running of the workshop though, the widget seemed to work successfully; aside from a few minor glitches, a system crash, and a fire alarm everything went smoothly!

I hope to post more on this and other pilots soon.

Posted in e-Portfolio Templates, Pilots and Implementation | 1 Comment