itsnews Over the last few days there have been a number of problems with the Blackboard service at which manifested itself as very slow log on times and problems navigating through the system.
This has caused problems for a large number of users and for this we apologise. We hope the information below illustrates the problems that we have faced and the fix.
IT Services first received warning notices about the problem on Wednesday 16 November at about 19:00, from our automatic monitoring services. The problems manifested themselves as high CPU load due to the Blackboard Java process and this in turn caused the slowness experienced due to a lack of system resources. These problems were dealt with by IT Services staff remotely and appeared to be fixed by a system restart.
The problems recurred throughout Thursday and continued despite system restarts around lunchtime. During this time IT Services had contacted the vendors who provided advice on some systems settings to change and a patch to apply to the system. These were implemented out of office hours between 1700 and 1800. Once these had been applied the system CPU load was much reduced, generally operating at around 50-60%, which was in fact substantially lower than typical load seen over the previous few weeks.
Today, Friday 18 November, we have been able to investigate what caused this sudden spike in load, and appear to have tracked it down to a sudden increase in the University’s Google-Mini search appliance scanning which started at about 1420 on the Wednesday. This increased the load on Blackboard that appeared to trigger a bug, which the subsequent patch fixed. Scanning of the Blackboard system from the Google-Mini appliance has now been halted and IT Services are continuing to monitor the Blackboard system load. We are continuing to investigate why there was a sudden increase in scanning on Wednesday.
If you continue to have problems with Blackboard, please keep us notified via its-help@reading.ac.uk or by phoning ext 6262