Succeed’s performance in the mornings this week has been slow. On Tuesday we did some diagnostic work to identify where the bottlenecks were amongst the servers that form Succeed (load-balancers, application servers and database server). We identified the application servers as the part of the system that were struggling most — they both had very high processor utilisation figures. This morning (Wednesday, October 1st) we increased the CPU cores on both application servers from 8 to 12 and hopefully this will improve the performance of Succeed. This in itself generated an issue on Wednesday as one server grabbed all the traffic and then couldn’t handle that level of traffic and started behaving very strangely. This was resolved late in the morning and the service should be a quicker in the mornings and for the rest of the day from now on.
The reason that Succeed has been struggling is because of a big increase in the usage of the system during the day. The peak usage has increased by over 40% from last year — a considerable and unexpected increase. We suspect this is related to the new wireless infrastructure and the ease of connection that it offers.