Wikipedia:Wikipedia Signpost/2005-01-24/Performance improvements

Performance improvements

Developers work on servers, speed picks up again

Following several weeks of poor performance, the Wikipedia website gradually returned to reasonably normal performance last week. After various efforts to diagnose problems and test solutions, MediaWiki developers managed to implement several changes that significantly improved the situation.

Problems continue

Early in the week and continuing over from last week, complaints continued to come in that Wikipedia was practically unusable, whether for editing or browsing. Particularly for peak periods of activity in North America, virtually any page load would take up to a minute, or more often it would simply time out altogether.

At one point, the OpenFacts project, which hosts an off-site page for Wikipedia status reports, was also not responding. The error messages being generated there resembled those on Wikipedia when a database query has failed. Since the status page on OpenFacts is linked on many of the error messages provided by Wikipedia's servers, it seems somewhat plausible to suggest that OpenFacts may have been experiencing a Slashdot effect of incoming traffic from frustrated Wikipedians (crashdotting, anyone?).

Some solutions implemented

On Wednesday, developer Jamesday reported that an increase in the capacity of Wikimedia's memcached system, along with some other improvements to it, was helping to improve page load times for logged-in users. In addition, he said this would reduce problems with servers that stopped responding when visited by web crawlers.

As the situation began to improve, Tim Starling took the time Thursday to publicly thank the other developers on the wikitech mailing list for their work. In recognition of the amount of work put in, Starling said, "I've lost count of how many problems we've identified and fixed just over the last few weeks." He commented that in spite of the complaints, most users are appreciative of the work being done, but, "They don't know what our achievements have been and they don't know the challenges we face."

In other changes, new load balancing software set up by developer Kate Turner helped distribute the work more effectively between the servers.

After Wednesday, editors continued to report sporadic problems, but the reports came in far less frequently than before. The remaining complaints seemed to focus on being able to save edits. In an update on Saturday, Jamesday indicated that the developer team had done additional work to increase the speed of saves, which should address those problems.

Since Wikipedia traffic continues to grow, and any signs of this growth leveling off have generally been due to performance limitations, the potential for future problems remains unless the server system can continue to accommodate the growth. Comments from developers and Wikimedia Foundation officials continued to emphasize that more servers will be needed. Since the Foundation still relies heavily on donations, the upcoming February fundraiser is an important element in keeping up with the demand for Wikipedia content.