Saturday, September 6, 2014

Performance Improvements and High Roller Clothing Revealed

1.17.1: Performance Improvements

For the past few months we've been running tests, adding logging, analysing data, going over client and server logs and reviewing network latency reports, all in an attempt to find out what the main causes of our performance bottlenecks are. Frustratingly, the answer is not entirely straight forward (we can't just blame Howard the server hamster, for example) as there are a range of potential issues in various areas of the game that could be causing problems. There are, however, a few systems that are primary candidates for potential performance hiccups. So - with 1.17.1, we've made changes to some of these systems that we believe should make a noticeable improvement to the game.

One of the primary candidates to optimize has been the garbage collection system - this has nothing to do with the Green Planet trucks and everything to do with how the game frees up memory by releasing objects no longer required by the game.

To give you a more technical overview of what we've done here, we're going to hand you over to the lead engineer who has been spearheading the development effort to improve performance, and has written an overview of the problems he's encountered and how he's gone about addressing them.

Identifying Lag Spikes

We've been looking again at server performance in 1.17.1 and have been analysing some of the spikes reported in server log files.

A server ticks (updates) 25 times a second which gives it 0.04 seconds (40 milliseconds) to update the game world. Every tick we measure the time taken and if this is over a threshold then we have a spike (slow update) and we log additional information about the causes of the spike.

Whether a spike affects your experience of playing the game depends on many different factors. Some will be noticed, others won't be, but the less spikes we have, the "cleaner" the server is and the more predictable its performance will be.

The Two Primary Candidates

A game server needs to do different amounts of work at different times, it does standard things all the time; for example, moving characters, simulating physics, and sending network traffic.

The server will often have to do extra work, however, when a character enters a district, and during garbage collection. I've been looking at these two cases.

Reducing Impact of Character Entry

When a character enters a district, the district server needs to grab information from the database and then initialise itself with this data. After observing that this involved a lot of time reading files from disk (which is a slow process because hard disks are often the slowest part of a computer system) I've been able to reduce the impact on the server of reading these files.

Optimizing Garbage Collection

Garbage collection is the process which tidies up unused objects in memory. It currently runs every minute, at 40 seconds past the minute, and runs in two phases:

The first phase is to identify which objects are no longer required. This needs to be completed in one step, otherwise memory could change and its analysis will be invalidated.

The second phase is to tidy up the objects which are no longer required. This was taking longer than was desirable.

For some time now, we've been adding extra logging to provide more information about the process of tidying up objects. From this logging I was able to identify which sorts of objects take the most time to tidy up. In fact - one type of object was taking up the majority of time. I've been able to rework some code so these objects do not always need to be created in the first place.

Improving server performance is an ongoing process, and there are still improvements we can make. We'll be continuing to work on identifying the points on the server where spikes occur and figuring out new ways to smooth these out.

High Roller: New Gear

Welcome to the first teaser of the High Roller Pack! In this installment, we'd like to showcase the new clothing items that come in the pack. As you can see from these pictures, the clothes convey an element of professional deadliness; a mixture of style and function.

First of all, we have some new pieces of wearable equipment that will help players intimidate their enemies. Enforcers will gain access to a vest with a multitude of spaces to store shotgun shells, while Criminals will be able to adorn themselves with bandoliers. Female characters will gain access to some very fashionable threads, including an underbust laced corset, fashionable shorts, skinny glossy pants and some high heeled ankle boots. Male characters will also be able to equip some stylish new attire in the form of a bakerboy hat, a waistcoat/tie & shirt combo, patched fashion jeans and a pair of very chic ankle boots.

Looking good has always been a priority for the residents of San Paro, and the High Roller Pack will give fashion-conscious players the chance to create trendsetting new outfits for their characters.

Join us tomorrow, when we'll be revealing the new shotgun and talking about the raft of balance changes we've been making to shotguns in general!

-The Reloaded Team