Patch 88 release successful
Last night PST / morning EU time we released patch 88 which performed a lot of minor tweaks to the game. The patch launch went well, and we saw quite an uptick of players after the patch. In fact - we ended up setting a record in EU today with tens of thousands of players logging in to Open Beta simultaneously from Europe alone. The game has been growing at a rate of 50% per day globally for the past 5 days, though we expect that pace of growth to taper off rather soon.
Packet Loss on Europe 1 & 2 networking equipment last night
That giant Open Beta turnout wave caused some unexpected network behavior during EU peak times yesterday. The servers were unaffected (since they scale horizontally and new districts will just start firing up), but we did run in to an unexpected problem with our main network edge devices that caused sudden severe packet loss conditions in Europe, even though the devices were in theory running at only 10% of capacity.
The result was that between 16:00 and 22:00 CEST on the networking gear connected to Europe-1 and 2 (ie Patriot and Obeya) we saw several sporadic but big spikes of up to 40% packet loss during those hours.
Inside the game those packet loss episodes manifested themselves as severe rubber-banding, even though ping time and server load would actually remain very low (however, you would be able to see the packet loss conditions if you happened to use the test ping IP we provided on the Forums).
We are working closely with the manufacturer of the networking gear overnight to see if they can resolve the issue before tomorrow's peak time. They are potentially able to provide a fix and if it can be confirmed, I will post the information here as an update to this post.
US-East and US-West did not experience the same issue, partly because those locations have of course been split between two different physical datacenters and in effect have double the amount of networking gear infront of the game servers at each location.
Again, we hope this issue will be resolved very swiftly, but keep in mind this type of testing is what Open Beta is all about - to test the crap out of all the equipment at full throttle. So, while we apologize for the un-fun experience in Europe during that time window, we hope people continue logging in and playing since that will help us troubleshoot the problem in much more detail tomorrow night.
New Items on Sale Tomorrow
This week we are going to start posting a lot of cool new items in the store, and we are going to start introducing some permanent items as well on a rolling schedule. We will announce the new items on Facebook tomorrow so stay tuned (or simply log in to the Armas Marketplace when we launch the items).
Renewal of Premiums during Beta - only $7.99 USD (£4.99 GBP / €5.99 EUR) !
So in the next two days our first batch of Premium Players are going to get reminder emails that their first 30 days are almost up. As we have said before - during the Beta period the cost of premium is only $9.99 for all of the cool premium benefits that let you vastly accelerate your progression and item drops, and gives you complete artistic freedom when you design your characters plus a whole range of other benefits including a 20% item purchase discount.
Thanks to this discount when Premium Players re-up (adds 30 additional premium days) before the end of his/her premium period, those premium extensions get an automatic 20% off (!) off the Beta $9.99 price.
The result is that premium renewals are only $7.99 in the US, £4.99 in the UK and €5.99 in Europe. And - any benefits that you accrue, including standing, achievements, customizations, permanent items or permanent weapons you buy during Open Beta carry over to the live production launch of the game as well.
And finally - a big thanks for Open Beta testing our game!
And a final word to everyone trying the game out; a really big THANKS for Open Beta testing our game (and liking us on Facebook!).
While we will try to keep the game as near production quality as possible at all times during OB, there will of course be some hiccups along the way, and that's expected at this stage of Beta. But we are working around the clock (litterally) to ensure the best possible "near production" experience along the way. In the next days and weeks we will continue performing network updates, traffic improvements, game tweaks etc., while also working on the major upcoming changes that we expect to launch before the big formal release of the game!