Ready To Move

This weekend I was not certain how much time I would get to play, because the weekend tends to be when we do things. For the most part, I got to hang out and play New World all Saturday because it was raining buckets outside, and neither of us had the desire to leave the house. Sunday however it became rapidly clear just how untenable our current server situation is. Because of just the nature of how our Sundays go, I spent time in queues three different times for a grand total of four and a half hours spent watching this box tick down. Essentially if you cannot log in before noon, you are not going to have what I would call a reasonable experience. Thankfully my machine is capable of running multiple games at the same time so on the longest of these queues… 2.5 hours, I spent time over in Final Fantasy XIV doing roulette.
One thing that I can attest to with a fairly high degree of confidence is that the tool we have been using to estimate how long a queue is… is pretty freaking accurate. The three queues that I entered yesterday were pretty much dead on for what the queue estimates were at the time of me logging in. These are some sample queues from the site at the time of writing this, and I sorted them by the most players in a given queue. This more or less tells the tale of New World right now. Notice that a few servers have seemingly been bumped up to 2250 players as a test, but I am starting to doubt that they plan on scaling these up very much higher. In fact 2000 players seems to be an important number for the design and stability of this game. I provided some server data on Friday when I wrote about the game and since then 131 more servers have been added… only further increasing the fragmentation of the player base. The current data centers look something like this:
  • Frankfurt, Germany – 230 Servers
  • Arlington Virginia, United States – 200 Servers
  • Sydney, Australia – 70 Servers
  • Umatilla Oregon, United States – 64 Servers
  • Sao Paulo, Brazil – 44 Servers
Amazon has made some attempts to flag players who are trying to get around the AFK timeout and force them out of the game. Cities right now are rife with players running in place against obstacles or auto-attacking while standing still via a macro. So while I am happy to see them making strides in keeping this from happening, I also don’t think it is going to really go very far in solving the problem. The core issue is that the day one, day two, and probably even day three servers are way too populated.
Amazon has officially locked many of these overly populated servers so that new characters cannot be rolled on them. However, this action probably came a little too late for it to make a difference either. If you have a fixed server size in mind as they did… maybe stop character creation at 2.5 times that number? On opening day there were servers with queues that were upwards of four times the size of the total number of players allowed on a server. Ultimately I am just not certain that staying where we are is going to be tenable much longer. There has been a discussion that Amazon is trying to rush a solution to allow players to migrate elsewhere, and right now I am thinking that is our next best hope. Once this opens up, find a smaller server and then coordinate a move to that location so that we can recreate the company again on the other side.
There is a lot of really fun group content available in this game. I spent some time over the weekend closing “Rifts” aka corruption breaches and they were great. I also got pulled into a random group with players from various factions and did Amrine Expedition the first dungeon. All of this is pretty awesome content, but also things that would have been so much more enjoyable had I been able to do it with my friends. The core problem with the game right now is actually getting players online. It is making all of the core systems function a little worse as a result. For example, I watched a video from a YouTuber that talked about how hard it is to make the War system work right now because they just can’t reliably get players online at a fixed time in order to meet up and defend a territory.
There are times I am committed and connected to the community on a server. For example, I would refuse to ever move away from Cactuar in Final Fantasy XIV, because that server is just phenomenal. In New World, however… I think pretty much every server is a dumpster fire right now and Minda is not really a place I care about significantly. I am deeply connected to my character and would not want to re-roll elsewhere. I am deeply connected with the people I am playing with… whenever we can actually play together. I am not however connected to any given server, so far as I am concerned our best bet is to move. This is going to still have some pain associated with it because I am in a place where I need those tier 4 crafting machines, but I can deal with being delayed if it means we can all play the game in a reasonable manner.
I think the plan going forward is to wait for the server transfer tool to open up, and see what it does to server populations. Then find a relatively low to medium population server and transplant our little group of players. The players that rerolled on low pop servers are seeing a lot more of these sweet sweet loading screens than those who have stuck by Minda. At level 33, I am too deep to contemplate starting over, so my last hope is for a server transfer. The post Ready To Move appeared first on Tales of the Aggronaut.

AggroChat #361 – Huge Queues Tiny Servers

Featuring: Belghast, Grace, Kodra, Tamrielo, and Thalen
Tonight we are back after taking last week off for Tam’s wedding.  We also record an exceptionally long show because we got into the topic…  and were not watching the clock terribly closely.  So it clocks in at around an hour and forty minutes of talking about Amazon’s New World release and all of the good and bad that comes with that.  While this makes for an interesting show…  it makes for some pretty boring show notes.

Topics Discussed

  • New World Launch
  • Infrastructure Woes
  • Great Game Though
  • Server Transfers
The post AggroChat #361 – Huge Queues Tiny Servers appeared first on Tales of the Aggronaut.

New World Launch Post Mortem

Good Morning Friends. I find myself still here and still playing New World. I did however take a break and play some Final Fantasy XIV… while the New World client was up and idling in the background. Yesterday I logged in over lunch to about a thirty-minute queue, and that was enough anxiety to have me just leave the client open all day long. After yesterday’s blog post, I took to “the twitters” and threaded up some commentary for anyone who might not read my blog. My advice this morning is still pretty much the same as that thread, if you are not already bought into the game then I would highly suggest just waiting to pick it up until the server congestion situation is more reasonable.
The New World team took to their Twitter account and posted this message, trying to put a positive spin on the current situation. The tweet talks about how many servers they created on the fly and how they are working to increase capacity on each server. It also goes on to say that if people just pick any available server now they are working on a way to get folks transferred onto a server with their friends. The problem here is this fundamentally is not how an MMORPG works. You need those friends the most in that early rush as you are attempting to do dungeons and collaborate on getting folks geared. Even if you are not actively playing with other folks, there is something necessary about just knowing that friendly voices are out there waiting on the other end of guild chat. There is something deeply flawed in the design of this game and I will go into detail a bit but first, we need to talk through some numbers. Right now at this very moment, we have the following server capacity:
  • United States East Region – 158 Servers available
  • South America East Region – 35 Servers available
  • Europe Central Region – 195 Servers available
  • Asia/Pacific Southeast Region – 40 Servers available
  • United States West Region – 49 Servers available
Looking through this you notice some strange data points. First off the EU region has significantly more servers than I would have expected at this point based on the sizing data from other MMORPGs. The United States West Coast region seems grossly undersized to what I would normally have expected based on my experience with other games as well. In grand total, you have 477 servers and if my math is correct a maximum capacity of 954,000 players could be playing the game at any time. There is a maximum single server capacity of 2000 players based on the metrics we have available to us, and there are many realms that have more players in the queue than actively playing on the server.
In yet another tweet thread, I surmise that there is something about their design model that is fundamentally easier to scale horizontally than to scale vertically. This 2000 player limit seems to be important, as does the 100 players in a single company. That tells me this game was more or less designed for 20 companies to be duking it out for the resources of Aeternum, which makes me wonder if New World more or less started its design cycle as a slightly larger scale Battle Royale game, and morphed into an MMORPG game as players violently rejected its PVPcentric design. We know without a shadow of a doubt that AWS can scale upwards extremely well, and you can very well keep throwing resources at a problem by adding more processing power, memory, and disk space ad infinitum. So that tells me there is something on a software level that makes it a challenge to scale above 2000 players in a single server.
Let’s talk about that 2000 player number a bit. Once upon a time both Everquest and Dark Age of Camelot showed the current player population of each server on the login screen. Quite frankly this was something they were both proud of because you have to understand… coming from games with a maximum player population of 16 in a single match, having thousands of players was quite the feat. The average peak for those servers generally speaking was in the 2000-3000 player range. As we move forward to the modern era of MMORPGs, I have heard that 6000 players are the point at which the login queue starts to kick in on a World of Warcraft server. Final Fantasy XIV requires a bit more factoring to come up with some sort of estimate. Based on the player knowledge I have… most people do not use the Steam client, and I would guesstimate that at most 1/5rd of the player base connects that way because you have a huge group that plays through their console. There are 68 total servers in FFXIV, and Steam recently reached a peak concurrency with the game of about 67,000 so for sake of making things simple let’s call that 1000 steam players per server which again is going to be a deeply flawed number. If we are taking as an assumption that the number represents a fifth of the player base then my guess would be that would give you around 5000 concurrent players per FFXIV server.
So you might be asking yourself… Bel, why did you go through all of that contorted math that you know is going to be completely wrong? Just trying to paint you a picture of what the modern MMO landscape looks like as far as player population caps, and how 2000 per server seems extremely small based on the demand already existing in mature games. In an immature game, there is going to be even more of a burst of players that needs to be dealt with as folks are currently hungry for any new games coming out regardless of genre. New World also is a special game, because it represents the first completely new “Western” MMORPG in years. Western players have developed a pretty sour taste in their mouth for the microtransaction hellscape that are Korean and Chinese localizations. Based on the recent flurry of activity in Final Fantasy XIV, there should have been signs that the community was primed and ready to latch onto something new and fresh given that the entire World of Warcraft player base seems freshly unmoored.
So once again I fall back on, that there is something specific to their design model that indicates 2000 players as a specific unreachable boundary. Otherwise knowing what I know about AWS, it would have been far simpler to add additional processing power to the servers that were already provisioned than to keep spinning up new nodes. There has to be some reason on the software side that they are not doing this, however, and it makes me question if they are confident that they can scale upwards. Alpha and Beta testing didn’t really give them an idea of what a real-world player demand looks like. We went into those tests just happy to be playing, and not much caring if we were playing with our core group of friends. However, an MMORPG launched is a totally different beast, and players organize into dense clusters as everyone tries to make landfall on the beach that their friends already arrived on. Having more servers doesn’t really do much to dilute the demand placed on those first few servers.
Now we are going to swing back around to why the servers per region numbers look fucked. Amazon shot themselves in the foot with this launch and rolled the game out in a manner as to encourage players to dogpile a limited set of servers. Amazon rolled the servers out by region in a staggered manner… causing the EU and NA East servers to fill up completely because once players are established somewhere, there is entirely too much inertia to get them to move elsewhere when more appropriate servers are opened. The schedule looked a little something like this:
  • European Servers Opened at 11 pm PDT on 9/27
  • South American Servers Opened at 4 am PDT on 9/28
  • Asia/Pacific Servers Opened at 4 am PDT on 9/28
  • North America East Servers Opened at 5 am PDT on 9/28
  • North America West Servers Opened at 8 am PDT on 9/28
The first servers to come online were in the European Data Center, and there was a mad rush to get in and reserve your name. This is another flaw that added to the problems we are dealing with, but the fact that character names are globally locked and not tied to a specific server. As result, there were a lot of folks that popped into those servers to create a character reservation, and likely a handful of people who just wanted to play the game period regardless of server environment stuck around and started playing the game. The next big avalanche came when the East Coast servers opened, and since there is a relatively insignificant difference for most players between the two environments… every single major guild chose to start the game at this point.
Regardless of how painful the launch might have been for some players, the staggered launch and the fact that there was a land run on virtual real estate… aka player and company names made everything that much more important that you got there first. The problems that we are dealing with in this New World launch are absolutely problems of Amazon’s making, in part due to sheer lack of experience in dealing with the MMORPG player base. There is only about a 100 ms ping difference between me playing on my NA East server and playing on a European server, which means for most people the choice of where they landed was largely meaningless. New World was treated like a highly localized experience when time and time again the players have proven that they are a global audience.
So here we are with this botched launch and questionable game design… that we could easily walk away from were it not for the fact that when you ARE able to get into a server the game is damned addicting. New World is a great gameplay experience when it is operating under optimal conditions. My hope is that they have engineers working on how best to scale the servers, and as one of my guild members commented… if they doubled the capacity per node most of the queues would go away instantly. I think they need to do precisely that and at the same time offer free server moves at a company level. Let entire companies transplant themselves on greener pastures, because quite frankly unless they specifically are holding territory there isn’t a lot connecting people to a specific node at this point. I hope they can make this work because the game legitimately is good. We haven’t had an MMORPG launch that was this solid in a very long time. My advice stills stand for anyone waiting and watching from the wings. Keep waiting. If they manage to pull out of this tailspin I will be the first to raise notice that it is time to start paying attention again. Until then, however, all that waits for you is a whole lot of frustration and anxiety buffered only by just how good the game actually is. The post New World Launch Post Mortem appeared first on Tales of the Aggronaut.

Bigger is Better

Good morning folks! So I thought I would start this morning’s post with a little bit of a tip. In New World, there are a lot of resources nodes that come in various shapes and sizes. This is not just a cosmetic thing, the larger the node the more resources you will get from it. This means that if you ever roll up on a cluster of nodes, always focus on clearing them from largest to smallest. What is often going to happen is another player will roll up and start to harvest a node beside you, which means you are in a race to gobble up as many resources as you can as efficiently as you can. This for example is Iron and it is among the most hotly contested materials and given that your mining skill, your strength, your faction standing cards, and the quality of your tools all determine how fast you can gather as compared to other players. Always start on the big node and work your way down from there.
Yesterday I spent most of my time wandering around and mopping up lower-level quests that I probably should have already taken care of before now. Of course, while going on a walkabout… I would discover resource nodes that would distract me from my goal. This is the core gameplay loop for me and honestly what I enjoy about this game and games like Skyrim or the modern Fallout series. I set forth on an adventure with a goal in mind, but the process of traversing the distance in between provides a real meaningful journey. The truth is in these sorts of games I tend to use fast travel or mounts very sparingly because it is that cross country trek that really provides me with joy. In New World, I like that I pretty much still need resources that I can find rarely out in the fields and forests on the way to my destination, so stumbling across a stand of hemp for example means I am always going to stop and gather it before continuing onwards.
So far surprisingly I have only really encountered a single bugged area, and that is surrounding a quest where you need to kill a named mob down in a cave. The problem with this quest is that it seems like folks are camping the area constantly and the spawn time so infrequent that whenever we do get a pop… it is almost immediately killed by splash damage. I am uncertain how much interaction is needed in order to earn credit for a quest, but it seems like I am always short because I am pretty sure I am interacting and dealing damage but never actually finishing the quest. I am hoping to roll over there really quickly after publishing this post and maybe just maybe the early morning crowd has moved on past this area.
At this point, I have made it to level 23 and I am thinking about maybe taking a bit of a break from the game and returning to my leveling over in Final Fantasy XIV. The key turn-off right now is knowing that I pretty much have to idle with the game running in the background all day if I have any hope of actually getting some playtime that evening. Then what makes that worse… is knowing that it is highly unlikely that any of the people that are starting the game are going to be able to roll on our server and join us. This whole only 2000 characters per server thing is a pretty debilitating problem, and it threatens to kill the experience for a lot of players out there that will likely bounce, or spend two hours in queue and be unable to return the game. I think someday this will be a great experience, but until they sort out the technical limitations it feels like a really fun game that is only available for a limited few players who can manage to land on the servers in a reasonable manner.
As far as the server goes, the other company still holds First Light but is at risk of losing it. There is a war that has been declared because Marauders have taken the fort and make war on the territory to claim it. Based on what I just raid on Twitter it sounds like the Marauders are playing dirty as the Trading Co. has just had a number of their highest level members given temporary bans. I guess this is the problem with contested territory and games that more or less have automated enforcement systems. I have no clue which group is gunning for the territory because the fort doesn’t say which force claimed it, but I am going to guess it is the Spartan Legion that controls Windsward, and that they are going to be taking their progress in that territory and beginning to take over the map. Honestly, I would be completely fine with that group taking more territory because they seem to be charging fair tax rates. As a citizen of Aeternum and not a combatant… I have a completely different view of these territories because quite honestly all I care about is having a beneficial overlord that isn’t going to charge odious taxes.
I am greatly enjoying the game, but the fact that I cannot reliably play with friends… because my friends cannot reliably get into the game is somewhat damaging the experience. As I said earlier, I am getting tired of feeling like I need to stay connected in the background in order to save the said connection for the moments I can play the game. There is simply no viable way to make it through the queues during prime time hours and I think for the moment I might just move on to doing other things with my time and hoping that at some point the server sizing problems are fixed and we can all move forward together. As much hype is surrounding this game can easily turn bitter if these growing pains are not resolved quickly. More servers are not the answer… bigger servers that allow more people to play together are the only path forward. If Amazon cannot provide that, then this game will die on the vine. Otherwise, folks are probably just better off playing something like Valheim if they can’t actually experience this reliably with friends who didn’t take off work to log in at non-primetime hours. The post Bigger is Better appeared first on Tales of the Aggronaut.