Performance Optimization for 25+ Players

Discussion in 'Bukkit Discussion' started by PseudoKnight, Feb 14, 2012.

Thread Status:
Not open for further replies.
  1. Offline

    PseudoKnight

    I'd like to talk about performance optimization for 25+ players, what your experiences are and my particular challenge.

    I've recently run into a ceiling in terms of performance after a player boom due to publicity. At up to 18 players, I had the luxury of various fun plugins and features that did operations more frequently. I've now stripped out or modified most plugins for performance now that we're going over 30 players every day. Yet, I still can't prevent the TPS from dropping below 20 when it goes above 20 players. I now only have one playermove event registered (Stargate), and that's calculated decently fast (which I've determined with BukkitInsider). I've dropped or modified a number of other plugins that had frequent triggers/schedules. I've reduced view distance. I've even changed the new bukkit config option to spawn monsters at every 10 ticks instead of 1. Trial and error is reaching an impasse now, so I'm reaching out to the community for tips for larger servers. I know I could fill it up to 50 players, but I don't have the performance. Memory use is fine, though. I can stay below 3GB easily.

    Plugins: WorldBorder, PhysicalShop, PlugMan, WorldEdit, BukkitCompat, CommandHelper, JukeboxControl, PermissionsBukkit, CommandBook, Deadbolt, Stargate, Multiverse-Core, XcraftChat, BananaChunk, LogBlock, WorldGuard, CleanroomGenerator, dynmap, Dynmap-WorldGuard, Multiverse-NetherPortals, Dynmap-CommandBook

    Some random TPS samples today:
    31 players @ 11-13 TPS
    26 players @ 12-13 TPS
    25 players @ 13-16 TPS
    24 players @ 12-15 TPS
    22 players @ 15-18 TPS
    21 players @ 18-20 TPS
    <=20 players @ 20 TPS

    I know we have around 1400 animals loaded during peak times. Monsters off on all worlds results in maybe 3 extra TPS, but I never measured that exactly. Thing is, we're a survival server, so turning hostile mobs off frequently can be really unpopular.

    Here's the tricky part. I don't know what the CPU is exactly. It might be two AMD server CPUs as /debug info says AMD64. The host info suggests it's a Dual Quad Xeon E5620 or Intel E3-1270 3.4Ghz. We have 16 threads (pretty useless except for maybe dynmap, but possibly indicative of what type of CPU it is). They're using SSDs and I have 3GB allocated. I've had no performance issues until this player count increase. It's around $50 a month, I believe.

    Anyway, thoughts? Ideas? Do I just need to move on to another host? What do you guys experience at 30+ players?

    (while this is bordering on a help topic, the reason it's not there is that I'm really curious about performance experiences to get a better idea of what to expect)

    I've also wondered about the impact of multiple worlds. Theoretically another world shouldn't add much overhead (especially if you don't keep their spawns in memory), but I've heard otherwise. What's the true impact? Is stitching multiple worlds into one really beneficial for performance? How do multiple worlds impact the loaded chunks that are used to calculate a hostile mob cap. Is it per world or global? Like, if I turn off mobs for one world, does it count those chunks towards the limit anyway?

    Any other good tools for measuring performance? I've used nolagg, bukkitinsider, and commandbook so far.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 24, 2016
  2. Offline

    Nathan C

    You are more than likely experiencing a CPU bottleneck. Therefore, like in most Minecraft servers you will have to use more RAM in order to alleviate the CPU. Also, since Minecraft can only really utilize on core, then you want to decrease the load on the CPU at all costs. RAM is cheaper than upgrading your CPU (if you can even upgrade any farther), so if it takes increasing RAM use, to take some load off the CPU, then do it.

    Do this:
    - Install Nolagg. This will drastically decrease overall CPU use on the server.
    - RAMdisk or SSD. You do not want any I/O bottleneck, as this will also cause a drop in TPS.
    - Mobs. This is HUGE. I have run servers on all flavors of hardware, from Core 2 Duos, all the way up to 2500ks @ 4 Ghz and E-1230s. I have noticed that even on the E3-1230, it will start to lag and the TPS will drop after a certian amount of mobs is spawned. On my E3-1230, anything over 1500 global would cause the tickrate to drop and CPU use to spike insanely high. I personally would set the max OVERALL mob limit to around 1000-1100 or less, depending on how good the CPU is on a dedicated server. If you are on a shared host and not a dedicated server, then I would recommend limiting them to about 600-700 max OVERALL. By overall, I mean this being your global limit in the whole server. Some will say that mobs do not affect CPU use, but they would be dead wrong, as mobs is one of the greatest contributing factor to high CPU use and tick degradation. Limit the mobs in all worlds..
    - View distance. Decrease the view distance to maybe 6 or 7 and you will get slightly better performance at the cost of view distance. You really shouldn't need huge view distance numbers anyway, unless you are flying.

    Finally, if this does not work, then you have some other issues. With a shared host there is not much you can do at this point, if you did all the above. With shared hosts, you are on a node with about 30+ other users sharing the same resources as you and most of the time the hosts oversell, which creates sub-par performance for the clients. If you want 100% dedicated resources and don't want to worry about lag, then get a dedicated server. Although, if you performed the above, your tickrate should stay at 20... Mobs are huge, do not take limiting them for granted.

    Also, multiworlds isn't that bad to handle, unless you constantly have people going back and forth between them. The biggest issue with multiple worlds is mobs......as you can imagine, each world will be having their own mobs spawn, therefore it could (depending on the server) double the total mobs on the server. Mobs are huge contributing factor to low performance and limiting them is key to solving this issue.

    Measure performance with http://dev.bukkit.org/server-mods/lagmeter/
     
  3. Offline

    PseudoKnight

    I've tried nolagg with a long chunk unload delay. It doesn't seem to help. Also, like I said, turning off hostile mobs doesn't seem to get me all my performance back. Today when I turned monsters completely off I went from 11TPS to 17TPS (edit: later measured at 19.6, so maybe). That's together with less frequent grow/decay checks.

    I noticed performance drops at around 1700-1800 total mobs on our server.

    We have an SSD, thankfully. I actually noticed the difference when they upgraded from SAS in RAID. So it's a solid recommendation from me.

    Dropping view distance is really something I hate doing. I used to have it at 12 with no problem. But I kept it at 11 most of the time. Now I have it dropped to 9, but I'm still not happy. Though no one on the server is complaining about it yet. I'll try 8, but 7 is just awful. :( Incidentally, this decreases RAM usage significantly, countering your earlier suggestion of putting more to RAM. How can I use more RAM if not by keeping more chunks loaded?

    Lagmeter gives a decent TPS measurer, but it never reads my player count.
     
  4. Offline

    JohnTheRipper

    Dynmap might be causing the lag. Try removing it and/or multiverse just for a little bit to test, and see if you get better tps.
     
  5. Offline

    Nathan C

    What kind of specs is your dedicated server?

    20 players should be fairly easy to do with most any hardware.
     
  6. Offline

    PseudoKnight

    20 players is fine, but north of that performance starts dropping. I'm making some progress, but it's inch by inch. I'm going to try tweaking dynmap again. I've reduced to 8 view distance, but no stats on it yet.
     
  7. Offline

    EvilSeph

    As per our policy we've had from the beginning of the project, we do not allow discussion of unofficial builds of CraftBukkit or Bukkit as it creates an unnecessary increase in our support load. As a result, I have deleted any posts referring to such.
     
    Sayshal likes this.
  8. Offline

    Nathan C

    I would recommend disabling all automatic updates in the config. Instead just have it manually update the map every few hours via cronjob.
     
  9. Offline

    MsInscrutable

    I understand this thread was calling for suggestions?
    If the suggestions lead to non-standard and approved variants of Bukkit for improving performance , then the question is why Bukkit can't deliver the goods.
     
    battlekid and Sleaker like this.
  10. Offline

    Fishfish0001

    I have my view distance on 5, you really don't need it extremely high to enjoy the game, and it will reduce chunks loaded.

    If on Linux, you can do
    Code:
    cat /proc/cpuinfo
    which outputs something like:

    Code:
    processor: 7
    vendor_id: GenuineIntel
    cpu family: 6
    model: 42
    model name: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz
    stepping: 7
    cpu MHz: 1600.000
    cache size: 8192 KB
    physical id: 0
    siblings: 8
    core id: 3
    cpu cores: 4
    apicid: 7
    initial apicid: 7
    fpu: yes
    fpu_exception: yes
    cpuid level: 13
    wp: yes
    flags: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 x2apic popcnt xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
    bogomips: 6384.67
    clflush size: 64
    cache_alignment: 64
    address sizes: 36 bits physical, 48 bits virtual
    power management:
    
    That will show what your CPU is and how fast it is.
     
  11. Offline

    PseudoKnight

    Where's the appropriate place to discuss this policy? Because I think it may make things worse when you don't know if someone is using an unofficial build. It's also odd considering the history and nature of CraftBukkit. But you do what you have to do and I'll do what I have to do.

    Do you mean a fullrender? Seems rather wasteful and it's costly when it's run. Maybe. I've since moved to just cheaply rendered flat maps. Anybody know what "attempted" means? (eg. /dynmap stats ... Chunk: loaded=26885, attempted=78978) I couldn't find much in their threads or wiki.

    I've managed to get it to 13-20TPS at 35 players. It varies, which is expected. I'm going to continue tweaking, but I'm less concerned about it now. 30 monsters per chunk is a nice balance, it seems. 20 ticks per monster spawn may be a little low, but it works fine.
     
    Inscrutable likes this.
  12. Offline

    Inscrutable

    There is a (relatively) new updaterender command for Dynmap that should only render tiles that need updating.
    Give that a try, it should be less costly than fullrender. Using the jpg image format may help too.
    As for what attempted means, ask Mike Primm. He's a pretty approachable guy :)
     
  13. Offline

    EvilSeph

    That's why the policy is in place. We assume everyone is using an official version of CraftBukkit or else my (testing) team will be spending wasted hours on a wild goose chase. If you're not, and the general consensus is your bug report is invalid, it's your responsibility to know what's running on your server. Expecting us to provide support for an unofficial build that contains completely unverified code is insane. I've already lost count of the amount of times a problem reported by someone has been the cause of an unofficial CraftBukkit, like serious world damage for example or people who have provided it 'for the community' but then later gone on those servers and said 'type /version - I made the server you're using' in an attempt to be malicious.

    As usual, discuss this policy and any other aspect of the project in our Bukkit Project and Community Feedback forum.
     
    Zothen likes this.
  14. Offline

    NotYetRated

    Where is this option? I can find nothing else about it, and would like to tinker with it. :)

    Also, a bunch of awesome info in this thread guys, will help me a lot, thanks for it all.
     
  15. Offline

    Daniel Heppner

    So how are we supposed to tell him to use that thing that Afforess makes?
     
    Inscrutable likes this.
  16. Gosh damit, dont be so stupid and maybe use a PM? Seriously, how hard is it to think before you post?
     
  17. Offline

    Daniel Heppner

    That was offensive. These are forums, and to prevent tons of people from posting the same question again and again, you need to make the information public. A PM, as you know, is not public. Another good point is that now this person's inbox is going to be filled with people suggesting you-know-what. Before accusing someone of not thinking, please think about why the person may have done something. Do some thinking for yourself. And there was no need to be rude.
     
    Sleaker and Inscrutable like this.
  18. Offline

    PseudoKnight

    Daniel, I understand your position, but we need to move off that subject. Keep that particular discussion to the above specified forum. http://forums.bukkit.org/forums/bukkit-project-and-community-feedback.12/

    It was just added in the latest RB. In bukkit.yml there's these lines:

    ticks-per:
    animal-spawns: 400
    monster-spawns: 1

    I suspect this is really great in combination with a mob limiter like EntitySuppressor. That way the spawning mechanism isn't fighting so hard against the limiter.

    I believe updaterender only works for areas that have never been rendered. Otherwise it needs a trigger to know to update it. I turned off triggers today and LagMeter seems to be giving higher TPS over time, but there's no way to be sure. But now that I'm just using a flat map, full renders are more manageable.

    So, which is the best way to measure TPS? Which is most accurate? I see a lot of people favor LagMeter. (though I can't get it to record player counts) CommandBook's /debug clock seems good on the surface, but it's lower sometimes than LagMeter. (all the first post measurements were with CommandBook) NoLagg's performance data is robust, which I like, but I can't verify its accuracy. Also, NoLagg comes with a lot of baggage, it feels like. Unless used right, it can cause performance loss. It hooks into so many things, I'm not sure I'm comfortable with it.

    Also, no one have any idea about my multiple world question on the second post? What's the resource implication of having multiple worlds instead one? (assuming spawn isn't kept in memory)
     
Thread Status:
Not open for further replies.

Share This Page