Optimize JAVA for MineCraft in a EXTREME way :P -- NEED FEEDBACK, THX !! [- I AM BACK -!]

Discussion in 'Bukkit Discussion' started by Therawroy, Jul 12, 2011.

Thread Status:
Not open for further replies.
  1. Offline

    Therawroy

    Some people asked me if i just test it with trial and error (or google), or if i have a real idea of what i am doing ....

    So here is a little screenie from one of the tools i use .... you can clearly see all the things which has been discussed here lately in a nice gui. And i can play with the garbage collectors .... to test and ensure you have a nice playing time ;)

    Its only a small part of the whole programm ... but i need to protect it for the haters ;)

    [​IMG]
     
  2. Offline

    pomo4ka

    Good. Tell me your arguments above, only work on Java 7? How many players are online?
     
  3. Offline

    Therawroy

    And if you wanna optimize it a little! for Multi Core .... try:

    apt-get update
    apt-get install irqbalance [nice one ... gives every free cpu something to do]
    apt-get install preload [The Name is programm ;) a must have]
    apt-get update
    apt-get upgrade

    Sorry pomo4ka, i didnt see your text.

    I tried almost 98% with Java 7, because i am a fan of new stuff .... and Java is since the beginning crappy ... yeah it works on almost every device ... but slow and shitty. So after i heard there are some usable Snapshots, i was the first who tried. There was a little speedplus ..... but nothing real. And i learned at this time that my server is too weak for my project. So what to do ?? Change the Server or optimize the Crap out of it (my favorite way) if its realistic ... and you read my results :p

    So i choose of knowledge, experience and some luck the commands for the Problems i monitor .... not all commands are from Java 7 .... there are also some real old ones in there :p
    But what makes the difference ??? I can really measure it and give it first hand from a poor vServer / VMWare on good PC / Hardcore Dedicated Server ...... and maybe Android soon xD

    Love me or hate me .... but i am away .... searching for food ;) cu later

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
  4. Offline

    will d

    Based on the heap size in his GC screenshots, I'd say there's ... one player on? Maybe a second client? Zero cpu, so they're definitely just sitting around.

    @Therawroy
    To address all your 'to you haters' lines, I don't have a problem with you coming up with anecdotal answers to problems, but you do not seem to be generalizing them or properly using them as evidence for any real-world application.

    As an example--from a 'theoretical' viewpoint, I don't see that you should value -- AT ALL -- free memory in relation to your maximum heap size. We all know that with java, there is the maximum heap size, and the accompanying java/stack overhead that comes with it. So a heapsize of 2048 in a running java instance we should expect to take > 2048 by a slim margin (something like 50MB). But looking at 'top' and seeing that instead of your server rocking out a 2048+ working set, you're finding victories in seeing 600MB(!!!).

    The way to look at it (as I see it) is like this: under maximum capacity of your server, you are permitting minecraft to use 2048 MB. If the heap size is currently at 600MB and you have 1400 free megabytes...that memory is doing nothing for you, at all. If Java has to slowly claim more and more memory until it hits the 'real-world' goal of high-pop server and reaches 2048, what was saved by having the heap size reduced to start?

    In other words, if you're basing successful java tweaks on the working set (memory footprint), so you can MINIMIZE it....then run OTHER STUFF in the background---that does not seem like maximizing your minecraft server at all--it feels like maximizing the multi-tasking ability of your server, which is a completely different goal.

    You're essentially finding how to do other things on your server, which other admins may or may not need/want to do. That goes back to my 'one-size-does-not-fit-all' issue, wherein your solution (java tweaks) may tackle a problem you're having, but you're claiming it fixes a problem for anybody who has unoptimized minecraft performance issues and 'wants more horsepower'.

    That's the point of so many other admins in the world, like myself, of making the minimum and maximum heap size the same; we are tailoring our servers to operate its other tasks, like dynmap, both when the maximum heap size is reached (minecraft load capacity reached), and when not at capacity--not 'dynmap' works great when theres a lull in the minecraft world and java can re-release memory to the system.

    if tldr: you're claiming to have struck oil when you really just found a spill on your driveway. Your tests are not rigorous enough to make these un-intuitive, counter-productive tweaks proof of improvement.
     
    mooman219 and Lunar Delta like this.
  5. Offline

    Ziden

    what could be changed on this config to reduce CPU usage ?
     
  6. Offline

    Zalastri

    I don't think you understand, man. These launch options will make your MineOS 2x longer.
     
  7. Offline

    EvilSeph

    All of these threads detailing how to optimise the JVM are misleading and inaccurate. Generally, the JVM is pretty good at optimising itself based on the environment the JVM is running in (with a few obvious exceptions). Any other optimisations require a lot of time and trial and error to figure out what configuration is right for your server. Every java application is different, and on top of this, every server environment is different too.

    By just throwing random flags into the launch line you run the risk of undoing the automatic, and correct, optimisations the JVM has chosen to use based on the system environment it is running in. While some of these flags can improve performance, most of this is just the placebo effect in play. Your post provides a list of flags with no explanation as to what each one does and what effect they have on the JVM and this is polarly contrary to how system optimisation should take place. Just like overclocking a PC, you need to do things in baby steps and spend countless hours rigorously testing each change for at least a week in order to really figure out what is beneficial to the performance of the JVM in relation to the program and what isn't.

    Usually, you can very quickly find the answer to a question by using Google. Has it not, then, struck anyone as odd that there is no proper guide written to tune your JVM? It's because one can't be written; it's not that easy.

    Nonetheless, here are some observations I've made based on what I've seen people trying:
    -server - this switches the compiler to an optimizing one that generally results in a significant increase in performance, at the expense of a longer warm up time.
    -d64 - this makes the JVM run in 64bit mode. However, if it isn't already doing so, you can't force it as such, so this flag is useless.
    -Xnoclassgc - this flag should not be used with CraftBukkit as we make EXTENSIVE use of reflection and could lead to a large memory leak as a result of all the dynamic classes not being garbage collected when they need to be, since you are telling it not to…
    -XX:+AggressiveOpts - this flag is a quick way to turn on multiple optimisations but I have not looked into if this would increase or decrease performance for CraftBukkit.

    Garbage Collector selection:
    Parallel Collector: -XX:+UseParallelGC
    Concurrent Low Pause Collector: -XX:+UseConcMarkSweepGC
    Incremental Low Pause Collector: -Xincgc

    Since you're selecting *a* garbage collector, it makes sense that you can only choose one. Having multiple flags choosing different GCs in the same launch line will just have the JVM pick the last one you specified. The only exception to this is if you want to run a Concurrent Low Pause Collector with a Parallel YOUNG Generation Collector:
    -XX:+UseConcMarkSweepGC -XX:+UseParNewGC

    Once you've chosen your garbage collector, if you even want to, you should then leave it up to the JVM to decide what parameters the GC should follow. For example, a lot of people tell you to specify how many threads a parallel GC should have, but you run the risk of specifying the wrong amount - the JVM knows best and has already set it as such.

    Anything that uses System.gc() to try and explicitly force a garbage collection (even though it won't) is detrimental to the performance of your system. If there's one flag I'd recommend everyone use, it is -XX:+DisableExplicitGC to prevent naughty software from trying to do this. Leave it up to the JVM, it knows best. Some plugins like Essentials and, I believe, Performance Tweaks do this and it is strongly recommended against. While it merely provides a hint to the JVM, there is still a chance that it could cause a GC to occur when it shouldn't and is not needed.

    In relation to the advice given in this specific thread, you have suggested people use conflicting flags like enabling the incremental garbage collector, then later enabling the concurrent one, followed by enabling the parallel one. Which one should the JVM use now!? It really seems like you scoured Google for JVM flags and just threw them into one launch line without any knowledge as to what effect they would have on the JVM.

    In conclusion: unless you understand how the JVM works and how the application you're trying to tune handles memory, you're better off just sticking with letting the JVM do what it does best, handling the optimisations. I've probably forgotten to address some issues, but I think this will help out some people for the time being.
     
  8. Offline

    Monstertke


    and /thread

    again....
     
  9. Offline

    Lunar Delta

    This. I think I told him at the beginning of the thread that Xincgc and ConcMark didn't go together, but he didn't listen... D:
     
  10. Offline

    Therawroy

    i never said anything against it :p I just wanted to check it myself ... and arent we all humans in the end ?? So mistake is what us defines :p
     
  11. Offline

    Deanouk87

    Hi,
    I would like to try this but have no idea how to install JDK 1.7 onto CentOS.

    I have downkloaded the tar.gz file but cannot find a guide on how to install. Can someone assist me?
     
  12. Offline

    Monstertke

    It wasnt hard, as soon as I get a free 10 minutes Ill look at my notes and edit this post.
    Centos has its peculiarities.
     
    Deanouk87 likes this.
  13. Offline

    Deanouk87

    Cool, thanks! It's my first proper experience with linux, nice learning curve so far.
     
  14. Offline

    Monstertke

    Lol, sorry man I lied. I have a management package with singlehop. I had them do it.
     
  15. Offline

    Deanouk87

    Nevermind then... I'll try and work it out.
     
  16. Offline

    dureiken

    Hi there,

    thanks for your work, I would try your optimized Java server but I can't achieve to install JDK 1.7 on my ubuntu server :

    When I follow your guide , downloading tar file on http://jdk7.java.net/download.html

    Code:
    jdk-7-fcs-bin-b147-linux-x64-27_jun_2011.tar
    I unzip it and copy it in /usr/lib

    Code:
    root@ICreeper:/usr/lib/jvm# ls -l
    total 8
    lrwxrwxrwx  1 root root   14 2011-06-28 18:44 java-1.6.0-openjdk -> java-6-openjdk
    drwxr-xr-x  5 root root 4096 2011-06-28 18:44 java-6-openjdk
    drwxr-xr-x 10  500  500 4096 2011-06-27 10:51 jdk1.7.0
    
    but when I run

    Code:
    sudo update-alternatives --config java
    It says "there's no alternative to config java in /usr/lib/jvm/java-6-openjdk/jre/bin/java"

    Where did I miss something ?

    Thanks :)
     
  17. Offline

    Phinary

    Im using the 64bit version. I have the starting ram set to 2gb, but when I start the server, it only starts with 1gb. Is something wrong?
     
  18. Offline

    bradgillap

    Can someone help me with using multicraft and java 7?

    Here is the conf file


    Code:
    #To create a custom config for a mod create a file with the same name
    # as the jar file and append ".conf", for craftbukkit this would be
    # "craftbukkit.jar.conf".
    #The following config is an example that lists all possible options,
    # without actually changing anything (it remaps the commands on
    # themselves)
    #
    #General configuration. If this contains a setting called "file" the
    # [commands], [start] and [parse] sections will be loaded from this
    # other file
    [config]
    name=Mod: Craftbukkit
    source=http://ci.bukkit.org/job/dev-CraftBukkit/promotion/latest/Recommended/artifact/target/craftbukkit-0.0.1-SNAPSHOT.jar
    configSource=http://www.multicraft.org/download/conf/?file=craftbukkit.jar.conf
    
    #encoding used for strings sent to and received from the server
    [encoding]
    ## The encoding to use to send data to the Minecraft process
    ## Use "system" for your current locale.
    ## default: system
    encode=latin-1
    ## The encoding to use to decode data received from the Minecraft process
    ## Use "system" for your current locale.
    ## default: system
    decode=system
    
    ##command line customization
    ## Available variables
    ## - {JAVA}
    ## - {START_MEMORY}
    ## - {MAX_MEMORY}
    ## - {JAR_DIR}
    ## - {JAR_FILE}
    ## - {JAR} -> {JAR_DIR}/{JAR_FILE}
    ## - {MULTICRAFT_DIR} The Multicraft installation directory
    ## - {BASE_DIR} The directory containing all the servers
    ## - {SERVER_DIR}
    ## - {WORLD}
    ## - {IP}
    ## - {PID_FILE}
    [start]
    command="{JAVA}" -Xmx{MAX_MEMORY}M -Xms{START_MEMORY}M -Djline.terminal=jline.UnsupportedTerminal -jar "{JAR}" nogui
    
    #command remapping
    [commands]
    say=say
    list=list
    save-on=save-on
    save-off=save-off
    save-all=save-all
    stop=stop
    tell=tell
    give=give
    tp=tp
    deop=deop
    op=op
    pardon-ip=pardon-ip
    ban-ip=ban-ip
    pardon=pardon
    ban=ban
    kick=kick
    help=help
    
    #adjust message parsing, contact us if you require this
    [parse]
    clean=(^[>\r]+|\x1b\[[0-9;]+m)
    hide=(There are |\d+ players are)
    players=^\s*(?:Connected\s*players|Online \([\d]+[^)]*\)): (.*)$
    
     
  19. Offline

    LegendOne

    hy guys, its a bit late here (GMT+1) and iam abit drunk.
    So iam running a server4you Vserver (2GB vram + 2x ~2Ghz) and try to run you little command but have this problem:
    Code:
    java -server -Xincgc -Xmx1539M -Xms512M -Xmn256M -XX:NewRatio=2
     -XX:CMSFullGCsBeforeCompaction=1 -XX:SoftRefLRUPolicyMSPerMB=2048
    -XX:+CMSParallelRemarkEnabled -XX:+UseParNewGC -XX:+UseAdaptiveSizePolicy
    -XX:+DisableExplicitGC -Xnoclassgc -oss4M -ss4M -XX:+UseFastAccessorMethods
     -XX:CMSInitiatingOccupancyFraction=60 -XX:+UseConcMarkSweepGC -XX:UseSSE=3
    -XX:+UseCMSCompactAtFullCollection -XX:ParallelGCThreads=2 -XX:+AggressiveOpts -cp
    "craftbukkit-0.0.1-SNAPSHOT.jar" -jar "craftbukkit-0.0.1-SNAPSHOT.jar" nogui
    Error occurred during initialization of VM
    Could not reserve enough space for object heap
    Error: Could not create the Java Virtual Machine.
    Error: A fatal exception has occurred. Program will exit.
    Does one of you know why im getting this problem?
     
  20. Offline

    Monstertke

    Lower your -Xmx in 50Mb increments until it starts. Remove the -Xnoclassgc and -XX:parallelGCThreads=2
     
  21. Offline

    LegendOne

    nice thank you.
    worked for me with following command string:
    Code:
    java -server -Xincgc -Xmx1152M -Xms512M -Xmn256M -XX:NewRatio=2
    -XX:CMSFullGCsBeforeCompaction=1 -XX:SoftRefLRUPolicyMSPerMB=2048 -XX:+CMSParallelRemarkEnabled
    -XX:+UseParNewGC -XX:+UseAdaptiveSizePolicy -XX:+DisableExplicitGC -oss4M -ss4M
    -XX:+UseFastAccessorMethods -XX:CMSInitiatingOccupancyFraction=60 -XX:+UseConcMarkSweepGC
    -XX:UseSSE=3 -XX:+UseCMSCompactAtFullCollection -XX:+AggressiveOpts -cp "craftbukkit-0.0.1-SNAPSHOT.jar"
    -jar "craftbukkit-0.0.1-SNAPSHOT.jar" nogui
    quick question, can i use the "screen" programm or will there be complications?

    i changed my commands to run with screen sessions. The server is giving me some errors:
    Code:
    #!/bin/sh
    screen -AS minecraft java -native -server
    -Xincgc -Xmx1152M -Xms512M -Xmn256M
    -XX:NewRatio=2 -XX:CMSFullGCsBeforeCompaction=1
    -XX:SoftRefLRUPolicyMSPerMB=2048 -XX:+CMSParallelRemarkEnabled
    -XX:+UseParNewGC -XX:+UseAdaptiveSizePolicy -XX:+DisableExplicitGC -oss4M -ss4M
    -XX:+UseFastAccessorMethods -XX:CMSInitiatingOccupancyFraction=60
    -XX:+UseConcMarkSweepGC -XX:UseSSE=3 -XX:+UseCMSCompactAtFullCollection
    -XX:+AggressiveOpts -cp "craftbukkit-0.0.1-SNAPSHOT.jar" -jar "craftbukkit-0.0.1-
    SNAPSHOT.jar" nogui
        
    19:45:03 [SEVERE] java.lang.OutOfMemoryError: unable to create new native thread
    19:45:03 [SEVERE]       at java.lang.Thread.start0(Native Method)
    19:45:03 [SEVERE]       at java.lang.Thread.start(Thread.java:697)
    19:45:03 [SEVERE]       at net.minecraft.server.NetworkManager.<init>(NetworkManager.java:68)
    19:45:03 [SEVERE]       at net.minecraft.server.NetLoginHandler.<init>(NetLoginHandler.java:21)
    19:45:03 [SEVERE]       at net.minecraft.server.NetworkAcceptThread.run(SourceFile:53)
    >

    Some ideas? I thin my ressources are just bad.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 17, 2016
  22. Offline

    Monstertke

    Try the same as above but manually invoking screen through screen -s
     
  23. Offline

    LegendOne

    its terminating out of nowhere. Log says everything is allright. I think its more of a ressource problem.
     
  24. Offline

    Jdbye

    What does the -native switch do exactly? It seems it can only be used together with -server, but on my server for some reason the server version of the Java VM slows down over time and causes constant server lag after a few hours without a restart, so I had to remove -native and -server from your arguments.
    Otherwise seems good, anything to improve how the server runs since a over-spec'ed server doesn't help much either, MC doesn't make good use of a multithreaded CPU.
     
  25. Offline

    saladfingers

    I've been using flags like this for quite some time, but after reading EvilSephs post I decided to try without them and in all honesty I think my server actually runs better when I leave java to config itself.

    All those flags look good but I really think its unnecessary. My CPU usage is lower and memory usage is exactly the same as when I used the flags.
     
  26. Offline

    Monstertke

    They are good for big ram servers. The best thing is setting -Xmn to the correct amount and enabling ParNewGC and CMS.
     
  27. Offline

    Therawroy

    -- I AM BACK --

    PLS jump to PAGE -1- to see the NEW UPDATEs
     
  28. Offline

    Monstertke

    Im gonna be real nice this time.... Did you read EvilSephs post on this thread about GC? Did you catch the part where he said NOT to use -Xnoclassgc? Because it can cause a runaway memory leak.

    It begs the question, if you ignore one of the guys who WROTE bukkit and included something that he said was dangerous, what else did you include that was dangerous?

    The other point he tried to make was that every JVM instance is different. JVM tuning options are for one machine and one instance under unchanging and carefully controlled circumstances, which bukkit+plugins definitely is not.

    So enjoy the spotlight, have fun, keep posting in 24pt type, broken English and bright colors.
     
    xtremetom likes this.
  29. Offline

    Therawroy

    Welcome to the discussion.
    Yeah and it was long time not an used option. But i had a real long time for self testing and it wasnt any problem maker. And i tried 3 different Java7 Versions without problems and like i also said its only one thing of the package. I made some other significant changes to increase the minecraft experience :p Like CustomKernel cpu specified and scheduler tuning with hughpachesizes cpu specific optimizations , zram and so on ;)

    Maybe i release my own LinuxDistro called "LongSPITAK" or whatever :p now i have the time because my server runs like a wet dream xD

    and for your information .... "I love bukkit" .... =)
     
  30. Offline

    spunkiie

    @Therawroy

    Don't you get it ? You are lame+ dude. Everything you say is pure crap.

    "you need JAVA "JDK 1.7.0" and not "JRE 1.7.0" !!!!!"

    It's just the stupidest thing i've ever readed in bukkit forum. JRE simple COMES with JDK, the exactly same thing, the exactly same code. Also, your flags conflict each other. You don't even know wtf are you talking about. You tell ppl to use java switchs that don't even exist anymore since java 1.4. This thread should be closed due to misinformation.

    Man, do the community a favor and KILL YOURSELF.

    KILL YOURSELF AND TAKE THIS USELESS THREAD WITH YOU.

    NOW
     
Thread Status:
Not open for further replies.

Share This Page