Server Setup?

Discussion in 'Bukkit Discussion' started by derpium, Mar 25, 2012.

  1. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Hello, Bukkit community! My father and I were having a discussion the other day about setting up a large minecraft server (looking for 1000+ slots) and after much research, we have been unable to figure out how servers do this. Hardware isn't a problem for us, but minecraft only utilizes a single processor core, which I doubt would be able to handle 1000 slots. We were going to run a rack of Mac Mini Servers with 60GB SSD drives, and 16GB RAM each, but minecraft servers aren't cluster compatible. Anyone have any ideas here?

    tl;dr: How does one go about setting up a large server (1000+ slots) when hardware and internet connection aren't a problem.
  2. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I've seen in multiple places talk of an SQL database of a world, but have been unable to find anything further on it.
  3. Offline

    Lolmewn BukkitDev Staff

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I'd place the world in a RAMdisk, as hardware isn't a problem. RAMdisk basically means that you are using your RAM as drive space. Do mind though, you have to back it up properly. As far as I know, RAM gets cleared when shutting down your PC, and of course you don't want it all removed.
  4. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I don't think that RAMdisk has cluster support for Mac OSX, which is what we'll be using. My father and I both work for Apple, and thus get discounts and such.

    EDIT: After looking further into it, clustering and RAMdisk on OSX is a possibility. However, how would this scale as far as the processor? I've heard horror stories of java hitting the 99% on that one core and it just can't keep up.

    This post has been edited 2 times. It was last edited by derpium Mar 25, 2012.
  5. Offline

    Lolmewn BukkitDev Staff

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I've got no clue. I recommend getting really strong cores (like 4GHz per core or more), that'll make it somewhat better.
  6. Offline

    Obliviator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    See if you can get a linux distro on your mac.
    I really wouldn't recommend anything but linux if you are going for a large server.
  7. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I put the server on a RAM Disk, and oh god the difference. Chunks loading instantly, it was delicious. The only thing I'm concerned about with a large server is the processor. I can get a Mac Pro with the following specs for about 7 grand, but I'm not sure it will be enough.

    Apple Mac Pro:
    • 3.33GHz Hexacore Intel Xeon "Westmere" Processor
    • OWC 240GB Mercury Extreme Pro SSD (containing OS X Server)
    • OWC 128GB RAM Upgrade Kit
    • Stock 1TB HDD (for media storage such as server promotional videos, etc.)
    • Sonnet Dual Port Wired Network Card
    With this setup, I would have approx. 110GB dedicated to the server, which allows 10 players per GB of dedicated RAM. The remaining 18GB would be split between running the OS, and the RAM Disk.
  8. Offline

    Sjf11

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
  9. Offline

    Sjf11

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Let me know the IP when you get the server done. =D
  10. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I'll be sure to start a thread. If you send me a PM I'll update you directly.
  11. Offline

    Sjf11

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Actually, I would suggest you make your own computer and then put windows on it... It could have the same specs for a lot cheaper. :)
  12. Offline

    Orcworm

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    You'll need something better than Westmere for that kind of project. I doubt Apple have the option of fitting 3930X/3960X chips in their server line yet, and I'm pretty sure their motherboards are locked down to any kind of overclocking as well since lets be honest, if you want it to be anywhere near playable it isn't going to happen running at stock)
  13. Offline

    TheBeast808

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    You're going to want something like an i7 for 1000+ slot sever.
  14. Offline

    Nathan C

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    1000+ ? Nope.

    Not possible with the current state of Minecraft and hardware. Come back in about 2 years and see.

    The biggest servers I see are 200-250 and they lag like no tomorrow. An example is legendarycraft and that servers lags hardcore.

    This post has been edited 2 times. It was last edited by Nathan C Mar 26, 2012.
    Mikeambrose3 and battlekid like this.
  15. Offline

    Sjf11

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Wrong. If a person is willing to spend some money, they will be able to do this.
  16. Offline

    Nathan C

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Ok, please do enlighten me a bit then. What kind of specs will hold 1000+ on one single Minecraft server Java process?
  17. Offline

    Rjames426

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    HAHA (I soooo wanna see his reply!)
  18. Offline

    rustymetal

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Realistically, OP is crazy. You are not getting 1000 player server without a years of testing and changing things every time you hit a wall at 200/400/600 where server lag is so constant.

    First, you are going to need to overclocked cpu.
    Second, all SQL databases should be hosted on another dedicated machine and accessed via a private VLAN or what have you.
    Third, your going to be fighting Java VM every second of the day
    good luck

    This post has been edited 2 times. It was last edited by rustymetal Mar 26, 2012.
    forresthopkinsa and levisn1 like this.
  19. Offline

    Sjf11

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
  20. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Goodness, what a gigglepit this has turned into while I was gone. Let's see if I can respond to everyone in one post...

    @Sjf11 - I will not be running Windows, I despise it. If I do switch my hosting OS, it will be to Linux variation.

    @Orcworm - The Mac Pro line is due for update and when updated, will include Ivy Bridge processors. Overclocking is possible, with a little bit of work. There are various overclocking applications for Mac Pros, and worst case scenario I can always poke around in the EFI.

    @Nathan C - It is possible, with enough work and funding. Whether I set up a cluster to appear as a single virtual machine and have the Java process running off that, or overclock the hell out of a single processor, I don't know yet. But, I can assure you it is possible.

    @rustymetal - Pretty much the same response as I gave to Nathan C.

    Overall, lag isn't a problem with the proper hardware and internet connection. Both of which are available to me. I'm looking to put out a truly superior server in terms of latency, gameplay, community, and development quality. 1000 players is a goal, not a must have. I don't expect to fill 1000 slots at once. There are a few 1000 slot servers out at the moment, but none have hit even close to full. I believe the closest to 1000 slots has been in the 600-700 range.
  21. Offline

    Sjf11

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Gigglepit. XD

    This post has been edited 1 time. It was last edited by Sjf11 Mar 26, 2012.
  22. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Yeup, gigglepit, lol. I use it as a synonym for "clusterfuck" and other various less savory phrases.
  23. Offline

    Sjf11

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    You're the best. XD (still cracking up)

    This post has been edited 1 time. It was last edited by Sjf11 Mar 26, 2012.
  24. Offline

    JohnTheRipper

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Ditch Mac OS and go for Linux. I like OSX also, but as a desktop OS, it's pure shit for server use (well, technically it's better than Windows server, but not by much).

    Mac Pros? Hahahahahaha. No, no, no, no. Build your own rackmount and add Linux, and you'll pay way less and get better hardware in the process... I too am waiting for the Mac Pros, but only for video driver support so that I can build myself a decent hackintosh. Also, the CPUs used in there are locked down.

    As for this.... NOPE. Unless you feel like paying tens of thousands, maybe hundreds of thousands, you won't see Minecraft running on a cluster any time soon. However, a Ivy Bridge i7 + watercooling could manage 5-6GHz, but I'm unsure how well it would work under 24/7 load. Besides, the 32GB RAM limit is an issue, you'd need Xeon CPUs, and the fastest you'll manage there is well under 4GHz.

    ...

    So, you have a gigabit line, a $20k+ budget, and unlimited free time to maintain this? I smell trolling :)

    Let me point something out. 1k slots would require a ridiculously overclocked CPU, and well over 90GB of RAM... This assumes that you're trying to run something with lowish lag, using Linux and a highly tuned server. You'd also need at least 1gbps, as you could see 100-200mbps of player traffic at full capacity.
  25. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @battlekid - Here it goes:

    Running the server off of Linux is a 50/50 thing right now. I know it's better for server, but I enjoy being in an environment I'm accustom to, even if I have to work around problems that will likely arise.

    I intend to get a baseline Mac Pro from Apple, and then put about 3k worth of upgrades into it via 3rd party sources. I've built a few Hackintosh's in my day, but with Mountain Lion coming up soon, and so many products awaiting updates, I'll be waiting before I start on a new Hackintosh project. While the Mac Pro processors can not be unlocked, ZDNet has successfully overclocked a 2.8GHz chip to 3.24GHz using their overclocking software (which still works on the current Mac Pros).

    If I could get the cluster to appear as a single virtual machine, with one core, and have all of the machines in the cluster contribute to said virtual processor core, it THEORETICALLY is possible. Whether something like this is actually going to happen, time will tell. An Ivy Bridge i7 is actually worse than the Sandy Bridge Enthusiast chips. The main difference between Ivy and Sandy Bridge chips is the dye process, Ivy Bridge using 22nm and Sandy Bridge using 32nm. The RAM limit on Ivy Bridge and Sandy Bridge i7 chips is 64GB, not 32GB. However, using a dual Xeon setup to bypass this, it is possible to get them up to 4GHz, but that's about where they would max out.

    I have access to a gigabit line, yes. I do not see where a $20k budget comes into place however. As a fully upgraded Mac Pro tops out at about $7k, and if I go with the whole building my own rackmount server running Linux option (which you said would cost way less) I would presume that would knock it down a few grand. As far as unlimited free time, yeah, I have that.

    To achieve the 1k slots, the RAM is no problem I've been outlining a plan entailing 128GB of RAM, 110GB of which is dedicated to the server. The only problem here is the CPU, which is something I'm working on figuring out.

    If anyone runs a fairly large server, I would appreciate it very much if you'd post your server specs and how a certain sized load affects your processor. For example, what is the % of processing power being used when you have "x" number of players on.
  26. Offline

    JohnTheRipper

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Linux has sooo much more options as far as server-aimed software goes. It's also extremely stable, and has easy 1.7 support.

    For $5500 you could easily build a rig that would blow away any Mac pro... Try dual LGA2011 Xeons, with 128GB of RAM, and SSD storage. I'd bet you'd be way under $5.5k, although you can't overclock... The only way this *could* work is if you get a 3960K and uber-overclock it, then stick 64GB of RAM and some SSDs into it.

    Those overclocks are tiny. People can manage 5+GHz on i7s nowadays, and you'd need every bit of that juice and more for 500 slots, never mind 1k slots.

    But you'd need at least 5-6+GHz for 500 slots... And I don't even have a clue how fast it would need to be for 1k... I don't even think that you can scale that high while maintaining playable performance...

    Oh really, gigabit? So you're going to colocate it, right? Good idea, but sadly, a Mac Pro cannot support 1k slots.

    $20k is for the developer you're going to hire to rewrite Minecraft so that it can run on multiple threads across a cluster of Mac Pros.

    Yeah, and good luck with THAT. Tell us how it works out for you.

    i3-2100 (16GB DDR3 RAM), 2% load with empty server, if you want to poke around in munin go to http://a304.net/munin. More bandwidth/player info in http://a304.net/info. I'm estimating that my CPU could handle ~100-150 slots, maybe more, maybe less. If I wanted to go higher, I'd need a higher end E3 or a i7.

    This post has been edited 1 time. It was last edited by battlekid Mar 28, 2012.
  27. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I'm aware that Linux has more server friendly options, but like I said before, I prefer an environment I'm comfortable in. If I do choose to go with Linux, it'll be with a juiced up Core 2 i7 Extreme on liquid cooling with 64GB of RAM, which hopefully will do me in for 500 slots. From there, I would get an identical rig, and just have the servers run on two different ports. I recall a plugin sometime back that allowed you to use a teleporation system to go from one server to the other, so players could simply log into the main world, and be able to take a portal to the second world. Another thing to consider, aside from conventional liquid cooling, is a Mineral Oil build, which I feel would surely disperse heat effectively.

    What form of Linux are you running?
  28. Offline

    JohnTheRipper

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Much more reasonable goal. You might even manage to hit 500 slots with zero lag if you're good enough at optimization and can OC it high enough.

    CentOS 6.2 64 bit, with Java 7.

    Keep in mind that most datacenters don't want liquid cooled servers, and a tank of Mineral Oil is even worse from their perspective :p.

    This post has been edited 1 time. It was last edited by battlekid Mar 28, 2012.
  29. Offline

    Refalm

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Just use Mac OS X Server if you're most comfortable with Apple, it's based on BSD anyway, so you should roughly have the same options as running Debian or something.
    I think people confuse it with the desktop version, which would be a bad idea.

    If that doesn't work out, try Debian. It's got a great balance between being easy to use (as opposed to Arch or Rock), and not adding bullshit (like Suse or Ubuntu).

    This post has been edited 1 time. It was last edited by Refalm Mar 28, 2012.
  30. Offline

    derpium

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @battlekid - Fuck datacenters :D

    @Refalm - This is true, they often get mistaken. With OS X Server I still have the problem with the CPU clock.

Share This Page