Massive block changes

Discussion in 'Plugin Development' started by V10lator, May 4, 2012.

  1. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I'm currently writing at a plugin which has to do massive block changes. In fact it changes all blocks at one layer of the Y axis in all loaded chunks at once. The code I'm currently using does that by manipulating the ChunkSection directly and it does that pretty fast. But there's one big problem: Minecraft still sends packages for each block change to the clients and that lags out the server (yes, it's the packages, without any player online the server does not lag out even if I manually keep chunks loaded).
    First off, here's my code:
    Code:java
    1. private void replaceBlock(net.minecraft.server.Chunk mcChunk, int x, int y, int z, int oldId, int id)
    2. {
    3. int j1 = z << 4 | x;
    4.  
    5. if(y > mcChunk.b[j1] - 1)
    6. mcChunk.b[j1] = -999;
    7.  
    8. ChunkSection[] sections = null;
    9. try {
    10. sections = (ChunkSection[])f.get(mcChunk);
    11. }
    12. catch(Exception e)
    13. {
    14. // TODO Auto-generated catch block
    15. e.printStackTrace();
    16. }
    17. ChunkSection chunksection = sections[y >> 4];
    18.  
    19. if(chunksection == null)
    20. {
    21. if(id == air)
    22. return;
    23.  
    24. chunksection = sections[y >> 4] = new ChunkSection(y >> 4 << 4);
    25. }
    26.  
    27. try
    28. {
    29. f.set(mcChunk, sections);
    30. }
    31. catch(Exception e)
    32. {
    33. // TODO Auto-generated catch block
    34. e.printStackTrace();
    35. }
    36.  
    37. chunksection.a(x, y & 15, z, id);
    38.  
    39. if(oldId != air)
    40. net.minecraft.server.Block.byId[oldId].remove(mcChunk.world, mcChunk.x * 16 + x, y, mcChunk.z * 16 + z);
    41.  
    42. if(chunksection.a(x, y & 15, z) != id)
    43. return;
    44.  
    45. chunksection.b(x, y & 15, z);
    46.  
    47. //mcChunk.l = true;
    48. }

    My question is: Does anybody know a way to prevent the packages to be send so I can manually send a chunk package to the player after I manipulated all the blocks?
  2. Offline

    codename_B

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Maybe edit the arrays manually then refresh all chunks?
  3. Offline

    d33k40

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Interesting, idk but i bump this.
  4. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
  5. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @V10lator you can do this more efficiently (i.e. with less load on the server) by manipulating the chunkCoordIntPairQueue of the EntityPlayer objects. That field stores a list of the chunk co-ordinates that the server needs to send to the relevant player; by adding chunks to that, you allow the server to send the chunks in its own time. Sending lots of chunks explicitly with Packet51MapChunk can put considerable load on the server, especially when there are many players online.

    See https://github.com/desht/ChessCraft.../java/me/desht/chesscraft/regions/Cuboid.java for an example of this, in particular the sendClientChanges() method. It has a couple of extra optimisations:
    • It won't add the same chunk to the queue twice
    • It only queues chunk updates for players who are actually within viewing distance of the chunk that's being updated - Bukkit.getServer().viewDistance() is useful here.
    I have used this method to update tens of thousands of blocks with no noticeable lag (unscientific estimate: about 10,000 blocks in 1ms).

    I also use the Chunk a() method for quick updating of individual blocks (see https://github.com/desht/ChessCraft...va/me/desht/chesscraft/blocks/BlockUtils.java) but I can you're using your own strategy for that.

    Credit goes to @bergerkiller for explaining the concept to me, but the code linked above is mine. You're free to use/adapt it if you want to.

    P.S. love the concept of your plugin :)

    This post has been edited 1 time. It was last edited by desht May 4, 2012.
  6. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @desht Thanks, that will help a lot as I just crashed a server with the method I was using... ;)
    Do you know if a chunk is being added to that queue before or after the onChunkLoad event is called? Cause I have to manipulate chunks there, too.

    //EDIT: Credit for the idea of the plugin goes to McHeaven ( @beleg )... ;)

    This post has been edited 1 time. It was last edited by V10lator May 4, 2012.
  7. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @V10lator I think it will be safe enough to do. chunkCoordIntPairQueue is processed in the EntityPlayer a(boolean) method, and that checks if the chunk at those co-ordinates is properly loaded before actually sending it to the client and removing it from the queue. So it should be ok to add the co-ordinates of a chunk that isn't actually loaded yet.

    However, that's just from a cursory inspection of the NMS code - only way to be certain is to try it :)
  8. Offline

    Komak57

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    A chunk is 16x16x256 isn't it? that's loading 65536 blocks =.= I highly suggest you unload the said chunk first, then alter, then load it back with the changes? No idea how to do that past unload() and load()
  9. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Really not seeing the benefit of that approach... you'd have the overhead of unloading/reloading the chunk, the visually jarring effect of a chunk unload for any players in or near the chunk, and you'd still have to resend the newly reloaded chunk to players. (And thinking about it, how do you propose to change the chunk data when the chunk isn't loaded? You want to edit the world files directly?)

    A whole chunk is 65536 blocks, yes - but a Packet51MapChunk won't be anything like that large in practice:
    • The data is compressed
    • With Anvil maps, empty chunk slices don't need to be sent to the client.

    This post has been edited 1 time. It was last edited by desht May 4, 2012.
  10. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @desht
    1)I wanted to know if I manually have to add the chunk to the players queue or if it gets added after the event has passed. ;)

    2)I tried your solution, this is just a test code which doesn't check if the player needs the chunk:
    Code:java
    1. ccip = new ChunkCoordIntPair(mcChunk.x >> 4, mcChunk.z >> 4);
    2. for(Player p: world.getPlayers())
    3. {
    4. chunkCoordIntPairQueue = (List<ChunkCoordIntPair>)((CraftPlayer)p).getHandle().chunkCoordIntPairQueue;
    5. if(!chunkCoordIntPairQueue.contains(ccip))
    6. chunkCoordIntPairQueue.add(ccip);
    7. }

    but it doesn't work. The water isn't shown to the player (but he get's damage and mobs are flying around). :(

    This post has been edited 1 time. It was last edited by V10lator May 4, 2012.
  11. Offline

    Komak57

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    well, you could always do a double-buffer with worlds XD load the chunks into world2, teliport players to their corrosponding coordinates in the new map, and then reverse the process for the next reload to world1?
  12. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @Komak57 and teleport the player every 10 ticks? Sure... <.<
    Really: Manipulating the ChunkSection directly is the most lightweight solution i could find. The CPU load is very low. The only problem is sending the changes to the players...
  13. Offline

    Komak57

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    So you want to update the players every 10 ticks? (half second)? That sounds like a LOT of client-side lag >_> What exactly is the purpose of mass-chunk changes?
  14. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    mcChunk.x and mcChunk.z are already chunk coordinates, aren't they? I.e. you don't want to be bitshifting them again - just use them directly.

    For point 1), I'm not sure TBH. I'd try with and without queueing the chunk in the event handler and see which works best :)

    This post has been edited 1 time. It was last edited by desht May 4, 2012.
    V10lator likes this.
  15. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    That did the trick! :)
    But, from your plugin:
    Code:java
    1. for (Chunk c : getChunks()) {
    2. pairs.add(new ChunkCoordIntPair(c.getX() >> 4, c.getZ() >> 4));

    c.getX() and c.getZ() are chunk coordinates, too, and you're bitshifting them, too... ;)
  16. Offline

    Double0negative

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    How fast have you got it too and how many blocks are you changing?
  17. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    This is for one chunk:
    Replacing blocks: 0ms - relighting chunk: 0ms - Adding package to queues: 0ms
  18. Offline

    Double0negative

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    oh ok, just wonder cause im trying to copy/paste an insane amount of blocks and trying to figure out the fastest way. Got it up to 6-7million blocks per minute right now
  19. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
  20. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Hmm. Interesting :) my code does appear to work so clearly I have some investigation to do...
  21. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Managing about 1 million blocks in a second here :)

    226 x 226 x 60 region in just over 2 seconds (creating a chessboard with ChessCraft and the "yazpanda" style).
    V10lator likes this.
  22. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    @V10lator investigation complete: it works by luck, basically :) I get the Chunk object in my getChunks() method by calling world.getChunkAt(x,z), but I misunderstood the parameters to that method - I'm passing it Block co-ordinates, when it's actually expecting Chunk co-ordinates. Then later on, I bit-shift (as you pointed out) the result of c.getX() and getZ() - thereby compensating for the earlier error. So in this case two wrongs actually do make a right :rolleyes:

    Had I done anything with those chunks other than just getX() and getZ(), I'd have probably spotted that mistake a lot sooner... but good catch, thanks :)
  23. Offline

    Vandrake

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Hey desht x.x since you seem knowledged on this, mind telling me how to refresh a chunk? I tried the world.refreshchunk but no luck. The kind of changes im trying to see can only be seen after relog. x.x you know any way?
  24. Offline

    Hoolean BukkitDev Staff

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    This thread is from May -.-
  25. Offline

    Vandrake

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    This comment is irrelevant -.-

    I didn't ask for help about the thread. I asked the person itself. :/ But yeah I just noticed the thread's date
  26. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    You have to get nearby players and use minecrafts internal chunk queue to send the changes to them. Here's a example:
    Code:java
    1. /**
    2.   * To finish changing chunks.
    3.   * This is used internally and normally you shouldn't have to call this.
    4.   * @param chunk The chunk to finish.
    5.   */
    6. @SuppressWarnings("unchecked")
    7. public void finishChunkChanges(net.minecraft.server.Chunk chunk)
    8. {
    9. if(chunk == null)
    10. return;
    11. List<ChunkCoordIntPair> chunkCoordIntPairQueue;
    12. ChunkCoordIntPair ccip = new ChunkCoordIntPair(chunk.x, chunk.z);
    13. Server s = plugin.getServer();
    14. if(threshold == -1)
    15. {
    16. threshold = (s.getViewDistance() << 4) + 32;
    17. threshold *= threshold;
    18. clearChunkChanges();
    19. }
    20. EntityPlayer player;
    21. int px;
    22. int pz;
    23. int[] pl;
    24. Location loc;
    25. for(Object o: chunk.world.players)
    26. {
    27. player = (EntityPlayer)o;
    28. if(locs.containsKey(player))
    29. {
    30. pl = locs.get(player);
    31. px = pl[0];
    32. pz = pl[1];
    33. }
    34. else
    35. {
    36. loc = player.getBukkitEntity().getLocation();
    37. px = loc.getBlockX();
    38. pz = loc.getBlockZ();
    39. pl = new int[] { px, pz };
    40. locs.put(player, pl);
    41. clearChunkChanges();
    42. }
    43. px -= (chunk.x * 16);
    44. pz -= (chunk.z * 16);
    45. if((px * px) + (pz * pz) < threshold)
    46. {
    47. chunkCoordIntPairQueue = (List<ChunkCoordIntPair>)player.chunkCoordIntPairQueue;
    48. if(!chunkCoordIntPairQueue.contains(ccip))
    49. chunkCoordIntPairQueue.add(ccip);
    50. }
    51. }
    52. }
  27. Offline

    Vandrake

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Wow
    o.o wow awsome! Im not currentl at my pc but ill test Itwhen Im Home. Thanks
  28. Online

    desht

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    Yeah, what @V10lator said :)

    I'm actually working on a slightly higher-level wrapper class (which ideally I'd like to get pulled into Bukkit - see discussion in https://bukkit.atlassian.net/browse/BUKKIT-3113).

    See https://github.com/desht/buster/blob/master/src/main/java/me/desht/buster/MassBlockUpdate.java for the interface and https://github.com/desht/buster/blob/master/src/main/java/me/desht/buster/CraftMassBlockUpdate.java for the implementation (and https://github.com/desht/buster/blob/master/src/main/java/me/desht/buster/buster.java for how it's used).

    Note a couple of changes from the description above:
    • The NMS World z(int, int, int) method is called if a changed block has different lighting or light-blocking properties from the old one. This method is slow, but it's needed to ensure the changes are correctly re-lit. This is still more efficient than the vanilla approach, which calls the .z() method on every single change.
    • The chunk.initLighting() call doesn't appear to be useful in this case; it doesn't cause lighting to be recalculated.
    • I have a (slightly) more efficient way of detecting players to be notified of changes. It's not necessary to calculate distance - just get a bounding box of all changes, expand it by the server view distance, and see if each player is inside that expanded bounding box.
    So my current implementation is slower, but more correct in terms of lighting. One possible optimisation would be to defer lighting recalculation for blocks that need it and carry it out over the next several ticks, limiting the number of calls to .z() in each tick to avoid lag. Care is needed with this approach though, in case the block is subsequently changed again, before it's re-lit.
    JazzaG and V10lator like this.
  29. Offline

    Vandrake

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    I'm sorry to bother again but x.x I tried to implement this importing the due stuff, however some variables were unknown so I tried to read this thread again XD I'm really clueless how to do this. Honestly I only need to update like 3x3x3 area which changed the location of an itemframe. I thought refreshing the chunk would be safer but idk. I never worked with chunks but I'm eager to learn if someone teaches me >.< I also tried to read the codes desht sent and I didn't see how it could serve me. Maybe I'm just damn dumb xD so sorry to bother again and ... halp?XD The only thing I saw that could help me depending on how I put it was updateBlocksMBU?

    This post has been edited 1 time. It was last edited by Vandrake Dec 17, 2012.
  30. Offline

    V10lator

    dev.bukkit.org profile:
    CFUSERNAME
    My Plugins (CFCOUNT)
    Minecraft account:
    MCUSERNAME
    If you just need to change 27 blocks block.setType() should suit you. If you ever need to change 30x30x30 (=27000) blocks then look at this methods again. Really, what you do isn't massive...

Share This Page