Storing persistant data...

(Title originally "Storing a persistant (hash)map?")

What is the best way to do this? SQL looks way more complex than I require.

 

if you mean Map (the java data type, not the game item map or the map itself), you could always use an object writer/object reader, granted you might have to make some wrapper classes depending on what your map is made of if the objects are not serializable

 

Yes I mean the data type. What does the writer/reader write it to and read it from..? lol.

I'm not sure what serializable means, but I don't use it. It would be made up of two strings. (three if I was using sql I guess, but two would work fine.)

 

a file, and strings are serializable, serializable means that it can be writen to a file. However if you are just using strings you could always just write them to a .yml file. If you want them to be user editable go with the .yml, if not look up the ObjectOutputStream and ObjectInputStream

 

Agree with wwsean08 on YML vs object/binary output. This page has some more information that will be useful, especially the explanations of what "serializable" means and how to use it.

http://wiki.bukkit.org/Introduction_to_the_New_Configuration#Advance_Topics

 

Actually, by definition of the term map, storing something in a yaml file is actually storing something in a map. So let's not make this more complicated than it has to be.

 

This isn't for config. Its for persistant data..

 

well you can still save it in that way if its just strings

 

Yes but we are taking about alot of strings.

Like, for this: http://forums.bukkit.org/threads/req-decoyblocks-v1-0.48818/#post-842671

So I need a very large list of blocks, and then a log of Player, Timestamp, and block.

 

I only brushed over this topic, so my apologies if I restate anyones perspective.

When I first began programming plug-ins for Bukkit that required data persistance I began with SQL, and I can honestly say it was horrible. The export times were terrible, the stability was horrible and the difficulty of exporting a collection to a table structure was limiting.

Serializable was an amazing alternative. I used Serialization with previous C# projects so I decided to give it a shot in java, and so far I have nothing but positive things to say. Export times are extremely fast (milliseconds), data stability is no longer an issue (with SQL I lost my DB 7 times, with Serializable I've never lost data in 2 months).

But the best selling point to me is that entire objects can be serialized as long as the values stored within it are serializable. So if you have a custom class and every variable within that class is serializable (basic data types like string and integer) then you can just send the entire object at once to be serialized, you don't have to break it down to it's individual variables and then configure a SQL statement to export the data.

I am rambling, and for that I apologize; but long story short I have been incredibly happy with java's IO serializable interface, it has been insanely easy to use, extremely stable, and implementing it takes a matter of minutes.

Good luck with your decision!

 

So... serialization... How do I do that without going through ...config?

 

Then you would make a class "DataRecord" or what ever, and inside that class you would create variables that store all the data you want. Just remember bukkit variables aren't seriazable, only default java value types. So if you wanted to store a location, rather than doing this:

Code:

Location loc = null;

you would do this:

Code:

Integer locX = 0;
Integer locY = 0;
Integer locZ = 0;
String locWorld = null;

The two methods store the exact same information, but one is serializable because its made of basic java data types while the other is storing a custom bukkit class.

Then you would just make a function in the record class "GetLocation" that would take those variables above and concert them back into a location:

Code:

public Location GetLocation()
{
    return new Location( Bukkit.getWorld( locWorld), locx, locy, locz);
}

So you would make a single class that stores everything you wish to save, make sure its all generic java data types, implement the serializable interface and your set.

Word of advice, when serializing make the first thing you save be the size of the collection you are saving, so if you have 54 records save the 54 integer, then loop through your records and save each one, then when you go to load read the first integer and bingo, you know how many records you need to load, then just do a for loop for each integer you just loaded, and bingo all 54 records are loaded back into the server.

I know what I've said is likely chinese to you if you've never worked with serialization before, but just remember you have to read the same amount of objects from the list in the same order they were placed in the list or it wont load properly, so...

P.S. everything here is psuodo code written from the top of my head, so only use them as food for thought and not literal code snippets.

I hope my aimless ranting was able to shed some light, good luck!

 

Not sure if this is serialization, but an example is TrainCarts where I store lots of groups as raw data:
Where I load my data
Where I read/write the object (example)

 

By the way, regarding this sentance:

If the list is within the serializable object it will be saved with the object! So if you have a class with 3 ArrayList being stored in it when you save the class it will also save the content of all 3 list aswell, again as long as the data in the list is serializable:

Code:

 List<Player> players = new ArrayList<Player>();

This is NOT serializable because the Player class is a custom class unique to Bukkit, not a default java data type, however if you did this:

Code:

 List<String> players = new ArrayList<String>();

you CAN serialize it because string is a standard java datatype, then to get the player you want you would simple retrieve their name from the ArrayList and execute the following code:

Code:

Player player = Bukkit.getPlayer( PlayerName );

It's just a matter of arranging your data properly in the beginning so that when the serializable interface tries to save it later it can just handle the entire class for you, no need to break it down into each variable that is being stored and serialize them one at a time.

I've done nothing but aimlessly ramble at you for 30 minutes, so if you manage to make any sense of my babble I should buy you a beer. ^^ or a Rootbeer (depending on your age! )

Hmm, you are using the exact same streams that serializable does, but you aren't actually utilizing the Serializable interface, rather you are simply writing each variable manually using the filesteam.

As I was saying above, that is what is nice about the serializable interface; rather than saying:

save this integer in object1
save this string in object1
save this boolean in object1
save this integer in object1

you can just say:

See every variable in object1, just save them all for me please!

Edit: you know what, RiftCraft is about to be pushed to github anyways, so this code is about to become pulic regardless; I might as well share it with you:

Code:

package me.FatherTime.RiftCraft;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.util.Map;

import org.bukkit.Bukkit;
import org.bukkit.plugin.java.JavaPlugin;

public class RCDataHandler
{
    public void Serialize()
    {
        try
        {
            FileOutputStream fstream = new FileOutputStream("plugins\\RiftCraft\\RCsave.dat");
            ObjectOutputStream ostream = new ObjectOutputStream( fstream );

            ostream.writeInt( RiftCraft.GetBookHandler().GetRiftBooks().size() ); // writes how many records we are recording
            for ( Map.Entry<String, RiftBook> KPV : RiftCraft.GetBookHandler().GetRiftBooks().entrySet())
            {
                String BookOwner = KPV.getKey();
                RiftBook Book = KPV.getValue();
                ostream.writeObject( (String)BookOwner );
                ostream.writeObject( Book );
            }
        }
        catch( IOException ioe )
        {
            ioe.printStackTrace();
        }
    }

    public void Deserialize()
    {
        File savefile = new File("plugins\\RiftCraft\\RCsave.dat");
        boolean exists = savefile.exists();
        if(exists)
        {
            FileInputStream ifstream = null;
            ObjectInputStream iostream = null;
            try
            {
                ifstream = new FileInputStream("plugins\\RiftCraft\\RCsave.dat");
                iostream = new ObjectInputStream( ifstream );

                Integer recordcount = iostream.readInt();
                for( int i = 0; i < recordcount; i++ )
                {
                    String BookOwner = (String)iostream.readObject();
                    RiftBook Book = (RiftBook)iostream.readObject();
                    RiftCraft.GetBookHandler().GetRiftBooks().put(BookOwner, Book );
                }
            }
            catch(FileNotFoundException e)
            {
                RCUtilities.LogPrint( "Could not locate RCsave.data", true );
                e.printStackTrace();
            }
            catch(IOException e)
            {
                RCUtilities.LogPrint( "I\\O error while attempting to read RCsave.dat", true );
                e.printStackTrace();
            }
            catch (ClassNotFoundException e)
            {
                RCUtilities.LogPrint( "Could not deserializing RCsave.dat, class not found.", true );
                e.printStackTrace();
            }
            finally
            {
                try
                {
                    ifstream.close();
                }
                catch ( IOException e )
                {
                    RCUtilities.LogPrint( "Error deserializing RCsave.dat, could not close stream.", true );
                    e.printStackTrace();
                }
            }
        }
    }

    public void RegisterSerializationTimer( JavaPlugin plugin )
    {
        Bukkit.getServer().getScheduler().scheduleAsyncRepeatingTask( plugin, new Runnable(){public void run()
        {
            Serialize();
        } }, 1200L * 5L, 1200L * 5L ); // saves every 5 minutes
    }
}

This is the Datahandler class for RiftCraft, a portal and teleportation plug-in that I made. This will contain a bunch of irrelivant code aswell, but it's a good working example of the serializable interface.

I hope this helps!

EDIT by Moderator: merged posts, please use the edit button instead of double posting.

 

It does. I don't understand how I'm going to write and read this, though?:

Integer Double locX = 0;
Integer Double locY = 0;
Integer Double locZ = 0;
String locWorld = null;

I mean, I can just write that, but how is it going to read it together properly?

 

Look at my sample deserialization function:

Code:

    public void Deserialize()
    {
        File savefile = new File("plugins\\RiftCraft\\RCsave.dat");
        boolean exists = savefile.exists();
        if(exists)
        {
            FileInputStream ifstream = null;
            ObjectInputStream iostream = null;
            try
            {
                ifstream = new FileInputStream("plugins\\RiftCraft\\RCsave.dat");
                iostream = new ObjectInputStream( ifstream );

                Integer recordcount = iostream.readInt();
                for( int i = 0; i < recordcount; i++ )
                {
                    String BookOwner = (String)iostream.readObject();
                    RiftBook Book = (RiftBook)iostream.readObject();
                    RiftCraft.GetBookHandler().GetRiftBooks().put(BookOwner, Book );
                }
            }
...

That is where the "read them back into memory in the same order they were saved in" comes into play. Its done with class casting. If you have 3 objects:

ClassA (stores 3 variables)
ClassB (stores 2 variables)
ClassC (stores 5 variables)

and then you try to serialize them all:

save ClassA (stores 3 variables)
save ClassB (stores 2 variables)
save ClassC (stores 5 variables)

when you go to load them back in it will just associate the bytes in memory with the class structure when you cast it

ClassA classa = (ClassA)load.data

But if you try and load them out of order the data saved won't match up with the data being loaded.

load ClassB (loads 2 data attributes, but this save has 3 data attributes)
save ClassA loads 2 data attributes, but this save has 2 data attributes)
save ClassC (stores 5 variables)

Which would cause the saved data for class B and A to not loading properly. You must load them in the same order they were saved, and while loading cast them to the appropriate data attribute type.

This is extremely hard to explain, so if you are still having a hard time understanding me I would advise googling "serializable java", the first link is a guide about how to serialize data in java that was released by Oracle.

Good luck, its a step learning curve but once you get it.... it's easier than stealing candy from a baby...

*snatches the lolly and runs like hell*

 

Right, so a modified version of you code isn't killing my comp, thats good, I think.

But It says the file doesn't exist, so I'm trying to use this to make the file:

Code:java

File savefile = new File("plugins\\DecoyBlocks\\DBList.dat");
		if (!savefile.exists()) {
			try {
				savefile.createNewFile();
			} catch (IOException e) {
				e.printStackTrace();
			}
		}

But It says It cannot find the directory?

 

File savefile = new File("plugins" + File.Seperator + "DecoyBlocks" + File.Seperator + "DBList.dat");

 

Yay! It finally works! Thanks guys!

Incase there are any other crazy hooligans looking at this wondering what my code looks like:

Code:java

public class DBDatabase {
	public static DecoyBlocks m;
	public List<Block> decoyList = new ArrayList<Block>();
	public DBDatabase(DecoyBlocks instance) {
		m = instance;
	}
 
	public void SaveList() {
		File savefile = new File("plugins" + File.separator + "DecoyBlocks" + File.separator + "DBList.dat");
		if (!savefile.exists()) {
			try {
				new File("plugins" + File.separator + "DecoyBlocks").mkdir();
				savefile.createNewFile();
			} catch (IOException e) {
				e.printStackTrace();
			}
		}
        try {
            FileOutputStream fstream = new FileOutputStream("plugins" + File.separator + "DecoyBlocks" + File.separator + "DBList.dat");
            ObjectOutputStream ostream = new ObjectOutputStream(fstream);
 
            long recordcount = 0;
            for (@SuppressWarnings("unused") Block b : decoyList) {
            	recordcount ++;
            }
            ostream.writeLong(recordcount);
 
            for (Block b : decoyList) {
            	recordcount ++;
 
            	ostream.writeObject(b.getLocation().getWorld().getName());
            	ostream.writeDouble(b.getLocation().getX());
            	ostream.writeDouble(b.getLocation().getY());
            	ostream.writeDouble(b.getLocation().getZ());
            }
 
            ostream.close();
 
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
 
    public void LoadList() {
        File savefile = new File("plugins" + File.separator + "DecoyBlocks" + File.separator + "DBList.dat");
        boolean exists = savefile.exists();
        if (exists) {
        	FileInputStream ifstream = null;
            ObjectInputStream iostream = null;
            try {
                ifstream = new FileInputStream("plugins" + File.separator + "DecoyBlocks" + File.separator + "DBList.dat");
                iostream = new ObjectInputStream(ifstream);
 
                long recordcount = iostream.readLong();
                for (int i = 0; i < recordcount; i++)  {
                   World w = Bukkit.getServer().getWorld(iostream.readObject().toString());
                   Double x = iostream.readDouble();
                   Double y = iostream.readDouble();
                   Double z = iostream.readDouble();
                   decoyList.add(w.getBlockAt(new Location(w, x, y, z)));
                }
 
            } catch (FileNotFoundException e) {
                m.l.info("Could not locate DBList.data");
                e.printStackTrace();
            } catch(IOException e) {
                m.l.info("I\\O error while attempting to read DBList.dat");
                e.printStackTrace();
            } catch (ClassNotFoundException e) {
                m.l.info( "Could not read DBList.dat, class not found.");
                e.printStackTrace();
            } finally {
                try {
                    ifstream.close();
                } catch (IOException e) {
                    m.l.info( "Error reading DBList.dat, could not close stream.");
                    e.printStackTrace();
                }
            }
        }
    }