Inactive [WEB/DEV/ADMIN] HTTPConsole 0.3.0 - Issue Console Commands Over HTTP [803]

Discussion in 'Inactive/Unsupported Plugins' started by BlueJeansAndRain, Apr 11, 2011.

  1. HTTPConsole - Issue Console Commands Over HTTP
    Version: 0.3.0

    I wrote this because I have a bash script which I use to backup the worlds and do mapping and various other manipulations directly on world data. So, I needed a way for the script to tell the server to save-all, then disable saving to copy the worlds, then re-enable saving. I know there are plugins out there that expose a limited API over HTTP, and there's even one that provides telnet access. But the APIs didn't provide the access I wanted and telnet access is too cumbersome.

    While I was at it I figured I would make it capable of issuing any console command. You never know what you might need :).

    WARNING: If you configure this plugin to listen on an ip address open to the internet, and don't restrict client IP addresses (using the white and black lists) to specific IP addresses you trust, someone will hack your server.

    This is my first plugin... actually this is the first thing I've ever written in Java, so be gentle with my source :p. Suggestions welcome.

    Features:
    • Issue any command over HTTP that you can issue on the console.
    • Change the listener IP address, port, and log-level through the config file.
    • Get back the output of the issued command.
      • This only works for some commands, specifically commands that are not "threaded".
    • Accepts GET and POST (url or json encoded) requests.
    • Client IP address whitelist/blacklist.
    • Host name filtering.
    Trouble Shooting: If it doesn't appear to be working, try the following solutions.
    • Change the "port: 8765" configuration option to a different number. Valid values are 1024 - 65565
    • Make sure your firewall is allowing access to the port.
    Download the plugin Jar

    Source Code
    Usage (open)

    Make requests to http://127.0.0.1:8765/console?command=<command> (assuming you're using the default port and the server is running on localhost). If you're running your server locally, just open up your browser and type http://127.0.0.1:8765/console?command=save-all and SAVE THE WORLD!
    Default config.yml (open)

    Code:
    # The IP address that HTTPConsole will listen on. "any" can be used to listen on
    #  any address.
    # The default is 127.0.0.1
    ip-address: 127.0.0.1
    
    # The TCP listening port that HTTPConsole will bind to.
    # The default is 8765.
    port: 8765
    
    # Controls how important a console message must be to be shown.
    # Possible values: severe, warning, info
    # The default is severe, meaning only severe log messages will be visible.
    log-level: severe
    
    # Controls whether messages beginning with "ConsoleCommandSender:" are filtered
    #  out of the console log.
    # The default value is true.
    filter-command-sender: true
    
    # This sets what ip addresses are not allowed to send requests to HTTPConsole.
    # To add more values, just add more "- ipaddress" lines to the list.
    # Add "- any" (no quotes) to the list to block all ip addresses from connecting
    #  to HTTPConsole.
    # Ranges of ip addresses can be set by using two ip addresses seperated by a
    #  hyphen (-), or by using an asterisk (*) in place of any number.
    # Hyphens and asterisks are special characters in yml files so they will have
    #  to be surrounded by quotes (see examples below).
    client-ip-blacklist:
    #   - 192.168.3.2
    #   - "192.168.4.*"
    #   - "192.168.5.1-192.168.6.254"
    #   - any
    
    # This sets what ip addresses are allowed to send requests to HTTPConsole.
    # All ip addresses and ranges that are valid for the blacklist are also valid
    #  for the whitelist.
    client-ip-whitelist:
    #   Example:
    #   - 192.168.1.100
    #   - "192.168.2.*"
    #   - "192.168.0.1-192.168.0.254"
    #   - any
    
    # This changes the order in which the whitelist and blacklist are checked.
    # Possible Values:
    #
    #  "Deny,Allow"
    #   The whitelist is only checked when an ip address matches the blacklist.
    #   If the ip address matches both lists, it will be allowed. If it matches
    #    neither list, it will be allowed.
    #   This behavior matches the Apache "Deny,Allow" Order directive.
    #   This is the default value.
    #
    #  "Allow,Deny"
    #   The blacklist is only checked when an ip address matches the whitelist.
    #   If the ip address matches both lists, it will be denied. If it matches
    #    neither list, it will be denied.
    #   This behavior matches the Apache "Allow,Deny" Order directive.
    #
    white-black-list-order: "Deny,Allow"
    
    # This is a list of the allowed hostnames (domains) that HTTPConsole will
    #  accept requests on. This means that if someone uses
    #  "http://my.domain.com:8765/console?command=stop" to issue the
    #  stop command to your server, and my.domain.com is not listed below, the
    #  connection will be refused.
    # The one exception to the above rule is that If no hostnames are listed than
    #  any hostname will be allowed.
    # This is similar to virtual hosting except that HTTPConsole only uses the
    #  hostname to allow or deny the request.
    # This feature combined with the cross site scripting restrictions in modern
    #  browsers is designed to make it more difficult for unauthorized people
    #  to target your server from client side (browser) scripts. This does not make
    #  it impossible however as they can always create a proxy if they really want
    #  to send requests from the browser.
    # If any hostnames are listed, requests to bare ip addresses will be denied
    #  unless the ip address is 127.0.0.1.
    # An asterisk (*) in the left most label will match any valid label or labels.
    # At the very least one label and a top level domain (com, org, net, etc.) must
    #  be given per hostname.
    allowed-hosts:
    #   - theanticookie.com
    #   - notch.tumblr.com
    #   - "*.minecraft.net"
    
    # Controls whether connection refused log messages are always sent, or only sent
    #  when log-level is set to info.
    # The default value is false. Connection refused messages will only be visible
    #  if the log-level is set to info.
    always-log-refused-connections: false
    
    Example Backup Bash Script (open)

    This is the backup script I use for my own server.
    Code:
    #!/bin/bash
    BACKUP_DAYS=28
    MINECRAFT_DIR=/home/me/minecraft
    BACKUP_DIR=$MINECRAFT_DIR/backup
    HTTPCONSOLE_URL="http://127.0.0.1:8765"
    
    BASENAME=`basename $0`
    LOCK_FILE=/tmp/$BASENAME.lock
    if [ ! -e "$LOCK_FILE" ]; then
            trap "rm -f \"$LOCK_FILE\"; exit" INT TERM EXIT
            touch $LOCK_FILE
    else
            echo "Backup already running."
            exit
    fi
    
    if [ "$1" == "" ]; then
            echo No worlds specified.
            exit
    fi
    
    TIMESTAMP=`date +%Y-%m-%d_%T`
    
    function console_command
    {
            curl --data-urlencode "command=$*" $HTTPCONSOLE_URL/console
    }
    
    function backup
    {
            WORLD=$1
            SOURCE=$MINECRAFT_DIR/$WORLD/
            TARGET_DIR=$BACKUP_DIR/$WORLD
            TARGET=$TARGET_DIR/$TIMESTAMP
    
            if [ ! -d "$SOURCE" ]; then
                    echo World directory \"$SOURCE\" does not exist.
                    return
            fi
    
            if [ -d "$TARGET_DIR" ]; then
                    if [ -e "$TARGET" ]; then
                            echo Backup path \"$TARGET\" already exists.
                            return
                    fi
    
                    NEWEST_FILE=`ls -t "$TARGET_DIR" 2>/dev/null | head -1`
                    if [ "$NEWEST_FILE" != "" ]; then
                            NEWEST_FILE=$TARGET_DIR/$NEWEST_FILE
                            echo Sourcing Most Recent Backup \"$NEWEST_FILE\"
                            cp -al "$NEWEST_FILE" "$TARGET"
                    else
                            echo No Prior Backups
                    fi
    
                    echo Deleting Backups Older Than $BACKUP_DAYS Days
                    find "$TARGET_DIR/"* -maxdepth 0 -mtime +$BACKUP_DAYS -delete
            elif [ -e "$TARGET_DIR" ]; then
                    echo The backup path \"$TARGET_DIR exists but is not a directory.
                    return
            else
                    mkdir -p "$TARGET_DIR"
            fi
    
            echo Syncing \"$SOURCE\" to \"$TARGET\"
            rsync -a --delete "$SOURCE" "$TARGET"
    }
    
    echo Saving All Minecraft Worlds
    console_command say Backing Up Worlds...
    console_command save-all
    console_command save-off
    echo
    
    for LEVEL in "$@"; do
            echo Backing Up \"$LEVEL\"
            backup $LEVEL
            echo
    done
    
    console_command save-on
    console_command say Backup Complete.
    
    Todo (open)

    • Wait for some log output before completing a request.
    • Return command output.
    • Have the response body be more meaningful than "Recieved Command <command>"
    • Respond to URL or JSON encoded POST requests.
    • Allow the listener address to be changed (not just 127.0.0.1 anymore).
    • Client ip whitelist.
    • Client ip blacklist.
    • Host name filtering.
    • Cookie based sessions. (finished, testing)
    • Get back buffered console output. (finished, testing)
    • Basic/Digest http authorization. (in progress)
    • Command whitelist/blacklist. Allow or disallow some commands.
    • Allow disabling of POST xor GET requests.
    • Write example web page for a console in your browser.
    • Possibly create a new service for restarting the server?
      • Not going to add this because it can be added by getting a plugin that does server restarts and then giving that plugin commands through HTTPConsole.
    • Expose a request handling API to bukkit allowing developers to easily add web service functionality to their plugins.
    • Profit?!
    Change Log (open)


    Version 0.3.0
    • Added blacklist.
    • Added host name filtering.
    • Changed default port to 8765.
    • Bug Fixes
    Version 0.2.1
    • Fixed input stream parsing.
    Version 0.2
    • Reworked logging and output.
    • Handle POST requests (url or json encoded data).
    • Return output from command (easier than I expected).
    • Abstracted a lot of the framework to pave the way for increased future functionality.
    • Listener ip address can be changes via the config.yml file.
    • A client ip address whitelist can be set via the config.yml file.
    Version 0.1
    • Initial Release
     
  2. Thanks :). I find it pretty useful.
     
  3. Offline

    blucat

    Here's the bash script I wrote that makes use of your plugin.

    Code:
    #!/bin/bash
    
    dir=world-backups
    max=240
    
    if [ -z "$1" ]; then
    world=world
    else
    world=$1
    fi
    
    for (( ; ; ))
    do
    curl --silent http://127.0.0.1:9000/console?command=save-all
    
    rm backup/diff_$world.log
    diff -ra -x '*level*' $world/ world-backups/temp_$world > backup/diff_$world\.log
    
    if [ "`ls -lah backup/diff_$world.log | awk '/M/{ print ( substr($5, 0, length($5)-1 ) > 5 ) }'`" -ge 1 ]; then
    
    if [ `ls world-backups/ | wc -l` -ge "$max" ] ; then
    rm `ls -1t world-backups/$1* | tail -1`
    fi
    
    rm -r $dir/temp_$world
    cp -r $world $dir/temp_$world
    zip -rq9 $dir/$world\_`date +"%Y%m%d"`-`date +%s` $world
    fi
    sleep 2000
    done
     
  4. Offline

    BlueFireGaming

    I haven't tested it yet, but it sounds very nice. :)
    Can I also use the plugin to execute commands on an other computer in the server's network?

    I'll probably test it tomorrow.
     
  5. Currently it only listens on 127.0.0.1, which means you can't execute things remotely without some hacking. I suppose I could let people config the listener address, I was just protecting them from themselves :).

    Version 0.2 coming soon (tonight or tomorrow):
    • Reworked logging and output
    • Handle POST requests (url or json encoded data)
    • Return output from command (easier than I expected).
    • Abstracted a lot of the framework to pave the way for increased future functionality.
     
  6. Offline

    fishyfish

    Awesome! Love it, thank you!

    Suggestion/idea: one thing i really miss is a simple wrapper, in other words enabling you to start/restart too. I know there is MCRemotekit (paraphrased), but it lacks console commands though.
     
  7. I'll look into it. This would require that the plugin start a new processes in order to restart the server after the plugin has stopped along with the server. I can think of several ways I might go about that: for example giving the plugin a main method so it's can run as a stand alone jar to perform that task.

    What I'm not sure of is how to determine what file was actually executed to start the server, because for instance, a CraftBukkit server is started with a bash or batch script usually.

    I suppose I could take the easy way out and just make it take a configuration option for restart-command which would have the script/binary + params to execute :). Would that be acceptable?

    I'm also definitely thinking of allowing people to hook into the server instance and handle http requests from other plugins. I saw one or two other plugins out there that also advertised that ability, but it sounds like an interesting thing to figure out :)

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 13, 2016
  8. Offline

    Niles

  9. That's odd, it doesn't even know the http return code for that... I'll give it a shot and get back to you. What are you using to hit that url (curl, lynx, firefox, wget, etc)?

    Version 0.2 released

    Changes:
    • Reworked logging and output
    • Handle POST requests (url or json encoded data)
    • Return output from command (easier than I expected).
    • Abstracted a lot of the framework to pave the way for increased future functionality.
    • Listener ip address can be changes via the config.yml file.
    • A client ip address whitelist can be set via the config.yml file.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 13, 2016
  10. Offline

    Niles

    I am using google chrome and am just about to update to 0.2
     
  11. Offline

    phaed

    Nice, though backups with cp are a bad idea, rather than removing and copying over the whole world every time, keep a mirror of your world which you then keep updated. This will lag your server a lot less during backups.

    Code:
    remove:
    rm -r $dir/temp_$world
    cp -r $world $dir/temp_$world
    
    add:
    rsync -r -t -v $world $dir/temp_$world
     
  12. Pushing some minor changes that might help with the issue you were having Niles. I honestly have no idea why you got that.

    I was being lazy about parsing the input stream and doing a best-attempt at reading the entire stream before parsing it (hey it works for files :p). That can cause some issues with sockets in two cases. It will either see an "end" of the stream and stop reading too early if data isn't ready as fast as the server is trying to read it, or it will read all the way to the end of the stream, which sounds okay except that for some reason it tends to tell client and/or the java socket (I'm not sure which) that you're done and no longer need either the input or the output stream. I think it happens if you read that ending EOF on the input stream.

    Anyway, it now parses the input stream as it reads it and stops at the end of the headers or at the end of content-length data (for post). This is theoretically the "right" way to do it so if you were seeing some cases where it was saying "HTTPConsole: Empty HTTP request recieved" or something to that effect, it was also probably due to this issue.

    Ya, rsync is awesome :D. It's how I do my world parsing. You don't want to zip or create maps from files that might change underneath you. My processing flow looks like this:
    • save-all
    • save-off
    • rsync all my worlds over to my mirrors directory
    • save-on
    • rsync snapshot the mirrors directory to keep a running history of world states.
    • map "nicely" on latest mirror while the server continues on non-the wiser :)

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 13, 2016
  13. Offline

    furyx639

    This Mod has piqued my interest.

    Is there any chance of an HTML-based UI to go along with this plugin?

    I'm going to play around with this. I think it'd be cool if there was a web admin that was aware of all the different functions for different plugin, and could automatically detect what the server has enabled. It'd be nice to be able to have a nice visual/intuitive interface to admin your server and avoid console commands all-together.
     
  14. Offline

    Niles

    Nevermind, it is working now. I think I had something else running on port 8080
     
  15. Ah! k, that makes sense :). I was so confused! I was looking at http return codes wondering if I didn't understand the side effects of some of them. I am planning on adding basic http auth in the future, but usually features don't implement themselves ;)

    Glad you're interested :).

    For an html based gui, the short answer is yes. The slightly longer answer is you could probably already put one together in apache or nginx or whatever web server you use, and it could use this to interface with minecraft. For instance, if you were to install Plugman, you would be able to issue console commands through HTTPConsole to do most of what you just described regarding plugins and their available functions.

    The even longer answer is, I'm planning on opening up the API to other plugins and adding enough functionality that you eventually wouldn't need a third party web server :D. I have some ideas about how to do this, but it will be a week or two before it's ready for testing. When that happens I will probably release another plugin called HTTPWebAdmin (which would have HTTPConsole as a dependency) to demonstrates the API functionality.

    Just tested it with RB #674 and it works just fine. No update necessary. There were no changes should have affected it, but you never know.

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 13, 2016
  16. Offline

    hjhj3686

    what a apply homepage?
     
  17. Pardon?
     
  18. Offline

    hjhj3686

  19. I'm sorry, I don't know what you're trying to ask.

    Tested RB #677. No update necessary.

    Just a little heads up: If I understand java obfuscation correctly, when notch releases 1.5 and assuming there is a server upgrade to go with it, there will probably be some lag time between the minecraft release and a new craftbukkit release. But HTTPConsole should be updated within 24 hours of the craftbukkit release :).

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 13, 2016
  20. Offline

    wickersty

    Are you intending to create a slick html ui for sending commands using the plugin? If not, I'd be interested in putting something together...

    Jeff
     
  21. I was, but you're welcome to as well :). It should get a lot easier once get the API for other plugins to use it setup.
     
  22. Offline

    hjhj3686

    What should I do to connect to a domain?
     
  23. Well, first you have to register a domain, then you'll have to setup the dns for it to point to your ip address, then you can use that domain in place of the ip address.
     
  24. Offline

    Smallie07

    is the link down for the JAR file?
     
  25. Sorry about that. It's back up now.

    EDIT: I changed the downloads to dropbox to hopefully avoid that in the future.
     
  26. Offline

    ks07

    @BlueJeansAndRain

    Wow, thankyou SO much for this plugin! I installed it during my last update to later try implementing it into backup scripts etc... It wasn't until now that I realised how awesome it is to be able to get some feedback on sent commands! (No more stupid screen -X).

    :D Thumbs up from Ultimate Minecraft!

    EDIT: Just one suggestion: add some example usage to the OP to show off the awesome-ness!
     
  27. Offline

    renxwar

    Is there a way you could add so I can look at the console? Like live console support, I guess you could call it. I have my server hosted for me, and I am always wanting to see my console XD
     
  28. Offline

    ks07

    By the looks of the To-Do, something similar might be in the works:
    If you want a full console remotely, you probably want to look at http://forums.bukkit.org/threads/admn-mctelnet-v1-2-4-remote-console-access-using-telnet-677.5299/
     
  29. @ks07 You're welcome :). Heh, adding some examples to the OP is a good idea.

    @renxwar Yup, working on it.

    The telnet access plugin works well. It's what I was using before I wrote this. For plain console access it's pretty hard to beat :). I just got tired of tinkering with expect to script it.

    Just for reference sake, here's my backup script:
    Code:
    #!/bin/bash
    BACKUP_DAYS=28
    MINECRAFT_DIR=/home/me/minecraft
    BACKUP_DIR=$MINECRAFT_DIR/backup
    HTTPCONSOLE_URL="http://127.0.0.1:8080"
    
    BASENAME=`basename $0`
    LOCK_FILE=/tmp/$BASENAME.lock
    if [ ! -e "$LOCK_FILE" ]; then
            trap "rm -f \"$LOCK_FILE\"; exit" INT TERM EXIT
            touch $LOCK_FILE
    else
            echo "Backup already running."
            exit
    fi
    
    if [ "$1" == "" ]; then
            echo No worlds specified.
            exit
    fi
    
    TIMESTAMP=`date +%Y-%m-%d_%T`
    
    function console_command
    {
            curl --data-urlencode "command=$*" $HTTPCONSOLE_URL/console
    }
    
    function backup
    {
            WORLD=$1
            SOURCE=$MINECRAFT_DIR/$WORLD/
            TARGET_DIR=$BACKUP_DIR/$WORLD
            TARGET=$TARGET_DIR/$TIMESTAMP
    
            if [ ! -d "$SOURCE" ]; then
                    echo World directory \"$SOURCE\" does not exist.
                    return
            fi
    
            if [ -d "$TARGET_DIR" ]; then
                    if [ -e "$TARGET" ]; then
                            echo Backup path \"$TARGET\" already exists.
                            return
                    fi
    
                    NEWEST_FILE=`ls -t "$TARGET_DIR" 2>/dev/null | head -1`
                    if [ "$NEWEST_FILE" != "" ]; then
                            NEWEST_FILE=$TARGET_DIR/$NEWEST_FILE
                            echo Sourcing Most Recent Backup \"$NEWEST_FILE\"
                            cp -al "$NEWEST_FILE" "$TARGET"
                    else
                            echo No Prior Backups
                    fi
    
                    echo Deleting Backups Older Than $BACKUP_DAYS Days
                    find "$TARGET_DIR/"* -maxdepth 0 -mtime +$BACKUP_DAYS -delete
            elif [ -e "$TARGET_DIR" ]; then
                    echo The backup path \"$TARGET_DIR exists but is not a directory.
                    return
            else
                    mkdir -p "$TARGET_DIR"
            fi
    
            echo Syncing \"$SOURCE\" to \"$TARGET\"
            rsync -a --delete "$SOURCE" "$TARGET"
    }
    
    echo Saving All Minecraft Worlds
    console_command say Backing Up Worlds...
    console_command save-all
    console_command save-off
    echo
    
    for LEVEL in "$@"; do
            echo Backing Up \"$LEVEL\"
            backup $LEVEL
            echo
    done
    
    console_command save-on
    console_command say Backup Complete.
    
    Usage: mcbackup world netherWorld

    Backs up both the world and the nether world hourly in a cron job :).

    EDIT by Moderator: merged posts, please use the edit button instead of double posting.
     
    Last edited by a moderator: May 13, 2016

Share This Page