Register forum user name Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.
 Entire forum ➜ MUSHclient ➜ General ➜ 'Harvesting' data from a MUSH.
'Harvesting' data from a MUSH.

It is now over 60 days since the last post. This thread is closed.     Refresh page

Posted by Linda   Sweden  (164 posts)  Bio
Date Fri 18 Feb 2005 12:48 AM (UTC)
Message
Properly speaking, this should probably go under one of the script forums, or maybe the plugin forum, but since I won't know which without determining what approach is best, I guess I'll stick it here. :)

The situation is this:

I have a database of characters on my MUSH. Each entry has its own object, and each of those objects then have the necessary attributes for holding the information about each character.

I'd like to harvest this data from the MUSH in some way and compile it into some sort of standardized database format which I then can import into a web-based database to make it accessible that way too. Precisely what format it should go into isn't decided, however.

My first question, before I continue on to discuss anything else, is simply: does this sound doable? And if so, what would it likely take? A script (what language?), a plugin or ... something else?
Top
Posted by Poromenos   Greece  (1,037 posts)  Bio
Date Reply #1 on Sat 19 Feb 2005 01:12 PM (UTC)
Message
This is easy enough, I have done it to collect item data, you just use Python with a MySQL library (whose name is on my other pc :/). But it's a few calls, I could look it up if you want.
Vidi, Vici, Veni.
http://porocrom.poromenos.org/ Read it!
Top
Posted by Linda   Sweden  (164 posts)  Bio
Date Reply #2 on Sat 19 Feb 2005 04:46 PM (UTC)
Message
When you get a chance, I'd love to see an example of how you do this. :)

Although, it sounds (since you use a mySQL library) as if you put the data straight into mySQL?

Since the mySQL db I'll be using isn't on the same server as the game, I probably won't be able to do a direct transfer. Also, I'll probably need to import the data into a pre-existing tabel structure (I will be using a blog/cms tool called Expression Engine to manage the database), rather than simply create a new one, which is why I was looking to put it into an importable format.
Top
Posted by Poromenos   Greece  (1,037 posts)  Bio
Date Reply #3 on Sat 19 Feb 2005 05:58 PM (UTC)
Message
Hmm, I see. The MySQL server doesn't have to be on the same server as the game, you just have to be able to reach both from your machine. You could use one you setup locally or wherever you want. You can also use ActiveX to create databases, as demonstrated in one of Nick's plugins. You can use this for python: http://sourceforge.net/projects/mysql-python
Vidi, Vici, Veni.
http://porocrom.poromenos.org/ Read it!
Top
Posted by Linda   Sweden  (164 posts)  Bio
Date Reply #4 on Sun 20 Feb 2005 01:41 PM (UTC)
Message
Hrm, looks like I'll have to give this some more thought. :)

Basically, creating a database out of the data from the MUSH straight off isn't what I need. Instead, I need the data organized in a format that I can import into my already existing database. And preferably import into the structure created by Expression Engine.

Lets say, since I think EE can handle this through plugins, that I want the data from the MUSH put into a specific XML structure. Could I do this via a script or a plugin too? And could that be a single large document, or would each character need its own XML file?

Another option would be to create some kind of .csv file that EE can parse, I suppose.
Top
Posted by Poromenos   Greece  (1,037 posts)  Bio
Date Reply #5 on Sun 20 Feb 2005 11:04 PM (UTC)
Message
What format is the database in now? Is it stored on your PC, or is it on the MUD?
Vidi, Vici, Veni.
http://porocrom.poromenos.org/ Read it!
Top
Posted by Linda   Sweden  (164 posts)  Bio
Date Reply #6 on Mon 21 Feb 2005 09:46 AM (UTC)
Message
Its on the MUSH. Format is homecooked (each 'entry' is an object with the 'fields' being attributes).
Top
Posted by Poromenos   Greece  (1,037 posts)  Bio
Date Reply #7 on Mon 21 Feb 2005 12:31 PM (UTC)
Message
Ah, I see. Well then, you could store it as anything, csv, xml (using Python's XML module), MS access, mysql, anything you want.
Vidi, Vici, Veni.
http://porocrom.poromenos.org/ Read it!
Top
Posted by Linda   Sweden  (164 posts)  Bio
Date Reply #8 on Mon 21 Feb 2005 03:20 PM (UTC)
Message
Ah, okay. :)

Has there been any previous discussion anywhere on the forum that you know of how to go about grabbing data and storing it as csv or xml?

I am pretty completely clueless about doing any scripting from the ground up, but I have been known to occasionally be able to figure out how to modify existing scripts. ;)
Top
Posted by Ked   Russia  (524 posts)  Bio
Date Reply #9 on Tue 22 Feb 2005 03:06 PM (UTC)
Message
Storing it as XML is not at all complicated, don't even think it warrants a discussion. Here's an example Python snippet for converting an internal representation (internal for the program this was used in) of a Mushclient trigger into its XML equivalent:


import xml.sax.saxutils as sax

def write_trigger(trig, section=1):
    output = ""
    if section: output = output + "<triggers>\n"
    output = output + "<trigger\n"
    
    for key in trig.keys():
        if key == "send":
            continue
        output = output + "  " + key + '="' + trig[key] + '"\n'

    output = output + ">"
    if trig.has_key('send'):
        output = output + "<send>" + sax.escape(trig['send'], {'"':'&quot;'})
    output = output + r"</send></trigger>" + "\n"
    if section: output = output + r"</triggers>" + "\n"
    return output

def write_triggers(trig_list):
    trigs = "<triggers>\n"
    if type(trig_list) == type(list()):
        for trig in trig_list:
            trigs = trigs + write_trigger(trig, 0)
    trigs = trigs + r"</triggers>" + "\n"
    return trigs


As you can see - it's all about converting a Python dictionary into formatted text, escaping any special characters using xml.sax.saxutils.escape(), which is part of a standard distribution. Reading the data from the MUSH probably wouldn't be any more difficult - you'd just use triggers, convert the wildcards into a Python object, then convert the object to XML as above.
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

26,926 views.

It is now over 60 days since the last post. This thread is closed.     Refresh page

Go to topic:           Search the forum

[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.

Home