[Home] [Downloads] [Search] [Help/forum]

Gammon Forum

See www.mushclient.com/spam for dealing with forum spam. Please read the MUSHclient FAQ!

[Folder]  Entire forum
-> [Folder]  MUSHclient
. -> [Folder]  Suggestions
. . -> [Subject]  Patch to serialize.save
Home  |  Users  |  Search  |  FAQ
Username:
Register forum user name
Password:
Forgotten password?

Patch to serialize.save

It is now over 60 days since the last post. This thread is closed.     [Refresh] Refresh page


Posted by Twisol   USA  (2,257 posts)  [Biography] bio
Date Thu 22 Apr 2010 08:26 AM (UTC)
Message
I was explaining how to use serialize.lua to a friend, and I found myself somewhat befuddled by the output of serialize.save. Specifically, I don't understand why it must declare the table globally. Here's an example output now:

tbl = {}
  tbl[1] = 1
  tbl[2] = 2
  tbl[3] = 3


The expected use of this seems to be something like:

assert(loadstring(serialized_stuff))()

That's fine, but 'tbl' is now defined globally, potentially overwriting something in the global namespace. I might also only want to use it locally within a function, also. You might well say that I should be able to trust my own serialized data, but there's no guarantee that this is strictly staying within one plugin, or that I might change a global variable name in the future, and updated plugins will use the old state and overwrite that global.

My suggested patch would output:

local tbl = {}
  tbl[1] = 1
  tbl[2] = 2
  tbl[3] = 3
return tbl


This would allow you to directly control how the data is managed:

-- if you want a global:
data = assert(loadstring(serialized_stuff))()
-- or a local:
local data = assert(loadstring(serialized_stuff))()


It also means that serialize.save doesn't need to know the name of the table to use. Since the loaded chunk has its own inner scope, you can use a specific name (in this case, tbl), and it will never clash, because the serialized table uses the "local" keyword. Thus, serialize.save can remove the "what" argument entirely.


Here's the patch. I don't have a diff tool handy, so I'll just paste by function.

function save (v)
  assert (v, "Must provide a valid variable for the 1st parameter!")

  local out = {}  -- output to this table
  save_item ("tbl", v, out, 0, {})   -- do serialization
  out[1] = "local " .. out[1]
  table.insert(out, "return tbl")
  return table.concat (out, "\n")  -- turn into a string
end -- serialize.save



Alternatively, if you want to keep backwards compatibility (I'm sure you do), ignore the above patch and just provide a serialize.load function like below

function load(name, str)
  local old  = _G[name]
  
  assert(type(name) == "string", "Argument 1 must be a string!")
  assert(type(str) == "string", "Argument 2 must be a string!")
  
  assert(loadstring(str))()
  
  local val = _G[name]
  _G[name] = old
  return val
end


I'd prefer the serialize.save improvement, but this would at least make it more natural to decode:

-- if you want a global:
data = serialize.load("name", serialized_stuff)
-- or a local:
local data = serialize.load("name", serialized_stuff)


You unfortunately still have to know the associated name ahead of time.

'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
[Go to top] top

Posted by Nick Gammon   Australia  (21,322 posts)  [Biography] bio   Forum Administrator
Date Reply #1 on Thu 22 Apr 2010 09:36 AM (UTC)

Amended on Thu 22 Apr 2010 09:37 AM (UTC) by Nick Gammon

Message
You don't need to worry about that. The recommended way of loading the serialized data is into a table that has its environment changed, thus the "global" table you refer to in your post is only global to the environment. For example:


local t = {}
setfenv (assert (loadstring (serialized_stuff)), t) ()


Since you are writing a plugin, you presumably can control the way the data is read back in, and that method is the recommended way.

Your suggested patch of "local" would defeat that BTW.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top

Posted by David Haley   USA  (3,881 posts)  [Biography] bio
Date Reply #2 on Thu 22 Apr 2010 01:19 PM (UTC)
Message
I also prefer the environment method, although I understand there's some kind of big change with how that works in Lua 5.2 (haven't looked into the details yet).

David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
[Go to top] top

Posted by Twisol   USA  (2,257 posts)  [Biography] bio
Date Reply #3 on Thu 22 Apr 2010 06:07 PM (UTC)

Amended on Thu 22 Apr 2010 06:15 PM (UTC) by Twisol

Message
Right then, that's a much better solution. It's not immediately obvious though, nor was it described anywhere I can see.

I'd still be in favor of adding a serialize.load convenience function like below:

function load(str)
  local t = {}
  setfenv(assert(loadstring(str)), t)()
  return select(2, next(t))
end



Nick Gammon said:
Your suggested patch of "local" would defeat that BTW.

In favor of something simpler, I feel. *shrug* It creates a local within the chunk, and returns it. That's more or less how serialize.save_simple works, too; you just have to tack on a "return " before you loadstring it. (It would be convenient to add that directly to the save_simple method instead, or provide a load_simple function for convenience)


David Haley said:
I also prefer the environment method, although I understand there's some kind of big change with how that works in Lua 5.2 (haven't looked into the details yet).

In Lua 5.2, you'd do this:

local t = {}
assert(loadin(t, serialized_data))()



EDIT: Here's the serialize.load_simple function as well.

function load_simple(v)
  return assert(loadstring("return " .. (v or "")))()
end

'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
[Go to top] top

Posted by WillFa   USA  (525 posts)  [Biography] bio
Date Reply #4 on Thu 22 Apr 2010 09:14 PM (UTC)
Message
There's already a save_simple function in serialize.lua. It has the advantage of only pickling what's in between the braces


t={}
foo = serialize.save_simple(t)

print(foo)   -->  "{ }"


Note you're passing in a table and not the name of a table. The drawback of save_simple is that


t={}
t.me = t

print ( serialize.save_simple(t) )


recurses forever or until stack overflow, which ever comes first. ;)




local t,foo
t={}


foo = serialize.save_simple(t)

local newtbl
newtbl = assert(loadstring(foo))()


gives your intended result.
[Go to top] top

Posted by Nick Gammon   Australia  (21,322 posts)  [Biography] bio   Forum Administrator
Date Reply #5 on Thu 22 Apr 2010 09:35 PM (UTC)
Message
Twisol said:


function load(str)
  local t = {}
  setfenv(assert(loadstring(str)), t)()
  return select(2, next(t))
end



Why not just return t?

- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top

Posted by Twisol   USA  (2,257 posts)  [Biography] bio
Date Reply #6 on Thu 22 Apr 2010 10:54 PM (UTC)

Amended on Thu 22 Apr 2010 10:56 PM (UTC) by Twisol

Message
WillFa said:

There's already a save_simple function in serialize.lua. It has the advantage of only pickling what's in between the braces

And the disadvantage that it doesn't work with tables with cyclical references.

To the rest: I know how save_simple works, but thanks.

Nick Gammon said:

Twisol said:


function load(str)
  local t = {}
  setfenv(assert(loadstring(str)), t)()
  return select(2, next(t))
end



Why not just return t?


Because then you don't get out exactly what you put in. You get a table containing the one you put in, and you have to either know its name or use next() to get it back.


EDIT: Actually, there is a benefit to returning 't', and it's for the same reason you use your 'saved' table in serialize.save. But then you lose the advantage I just gave above, because there's no simple way to track what was just added to the table.

'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
[Go to top] top

Posted by Nick Gammon   Australia  (21,322 posts)  [Biography] bio   Forum Administrator
Date Reply #7 on Thu 22 Apr 2010 11:31 PM (UTC)
Message
It's true that the serialize.save seems to create another level, which is in fact the thing you are serializing, eg.


mobs = {}  -- create mobs table

mobs.kobold = {
  name = 'killer',
  hp = 22,
  gold = 5,
  location = 'city square',
  treasure = { "sword", "gold", "helmet" } -- sub table
  }

-- and another one ...

mobs.worm = {
  name = 'gordon',
  hp = 4,
  gold = 15,
  location = 'underground',
  treasure = { "food", "knife" },
  attacks = { "bite", "poison" }
  }


require "serialize"
require "tprint"

tprint (mobs)

s = serialize.save ('mobs')

print (s)

local t = {}
setfenv (assert (loadstring (s)), t) ()

tprint (t)


In the case of a plugin, you serialize (say) the mobs table, and then next time around, in global namespace or somewhere else, you loadstring it. So it creates the mobs table again for you. Your objection may have some weight if you use the setfenv method, because now things are in t.mobs, but if you just do a loadstring, you don't want all the mobs data in _G you want it in mobs.

Please, enough. It is working fine. You just need to be aware of the way it works.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top

Posted by Twisol   USA  (2,257 posts)  [Biography] bio
Date Reply #8 on Thu 22 Apr 2010 11:32 PM (UTC)

Amended on Thu 22 Apr 2010 11:52 PM (UTC) by Twisol

Message
Nick Gammon said:
Please, enough. It is working fine. You just need to be aware of the way it works.


Yes, it works fine... I was just trying to make it more convenient. At least make a note of the setfenv() technique somewhere, or point me to where it says it already? I couldn't find it mentioned.

EDIT: method -> technique

EDIT 2: Thanks, I appreciate it.

'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
[Go to top] top

Posted by Nick Gammon   Australia  (21,322 posts)  [Biography] bio   Forum Administrator
Date Reply #9 on Thu 22 Apr 2010 11:53 PM (UTC)

Amended on Fri 23 Apr 2010 01:14 AM (UTC) by Nick Gammon

Message
http://www.gammon.com.au/scripts/doc.php?lua=setfenv

The exact method using loadstring was mentioned in the first post here:

Template:post=10043 Please see the forum thread: http://gammon.com.au/forum/?id=10043.


I also mentioned the method more than once in the mudstandards forum.

I have added a clarifying post into the main post about serializing variables into a string:

Template:post=4960 Please see the forum thread: http://gammon.com.au/forum/?id=4960.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top

Posted by Twisol   USA  (2,257 posts)  [Biography] bio
Date Reply #10 on Thu 22 Apr 2010 11:57 PM (UTC)

Amended on Thu 22 Apr 2010 11:58 PM (UTC) by Twisol

Message
Nick Gammon said:
(function=setfenv)

Says "Function setfenv does not exist.". :P

Nick Gammon said:
The exact method using loadstring was mentioned in the first post here:

(post=10043)

I also mentioned the method more than once in the mudstandards forum.

I meant specifically using setfenv() with the output from serialize.save, since serializing is a rather more common task than dealing with telopts.

Nick Gammon said:
I have added a clarifying post into the main post about serializing variables into a string:

(post=4960)

I appreciate that a lot, thanks!

'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
[Go to top] top

Posted by Nick Gammon   Australia  (21,322 posts)  [Biography] bio   Forum Administrator
Date Reply #11 on Fri 23 Apr 2010 01:15 AM (UTC)
Message
Modified my post. It was a Lua function, not a normal function.

As for the other forum, I specifically mentioned that method for un-serializing Lua data, however that data might have been received.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
[Go to top] top

Posted by Twisol   USA  (2,257 posts)  [Biography] bio
Date Reply #12 on Fri 23 Apr 2010 01:17 AM (UTC)
Message
I don't think most users of MUSHclient frequent MudStandards. ;) Anyways, I more or less got what I wanted, so I'm happy.

'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
[Go to top] top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.


7,472 views.

It is now over 60 days since the last post. This thread is closed.     [Refresh] Refresh page

Go to topic:           Search the forum


[Go to top] top

Quick links: MUSHclient. MUSHclient help. Forum shortcuts. Posting templates. Lua modules. Lua documentation.

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.

[Home]


Written by Nick Gammon - 5K   profile for Nick Gammon on Stack Exchange, a network of free, community-driven Q&A sites   Marriage equality

Comments to: Gammon Software support
[RH click to get RSS URL] Forum RSS feed ( https://gammon.com.au/rss/forum.xml )

[Best viewed with any browser - 2K]    [Hosted at FutureQuest]