Register forum user name Search FAQ

Gammon Forum

Notice: Any messages purporting to come from this site telling you that your password has expired, or that you need to verify your details, confirm your email, resolve issues, making threats, or asking for money, are spam. We do not email users with any such messages. If you have lost your password you can obtain a new one by using the password reset link.

Due to spam on this forum, all posts now need moderator approval.

 Entire forum ➜ MUDs ➜ General ➜ Suggested protocol for server to client "out of band" messages
Suggested protocol for server to client "out of band" messages

It is now over 60 days since the last post. This thread is closed.     Refresh page

Pages: 1  2  3  4  5  6  7 8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  

Posted by Twisol   USA  (2,257 posts)  Bio
Date Reply #90 on Thu 04 Feb 2010 09:42 PM (UTC)

Amended on Thu 04 Feb 2010 09:44 PM (UTC) by Twisol

Message
Nick Gammon said:
Hey, and this is why you don't use 0x00 as an internal delimiter when defining a protocol. :P

No, this is why you don't allow NUL bytes as content. Using NUL as a delimiter is fine, just split by NUL and pass the resultant list.

EDIT: If you take it upon yourself to always split protocol entries by NUL, it might cause issues with other protocols that do allow NUL for one reason or another. This is another of those nuances that I would prefer to be handled by the protocol plugin itself, and I don't think it's unreasonable to suggest that if a protocol uses NUL bytes, you're better off using Lua. The Lua plugin can always make the content available to other plugins, either through broadcasts or through registration-over-PPI.
'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #91 on Thu 04 Feb 2010 09:59 PM (UTC)

Amended on Thu 04 Feb 2010 10:01 PM (UTC) by Nick Gammon

Message
Twisol said:

No, this is why you don't allow NUL bytes as content. Using NUL as a delimiter is fine, just split by NUL and pass the resultant list.

EDIT: If you take it upon yourself to always split protocol entries by NUL, it might cause issues with other protocols that do allow NUL for one reason or another.


So it's fine, but it isn't fine, is that it?

OK, that's it. I just don't have the energy to make another plugin callback calling routine, just to handle one case.

I'll go with Twisol's suggestion. If you want to handle ZMP, you have to use Lua, or make a Lua intermediate plugin to break up the parameters for you.

And I just suggest to the ZMP author, whilst using 0x00 delimiters might be defensible on all sorts of technical grounds, it just is another barrier to implementors, given that the C libraries (and other things like data types in the scripting engines) don't like imbedded 0x00 in strings. Sure, it can be worked around, but it just makes the job harder.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Twisol   USA  (2,257 posts)  Bio
Date Reply #92 on Thu 04 Feb 2010 10:18 PM (UTC)
Message
Nick Gammon said:
So it's fine, but it isn't fine, is that it?

Heh, it's "better" than using nulls as content. Using NUL as part of the protocol itself means it's easier to deal with earlier in the process. Putting them in the actual content creates migraines.
'Soludra' on Achaea

Blog: http://jonathan.com/
GitHub: http://github.com/Twisol
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #93 on Thu 04 Feb 2010 10:22 PM (UTC)
Message
Well, for MUSHclient, to use it for a specific purpose like breaking up parameters means I need to "teach" the client the exact protocol at the low level. However as you noted, breaking up at 0x00 bytes (into a table, say) is very specific to ZMP. If ZMP had chosen any other character as the delimiter (eg. 0x01 which doesn't normally occur in text) then the whole subnegotiation text would be the content, it could be passed to a plugin, and that plugin could say "hey, I'll break it up at the delimiter as I happen to know that is how ZMP works".
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #94 on Thu 04 Feb 2010 10:23 PM (UTC)
Message
Even now it works with Lua, so the basics inside MUSHclient are fine. It just limits its use with any language that happens to use null-terminated strings internally to hold "content".
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #95 on Thu 04 Feb 2010 10:25 PM (UTC)
Message
Quote:
I think we discussed as much as we could possibly discuss. Lasher isn't taking part in the discussion, and he'd pretty much be the key part of any implementation.

Are we developing something for Lasher or as a proposed reusable community protocol?

I don't really care what Lasher ends up wanting to do with Aardwolf, he can do whatever he like and if Nick supports it that's great for Aardwolf. But the nominal topic here is to suggest a standard of some sort, which -- as I understand things -- means that it's not just an Aardwolf feature.

So, I'm just trying to get context here...



A practical concern that has not yet been addressed is what kind of limit is imposed on the Lua expressions, because (as it stands now) some expressions can actually be very difficult to parse in other languages, even without going to actively malicious expressions. A protocol that aims to be simple and language neutral, and yet has this issue, has failed one of its stated design goals.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #96 on Thu 04 Feb 2010 10:39 PM (UTC)

Amended on Thu 04 Feb 2010 10:42 PM (UTC) by Nick Gammon

Message
The topic has really broken into two pieces as far as I am concerned.

Graceful handling of telnet negotiation


This developed after discussion about how a new protocol wasn't really needed, but we did need:


  • A way to easily capture telnet subnegotiations without having to get into delving into incoming packets in a script

  • Removal of the restriction on the length of such content packets (eg. for my proposed system, or any other)

  • Better handling of IAC IAC

  • Fixing some bugs in the current implementation where you could not have imbedded CR/NL/ESC inside telnet negotiations


This goal has been achieved to my satisfaction by adding various new things:


  • Rewritten telnet subnegotiation state machine

  • IAC IAC now allowed inside telnet subnegotiation

  • CR/NL/ESC now allowed inside telnet subnegotiation

  • Telnet subnegotiation can be any length

  • Once completed, the "content" part (the <data> part in IAC SB x <data> IAC SE) is passed to any plugin that wants it, along with x

  • Querying plugins on receiving IAC DO x and IAC WILL x to see if they want to handle it, and if so, replying IAC WILL x or IAC DO x.

  • Support for any telnet protocols (including ones I haven't heard of) by sending all unknown subnegotiation messages to the above plugins. Known ones continue to be handled in the existing way (eg. MCCP).


This effectively supports all protocols, both existing and future, by providing a framework to get at the content (the part inside the telnet subnegotiation) without needing to get to the packet level in plugins.

Effectively, it is irrelevant whether the content is Lua, ATCP, the Aardwolf protocol (which at present is not Lua, but simply various settings), ZMP or whatever.


My proposed protocol for server -> client


My suggestion there stands, although now it would be easier to use any other telnet type (than 102) because the new MUSHclient handler would let you choose 103, 104 or whatever. It will also work more reliably (you don't need to worry about newlines in the "code").

I take your point about someone sending "while true do end" but as a server writer don't really care about that. I won't send that, any more than I will send 2 Gb of data in one message, or put insulting messages in my MOTD.

I think that if I wanted to write my own client/server pair (for instance, a server where *all* traffic was out-of-band) then my proposed method will work fine. I will control the server, and will write the client plugins. I won't moan to myself that, say, Python isn't supported because I am working closely with myself.

However anyone that wants to press on with ZMP, ATCP or any other protocol is welcome to, and the changes in MUSHclient will make that much easier, and in no way limit their ability to do so.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #97 on Thu 04 Feb 2010 10:46 PM (UTC)
Message
I think there are non-contrived reasons why somebody might want to output expressions that involve actual expression evaluation. (You could think of this as similar to "entities" in MXP; you send a definition once in a command, and then reuse it.) Regardless, allowing expression evaluation is not consistent with the idea that this is easy and simple in general. If languages than Lua aren't on the table, that's fine. So this is not a protocol for the server to send "out of band" messages to the client: it's a protocol to do the same using Lua.

Nick Gammon said:
I won't moan to myself that, say, Python isn't supported because I am working closely with myself.

But this isn't really the point, is it? Your comment here makes sense if you're doing something for one MUD. It doesn't make sense if we're trying to develop something that many MUDs can use, potentially with many clients.

This started out sounding like a general protocol, that other clients could pick up, and that people could handle in many scripting languages.
Now it sounds like a protocol meant for some rather specific server to talk to some rather specific client, using a rather specific plugin language, with a rather specific person (or small group of people) in full control of all steps involved.
This isn't a "bad thing" at all, it's just that it's different. As I said re: Lasher & Aardwolf, I really don't care what people feel like doing when it comes to their own client/server pairs. But I do care what happens if we're trying to develop a reusable standard.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #98 on Thu 04 Feb 2010 11:07 PM (UTC)
Message
David Haley said:

This started out sounding like a general protocol, that other clients could pick up, and that people could handle in many scripting languages.


My original post on page 1 was really trying to suggest a method (not propose a standard that everyone should agree on), but a method that a server could use to simply send extra data (like your hit points) to a client, so that client, any client, might display that in a bar, window, speak it out loud or whatever.

Now my suggestion was to use Lua simply because some mainstream clients I know of already implement Lua (MUSHclient, cMUD, Mudlet), so interpreting the data would be simple for them.

Other clients that don't have Lua incorporated could, if they are still being maintained, add it. But if they don't no big deal. However I should point out that any alternative, such as JSON, or XML, or the ATCP, or MCP protocols are not automatically there in every client anyway. So if your objection is "they would have to add Lua to make it work", I would reply, "so what? if you go with JSON then then have to add the JSON libraries, what is the difference?", or XML, or stuff to handle ATCP.

My question is, if you have a server that is based on Smaug, and you incorporated my suggested C code into it, would harm would it do? Your players could then install the health bar plugin if they happened to use MUSHclient. If not, the server would see the initial negotiation failed, and not do any extra work. If so, they get a better experience. Then someone who uses cMUD might do a cMUD plugin as well. Ditto for Mudlet. Then other clients might incorporate Lua.

I don't quite see what the objection is.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Worstje   Netherlands  (899 posts)  Bio
Date Reply #99 on Thu 04 Feb 2010 11:15 PM (UTC)

Amended on Thu 04 Feb 2010 11:18 PM (UTC) by Worstje

Message
Nick Gammon said:

... But when creating the BSTR value it uses SysAllocString and the documentation for that says:


Parameter

sz

A zero-terminated string to copy. The sz parameter must be a Unicode string in 32-bit applications, and an ANSI string in 16-bit applications.


So it doesn't look possible. Hey, and this is why you don't use 0x00 as an internal delimiter when defining a protocol. :P

I can work around it, whilst I still have the string with the nulls, break it up myself at the nulls, create a table, and call the callback with that table.


Why go through all that effort? I raise you SysAllocStringLen (http://msdn.microsoft.com/en-us/library/ms221639.aspx):

Quote:
BSTR SysAllocStringLen(
const OLECHAR *pch,
unsigned int cch
);

Parameters

pch

A pointer to cch characters to be copied. If pch is NULL, a new string is allocated that contains only the null character.

cch

The number of characters to be copied from pch. A null character is placed afterwards, allocating a total of cch plus one characters.


If you are copying non-NULL-terminated buffers into a new non-NULL-terminated buffer, don't use a NULL-terminated buffer creation method. :)
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #100 on Thu 04 Feb 2010 11:22 PM (UTC)
Message
The changes to the code have been pushed to GitHub, if you want to check them out:

http://github.com/nickgammon/mushclient

They are not fully tested yet, but you can see the scope of the changes to date.

- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #101 on Thu 04 Feb 2010 11:46 PM (UTC)
Message
Worstje said:

If you are copying non-NULL-terminated buffers into a new non-NULL-terminated buffer, don't use a NULL-terminated buffer creation method. :)


Well I didn't do it deliberately, that was all in the MFC libraries. However thanks to your pointer I have adapted your idea and it seems to work.
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top
Posted by David Haley   USA  (3,881 posts)  Bio
Date Reply #102 on Thu 04 Feb 2010 11:48 PM (UTC)
Message
Quote:
So if your objection is "they would have to add Lua to make it work", I would reply, "so what? if you go with JSON then then have to add the JSON libraries, what is the difference?", or XML, or stuff to handle ATCP.

There are well-known libraries for handling e.g. JSON or XML in many libraries. To handle "LNL" as you have defined it so far, one must fully embed Lua.

This issue goes away, incidentally, if you use the relatively simple legal expression definition I gave earlier.

Quote:
My question is, if you have a server that is based on Smaug, and you incorporated my suggested C code into it, would harm would it do? Your players could then install the health bar plugin if they happened to use MUSHclient. If not, the server would see the initial negotiation failed, and not do any extra work. If so, they get a better experience. Then someone who uses cMUD might do a cMUD plugin as well. Ditto for Mudlet. Then other clients might incorporate Lua.

It depends on whether you're trying to achieve standardization by design or by accident. MXP attempted to achieve standardization by design but mostly failed due to issues both with the spec and with "famous" implementations of the spec. It sounds like you don't care about standardization, but if other people use it, that's great too.

As a server author, I want it to be easy for players to use this thing I'm going to spend time working on. Prompt data is really only a fairly trivial application of out-of-band data. Invariably, more complex information will be sent (as you are already sending with Aardwolf).
Therefore, to maximize utility for the time I spend, I want to make sure it's pretty easy to use this stuff in other clients as well. If all the other clients already support Lua as a plugin language, that makes life easy -- although then I have to assume that the clients have cross-plugin communication, or I force people to write associated plugins in Lua as well. If clients don't already support Lua, then life gets pretty complicated all of a sudden. Either I fully support the spec and allow full Lua expression evaluation, or I go against the spec and only allow primitives (for example) which then creates confusion because the spec is no longer trustworthy as an indication of what implementations support.

I just don't want to see something that has potential to be great end up falling to the wayside due to issues that we've seen with standardization in the past. We are in a fairly unique situation here of being able to drive a standard, assuming that MUSHclient implements whatever standard we end up deciding upon -- if indeed we're even trying to do that.
David Haley aka Ksilyan
Head Programmer,
Legends of the Darkstone

http://david.the-haleys.org
Top
Posted by Worstje   Netherlands  (899 posts)  Bio
Date Reply #103 on Thu 04 Feb 2010 11:54 PM (UTC)
Message
@ Nick Gammon:

Yeah I understood that much. I specified the logic (although I'll declare guilty at phrasing it a bit smart-alec-like) to point out my reasoning and how I found the replacement function. I knew neither function up till just now. :)

@ David Haley:

I completely agree with your concerns. It is why I put forward the minimized table syntax minus expressions bit a few pages ago.

Of course, personally I am still fond of bencoding. It is written to have very little overhead (not all servers use MCCP), and assuming we'll get out of band data every prompt, I think saving on the amount of characters transmitted is still a quite important endeavour. People might not be using old-fashioned 56k telephone modems where I live anymore, but I know plenty of people in the USA and other countries do still.
Top
Posted by Nick Gammon   Australia  (23,173 posts)  Bio   Forum Administrator
Date Reply #104 on Fri 05 Feb 2010 03:42 AM (UTC)

Amended on Fri 05 Feb 2010 03:46 AM (UTC) by Nick Gammon

Message
Well I tested my new modifications on Achaea with reasonable results.

Armed with this simple plugin:


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>

<muclient>
<plugin
   name="Achaea_Test"
   author="Nick Gammon"
   id="3d13120b92f9f93af9ac6f42"
   language="Lua"
   purpose="Tests Achaea ATCP"
   date_written="2010-02-05"
   requires="4.48"
   version="1.0"
   >

</plugin>

<!--  Script  -->

<script>
<![CDATA[

function OnPluginTelnetSubnegotiation (type, option)
  AppendToNotepad ("Achaea", option .. "\r\n")
end -- function OnPluginTelnetSubnegotiation

function OnPluginTelnetRequest (type, data)
  if type == 200 and data == "WILL" then
    return true
  end -- if
end -- function OnPluginTelnetRequest

]]>
</script>

</muclient>


I got the following results (in part):


Room.Num 23214
Room.Brief Inside the World Tree
Room.Exits e
Room.Num 23214
Room.Brief Inside the World Tree
Room.Exits e
Room.Num 21410
Room.Brief A dimly lit cave
Room.Exits o
Room.Num 21410
Room.Brief A dimly lit cave
Room.Exits o
Room.Num 22698
Room.Brief Upon a flower-laden hill
Room.Exits ne,nw,u,d,i
Room.Num 23812
Room.Brief Path hugging the side of a hill (road)
Room.Exits n,sw
Room.Num 23188
Room.Brief Mud-covered trail (road)
Room.Exits e,s
Room.Num 21494
Room.Brief Beneath a large oak tree on the hillside
Room.Exits w
Room.Num 23188
Room.Brief Mud-covered trail (road)
Room.Exits e,s
Room.Num 21494
Room.Brief Beneath a large oak tree on the hillside
Room.Exits w
Room.Num 23188
Client.GoodBye Farewell, adventurer.


So that is a lot simpler than mucking around grabbing stuff out of packets. :)
- Nick Gammon

www.gammon.com.au, www.mushclient.com
Top

The dates and times for posts above are shown in Universal Co-ordinated Time (UTC).

To show them in your local time you can join the forum, and then set the 'time correction' field in your profile to the number of hours difference between your location and UTC time.

1,010,907 views.

This is page 7, subject is 23 pages long:  [Previous page]  1  2  3  4  5  6  7 8  9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  [Next page]

It is now over 60 days since the last post. This thread is closed.     Refresh page

Go to topic:           Search the forum

[Go to top] top

Information and images on this site are licensed under the Creative Commons Attribution 3.0 Australia License unless stated otherwise.

Home