Triggers across lines

Posted by Zendu on Tue 20 Apr 2004 06:58 PM — 46 posts, 199,727 views.

#0
I have the text

You smile and congratulate Salazar, conferring upon him the right to wear the

red belt.

In game, and was wondering how i whould make a trigger for it? This is all i could get

^You smile and congratulate (.*?)\, conferring upon (.*?) the right to wear the \x0D\x0A(.*?) belt\.$
Russia #1
Have a trigger matching on the first line enable a trigger that's supposed to match on the second line. When the second trigger matches it should disable itself before doing anything else.
Amended on Wed 21 Apr 2004 09:51 AM by Ked
USA #2
Hmm. It occures to me that you can 'fake' multiline triggering with some forethought. Basically you need:

a) main trigger.
b) pending trigger.
c) disabling the pending trigger of some different line arrives.

Imho it would be so much easier if Mushclient handled the creation of temporary triggers for multilines that either match or get deleted if no match can be found, but you can fake it:

<triggers>
  <trigger
   enabled="y"
   match="(.*?) belt\.$"
   name="Pending_belt"
   regexp="y"
   send_to="12"
   group="pending"
   sequence="0"
  >
  <send>enablescript &quot;pending_belt&quot;, 0</send>
  </trigger>
  <trigger
   enabled="y"
   match="^You smile and congratulate (.*?)\, conferring upon (.*?) the right to wear the"
   regexp="y"
   send_to="12"
   sequence="0"
  >
  <send>enabletriggergroup &quot;pending&quot;, 1</send>
  </trigger>
  <trigger
   enabled="y"
   match=".*"
   name="kill_pending"
   keep_evaluating="y"
   regexp="y"
   send_to="12"
   sequence="1"
  >
  <send>enabletriggergroup &quot;pending&quot;, 0</send>
  </trigger>
</triggers>


So, point is that if all of your triggers fail, then the last trigger disables the pending trigger. This has one 'major' flaw. If you position any trigger in between them, and one of those does not use 'keep_evaluating' then the trigger that needs to disable the one that is pending will never match and never turn off the pending trigger. This is why the sequences are set to 0 for all the multiline triggers and 1 for the one that turns them all off. You could in theory use this trick to nest dozens of lines deep, by having the second trigger enable a third, even as it disables itself, etc. until you run out of stuff to find.
Russia #3
It would be better to keep the secondary triggers and a closing (.*) trigger in one group and disabled though. If you have several multiline triggers running at once and they are all set to the same sequences, then they can possibly interfere with each other, while if each secondary group is enabled by the openning trigger and disabled when you finish searching for a match, it is much less likely to happen.
Australia Forum Administrator #4
Perhaps it is time to revist multi-line triggers. Would anyone be interested in them? ;)
Greece #5
Yes yes, multiline triggers please! I still don't understand why they would be that much trouble to implement... It's just two triggers in one, if the first line AND the second line matches, you trigger... Simple...
Russia #6
Wasn't there a problem with lines falling out of packets or something? I am used to doing it the old-fashioned way actually, and would only be content with that changing if it wouldn't slow the client down.
USA #7
Well. Lines falling out of packets may be an issue, but that is in cases where you try to match letter by letter. A system that took a trigger like:

^You have:$.*$.*$^$

in order to match:

1> You have:
2>  A floopy hat
3>  A sack of corn
4>


could be handled the way other clients do, by trying to read it all at once. *Or* you could use a special temporary trigger system:

Trigger 1: match="^You have:$" activates="Trigger 2"
Trigger 2: match=".*$" disables="Trigger 2" activates="Trigger 3"
Trigger 3: match=".*$" disables="Trigger 3" activates="Trigger 4"
Trigger 4: match="^$" disables="Trigger 4"

or at least some internal logic that does the above. There is no reason that the user needs to even be aware that each segment of the original trigger is being split up into nested triggers that will only become active if/when the first one matches. It would still work basically the same as the existing trigger system, but will automatically be terminated the moment the second pass through the trigger list is made, for the next arriving line, and it doesn't match. In effect, the moment the temporary triggers that have been nested succeed *or* fail, then disable themselves. The next trigger in the chain only gets checked on the next line 'if' the prior one in the chain did. The moment one of them doesn't, the entire chain gets disabled, except for the main trigger they are derrived from.

Of course.. *An even simpler method* would be to have a hidden 'index' in the triggers. When ever a multiple lines are detected in the trigger text, the client would automatically stop testing at the first $ it finds. The index would then be used to direct it to skip past that first $ and attempt to match the next segment when a new line arrives. Each time a segment successfully matches, the index is moved to point where the next line's trigger starts. If the match fails, the index is set back to 0. If the next index position is beyond the end of the entire match string, i.e. the entire string has been tested and the last $ isn't followed by anything, the trigger also resets the index to 0 and starts over.

Of course, for scripting it may be necessary to have a World.GetTriggerIndex ("Trigger_Name") option, so the coder can tell which set of wildcards are being returned. After all the same trigger is passing the text each time, so it will have the same name, but could have completely different wildcards. In my previous example, index 0 would return none, while index 11 and 14 would both return the entire lines, and the blank line at index 17 would also return nothing. Some triggers are bound to be far more complex, so knowing which 'section' of the trigger matched and thus what wildcards to look in for that part is very important.
Australia Forum Administrator #8
Quote:

Wasn't there a problem with lines falling out of packets or something?


No, that related to triggers matching lines before a newline was received. That has been addressed by the plugin OnPluginPartialLine callback.

I have resisted this before because I think it would be clumsy, but perhaps there is a way ...

I think what would work would be to specify a number of lines to go back, and then MUSHclient would assemble a text block consisting of "x" lines (eg. 5 lines) by working backwards through the output window, putting newlines between each line (with a linebreak) and then feed the lot into a multi-line regular expression.

Yes, it may be slower than normal triggers, but it would be optional, so it would only apply if you wanted it, and in that case would probably be *faster* than trying to do it in VBscript.

With a judicious choice of the number of lines, and the regular expression, it should be possible to match on a lot of cases (eg. multi-line room descriptions), or the case at the start of this thread, without too much trouble.
Australia Forum Administrator #9
An interesting side-issue here is what would happen with omit from output lines? Here is an example ...


The Shining Emerald
A locked display box dominates this particular shop.  Under the glass
of the box are a myriad of gems, each lying gently upon a miniscule cushion.
Hanging from the walls are various paintings, while sculptures take up the
rest of the floor space.  To the east is Darkhaven's courier, while an
opening in the tent to the north leads back out onto Market Street.

Exits: north east.
The jeweler smiles warmly.


Say you have a trigger that omits from output "Hanging from the *" (the line in bold). Then you use a multi-line trigger to match the description. Since the omitting occurred earlier (the other lines hadn't arrived yet) does the omitted line count, or not?

I think the answer is "yes" - the multi-line triggers will match what actually arrived, not what happens after other triggers get to them. I think this also simplifies the code.
Amended on Thu 22 Apr 2004 10:20 PM by Nick Gammon
USA #10
Hmm. Any reason why my indexing method wouldn't work? Tracing backward takes extra time and still runs into the same issue as existing triggers, there are only so many wild cards it could return and how the heck do you deal with a situation where you need to look at the entire result of the match? By allowing the trigger to shift and index farward to the next section to match, you in effect match each line seperately and can handle those lines and wildcards in them seperately, but still use only a single trigger. What you are talking about sounds a lot more clumsy, slow and not exactly that functional.

Now my idea would require the user to 'save' wildcards themselves between matches, so that if you got two on the first line and one on the second, you could use them all together, but that is trivial imho and a lot easier that trying to make sure you don't exceed the 'per trigger' limit on wildcards.
Australia Forum Administrator #11
Er, I don't directly control the trigger match routine, that is the PCRE code which is an externally-written thing.

Thus, "stopping at a $" is not particularly easy, and indeed may not solve things like this:

^You see (.*)$^Exits lead (.*)$

The first wildcard in this example may be multiple lines - not just one. However I have to test to see if that works. :)

I think the tracing backwards can be avoided by maintaining a deque (STL double-ended queue) which is the last x lines (say, 100) which can be quickly used to build a text block of (say) 5 lines with \n between them.
USA #12
Well. I am thinking in terms of adding a bit of extra logic to the triggers, something like:
if regexp = "y" then
  a = len(match)
  cmatch = right(match, a - index)
  b = instr(cmatch, "$")
  if b and b < len(cmatch) then
    regmatch = left(cmatch,b)
    index = b
  else
    regmatch = cmatch
    index = 0
  end if
  result = ParsePCRE(cmatch)
  if result then
    ... *do the stuff for a match*
  else
    index = 0
  end if
else
  ... *stuff for normal triggers*


This may add a 'tiny' bit of overhead to regular expression triggers, but it would be very minor compared the what you are thinking. Or at least I would think so.
Greece #13
About the text omitting, I would say that it should be a precedence thing, i.e. the trigger with the higher order should take precedence, but that would require a lot of extra code, so I think you would be better off just writing a rudimentary multiline trigger system for starters, which may even be adequate for everything we need. If not, you can change it later... Also, for something irrelevant, I think that MUclient versions are being released a bit slowly... How about having beta versions (i.e. v. 3.47b build whatever) which we could download and help you test. That would be like the current system where you released versions 3.46 and 3.47 on the forums, but it wouldn't change version numbers this frequently, since they would be different builds... I don't think packaging another build and uploading it is too hard to do...
Amended on Tue 15 Jun 2004 07:08 PM by Poromenos
Australia Forum Administrator #14
I don't think pulling apart the regexp at the $ signs will be a big success.

For example, here is a multi-line trigger that matches my inventory:


<triggers>
  <trigger
   enabled="y"
   lines_to_match="8"
   match="^You are carrying:$((?s)(.*))^$"
   multi_line="y"
   regexp="y"
   send_to="2"
   sequence="100"
  >
  <send>Match = %0
</send>
  </trigger>
</triggers>


The important line is the match text, which you notice contains two $ signs, however it actually matches up to 8 lines, eg.

output


You are carrying:
     a magic mushroom
     a torch
     a dragonskin
     a bag
     recall scroll


matching text



Match = You are carrying:
     a magic mushroom
     a torch
     a dragonskin
     a bag
     recall scroll


Thus %0 is a multi-line match (containing the newline character).

The (?s) is a "dot matches all" option to the regexp, which means the "." character can match newlines, which it normally doesn't.

As for release times, I should be able to release this one for testing soon.
USA #15
Umm.. Frankly I would have never considered actually specifying the number of lines to expect. It frankly makes no sense to do so. Exa:

^You are carrying:$((?s)(.*))$You are carrying (.*) items\.$

wouldn't work. Which part are you going to match multiple times, the first, middle, last or every item? Take for instance:

You see .....
and a large toad.

and

and a large toad.
You see .....

It appears under your logic, both of these would create a match, but the point of multiline triggers is to match known information that spans multiple lines in a *specific* order. There are imho far better ways to match lists and it makes no sense at all to use a multiline trigger with a predefined number of lines to match an inventory that could contain more or less lines that you ask it to. And if it does work in the right order, then there are still cases where it doesn't make sense. It is also certain to be slower than necessary, the behaviour is frankly going to confuse people that use such things in other programs and if I am right can even be completely wrong in cases like my example above.

Ok, maybe for a list with a known length where you have a single line that starts it and you *know* that it will only have x lines, this is an interesting idea, but the other 99.9% of the time it makes no sense. However, even for such lists, this would make more sense:

T1> "^You are carrying:$" - enable T2 and T3.
T2> "^((?s)(.*))$" - calls script, etc.
T3> "^((?s)(.*))$^$" - disables T2 and T3.

You get the exact same result, it still works as intended and it doesn't need to know how many lines to look for. This to me makes sense and is how 'everyone' that uses other clients that support multiline matching would expect to handle it.

So, if you really have a good reason for your method... But I frankly just don't see it.
Australia Forum Administrator #16
Huh? Matching in any order? That wasn't my intention. My earlier example, which works, was this:

match: ^You are carrying:$((?s)(.*))^$

Analysing that, you are matching:

You are carrying:
(x lines containing your inventory)
(a blank line (^$) )

Where "x" is limited to the number of lines you specify as the trigger match limit (eg. 10). As triggers are still matching in sequence, it will only match lines that occur in that order, not some other random collection of lines which might happen to have "You are carrying:" somewhere in them.

I think I see what you are getting at with the wildcards, but it is no different than before. It will try to match where it can. I have done an example with a who list. Here is the list:



-----------------------------------[ IMMORTALS ]------------------------------

Supreme Entity  Nick the Recruit.
Supreme Entity  Shadowfyr the Supreme Entity.
2 players.


This has two wildcards in it - the list of names, and the number of players (2 in this case). Now to make it simpler the "2" could be represented as \d* however to demonstrate the point I'll use normal wildcards. Here is the trigger:


<triggers>
  <trigger
   enabled="y"
   lines_to_match="10"
   match="^\-*\[ IMMORTALS \]\-*$(?s)(.*)^(.*) players?\.\z"
   multi_line="y"
   regexp="y"
   send_to="2"
   sequence="100"
  >
  <send>Match = 
"%0"</send>
  </trigger>
</triggers>


And this is what I see:


Match = 
"-----------------------------------[ IMMORTALS ]------------------------------

Supreme Entity  Nick the Recruit.
Supreme Entity  Shadowfyr the Supreme Entity.
2 players."


Thus the matching text is the whole thing. Let's look at wildcard 1:


Match = 
"

Supreme Entity  Nick the Recruit.
Supreme Entity  Helen the Supreme Entity.
"


That is indeed the multiple lines between the opening (identifying) line and the closing line. This seems fine to me.
Amended on Fri 23 Apr 2004 01:19 AM by Nick Gammon
USA #17
Ok.. I see what you mean. I still think it is inconsistent with what people will expect them to do. You are adding a feature that accounts for a situation that people don't use these kinds of triggers for in the first place. Most of the time if I am trying to match multiple lines it isn't going to be a list. It will be more specific information like a room description, that may in fact appear in a dozen different rooms, but with 'one' room having an extra line that I am looking for. Or something like the example that started this thread, which involves two lines that are technically part of the same line, but just happen be be broken up by the mud before the client gets them. Your adding functionality, extra settings people are not likely to expect and additional overhead to something that most people would never even consider using the way you suggest imho.

I wish a few other people that have talked about this feature before would post comments about it, because I expect that while some would consider your design to be useful, most would have never suggested such a radical shift from the way such triggers work in existing clients. They certainly wouldn't expect it in your implimentation. I don't think that specifically telling the client to look for x lines is necessary or so useful that it is a must have feature, nor do I think requiring that people add extra settings to make multiline matching work is an improvement over my idea, which simply extends the behaviour to do what a number of people expected it to do in the first place.

Again, *lists* are not the reason people are looking for this, they just want the client to behave the way the existing specification for regular expressions and multiple lines as well as they way they work in existing clients that support such. Who, inventory, etc. are literally at the bottom of my list of things I would use such a trigger for and I can think of 0 posts on this forum that even suggest that they are a problem that needs to be fixed. In most cases where I would want/need to match multiple lines, I know what the end looks like and what starts it, but I haven't a clue how many lines I need to have to look through to complete the match. If I ask it to look too far back, then it could even read the wrong line as the start. For example, lets say I am capturing gossip traffic:

[gossip] fred: Yeah. That was fun.
[gossip] Jim: Well, not really. After all...
...
...
[sales] grung: WTS fred.
Youn enter a forested area ...
...
...

Exits: ...
[gossip] fred: True.

Ok... How the heck do I capture multiple lines in this case if I have to tell it which lines to look for? You can't, because it can be anything from 1 line to whatever the buffer limit of the mud is, which is about 12-13 lines on Ages of Despair. If I set it at 13 lines, your method would look back 13 lines and capture the first "[gossip] fred: Yeah..." line, even though it *should* be capturing the line "[gossip] fred: True.". Also, in the above example, there isn't going to be any 'end line', but there may be a change in color that is just as critical to notice as the stop point. I expect you have it stop if it ever fails to match, but if someone dumps some ansi graphics in there right after one of those lines, is it by pure coincidence going to match anyway?

Basically -

1. I can see cases where it can completely fail or doesn't work.

2. It breaks both the expected behaviour of regexp, which only matches what you tell it, not 'everything between these lines'.

3. It requires an extra setting people are not going to expect.

4. It isn't compatible with any existing implimentation that people transitioning to Mushclient are going to be expecting.

5. Anything it does with the 'find x' lines trick can be done more effectively in other ways.

6. It does not and cannot fix the problems that already exist with capturing lists, since it only provides a way to match a special line at the end of the list, which isn't always possible even using existing methods.
USA #18
Oh.. BTW, doesn't regexp already allow for what you are doing the hard way anyway:

match="^\-*[ IMMORTALS ]\-*$((?s)(.*)){,10}^(.*) players?\.\z"
                                     ^^^^^


Or am I confused?
Amended on Fri 23 Apr 2004 02:06 AM by Shadowfyr
Australia Forum Administrator #19
Quote:

Frankly I would have never considered actually specifying the number of lines to expect. It frankly makes no sense to do so.


Actually, it works around the problem that I had years ago when I explained why it wasn't easy to do multi-line triggers. (Now that I want it I can't find a forum post that covers that exact subject in detail, giving all the reasons).

The problem with multi-line triggers is *limiting* the trigger somehow. If you have something like:


You are carrying:
  (x lines here)
  (blank line)


Where "x" could be 1 to 70, say, then how does MUSHclient know when to start matching, and when to stop? Clearly it can't when it gets the line "You are carrying" because it doesn't have the rest yet, and how does it know when to start matching if the lines that it is matching on are themselves variable?

The solution I thought of, which you don't have to use if you don't want to, is to allow you to specify a limit (eg. go back 60 lines). This then lets MUSHclient assemble a block of lines (60, say) and pass them to the regexp processor.

Clearly the more lines you specify the slower it will be, however if you specify too few, you may miss a match (eg, 5 would be too few if you had 10 items in your inventory).

Thus, you compromise between speed and whether you match a lengthy sequence or not.

What you could do, if you wanted to speed things up, would be to have a normal trigger match the first line (eg. "You are carrying") and have that enable the multi-line trigger. That means you know that the multi-line trigger should match soon. Then when the multi-line trigger matches it could disable itself, ready for next time.


(edit) This was written while you were writing your recent post, so it doesn't address the issues you raised there.
Amended on Fri 23 Apr 2004 02:09 AM by Nick Gammon
USA #20
Well yeah. My point is that I think regexp already provides and internal limitation on that, though you are right, it would complicate things a bit to check to see if something ending with a $ is in () and followed by such a limiter. In most cases though the way the matching multiline trigger are designed, there is an assumption that they will only match as many lines as there are $ in them, not including of course anything that followed the last $. Basically, if you tell it ...$..., it should always be two lines, unless you actually do something silly like using (^ ... $)+, which would make the number theoretically unlimited. That could definitely be a problem, but a user error type problem, not something the client necessarilly has to fix for them.

In any case, I expect that using a look-back would be bound to add extra overhead that is not really needed in most cases it will be used, if at all. That is more worrysome that needing to say build an endless loop manager that only lets stuff like {1,} or + match 50 lines before it automatically bails out as a safe guard. Most will not be that open and will already match by definition a very specific number of lines.

At this point I am getting the feeling that maybe you are mixing up two similar but completely unrelated issues.
Amended on Fri 23 Apr 2004 02:54 AM by Shadowfyr
Australia Forum Administrator #21
I really don't see how that will work. Your suggestion of counting "$" signs falls down on a number of counts. For one, regexps allow for .* to match newlines if you use the "dotall" option, thus there could be *no* "$" in the regexp.

Second, you gave the example of match room descriptions, or gossips. Here is a room description:


Darkhaven Academy
You stand inside the Darkhaven Academy, an establishment designed to teach
the basics of play inside this game.  Each room has a specific purpose and
contains information on the various commands to maneuver around and interact 
with the players.  We recommend you explore the Academy in full, taking the 
time to read the instructions in each room.


And another:


North-West Corner
This area will allow you to further hone your combat skills. Kill the beasts
that wander here. If you wish to leave, exiting up will take you to the room
for Toric, the Lord of Healing, in the Darkhaven Academy.


They have different numbers of lines. How can I make a regexp that counts "$" signs to match a variable number of lines in a description? I can't. (I think I see where you got this idea, zMUD seems to do just that).

The regular expression matcher (which I didn't write) is called like this:

match = regexec (regular_expression, text_to_match);

Now the regular_expression bit is easy - you supply that. But the text_to_match? How do I assemble the text if you want me to match multiple lines, and I don't know how many are going to match (eg. a multi-line room description)?

In your simple example, of say two lines, in my system you would do this:

match = line 1 \n line 2
number of lines = 2

That will be as quick as any other method.

However for multi-line things (like room descriptions) you basically have to give a limit (eg. 10 lines) and it will go back that many lines and try to match for you.
USA #22
Hmm. I think the problem here is that such expressions in most clients don't use 'dotall' and .* to mean everything including the end of a line. This is probably why most clients don't even provide that as a possibility. In the case of room description like you mention, or even with channels, there may be no way to know how many I need to match. Even attempting to specify a number will result in incorrect results. You can't tell the trigger how many to look for unless there is something very specific, like a color change or obvious change in a line. I doubt other clients do any better of a job with that sort of thing than mushclient, even with multiline ability. You method flat out cannot and does not solve it. But the fact that people transitioning to Mushclient, that wonder why the heck they need to tell it to match two lines, when the match text appears to already specify that, are likely to be a bit confused to be more of an issue.

Your concept works, but only in cases where you are sure of how many lines for it to look for. Mine can't match multiple instances of the same line in a group of lines (and making it do so would be complicated, so imho should probably produce an error if attempted), but behaves as I expect it to in the first place when defining a normal multiline trigger. Both are useful, but both have flaws as well. I am not sure that telling something to match:

^You smile and congratulate (.*?)\, conferring upon (.*?) the right to wear the $(.*?) belt\.$

as in the start of this thread makes sense to also require that you tell it two lines. It should be implied in this case. Allowing someone to specify 'dotall' to mean 'everything including the end of line' is frankly just complicating what 'should' be very simple. It makes sense in a text parser with unlimited run time and no interactivity, but having to artificially limit it, even when the limit is implied, in a client to make it work right means you didn't design the implimentation right in the first place imho. Again, there are better ways to do lists, which don't require obfuscating syntax that people migrating to mushclient are going to expect to work another way. Most people are never going to use it the way you suggest and as I said before, there are already ways to handle those cases when it does make sense.

Now.. I 'may' be willing to accept it is the line count was automatically set when the expression can and does imply a specific number, but I don't think it is reasonable to force people to set it manually when it is completely unneeded or setting a special flag to activate what *should* be implied in the expression in the first place.

Frankly, I doubt I would even try using such triggers for capturing room descriptions or channels or anything like that anyway. If I had to guess, though they make useful examples. I would be extremely surprised if even 1% of the uses for multiline triggers exceeded 2-3 lines or *ever* tried to capture generic unknowns like lists. I have never seen any such use, possibly because other client designers didn't think that was or should be a feature of multiline triggers any more than I would have thought it was, until you suggested it should. I would rather that the syntax used do what I specified and generate an error when you try something stupid like 'dotall' or (^...){some_limits}, which won't break the behaviour people are expecting (by requiring extra settings) and that there be a seperate feature for what you are suggesting that does break it, but provides added functionality. If you forget about the whole 'what happens if it needs to match on more lines than there are $' issue, then it may be technically broken with respect to the full regular expression syntax (but what client isn't), but it doesn't require all the extra junk you are doing to make it work, unless you really truely absolutely need it.

splitting out the two problem and not trying to treat both as equal situations should make everyone happy. The overhead for implied limits would be limited and apply to 'all' regular expressions, while your extended function would work for those cases it makes sense to do it the hard way. No one said you had to implement everything standard regular expressions do, including the concept that 'dotall' really means everything. For a mud client allowing that is imho probably a stupid idea anyway and not particularly useful.
Amended on Fri 23 Apr 2004 04:03 AM by Shadowfyr
Australia Forum Administrator #23
Quote:

Most of the time if I am trying to match multiple lines it isn't going to be a list. It will be more specific information like a room description

...

Frankly, I doubt I would even try using such triggers for capturing room descriptions or channels or anything like that anyway.



Can you clarify what you want to match multiple lines for? Your two posts here really contradict each other. First you say you want to capture stuff "like a room description" and then you say "I doubt I would even try using such triggers for capturing room descriptions ".


Quote:

Your concept works, but only in cases where you are sure of how many lines for it to look for.


The "line count" I specify is only a maximum number to go back. You can specify an "anchor" (eg. \z) at the end of the regexp to anchor it to the end of the group (as I have done in some examples here) so that if you have two inventories it won't find the first instead of the second.

Quote:

Allowing someone to specify 'dotall' to mean 'everything including the end of line' is frankly just complicating what 'should' be very simple.


I can't agree with that. Allowing a match of multiple lines (eg. a room description) seems to me to be the very thing that multiple line matching would be about. Matching multiple lines, including newlines, and with an indefinite number to match. The line count is only an upper limit, for speed purposes. For instance, if your inventory is normally 15 items or less, and you want to capture it, then you would set the line count at 15. However that handles cases of 14, 13, 12 and so on as well.

In the case of the example that triggered off this thread, he could do it like this:


<triggers>
  <trigger
   enabled="y"
   lines_to_match="2"
   match="You smile and congratulate (.*?), conferring upon (.*?) the right to wear the\nred belt."
   multi_line="y"
   regexp="y"
   send_to="2"
   sequence="100"
  >
  <send>match = %0
%%1 = %1
%%2 = %2</send>
  </trigger>
</triggers>


Testing this reveals this match:


match = You smile and congratulate Salazar, conferring upon him the right to wear the
red belt.
%1 = Salazar
%2 = him


This seems pretty natural to me, and is pretty-well what he asked for in the first place.



Amended on Fri 23 Apr 2004 04:12 AM by Nick Gammon
USA #24
Well.. I used them as an extreme example, but virtually every case I have seen has either been someone saying "I can do this in zMud, but how do you do it in mushclient" or are otherwise trying to match fairly specific cases where the number of lines are implied by default. The problem involving lists doesn't show up when trying to search on multi-line issues, because none of us expected that sort of ability in such a trigger. The problem was more of a "how to get it to stop, if we don't know where it will end." Tacking a termination line on it doesn't work, since the cases where these problem happen don't have any line you can match on to stop the capture, multiline triggers are never even mentioned in those threads.

Yeah, you saw it as a problem, but we never thought of it as one, because we never expected such triggers to match more than a few lines in the first place. We never mentioned lists and multiline triggers in the same threads, *that I know of*.

All I am saying is that in general, the same problem exists with your trigger as already exists with existing methods. You haven't really solved anything that isn't still a problem for other methods and you have added the need to specify the number of lines to look at, even in cases where the trigger itself implies that it can't match more lines than that anyway, which is redundant. As for 'dotall', as I said, I don't plan to use it, people migrating from zMud won't expect it, other clients don't allow it and providing it adds extra settings that are unnecessary for most uses, and doesn't fix the original problem that lists, without known termination lines, already have using the existing methods. What exactly does the extra overhead and the requirement for use to specify how many lines to look for actually provide? Not much as far as I can see, it just adds one more setting we have to import or set in scripting to get the trigger to work right when we do use it.

But that is unfortunately only my opinion, Magnum and most others probably haven't even bothered to check this thread, since it doesn't indicate that we are discussing a feature to be added, not just someone elses problem. For that matter, I haven't seen Magnum on the mud for a few days, so he may not have even seen this thread. I would prefer more feedback from some other people on the subject, especial those that previously asked about the implimentation of such a feature in the first place. :(
Amended on Fri 23 Apr 2004 05:00 AM by Shadowfyr
Australia Forum Administrator #25
Quote:

... we never expected such triggers to match more than a few lines in the first place ...


So it does *more* than you expect? Excellent!

To cover your other points:

  • This is an optional extra. If you don't specify the multiline flag then triggers will behave the same as before, so writing plugins etc. will not change
  • By using a reasonably powerful regexp I can see lists being matched fairly well. For instance:


    Match: You are carrying:\n(([ ]{5}.*\n)*)(?![ ]{5})\z


    This will match my inventory list starting with "You are carrying:", capturing lines starting with 5 spaces (the inventory) and stopping at a line that *does not* start with 5 spaces. The final \z anchors it to the end of the buffer, so if it happens to have more than one inventory in it, this trigger will match the latest one.

    Effectively this encapsulates the original idea of 3 triggers (first, middle, last) into a single trigger which doesn't need enabling/disabling, so it seems neater to me, not more complex.
  • It won't break existing behaviour, as you have to turn it on.
  • For people that just want to match 2 consecutive lines, like the example at the start of this thread, it does just that, in almost exactly the way the thread originator expected.
  • As for "automatically counting lines" - apart from the issue that the line count could vary, if you are matching a 3-line sequence, that you see in front of you, it isn't too complex to type in the number "3" into the "number of lines" box. I suppose it could count \n characters, but in the example above of the inventory, the \n is inside a "repeat count" which could be any number, so it would get the count wrong.


Comments from other "multi-line" users are also solicited. :)

Amended on Fri 23 Apr 2004 07:11 AM by Nick Gammon
USA #26
Car planes do more than expected to, but they tend to perform badly as both cars and planes. Not to mention the inconvenience of having, for most designs, to dig the wings out of the trunk to strap them on every time you need to fly one. Using /n and telling it the number of lines it needs to look at, even when it is technically redundant, just seems cludgy and is definitely not the behaviour any of us where looking for. Yeah, it does more than we asked, but in some cases that isn't a good thing. I personally think that what you are doing makes sense as a special list reader, but that making regexp behave more like they do in other clients when using $ is a different issue. Trying to shoehorn both the wanted behaviour and something requiring an extra setting to work all into one function isn't practical. I could care less about how easy you think it is to tell it '3 lines', it still isn't directly compatible with existing implimentations and the behaviour of regexp, unless you explicitly give it a number is still *broken*.

It is also an example of why most clients impliment their own modified regexp system, instead of relying on one that was never intended to parse streaming text from an interactive source. I seriously doubt any of them have to even worry about matching multiple lines using this method. It doesn't make any sense for them to do so, unless it is static text. Your trying to turn what should be a line by line matching system into one that attempts to manipulate the mud output as static. This is not the way regexp in mud clients do things for a very good reason, the lines can change (as you have pointed out) in ways by other triggers that make them unparsable. Also, you *only* want to parse the *muds* output, not anything you may post as a note in the same window. If the note is timed output, it could very easilly end up in the middle of what is being parsed, a possibility which also breaks the expected behaviour, which should only parse the mud output, not anything else.
USA #27
Oh yes.. And also:

> Thus the matching text is the whole thing. Let's look at wildcard 1:
>
>
> Match =
> "
>
> Supreme Entity Nick the Recruit.
> Supreme Entity Helen the Supreme Entity.
> "

Ooh! Brilliant, if what I wanted to do is capture every name seperately as one wildcard and deal with them individually, your method slaps me with one huge wildcard that I am now forced to reparse all over again to return the results I originally wanted... How is that an improvement over using the three trigger method to capture each line as it arrives? Having wildcard 1 contain the 'entire' list would be completely useless. I doubt anyone else would disagree. In then end I end up doing more work to retrieve the results than I did making three seperate triggers to start reading the list, capture the list contents item by item and finally terminate it.

I was so busy arguing about the implimentation of simple muliline triggers that I failed to notice just how (forgive me for saying so) dumb this idea really is. It doesn't make it easier to capture lists, it actually requires extra and normally totally unnecessary work to use the information it captures in any useful way.
Australia Forum Administrator #28
OK, thanks Shadowfyr, I gather you don't like the idea? ;)

I was interested to look into your claims that "other clients" handle it much more easily. I checked out zMUD, and I notice a couple of things. First, in the help file I see this:

Quote:

Sometime you just need to capture a bunch of lines. Things like who lists, room descriptions, inventory lists, etc. can be tough to grab with single triggers. This script captures multiple lines and stores them in whatever format is needed, this is a highly advanced script, and while I will try to explain it thouroughly it may still be difficult to follow.


Then a lengthy script follows which is similar in concept to what MUSHclient users have needed to do so far, enable/disable multiple triggers.

However I also notice that simple multi-line capture is possible by simply using the "$" symbol. I presume it must count them and go back that many lines, which is what you want, I gather.

But, that is using his own regular expression matching syntax, which he has control over the parsing of. Using the publicly available (and more powerful, if I say so) PCRE syntax that wouldn't be as easy.

I note also that zMUD is now offering PCRE regexp matching as an option, I wonder if multiple lines are supported if you use that?

Quote:

Ooh! Brilliant, if what I wanted to do is capture every name seperately as one wildcard and deal with them individually, your method slaps me with one huge wildcard that I am now forced to reparse all over again to return the results I originally wanted...


You use Split on the newline character. Or, the new table-management code in MUSHclient if you are not using a script language that supports Split or something similar.

I don't think there is an easier way to handle multiple lines, where you might have 50 matches.

However, again, you don't need to do it that way if you don't want to. Use the old method if you prefer it.

I think however this system would make things like stat rollers much simpler. Something like this:


Rerolling please wait.
-====================== CHARACTER ATTRIBUTES SELECTION ======================-
            Strength (ST): 30( 77)   Constitution   (CO): 33( 53)
            Quickness(QU): 40( 40)   Agility        (AG): 52( 52)
            Presence (PR): 24( 84)   Self-Discipline(SD): 29( 39)
            Empathy  (EM): 20( 68)   Memory         (ME): 40( 40)
            Intuition(IN): 26( 47)   Reasoning      (RE): 29( 39)
            
Are you happy with these attributes?


This could be a single trigger with a single regular expression (with \n to indicate line breaks), rather than the previous method of about 7 triggers that have to turn each other on and off, remember variables and so on.

I really can't see what the big objection is to that. :)
Australia Forum Administrator #29
Quote:

Also, you *only* want to parse the *muds* output, not anything you may post as a note in the same window.


My system does exactly that. It keeps track of the last 100 lines of output from the MUD, and uses that to assemble the text to match on. Notes, timers, commands etc. will not be included.

Quote:

It is also an example of why most clients impliment their own modified regexp system, instead of relying on one that was never intended to parse streaming text from an interactive source.


Strange, then, that zMUD now offers PCRE matching. Perhaps the modified regexp system isn't all it is cracked up to be.




Anyway, comments from other interested parties?

USA #30
I think all the other interested parties must be on vacation or something... lol That or they simply have no clue what is being discussed in this thread. Not everyone checks and reads every post, so they would have no idea that we are talking about implimentation in this case, until after it has already been included. I sent Magnum an email, but got no response and unfortunately don't know anyone else to send one too. Well.. Maybe Fyxen, but I am not sure how much she reads the posts either... :(

As for zMud supporting PCRE.. That is interesting, but I suspect broken. As for how you think it parses, I would tend to think it uses something closer to my suggestion, programatically it makes more sense to handle things as they arrive, than to retest everything for the last x number of lines, attempting to find a match. Then again, zMud isn't known for logical, complete, fast or even in some cases 'working' implimentations. lol
USA #31
I've tried tackling multiline triggers myself in the past without much success. The problem I found personally is that different situations have required different solutions, so while one method worked great in one situation, it failed dismally in another. The hardest thing was finding something that covered ANY multiline-type of triggering response, and unfortunately I never found a solution that was 100% fool proof that covered the multitude of differing possible scenarios.

If a trigger could fire using \n somehow.. but no.

I tried using world.getcommandlist on another occasion (for a kind of related situation) that sort of worked, but if I sent any commands in between (whether sent manually or by an unrelated trigger) I ended up doing something entirely undesired. It was amusing for a while anyway.. kinda like playing Russian Roulette ..hehe

Another method I tried that was extremely basic was having a first trigger (firing on the first line) that did 2 things: it enabled a 2nd trigger (that fired on the second line of the multiline trigger required obviously) and also called a script that had a short delay using world.DoAfterSpecial that disabled the 2nd trigger if it had not disabled itself by firing before that time period expired (and this could obviously be a chain of triggers). That actually did work in the right situation, but somehow seemed too easy to be a good solution, and again, this probably would not cover all situations.

My final solution was to just not worry about multiline triggers, and that has worked very well :)

As to the options in this thread so far, I think each has merit but I'm not knowledgable enough to give a definitive and qualified opinion one way or the other. :/

*sits on the fence and raises an eyebrow*
Australia Forum Administrator #32
Here is another example of a simplified trigger (if you can call it that, *cough* *cough*). At least it uses one trigger to do a stat roller rather than about 8 of them.

This illustrates a few things:

  • Multi-line trigger
  • 10 wildcards matching (normally MUSHclient matches 9 wildcards plus an extra for "the matching line")
  • Named wildcards to simplify the end-processing


Expected output:


Rerolling please wait.
-====================== CHARACTER ATTRIBUTES SELECTION ======================-
            Strength (ST): 30( 77)   Constitution   (CO): 33( 53)
            Quickness(QU): 40( 40)   Agility        (AG): 52( 52)
            Presence (PR): 24( 84)   Self-Discipline(SD): 29( 39)
            Empathy  (EM): 20( 68)   Memory         (ME): 40( 40)
            Intuition(IN): 26( 47)   Reasoning      (RE): 29( 39)


Trigger match:


Rerolling please wait\.\n-=* CHARACTER ATTRIBUTES SELECTION =*-\n\s+Strength \(ST\):\s+(?P<st>\d+)\( \d+\)\s+Constitution\s+\(CO\): (?P<co>\d+)\( \d+\)\n\s+Quickness\(QU\): (?P<qu>\d+)\( \d+\)\s+Agility \(AG\): (?P<ag>\d+)\( \d+\)\n\s+Presence \(PR\): (?P<pr>\d+)\( \d+\)\s+Self-Discipline\(SD\): (?P<sd>\d+)\( \d+\)\n\s+Empathy \(EM\): (?P<em>\d+)\( \d+\)\s+Memory\s+\(ME\): (?P<me>\d+)\( \d+\)\n\s+Intuition\(IN\): (?P<in>\d+)\( \d+\)\s+Reasoning\s+\(RE\): (?P<re>\d+)\( \d+\)


Example of extracting the data (send box) using named wildcards:


strength = %<st>
quickness = %<qu>
presence = %<pr>
empathy = %<em>
intuition = %<in>
constitution = %<co>
agility = %<ag>
self-discipline = %<sd>
memory = %<me>
reasoning = %<re>


Results:


strength = 30
quickness = 40
presence = 24
empathy = 20
intuition = 26
constitution = 33
agility = 52
self-discipline = 29
memory = 40
reasoning = 29


Time to match: 0.000171 seconds.
Amended on Sat 24 Apr 2004 07:35 AM by Nick Gammon
USA #33
Ok.. Two things have occured to me that I think are major problems with a look-back method. 1) coloring lines and 2) omitting from output.

Both of these are issues because while you use a buffer to do the look-back, the actual lines in the display are not necessarilly going to correspond to where they are in the output. This requires not only keeping track of where those lines are in the output. Ironically, it causes another odd problem. If a user types a command or a script generates a note when these lines are being displayed, then the trigger will correctly match and theoretically could color the right line, but the command, note or tell could cause a break in that line, which means it would see the right line, start to color it, then incorrectly color everything else posted to that line, though it is not actually part of the line that matched. This gets even more confusing if a) your are using the trigger to 'omit from output' or need to correctly retrieve the style information for the lines matched, since there is no practical way to know which line is really the start and if the line gets broken, the styles retrieved won't match, even though the actual contents for the line returned is correct inside the wildcard. Simply disabling these features for such triggers would more or less solve the problem, but begs the question of how useful they really are when you need to both use the data and color the information.

In the case of the stat roller, where a clever person might use colourtell, colournote and 'omit from output' to not just have the mud reroll things, but show a colorized version of the results of each roll, to get a sense of the trend the rolls are taking. While this specific instance may not quite be that helpful, since it is automated, a mud which will never give optimal results may require a less automated system, but use the same trigger to provide such a visual comparison. Other similar things could be tracked 'in-game' (say space ports stats or the like in a space trading mud) where, as I mentioned, commands or displayed lines could royally muck up the function of such a trigger. Not that they don't unfortunately tend to sometimes do so on a small scale already, but this requires still another level of complexity to partially fix the problem. A complete fix would require some way to mark lines that get broken internally, so that the commands to return styles or other line info can find the part that got chopped off. Of course since triggers also fail to match lines that get broken that way...

Anyway, it is another possible issue to consider.

I am almost tempted to try to write a PCRE parser that does properly manage streamed text. The only real hang up is getting it to split on $ or \n, which is easy and deal properly with stuff like (^...$){1,10}. From a practical standpoint it means pre-parsing the regexp to break it into seperate triggers and adding logic so it knows when each one can be expected to find a match. Not exactly an impossibility, but it would admittedly take a bit of work. In the case of something like:

^You see\:$(.*\n){1,}^$

it would simply need to create internal logic to do:

if Trigger "^You see\:" matched then
  call sub, etc.
  loop
    if match = "$(.*\n)" then
      call sub, etc.
  until if match = "^$" then
    call subs, etc.

If you used {1,10} and added ^Exits\: .*$, then:

if Trigger "^You see\:" matched then
  call sub, etc.
  loop
    if match = "$(.*\n)" then
      call sub, etc.
  until count = 10 or if match = "^$" then
    call subs, etc.
    if match = "^Exits\: .*$" then
      call sub, etc.
      reset trigger to first state.

The only real issue is having the pre-parser figure out how the bits get seperated out. Would probably take me forever to get right though. :( I still think it is a better design and uses PCRE syntax correctly, while your design actually ignores the rules to provide a feature that is in theory already in the syntax. I'm not entirely sure all the extra stuff you are doing, like the look-back buffer isn't actually more complicated that a good pre-parser, for generating a set of rules like above, to guide the trigger behaviour. It may be easier to design, but more efficient or less complicated overall? Hard to say. Sometimes 'easy to design' is a synonim for bad idea. I'm just not a good enough programmer to tell the difference most times. I suspect that is true about most of us.

On a side note: Interesting quirk with the above idea. If the pre-parser actually built vbscript, or javascript, etc. to handle it and used the scripts regexp parsing to handle the matching it would be even easier, no need to design a messy internal method to do the logic, just one to build the script. But that's not likely that practical and you never know what version of PCRE the available script engines will properly support, if actually any. Still, it is kind of an interesting idea. lol Never know where anything with Mushclient will take you. ;)
Australia Forum Administrator #34
Quote:

Ok.. Two things have occured to me that I think are major problems with a look-back method. 1) coloring lines and 2) omitting from output.


I have now updated the release notes for version 3.48, and you can see from reading those that I have taken those problems into account (before reading what you said).

Clearly a multiple-line trigger cannot do all a single-line trigger can, for instance:

  • It cannot omit from output, for one thing the line(s) may be already omitted, for another it would be hard to work out which ones they were, for another there may be note or command lines inbetween.
  • It cannot colour lines, they may have been omitted, already coloured, other lines may intervene, and again it would be hard to say exactly which lines where to be coloured.
  • It cannot omit from log, the lines may already be logged with other lines after them.
  • It cannot match on colours or styles, again it would be hard to say exactly where those colours or styles were in the original buffer, and in any case the original lines may be have omitted by the time the trigger matched.


It was really intended to solve those problems where you may want to handle a "batch of lines" (the stat roller is quite a good example) without having to write lots of separate triggers and have "trigger states".
Australia Forum Administrator #35
Quote:

I am almost tempted to try to write a PCRE parser that does properly manage streamed text.


I'll comment on that in your new thread.
Australia Forum Administrator #36
Some parts of this thread are continued at RegExp VM.
Australia Forum Administrator #37
Quote:

a clever person might use colourtell, colournote and 'omit from output' to not just have the mud reroll things, but show a colorized version of the results of each roll, to get a sense of the trend the rolls are taking


I honestly don't think randomly-rolled stats will "show a trend".
Australia Forum Administrator #38
You can now play with multi-line triggers in version 3.48, which has just been released.
#39
I for one am excited about this feature. In the past, I never wrote triggers to record things like mob vulns or ID'd items because it is annoying to coordinate the various triggers.

I just hope this system can allow for sometimes having a line and sometimes not. i.e. sometimes a mob has a vulnerability and sometimes they do not.
Australia Forum Administrator #40
Yes you can do that. Basically a multi-line trigger is like an ordinary trigger regular expression except that it might have the \n character in it to match line breaks.

Like a normal regexp you can have optional parts, where that optional part might be a whole line.

Here is an example:



Match: ^You see Nick\n(He is carrying (?P<gold>\d+) gold\n)?He is looking sleepy\z


This matches both:


You see Nick
He is looking sleepy


... and ...

You see Nick
He is carrying 13 gold
He is looking sleepy


If it matches the second case (with the gold line) then the named wildcard %<gold> contains the amount of gold he has.

I set the number of lines to 3 to test this. This is a good example of how you cannot predict exactly how many lines to match, so you allow for the maximum number, in this case, 3.
#41
Nick Gammon's way works good,then another problem:how can the script received these data(more than 10) ?
USA #42
Release notes of 3.48 (which means youll need at least 3.48), there are three methods, naming your wildcards, simply referencing them (much like the normal numbers) and then finally theres a script function to do it.


14. Added support for named wildcards in triggers or aliases. Inside regular expressions you can specify a named wildcard, like this:

Match: (?P<who>.*) tells you '(?P<what>.*)'

In this case we have 2 wildcards, named "who" and "what".

Then in the "send" box you can access named wildcards like this:

Send: %<who> just told you %<what>


15. Added support for up to 1000 wildcards in triggers and aliases (rather than 10). To access the additional ones you can use named wildcards as described above, the new script routines GetTriggerWildcard and GetAliasWildcard described below, or use this syntax in the "send" box:

Send: %<11> just told you %<12>

This example would return wildcards 11 and 12.


16. Added new script routines GetTriggerWildcard and GetAliasWildcard - these let you get a named or numbered wildcard from a trigger or alias.

eg.

x = GetTriggerWildcard ("mytrigger", "target") ' get named wildcard
x = GetTriggerWildcard ("mytrigger", "22") ' get wildcard 22

edit: changed the version to prevent future confusion, thank you Nick.
Amended on Sat 03 Jul 2004 10:26 PM by Flannel
Australia Forum Administrator #43
Quote:

Release notes of 4.48 (which means youll need at least 4.48), ...


That's version 3.48. You'll wait quite a while for 4.48.
#44
thanks a lot
Australia Forum Administrator #45
After a bit of recent testing I found the suggested trigger further up matches too often.

This seems to work better:


<triggers>
  <trigger
   enabled="y"
   lines_to_match="15"
   match="You are carrying:\n(([ ]{5}.*\n)*)\n(?![ ]{5})\z"
   multi_line="y"
   regexp="y"
   send_to="2"
   sequence="100"
  >
  <send>Found %0</send>
  </trigger>
</triggers>


There is an extra \n before the last bit, which forces it to assert that it gets a newline followed by *not* 5 spaces, to signal the end of the inventory.

Test data:



You are carrying:
     a magic mushroom
     a torch
     a dragonskin
     a bag
     recall scroll



Without the \n it matched on each line of inventory.