Nick, Question about output window & scrollback buffer...

Posted by Krenath on Sat 06 Sep 2003 01:48 AM — 18 posts, 71,115 views.

USA #0
Are the output window and scrollback buffers extremely closely tied together? As in, does the output window control itself serve as the buffer or is the output window just set up to display sections of a separate buffer?

The reason I ask is because, for as long as I've had MUSHclient, way back to version 3.1-something, any time I set up a trigger to Omit From Output, any text matching that trigger is first drawn on the screen, scrolling if necessary, then removed from the screen almost, but not quite, immediately.

Expected behavior is to never see the text appear at all.

If you've got someone annoying you and you set up a trigger to /gag them, you can't gag them, you can only shush them. Any insults or profanity sent your way are *still* flashed on the screen for about a half-second before being cleaned away. And the screen jumps as the text is scrolled onto, then scrolled off of the output window display.

This has occurred on all machines I've ever installed MUSHclient on over the years, on all operating systems so far, including Windows 95, 98, ME, NT, 2000, and XP Pro.

Is there no way to inhibit redraw of the window between the time new text arrives and the time triggers get done processing?
USA #1
It has to do with the fact that triggers arent evaluated (and therefore gagged) until after a newline has been recieved. So until then, MC doesnt know if the line should be gagged or not.
Australia Forum Administrator #2
This has been covered quite a few times in the past. However the short answer is that the buffer and window are not directly related (you can have multiple windows for the one buffer for instance, if you do "new window").

Early versions of MUSHclient did *not* show partial lines - that was when it was written for a MUSH.

However what happened was that when it was used on a Diku MUD, about the first thing that happened was the MUD sent:

What is your name? (no newline here)

This then did not appear. So, you didn't know to enter your name, password, or whatever.

So, it had to change to show stuff as it arrived, and not defer it.

However triggers still match when the newline arrives, as TCP/IP is a streaming protocol and there is no guarantee how many packets one line might be broken into, so it can't match the trigger, and therefore delete the line, until it is first drawn. Sorry.
Greece #3
Hmm, that never happens to me, and I'm using dialup... The MUD should always send a newline after its lines (except maybe the prompt), so that would only happen if your connection is so slow that you receive a packet every half a second... That can't be, can it? I have set up a trigger to gag people, and it always works, I never see any of their text...
USA #4
Poromenos, you never know how many packets the MUD's system (OS, not the MUD itself) might break the lines into. It could be as bad as one packet per character; or you could be sending 1024 or more bytes per packet. Of course, you almost never actually do receive one character per packet, but as Nick said there is no guarantee whatsoever that the line is sent in one packet. Usually, the OS sends as much as it can, but if there is heavy load it might break up packets. This is why, for example, the SMAUG network code isn't very good, because it doesn't properly consider this problem (of the OS not being able to send more packets of size x right now.)


I'm assuming that packets are read, and then drawn to the screen, and then triggers are processed. If so, would it be possible to have triggers intercept the text before it gets drawn? You could then specify in the trigger something to the effect of: "Gag until next newline."

So, let's say we have:

trigger:
^Annoying player (.*)
action:
gag until newline

Packet 1:
Annoying player s
Packet 2:
ays, 'Hahaha y
Packet 3:
ou are stupid!'\r\n


Our trigger will recognize "annoying player *" and enter gag-mode. This would be used for "annoying player tells you", "annoying player says", etc.

This, of course, is completely based on the hope that you receive the first part of the trigger in one piece.

So perhaps a "hopeful" implementation could be used? In this way, if the trigger matches what it needs before the newline is received, all the better... if not, oh well. The user can always gag the split second later, as Krenath described. Of course, the user would have to be aware of the dangers of such a trigger (i.e. first part matches but second part doesn't), and it should be highly recommended to use regular expressions and use the "line-begin" character, which is ^ I believe, to not mess up lines that may have already started and might mean something else.

Is this too much work for too little benefit?
Greece #5
Yes, it could break the line to as many packets as it wants, but for normal connections you usually don't even notice the different packets coming... They all come almost instantly...
USA #6
The point is not when the user sees the packets, it is when the program sees the packets. And programs aren't bright, they process things in the order they receive them.

Let's say you keep all your packets in a buffer, and only send the "line" to the display window when you actually receive a newline. As Nick said that works just fine for MUSHes, but on SMAUG, for instance, many lines - e.g. the intro lines, the prompts - do NOT have newlines. So how does your program know to display it?

You could do time delays - e.g. display packets a second after you receive them, so that you can process the "whole" line - but then you run into problems... for one, I don't want to have my display lagged by a second. For two, this does not promise accurate or even desirable behavior.

So the point is that you don't know which packet is the "last one" in the line, until you receive the "newline" - but in some cases, you won't EVER receive a newline. Therefore your program cannot depend on a newline as a message for "ok, display the line now after its processed" because you will throw away any message that comes without a newline.
Greece #7
Yes, but SMAUG always sends a newline after lines. The only time where it doesn't send a newline is the prompt, and you were not concerned about matching the prompt...
USA #8
The problem here is displaying the text packets we receive, and maybe gagging them. You can't design your program around the assumption that only prompts (and you forgot the welcome lines) don't end with newlines - it might change and then your whole functionality is broken.

See, you have to know when to begin evaluating the input buffer for on-screen display. If you don't display it as you receive it, you'll lose any "line" that is not terminated ("Enter your password: " ...). This means that you can't wait for a whole line to appear before you start matching triggers.

The basic process is thus:
a) we receive a packet, and put it on screen
Screen now contains: The Annoying player sa
b) we don't have a newline, so we don't process triggers
c) we receive another packet, and put it on screen
Screen now contains: The Annoying player says 'hahaha you are stupid!'
d) still no newline, so no triggers
e) we receive another packet, and put it on screen
Screen now contains: The Annoying player says 'hahaha you are stupid!'\r\n
f) we have a newline, so we can process triggers now
g) we match the annoying player trigger, gag the text, and remove it from the screen

Meanwhile, the text was still put on the screen, simply because we don't *know* that it will be the annoying player text (or anything else for that matter) until we have the whole thing.

Try writing out the logic process for such a thing, you'll see where the problems come up. You may find a solution that works *most* of the time, but that under some (more or less common) circumstances does not give the expected results.
Australia Forum Administrator #9
You could conceivably have an option for a trigger to match before a newline, however that makes it somewhat more complex.

For one thing, you then have some triggers that match in one place - when text is initially received - and others that match on a newline. Since you probably don't want to match twice you then have to flag the ones that got matched before the newline appeared so they don't match twice.

Then you have this issue ...

Say you want to gag "A turkey says, *"

OK - now this "turkey" player sends enough text to wrap around two lines.

MUSHclient matches on the first part, and gags the first line (throws it away). Now when the rest of the line arrives it has to know that it should be thrown away as well.

And what about this?

You get this:

A turk

and then in a separate packet:

ey says, xxxx\n

The trigger matches the first packet, finds no match, and flags it as "done". Then the rest arrives but because we don't want to match twice, we don't pick up that the *assembled* line - not just the parts - is what we want to match.

Sounds error-prone to me.
Greece #10
Quote:

a) we receive a packet, and put it on screen
Screen now contains: The Annoying player sa
b) we don't have a newline, so we don't process triggers
c) we receive another packet, and put it on screen
Screen now contains: The Annoying player says 'hahaha you are stupid!'
d) still no newline, so no triggers
e) we receive another packet, and put it on screen
Screen now contains: The Annoying player says 'hahaha you are stupid!'\r\n
f) we have a newline, so we can process triggers now
g) we match the annoying player trigger, gag the text, and remove it from the screen

Exactly. The packets before the newline all come almost instantly, fast enough for you not to see them.
Amended on Sun 07 Sep 2003 01:00 AM by Poromenos
USA #11
I quote Krenath:
Quote:

If you've got someone annoying you and you set up a trigger to /gag them, you can't gag them, you can only shush them. Any insults or profanity sent your way are *still* flashed on the screen for about a half-second before being cleaned away. And the screen jumps as the text is scrolled onto, then scrolled off of the output window display.


So obviously they AREN'T fast enough for you to not see them... either that or Krenath's connection is going at 1kbit/s, which is highly unlikely...
Australia Forum Administrator #12
Here is a rather strange idea, but it might work if the MUD in question is only white text on a black background, or otherwise does not use colour heavily.

You could set your colours to something like black-on-black (at least the colours that the "says" lines are arriving in) so that you don't actually see it. You may see the screen jump, but that is all.

Then you make a trigger to gag the annoying lines in the usual way.

Then you make another trigger that recolours the line so it becomes visible (eg. white on black).

You may need to defer such line colouring until the initial login screen (the username/password one), unless you have that part automated.
USA #13
Well, I think the main point here was to prevent the screen from jumping. After all the end result is the same - you don't really see the text - but it means that you jump and lose your scrollback position. Being able to do this "properly" would mean having:

a) the client not jump on new text
b) being able to intercept things before they are displayed, but this goes back to the newline problem

Come to think of it, you could probably have a set of triggers that jumped the buffer if the output is something important. For example, a say would jump the buffer. However, the gag trigger would be processed BEFORE the jump trigger, which would prevent the jump one from processing and therefore jumping. I'm not sure if this is possible now - e.g. that a trigger can jump the buffer - but if you use autopause then it would only jump if the trigger told it to.

Of course, you'd have a lot of work defining triggers to mark things as "important", but at least it'd work... I think... :)
USA #14
I have a cablemodem connection and, while this problem happens on any MUSH I connect to, the one in question is in my own house, running on one of my own computers. Connection speed is 100 megabits. The speed of the computer running MUSHclient is 2Ghz, and it has 1.5Gb of RAM and a 40GB hard drive. It's no slouch. Yet it still shows gagged/triggered lines for enough time that I can read most if not all the text before it vanishes. (I am, however, a fast reader. It's up for about a half second.)

Personally, I don't play on MUDs. All the games I play on are based on the PennMUSH server, so I have no personal use for matching text before a newline is received.

I don't suppose there's likely to be an option in the future for enabling/disabling the matching-on-partial-lines? For MUSh users, as opposed to MUD users, there's no need to try matching on or immediately displaying each packet, and MUSHclient could defer matching and display until the newline is received.
Canada #15
Nick, I don't think your suggestion will work. Colouring triggers suffer from the same 'lag' that gag triggers do. There have been plenty of times I've seen certain lines in their original colour before the line is completely received, after which the trigger kicks in and changes the colour.

Using the same foreground and background colours by default, and then using colour triggers to "white-list" all approved lines... that sounds insanely complex, and would be error prone due to text with no 'newline', which would be "invisible" forever.
Australia Forum Administrator #16
It doesn't matter if there is a lag, the line will be invisible until it is recoloured correctly (ie. black on black will be the default, *before* a trigger gets to it). Then the trigger changes it back to "normal" colours.

As for the initial prompts, the trigger could be disabled until you got past them - or you could have something that detects the "welcome" screen - after your name/password, and then changes the foreground/background colour to black on black. Depends on how much the spam is annoying you. :)
Australia Forum Administrator #17
Quote:

Is there no way to inhibit redraw of the window between the time new text arrives and the time triggers get done processing?


You can use the new behaviour for OnPluginPacketReceived to pre-process packets before they hit the main MUSHclient incoming text handler. There it would be possible to totally omit certain types of lines, as if they had never arrived.