1x1 hotspots

Posted by Twisol on Mon 21 Dec 2009 05:43 AM — 45 posts, 141,370 views.

Twisol USA Mon 21 Dec 2009 05:43 AM #0

I seem to be unable to create 1x1 hotspots, i.e. hotspots that only affect a single pixel. I'm trying to layer a window entirely in 1x1 hotspots for fine-tuned tracking of the mouse using the below code, and for testing purposes I'm using the hotspot's ID as the tooltip text. The problem is that no tooltip appears at all, no matter where I point the mouse.


for y = 0, self.height do
  for x = 0, self.width do
    local id = self.name .. "-h(" .. x .. "," .. y .. ")"
    check(WindowAddHotspot(self.name, id, x, y, x+1, y+1,
       "MWidget.View.hotspot_handlers.mouseover",
       "MWidget.View.hotspot_handlers.cancelmouseover",
       "MWidget.View.hotspot_handlers.mousedown",
       "MWidget.View.hotspot_handlers.cancelmousedown",
       "MWidget.View.hotspot_handlers.mouseup",
       id, 0, 0))
    check(WindowDragHandler(self.name, id,
       "MWidget.View.hotspot_handlers.dragmove",
       "MWidget.View.hotspot_handlers.dragrelease",
       0))
  end
end

If I make the hotspots 2x2 instead, and change the for-loops to have a step of 2 instead of the default of 1, everything works fine. 2x2 isn't sufficient for my purposes, though, and it seems odd that 1x1 wouldn't work - it seems perfectly valid.

Another issue is that this takes a long time to execute, taking around 2 seconds on a 184x299 window. I think it wouldn't be such a bad idea to start using the 'flags' parameter to WindowAddHotspot, allowing you to modify the behavior of a single hotspot to send mouseover events for every mouse movement within the area, rather than just crossing into it from the outside. That way, I could just use one large hotspot instead of hundreds of tiny ones.

For reference, the reason I am doing this is because it provides the most convenient and intuitive way to create an abstraction layer over MUSHclient hotspots, allowing each widget in my framework to manage its own set of Hotspot objects (even if the widget is a child (contained within) of another widget). This solves the issue of how to percolate hotspot changes from subwidgets up to the visible frame, because my abstraction layer (implemented by the set of generic handlers you see in the code above) can use the information available to them to travel down the tree of subwidgets to locate the appropriate Hotspot to trigger.

Of course, if anyone has any other ideas I'm all ears, but I've been thinking about this for quite a while and I think it's the most viable way to go about it. After all, I think Win32 sends mouse-movement messages to its windows for every twitch of the mouse, doesn't it? So there is definitely precedent. [1]

[1] http://msdn.microsoft.com/en-us/library/ms645616(VS.85).aspx

EDIT: tl;dr summary, because I think I rambled a bit there:

* 1x1 hotspots seem to be malfunctioning, though I could just be doing something wrong.
* Fine-tuned mouse movement notification within a given hotspot would make a great flag for the unused 'flags' parameter in WindowAddHotspot.

Nick Gammon Australia Forum Administrator Mon 21 Dec 2009 07:48 AM #1

Twisol said:

Another issue is that this takes a long time to execute, taking around 2 seconds on a 184x299 window.

So it creates 55,016 hotspots in 2 seconds? And that is slow? That is 27,508 per second. Each hotspot causes memory allocations, and needs to be stored in a list.

Whilst there may be a bug in hotspots where you can't make a 1x1 pixel hotspot, I think you are abusing the concept here. Hotspots are supposed to be GUI elements, like buttons, not individual pixels.

Other parts of miniwindows documentation state that you should make, say, lines by calling a single function (WindowLine) rather than doing it individually for every pixel (WindowSetPixel) - this can reduce the number of function calls from thousands, to one.

Nick Gammon Australia Forum Administrator Mon 21 Dec 2009 07:55 AM #2

Even if the bug wasn't there, you are asking for a Lua script to be called for every pixel that the mouse moves over a window. I think that would be slow too. The miniwindow mouse handlers are trying hard to abstract away a lot of that low-level stuff, so individual mouse moves are only relevant when they move to or from graphical elements (hotspots) like buttons.

Twisol USA Mon 21 Dec 2009 07:56 AM #3

Everything you just said is precisely why I suggested the modification to the 'flags' parameter in WindowAddHotspot. I fully admit that creating 55,016 hotspots is completely inefficient, and using just one that's pixel-sensitive - akin to using a line or an image rather than pixel blits - would be much more effective. I'm simply working with what I have, and suggesting improvements that would be useful. (It's important to note that such a change would be quite useful even outside my widget framework, by the way - as I said, Windows itself fires a WM_MOUSEMOVE on every movement per-pixel.)

Twisol USA Mon 21 Dec 2009 08:04 AM #4

Nick Gammon said:
Even if the bug wasn't there, you are asking for a Lua script to be called for every pixel that the mouse moves over a window. I think that would be slow too.

Graphical games (like World of Warcraft, which also uses Lua) do far more intensive number-crunching per frame, so I don't see this as an issue unless you're doing really heavy things in the handlers. And the most likely place for heavy things to go would be mousedown (or mouseup), not mouseover.

Nick Gammon said:
The miniwindow mouse handlers are trying hard to abstract away a lot of that low-level stuff, so individual mouse moves are only relevant when they move to or from graphical elements (hotspots) like buttons.

The issue is that at a fundamental level, composition of images and composition of hotspots are very different things. It is very simple to redraw widgets and pass the finished buffer up to the parent widget, and so on. It is not so easy to do the same with hotspots, especially when they need to be able to be removed or modified. This approach is the only one I've come up with so far that seems both viable and intuitive.

As a sidenote, if someone wanted to create a custom 'cursor'-like graphic when hovering over a minwindow, how would they do it? This is competely apart from my widget framework. You would need either a layer of 1x1 hotpixels, or a single high-sensitivity hotspot.

EDIT: Second sidenote. Yes, 2 seconds is very speedy considering the sheer amount of work being done, but it is incredibly slow compared to the rest of what I've done so far. In other words, it is a bottleneck with an, in my opinion, obvious and relatively easy solution.

Twisol USA Mon 21 Dec 2009 08:15 AM #5

A cursory glance at the code in CMUSHView::Mouse_Move_Window suggests that maybe the following change might allow this functionality, though I am by no means certain.

      // if different hotspot from before or this hotspot is pixel-sensitive
      if (sHotspotId != mw->m_sMouseOverHotspot || (pHotspot->m_Flags | 1))
        {
        // this is our new one
        mw->m_sMouseOverHotspot = sHotspotId;
        Send_Mouse_Event_To_Plugin (mw->m_sCallbackPlugin, 
                                    pHotspot->m_sMouseOver, 
                                    sHotspotId);
        }
      }

1 is just an arbitrarily selected flag to mark it as pixel-sensitive.

(EDIT: It's late and I should head to bed, I'll check back in tomorrow. >_>)

Nick Gammon Australia Forum Administrator Mon 21 Dec 2009 07:35 PM #6

Twisol said:

I seem to be unable to create 1x1 hotspots, i.e. hotspots that only affect a single pixel.

I can create 1x1 hotspots OK, see this:

To save and install the Hotspot_bug_test plugin do this:

Copy the code below (in the code box) to the Clipboard
Open a text editor (such as Notepad) and paste the plugin code into it
Save to disk on your PC, preferably in your plugins directory, as Hotspot_bug_test.xml
- The "plugins" directory is usually under the "worlds" directory inside where you installed MUSHclient.
Go to the MUSHclient File menu -> Plugins
Click "Add"
Choose the file Hotspot_bug_test.xml (which you just saved in step 3) as a plugin
Click "Close"
Save your world file, so that the plugin loads next time you open it.


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>
<!-- Saved on Tuesday, December 22, 2009, 7:29 AM -->
<!-- MuClient version 4.45 -->

<!-- Plugin "Hotspot_bug_test" generated by Plugin Wizard -->

<muclient>
<plugin
   name="Hotspot_bug_test"
   author="Nick Gammon"
   id="b40cc6e35092067ad28e5616"
   language="Lua"
   purpose="Tests a 1x1 hotspot"
   date_written="2009-12-22 07:22:57"
   requires="4.40"
   version="1.0"
   >

</plugin>


<!--  Script  -->


<script>
<![CDATA[
function mouseover (flags, hotspot_id)
  print ("in mouseover, flags=", flags, "hotspot_id=", hotspot_id)
end -- mouseover

function cancelmouseover (flags, hotspot_id)
  print ("in cancelmouseover, flags=", flags, "hotspot_id=", hotspot_id)
end -- mouseover

function mousedown (flags, hotspot_id)
  print ("in mousedown, flags=", flags, "hotspot_id=", hotspot_id)
end -- mousedown

function cancelmousedown  (flags, hotspot_id)
  print ("in cancelmousedown, flags=", flags, "hotspot_id=", hotspot_id)
end -- cancelmousedown

function mouseup  (flags, hotspot_id)
  print ("in mouseup, flags=", flags, "hotspot_id=", hotspot_id)
end -- mousedown

-- make window
win = GetPluginID ()  -- get a unique name
WindowCreate (win, 0, 0, 200, 200, 12, 0, ColourNameToRGB("white"))  -- create window

-- hotspot location
LeftA, TopA, RightA, BottomA = 10, 10, 11, 11

-- fill it
WindowRectOp (win, 2, LeftA, TopA, RightA, BottomA, ColourNameToRGB ("red"))

-- add hotspot
WindowAddHotspot(win, "hs1",  LeftA, TopA, RightA, BottomA, 
                 "mouseover", 
                 "cancelmouseover", 
                 "mousedown",
                 "cancelmousedown", 
                 "mouseup", 
                 "tooltip - hs1",
                 1, 0)

WindowShow (win,  true)  -- show it 


]]>
</script>


</muclient>

Nick Gammon Australia Forum Administrator Mon 21 Dec 2009 07:43 PM #7

And in case it was the juxtaposition of the hotspots, this test creates an array of 100 x 100 of 1x1 hotspots, and that works OK too.

To save and install the Hotspot_bug_test2 plugin do this:

Copy the code below (in the code box) to the Clipboard
Open a text editor (such as Notepad) and paste the plugin code into it
Save to disk on your PC, preferably in your plugins directory, as Hotspot_bug_test2.xml
- The "plugins" directory is usually under the "worlds" directory inside where you installed MUSHclient.
Go to the MUSHclient File menu -> Plugins
Click "Add"
Choose the file Hotspot_bug_test2.xml (which you just saved in step 3) as a plugin
Click "Close"
Save your world file, so that the plugin loads next time you open it.


<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE muclient>
<!-- Saved on Tuesday, December 22, 2009, 7:29 AM -->
<!-- MuClient version 4.45 -->

<!-- Plugin "Hotspot_bug_test2" generated by Plugin Wizard -->

<muclient>
<plugin
   name="Hotspot_bug_test2"
   author="Nick Gammon"
   id="5a796a8c2d3e5735d5cd1f2d"
   language="Lua"
   purpose="Tests 100 1x1 hotspots"
   date_written="2009-12-22 07:22:57"
   requires="4.40"
   version="1.0"
   >

</plugin>


<!--  Script  -->


<script>
<![CDATA[
function mouseover (flags, hotspot_id)
  print ("in mouseover, flags=", flags, "hotspot_id=", hotspot_id)
end -- mouseover

function cancelmouseover (flags, hotspot_id)
  print ("in cancelmouseover, flags=", flags, "hotspot_id=", hotspot_id)
end -- mouseover

function mousedown (flags, hotspot_id)
  print ("in mousedown, flags=", flags, "hotspot_id=", hotspot_id)
end -- mousedown

function cancelmousedown  (flags, hotspot_id)
  print ("in cancelmousedown, flags=", flags, "hotspot_id=", hotspot_id)
end -- cancelmousedown

function mouseup  (flags, hotspot_id)
  print ("in mouseup, flags=", flags, "hotspot_id=", hotspot_id)
end -- mousedown

-- make window
win = GetPluginID ()  -- get a unique name
WindowCreate (win, 0, 0, 200, 200, 12, 0, ColourNameToRGB("white"))  -- create window

for y = 0, 99 do
  for x = 0, 99 do
  
  -- hotspot location
  local LeftA, TopA, RightA, BottomA = x, y, x + 1, y + 1
  local id = string.format ("x=%i, y=%i", x, y)
  
  -- fill it
  WindowRectOp (win, 2, LeftA, TopA, RightA, BottomA, 
                bit.bor (
                  bit.shl (math.random (0, 255), 16),
                  bit.shl (math.random (0, 255), 8),
                           math.random (0, 255)))
                  
                
  
  -- add hotspot
  WindowAddHotspot(win, id,  LeftA, TopA, RightA, BottomA, 
                   "mouseover", 
                   "cancelmouseover", 
                   "mousedown",
                   "cancelmousedown", 
                   "mouseup", 
                   "tooltip " ..  id,
                   1, 0)
  
  end -- for x
  
end -- for  y


WindowShow (win,  true)  -- show it 


]]>
</script>


</muclient>

Twisol USA Mon 21 Dec 2009 07:44 PM #8

The events appear to work fine. The tooltip doesn't. So, I suppose this is more of a tooltip-event issue...

Since I'm not planning on using the tooltip parameter except for testing, I guess it's a non-issue for me personally. However, I still think that using one high-sensitivity hotspot beats using 55016 hotpixels. ;)

Nick Gammon Australia Forum Administrator Mon 21 Dec 2009 07:48 PM #9

Twisol said:

The events appear to work fine. The tooltip doesn't. So, I suppose this is more of a tooltip-event issue.

I don't seem to be able to get tooltips at all, so I agree there seems to be a problem there.

Nick Gammon Australia Forum Administrator Mon 21 Dec 2009 09:16 PM #10

I am getting very variable results with the tooltip text, not only with 1x1 hotspots. There seems to be something strange going on there.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 01:41 AM #11

After extensive investigation I am coming to the conclusion that the tooltip code is somehow responsible for this. MUSHclient activates the tooltip when you move onto a different hotspot from before, however I think the tooltip code only shows the tooltip if you move *inside* the tooltip area, and then stop moving. Thus, it is impossible in a 1x1 pixel hotspot for it to be activated (by moving onto the pixel) and then move around *inside* that pixel for the the tooltip display to be activated - since by definition moving around would move you outside that pixel.

So basically this is an artifact of having tiny hotspots, which I may say you wouldn't use in practice anyway.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 01:43 AM #12

Your original report was a bit misleading:

Twisol said:

I seem to be unable to create 1x1 hotspots, i.e. hotspots that only affect a single pixel.

You can indeed create 1x1 hotspots, as you agree yourself:

Twisol said:

The events appear to work fine. The tooltip doesn't.

Twisol USA Tue 22 Dec 2009 01:45 AM #13

At the time, I didn't know it was just the tooltips; I really did think it was the actual hotspot. Your later two plugin examples set me straight. My apologies if I wasn't clear in the second post you quoted.

As to the tooltip thing... interesting. Will that tidbit be added to the docs? ;)

EDIT: This may be a stupid question, but what happens if you enable the tooltip on every hotspot that provides one? Would the tooltip float up no matter where you hover, or would it only appear over the hotspot? It seems like maybe some of the CToolTipCtrl members could be used to limit the area the tooltip hovers over (::SetToolRect?), but this is my first exposure to the tooltip class and I'm just shooting in the dark.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 01:57 AM #14

Don't really understand the question, and not sure about the answer, as the CToolTipCtrl control seems a bit obscure anyway.

Twisol USA Tue 22 Dec 2009 02:00 AM #15

The issue is that, once a tooltip control is Activated, it (probably) needs to receive a mousemove event for the tooltip to show up. My proposed solution, assuming I'm even close to being correct, is to immediately Activate a hotspot's tooltip during WindowAddHotspot, and then limit the tooltip control's rect using CToolTipCtrl::SetToolRect(). That way, the tooltip would be alerted the moment you wander into a hotspot, rather than once you move around inside one.

EDIT: I think I am right; see [1] and [2].

[1] http://msdn.microsoft.com/en-us/library/6b4cb3a5(VS.80).aspx

Quote:
A "tool" is either a window, such as a child window or control, or an application-defined rectangular area within a window's client area. A tool tip is hidden most of the time, appearing only when the user puts the cursor on a tool and leaves it there for approximately one-half second.

[2] http://msdn.microsoft.com/en-us/library/6hh5zw0e(VS.80).aspx
CToolTipCtrl:SetToolRect()

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 02:46 AM #16

There is one tooltip control for the entire output window, not one per hotspot. After all, only one tooltip is visible at once.

Twisol USA Tue 22 Dec 2009 02:46 AM #17

Anyways, what do you think about the initial idea, adding a flag for WindowAddHotspot to allow hotspots to pick up on pixel-sensitive movement? It wouldn't affect hotspots prior to the change, and would be fully optional (more of an advanced feature, if you will). But just as you said, it's better to blit a whole image at once rather than its component pixels separately, and that's exactly what the flag would do.

EDIT: Good timing on the posting... replying again to respond.

Twisol USA Tue 22 Dec 2009 02:48 AM #18

I'm not sure I see the issue. You can have multiple CToolTipCtrl's and attach them all to the output window's HWND, and then use ::SetToolRect() to map the tooltip to a specific rectangle of the output window.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 04:04 AM #19

You could, but so far I haven't. I can see quite possibly, especially if you have a tooltip per pixel, dozens of tooltips popping up everywhere driving you mad, like confetti. The current design limits you to one tooltip at a time.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 04:09 AM #20

Twisol said:

Anyways, what do you think about the initial idea, adding a flag for WindowAddHotspot to allow hotspots to pick up on pixel-sensitive movement?

Ah yes, well I got distracted trying to see if there was a genuine bug.

You are right WoW uses Lua, however that is somewhat more coarse-grained than calling a Lua routine for every pixel the mouse wanders over. User functions are called for things like incoming messages (eg. battle damage) or the player clicking on a button, exactly like what you get now in MUSHclient with triggers and hotspots.

Your proposed change of getting a mouse-move message for every time the mouse moves, even by a pixel, would break the documented effect for mouse moves, that you get a mouse-move followed by a mouse-cancel event. Unless you also want a mouse-over cancel event for every pixel as well.

I can't help feeling you are trying a brute-force approach to a problem that should have a more elegant solution. Surely even nested hotspots can be accommodated by simply keeping them in a table, and finding the topmost one by a quick search?

Twisol USA Tue 22 Dec 2009 04:21 AM #21

Nick Gammon said:

You could, but so far I haven't. I can see quite possibly, especially if you have a tooltip per pixel, dozens of tooltips popping up everywhere driving you mad, like confetti. The current design limits you to one tooltip at a time.

That's fine, I was just proposing a possible fix. It's definitely not a major concern.

Nick Gammon said:
Your proposed change of getting a mouse-move message for every time the mouse moves, even by a pixel, would break the documented effect for mouse moves, that you get a mouse-move followed by a mouse-cancel event. Unless you also want a mouse-over cancel event for every pixel as well.

No, not really. The cancelmouseover would only occur when you leave the hotspot, as it is now, and that makes sense to me. The mouseover event just responds as the mouse moves over it.

Nick Gammon said:
I can't help feeling you are trying a brute-force approach to a problem that should have a more elegant solution. Surely even nested hotspots can be accommodated by simply keeping them in a table, and finding the topmost one by a quick search?

I find this approach very elegant, personally, but we're allowed to disagree. The issue is that each widget needs some way to keep track of its hotspots, change their handlers, modify their location/size, and remove them if need be, and it is a bonus that the hotspots wouldn't lose their current mouse-state just by getting moved or resized. They would naturally get updated on the next mouse movement, of course, if it moved itself away from the mouse.

Taking this approach - decoupling the hotspots from the events - allows widgets to manage their own hotspots dynamically, without needing to run through complex operations just to make simple changes. I had spent around two or three days just trying to come up with a good design for hotspots, and all of them had some major drawbacks. When I came up with this design, I felt like it was a eureka moment; the only real drawback I could see was the inefficiency of creating so many 1x1 hotspots, which is what I am proposing a (simple) solution to. The layer of hotpixels would track the events, traverse the tree to locate potential callbacks, and execute callbacks when appropriate. The whole widget/hotspot tree can be changed dynamically, and the view need never know.

If you think you have a better solution, I am completely all ears! :) The source is also available on GitHub (now linked in my signature), if you want to look at it for a better idea of what's involved. I'll soon be pushing up a version using the 1x1 blanket of hotpixels, since that approach currently works, if you'd like to see exactly what I'm planning. Most of the infrastructure for the hotspot functionality is already in this version, actually, it's just not exposed.

EDIT: To note, by supplying the flag to enable pixel-sensitive mouse movement, you'd know exactly what would happen. I don't think the documentation issue is a valid concern because of this; just note that the pixel-precision has this behavior and it should be kept in mind while using it.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 06:04 AM #22

Twisol said:

EDIT: tl;dr summary, because I think I rambled a bit there:

I forgot to say, purely for humor you understand: TLDR

David Haley USA Tue 22 Dec 2009 06:15 PM #23

I think that two issues have been somewhat conflated here:

1. Twisol wants per-pixel knowledge of where the mouse cursor currently is.

2. Twisol has a solution to this problem that abuses the notion of hotspots. (Twisol also believes he doesn't have much of a choice in the matter given the current API.)

I think that it's pretty clear that blanketing the entire window with 1x1 hotspots is not ideal for several reasons. So I don't think we should argue against the 1x1 method; we should be trying to establish if having per-pixel mouse control is a good idea or not.

First, note that most GUI frameworks send mouse movement events to the whole (sub-)window, with the coordinates as extra parameters. Applications can then deal with or ignore these messages as they see fit.

An argument given against per-pixel control was efficiency. Many applications do in fact track the mouse rather particularly. For example, many games change the cursor depending on what's under the mouse; many others track mouse movements in order to move the camera. So (in principle) tracking every mouse movement is not prohibitive.

I threw together a quick benchmark:


$ cat test.lua 

function do_something(x, y)
    return x * 2 + y
end
function do_something_else(x, y)
    return x * 3 + y
end

function HandleEvent(event)
    if event.type == "mouseover" then
        if event.x > 50 and event.y < 20 then
            do_something(x, y)
        else
            do_something_else(x, y)
        end
    end
end


for i = 1, 10000000 do
    x = math.random(1, 100)
    y = math.random(1, 100)
    HandleEvent({type = "mouseover", x = x, y = y})
end

$ time lua test.lua
lua test.lua  31.21s user 0.00s system 99% cpu 31.223 total
$

(This is on an Intel Celeron 2.4 GHz, running Debian. Not exactly a fast computer.)

This is obviously only an approximation of what happens for mouse events. Note that it takes ~30 seconds to run 10 million events. That's 0.000003 seconds, or 0.003 milliseconds, per event handling.

It can be empirically tested, but I wager that you probably won't be getting more than several thousand mouse events per second (if that many!). This means that we'd be spending a few milliseconds processing events at maximal "mouse moving bandwidth". That's probably acceptable.

For windows that do not need fine-grained control, it would suffice to create them with a flag set to false (which could be the default value for the parameter) -- in this case they would not generate the mouse events in the first place. So, miniwindows who don't care about mouse events will see no difference.

I think that having per-pixel mouse control is an important feature of any GUI toolkit. The fact that Twisol is trying to do his own stuff on top of this is kind of irrelevant in my opinion (it just happens that his work is exposing issues in the API). There are many reasons to have per-pixel control. One reason is that it makes some tasks much easier; imagine you have some sort of map that displays things. You could create a hotspot per thing, but this would require complex management (addition, removal, modification) of hotspots each time the 'things' changed. But if you could simply capture all pixels, you could look up which 'thing' is under the cursor at that point in time.

Perhaps one way to focus the discussion is to examine the objections to per-pixel control? The main ones I see are: "it's inefficient" (which I think has been addressed) and "it creates extra work" (which I think is true but unavoidable :P).

Twisol USA Tue 22 Dec 2009 07:55 PM #24

Just to note briefly, I have just pushed a version of the framework with a working pixel-sensitive layer of 1x1 hotpixels. It's not even remotely close to being done, but it does work, and my Tic Tac Toe plugin is behaving very well (it uses hotspots extensively). I hope you will check it out!

EDIT: To briefly note, I'm only using the 1x1 layer because I have no other recourse. It's an odd workaround, to be sure, but that's all I've got.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 08:07 PM #25

OK, well if everyone insists. :)

I have added a new plugin callback to version 4.45. This detects mouse movement in the output window. For example:


function OnPluginMouseMoved (x, y, unused)
  TraceOut ("mouse move: x=", x, ", y=", y)
end -- function OnPluginMouseMoved

Tests seem to show that with continuous movement of the mouse you get about 60 calls per second. Using the example code above I didn't notice any particular slowing down of the client.

Twisol USA Tue 22 Dec 2009 08:16 PM #26

Not to be like "give them an inch, they take a mile" or anything - I appreciate that you're doing this at all - but I really, really don't think this is the best solution. In relation to the framework (as an example), it requires that either (a) the user write an OnPluginMouseMoved themselves and pass it along to an overall framework callback, or (b) the framework write one itself, thus rendering it wholly unavailable to the user if they want to do anything different with it.

Doing it this way would require not only that code be written to discern which hotspot to execute, but also which -window- to check in the first place! Further, it's wholly decoupled from the miniwindow system, which is really the only place this would make any sense. Finally, because there is no sense of 'which window is being moused over', you could have two windows on top of eachother, both created in separate plugins, both moused over at some point, and both of them doing something in response to the mouseover, even though only one is on top. This wholly breaks expected behavior, and is the largest issue with this approach.

I would honestly like to know why you're averse to adding a flag to WindowAddHotspot. Is there an issue with it I'm unaware of?

EDIT: note the "both created in separate plugins" bit, I forgot that part.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 08:20 PM #27

Well, given that you seem to want cursor movements only for a particular hotspot, why is that? I assumed from your earlier posts that you were trying to detect movements in general, and then farm them out to a particular "control" or whatever it is.

I mean, if there is an OK button, why is it exciting to know the mouse has moved around inside it?

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 08:24 PM #28

Twisol said:

As a sidenote, if someone wanted to create a custom 'cursor'-like graphic when hovering over a minwindow, how would they do it? This is competely apart from my widget framework. You would need either a layer of 1x1 hotpixels, or a single high-sensitivity hotspot.

I have added a new selector to SetCursor, namely -1 for "no cursor".

Combined with OnPluginMouseMoved, you could hide the normal cursor, and then move around another miniwindow, which could be your graphical cursor. This is one of the reasons I added the "ignore mouse" (8) parameter to miniwindow creation a while back. Such a "cursor" window would not want to get mouse clicks, or it would be impossible to click on the underlying window.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 08:26 PM #29

Twisol said:

(b) the framework write one itself, thus rendering it wholly unavailable to the user if they want to do anything different with it.

Why? They can't "do anything" with it. It just tells them where the mouse is. All plugins will get it.

Twisol USA Tue 22 Dec 2009 08:32 PM #30

On the SetCursor thing, brilliant, I was going to ask if you could do exactly that at some future point. :) The use I was thinking of, though, was that you'd control the cursor you draw per-window, though - i.e. just blitting your own graphic to the mouse position on your window - not actually changing the mouse for the whole output window. (A simpler way to do that would be for a variant of SetCursor where you supply the name of a miniwindow or some such thing.)

Nick Gammon said:
Well, given that you seem to want cursor movements only for a particular hotpost, why is that? I assumed from your earlier posts that you were trying to detect movements in general, and then farm them out to a particular "control" or whatever it is.

Yes, I will have a single pixel-sensitive hotspot overlaying the whole window, specific to my framework. It would handle events and pass things along to Hotspot objects owned by individual widget controls.

Nick Gammon said:
I mean, if there is an OK button, why is it exciting to know the mouse has moved around inside it?

It's not. If you didn't want to use my framework, for example, a pixel-sensitive hotspot could be placed alongside normal hotspots, allowing you to use normal hotspots for buttons and the like, while using the sensitive hotspot for more interesting interaction.

The only reason my framework requires a pixel-precision hotspot to cover the whole window is precisely because it's a framework. It can't assume the locations and sizes and uses of the hotspots it will own, which is the point of the Hotspot objects I've created. It makes it easier for users to manage the interactivity of their widgets, without compromising how the view is drawn or acted upon.

EDIT: Not the only reason, but a big one. The other is the issue of applying hotspot changes to the visible window.

Twisol USA Tue 22 Dec 2009 08:33 PM #31

Nick Gammon said:

Twisol said:

(b) the framework write one itself, thus rendering it wholly unavailable to the user if they want to do anything different with it.

Why? They can't "do anything" with it. It just tells them where the mouse is. All plugins will get it.

Well yes, that's the big issue I mentioned in the very next paragraph.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 09:03 PM #32

Twisol said:

Yes, I will have a single pixel-sensitive hotspot overlaying the whole window, specific to my framework. It would handle events and pass things along to Hotspot objects owned by individual widget controls.

You are going to have to spell out for me the difference between a plugin callback that gets mouse movements for the whole window, and a "pixel-sensitive hotspot overlaying the whole window" which gives you mouse movements. Sounds the same to me. Is there a terminology confusion here?

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 09:13 PM #33

Twisol said:

Doing it this way would require not only that code be written to discern which hotspot to execute, but also which -window- to check in the first place! Further, it's wholly decoupled from the miniwindow system, which is really the only place this would make any sense. Finally, because there is no sense of 'which window is being moused over', you could have two windows on top of eachother, both created in separate plugins, both moused over at some point, and both of them doing something in response to the mouseover, even though only one is on top. This wholly breaks expected behavior, and is the largest issue with this approach.

I thought having a single point of reference would help, particularly as you seem to be developing a framework system. All you have to do is do what MUSHclient does itself anyway:

Iterate through the windows (WindowList gives you that) - in reverse order, so the topmost one is considered first.
Check the x/y coordinates to be within the miniwindow location (up to 4 integer compares)
If not, move onto the next one
If so, stop processing more windows (so you don't get an underneath one too), and then repeat the process for hotspots inside that miniwindow if required to identify a hotspot.

Twisol said:

I would honestly like to know why you're averse to adding a flag to WindowAddHotspot. Is there an issue with it I'm unaware of?

It seemed a more generic solution, somehow.

I still don't fully get why your framework can't just get the mouseover calls for individual hotspots anyway, without going down to the pixel level.

Twisol USA Tue 22 Dec 2009 09:28 PM #34

Nick Gammon said:
You are going to have to spell out for me the difference between a plugin callback that gets mouse movements for the whole window, and a "pixel-sensitive hotspot overlaying the whole window" which gives you mouse movements. Sounds the same to me. Is there a terminology confusion here?

I meant that it covers the whole miniwindow. Your callback would be called on every mouse movement whether or not there's a window there; see my response to the next quote below.

Nick Gammon said:
I thought having a single point of reference would help, particularly as you seem to be developing a framework system. All you have to do is do what MUSHclient does itself anyway:

*Iterate through the windows (WindowList gives you that) - in reverse order, so the topmost one is considered first.

*Check the x/y coordinates to be within the miniwindow location (up to 4 integer compares)

*If not, move onto the next one

*If so, stop processing more windows (so you don't get an underneath one too), and then repeat the process for hotspots inside that miniwindow if required to identify a hotspot.

One point to make against doing it myself is that MUSHclient already does it for me. As you said, MUSHclient goes to great lengths to hide these details. (For the record, I don't count pixel-sensitive mousemoves to be an incredibly harsh detail to expose.)

Another issue is that this OnPlugin* callback will be called on every mouse movement within the output area. That means that not only do I have to do work that MUSHclient already does on its own, I also have to do it even when there might not be a window beneath the cursor. This is something MUSHclient does anyways, to pass normal mouseover events, which means we're doing the same work twice.

Lastly, unfortunately the "single frame of reference" doesn't really work out here. The framework is window-based, not plugin-based or output-window-based. Every view has its own state, though it might share descendants with other views. It is -much much much- easier to start with the view than the output window.

Nick Gammon said:
It seemed a more generic solution, somehow.

I still don't fully get why your framework can't just get the mouseover calls for individual hotspots anyway, without going down to the pixel level.

What do you mean by a "more generic solution"?

Here is how my framework works. There is one view, into which all widget canvases are compiled to get the visual elements you see. ('widgets' aren't miniwindows themselves, they are images blitted onto their parent, drawn on hidden canvas miniwindows that are never seen directly). Each widget can add their own logical hotspots, that is, Hotspot objects that are pure data. Hotspot objects have no physical presence on the view window, because it is much easier to manage them separately, and it makes it possible to resize, move, and change the behavior of them very easily.

The view, the physical area on-screen that you see, is sensitive to every movement of the mouse. It catches all events, figures out what hotspots are involved, and passes them along. It also will (not done yet) track the currently moused-over hotspot, firing a cancelmouseover when the mouse leaves its region. Because of the nature of the abstraction layer, Hotspots can be freely moved and resized (i.e. their internal rects changed) without any repercussions.

It is incredibly difficult (bordering on impossible) to literally give each Hotspot its own MUSH hotspot on the view, not least because there can be multiple views that happen to show the same widget, and it would be chaos to figure out what changes go to what view. (If you wonder when this would ever happen, it would be more likely to occur during widget composition, when you're viewing two separate widgets that happen to have the same one as a descendant further down)

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 10:25 PM #35

Twisol said:

Anyways, what do you think about the initial idea, adding a flag for WindowAddHotspot to allow hotspots to pick up on pixel-sensitive movement?

...

It is incredibly difficult (bordering on impossible) to literally give each Hotspot its own MUSH hotspot on the view, not least because there can be multiple views that happen to show the same widget, and it would be chaos to figure out what changes go to what view.

OK, if I went along with this, you want a flag added to WindowAddHotspot to track mouse movements inside hotspots, but you are not using MUSHclient hotspots?

If you have a "full-window" hotspot, and it is high up the testing order then it hides the lower hotspots.

So I don't see how this works.

I have amended the earlier change to the OnPluginMouseMoved callback to change the 3rd argument to "miniwindow ID".

Now, you know immediately which miniwindow the mouse is over, without mucking around checking coordinates. And, if it is over none, you can quickly exit, eg.


function OnPluginMouseMoved (x, y, mw)

  -- ignore if not on miniwindow
  if mw == "" then return end 

  -- process miniwindow "mw" here ...

end -- function OnPluginMouseMoved

Now, given the miniwindow ID, you can check for some movement over the "logical hotspots" you describe.

Twisol USA Tue 22 Dec 2009 10:29 PM #36

Nick Gammon said:
OK, if I went along with this, you want a flag added to WindowAddHotspot to track mouse movements inside hotspots, but you are not using MUSHclient hotspots?

If you have a "full-window" hotspot, and it is high up the testing order then it hides the lower hotspots.

So I don't see how this works.

No, no, there's only one real hotspot altogether. All of the others are virtual hotspots, they're not added with WindowAddHotspot at all. There's only one MUSHclient hotspot, and that's the full-window hotspot.

It's not just my framework that would use these kinds of hotspots. My framework only uses one hotspot, but non-framework miniwindows could still use these hotspots (non-full-window) for their own purposes.

Nick Gammon said:
I have amended the earlier change to the OnPluginMouseMoved callback to change the 3rd argument to "miniwindow ID".

Now, you know immediately which miniwindow the mouse is over, without mucking around checking coordinates. And, if it is over none, you can quickly exit

This is feeling exceedingly hacky by now, when the exact same functionality is already given by the mousemove hotspot event. By this point I'd almost rather use the 55,016 layer of 1x1 hotspots >_>

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 11:02 PM #37

Well in the interests of keeping plugin developers happy, I have done what you suggested. There is now a flag (1) in the WindowAddHotspot function, which causes mouseover events to be reported for all mouse movements, not just the first.

So you know which is which, the flags parameter passed to the mouseover function will have the 0x80 bit set for subsequent calls, so you can tell the difference between the first call and other calls.

Twisol USA Tue 22 Dec 2009 11:05 PM #38

Thanks! I really appreciate it.

It seems like I'm asking for/about a lot lately. I'm sorry if it's annoying or an inconvenience - and oftentimes you really do suggest better alternatives - but it's just because I want to help keep this client great. >_> So yeah, apologies just in case.

Twisol USA Tue 22 Dec 2009 11:08 PM #39

By the way, I have a minor idea for you, relating to your previous SetCursor change. Since you can now pass -1 to SetCursor, what about passing -1 to WindowAddHotspot for the Cursor parameter?

EDIT: And when is OnPluginMouseMove executed? Is it before or after hotspot handlers have been executed? If after, and if SetCursor is called in OnPluginMouseMove, it would be very difficult for miniwindows to draw their own cursors.

Just something that occured to me while mulling over the new changes.

Nick Gammon Australia Forum Administrator Tue 22 Dec 2009 11:28 PM #40

Yeah, why not? I have changed hotspots to allow for a cursor of -1.

OnPluginMouseMove is executed early, before any hotspot handlers.

Twisol USA Tue 22 Dec 2009 11:28 PM #41

Great, thanks! ^_^

Onoitsu2 USA Wed 23 Dec 2009 07:10 AM #42

I am liking the idea of what this can become, something like a visual inventory even, that could show bags of all kinds, and then when an item is dragged it could effectively send the remove command, and the the put command when it is dropped on another bag.

This has other merits, such as for those that are staff on MUSHes, it could be a visual job control, so you could alter the buckets of jobs, double click on a bucket to expand it, and then display its contained jobs, then double click on a job to view it. Or perhaps even on a BBOARD for posts.

I cannot wait to see the amazing things this can be used to make.

-Onoitsu2

KaVir Germany Thu 24 Jun 2010 02:10 PM #43

D'oh, I'd just written up a lengthy post asking if it was possible to do something along these lines, and only noticed this post when I did a final check to make sure it hadn't already been covered.

This is great stuff, though. Combined with a graphical map, it would allow Twisol-style virtual hotspots shaped exactly to fit the monster and object icons, overcoming the problem of transparent pixels blocking overlapping rectangles. Hover over the monster or object to view its stats, click to attack or pick up.

I'm still not quite sure how the custom cursors would work though. Supposing I want the cursor to be a boot when hovering over the map, replaced with a sword when hovering over a monster and a glove when hovering over an object...how exactly would that be done? I'd pass -1 to SetCursor() and then what? Would it just be a small miniwindow containing an image displayed relative to WindowInfo 17 and 18?

Twisol USA Thu 24 Jun 2010 05:40 PM #44

KaVir said:
I'm still not quite sure how the custom cursors would work though. Supposing I want the cursor to be a boot when hovering over the map, replaced with a sword when hovering over a monster and a glove when hovering over an object...how exactly would that be done? I'd pass -1 to SetCursor() and then what? Would it just be a small miniwindow containing an image displayed relative to WindowInfo 17 and 18?

Yep, that's pretty much exactly it.