ATL tricks

Some time back I had an idea.. Why not design a program to be a companion to Mushclient, which would let you design the layout of a window, save that as XML and then have a plugin parse that to reproduce the GUI you just made. There where too "critical" problems doing this. The first is that Mushclient scripts/plugins can't respond to events from such a control. Basically, if you made a window with a button, the button would never do anything. The second problem was a designer. The code of impliment one seemed to be buried so far down in the bowels of MS' "Why do you need to know this?" docs that it would never see the light of day.

The solution to the first problem turns out to be an ATL bridge. I sent Nick a link to it, but so far I have no idea if he has looked into it much or figured out how to make it work. Basically, it would give Mushclient a new script function, which would let you say: utils.connect("MyButton", "Button1"), where "MyButton" is the event and "Button1" would be an function/subroutine in the script/plugin. Problem solved and it would work for even existing controls, like the Winamp control dll that was frustratingly unable to tell Mushclient when the song changed to a new one.

The other problem is solved with:

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/vcmfc98/html/vcrefiaxwinambientdispatchput_usermode.asp

That's right.. The secret of the ages revieled. Or at least one that is damn hard to find. The "key" to being able to change properties that are normally locked and move controls around a windows is "iaxwinambientdispatchput_usermode". This can "probably" be accessed within some scripts, though probably not VB. But as usual the exact method to correctly impliment it is obscure in the MS documentation. Gosh, would it have hurt to include %$@$@#@$ example code once in a while guys? lol

But, point is it "should" be possible to use this and the bridge together to impliment a designer plugin. Just finding it is about the limit of my skill in this matter though. :p Figured maybe someone else with more of a clue how the heck to make some of this stuff work could try fiddling with it.

As I said before, MUSHclient is not written using ATL, and I am unsure how easily, if at all, an MFC program can be made to use ATL calls.

Microsoft is notorious for improving its products (its database interface is an example) in such a way that developers who use the "flavour of the month" for a couple of years find that this has now been superseded with a different method of doing things, which is not backwards compatible with the old method.

And won't the Lua-wxWidgets or Lua-GTK libraries let you do this anyway? Admittedly a bit clunkier, but much easier.

Somehow I doubt that. I think Shadowfyr wants a method of having an external program (like WinAmp) notify MUSHclient asynchronously of an event occuring.

Actually, I am not sure how well Lua or others do handle this. See.. The problem is that when you create a window it has to be tied to something, but most cases (I tried it in python once) you *can not* define the windows as a top level application window. I.e., if you use NULL for the parent, it will crash both the script and Mushclient. So.. the only choice is to use Mushclient's window as the parent.

This is where the problems start... Instead of the script recieving the events from the object, Mushclient does. Only Mushclient has no clue what to do with them, so simply ignores them. The result is that any events from that window, or any control placed on it, fail to do anything. As far as I know the only way this can work, even in Lua, is to run the script stand alone, instead of through Mushclient, but then you have an entirely different mess.

Now, Nick. This isn't a flavor of the week thing I am talking about. This is ***exactly*** how IE handles objects. There are only two ways to instance and object. Early and late binding. Early binding links the ActiveX objects functions directly into the program at compile time, so the compiler can directly connect a function or subroutine to the event message stream for the application. The other method is late binding, which you *can't* bind events with. All web controls in IE which are declared using CLSID's, or instanced through createobject in things like VBScript, are late bound. This means they "must" have a bridge to connect the events they generate to the instanced object. IE has a built in bridge and VBScript even has a command called GetRef, which works by simply tying the "event" IE is already aware of from when it instanced the object to a function:

set Window.close = GetRef("IClosed")

JScript uses something much closer to what the example I gave does:

attachevent("close", onclose);
function onclose()...

However, *neither* of these will work with Mushclient, because Mushclient does not impliment an ATL bridge that is compatible with the functions in either script language.

So.. I have to question the idea that something MS uses to impliment basic functionality in IE itself would get changed so it no longer works in some later version. ATL is the ActiveX Template Library. The functions used for "both" of these tricks are standard for everything from Internet Explorer to, in the later case with iaxwin... their own form designer in all their compilers. Heck, for that matter, their IDE wouldn't work right without a similar bridge system, since it allows you to "test" components without compiling the actual code in some cases, which would be quite impossible otherwise.

In any case, the point here is Lua and others can handle events for objects they create "when running in a stand alone environment where ***they*** are the top level application window. When running as a script engine for something like Mushclient, the top level already exists, so any attempt to create such a window will crash the engine and the client its tied to. But, if you use Mushclient as the top level window when creating a new one, it is "Mushclient", not the script that recieves event messages. This can only be fixed by doing the same thing IE does when dealing with this sort of problem and using an ATL bridge to intercept and connect the events to the script function. Its also the only way you can add unknown post-compilation objects into "any" application, no matter what language, and have them respond to events. Lua and others have their own bridges, but they only work when "they" are the top level application window, so they are already receiving the events.

Without a bridge, they couldn't work either, since obviously with a just in time system like scripts run with, there is **no** way to pre-define what an object is so that its events can be handled. That is why you can't place a line like "dim MyObject WithEvents as SomeObject" in vbscript. All scripting objects are late bound, even in Lua. Yet, somehow they manage to handle them. The only method I have found "anywhere" that does that is an ATL bridge, since it requires using the ActiveX atl.dll to trap the events and redirect them to the script functions. If I am wrong, someone show me an example of creating a window with a button in Lua (or anything else) inside a plugin, where the button or the window actually work right. It still wouldn't solve the problem of anything from existing controls to stand alone applications, but it would still suprise me a lot, given I already tried once and failed completely with Python.

----

Now, as for if you can use ATL with MFC... Why the hell not? MFC already uses ATL internally to work at all. The only thing you are doing when using the ATL library, instead of MFC, is cutting out one of the middle men. In fact, half the tricks in the toolkit of most programmers for professional development involves using lower level libraries to alter the behaviour of things like MFC. Sorry Nick, but I personally think you have far more to worry about when it comes to the possibility that MS will change the high level, miles away from the core dlls, MFC implimentation than the critical components like ATL, which if changed would break everything from MFC itself to their precious Explorer. Your being imho paranoid.

Sorry for the long and sightly ranting post, but I have done a lot more research on just why events can't be handled right in late bound objects and what the "only" solution for fixing it, which is discussed a lot on Microsoft's own site though sans any useful examples of how to use it (I get real tired of that), than you have. The fact that I don't know enough C++ to successfully impliment "Hello World" doesn't alter the fact that I do know something about programming in other languages and do have a fair understanding of what is going on in this case. Hell, I have been trying to find solutions to these two issues for over three years. I don't need to prove that it works. IE already uses an implimentation of it, as do many other application. I also know that the usermode trick, despite its obscurity, has at least a dozen $200+ applications floating around that impliment it, not including every form designer made by MS themselves for their own compilers. I wish I could design a test example to prove it, but.. Right now I have only a version of VC++ 4 and its on a machine that isn't working right by even the most limited definition of "working". :(

I'll give you a pretty good reason "why the hell not"... the APIs are different. It doesn't matter that one builds on top of the other. Besides, MFC doesn't use ATL to begin with. (ATL might use parts of MFC, but it's not a superset; and MFC sure as heck isn't a superset of ATL.) The whole point of ATL is that it provides an "advanced template library" that MFC doesn't provide. ATL and MFC are simply different technologies. It's true that it's possible to use ATL controls in an MFC program, but it's not as zip-zip-here-we-go as you seem to be suggesting.

I really think you need to write some C/C++ programs, even some small ones, and try this kind of thing yourself before jumping out and making statements like that and accusing Nick of being paranoid etc. :/

For a start, MFC does not use ATL. MFC is a set of C++ libraries that do not, largely speaking, use templates at all. Whereas ATL was designed later, and relies on the template mechanism to simplify its implementation. It is hard to Google for an authoritative definition of the difference but I found this quote on a site:

---


There are several programming foundations provided for the Visual C++ environment. Two of these that are commonly used are the Microsoft Foundation Classes (MFC) and the Active Template Library (ATL). MFC has been the most commonly used basis for application development in Visual C++ in the last few years. ATL has gained more and more popularity with the increased use of ActiveX technology and components. Some Visual C++ programmers might wonder why our tools are focused around use with MFC instead of ATL. MFC is designed as a foundation for GUI application development. ATL is designed as the foundation for ActiveX component development.


---

Thus it is unsurprising that I am using MFC for MUSHclient, but you - who seem to be trying to get a control to work - have found posts about how ATL might help.



Perhaps they will change them, but I won't notice. I am still using the compiler I purchased about 5 years ago, and the MFC libraries that came with it. I no longer want to get on the treadmill of upgrading the libraries, which means upgrading the compiler (paying to upgrade it), which means upgrading the operating system (paying again), which means a faster PC (paying again), to get some new whizz-bang technology that won't be compatible with what I have currently developed.

Again, if they changed ATL it won't break MFC.

---

Let's get back to basics. What is the problem we are really trying to solve?



So, I gather you have added a control to the MUSHclient window by using the main window handle that you can get from the GetFrame script function? Is that correct?

Now, various events are generated which end up in the main MUSHclient event loop, which are really intended for the control? Is that correct? Or are generated by the control and intended for the application?

Now, you want the control to get them? Or a script you have written?

Can you give me an example event? Like, the link to its definition?

Maybe this would work ... you could make a script call like this:


WantEvent (12345, pluginID, functionname)


If MUSHclient sees this event (preferably after realising it doesn't need to process it itself, or maybe as well as), it calls your plugin "pluginID", calling the function "functionname", and passing it the event details.

Is that what you need? Or am I missing the point here?

Ok.. Been doing some hunting. Your right, MFC probably doesn't use atl, "technically", however is does use parts of the same ole library, so in effect anything that broke ATL would probably break anything using a lot of ActiveX controls, never mind ATL. Since its a template system, the dlls are likely to remain the same, but the templates themselves may change. Note: There is in fact an error in the .idl file for some functions, so its tehcnically "already" broken in older versions, though only with reguard to those specific functions and then only if you don't know how to fix it. lol

That said. Your getting it exactly. As I understand it, any object instanced by a script engine while it runs in a client like Mushclient, even if you don't need to tell it what parent to use "is" using the hosting client as the parent. I may be wrong, but I doubt it, since if I was wrong there would be no reason why the GetRef function works in IE, but fails in Mushclient. The "parent" in such cases must be something, since its "not" the script engine itself, that leaves what?

Now, the reason Lua, Python and others don't need this when run stand alone is because the command line application for running them provides the top level window and all objects are instanced within the memory space of that top level application. I.e. The stand alone version is very simple client that creates a window, loads the engine and has it execute the code. But it has built into it a means to handle objects created in that context.

That said, your 100% right about what I am talking about. As I understand it the bridge is usually a DLL and differs from what you are saying only in that it uses the IUknown to find a list of event names from the internal table of the object. It then keeps this list, so all you need to do is hand it off the name of the object, event and the function you want it to call. Basically exactly what you are talking about, but as I understand it what ATL does is make it a bit easier to figure out what the heck your doing in this case (maybe).

In your example.. Is the first number there the "object"? I mean, it wouldn't be a big deal to have something like:



But if we needed to know the correct event number we would be right back to square one, with no way to determine what the number "should" be, so we can link it, especially given that you might have multiple buttons on a window.

However, yes, this is basically what I mean. Of course, if we could just find some code on how the hell IE actually impliments this connectiong through things like GetRef in VB, CreateEventObject in JScript and the methods in others for the same... The bridge simply seemed like the most promising, since we actually have code for how it works. Anything that would produce the needed result though would solve the problem. ;)

What frustrates the hell out of me though is when you ask anyone about how to impliment most of this stuff you get BS like, "Well, if IE already has that, why not just use IE?" Its like someone asking you why if the house your in already has a shower you want to build a new house someplace else and find out the correct way to install showers.... :p

OK, back to some Windows basics. In Windows (as in other operating systems) various things generate events, and others respond to them. For example, mouse down, mouse move, key click, and so on. Event messages look like this in Windows:



Here are some example message numbers (event numbers):



Thus effectively each event is tied to a window (the window in which the mouse was clicked, for example), and has a location (pt) which might be *where* the mouse was clicked, a time (when it was clicked) and two parameters (wParam and lParam) which can be used for things like (in the case of key presses) which key was pressed, and whether or not control or shift was also pressed.

Now I am assuming that if you want to trap some events generated by your control, the event numbers must be documented somewhere, and you thus might request notification of event left-mouse-down (0x201 or decimal 513).

Then your script called be called, saying that event 513 has occurred at a certain time, at a certain place, and with the lparam and wparam containing certain values.

Hmm. Ok, that makes sense. Might be nice to have something like:

WantEvent (hwnd, PluginID, FunctionName)

Why? Because some place in that return message is the information for if its the OK or the Oops! button. Assuming someone wanted to do something as silly as make a dialog window. For more complex things you are going to have a huge mess of buttons and each is going to have its own response code. Compilers take care of this quite nicely by handling most of the nasty bits for you. Your own window in a script is likely to be a tad more complicated. *And* some other persons control may not be so easy to find information on, especially if its a control developed for VB and they assume you don't need to know what the actual numbers generated are.

I admit I haven't looked too close at that part of the whole mess. I knew on some level, but thought in terms of the application connecting the "name" of the event to the ID, so you didn't have to muck around with numbers.

But, see if you follow me here. If the event handler simply passes on "all" events to a handler function, the user can always reverse engineer the numbers needed, by dumping the returned info from the events to output. In other words, if you don't actually know that the button labelled "Don't Press Me!" generates an event number of 0x995, you could just press the button, then use world.note to dump the result to the display. Imho, this makes a lot more sense than having to tell Mushclient about "every" event you want to deal with, instead of just coding function to split off what you "need" to. For example, in the case of a button that does nothing but toggle something on and off, having one whole function and a call to WantEvent to set it up is a lot less sane than say:



If they really need to call a seperate function for something, let them. And if your handler simply passes "all" of the information, including window, there is no real reason why you couldn't simplify it even more and do:

WantEvents (PluginID, EventHandlerFunction)

At worst you might need to, if you don't want the user to get them, add a bit of code to refuse to pass on Mushclients own windows events. But that has got to be simpler to code than running through a list of plugins, to see if they want events, then look at if the event is even one that has been asked for, then finally looking up the function needed to pass it too. Right? That just seems like a lot of work that the event function could do instead.

If you ever run an "event debugger" you will see hundreds of events occur every second, for example, simply moving the mouse.

I think to pass every one to a script would slow it down enormously. However if you specify a subset of two or three, it might be quick enough to handle that.

Hmm. Yeah. Passing all of them would be bad. lol

Just got my ATL Internals book today and have been reading up one stuff. Unfortunately it doesn't seem to cover the ambient property feature I got it for. :( A few interesting things though:

1. ATL, while it includes some pre-built function in the atl.dll, is actually a code generator and set of pre-built helper classes. In other words all it is really doing is providing some ways to impliment the existing interfaces, using a few tricks that make life easier, like one example where you have three different functions, all of which contain a Draw() function, and need to be able to tell them appart. The ATL wizard just generates C++ code that enumerates them in a way that you can tell which is which.

2. The stuff in the ATL bridge is probably using those ATL helper wrappers over top of an interface used in "all" ActiveX components, called IConnectionPointContainer. This is used by ActiveX controls sort of like this:

a. Create control.
b. Get a handle to a function.
c. Call the controls IConnectionPointContainer::Advise function with the function handle.
d. Events for the object are now passed to that function.

This is basically what you are trying to reinvent. ;) ActiveX containers, Internet Explorer and script engines, according to the book, all use this through a dispinterface, since they cannot direct bind the events to the client. This may mean that events for the objects, if this inteface is not implimented in the client application, end up in limbo.

3. This paragraph also caught my eye:

"The only abcolutely correct way to obtain a DISPID is to ask the object at startup for the DISPID corresponding to an event or to read the object's type library at runtime. However, ATL doesn't support this. Visual Basic and MFC don't either. They all assume that the DISPIDs in a dispinterface will never change as a questionable performance optimization."

So... As I thought, if something breaks ATL it will also break MFC, since both ignore the "correct" interface in favor of assuming that the dll's or exe's interfaces will remain the same between versions. Oops!

Unfortunately, it looks like having more clear info on how the interface in I.E. actually works means I would also need the book "Inside OLE" (another $60...), if they don't rob you blind with the OS and compilers they get you with the books needed to actually use them. :p The details on just how all the COM interfaces work and why is buried in there some place, along with what I needed for controlling, not just reading, the Ambient_Usermode properly. :(

But, one thing this has done is make me wonder if I am in fact correct about where unsinked events go. If they end up in limbo or some place in the script engine, which can't actually handle them without the client being able to bridge the gap, then... Ones explicitly created with the Mushclient window as the parent, or thereby hosted by it in some direct fasshion, would likely do what they are supposed to, but the rest.. Some of the stuff in ATL is actually code to start with a main window, interate through all the children, until it finds a specific control, then query its interfaces to do stuff with it. All of which can be coded by hand and isn't that disimilar to the code I came up with for my Fireworks gadget (and used to experiment with seeing if I could make Mushclient's main window change its text at the top, that sort of worked, but as soon as a redraw happened Mushclient over-wrote the change. ;) Definitely interesting reading..

Hmm. Just had a thought. While I still think giving the window handle to track makes more sense than individual events, things like stand alone objects, which do not require a specific window to be instanced, like the Winamp COM inteface that started this whole mess, are not going to be "on" a window that I can pass a handle back to. They "may" be tied by default to Mushclients main window though, so I would have to ask the WantEvents to track things Mushclient is doing in order to deal with it.

This isn't a major problem and at worst there might be some way to reassign the "parent" for the object to a user created window, maybe.. Its only really a problem if you try to put a sanity check in, like disallowing:

WantEvents(GetFrame, "EventHandler")

Also, while something like all movement from the mouse would be a bad idea to track, keep in mind that someone with a map window, or other similar stuff, "may" be use a picture of some sort and need to track the location where someone clicks on the image. This is likely to be far less of a mess than handling all events though, obviously. Just figured it would be a good idea to think about that, since it would be a bad thing (tm) to artifically limit things more than necessary when trying to keep events from slowing the client.

Its sort of a trade off. How do you handle events when you may not be 100% sure what the event is you are looking for, but keep rapid fire events from messing you up. This is imho probably why connection points are generally used instead, despite the efficiency issues with them. But I am not sure ordinary windows or objects, like those created through Lua, Python, etc., "have" connection points. They are not generally ActiveX based. In the end you idea might work better than trying to use the other method.

Found someone that actually knows how this stuff works. Yep, I got a real reply from someone with a clue, and they pointed me to the horses mouth. lol Basically, the assumption I was making may be 100% wrong. Created objects in late binding have "no" events generated. Those events fire only if they are connected to the internals of an application. To solve this they implemented IProvideClassInfo, which some objects unfortunately don't have. Some of those though, the applications that create event sinks use the system registry to look up. If for what ever reason neither the registry nor the object have a means to return information on the interface, you're screwed. :p But "some" form of sink must be used to make it work:

http://blogs.msdn.com/ericlippert/archive/2005/02/15/373330.aspx
http://blogs.msdn.com/ericlippert/archive/2005/09/09/463215.aspx
http://blogs.msdn.com/ericlippert/archive/2005/09/21/472465.aspx

More is probably around on the site and it can be done in raw C++, instead of MFC or ATL, both of which just add layers of abstraction and imho confusion to what is actually going on in cases like this.