New Live Messenger APIs

While the old messenger APIs have been steadily crumbling the past few versions, and the latest WLM10 beta having made it nearly useless, I was amazed to find several new APIs.
One of them is the “LivePlatform” API, which seems to be what messenger is using to retrieve the contact list and social news.
I don’t have time to investigate this fully, but I did get to put together a small demo. To try it, create a new console application in C#, add C:\Program Files\Windows Live\Contacts\LivePlatform.dll as a reference, and put in the following code in Main:

 LivePlatform.LivePlatformFactory factory=new LivePlatform.LivePlatformFactory();
            LivePlatform.ILiveIdentityCollection ids = factory.IdentityService.CachedIdentities;
            System.Console.WriteLine("Cached identities");
            for (int i = 0; i < ids.Count; i++)
            {
                LivePlatform.ILiveIdentity id = ids[i];
                if (id.HasPassword)
                    System.Console.WriteLine("\t"+id.Username);
            }
            System.Console.Write("Username: ");
            string szUsername = System.Console.ReadLine();
            LivePlatform.ILiveIdentity found = null;
            for (int i = 0; i < ids.Count; i++)
            {
                LivePlatform.ILiveIdentity id = ids[i];
                if (id.HasPassword && id.Username.ToLower().Equals(szUsername.ToLower()))
                {
                    found = id;
                    break;
                }
            }
            string szPassword = "";
            if (found == null)
            {
                System.Console.Write("Password: ");
                szPassword = System.Console.ReadLine();
            }
            platform=(found==null)?factory.CreateEx(szUsername,szPassword):factory.Create(found);
            platform.Config["BlockingSignin"] = true; //Don't set this to make Signin non-blocking. You'll have to wire up events to listen for OnReady before you do anything, though.
            platform.Signin(null);
            System.Console.WriteLine("Full name: "+platform.Me.FullName);
            LivePlatform.ILiveObjectCollection query = platform.Query("type(contact)");
            System.Console.WriteLine(String.Format("Contacts: {0}", query.Count));
            for (int c = 0; c < query.Count; c++)
            {
                LivePlatform.ILiveObject obj = query[c];
                LivePlatform.ILiveContact contact = obj as LivePlatform.ILiveContact;
                System.Console.WriteLine(String.Format("{0} ({1})", contact.FullName,(contact.Emails.Count>0)?contact.Emails[0].Address:"none"));
            }
            System.Console.WriteLine("Press enter to quit");
            System.Console.ReadLine();

It’ll print your full name, as well as your full contact list. Also note that if you added your facebook account, those contacts will show up too.
Some other queries I saw when debugging messenger:

type(invite)
type(contact) and contacts:canHideFrom(true)
type(contact)
type(contact) and contacts:isHidingFrom(true) and contancts:canHideFrom(true)
type(contact)
type(circle)
type(category)
type(category) and contacts:isfavorite(true)
type(contact) and contacts:isonline(true)
type(category) and contacts:isfavorite(true)

I believe that hooking this API in messenger might allow for some pretty cool tricks, for example something like BuddyFuse.

As a final note, I’d like to say that I’m posting this because I’m hoping more sharing might make the messenger community more like it once was. If you find out more about this API, or other new APIs, please be so kind to share them too so a new sharing eco-system might emerge. Thanks.

EDIT: Added cached identities.

Disappointing discoveries

Normally it’s pretty cool to find new APIs in Messenger. Imagine my excitement when I found a new interface named “IMSNMessengerConversationWnd2” that seemed to inherit from the already known IMSNMessengerConversationWnd interface. Two new functions appeared to have been added. Hurrah!

So I spent 3 hours of my time debugging and testing these new functions. Turns out they’re not nearly as exciting as I had hoped:

	[
	  odl,
	  uuid(929BB08C-B50C-4ECD-B68C-175F0B69A544),
	  helpstring("MSN Messenger Conversation Window Interface Ex"),
	  dual,
	  oleautomation
	]
	interface IMSNMessengerConversationWnd2 : IMSNMessengerConversationWnd {
		[id(0x0000080c), propget, helpstring("Get the window text.")]
		HRESULT WindowTitle([out, retval] BSTR* bstrWindowTitle);
		[id(0x0000080c), propput, helpstring("Set the window text.")]
		HRESULT WindowTitle([in] BSTR bstrWindowTitle);
	};

I’ve tested this in Vista, it gets and sets the window titlebar. As Vista doesn’t use the borderless-windows that WLM9 uses on XP, I’m not sure if it works on that, but I suspect that this interface has been specially generated for precisely that.

Well, either way, have fun with it. Sure hope my time has been useful for someone :p

Javascript Haskell mash-up

With LiveScratcher I’m working a lot with JavaScript, and I’ve really grown to love the language. For example, today I stumbled upon this blog post. It’s a JavaScript implementation of a lot of Haskell tricks and functions, including a quick “curry” function (read the blog post to find out what it is).

Just felt that was worth sharing πŸ™‚

LiveScratcher: Now what?

LiveScratcher, and it’s early abandoned brother project Modumes, have been in the pipeline for months now. I think I had the initial idea of a modular extension platform for Messenger well over a year ago. Either way, I never really had a lot of time to put some real work into it. Until now, that is. Holidays have started two weeks ago, and I’ve been crashing at my parent’s place (free food πŸ˜‰ ) and working on LiveScratcher non-stop since. I hadn’t quite expected it (what would you expect from a project that’s been in the making for months without any results), but I feel that the end of the road (or at least something worth showing) is near πŸ™‚

I’ve put up the initial scaffolding for the Messenger API, and am currently filling in the functionality for the Messenger, Contact, and Conversation classes. The DirectUI (messenger’s user-interface library) module is in a somewhat usable state, and I’ve even written an old-style UIFile parser to it (written completely in Javascript). One package I haven’t started on is the Interop (= communication with external libraries) one, but the plan for that one is well established in my head, and shouldn’t be too hard to implement.

For now I’m aiming for something that is on-par with Messenger Plus!’s scripting, but once I reach that I hope to keep expanding and surpass it greatly. One of the advantages I have is the extensibility. Each package can export certain functionality, and as such anyone is free to extend the API as they wish. Someone could make a package solely dedicated to communicating with facebook or twitter, and other developers can then use those packages from their scripts, for example creating a script that keeps your facebook or twitter friends updated on how many contacts you have.

Here’s a list of a few things that are done and worth mentioning:

  • Minimal main system: the main system *only* contains the bare necessities needed to load the modules and scripts. As such you can replace virtually anything with your own implementation; Don’t like the standard Console? take it out and write your own.
  • Google’s V8 JavaScript engine: The same engine that’s behind the blittering speed of Google Chrome. A downside is that things like “new ActiveXObject” aren’t supported, but upsides are plenty: It’s fast, it’s flexible, it’s standards compliant, and it actually allowed me to do the DirectUI stuff (Microsoft’s JScript was a bitch when it came to threading, V8 just sticks to one)
  • Code searching package: No more hard-coding of patch offsets, simply define what kind of pattern you’re looking for, and this package will find the offset(s) for you. It’ll optimize multiple patterns so only one pass over the memory is needed, so it should be really fast, and it’ll run in the background and only keep your script waiting minimally: it will return the offset the millisecond it’s found, and continue searching in the background.
  • Quick and simple console package that outputs script errors, and exports functions to add messages, warnings, or errors so you have an easy way to output data to the user.
  • DirectUI access: Want to remove a button from messenger? add a new one? Move the wordwheel to the bottom of the contact list? It’s all possible.
  • UIFile parser completely implemented in JavaScript, allows you to define your own output callbacks (or use existing ones), so someone could easily stick in a UIB compiler.
  • Packages can consist of modules (DLLs) or scripts, or even a combination of both.
  • The actual Messenger API is a mess, there’s about three different ways to do things, and half of the functionality is often broken. But as a package developer you don’t have to worry about that: I’ve taken the time to write a Messenger package that takes care of all that for you, and exports an easy to use interface for your scripts to use πŸ™‚
  • Need some functionality that’s not yet in LiveScratcher but is in Plus? For now there’s a PlusCom package that allows you to make a (string values only) bridge between a Plus! script and a LiveScratcher package.
    I’m not sure if this will be useful when it’s all done, but it was a nice experiment, and might come in handy with other things.
  • Need to pop up a toast? That’s already in the Messenger module πŸ™‚ with callbacks and everything. And not fake lookalike toasts, nope, actual real Messenger toasts with custom text.

Now, it all sounds really great when I put it like this, but there’s also a bunch of things that are missing (but that I do plan to implement some day):

  • Debugger: as of now, all you have is some errors in the console if something goes wrong. I’m planning on exposing the V8 debugger protocol and write a nice debugger plugin for Eclipse or Notepad++ or something like that.
  • More advanced Interop: For now I’m focussing on an interop approach like Plus!. A simpe way to call external libraries with integers or strings, and some memory block class for pointer stuff. When I have the time I’m hoping to come up with a nicer approach that allows you to specify a structure and it will make a memory-block automatically and catch all get/set calls to it.
  • Protocol level stuff: For now I’m focussing on exposing all the API based stuff, but at some point I hope to write a proxy/protocol-analysing package that would allow for similar functionality to Messenger Discoveries’s plugins.
  • RichEdit hooking: It’s going to be a big project, but I do hope to implement some kind of RichEdit hook and expose that, so scripts can implement new mark-up stuff. For example implement Plus!’s coloring, or implement a LateX package that automatically converts latex to images.

Well, now you know most of what it is and my plans about it. If for some reason you got excited (sure hope you did πŸ˜‰ ) and want to take a look and give some feedback, please don’t hesitate to contact me. Just leave a comment and I’ll get in touch with you. I’ll gladly send it over and give you all the help you need to get using it. At this point however I’m not quite comfortable with just uploading it here (as it’s still very very crude and rough around the edges), so you will have to personally ask me if you want to take a peek πŸ˜‰

Aho-Corasick with wildcards

A lot of Messenger plugins, addons, and scripts nowadays use some form of memory patching. Some are quick patches like disabling the nudge limit, others like the chat-only name of StuffPlug 3 are much more complicated. But no matter how simple or complicated, they all have the same downsides: the location of the patch changes with almost every new release (even minor ones) of Messenger. Most developers simply keep a small table of locations in different versions, but this forces you to update your software with each and every messenger update. With StuffPlug 3 I opted to write a quick search routine that would search the messenger executable for specific patterns, and would find patch locations using that. This pretty much garantueed StuffPlug to keep working with minor messenger
updates, and sometimes even major updates.

One of the downsides of my implementation in StuffPlug 3 was however that the search routine was pretty simple and as such slow. With each new pattern that needed to be looked for, the time it took to search increased, and at some point this reached a full 3 seconds on my machine (which is pretty high-end compared to my average user). I solved this partially with caching, but for some patterns I needed all hits and caching was not possible. In the end the searching generally took 1 second, or 3 if you just updated messenger. Now even though 1 second (or possibly 5 on slower PCs) doesn’t sound like a lot, it bothered me, and it hasn’t even stopped bothering me. I’ve always wanted to look into other algorithms to speed this up, but generally I had other priorities or didn’t have time at all.

Somehow that changed a week or so ago, when we revisited string matching algorithms in my current course. I just had to try, and I did. As I’m looking for multiple patterns at once, Aho-Corasick should be the ideal approach. One downside: I need wildcards (characters that match anything) in my patterns, and Aho-Corasick wasn’t really made for it. I did see a suggestion somewhere to simply split your search string on the wildcards and search for the constant sub-patterns, but this sounded a bit silly. Especially as there was no motivation about why this was the best approach, I felt I could do better. And today, I thought I had. I had an Aho-Corasick implementation working, with wildcards and everything.

Or so I thought. Yes, the algorithm works. Yes, it can take a wildcard or two. But once you add a few more wildcards, the number of states in the state tree just explodes. With just two memory patterns (38 characters in total, of which 12 wildcards) needed a whopping 7007 nodes, and generating the tree alone took a full 12 seconds. I also tried with all 20 patterns in StuffPlug 3, but that just didn’t seem to finish, ever.

In conclusion: For Aho-Corasick with wildcards, just use the suggested technique of splitting the patterns up. It is possible to adjust the algorithm itself for wildcards, but the number of states needed gets so big that it’s no longer feasible, and will most likely not run on any current computer when you use more than 5 wildcards in total.

I’m not giving up though πŸ™‚ I’m currently working on the Aho-Corasick bit with splitting up, and I have another tree/state-machine based approach in my head that I want to try. At least if I finish this I can truthfully say that I can’t possibly make that part of StuffPlug any faster…

JavaScript madness

There’s quite a lot of people out there that absolutely hate JavaScript or ECMAScript, mostly because the different DOM/DHTML implementations in browsers are often incompatible. Yet it appears there are also people out there that love the dynamic nature of the language itself, and I think I’m becoming one of them. To me it’s a language that allows you to hack small things together relatively easy (when writing some exercises that were supposed to be written in pseudo code in JS a professor noted that it resembled pseudo-code really closely, and I should recommend it to the lecturer), whereas you’re also able to write bigger projects by leveraging the OO capabilities.
Anyway, enough about that, ever since I started using Google’s V8 engine in LiveScratcher for the scripting I’ve been experimenting with JS, and after having done some Data structures exercises with it, I started getting passionate about implementing some data structures in JavaScript. I’ve currently written a Heap (with a PriorityQueue using it), and have just finished an AVLTree implementation (which I will use for a map), and will probably move on to implement all structures from my book πŸ™‚ I’m not sure if I’ll ever release this into the wild, or if it’s even worth releasing into the wild, but I’m pretty proud of it so far, and it forces me to learn more about the data structures involved, as well as keeps my JS skills up to par.

On a different note: I’ve set up a website for LiveScratcher with a wiki and an installer for an alpha version. Check it out at http://www.livescratcher.com/

Compatibility out of the box

My current boss wanted to know if the LiveScratcher Script-to-DUI interface would work with Windows Live Messenger 8 too. I’d kept in mind that I wanted it compatible with WLM8, but hadn’t tested it yet. To my suprise, there were only two tiny (non-fatal) errors, and I could get the advertising removal script working with just one extra line.

Behold, the Windows Live Messenger 8 *and* 9 compatible advertisement patch:

//Compact method
function CL_onCreatedElement(resid, root) {
	if (resid=="mainContentResID") {
		var ads=root.FindDescendent("TabsAndAds") || root.FindDescendent("AdBanner");
		ads.LayoutPos=-3;
	}
}

//Verbose method
function Convo_onCreatedElement(resid, root) {
	if (resid=="convframeresid") {
		var adbannergutter=root.FindDescendent("adbannergutter");
		if (adbannergutter)
			adbannergutter.SetValue("LayoutPos",Value.CreateInt(-3));
	}
}

//Patches contained in this file
Patches=[
	{
		id: "ContactListAdvert",
		name: "Remove Contact-list Advertisement",
		version: "1.0",
		author: "A. Nonymous",
		website: "http://www.google.com/",
		onCreatedElement: CL_onCreatedElement
	},
	{
		id: "ConvoTextAdvert",
		name: "Remove Conversation Text Advertising",
		version: "1.0",
		author: "A. Nonymous",
		website: "http://www.google.com/",
		onCreatedElement: Convo_onCreatedElement
	}
];
//Register patches
for (var i in Patches) {
	RegisterPatch(Patches[i].id,Patches[i]);
}

The whole thing started off as an extension to a DUI compatibility layer I wrote for Iminent, but with every hour I stick in I’m falling in love with it more πŸ™‚

What if…

… You could write Messenger patches in JavaScript? I guess it could look something like this:

function CL_onCreatedElement(resid, root) {
	if (resid=="mainContentResID") {
		var TabsAndAds=root.FindDescendent("TabsAndAds");
		if (TabsAndAds)
			TabsAndAds.SetValue("LayoutPos",new Value(-3));
	}
}

function Convo_onCreatedElement(resid, root) {
	if (resid=="convframeresid") {
		var adbannergutter=root.FindDescendent("adbannergutter");
		if (adbannergutter)
			adbannergutter.SetValue("LayoutPos",-3);
	}
}

//Patches contained in this file
Patches=[
	{
		id: "ContactListAdvert",
		name: "Remove Contact-list Advertisement",
		version: "1.0",
		author: "A. Nonymous",
		website: "http://www.google.com/",
		onCreatedElement: CL_onCreatedElement
	},
	{
		id: "ConvoTextAdvert",
		name: "Remove Conversation Text Advertising",
		version: "1.0",
		author: "A. Nonymous",
		website: "http://www.google.com/",
		onCreatedElement: Convo_onCreatedElement
	}
];
//Register patches
for (var i in Patches) {
	RegisterPatch(Patches[i].id,Patches[i]);
}

The above code works just like you’d expect it to, thanks to a new project I’m working on. For now it features only a very very small subset of the DirectUI library, just enough to make the above work (+ some debugging, obviously), but I plan to expand it to create an incredibly flexible platform for writing patches, skins, and later maybe even plugins.

It’s the spiritual successor of the abandoned Project M or Modumes project. Basically Project M was too complex, by using Windows JScript and allowing any other language to write modules in, I faced some problems (mainly threading) that I could not reliable fix.

To create something that could reliable interface with the DirectUI, I needed to have an enviroment where everything, or at least the parts interfacing with DirectUI, would be running from the main thread. My eye first fell on LUA, later on Squirrel, but in the end I figured that although those two scripting languages were incredibly powerful and small, I wanted something a tad more manageable. Eventually I figured the best language would be JavaScript, and Google’s V8 EngineΒ allowed me all I needed:

  • Quick execution (although Mozilla’s TraceMonkey is apparently slightly faster, V8 does a really good job, and was easier to embed)
  • Easy embedding
  • Easy extensible (adding new objects)
  • Easy to learn (Plus! uses a variant of JavaScript too, and all client-side web development is JavaScript too)
  • No threading issues

A weekend plus several hours in the evenings further, and I have a working prototype, yay! πŸ™‚

My current goals:

  • Ability to write quick patches
    • Without having to replace the entire UIB/XML
    • Without having to convert back and forth from UIB
    • Without having to update for each and every minor update
  • Ability to bind to element events (e.g. onMouseOver, onClick)
  • Ability to interface with the Messenger API
  • Ability to set timers from patches
  • Ability to interface with outside programs
  • Ability to share functions between scripts

Messenger Addons

Since the new version of Messenger 9, none of the major Messenger add-ons have been updated to support it: I myself have attempted to port the current StuffPlug 3 code to WLM9, but concluded that there have been so many changes that for it to be fully compatible I’d have to do a re-write. Patchou had made a quick beta compatible with the earlier WLM9 beta, but he hasn’t posted anything on his own forums since December, and as far as I know his latest version of Plus! is not compatible with the (January released) WLM9 Release Candidate. Messenger Discovery only states “limited support for 9.0”, and to top it all off all current patches (with the notable exception of A-patch, who did a limited version) are not compatible with the new messenger file format.

Basically I think that the comment absorbation made on my earlier postΒ was probably dead on: everyone seems to have moved on to other projects. I’ve seen my own interest in Messenger getting lower at times, resulting in no updates for StuffPlug, but now it’s not only me: even Patchou, the one person who’s kept updating Plus! for ages even when SP and MD would not, seems to be moving on to other projects.

I sometimes wonder how it is that old developers are moving on while no new ones are taking any interest in Messenger anymore. Is messenger development going too slow? too fast? is it because they completely abandoned all the APIs available? Is it because the messenger team has been more and more secretive, and the only real outwards communication being mainly focused on normal users while no longer appealing to developers? I’d really love to hear people’s opinions on this, and would be especially interested in hearing what people at Microsoft think of it. Please, if you read this and have any thoughts about this subject at all, share them with us through the comments below.

On a slightly different note, I’d like to comment on what warmth said in reply to an earlier post. Yes, StuffPlug currently is pretty much dead: the code I have is just no longer compatible with the changes in WLM9, and I’d have to start a re-write sometime. Problem is that most of StuffPlug’s features aren’t really relevant anymore on Messenger 9, and so trying to re-write the entire beast for WLM9 won’t really get me anywhere. I’d much rather start off with something small and build it up from there, much like StuffPlug was in the beginning. So if you have anything, anything at all, that bothers you about Messenger 9, please do tell me, and it might spark that required bit of passion to start working on something again.

As for helping development of other projects; I have actually helped and contributed to both Plus! and MD in the past, and if I can help them again I’d still love to, but setting up an environment for different people to work on the same (hobby) project takes time and effort, and I’m not sure either of us are really ready to make that kind of effort.

So, if you read this: please do leave a comment if you have any views on the whole lack of third-party development on Messenger 9, or if you have something that you’d like to see changed in Messenger 9 by a third-party add-on.