On Minecraft Mods

Introduction

I’ve been running a Bukkit Minecraft server for my friends and I, and recently someone insisted we try out Tekkit instead. Tekkit is a combination of several mods like BuildCraft, IndustrialCraft, and some others, running on a Bukkit 1.1 server. Seeing as Minecraft had become pretty stale for me, all these mods were a welcomed extension. Especially considering they took out most of the grinding of the vanilla game (e.g. constantly having to manually chop wood), I’d come to love all the mods in the Tekkit pack.

However, one problem with the Tekkit pack is that it’s horribly outdated, and most of the mods are pretty buggy, even though most of those bugs have been fixed by newer versions. Being unhappy with this state, I set out to look for updated mods and stumbled upon a few problems:

  • Most mods are incompatible with Bukkit.
  • Most mods are specifically tailored for a specific version of minecraft.
  • Most mods are not updated to the newest version of minecraft at all.
  • Combine the two before, and you see that even though there’s a newer version of mod A, you’re still stuck with an old one because mod B forces you to use an older version of minecraft.

Luckily there is a team at mcportcentral that tries to port all the mods to Bukkit and thus solves most of my problems, but the whole infrastructure of mods is pretty horrible ๐Ÿ™

The compatibility problem: Obfuscation

From what I’ve gathered this far, the main problem is Minecraft’s obfuscation, and the fact that different teams are using different deobfuscation keys.

The version bit comes down to the fact that each version of minecraft is obfuscated differently, and as such the HelloWorld class might’ve been called a in version 0.1, but is now called b in version 0.2. If a mod depended on the HelloWorld class, and as such was compiled against the a class in v0.1, it will fail on v0.2 because that class is now called b. Even if nothing actually changed in the HelloWorldย class, and even though the mod shouldย remain mostly compatible, the obfuscation breaks everything.

The problem gets even worse when there are different deobfuscation keys being used. For example, let’s look at Minecraft Coder Pack (MCP) and CraftBukkit. The first will decompile and deobfuscate a Minecraft jar file, and spit out the source code and possibly a deobfuscated jar file. CraftBukkit is an already decompiled and deobfuscated version of the server, with some extra changes to allow for the Bukkit API. Both are, however, using a different deobfuscation key. So let’s say we have the HelloWorldย class again. MCP could deobfuscate this to OhHaiWorld, whereas CraftBukkit might have deobfuscated it as WelcomeWorld.

So let’s say that instead of writing my mod for the vanilla minecraft server, I pick a deobfuscated one because it will allow me to keep it easily updated. (Note: this is what most modders do, but they re-obfuscate it with release so it’s easier to apply for normal users).Even though it’ll now probably work on Minecraft 1.2.1, 1.2.3, and 1.2.5 (provided the user deobfuscates first), it will still not work on both CraftBukkit and the Vanilla server.

Combination problem: Overwriting files

Another problem in the actual modding is that often you need to intercept parts of Minecraft, or change bits of it. Usually this is done by simply changing some of the Minecraft classes. The problem is that you can’t combine two mods that modify the same class, and if you’re using a different brand of server (e.g. CraftBukkit) that already had several changes in that class you can’t use a mod that modifies it too as it will undo the changes in that specific brand server.

Solutions

Luckily, there are a few solutions available, although most include a shitload of manual work too.

For example to manually port a mod from 1.2.3 to 1.2.5, you could get the MCP for 1.2.3, deobfuscate the mod, then get the MCP for 1.2.5 and reobfuscate it. If it doesn’t actually overwrite any of Minecraft’s files (e.g. it’s using something like Forge or ModLoader(MP) to do the actual hooks in minecraft’s code), this should be enough. If it does overwrite files, you should compare the (decompiled and deobfuscated) changes from the vanilla 1.2.3 server, and reapply those changes in the 1.2.5 server manually.

Porting a mod from vanilla to CraftBukkit usually involves more work. Maeyanie luckily keeps a list of name changes from MCPs deobfuscation key to Bukkit’s deobfuscation key which you can use to automate a part of the work. However, as CraftBukkit usually entails a lot of changes from the vanilla server, you still need to manually sift through any changes to Minecraft’s code. If the mod only adds files and uses Forge or something, that won’t be a lot of work, however if you’re porting Forge itself, you’re up for a lot of manual comparing and coding.

My involvement

I’m hoping to get more involved in the minecraft, and mostly CraftBukkit and Forge, mod scene at some point. I’ve got several ideas to try to automate more of the porting process, even though I’m not sure if I have the time to implement them all. Feel free to keep track of this blog for any future work on this.

Also, if there’s anything unclear, plainly wrong, or confusing, do feel free to use the comment field or contact me through other means ๐Ÿ™‚

Getting the Media Center Remote working

XBMC was supposed to work out of the box with the Media Center Remote, but for me it never did. Apparently this is because in linux kernel 2.6.something they merged some of those drivers into the kernel, X started handling them, and stuff like that.

To get it working, I had to take the following steps, most of which came from or were based on a thread on the XBMC forum.

Normally the up/down keys seem to work, but that’s only because X11 is treating the remote as keyboard input and is channelling the arrow keys properly whereas the other keys are a mystery to XBMC. Although it would be possible to bind all the other keys manually, I’d rather use XBMC’s normal way of doing this, namely through lirc. So the first step is to get X11 to *not* do anything with the remote input.

To prevent X11 from handling the remote input, add the following lines to /etc/X11/xorg.conf:

Section “InputClass”
Identifier “Remote”
MatchProduct “Media Center Ed. eHome Infrared Remote Transceiver”
Option “Ignore” “True”
EndSection

Then we need to get lirc up and running, so first let’s install it:

apt-get install lirc

Next we need to tell lirc what device the remote is on, seeing as the new kernel driver doesn’t use the normal /dev/lirc but /dev/lirc0. Edit /etc/lirc/hardware.conf and change the following three lines (just find the line where everything before the = is the same)

DRIVER=”default”

DEVICE=”/dev/lirc0″

Then we need to load the mceusb config file shipped with lirc:

cp /usr/share/lirc/remotes/mceusb/lircd.conf.example /etc/lirc/lircd.conf

At this point you should restart lirc with:

/init.d/lirc restart

And then try running

irw

and pressing some keys on the remote. It should give some output. Note that the ^]B like output is output generated by the kernel for the arrow keys. Try the big “OK” button to be sure, I’m fairly certain that the kernel doesn’t map that one. Ctrl+c out of this when you’re ready testing.

Finally, XBMC expects the MCE driver to give slightly different output. To fix this, we should print or open on another computer the file/etc/lirc/lircd.conf and open /usr/share/xbmc/system/Lircmap.xml. Then for each item in the <remote device=”mceusb”> section, change the middle bit to something from the lircd.conf file. For example:

<play>Play</play>

should become

<play>KEY_PLAY</play>

After you’ve changed all those and restarted X and XBMC, it should be working.

Finally because I wanted the box to sleep when pressing power instead of shutting down completely, I opened /usr/share/xbmc/system/keymaps/remote.xml and changed the line:

<power>XBMC.ShutDown()</power>

to

<power>XBMC.Sleep()</power>

Now I had actually thought to be done by now, but after a few tries it seems the computer randomly shut down sometimes. This was caused by the new kernel drivers also passing all keypresses to the input subsystem, and the input system deciding that KEY_POWER really meant KEY_POWER. To fix this, add the following line to /etc/rc.local, preferably at the start.

echo none +lirc > /sys/class/rc/rc0/protocols

This will tell the kernel to not attempt to decode the IR signal using any protocols, but instead only send it on to LIRC.

XBMC on Asus AT5IONT-I using Debian

I finally figured out how to install a mostly working XBMC system on my new media center, a shiny Asus AT5IONT-I board with 4GB of ram.

Because it was quite a hassle, here’s the guide on how I did it, hoping that other people might benefit from it, and possibly myself too if I ever forget this when I need to reinstall ๐Ÿ˜‰

First things first, a quick overview of what I’ll be installing. I’ll be using Debian wheezy, or testing currently, with XBMC from the www.debian-multimedia.org repository.

We’ll start by inserting the Debian Wheezy CD and starting the installation. It’ll complain about firmware for the network interface, just hit continue on that, we’ll manage that later.

When asked about which software to install, untick everything but the the SSH server (as that will make it much easier to troubleshoot if something goes wrong). You could possibly also add the Standard System Utilities, but I didn’t as I wanted the system to be as lean as possible, and if I needed anything specific, I could just apt-get it.

From here on, most of the configuration can be done either through SSH, or through the terminal (ctrl+alt+f1 after we’ve installed the graphical bits)

Once Debian is installed and running, the first step is to get audio working. Why audio? Because it’s a bitch to get working on this motherboard, and I’d rather know sound is working before I install any other parts. So on we go, installing alsa and pulseaudio:

apt-get install pulseaudio alsa-base pulseaudio-utils alsa-utils

At this point, audio should be able to play through the analog channels (over the Intel chipset on the board), but HDMI is still a bitch to get working (it’s possible, just more difficult). The problem seems to be that the nVidia chipset should have only one output (HDMI), yet due to some driver issues multiple are recognized. To fix this, we need to set some module options on the HDA Intel driver. For other boards, see the XBMC guide on setting up HDMI audio for NVidia ION and other cards. For my card, the you need to add the following line to /etc/modprobe.d/sound.conf using, for example, nano /etc/modprobe.d/sound.conf:

options snd-hda-intel enable_msi=0 probe_mask=0xffff,0xfff2

Then type:

modprobe -r snd-hda-intel
modprobe snd-hda-intel
aplay -l

The first two commands will unload and reload the intel HDA driver, the second should lits all audio devices. At this point, there should be two device on card0, one analog and one digital, and just one device on card1, the HDMI one.

Now to actually test HDMI, we need the nvidia drivers (and X11) to be loaded. First, open /etc/apt/sources.list and add the words contrib and non-free to each of the sources. Then type:

apt-get update
apt-get install nvidia-glx nvidia-xconfig xorg
nvidia-xconfig

Seeing as apt is now again whining about the missing firmware bits, this is also a good time to fix that:

apt-get install firmware-realtek

For some reason, the nvidia driver won’t properly load unless the system is restarted, so let’s do so now.

shutdown -r now

Now let’s get to installing the actual XBMC packages. open /etc/apt/sources.list and add the following line:

deb http://www.debian-multimedia.org/ wheezy main non-free

Then do:

apt-get update
apt-get install debian multimedia keyring
apt-get update

And finally install XBMC:

apt-get install xbmc-standalone

Finally we need to have XBMC start when the machine boots, for which I’m using this solution. Simply edit /etc/rc.local and add the following line just before the exit line:

su – xbmc startx &

And then make xbmc the window manager with:

ln -s /usr/bin/xbmc-standalone /usr/bin/x-window-manager

And finally allow XBMC to actually start the X daemon, by running

dpkg-reconfigure x11-common

and specifying that Anybody can start it.

At this point you should try another reboot and see if XBMC starts by default. Now to get sound through HDMI, we’ll have to set it up in XBMC. Go to System -> System -> Audio Output and

  • Audio output: HDMI
  • Audio output device: High definition audio controller … (the HDMI one)
  • Passthrough output device: Custom, hw:1,3 (note, this is for this specific motherboard, use aplay -l to find which one you need)
  • If your display doesn’t support passthrough output, disable those.

This should make video and music go through the proper channel. Test this by starting up a song or movie, and seeing if you hear anything. If you don’t, start alsamixer and play around with the volume controls. I needed to press F6 to change to the nvidia card, and then unmute the SPDIF switch the the m key.

Video and music should now work, but XBMC-menu-blips will still go through the analog ports. ย To fix that we need a shell as the xbmc user, and execute pacmd which allows us to talk to pulseaudio. Make sure XBMC is running and in the menu.

First, find the sink you want it to output on with list-sinks, memorize the index. (use shift+pageup/down to see more on a normal terminal).

then type:

set-default-sink the_index_you_just_found

Next, we need to force XBMC on that sink too (as else it’ll save which one it was on before). Find it’s index by doing list-sink-inputs, and then type:

move-sink-input xbmc_id sink_id

And the blips should work now. At first with some delay, but that will go away after playing music or video (XBMC just needs to reopen pulse). You can type exit to exit the pulseaudio command shell.

Next up is allowing XBMC to shut down or put the computer to sleep, I used this guide.

apt-get install upower acpi-support

Then open /var/lib/polkut-1/localauthority/50-local.d/custom-actions.pkla and add the following lines:

[Actions for xbmc user]
Identity=unix-user:xbmc
Action=org.freedesktop.upower.*;org.freedesktop.consolekit.system.*;org.freedesktop.udisks.*
ResultAny=yes
ResultInactive=no
ResultActive=yes

While we’re at it, also install the following packages to allow USB auto-mount:

apt-get install udisks usbmount ntfs-3g

And open /etc/usbmount/usbmount.conf to add ntfs to the allowed filesystems, simply add it at the start of the FILESYSTEMS=”…” list.

Now to speed up booting a bit, I edited /etc/default/grub and set GRUB_TIMEOUT to 0, after which I ran

update-grub

At this point the XBMC is pretty much ready. All that’s left to do is set up the network access for smartphone remote control, and add all movie sources.

If any new developments are added, I’ll post a new item on this blog ๐Ÿ™‚

Fixing backslashes in SVN from windows

At Iminent someone did something horrible:
The trunk pretty much contained:

  • file a
  • dir b
    • dir c
      • file d
    • file e

And someone ,on presumably a mac or a *nix machine, added the directory “b\c”. In *nix, where the SVN server was run, this was a perfectly valid directory name. On Windows, what most of us run on, the backslash is a directory seperator.

So when TortoiseSVN tries update or check-out, it fails to create a directory like that. Even worse, when you try to delete the offending directory from the “Browse Repository”, it actually interpreted the \ as a directory seperator, so while you wanted to delete directory “b\c”, it ended up deleting directory c in b. My boss had fixed this by completely removing the trunk, and putting in a new one from his working copy, but this way you lose all history of the files, and any changes that he had made to the working set were not visible anywhere.

I fixed it by doing the following:

  • First, you branch from the Trunk revision just before those faulty directories were added.
  • Then you carefully merge all changes from trunk from that point to HEAD to that branch, while excluding all revisions that add those offending directories.
  • Then you branch the current trunk to a branch like “MessedUpTrunkBackup” using TortoiseSVN Repository browser (You can’t do it by checkout + branch, as checkout fails)
  • Then you delete the current trunk, again using the repository browser.
  • Then you copy the clean branch you made earlier to trunk, using either the repository browser or by using normal branching methods.

Tada: You now have a new trunk, without the offending changes, but with full history on all files, and you can safely issue a checkout, or an update on an earlier working copy ๐Ÿ™‚

I am aware it’s probably possible to cleanly revert those changes from a *nix box, but we didn’t have any available at the moment, so I had to do it this way.

V8 crashing with ‘Uncaught RangeError: Maximum call stack size exceeded’

While working with V8 in another process, it kept crashing at initialization with the message ‘Uncaught RangeError: Maximum call stack size exceeded’. I wasn’t doing anything big, just initialization.
After about 3 hour of debugging, I found out what was wrong. Normally, V8 will assume that you have no more than 512KB of stack space. To prevent stack overflows, it will take a stack address, substract 512kb from it, and remember that address. If it ever passes that address, it’ll throw a RangeError.

The problem lies in the fact that the program I was working with had a stack somewhere around 0x60000 or 384kb. V8 then substracts 512 from that, but instead of getting a nice stack-boundary, it ends up with an incredibly big number due to integer underflow. The next time it checks the stack, it compares the stack address (still somewhere around 0x60000) with it’s calculated stack limited (which, due to the underflow, is about 0xFFFE0000), and assumes it has a stack overflow.

To fix this, I had to manually set the stack limit. Basically, the following code takes a random stack address, divides it by 2, and uses that as the lower-limit to the stack:

	v8::ResourceConstraints rc;
	rc.set_stack_limit((uint32_t *)(((uint32_t)&rc)/2));
	v8::SetResourceConstraints(&rc);

New Live Messenger APIs

While the old messenger APIs have been steadily crumbling the past few versions, and the latest WLM10 beta having made it nearly useless, I was amazed to find several new APIs.
One of them is the “LivePlatform” API, which seems to be what messenger is using to retrieve the contact list and social news.
I don’t have time to investigate this fully, but I did get to put together a small demo. To try it, create a new console application in C#, add C:\Program Files\Windows Live\Contacts\LivePlatform.dll as a reference, and put in the following code in Main:

 LivePlatform.LivePlatformFactory factory=new LivePlatform.LivePlatformFactory();
            LivePlatform.ILiveIdentityCollection ids = factory.IdentityService.CachedIdentities;
            System.Console.WriteLine("Cached identities");
            for (int i = 0; i < ids.Count; i++)
            {
                LivePlatform.ILiveIdentity id = ids[i];
                if (id.HasPassword)
                    System.Console.WriteLine("\t"+id.Username);
            }
            System.Console.Write("Username: ");
            string szUsername = System.Console.ReadLine();
            LivePlatform.ILiveIdentity found = null;
            for (int i = 0; i < ids.Count; i++)
            {
                LivePlatform.ILiveIdentity id = ids[i];
                if (id.HasPassword && id.Username.ToLower().Equals(szUsername.ToLower()))
                {
                    found = id;
                    break;
                }
            }
            string szPassword = "";
            if (found == null)
            {
                System.Console.Write("Password: ");
                szPassword = System.Console.ReadLine();
            }
            platform=(found==null)?factory.CreateEx(szUsername,szPassword):factory.Create(found);
            platform.Config["BlockingSignin"] = true; //Don't set this to make Signin non-blocking. You'll have to wire up events to listen for OnReady before you do anything, though.
            platform.Signin(null);
            System.Console.WriteLine("Full name: "+platform.Me.FullName);
            LivePlatform.ILiveObjectCollection query = platform.Query("type(contact)");
            System.Console.WriteLine(String.Format("Contacts: {0}", query.Count));
            for (int c = 0; c < query.Count; c++)
            {
                LivePlatform.ILiveObject obj = query;
                LivePlatform.ILiveContact contact = obj as LivePlatform.ILiveContact;
                System.Console.WriteLine(String.Format("{0} ({1})", contact.FullName,(contact.Emails.Count>0)?contact.Emails[0].Address:"none"));
            }
            System.Console.WriteLine("Press enter to quit");
            System.Console.ReadLine();

It’ll print your full name, as well as your full contact list. Also note that if you added your facebook account, those contacts will show up too.
Some other queries I saw when debugging messenger:

type(invite)
type(contact) and contacts:canHideFrom(true)
type(contact)
type(contact) and contacts:isHidingFrom(true) and contancts:canHideFrom(true)
type(contact)
type(circle)
type(category)
type(category) and contacts:isfavorite(true)
type(contact) and contacts:isonline(true)
type(category) and contacts:isfavorite(true)

I believe that hooking this API in messenger might allow for some pretty cool tricks, for example something like BuddyFuse.

As a final note, I’d like to say that I’m posting this because I’m hoping more sharing might make the messenger community more like it once was. If you find out more about this API, or other new APIs, please be so kind to share them too so a new sharing eco-system might emerge. Thanks.

EDIT: Added cached identities.

CyanogenMod modded

For my Nexus One, I’ve been trying a few new ROMs. One that is supposedly very good is CyanogenMod, and I personally kinda like it.
One downside however, is that the power widget colors for features turned on have been changed to an ugly light blue-ish color, that completely mismatches with any other android user interface parts. So I did what any respectable tinkerer would do, and changed them back to green. Here’s how I did it, hoping these instructions might help other people with the same or related problems. Note that all the tools I uesd should be easy to find using google ๐Ÿ™‚

Apparently, the power widget is in Settings.apk. This file also contains a lot of other stuff, and not suprisingly the settings dialogs are among those. Seeing as CyanogenMod adds quite a bit to the settings dialogs, simply putting back a stock Settings.apk is not an option.
So instead, we should change something inside of Settings.apk. This is what I did to get the Settings.apk (and back it up) using the Android SDK:

adb remount
adb shell
cd /system/app
cp Settings.apk Settings.bak
exit
adb pull /system/app/Settings.apk Settings.zip

Then I opened Settings.zip with winrar, and browsed around. I found the files needed in res/drawable-hdpi under filenames appwidget_settings_ind_on_?.9.png and appwidget_settings_ind_mid_?.9.png (where ? is c, r and l).
My first attempt was simply to change the colors using GIMP, but that messed up the scaling of the pictures.
To fix that, I used a program called tweakpng, and removed all PNG chunks not in the original file, and copied the one chunk that the new file didn’t have from the old file.
This targets the scaling issues, but somehow I still had some weird blue lines through the images.
Instead of turning it to white as I wanted, I figured the normal Android green would be sufficient too. So I set out to get it from a stock ROM.
I found the stock Nexus ROM, and found the system.img. Apparently it’s a YAFFS2 filesystem, and after some googling, I found a tool unyaffs (needed to compile that on a linux box) to extract it. As said before I couldn’t just take the Settings.apk from there, so instead I renamed it to Settings_orig.zip, and with winrar moved the files I found earlier from the original into the modified zip file.
Then I used

adb push Settings.zip /system/app/Settings.apk

to push it back onto my device.
After re-adding the widget, the colors were green again ๐Ÿ˜€

Compiling V8? Do not use Python x64!

When compiling Google V8 on Windows, do not use an AMD64 version of Python.
Firstly, the scons installer will not function, as it can’t find a python version in the registry.
Secondly, even if you just install scons manually, the V8 SConstruct file will fail. Because the return value from platform.machine() is now AMD64, the GuessArchitecture function in V8’s build script will not recognize a valid architecture, and the entire thing will fail.
If you just uninstall Python, and re-install an x86 version, everything will function fine.

My rtorrent setup

I hate leaving my main desktop turned on at night. I’m pretty sure it has something to do with the fact that it’s: a) noisy, b) in the same room I sleep in, and I: c) can’t sleep with a lot of noise or light in my room. This is why I have a seperate computer (without monitor that is) in another room that does all the stuff I’d want to be doing at night. It used to record from the TV channel, but dutch television sucks when it comes to good series, so a while ago I put a torrent client on there. Here’s a description of my old setup, and how I set it up today.
Continue reading My rtorrent setup