Friday, July 30, 2010

Wii + Python = Motion Capture + Gesture recognition

Here’s the second installment in the series of videos I’ve been uploading recently on using the Nintendo Wii and Python to provide intuitive means of controlling applications on Ubuntu.

I plan to demonstrate this and much more at PyCon India, 2010, if my proposal is accepted. The Nintendo Wii has an IR Camera built into it with a really powerful on board controller (an SOC) that basically does a lot of hard work to present simplified coordinate based tracking. Each controller can track up to four points in two dimensional space (there is an element of depth as well, but more on that later).

Using the “CWiiD” Project, an open source interface for the Wii remote on Linux, I have developed a small “sandbox” application that allows you to manipulate an object using my hands, as opposed to the traditional forms of control viz. the mouse and keyboard. What’s even more impressive is that the entire hardware will cost only under $50. This provides almost limitless possibilities for the amount of intuitiveness that we can add to an application.

Currently, there is no device driver in place, but with a little help from the open source community and a little “rocket science” (basically a library for recognizing commonly used gestures), we can change the way we interact with our PCs forever.

Here is the video. Hope you enjoy this as much as I enjoyed making it! Cheers to PyCon 2010.

If you like this video, comment on this post, share it with your peers and support PyCon in India and around the world.

Tuesday, July 27, 2010

Wii Remote + Python + Linux

Here’s a teaser on the applications that I’ve been building recently for my proposal to present a talk at PyCon India, 2010. The cwiid project provides a simple python wrapper which allows application developers to interface with the Nintendo Wii remote over bluetooth.

I have build a small wrapper on top of the cwiid module that allows you to run a thread which connects to the Wii remote and sample data from it at regular intervals and feed that data to a designated callback routine. This approach improves the usability of the cwiid module and allows a developer to easily embed Wii remotes into their applications.

Intuitive control is something that never comes cheap! Touch screens cost a fortune and are not supported by every application. Using the Wii, and some clever programming, gesture and pattern recognition can be brought to almost every desktop. Brian Peek’s library has been a rich source for developers on Windows using the .NET framework, but it seems like there is little happening on Linux as far as expanding or maintaining a decent Wii library is concerned.

Here is a taste of what we can achieve with a little imagination and a few lines of code. Check out this video I made today, that allows me to set my desktop on fire :D

There will be more updates headed your way very soon. If you like what you’ve seen, comment on my blog or “like” this video on YouTube. Support PyCon India, this year and the next.

I dedicate this video to Brian Peek, whose work on creating a Windows based library for the Wiimote inspired many others to take it beyond the barriers of operating systems and languages. Cheers to you mate!

Saturday, July 24, 2010

My thread burnt down my computer!!!

I've been working on developing applications for my PC using the wii lately. In order to get me going, I found an open source library called "cwiid" with a python wrapper, "python-cwiid". Yesterday, while coding, I kept facing a peculiar problem. Every time I ran my application (which was merely something that connected the nearest wii remote and started receiving accelerometer and key press information from it), my laptop got extremely hot and my computer shut down.

I was quick to understand that this had something to do with the CPU being pushed into thermal cutoff. Interestingly, a previous application that I wrote came in handy. My TempMonitor application allowed me to get a visual comparison of the CPU usage and temperature against time. So I decided to see for myself, how brutally my CPU was being ravaged by the wii application I had made. The results were quite astounding. My application pushed my CPU upto a 100 degrees centigrade in under 15 minutes (coupled with a few other apps like games or presentations for which the remote was suppose to act as a new interface) and caused it to shut down.

So I took a deeper look at my code. My app was essentially a wrapper for the "Wiimote" class provided by python-cwiid. It allowed me to run a thread that received data from the wii remote and invoke a callback routine to handle the processing. This was extremely essential as it was the only way to abstract the nitty gritties of the wii from the actual application. After an hour or two of breaking my head, it hit me like a tonne of bricks.

My main routine started the thread and died, making a "orphaned zombie" (Now this might sound strange since, an orphan thread is one whose parent thread has died. A zombie process is a defunct or redundant entry in the process table. In our case, the cycle of calls is Python Interpreter --> main --> background process. Here, the main routine dies, but the interpreter doesn't. So the background thread has lost its parent but still has a grandparent viz. the interpreter. And technically, since the application is over after the death of the main routine, the python interpreter is a redundant entry that continues to live on in the process table. So the nomenclature!). I quickly added a semaphore to my main routine, that kept it alive until the background task was not complete, and "peace was once again restored to the empire". But then I decided to put my theory to the test. I wrote a script that mimiced the structure of my Wii application.

It had a background task with some heavy processing (well not as heavy as my app, but enough to put a little squeeze on the core), and in order to test its effect on the CPU load and temperature, i create three kinds of main routines:
  • the first one died after starting the background task, orphaning it
  • the second one had a semaphore but was a bigger time hog than the background task (i.e. it didn't sleep and fairly share time with the background task)
  • the third one had a semaphore that made the main routine doze every once in a while and provide more time to the background task
The processing was not too heavy, as i wanted to put just enough load to cause a steady rise in temperature. The background task's job was to open a JPEG and read out all the pixels (values between 0-255). Now the question to ask was, "what would be the threshold temperature"? I wanted each of the cases to eventually break down, and yet give comprehensive results on the effect a simple "sleep statement" can have. Normal range of operation of most cores is between 55 to 90 degrees centigrade. An average load on my computer is between 65-70% (which is higher than most regular computer users) and it usually works between 60 C to 70 C. So after a little deliberation, I chose 70 degrees centigrade as my threshold. The load that the application was going to put on my 2.8 GHz CPU was going to be much above ordinary and more importantly, it was going to be persistent (yes, that's something we normally don't face. Most applications have asynchronous demands which are resource heavy indeed but for brief periods of time. Therefore, we don't really feel the CPU running out of steam or heating up too much. Persistent applications are the best kind of load to make your CPU sweat!).

Here is the script:


import time,sys
from threading import Thread

#---------------------------------------------------------------------------
class backgroundTask (Thread):

""" simple background task """

def __init__(self):
""" simple thread constructor """
Thread.__init__(self)
self.stop = False
self.cnt = 0
#------------------------------------------------------------------
def bizzareProcessing(self,num):
""" some heavy duty processing for testing pruposes """
fileRead = open('APPLE.jpg')
image = fileRead.read()
for eachbyte in image:
print ord(eachbyte)
fileRead.close()
print "_"*30
#------------------------------------------------------------------
def run(self):
""" worker routine for the thread """

print 'starting background task'
while not self.stop:

self.bizzareProcessing(self.cnt)
self.cnt+=1
time.sleep(0.1)

print 'exiting backround task'
#------------------------------------------------------------------
def stop(self):
""" This is the kill switch """
self.stop = True
#------------------------------------------------------------------
def getCount(self): return self.cnt


#---------------------------------------------------------------------------
def mainZombie():

bgTaskZombie = backgroundTask()
bgTaskZombie.start()
#now the parent thread, i.e. this function will die, but the
#task bgTaskZombie is still going to be running
print '\nMainZombie is dying now'
#---------------------------------------------------------------------------
def mainHog():

bgTaskHog = backgroundTask()
bgTaskHog.start()

while bgTaskHog.getCount == 20000: pass
#the semaphore above keeps the parent thread from dying before
#the child thread, therefore not making it a zombie but no
#sleep in the loop causes the main routine to hog cpu

print '\nMainHog is dying now'
#---------------------------------------------------------------------------
def mainProper():
bgTaskProper = backgroundTask()
bgTaskProper.start()

while bgTaskProper.getCount == 20000: time.sleep(1)
#same as the mainHog routine, only with a little nap for the main
#routine. This causes the main routine to share time and stay alive

print '\nMainProper is dying now'
#---------------------------------------------------------------------------
if __name__ == '__main__':

arg = int(sys.argv[1])

if arg == 1: mainZombie()
elif arg == 2: mainHog()
elif arg == 3: mainProper()



The results are shown below. My temperature monitor application uses google charts API to generate a few graphs of CPU load (blue trace) and CPU temperature (green trace) against time. It records the changes in these two quantities over the last 400 seconds. I ran each of the main routines and clobbered the images together for a clear comparison. Now I did run each main routine over five times to make sure there was consistency in my readings. Here are the plots for the last run:


As you can see, it takes longer for the most properly structured application to reach the threshold, while the one with the zombie hanging around is quick to touch the threshold temperature in no time.

Problems like this may go undetected and surface in an untimely fashion to make your life a living hell. So structure your threads well. Make sure all the babies have their parents and everyone gets enough sleep.

Thursday, July 22, 2010

Automated Wallpapers for your Ubuntu Desktop

I was quite bored with the ill written dynamic wallpaper utility on Ubunutu. It uses the wallpapers stored in /usr/share/backgrounds directory in union with an XML file, to change your desktop wallpapers. If you wanted to add your own collection of wallpapers, you'd have to put them in that directory and edit the XML file and god save you if you accidentally deleted some wallpapers, 'cause then you're going to get a plain screen every time it tries to load a dead link.

So I wrote my own little desktop utility called "Wallshaker". It allows you to load multiple directories as sources of images, monitors them and doesn't load dead links. If new images are added to any of the monitored directories, they are picked up as well. You can even configure the timing between wallpapers (in seconds). But hey, this is pretty regular stuff. You get a truckload of applications that do just the same thing. "DRAPES" for one is a decent app and is available through the repositories. So what is it that sets this application aside.

Well, my little app has a checkbox that reads "Live World Map". If you click that, satellite images of the sunlight map of the world is grabbed from the web and set as your wallpaper. The map is refreshed multiple times during the day as and when new images are available. Cloud patterns are update every few hours and you can actually see how one part of the world moves from light to darkness during the course of the day.

Now you can simply look at your desktop and see which parts of the world are "awake" or "asleep" :D

But simply using this was not enough. So I wrote another utility that would grab my desktop every now and then and made a movie out of it. Its a small clip, spanning almost six to seven hours of the day squeezed into 15 seconds. But it clearly shows you how your desktop changes as the earth moves about its axis and as the day blends into the night.

Here's the link to the video. As always, you can download the app from my projects page.

video

=-=-=-=-=
Powered by Blogilo

Wednesday, July 21, 2010

Taking periodic snapshots of your Ubuntu Desktop

I've been working an application (will be posting more about that a bit later), but in order to test it, i needed a tool that would constantly take snapshots of my desktop every few minutes and save those snapshots in my home folder.

I thought long about doing this, without support from external applications and using only those that were native to gnome. Unfortunately, that didn't work out too well, simply because the snapshot tool in gnome captures the image and pops up a save dialog box with a preview of the snapshot taken. I didn't want to sit and record a macro as they don't always work too well.

That's when the guys at google tossed out "imageMagick" a.k.a "import" at me. This is a utility that comes packaged with Ubuntu (i'm using 10.04 LTS). And it works purely off the command line without any confirmations or dialogs. Here's a sample of how its used:

shell$ import -window root some_filename.png

This saves the image "some_filename.png" into the current working directory. Its really compact and works inline at the shell. The next thing I wanted to do was to minimize all the applications (CTRL + ALT + D, show the desktop basically) before the snapshot was taken and restore them after my work was done. This is where I feel the .NET framework's SendKeys("keys") makes so much sense. You can use that to send any key combination to the OS. Unfortunately, none of that was available to me on Ubuntu, so I found another command line utility (a really really neat one at that!) called "wmctrl". It can be used for a variety of tasks, and here's how it can be used for minimizing and maximizing all windows:

shell$ wmctrl -k on #minimize all windows to show desktop

shell$ wmctrl -k off #maximize all windows

Finally, combining the two, I came up with a small python script, that allowed me to take snapshots of my desktop at regular intervals. The interval, in minutes, can also be specified at the shell. Here's the script.


Tuesday, July 20, 2010

Small program "logger" module in Python

I was sitting around and a thought crossed my mind, "How can I get the function name of a function that the trace is in?". I thought about it for a few minutes, and said to myself, "The stack must know". But how do I read the function names of the stack in Python??
For all these system level hacks, usually the sys module has something to offer, and this case was no exception. Here's how you can do it (using something called a "frame object"):


print sys._getframe().f_code.co_name

Another useful modification of the snippet above can give us the name of the function that is calling the current function. This is how it looks:


print sys._getframe(1).f_code.co_name

So as you saw, the argument of sys._getFrame() can take you up or down the stack as your need may be. This utility of the sys module is extremely useful while logging for errors. Almost every good programmer uses some kind of logging in his code. So far most of my logging was using independant file writes or console writes. With my newfound knowledge, all that was going to change, so I wrote myself a small, yet handy logger for my forthcoming projects. Here's the script.

I integrated this into one of the apps I was making and well, it was simply fantastic. Now I just piped out the log messages to this module and it would automatically determine which function was sending the messages. I wrote an app with a GUI in PyQt4, and as will all GUI apps, tracking events of widgets is extremely important in figuring out what went wrong during debugging. This is what the output of my new logger looked like:


Tue Jul 20 12:32:53 2010: <loadAppData()> Trying to load application data
Tue Jul 20 12:32:53 2010: <loadAppData()> Values picked up from appdata file. Interval:43; Dirs:[]
Tue Jul 20 12:32:53 2010: <loadAppData()> Application data loaded successfully
Tue Jul 20 12:32:56 2010: <btnAdd_Click()> Add button clicked
Tue Jul 20 12:33:12 2010: <saveAppData()> Trying to save application data
Tue Jul 20 12:33:12 2010: <saveAppData()> Application data saved successfully!
Tue Jul 20 12:33:26 2010: <trayIcon_Click()> Tray icon clicked. Current visible status:True
Tue Jul 20 12:33:26 2010: <trayIcon_Click()> Tray icon event complete. Current visible status:False
Tue Jul 20 12:33:30 2010: <trayIcon_Click()> Tray icon clicked. Current visible status:False
Tue Jul 20 12:33:30 2010: <trayIcon_Click()> Tray icon event complete. Current visible status:True
Tue Jul 20 12:33:35 2010: <closeEvent()> Form close event raised
Tue Jul 20 12:33:35 2010: <saveAppData()> Trying to save application data
Tue Jul 20 12:33:35 2010: <saveAppData()> Application data saved successfully!
Tue Jul 20 12:33:35 2010: <closeEvent()> Program is exiting.
This is so neat :D

Monday, July 12, 2010

Remote control your PC using a Wii remote

I’ve been tripping on the Wii lately, and another application that I cooked up over the last couple of days was something I like to call “WiiCONTROL”. Its an application that allows me to control navigation on Windows 7 using the Nintendo Wii remote.

I have built my app on top of Brian Peek’s Wiimote library. I hope you’re reading this Brian :D

The need for the application arose while I was watching a movie a night ago, and I realized that using a wireless keyboard while lying in the comfort of my bed was rather annoying. I needed something that didn’t take much space and was “like a remote”. Lying next to my computer was the Wiimote I’ve been using off late.

The application took me nearly a day to make. The major portion of the app was figuring out how the navigation on windows could work solely through keyboard events and then tying them up to the button press events generated by the Wiimote.

Here’s what it looks like:

image

You can clearly see which functions are tied up to which buttons and the battery indicator gives you a rough idea about how much juice you have left in your remote. The experience counter may be used to control how responsive (key de-bouncing is such a pain!) your remote should be. The preset value works really well for me. The box titled “function” can be used to check which key corresponds to which function. You push a button and it shows up on that box.

Here’s my personal favorite. When you click “test feedback”, the app sends a signal back to your remote causing it to vibrate momentarily :D confirming a closed loop between your PC and your remote.

The app iconizes to your system tray and can be brought back to focus by double clicking the “YODA” icon (heehee!!!) therein.

Whats cooler, is that if you use VLC player, you can use the same remote to Play/Pause/FF/Rewind/Control Volume.

Here’s a video I made last night of me using the app on my home computer. Its a bit fuzzy thanks to the compression, but you can see what the deal is all about.

Go to my projects page to download the app. Please note that you have to hook up your Wiimote to your PC before starting the app. If the Wii is not found, it will simply notify you and exit. For a tutorial on how to connect your Wiimote to your Windows PC, go here.