A new kind of music video: Radiohead's "House of Cards"

Radiohead’s latest music video, “House of Cards”, was shot using LIDAR (Light Detection and Ranging) rather than cameras:

Learn more about how it was made at the Google Code Radiohead page. The coolest part is that Radiohead is making the data available to anyone who wants to play with it.

They provide some sample Processing code to view the data, but for whatever reason I couldn’t get it to work, so I wrote a simple little iPhone OpenGL ES application:

I’m unable to test it on a real iPhone since I don’t have a paid developer account, but presumably it should work.

View other people’s visualizations at the YouTube House of Cards group.

Multitouch JavaScript "Virtual Light Table" on iPhone v2.0

Now that iPhone 2.0 is out I started playing around with some of the new web features, and soon found that I had created the prototypical virtual light table that’s an essential demo for any new multitouch technology.

It’s about 100 lines of JavaScript. It grabs the 10 latest photos from Flickr’s “interesting photos” API and randomly places them on the screen for you to play with:

This is great if you have an iPhone with the 2.0 software, but desktop browsers should get some multitouch love too. So I started writing a little bridge that fakes multitouch events in desktop browsers. It’s far from complete, but it’s just good enough to get the virtual light table demo working.

So go ahead and load it up in the new iPhone MobileSafari or Safari 3.1+ / WebKit nightly (requires CSS transforms):


In desktop browsers it uses the previous clicked location as a second “touch”, so you can click a photo then click and drag another spot on the photo to resize and rotate (notice the yellow dot).

For a good overview of touch events and gestures, check out this SitePen blog post and Apple’s documentation.

Here’s the source for the fake multitouch bridge:


Clearly the reverse of this bridge would be even more useful, since iPhone only sends mouse events under specific conditions. The mousedown, mouseup, and mousemove events could be emulated using the touch equivalents to make certain web apps work on the iPhone without much additional work. Of course you would need to either cancel the default actions (i.e. panning and zooming) on touch events, or have some way to manage the interactions between them.

Chipmunk Physics engine running on the iPhone

A couple months ago I hacked together a demo of [Chipmunk Physics](http://wiki.slembcke.net/main/published/Chipmunk) engine running on the iPhone using the unofficial SDK. It shows the standard Chipmunk demos, but also it reads in accelerometer data using the method described on the [Medallia blog](http://blog.medallia.com/2007/08/iphone_accelerometer_source_co.html).

For the most part it was a fairly simple translation from standard OpenGL to the OpenGL ES the iPhone uses. If anyone is interested in the source code, let me know and I’ll see if I can find it… *edit: source is posted. see below*

[iPhone Chipmunk Physics demo application](http://tlrobinson.net/misc/iphone_chipmunk_physics/chipmunk_demos.tar.gz) – run from the command line (either using MobileTerminal.app or over ssh), only tested on iPhone software 1.0.2.

*Update: [the source](http://tlrobinson.net/projects/iphone/Chipmunk-iPhone.tar.gz) – this is just modified from the original Chipmunk Physics demo, so the original license applies. I can’t support this at all, so you’re on your own. I believe I used the hacked iPhone toolchain version 2 (or 3?), so you may have to edit the Makefile if you have a different version or installed it somewhere other than /usr/local/bin/.*

Geolocation possibilities on the iPhone

Ever since the iPhone was announced, I thought it would be really cool if it had a GPS receiver, or at the very least was able to find your general location via cell phone tower triangulation (technically [trilateration](http://en.wikipedia.org/wiki/Trilateration)). The obvious benefits of such a system would include automatic positioning of Google Maps to your location, or geotagging of photos taken on the iPhone’s (surprisingly decent) camera. Unfortunately, no such feature was included in the iPhone.

Now that a decent set of developer tools has been put together by hackers, there is the possibility of adding something like this. A few other people, including the famed [geohot](http://iphonejtag.blogspot.com/2007/08/im-at-rit.html), have also expressed interest in such a project. This is a summary of my endeavor so far to add some sort of location finding tool to the iPhone.

### Cell Tower Triangulation / Trilateration ###

The basic idea behind cell tower [triangulation](http://en.wikipedia.org/wiki/Triangulation) and [trilateration](http://en.wikipedia.org/wiki/Trilateration) is to locate the phone’s position relative to the cell towers by using the angles and distances between them, respectively.

Triangulation on the phone is impractical since there’s no way to easily measure the angles between the phone and the cell towers.

Trilateration is *sort of* possible by using the signal strengths to estimate the distance to the cell towers. This is very inaccurate due to multipath interference from stationary and moving objects, variation in cell tower antennas and equipment, etc. However, it would be *good enough* to get an approximate location to be used in Google Maps or rough geotagging of photos, assuming you know where all the cell towers are and can get their identities and signal strengths from the phone. That’s where the problem is.

### Issues with Trilateration ###

Cell towers locations are *supposed* to be public information, and the FCC does have a database called the [Antenna Structure Registration](http://wireless.fcc.gov/antenna/) (ASR)… BUT it’s missing many towers (it’s rumored to be about 25% complete), and the bigger problem is the networks aren’t required to provide the cell IDs along with the locations (and I doubt they would want to), so there’s no way to directly match a cell ID / signal strength from the phone to an actual physical location in the ASR database.


There’s *supposedly* more complete databases, such as [towermaps.com](http://www.towermaps.com), but again I don’t think they have the cell tower IDs, and even worse they charge $500 per COUNTY. There’s also [cellreception.com](http://www.cellreception.com) but it apparently just repackages the ASR data. I’m sure there’s others, but nothing I’ve come across is suitable. Let me know if you find something.

### Signal Strength Signatures ###

An alternative to triangulation and trilateration is to simply record a “fingerprint” of the cell tower signal strengths at every location you wish to be able to recognize. Then, when you want to find out where you are, simply take a reading of signal strengths, and compare them to the recorded fingerprints, picking the one that matches most closely.

The advantage of this is that you don’t need to know the exact locations of every cell tower, and it takes into account interference from buildings, land masses, etc. The obvious disadvantage is that you must map out every location where you want coverage. You must “cell-stumble”, the cellular equivalent of war driving.

A project that’s similar to this, but with Series 60 phones, exists at [GSMLoc.org](http://www.gsmloc.org) but they only have very limited data in a few scattered locations, and they appear to have pretty much abandoned the project.

### GPS ###

A built in GPS receiver in the iPhone would be the ideal solution, but of course this is out of the question at least until iPhone “2.0” (wink wink Apple).

Bluetooth GPS support for the iPhone would be awesome as well. It would require the Bluetooth serial port profile, which I would be surprised if the iPhone already has. iPhone uses the [BLUEmagic Bluetooth stack](http://www.oi-us.com/Uploads/File/Bluemagic_Brief.pdf) which is “modular”… it’s unlikely that Apple would pay for unnecessary modules or include them if they did. Maybe someone could port an existing open source Bluetooth stack to the iPhone ([Bluez](http://www.bluez.org/), perhaps? OS X probably isn’t similar enough to Linux for that to be easy). Worst case, we could use a laptop as the missing link (iPhone -> WiFi -> laptop <- serial/USB/bluetooth <- GPS) but that seems like a really poor solution. Maybe iPhone <- serial <- GPS? ### Moving Forward ### The first thing that needs to be done is to figure out how to get the cell tower IDs, signal strength, etc programmatically from the iPhone. Obviously this is possible since FieldTest.app is able to do it. I don't think it's as simple as sending AT commands directly to the baseband, since you can't do that unless CommCenter is disabled, which of course disables the phone features (obviously unacceptable in this case). Also, I have tried disabling CommCenter and sending "AT+CREG=2" then "AT+CREG", which should give signal strength, but wasn't getting anything useful back (perhaps because CommCenter was disabled and the phone wasn't connected to any cells?) Once that's done, we could interface with GSMLoc.org but again, their coverage is extremely sparse, and I don't think their approach is the best (they don't store signal strengths, which would be good for fingerprinting). It would be a good first step though. The mapping of fields in FieldTest.app to the GSMLoc.org database fields is as follows:

FieldTest.app GSMLoc.org Notes
# Net #1
Network Net #2
Location LAC
Cell id CellId Only the first entry appears to be valid, all others are “65535”
Rx Level

The next step would be to do our own “cell stumbling”, but that would require interfacing the iPhone with a real GPS receiver… again, problematic. See above.

Once we’re able to get cell tower IDs and signal strength correlated to lat/lon coordinates we can either add to the GSMLoc.org database or start our own. GSMLoc’s database of coordinate/cell id pairs seems too simplistic for accurate fingerprinting or trilateration.

Joel De Gan wrote about [matching this data up with the FCC records](http://blog.peoplesdns.com/archives/34), but I’m not convinced that’s necessary. I think the fingerprinting approach would be better. Simply store the raw readings of cell tower ID and signal strength mapped to locations, and use a little AI to intelligently infer your unknown location from known cell IDs and signal strengths. This could take into account interference from buildings, mountains, etc, rather than trying to map out every single cell tower then perform the trilateration. Actual trilateration using signal strength would assume every tower has identical antennas and transmit power, and is uniform 360 degrees around.

Finally, we need massive armies of people willing to cell stumble across the country (or at least every major metropolitan area).

Once that’s all done, we can pretty much take over the world (or at least make Google Maps way cooler and tag photos with approximate lat/lon coordinates…)

### Another Option: WiFi ###

One more possibility would be to use the known locations of WiFi access points for geolocation. There are already free and open repositories of millions of WiFi access points: [wigle.net](http://www.wigle.net/) seems to be one of the largest.

The problem is that WiFi signals are much shorter range than cellular signals, thus the coverage of a single WiFi access point is much smaller than a single cell tower. Of course there are many more WiFi access points than cell towers (11 million registered on wigle.net), but this just means you need to wardrive along much finer resolution “grid” (essentially at the individual street level) to get good coverage.

### Conclusion ###

There are plenty of obstacles to making a useful location service for cell phones, but I think it would be fun to try anyway. Even if we can’t cover the entire U.S., as long as there’s on person per major city willing to cell stumble for a weekend, I think you could get a pretty useful system.

If you’re interested in helping, email me at .

Update: it appears a company called Navizon has released [a product exactly like this](http://navizon.typepad.com/my_weblog/2007/09/a-version-of-na.html). Unfortunately it costs money, but supposedly the money goes to the users who upload new cell tower locations and WiFi access points. $20 for 1000 unique cell towers or 5000 unique WiFi access points doesn’t seem like a very good deal though.

Update 2: With the release of the iPhone 1.1.1 software updates all 3rd party applications are essentially locked out (for now). Navizon is offering a refund for customers who purchased their product.

So it’s on Apple to do this. If they partner with AT&T this would be trivial to implement, since of course AT&T knows where all their own cell towers are located.

Update 3: This exactly functionality is now built into iPhone software version 1.1.3. Apple is using both cell tower data from Google, and WiFi access point data for approximating your current location.

iPhone Accelerometer Fun-ness

If you’re an Apple geek like myself, you might remember back in 1999 when WiFi was a brand new technology, and [Phil Schiller](http://en.wikipedia.org/wiki/Philip_W._Schiller) demonstrated the new iBook’s Airport feature by jumping off a balcony holding an iBook with an attached accelerometer, which sent the accelerometer data wirelessly to Steve’s demo machine for display (see below).

Well, when I found out someone figured out [how to access the raw accelerometer data](http://blog.medallia.com/2007/08/fun_with_the_iphone_accelerome.html) from the iPhone I decided it would be cool to do something with it, and I used Phil’s demo as inspiration.

The demo consists of two simple applications. One application runs on the iPhone collecting accelerometer data and sending it over the wireless network to the other application running on a computer to display the data:

The three lines correspond to the three axes of the iPhone’s accelerometer plotted over time (if you’re looking at the iPhone from straight ahead then x is red, y is green, z is blue). Source code for the applications will be up eventually, but it’s really pretty straightforward.

Of course this is just a simple little demo, but I’ve got some plans for other cool things…

As an added bonus, I found the video of Phil’s demo:

iPhone headphone jack

As much as I love the iPhone, it certainly has a few quirks. My number one complaint is the ridiculous recessed headphone jack that won’t fit nearly any headphone plug, other than skinny iPhone and iPod headphones. And it’s not like this was some naïve accident that Apple didn’t know about… since the first day the iPhone went on sale Apple has stocked lame, overpriced adapters.

I didn’t want to spend $10, nor did I want to carry around a stupid little adapter, so I went with the free and obvious solution, trim the excess plastic off each of the audio plugs I wanted to use with my iPhone:


(left to right: Shure E3C in ear headphones, Monster FT transmitter, standard iPhone headphones)

Use a sharp knife like an Xacto to trim away just as much plastic as necessary to make the plug fit. Be careful you don’t cut too deep and slice the actual wires.

Security concerns in "web installer" and XBMC's web server

There have been some questions as to the security implications of the “web installer” available for installing the [XBMC iPhone Remote](http://tlrobinson.net/projects/xbmciphone/) and other XBMC applications. This page explains how it works and the problems with XBMC’s web server [HTTP-API](http://www.xboxmediacenter.com/wiki/index.php?title=web serverHTTP-API) which make this installer possible, as well as the solution to securing your XBMC web server.

Please note that simply installing XBMC iPhone Remote using EITHER the manual installation or web installation does NOT make your Xbox any less secure. It’s just HTML, CSS, and JavaScript, which are all harmless. There’s no server side ASP or Python code at all, other than in the web installer. Enabling XBMC’s web server in the first place (without password protection) is the problem.

Luckily there is an easy (although flawed) fix. See the “Solutions” section for instructions on how to enable password protection on the XBMC web server.

### Background ###

Any Xbox Media Center installation that has the web server enabled also has an HTTP interface that can be used to control various things on the Xbox, including pausing/playing, listing of files, etc, which are used extensively in the XBMC iPhone Remote and make it possible. It also allows you to control such things as downloading files, executing scripts, deleting files, etc, which are used in the “web installer”. Anyone who thinks about this for a moment will realize that this is rather unsafe, and could easily be used by malicious hackers to delete files, steal passwords, or gain complete control of your Xbox.

Normally this would not be a big problem because most users have their Xbox and computers on a LAN behind a home router (like Linksys, DLink, etc) so this interface isn’t available to the public Internet unless they specifically forward port 80 (HTTP) to their Xbox.

BUT, since for the web interface to be useful the user’s computer, iPhone, etc must have access to the Xbox’s web server. So the problem (or solution, in the case of the web installer) is that the user’s browser has unrestricted access to the web interface. When the user visits a webpage, that webpage could easily execute any command on the Xbox simply by loading an appropriate URL in a browser window or iframe.

Due to the [same origin policy](http://en.wikipedia.org/wiki/Same_origin_policy) in place on web browsers, the page that executes the command cannot access the results of the command, but that doesn’t matter since we can use the API to download and execute arbitrary PYthon scripts.

### How the web installer works ###

In fact this is exactly how the web installer works. It has to bootstrap itself by downloading a simple install script, then executes that script to do the actual installation. Credit goes to LiquidIce for the original web installer, I just modified it to be a little more automated and to suit the XBMC iPhone Remote.

The user enters their Xbox’s IP address into a text field on the project page, then presses the “Install” button:

– http://tlrobinson.net/projects/xbmciphone/index.php

The “Install” button executes a little JavaScript that creates a special URL with the user’s specified IP address, the “FileUpload” command, and the file to be uploaded which is a Python script that has been encoded into [Base64](http://en.wikipedia.org/wiki/Base64):

– http://tlrobinson.net/projects/xbmciphone/webinstaller.js (the JavaScript)
– http://tlrobinson.net/projects/xbmciphone/webinstaller-iphone.spy (the Python script before Base64 encoding)

The URL looks something like this (with replaced with the actual Xbox IP, and ABCDEFG replaced with the actual long string of Base64 encoded Python, webinstaller-iphone.spy):;ABCDEFG)

The following commands can be used on Mac OS X (and others?) to encode and decode Base64, respectively:
– openssl enc -base64 -in webinstaller-iphone.spy > webinstaller-iphone-base64.txt
– openssl enc -base64 -d -in webinstaller-iphone-base64.txt > webinstaller-iphone.spy

The URL is then loaded into the iframe, which uploads the script to Q:/web/webinstaller-iphone.spy on the Xbox. After a short delay to ensure it had time to upload, the JavaScript then executes the just uploaded Python file by loading a URL similar to the one below in same iframe:


That executes the Python code that downloads the actual XBMC iPhone Remote files to the correct directory, cleans up, and finally redirects the page to to display it.

– http://tlrobinson.net/projects/xbmciphone/iphone.rar (the rar file containing XBMC iPhone Remote)

So there it is. It’s nothing too sneaky. Feel free to look at all the files for yourself, decode the Base64 Python code, etc.

### The Problem ###

While this web installer is harmless, it would be trivial to write something more destructive to delete files, steal stored passwords for SMB, FTP, etc.

For the web installer, we simply ask the user for their Xbox’s IP address since we assume the user trusts us, but it wouldn’t be hard to brute force try all the common home LAN IP addresses (commonly 192.168.x.x). Also, while the web installer requires users interaction to initiate the installation, there’s no reason a malicious script couldn’t execute automatically.

Fortunately, XBMC provides password protection for the web server. Unfortunately, it’s not enable by default, and most people don’t bother to enable it.

### The (Partial) Solution ###

The solution is simple: enable password protection on the XBMC web server. In the Network settings menu, under Servers, there’s an option for a password.

There is one major problem with this: once you log into web server, the browser remembers that you have logged in for the remainder of the time it is open. This is convenient, as it would be very annoying to have to enter your username and password every time you issue a command to the, but could potentially be taken advantage of if you happen to log in prior to visiting a malicious page.

To test this (with the harmless web install example), enable password protection, then log into your Xbox’s web interface by visiting http://xbox/ where “xbox” is replaced with the actual Xbox IP address. Then use the [web installer](http://tlrobinson.net/projects/xbmc/) to perform the installation. It won’t ask for your password.

Additionally, you can change the web server port or use an obscure IP address for your Xbox. Although this only amounts to security through obscurity, it would likely be enough, bar some really determined hacker with something like [Jitko](http://ha.ckers.org/blog/20070402/jikto-leaked/).

### Conclusion ###

While it’s pretty unlikely that anyone cares enough about your Xbox to hack it, it’s a definite possibility, and would be nearly trivial with all the dangerous (yet useful) commands the HTTP-API gives us. In addition to being a neat and easy way to install a cool app on your Xbox, the web installer serves as a harmless proof of concept.

Thanks to LiquidIce for providing the original web installer code and idea.

iPhone headphones survive the ultimate test: washer and dryer

I was folding my laundry tonight, and found these at the bottom of the pile:


Yup, my nearly brand new iPhone headphones rolled up into a neat little ball. They got run through the washing machine and dryer… oops.

After several minutes of detangling, I was left with a thoroughly kinked and curled set of iPhone headphones:


I have a terrible record of destroying iPod headphones, so I thought for sure they were ruined. Nope! They work fine. Even the microphone and little clicky button thing. Amazing.

The iPhone, on the otherhand, doesn’t leave my sight. I’ll just let someone try it with theirs first…

The iPhone and Apple's 3rd party developer solution… web apps

Not many developers are happy with Apple’s solution for developing applications for the iPhone: web applications. Personally, I’m ok with it because I’ve seen what can be done with JavaScript.

Except for a few things:

– Lack of access to the hardware.
– Inability to run applications offline.
– Limited integration with the rest of the user interface.

There’s not much Apple can do to give developers access to the hardware short of providing a “real” environment to develop in, i.e. Cocoa. It would be great to be able to write applications for hardware like GPS receivers, or to write network utilities that require sockets, such as SSH clients.

The ability to run applications offline isn’t as important if we always have access to the EDGE network. It would be nice for on airplanes, etc though.

But to satisfy us until Apple gives us an Objective-C API (you ARE working on that…right Apple?), there’s a few things they could to do fix the third issue.

JavaScript is a very capable programming language, there’s not much you can’t do given the right APIs. The iPhone needs the following things to make these web apps seems more like native iPhone apps:

– Shortcut icons on the home screen. Give us one click access to our apps.
– Native looking UI widgets. At least give developers graphics for the standard buttons, menus, text boxes, lists, etc.
– Full screen. No reason to have a URL bar and browser controls for our own applications.
– More JavaScript events. We have click events… and that’s about it. Give us cool multitouch events like pinch and double tap. Let us redefine them to suit our own applications.

Given these things, iPhone web apps would be acceptable, at least for now.

Warped Tour – Ventura (and my first iPhone camera photos)

I drove out to Ventura on Saturday to see my friend Jon who is on the Warped Tour doing some promotion for Capitol Records and [absolutepunk.net](http://www.absolutepunk.net). The drive there was ridiculous… about 3 hours from Los Angeles to Ventura (only about 60 miles), as was parking (I ended up parking a mile away and walking) but I got there just in time to see one of my favorite bands from my high school days, [The Matches](http://www.thematches.com/). Last time I saw them was two years ago on the same stage at Warped Tour, which was cool. I also saw New Found Glory and Bad Religion, which were both great.

I also got a chance to test out (and show off) my iPhone. It was great for getting directions, figuring out where the hell I was, checking out traffic (which as I mentioned, was awful), and taking photos…

Here are a few photos from the day:









As you can see, the iPhone camera is quite good for a cell phone. The resolution isn’t high (2 megapixels), but it’s perfectly fine for photos destined for the web.

At first I took mostly portrait oriented photos, even when landscape would have been better. Once I realized I could take landscape photos I switched, although it was a little awkward holding the camera in landscape orientation without covering the lens.

I assumed iTunes would somehow transfer photos I took on the iPhone back to my computer, but it does not. To do that, you must use an application like iPhoto, Graphic Converter, or the free Image Capture application included with Mac OS X.