The Best Geotagging Solution Yet

Back in December, I talked about the attempt Jeannette and I were making on an elegant geotagging solution where the key components would come from our employer.

For various reasons, that project is on hold. But I still want to use the capabilities built into my camera to automatically geotag my photos, but I was not happy in Strasbourg and Barcelona carrying around the setup I had. (see below)



I have been yearning for a solution, and even tried out a few things (unsuccessfully). With the upcoming vacation season, the sense of urgency has increased, so I bit the bullet and invested in this:



The two small boxes replace the Nikon MC-35 cable, the 6-foot long GPS serial cable and the old & clunky Garmin GPS reciever.

The larger box is a Holux GPSlim 236 bluetooth GPS receiver and the smaller box is a bluetooth-to-serial converter doodad from Foolography that plugs into the camera's 10-pin accessory port. I don't think the creator has decided on a model number yet...

These two things completely eliminate the cable clutter. The only downside is the loss of the 10-pin connector for my remote release. But I am working on that...

So far this system has worked quite well, especially considering its small size. After extensive testing on our trip to Paris over the long Easter weekend, I can say I am quite happy. But there are some strange points I will bring up:
* The Bluetooth connection is flaky. Once the connection is made, it is pretty solid. I'd say about 66% of the time, the connection between the GPS and the module on the camera is made within a second or three of turning on the camera. About 25% of the time it will take much longer. Maybe 10 -15 seconds. The remaining 10% or so of the time it simply will not connect, and I have to switch off the camera and back on before the connection is made.
* The Bluetooth connection is much more reliable when I keep the GPS close to the camera. I had originally intended to keep the GPS unit in my backpack, but it was unreliable. Currently I have a cell-phone pouch (thank you Jeannette!) attached to my camera strap. Not perfect, but working acceptably well.
* The Holux GPSlim 236 battery life is incredible. I get at least 12 hours on a charge.
* The battery on the D200 is not seriously diminished, if I turn off the camera between bursts of shooting. Nikon made an interesting decision (or maybe it was just a mistake) to keep the exposure meter on when there is incoming GPS data. This means the camera is burning a lot more power than it does in its idle state when nothing is happening. The battery will die in a few hours if I don't turn the camera off. I think it is more the exposure meter remaining on than the Bluetooth-to-serial module.
* The Holux GPSlim 236 normally locks in pretty quickly (less than 1 minute), assuming I haven't moved a great distance since the last time it was on (and locked in). But sometimes, and I haven't figured out what causes this, it will take forever, like it is reading the entire almanac. It is very frustrating when the GPS doesn't lock in quickly, more so than when the Bluetooth connection isn't made.
* The Holux GPSlim 236 is very good (but not perfect) at keeping a lock in a city. It beats the heck out of my old Garmin, which is no surprise considering the Garmin is around 8 years older than the Holux. I think the quality of the position reported by the Holux is better too. Comparing geotagged photos with both GPS units using Google Maps, the Holux geotagged photos seem much closer to the right place than the Garmin. I really like that.

The Next Aperture Script

Recently I talked about the Applescript I worked on to add extra EXIF metadata to the Aperture database. It works great for me, but a couple people that have tried it have had problems with it, mainly because they use some options in Aperture that I don't that causes problems. Most of those problems, if not all, would be solved by doing the metadata extraction at import, when I know the name of the source file and the reference to the file in the Aperture database.

The basic principle was not very hard, in fact, it came together a lot quicker than I ever would have expected. But it took a while to bring it to a state that I would want the rest of the world to see it. I can put up with some things that I am sure others wouldn't, and I had to make it easy for the end user to use it in their own way - which meant taking out all the hardcoded paths and stuff...

Then I had to get all fancy. As I have mentioned in the past, I am a big proponent of geotagging, either with a direct connection between the camera and a GPS or software that syncs GPS track logs with photos via the time stamps. Since I was working on all this metadata (thanks to exiftool) on the front-end of the import process, and I have the GPS EXIF data available, why not query the GeoNames database and write the City, State & Country IPTC tags as well?

I was all ready to write my own parser of the GeoNames, but somehow I stumbled across the Geo::GeoNames perl module on CPAN. That sure made things easy.

The perl part came together pretty fast, but the Applescript part was slower - Applescript is probably the most readable language I have ever seen, but the elegance (if you want to call it that) of it is only in the finally working code. There are seriously some major syntax gymnastics going on when I try to figure out how to put things together. I think the open-endedness of the syntax is what makes it so hard. I think that anything should work, or that what I wrote originally is just the same as what I eventually got to work.

In the end, I cobbled together something that works great for me. Yeah, there are some rough edges I would like to work on, but for now, I need to live with what I have and spend some more time with my wife.

If you are interested in learning more, or using it for your own needs, or you just want to criticize my sloppy code, you can find more info, and a download link, here.

Followup on the Applescript

Here are some of the technical details about the Applescript I mentioned Yesterday.

The first key line of the code I added to Mr. Gross' Applescript, is just a list of the EXIF tags I want to grab. Not exactly anything fancy, but this actually requires a modicum of thought, since, by default, exiftool will output the fleld names in a human readable format with spaces, but in order to keep the command-line happy is to use the form of the tag without spaces. It is usually pretty easy to figure out.

set DesiredEXIFData to "-LensID -Lens -LensType 
     -FocusMode -FocusDistance 
     -AFMode -AFPoint -AFPointsUsed 
     -ShootingMode -MeteringMode 
     -Flash -FlashSetting -FlashType -FlashMode"

I chose tags that I think would be interesting and help me learn as a photographer. Some are redundant - Lens ID, for example, is a superset of "Lens" and "Lens Type". I also added a couple that Aperture will read, such as "Flash". The trouble is, Aperture will only print "0" or "1" for flash, or something else similar. I understand the binary thing, but sometimes the number is "2". Exiftool will report this field as words with meaning: "flash did not fire" or "flash fired, return detected".

The tags need to start with the "-" character. And they shouldn't end with an "=" or you will erase the value in the source file. (By default, exiftool only does potentially destructive things after making a backup).

I decided to separate the fields desired from the actual call to exiftool, because I was distributing this script to the rest of the world and recommending that the user edit the desired fields based on their own needs and equipment. I figured it was harder to screw up the whole shebang this way.

-- Ask exiftool for tab-separated fields
set output to do shell script "exiftool -t " & DesiredEXIFData & " " & (quoted form of m_imagePath)

Pretty self explanatory. The "&" symbol is Applescript's text concatenation symbol. It's how you stick the fixed command stuff to the variable stuff.

Sharp readers will notice that I don't provide the full path to exiftool. That might come back to bite me sometime. The thing is, I don't have any idea if I installed it in the standard location or not. And I have no idea whether the end user will either. It works for now, so I am gonna live with it.

Now that we have the exiftool output, I have to do something with it.

-- Separate output by lines
set output to the paragraphs of output
-- Tell Applescript that tab is the delimiter
set AppleScript's text item delimiters to {ASCII character 9}

The exiftool argument "-t" will output the outputs the field name and the field value separated by a tab.

-- Do the following for each line of output
repeat with fields in output
-- split line into two fields
  set fields to fields's text items

fields's? cringe!

-- the next two lines are probably not needed, but it makes things easy to read
  set MetadataField to item 1 of fields
  set MetadataValue to item 2 of fields

-- this does the actual write to the aperture database
  tell curImg
    make new custom tag with properties {name:MetadataField, value:MetadataValue}
  end tell
end repeat

I think between the comments and Applescript's syntax, it is pretty easy to figure out what is going on.

Speaking of Applescript syntax, I think it is great at how readable it makes the code. But I always find myself struggling to try to figure out how to write it when I am coding something myself. Coming from C or Perl or bash, I find it frustrating to try to figure out how to string together things to make a working program.

Well, that's all I added to Mr. Gross' Applescript. I tested it quite a bit before I unleashed it upon the world. Within a couple hours, someone had come up with a fix for the one known bug that I decided I didn't want to fix because it rarely affected me (and never after I figured out what caused it).

There are at least two people who have tried it out, it worked for them and they like what it does. Feels good to give something to the world.

My First Ever Applescript

A discussion on dpreview.com about Aperture and how it doesn't read maker note EXIF fields, specifically the Lens ID, got me thinking. The wonderful exiftool can read this data, why can't Aperture with its team of crack Apple programmers?

I thought of a workaround. Aperture supports the addition of arbitrary metadata to images, and I use it for things like saving the URL and flickr ID of the photos I upload there. Why not read the data from the image using exiftool and stick the result into a new custom field.

I have diddled around with Applescript in the past, but never really did anything that worked. I got bits & pieces working here and there, but never finished because the need went away, I figured out how to do it some other way (like in Perl), found someone else's solution or some similar result. This task didn't seem too hard, so I poked around and found out that there is Applescript support to write custom metadata.

I then encountered the big hurdle. Talking to Aperture with Applescript is easy and you can do a lot of stuff, but everything is relative to the program's internal database, yet I need the path to the actual image file so that exiftool can do its thing. I didn't see an easy way to get Aperture to reveal a path to the original file or get a reference to the image in Aperture's database based on the path of the image file.

Frustrating! I know both bits of data, I just had no clue how to use them to get to my end result. And I still don't, really, because somebody else figured it out and did the heavy lifting for me: Brett Gross' "Reveal Photo File" Applescript. This handy tool will reveal in the Finder the currently selected image(s) in Aperture. Mr. Gross was kind enough to distribute his Applescript in a readable form, so I was able to yank out the bits that open the finder window and instead squirt the path over to exiftool, get the data I need and then write it back to Aperture.



So now this script is chugging away on my 15k images. It's not the fastest, but most of the slowdown seems to be due to the size of my Aperture library. I tried it on a smaller library of about 200 images and it was done in a minute or two. The current task has been running for a couple days, and at the rate it is going, it will require a couple more. Really strange. And frustrating, because my CPUs are never going above 50% utilization and I have a >1GB of free RAM. I guess the disk is the bottleneck here. I just don't understand why the execution time is so non-linear.

My next step will be to write a new version of the script that will do this all at import, which I hope will be faster.

Please read my post at dpreview.com for the download link and instructions for use.