85: Adjusting photo timestamps with Exiftool

The nicest camera in my house is a Canon T3i. It's about five years old and for my needs it still takes fantastic pictures. One thing it doesn't do is keep accurate time. That's not its purpose; it's precision optics, not a precision chronograph.

But photo timestamps matter, and I always forget to check my camera's clock before going on a big trip. A week later, I have 1500 photos from the T3i that don't line up with the 100 or so photos I took on my phone (largely panoramas, something the iPhone does really well and SLRs don't handle). If the camera kept good time, this wouldn't happen, or at least the solution would be simple.

Most photo organization apps, like Lightroom, which I use, have the ability to batch correct photo timestamps — if you forgot to adjust your camera's time zone. I was in California, so I had that problem, but correcting three hours westward wouldn't solve it; my timestamps were some 3 hours and 18 minutes off from my iPhone photos.

I suspected my solution would lie on the command line — spoiler alert: it was — but I searched for a GUI tool designed for this task. An $8 mini version of A Better Finder Attributes claims to be a timestamp unitasker, but I couldn't sort out the ABFAX demo's interface. HoudaGeo, primarily designed for geotagging, will tackle the problem, but at $40 it's priced for users who will take advantage of all of its features.

So I turned to Exiftool, an omnibus photo metadata munger for the command line. It was quick to install with Homebrew, and I found the one-line command I needed in its documentation. (I had to run it twice on the first batch of photos, because I adjusted 18 minutes in the wrong direction!) Adding a simple Bash for loop blasted through 1200 more photos in no time.

for d in 2016-08-*; do exiftool "-DateTimeOriginal-=0:0:0 0:17:32" $d; done

Another loop removed the archived originals that Exiftool thoughtfully creates…although that can amount to gigabytes of extra storage when processing lots of photos.

Even after that, my photos weren't sure what time zone they were in. Lightroom displayed the unaltered iPhone photos in what would be Eastern Standard Time, even though we're currently observing Daylight Saving Time. I gave up on aligning them to a "true time" as long as SLR and iPhone would happily interlace. And they did, except for the iPhone videos (7 hours off) and some iPhone shots identified as coming from an "Unknown Camera" (5 hours off). But with a few tweaks, all is in order. Now, thanks to a little command line magic and some persistence, I can relive my trip in order.

84: How not to use in-flight Wi-Fi

You probably already know the first rule of using in-flight Wi-Fi: buy the pass before you get on the plane. The second rule is to make sure that you have everything you need to cash in the pre-purchased pass. That's the rule I forgot.

A week ago, I was traveling to San Francisco and had some podcast work that needed to get done along the way. Four hours in the air would be enough to do my work, and also enough to justify a $16 Gogo plan. I'd never crossed that value threshold before, so I didn't have a Gogo account. No problem, right? As I was packing, I grabbed the closest computer (my iMac), created an account, and made my purchase, all with the help of 1Password. I got on the flight, opened up my MacBook, and realized that my Gogo credentials were…nowhere.

This was a combination of user error on my part and some recent changes to how 1Password does syncing. A few months ago, I realized that when syncing 1Password vaults with Dropbox, a lot of information is stored in the clear — no passwords, of course, but the titles of all your accounts. It's a small chance that my vault would be compromised this way, but confirmation that I have an account on a certain site could make it a greater target. As a result, I switched to iCloud syncing, which makes all that data opaque. iCloud syncing Just Works™, so there's no control over exactly when it syncs. If I'd still been using Dropbox, my vault would've synced over as soon as I logged onto the ground-based airport Wi-Fi prior to boarding. iCloud didn't think the sync was urgent — the 1Password app wasn't open, after all — so it just didn't perform it.

All that aside, I would've been fine if I could have reset my Gogo account password. There's an option for that, even on the plane, but it requires answering a security question. And what is my mother's maiden name? It's…a randomly generated string saved in 1Password. (I used to use all lowercase strings with no spaces; a recent 1Password update seems to have removed that option in the password generator, favoring diceware instead. That was even more frustrating, because I could remember two of the words but not the third!) That left me completely sunk. I could buy another pass, at the increased in-flight rate of $27, or disconnect for four hours. I opted for the latter, and still managed to squeeze in my work. At least the first rule of plane Wi-Fi did some good: those pre-paid passes are valid for a year, so I got to use it on the way home — after I triple-checked that 1Password had synced.

83: A drought in the App Store

Since I last mentioned weather apps when I did some spring cleaning on my iPhone, I've settled on a single solution that works best for me: Storm by Weather Underground. (It replaced their other, slightly less geeky "Wunderground" app.) But the fact of the matter is that, unlike a few years ago, I haven't tried many new weather apps recently. Patrick Dean summed this up on twitter, pointing out that just about any competently made app reaches the threshold to be featured, as weather is still a top-level category in the App Store.

I think there's one clear reason for this: homogenization of weather data. In the early days of the App Store, weather apps could differentiate themselves based on the source and accuracy of their data; attractive presentation came second. That all changed three years ago with the introduction of forecast.io. If you don't know the history, forecast.io is the open API by the creators of Dark Sky.

When Dark Sky came onto the scene in 2011, it was head and shoulders above other weather apps, with groundbreaking features like predictive radar maps and push notifications for impending precipitation. But even in version 1.0, it was clear to me that some things were amiss. It was less accurate on the iPad than on the iPhone because it made its weather predictions client-side, using the GPU. In fact, Dark Sky is a classic example of "machine learning applied to a problem we (and the machine) know nothing about". Its algorithm doesn't analyze weather data, it analyzes images, specifically radar map frames. This is what gives its future maps an unearthly, unrealistic quality; storms that have been developing or diminishing suddenly turn into zombie clouds drifting in straight lines across the map. In fact, if you click the play button on the map in the latest version of Dark Sky, it stops at the present time, as if it's ashamed of how bad those future maps look.

View fullsize

In serious contrast, Storm offers a 5-hour future radar map underpinned by meteorological data. Storms can appear out of nowhere along front lines, or die down before reaching your location. It's this one feature, plus its excellent daily and hourly graph views, that make it my top weather app. Almost everything else in the App Store is just the same forecast.io data — a computer's idea of weather — dressed up in new skins. The greatest innovation has been to make the interface super-snarky; an advance in entertainment, but not in utility. There may never be another great iOS weather app, because the hard part is gathering the data. Weather Underground operated stations for years, and is now (regrettably) under the weather.com umbrella. There is free weather data out there, but you get what you pay for: a slick interface for a couple of bucks.

82: Trapped Flickr geotag data

I joined Flickr in 2005, when it was young enough that, like many fledgling internet services, you could refresh the global timeline and keep tabs on everyone's activity. In the past 11 years, I've uploaded over 9000 photos to Flickr and geotagged 7500 of them. And now I have a problem.

Flickr was the first piece of software, native or on the web, that let me add location data to my photos. It was all done by hand, as this was before my first iPhone or any other device that added geolocation automatically. I painstakingly dragged and dropped photos onto the map, often — because of Yahoo's crummy map data — with Google Earth side by side, especially for places that a street map considers the middle of nowhere, like archaeological sites in Sicily. Later, apps like iPhoto added geotagging capabilities (and removed them, and added them back), but I kept on with Flickr because it was my central photo hub.

Some of my fine-grained Flickr geotags in the Forum Romanum.

Flickr is still alive and kicking despite several apparent death knells in the past. But this week's news that Verizon is buying Yahoo gave me another push to liberate my geodata and keep it stored in my local Lightroom photo library. There's only one problem: I don't know how to do it. Flickr allows bulk downloads of photos, but it doesn't alter the original EXIF data, so that means the geotags aren't baked in. And the native map view on the Flickr site has barely changed since its introduction almost 10 years ago, including limiting you to viewing less than 100 geotagged photos at a time.

Perhaps I'm missing something, but there is no direct or even indirect conduit for my 7500 pairs of latitudes and longitudes to get matched to photos in the Finder or Lightroom. The closest answer I've gotten is "use the API to write something yourself", but that's far from a simple or complete solution. But unless an alternative presents itself, that's likely the route I'll take. Hopefully it won't be a race against the clock, facing an API shutdown date. (None has been announced, but it's hard to be optimistic about Flickr's bright future.)

When I get my geolocation data out, I'll know better than to lock it into a new proprietary system — especially one that I don't control. I was way more lax about my data in 2007 or 2008, when I started geotagging. I didn't even have regular backups, so Flickr actually saved my bacon when I had a catastrophic hard drive failure and would have otherwise lost about 2500 photos for good. I treat my data much more carefully today, or at least I will once I can get my hands on it.