DwellableTrends

I recently had the pleasure of creating the Dwellable iPhone app using our existing iPad app as a starting point. Back at Urbanspoon, the iPhone and iPad apps used entirely different codebases, so this was a new experience for me. To begin with, I checked the magic "Universal" checkbox in Xcode and fired up the iPhone simulator to see how things looked. Shockingly, nearly everything worked straightaway. Once I recovered from that very pleasant surprise, I got to work.

It took just EIGHT DAYS of engineering effort to whip the iPhone app into shape, including design work and assets. I had to carefully tweak the UI, but the fundamental interface design was sound. Everything worked beautifully on my iPhone 4.

I dug my trusty old iPhone 3 out of storage and brought up Dwellable. Uh oh - it was very slow. Incredibly slow. Like, bringing up the home screen took around 20 seconds. I quickly realized the problem - the photos on the home screen were 512x512, and there were fifteen of them. I was forcing my old iPhone to load up 15 JPGs, each of which was larger than the display! Whoops.

These performance issues would undoubtedly affect Android as well - we had to make the app run faster. Dwellable is all about big, beautiful hi-res photos, but people won't use the app if it takes forever to load.

Nathan and I parked in front of the whiteboard and brainstormed ways to optimize our images. Eventually we came up several ideas:

• image pipeline: serve the right image to each device
• image sprites
• use 16 bit images on older devices, to conserve memory

Image Pipeline: Serve the Right Image to Each Device

We re-jiggered the back end to serve up a variety of image sizes. We already had most of the ImageMagick plumbing in place on the back end so this change wasn't very difficult. The URL routing goes something like this:

/house/512x512/1.jpg    # 512x512, for older iPhones
/house/1024x1024/1.jpg  # 1024x1024, for retina iPhones
/house/1.jpg            # full size, for iPads

This way, each device can simply ask for the appropriate size. My old iPhone could request 100x100 photos for the home screen instead of the 512x512 photos needed for the retina iPad. For those of you keeping score, that's 1/25th the data. The new image pipeline would make a huge difference over cellular.

Image Sprites

The iPhone 3 still felt slow, unfortunately, and benchmarks revealed that it was still taking upwards of 12 seconds to render all 15 images. That doesn't include network time.

We decided to try spriting to see if we could speed things up. I spent a bit of time with Google and couldn't find anyone else using this technique.

We whipped up a new Rails action to serve up our sprite and added a handy category to UIImage for cropping an image. It would reduce requests, and surely one big JPG decompress was better than 15 small ones. Surely?

Rendering time for those buttons dropped from 12 seconds to 4. A huge win!

16 bit images

I was still nervous about one thing. There were a lot of full screen images flying around and I was getting the occasional memory warning. Most of those images were tied up in paging scroll views, and it would be difficult to teach everything to unload itself properly. Instead, I decided to reduce the bit depth of the images to save some space. A quick refresher on bits per pixel and bits per channel:

• 32 bit RGBA is 8 bits for red, green, blue and alpha. Every pixel has an alpha value, so you can use it with transparent images.
• 16 bit RGB555 is 5 bits for red, green, and blue. There is NO alpha channel and the first bit is skipped. This image format doesn't have an alpha channel, so it only works for images that are fully opaque. There are fewer available colors compared to 32 bit images, but most people can't tell the difference.

Using 16 bits instead of 32 bits cuts memory usage in half:

512x512 image at RGBA   - 512 * 512 * 32 / 8 = 1mb
512x512 image at RGB555 - 512 * 512 * 16 / 8 = 512kb

I ended up adding the following category to UIImage. Some of this was cribbed together from Stack Overflow, naturally:

@implementation UIImage (Dwellable)

- (UIImage *)crop:(CGRect)rect
{
  return [self copyFromRect:rect toSize:rect.size];
}

- (UIImage *)copyToSize:(CGSize)dstSize
{
  return [self copyFromRect:CGRectMake(0, 0, self.size.width, self.size.height)
               toSize:dstSize];
}

- (UIImage *)copyFromRect:(CGRect)srcRect toSize:(CGSize)dstSize
{
  //
  // apply scale if necessary
  //

  int scale = self.scale;
  if (scale > 1) {
    srcRect = CGRectMake(srcRect.origin.x   * scale, srcRect.origin.y    * scale,
                         srcRect.size.width * scale, srcRect.size.height * scale);
    dstSize = CGSizeMake(dstSize.width * scale, dstSize.height * scale);
  }

  //
  // create the context
  //

  int bpc = 8;
  CGBitmapInfo bmi = kCGImageAlphaPremultipliedFirst;
  CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
  if (gIsLowRam) {
    // 16bit if we don't have an alpha channel (this saves a ton of memory)
    CGImageAlphaInfo alpha = CGImageGetAlphaInfo(self.CGImage);
    if (alpha == kCGImageAlphaNone || alpha == kCGImageAlphaNoneSkipLast ||
        alpha == kCGImageAlphaNoneSkipFirst) {
      bpc = 5;
      bmi = kCGImageAlphaNoneSkipFirst;
    }
  }
  CGContextRef context = CGBitmapContextCreate(NULL, dstSize.width, dstSize.height,
                                               bpc, 0, colorSpace, bmi);
  CGColorSpaceRelease(colorSpace);

  //
  // interpolation quality set to high on everything but old iphones
  //

  CGInterpolationQuality quality = kCGInterpolationHigh;
  if (gIsLowCPU) {
    quality = kCGInterpolationDefault;
  }
  CGContextSetInterpolationQuality(context, quality);

  //
  // choose srcImage
  //

  CGImageRef srcImage;
  if (!CGRectEqualToRect(srcRect, CGRectMake(0, 0, self.size.width * scale,
                                             self.size.height * scale))) {
    srcImage = CGImageCreateWithImageInRect(self.CGImage, srcRect);
  } else {
    srcImage = CGImageRetain(self.CGImage);
  }

  //
  // copy
  //

  CGContextDrawImage(context, CGRectMake(0, 0, dstSize.width, dstSize.height),
                     srcImage);
  CGImageRelease(srcImage);
  CGImageRef dst = CGBitmapContextCreateImage(context);
  CGContextRelease(context);

  UIImage *result = [UIImage imageWithCGImage:dst scale:scale
                             orientation:UIImageOrientationUp];
  CGImageRelease(dst);  
  return result;
}

Note that gIsLowCPU and gIsLowRam are defined earlier (thanks again Stack Overflow!):

//
// gModel and friends
//
// iphone
// 1:   412mhz 128mb iPhone1,1
// 3g:  412mhz 128mb iPhone1,2
// 3gs: 600mhz 256mb iPhone2,1
// 4:   800mhz 512mb iPhone3,1-3
// 4s:  2x1ghz 512mb iPhone4,1
//    
// ipad
// 1:   1ghz 256mb iPad1,1-2
// 2: 2x1ghz 512mb iPad2,1-2
// 3: 2x1ghz 1gb   iPad3
//    
// ipod
// 1: 412mhz 128mb iPod1,1
// 2: 533mhz 128mb iPod2,1
// 3: 600mhz 256mb iPod3,1
// 4: 800mhz 256mb iPod4,1
//

struct utsname u;
uname(&u);
gModel = [NSString stringWithCString:u.machine encoding:NSASCIIStringEncoding];

// < 600mhz
gIsLowCPU = [gModel matches:[NSRegularExpression re:@"^(iPhone1|iPod[12])"]];
// old iphones/ipod/ipads
gIsLowRam = [gModel matches:[NSRegularExpression re:@"^(iPhone1|iPod[12]|iPad1)"]];


//
// For anybody trying to copy 'n paste:
//

@implementation NSRegularExpression (Dwellable)

+ (NSRegularExpression *)re:(NSString *)s
{
    return[NSRegularExpression regularExpressionWithPattern:s options:0 error:nil];
}

- (BOOL)matches:(NSString *)s
{
    return [self numberOfMatchesInString:s options:0 range:NSMakeRange(0, s.length)];
}

@end

@implementation NSString (Dwellable)

- (BOOL)matches:(NSRegularExpression *)re
{
    return [re matches:self];
}

@end

Conclusion: Awesomeness

Ultimately, we were able to reduce load times on our old iPhone 3 from 20 seconds to around 3. Other devices were noticably snappier too, especially since the new image pipeline dramatically reduces the amount of data to be fetched. It added an extra day or two to the project, but it was well worth the effort. Especially as we look forward to Android (hint hint). Happy hacking!