Symbolication (or how to analyze Apple’s crash logs) under the hood   Leave a comment

Difficulty: easy moderate challenging

If you ever have experienced crashes of your app and would like to fix them, you would like to be able to analyze crashes.

Crashes that happen on the same Mac you are compiling are automatically being intercepted, interpreted, and displayed using GDB.

However, if another person is QAing your app, of if a customer contacts you through the published developer email, or if you download crash logs from iTunes Connect, you would notice that the crash files are readable only for native hex-speaking computers, for instance:

6   Ohad                        0x000cc300 0x1000 + 832256

Luckily, in order to decrypt this crash log, all you need to do is:

  1. Open Xcode.
  2. Open Organizer (Window -> Organizer).
  3. Drag the crash log into iPhone Development -> Crash Logs.
  4. Ta-da! Xcode will symbolicate (technical term) it using the pre-compiled files on your computer and you will end up with a sensible crash file.

But for it to work, you must save the compiled folders that you supplied that person.

So, on a more general scale, you should save all the compiled folders for ALL compiled versions that you supply users.

Now, let’s see how it works under the hood:

If you open a crash file, you would see that there is a section named Binary files at the bottom of it.
This section contains all the code segments (your compiled code and all frameworks) being used in the app.
As an example:

Binary Images:
0x1000 –    0xf3fff +Ohad armv6  <feec92701a8a91f27d9abdd1650ec51f> /var/mobile/Applications/3879CB7F-3E67-4D47-A611-BCAA8DA9606D/
0x1f2000 –   0x1f3fff armv7  <16fad82d8b9ba83f1b97894eec8b9249> /usr/lib/info/
0x2fe00000 – 0x2fe24fff  dyld armv7  <74c5ceb6354fd165a25d02fbd22aa746> /usr/lib/dyld
0x30013000 – 0x30089fff  CFNetwork armv7  <c6ad6e56e0763566e9a9d176f9f8f53c> /System/Library/Frameworks/CFNetwork.framework/CFNetwork

Each binary has a UUID (bolded above), which uniquely identifies it.
The Crash Reported gathers this by running the following command on the executable that crashed:
[~]$ dwarfdump -u /Users/ohad/Downloads/ohad-iphone-1.6.3/build/Debug-iphoneos/
UUID: FEEC9270-1A8A-91F2-7D9A-BDD1650EC51F (armv6) /Users/ohad/Downloads/ohad-iphone-1.6.3/build/Debug-iphoneos/

Now, if you copy-paste this into Apple Spotlight, assuming the dSym folder is somewhere on your computer, you would notice that Spotlight finds the correct dSym folder. How does it do this? Apparently your dSym folder contain some metadata in it. If you run the following command, you should see:
[~]$ mdls  /Users/ohad/Downloads/ohad-iphone-1.6.3/build/Debug-iphoneos/
com_apple_xcode_dsym_paths     = (
com_apple_xcode_dsym_uuids     = (

Spotlight indexes this metadata for your searches. You can also search this through command line using:
[~]$ mdfind ‘com_apple_xcode_dsym_uuids == FEEC9270-1A8A-91F2-7D9A-BDD1650EC51F

When you are using Xcode organizer to symbolicate your crash file, that’s how it picks up the right dSym file, which contains all of your code’s symbolic table needed to correlate stack frames with actual lines of code.

Now, using this dSym file, Xcode uses a script named symbolicatecrash (located at /Developer/Platforms/iPhoneOS.platform/Developer/Library/PrivateFrameworks/DTDeviceKit.framework/Versions/A/Resources/symbolicatecrash on Xcode 3.2.3), which uses atos to replace in your crash log each line such as:
6   Ohad                        0x000cc300 0x1000 + 832256
With this (in bold):
[~]$ atos -o /Users/ohad/Downloads/ohad-iphone-1.6.3/build/Debug-iphoneos/ -arch arm 0x000cc300
-[MyManager establishId:] (in Ohad) (MyManager.m:150)



Posted October 7, 2010 by Ohad Kravchick (myok12) in iOS Programming

Tagged with , , , , , ,

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: