symbols.rst 15 KB

Raw Blame History Permalink


Symbols on macOS


Shell Script Property List Format
Embedding UUID property lists inside the dSYM bundles
File Mapped UUID Directories
dSYM Locating Shell Script Tips


On macOS, debug symbols are often in stand alone bundles called dSYM files.
These are bundles that contain DWARF debug information and other resources
related to builds and debug info.
The DebugSymbols.framework framework helps locate dSYM files when given a UUID.
It can locate the symbols using a variety of methods:

Spotlight
Explicit search paths
Implicit search paths
File mapped UUID paths
Running one or more shell scripts

DebugSymbols.framework also has global defaults that can be modified to allow
all of the debug tools (lldb, gdb, sample, CoreSymbolication.framework) to
easily find important debug symbols. The domain for the DebugSymbols.framework
defaults is com.apple.DebugSymbols, and the defaults can be read, written
or modified using the defaults shell command:
% defaults read com.apple.DebugSymbols
% defaults write com.apple.DebugSymbols KEY ...
% defaults delete com.apple.DebugSymbols KEY

The following is a list of the defaults key value setting pairs that can
be used to enhance symbol location:
DBGFileMappedPaths
This default can be specified as a single string, or an array of
strings. Each string represents a directory that contains file mapped
UUID values that point to dSYM files. See the "File Mapped UUID
Directories" section below for more details. Whenever
DebugSymbols.framework is asked to lookup a dSYM file, it will first
look in any file mapped UUID directories for a quick match.
% defaults write com.apple.DebugSymbols DBGFileMappedPaths -string /path/to/uuidmap1
% defaults write com.apple.DebugSymbols DBGFileMappedPaths -array /path/to/uuidmap1
    /path/to/uuidmap2

DBGShellCommands
This default can be specified as a single string, or an array of
strings. Specifies a shell script that will get run in order to find the
dSYM. The shell script will be run given a single UUID value as the
shell command arguments and the shell command is expected to return a
property list. See the property list format defined below.
% defaults write com.apple.DebugSymbols DBGShellCommands -string /path/to/script1
% defaults write com.apple.DebugSymbols DBGShellCommands -array /path/to/script1
    /path/to/script2

DBGSpotlightPaths
Specifies the directories to limit spotlight searches to as a string or
array of strings. When any other defaults are supplied to
com.apple.DebugSymbols, spotlight searches will be disabled unless
this default is set to an empty array:
# Specify an empty array to keep Spotlight searches enabled in all locations
% defaults write com.apple.DebugSymbols DBGSpotlightPaths -array

# Specify an array of paths to limit spotlight searches to certain directories
% defaults write com.apple.DebugSymbols DBGSpotlightPaths -array /path/dir1 /path/dir2


Shell Script Property List Format
Shell scripts that are specified with the DBGShellCommands defaults key
will be run in the order in which they are specified until a match is found.
The shell script will be invoked with a single UUID string value like
"23516BE4-29BE-350C-91C9-F36E7999F0F1". The shell script must respond with a
property list being written to STDOUT. The property list returned must contain
UUID string values as the root key values, with a dictionary for each UUID. The
dictionaries can contain one or more of the following keys:


Key
Description


DBGArchitecture
A textual architecture or target
triple like "x86_64", "i386", or
"x86_64-apple-macosx".


DBGBuildSourcePath
A path prefix that was used when
building the dSYM file. The debug
information will contain paths
with this prefix.


DBGSourcePath
A path prefix for where the
sources exist after the build has
completed. Often when building
projects, build machines will
host the sources in a temporary
directory while building, then
move the sources to another
location for archiving. If the
paths in the debug info don't
match where the sources are
currently hosted, then specifying
this path along with the
DBGBuildSourcePath will help
the developer tools always show
you sources when debugging or
symbolicating.


DBGDSYMPath
A path to the dSYM mach-o file
inside the dSYM bundle.


DBGSymbolRichExecutable
A path to the symbol rich
executable. Binaries are often
stripped after being built and
packaged into a release. If your
build systems saves an unstripped
executable a path to this
executable can be provided.


DBGError
If a binary can not be located
for the supplied UUID, a user
readable error can be returned.


Below is a sample shell script output for a binary that contains two
architectures:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>23516BE4-29BE-350C-91C9-F36E7999F0F1</key>
    <dict>
        <key>DBGArchitecture</key>
        <string>i386</string>
        <key>DBGBuildSourcePath</key>
        <string>/path/to/build/sources</string>
        <key>DBGSourcePath</key>
        <string>/path/to/actual/sources</string>
        <key>DBGDSYMPath</key>
        <string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string>
        <key>DBGSymbolRichExecutable</key>
        <string>/path/to/unstripped/executable</string>
    </dict>
    <key>A40597AA-5529-3337-8C09-D8A014EB1578</key>
    <dict>
        <key>DBGArchitecture</key>
        <string>x86_64</string>
        <key>DBGBuildSourcePath</key>
        <string>/path/to/build/sources</string>
        <key>DBGSourcePath</key>
        <string>/path/to/actual/sources</string>
        <key>DBGDSYMPath</key>
        <string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string>
        <key>DBGSymbolRichExecutable</key>
        <string>/path/to/unstripped/executable</string>
    </dict>
</dict>
</plist>

There is no timeout imposed on a shell script when is it asked to locate a dSYM
file, so be careful to not make a shell script that has high latency or takes a
long time to download unless this is really what you want. This can slow down
debug sessions in LLDB and GDB, symbolication with CoreSymbolication or Report
Crash, with no visible feedback to the user. You can quickly return a plist
with a single DBGError key that indicates a timeout has been reached. You
might also want to exec new processes to do the downloads so that if you return
an error that indicates a timeout, your download can still proceed after your
shell script has exited so subsequent debug sessions can use the cached files.
It is also important to track when a current download is in progress in case
you get multiple requests for the same UUID so that you don't end up
downloading the same file simultaneously. Also you will want to verify the
download was successful and then and only then place the file into the cache
for tools that will cache files locally.

Embedding UUID property lists inside the dSYM bundles
Since dSYM files are bundles, you can also place UUID info plists files inside
your dSYM bundles in the Contents/Resources directory. One of the main
reasons to create the UUID plists inside the dSYM bundles is that it will help
LLDB and other developer tools show you source. LLDB currently knows how to
check for these plist files so it can automatically remap the source location
information in the debug info.
If we take the two UUID values from the returns plist above, we can split them
out and save then in the dSYM bundle:
% ls /path/to/foo.dSYM/Contents/Resources
23516BE4-29BE-350C-91C9-F36E7999F0F1.plist
A40597AA-5529-3337-8C09-D8A014EB1578.plist

% cat /path/to/foo.dSYM/Contents/Resources/23516BE4-29BE-350C-91C9-F36E7999F0F1.plist
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
   <key>DBGArchitecture</key>
   <string>i386</string>
   <key>DBGBuildSourcePath</key>
   <string>/path/to/build/sources</string>
   <key>DBGSourcePath</key>
   <string>/path/to/actual/sources</string>
   <key>DBGDSYMPath</key>
   <string>/path/to/foo.dSYM/Contents/Resources/DWARF/foo</string>
   <key>DBGSymbolRichExecutable</key>
   <string>/path/to/unstripped/executable</string>
   <key>DBGVersion</key>
   <string>3</string>
   <key>DBGSourcePathRemapping</key>
   <dict>
       <key>/path/to/build/time/src/location1</key>
       <string>/path/to/debug/time/src/location</string>
       <key>/path/to/build/time/src/location2</key>
       <string>/path/to/debug/time/src/location</string>
   </dict>
   <key>DBGSymbolRichExecutable</key>
   <string>/path/to/unstripped/executable</string>
</dict>
</plist>

Note that the output is very close to what is needed by shell script output, so
making the results of your shell script will be very easy to create by
combining two plists into a single one where you take the UUID and use it a
string key, and the value is the contents of the plist.
LLDB will read the following entries from the per-UUID plist file in the dSYM
bundle: DBGSymbolRichExecutable, DBGBuildSourcePath and
DBGSourcePath, and DBGSourcePathRemapping if DBGVersion is 3 or
higher. DBGBuildSourcePath and DBGSourcePath are for remapping a single
file path. For instance, the files may be in /BuildDir/SheetApp/SheetApp-37
when built, but they are in /SourceDir/SheetApp/SheetApp-37 at debug time,
those two paths could be listed in those keys. If there are multiple source
path remappings, the DBGSourcePathRemapping dictionary can be used, where
an arbitrary number of entries may be present. DBGVersion should be 3 or
DBGSourcePathRemapping will not be read. If both DBGSourcePathRemapping
AND DBGBuildSourcePath/DBGSourcePath are present in the plist, the
DBGSourcePathRemapping entries will be used for path remapping first. This
may allow for more specific remappings in the DBGSourcePathRemapping
dictionary and a less specific remapping in the
DBGBuildSourcePath/DBGSourcePath pair as a last resort.

File Mapped UUID Directories
File Mapped directories can be used for efficient dSYM file lookups for local
or remote dSYM files. The UUID is broken up by splitting the first 20 hex
digits into 4 character chunks, and a directory is created for each chunk, and
each subsequent directory is created inside the previous one. A symlink is then
created whose name is the last 12 hex digits in the deepest directory. The
symlinks value is a full path to the mach-o files inside the dSYM bundle which
contains the DWARF. Whenever DebugSymbols.framework is asked to lookup a dSYM
file, it will first look in any file mapped UUID directories for a quick match
if the defaults are appropriately set.
For example, if we take the sample UUID plist inforamtion from above, we can
create a File Mapped UUID directory cache in
~/Library/SymbolCache/dsyms/uuids. We can easily see how things are laid
out:
% find ~/Library/SymbolCache/dsyms/uuids -type l
~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1
~/Library/SymbolCache/dsyms/uuids/A405/97AA/5529/3337/8C09/D8A014EB1578

The last entries in these file mapped directories are symlinks to the actual
dsym mach file in the dsym bundle:
% ls -lAF ~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1
~/Library/SymbolCache/dsyms/uuids/2351/6BE4/29BE/350C/91C9/F36E7999F0F1@ -> ../../../../../../dsyms/foo.dSYM/Contents/Resources/DWARF/foo

Then you can also tell DebugSymbols to check this UUID file map cache using:
% defaults write com.apple.DebugSymbols DBGFileMappedPaths ~/Library/SymbolCache/dsyms/uuids


dSYM Locating Shell Script Tips
One possible implementation of a dSYM finding shell script is to have the
script download and cache files locally in a known location. Then create a UUID
map for each UUID value that was found in a local UUID File Map cache so the
next query for the dSYM file will be able to use the cached version. So the
shell script is used to initially download and cache the file, and subsequent
accesses will use the cache and avoid calling the shell script.
Then the defaults for DebugSymbols.framework will entail enabling your shell
script, enabling the file mapped path setting so that already downloaded dSYMS
fill quickly be found without needing to run the shell script every time, and
also leaving spotlight enabled so that other normal dSYM files are still found:
% defaults write com.apple.DebugSymbols DBGShellCommands /path/to/shellscript
% defaults write com.apple.DebugSymbols DBGFileMappedPaths ~/Library/SymbolCache/dsyms/uuids
% defaults write com.apple.DebugSymbols DBGSpotlightPaths -array

Hopefully this helps explain how DebugSymbols.framework can help any company
implement a smart symbol finding and caching with minimal overhead.