The Universal Loader for Go

As promised in the previous post, Go Assembly on the arm64, I have been working on a very special project for the past couple months, and I’m very pleased to announce the Universal Loader!

This Golang library provides a consistent interface across all platforms for loading shared libraries from memory and without using CGO. When I say “all platforms”, I mean Linux, Windows, and OSX including the new M1 Apple chip.

Until someone tells me differently, I am claiming that this is the very first loader to work on the M1, Golang or not. I haven’t tried it myself yet, but it will likely also work on any POSIX system, like BSD variants, without changes. If you try this on a new platform, let me know!

Additionally, the Linux backend of the loader does not use memfd, and I believe this is the first Golang loader to do that as well. The Linux ramdisk implementation memfd is very easy to use, but it’s also relatively easy to detect.

Consistent Interface

On all platforms, this is a basic example of how to use Universal to load a shared library from memory and call an exported symbol inside it:

Just pass in your library as a byte array, call LoadLibrary on it and give it a name, then you can Call() any exported symbol in that library with whatever arguments you want.

All you have to do differently on a different platform is load the right type of library for your platform. On OSX, you would load myLibrary.dyld, on Linux myLibrary.so, and on Windows myLibrary.DLL.

Check out the examples/ folder in the repo and the tests for more details.

Algorithms and References

The Linux and Windows implementations are very straight-forward and based on the same basic algorithms that have been widely used for years, and I used malisal’s excellent repo as a reference, with some minor changes.

For the OSX loader, I referred heavily to MalwareUnicorn’s wonderful training, but I did have to make a few updates. For one thing, dyld is not guaranteed to be the next image in memory after the base image.

Also have to give a heartful thank you to C-Sto and lesnuages, who contributed code to the Windows and OSX loaders, respectively.

Last but not least, this library makes heavy use of our own fork of the debug library, so this would not have been possible without contributions over the years from the whole Binject crew, and it’s a perfect example of the power of the tools we’ve made.

This isn’t the first tool to come out of our work in Binject, and it definitely will not be the last.

Once you get locked into a serious m*****e collection, the tendency is to push it as far as you can.

Mach-O Universal / Fat Binaries

This is part two on our series on Mach-O executables, where in the first part we did a deep dive on the Mach-O file format. During the Apple-Intel transition, when they were moving from PowerPC to Intel x86_64 architecture, Apple came out with the Mach-O Universal Executable, a play on existing Fat binaries. This is one of the more interesting binaries types I’ve ever encountered. The Fat / Universal binary format acts as a wrapper for multiple Mach-O binaries of different architectures and CPU types. Meaning these binaries can run on multiple different architecture CPUs. Already having the ability to parse and write the Mach-O file structures, parsing the additional Fat headers is actually pretty simple. We’ve added both parsing and writing support for this file in our binject/debug library. Originally this was to include functionality for such 3rd party signing attacks as described in this set of posts. But legacy file types often contain many exploitable properties, and this file type may even be coming back as Apple is rumored to be moving to ARM CPUs from the Intel CPUs in the future, so let’s dig into the file format and learn some more.

Universal Binary Overview
A high level overview of a Fat / Universal Mach-O file is actually pretty simple, as the Fat headers simply act as a wrapper for the multiple Mach-O files contained within. While these can hold any number of Mach-O files within, on MacOS they are often found with two, being used to bridge between CPU transitions. Below we dive into what these Fat headers actually contain, but short to say it’s not too much. Here is an example of a Fat file structure that contains two Mach-O structures within one Universal file:

An example of the Fat / Universal Mach-O binary format

Universal Header
The Fat Mach-O file header starts with some special magic bytes, then moves on to a 4 byte arch size variable; the arch size is the length of the next dynamically sized section, the arch headers array, with one descriptor for each Mach-O structure contained within the Fat binary. Each arch header contains the same 5 properties, each 4 bytes in length. Each arch header contains the cpu and subcpu fields, which specify the architecture and cpu type the corresponding Mach-O file is for. When a Universal binary is loaded into memory this is what is checked to see what Mach-O file contained within should be loaded. Following that is the arch offset, arch size, and arch alignment, all of which specify where the corresponding Mach-O structure is located and how large it is.

N Mach-O Structures
Following the Fat binary header, there is a Mach-O structure for each of the corresponding arch headers. Each Mach-O file starts at it’s specified offset with large pads of 0 bytes between the headers and each respective Mach-O file. The Mach-O files themselves follow the exact structure outlined in the first post on Mach-Os, except for a slight difference between the reserved bytes (the i386 arch Mach-O binaries don’t seem to have the four reserved 0-bytes after the header). Simply put, these are normal Mach-O files.

Conclusion
This is an interesting file format that is still around on modern MacOS versions, and may even be coming back if Apple makes the move from Intel CPUs to ARM CPUs in the future. Ultimately, this is a pretty simple file format, but I think there is a lot of room for abuse here which we plan to experiment with in our Binjection library! Stop by soon for more interesting low level file format posts!

ahhh
Latest posts by ahhh (see all)

So You Want To Be A Mach-O Man?

Probably one of the least understood executable file formats is Mach-O, short for the Mach Object File Format, this was originally designed for NexTSTEP systems and later adopted by MacOS. When researching this file format there were a lot of documents that were misleading and the documents we did find were pretty old, prompting us to write this post. We’ve mirrored two docs (one and two) we found particularly helpful, although a little outdated. This post will also not cover Universal binaries, otherwise known as Fat Mach-O binaries, which can hold multiple Mach-O objects for different CPU architectures.

Tooling
While there are limited tools for investigating Mach-O files, there are some important ones you will want in your toolbox. One of the most useful tools you can use for parsing and visualizing a Mach-O file is MachOExplorer. otool is also a very useful native tool when investigating these files. Finally, bintriage is an interface to our custom debug library and our command line utility for investigating these files.

Intro / Background
The Mach-O file format on MacOS started with NexTSTEP CPUs targeting CMU’s Mach kernel design and Steve Jobs winning his way back into Apple’s good graces. The merger of the computing models, known as the Apple-Intel transition, gave way to multi-architecture universal file formats, which will be the subject of a follow up post. Out of those came the general Mach-O, intel 64 bit executable binary format for MacOS. That Mach-O executable file is the modern executable on MacOS (inside all Apps), and will be subject of our deep dive today.

High-Level Overview of the File Structure
Mach-O files themselves are very easy to parse, as almost every field is 4 bytes or a multiple of 4 bytes in length. Only larger abstract structures within the file format have offsets and lengths which must be observed, and even fewer data structures have required alignments. If I were to draw a diagram of the file structure based on my interactions and understanding of critical components, it would look like:

High level view of the Mach-O file structure

Mach-O Header
The header is extremely important, it helps provide data such as the magic bytes, the cpu type, and the subcpu type, which indicate the architecture and exact model cpu the binary is for. There is also the type, a 4 byte field that says whether this is an object file, a dynamic library (dylib), or an executable Mach-O file. Next is the ncmd and cmdsz fields, indicating the number of load commands and total size of the load commands. Finally there is the flags field, which is 4 bytes of bitwise flags specifying special linker options. At the end of the header is a reserved 4 byte space.

Example of a Mach-O file header

Load Commands
Probably the most important part of the Mach-O file structure are the load commands. Our debug library parses all of the critical structures required to rebuild a Mach-O file, however there are still several load commands we don’t parse but rather copy directly over. Each load command structure varies and needs to be parsed differently based on the type of load commands. Some load commands are self contained, telling the loader how to load dynamic libraries within the load command itself, where as other load commands reference other structures of data contained within the file, such as sections and tables that get loaded into segments. These are what truly represent how the file is mapped into virtual memory.

Example of some Mach-O load commands

Segments with Sections
Several segments are loaded after and corresponding to the load commands, with their respective sections. This is where a lot of the machine executable code, variables, and pointers to various functions from program code come from. Sections from the _TEXT segment and _DATA segment will almost always be present, while other segments and sections may be optional, depending on the various libraries they call and how they are compiled.

Example of some Mach-O sections

Optional Segments
Some segments are only present with certain Mach-O binary types, based on if the load commands are in place or not. An example of an optional segment would be a _DWARF segment and sections, which are used in debugging. We will likely write more on DWARF sections in a later post as there is often similar optional debug data present in ELF files.

Example of an optional _DWARF segment in a Mach-O file

LinkEdit Segment
The _LINKEDIT segment is the final segment in most Mach-O binaries and contains a number of critical components, such as the dynamic loader info, the symbol table, the string table, the dynamic symbol table, the digital signature, and potentially even more properties. This is one of the most important segments and is unfortunately left out of a lot of documentation, but make no mistake, this segment is critical for the binaries to run.

Example of some of the data in the LinkEdit Segment

Conclusion
Those are the major parts of the Mach-O file format as I understood them and worked with them to recreate working executable files. Stay tuned for a follow up post covering Fat / Universal Mach-O binaries, which act as a wrapper for multiple Mach-O binaries of different architectures. I hope this information is useful to people, and if there are questions or discussions lets talk about them in the comments and forums! If I missed anything major please let me know, and pull requests are definitely welcome on our Binject projects!

ahhh
Latest posts by ahhh (see all)