An adventure 13 years in the making.

iPhonelinux: A decade later

Over ten years ago, our co-founders David Wang (@planetbeing) and Chris Wade (@cmwdotme) started a project to port Android to the original iPhone.

You can view the original project here: http://linuxoniphone.blogspot.com
Or check out the original code base here: https://github.com/planetbeing/iphonelinux

This original project took over a year to complete, with input from many of the top iOS engineers in the world. Today, we were able to achieve this for the latest version of Android, on a more sophisticated iPhone model, in less than a month.

THe power of virtualization

We would not have been able to port Android nearly so quickly, if at all, without relying on Corellium’s revolutionary mobile device virtualization platform. Our platform creates software-based models of mobile device hardware, enabling users to run ARM-based operating systems on ARM-based enterprise servers. This groundbreaking advancement empowers engineers with scalability, efficiency, and innovative new tools for research, testing, training, and development purposes.

By leveraging our virtual devices, along with our deep knowledge of both the Android OS and the iPhone hardware, we were able to rapidly iterate to bring Android to life.

The Linux kernel

An Android port starts with a working and stable Linux kernel. The kernel is very flexible and its design can accommodate almost all quirks of our target hardware, but drivers are another matter.

The phone we chose for our adventure shares very few peripheral blocks with any other device supported in Linux, and the ones it does share are generally customized. Fortunately, Corellium VMs can model the H9P SoC, so most of the early kernel port did not involve real phones. The VM has a nice fast debugger, a number of logging and patching facilities to improve kernel bring-up, and most importantly is really fast to reboot. All we had to do is enable the Linux kernel loader on these VMs.

Of course, the first thing you need to do is print, so our first driver was for the uniquely modified UART. It shares some heritage with UARTs in the Samsung S3C series SoCs, but did not run with an unmodified driver in the Linux kernel.

The SPI controller was a similar story - also "inspired" by Samsung, but not compatible (and the driver in the kernel tree was not designed for our sort of application - it does not support interrupt driven transfers), so we had to write our own driver.

More interesting is the heritage of the I2C block. It was neither a Samsung derivative, nor a common Synopsys DesignWare core; instead, it derived clearly from a PASemi product. The PASemi driver in the kernel tree was, again, not capable of interrupt operation, and generally somewhat unfinished, so we wrote our own.

All SoCs in the family share a unique, non-standard interrupt controller, called AIC. AArch64 (64-bit ARM) devices running Linux pretty much always use one of the ARM GIC standard compliant controllers (which we're familiar with, since it's shared by most of our models as well as the servers Corellium runs on). Additionally, in an equally non-standard manner, the SoCs deliver timer interrupts via the so-called FIQ path, a nearly forgotten feature from the old 32-bit ARM times, and completely unsupported in AArch64 Linux. So we had to build FIQ support from scratch - more than just replacing an interrupt controller driver.

Armed with the power of VM debugging, though, very soon we found ourselves at a ramdisk-based command prompt, with a few other peripheral devices - like the RTC or GPIO - supported.

However, because our VMs were constructed without access to real hardware (Corellium supported this phone before the checkm8 exploit was announced), we would occasionally run into surprises when checking against real devices.

After a cheerful pre-boot 'Hello, world!' hand-coded in start-up code assembly, we ran into the first one: it turned out that the processor cores don't merely support 16kB pages, but they actually require them. The Linux kernel, however, took this in stride, and as soon as we switched the page size configuration option, it was happy to run.

Another unexpected delay happened when we tried bringing up the second CPU core. Most ARM systems use one of two mechanisms for this: poll-table, which has the boot CPU write the start address for another CPUs into a special memory location, or PSCI, which asks the bootloader to start another CPU in a portable manner. From our VM models we knew this was not the case; instead, a pair of magic MMIO registers is used to set the boot address and then bring the other CPU out of power-off state. What we did not know is that the boot address register would only let itself be written once per system start, and we needed to keep it unlocked during the boot process to be able to write it in Linux.

After we got a real device to an interactive ramdisk-based shell, we moved on to bring up the complex PCI Express interface. Again, initial bring-up in a virtual machine let us write most of the driver without having to dirty our hands with real hardware. Only some register writes that perform low-level adjustments had to be verified against a physical phone, and in a short few days we had the NVMe storage chip visible in `lspci'.

The Corellium VM uses a generic, standards-compliant NVMe model we created for all systems that have NVMe storage. We add specific quirks to better model the variants used by them, but fundamentally the NVMe model is quite complete in each. This was proven slightly and subtly wrong, when the actual NVMe chip would fail at writing large blocks to memory via the regular DART (IOMMU). Instead of data read from Flash, we'd get sections of zeros in RAM, until we switched our memory access to the little-known NVMMU feature present in H9P.

As far as Linux is concerned, NVMMU is basically another IOMMU, optimized for quickly creating and removing contiguous blocks of bus addresses mapped to system RAM. It turned out that as soon as we switched from DART to NVMMU, our zero-memory issues disappeared and we had working Flash access.

Android

Corellium has built up a considerable amount of experience with Android by now, supporting multiple virtual models for which we build firmware ourselves, and physical phone models where we just model the hardware. But all of those Android devices share one characteristic: 4kB memory pages. In fact, the 4kB page size is built into Android in so many places!

At this point, our VM was running with 16kB page size in kernel, so it would catch the 4kB assumptions we missed pretty much instantly. But they were still very laborious to fix. Eventually, we (probably) found them all and we could see the Android logo on the screen give way to the familiar AOSP launcher we've seen so many times before.

Unfortunately, Android developers enshrined the 4kB page size in the build system, breaking AArch64 convention; third-party applications containing binary libraries built for these small pages will not be able to start on our Android port and will need to be rebuilt. It's not intrinsically hard - one or two command-line options - but it can't be done if all you have is an APK.

Less limiting is the lack of 32-bit code support on our platform. While Android these days requires support for pure 64-bit systems from application developers, the system itself still has moldy chunks of 32-bit only code in unexpected places. We forced them to build as 64-bit, resulting in what may be the first pure 64-bit Android device around - an experiment we have tried before on one of our 64-bit only VM servers (these days, Corellium Android VMs support 32-bit code even on 64-bit only servers, thanks to our transparent binary translator in the hypervisor).

To finish off the Android port, we added support for the touch controller and WiFi chip. The touch controller is not very complex to interface with, but unlike the Broadcom WiFi device it had no driver in Linux. What the devices share is the need to load firmware.

That firmware is loaded directly from the phone's Flash memory. It is stored there on an unusual filesystem called APFS, which has no driver in mainline Linux kernel. Fortunately, a group of Linux hackers under the direction of Ernesto Fernandez have been working on implementing a kernel driver for it. We just had to extend the driver by adding support for compressed files and concurrent mounting of subvolumes.

At the very end, we customized our Android build with a more pleasant launcher, OpenLauncher, and built the Signal app into the Flash image.