It's a blong, blong, blong road...: Rooting a HTC Sensation XE

A few months back I rooted my old phone, a HTC Desire, and put a custom ROM on it. I did this rather late in the day, after having upgraded to a newer HTC Sensation XE and encountered some stumbling blocks along the way thanks to various OTA (over the air) updates that had been installed on the older phone, which made many available rooting guides and tools out of date and unable to help. However, when eventually done, the phone received a custom ROM (Supernova), demonstrates a fabulous battery life and has all apps and data regularly backed up.

The newer dual-core Sensation XE was demonstrating a sub-optimal battery life (whether or not this related to the recent OTA update to Android 4.03, or Ice Cream Sandwich (ICS), I couldn’t really be sure) and lacked a good solution for a thorough backup, so I thought it might be an idea to root this one also. This post is a write-up of various notes I’ve made getting my head around the quite complicated process of rooting a specific HTC Android phone.

Background information

HTC’s Android phones run a modified version of standard Android (AOSP, or Android Open Source Project) with an extra UI layer installed, called HTC Sense, to establish the brand and offer some potentially useful apps and widgets.

Android itself is a Linux-based OS embedded into a given device that may or may not have various hardware capabilities, such as cellular communication, camera, keyboard, etc. The OS and loader, as well as some internal storage and cache are typically described as being in the device’s ROM. However the ROM is really some NAND flash storage (much like in a USB flash drive) and so, given appropriate circumstances, is read/write - clearly you can install applications on an Android device and those applications store operational data.

The NAND storage is split up into partitions, some standard ones and some device-specific ones. Some partitions are used for the boot process and are not available at runtime to Android. Those partitions that are available are mounted either read-only or read-write as deemed appropriate. Some of the standard partitions on an Android device include:

/boot – the Android boot files, including Android’s Linux kernel
/system – the main Android OS files, HTC Sense, the boot animation
/recovery – the recovery OS, which includes its own small kernel
/data – user applications and data, Android settings etc.
/cache – OS cached data

The /system partition is mounted read-only to ensure key system binaries are not changed or broken during system uptime. The /data and /cache partitions are mounted read-write so apps can be installed and settings can be changed. When you perform a factory reset on a device the /data and /cache partitions are emptied or formatted. /boot and /recovery are not mounted for runtime Android use.

This information is backed up by these two screenshots showing the same information, one from a phone, one from a PC:

The ro in the response regarding /system indicates read-only, rw in the next two indicates read-write, and the requests for mount information for /boot and /recovery yield nothing.

On a phone there will also be a /radio partition containing the radio firmware used for cellular communication, data communication, GPS and Bluetooth. Additionally there is a small /hboot partition containing the bootloader, which is usually referred to as HBOOT.

When the device is first switched on, HBOOT is the first thing to run. HBOOT is the equivalent of a PC’s BIOS (a primary boot loader) or a Linux boot loader (a second-stage boot loader). It decides whether to launch Android by running /boot or whether to launch recovery by running /recovery.

The recovery system is designed to update your system when OTA updates arrive. When launched, it mounts /system as read-write so it can change things as required. One of the first things required to root a device or install a custom ROM is to replace the stock recovery software with custom recovery, which has a UI and can be told what you want it to do. A popular custom recovery is ClockworkMod Recovery. You can find a ClockworkMod recovery image for your device on their download page.

This is where the usual disclaimers kick in – as soon as you start replacing the standard software set up on the device you risk making the device unusable. An unusable smart phone is no better than a paperweight or brick, hence the term bricking your phone. If it all goes wrong you have only yourself to blame for taking the risk.

Given a custom recovery you can install update files. These can do something as simple as install one or more files on the /system partition (by first mounting /system as read-write), or alternatively set up an entire new system across several partitions (i.e. install a custom ROM).

HBOOT has a UI that you can use to manually launch into /recovery or continue to a regular system start (or indeed do a factory reset). It can be invoked through SDK command-line tools or be launched manually. To do the latter you ensure Fast Boot (a quicker turn off/turn on cycle) is disabled (in ICS this is done by selecting Menu, Settings, Power and disabling Fast boot) and then turn off the phone. To ensure the procedure works, remove and reinsert the battery (you probably won’t need to do this). Now turn on the phone with the Volume Down button pressed in. The phone should start in HBOOT/bootloader mode showing a text menu. You navigate this menu using the Volume keys and select an item with the Power button. Note that until you install a custom recovery image there is little point booting to recovery.

Rooting an Android device involves allowing the user to run with administrative privileges. On Linux the administrative user is called root (which is a little ambiguous, given the top level directory in a Linux directory hierarchy, /, is also called root or, more specifically, the root directory). By default the phone user has permissions to store data on the SD card and browse /system, /data is not accessible at all – the OS manages that area. Running with administrative privileges removes these restrictions and this is done using the Linux su (substitute user, often thought of as super user) command. Custom recovery can be used to install this (absent by default) command. Adding the ability to run with root privileges is called rooting the device or gaining root. This is the equivalent of what iPhone users call jail-breaking their phones.

Installing a custom ROM onto an Android device involves wiping the /boot, /system, /data and /cache partitions and writing replacement content to /boot, /system and /data.

Custom recovery also typically offers a NANDroid backup. This is a backup to your SD card of the /boot, /recovery, /system, /data and /cache partitions. If a subsequent custom ROM behaves poorly or breaks operation then the NANDroid backup can restore the device to a known working state.

The phone’s vendor (HTC in this case) also has internal ways of flashing a whole stock ROM onto the device using an RUU (ROM Update Utility). These are internal apps that will set the device to a given OS version, and are carrier-specific. For example, an Orange-branded HTC device can only have an RUU installed that is designed for an Orange-branded HTC device. This installation restriction is achieved by its checking the CID (Carrier ID) of the device and ensuring the CID matches the signed ROM/firmware update inside the RUU match. Various leaked ROMs for different regions and carriers for the HTC Sensation (internal model name is Pyramid) and HTC Sensation XE (internal model name is Pyramid LE) are listed here and for the HTC Desire (model name is Bravo) are listed here.

If you wanted to de-brand a branded HTC phone you can set a Super CID. This is quite straightforward and a little searching will show you how to do this using a couple of command-line tools from the Android SDK. It wasn’t an issue for me as neither my HTC Sensation XE nor HTC Desire were carrier-branded.

HTC Specifics

A few years back, HTC decided that since the NAND storage was crucial to the successful functioning of the device they would add in a security layer to ensure it remains protected. When you bump into the terms S-ON and S-OFF they are referring to this security layer.

When starting up, HBOOT checks the security setting, which is stored in the device’s radio NVRAM in a flag call @secuflag. If the flag is set, you are S-ON (NAND security is enabled) and the NAND memory is locked. So permanently rooting the phone or installing custom ROMs is not possible until S-OFF is attained (NAND security is disabled). The exception to this rule is that if you are flashing a file with the standard recovery, and the file is signed by a special HTC key, then the security is bypassed; this is how official OTA updates are able to work.

S-OFF can be attained either by managing to flash a new radio image with @secuflag turned off, or alternatively flashing a new HBOOT that does not check the radio’s @secuflag.

There are various tools, such as unrEVOked Forever and Revolutionary, that take advantage of exploits that have been discovered to facilitate gaining S-OFF. However, HTC’s OTA updates have been updating HBOOT to close down these exploits. Indeed one of the issues I had rooting my HTC Desire was that an OTA update had updated HBOOT making the above tools unable to make it S-OFF.

To help with this S-OFF issue there is another technique available – a tool called JuopunutBear, currently in beta and supporting a few recent HTC phones including the HTC Sensation and HTC Sensation XE. This tool should establish S-OFF but requires some (simple) hardware hacking in the form of the so-called “wire trick” (involves briefly connected the SD card case to an exposed contact).

However, resorting to tricks to gain S-OFF may perhaps not be necessary after all. As of late 2011 HTC has yielded to popular demand and provided an official way to lower the NAND security. By default recent HTC phones are shipped both S-ON and bootloader-locked. By following along on their dedicated web site (and acknowledging their various disclaimers) HTC will allow you to unlock your bootloader.

Note: the HTC site does make it quite clear that if you unlock your bootloader and then brick the device your warranty won’t cover it.

Having an unlocked bootloader is similar to S-OFF but not quite as liberating. S-OFF allows you to walk all over all NAND memory but being unlocked keeps /hboot and /radio (among other device-specific partitions) protected and does not permit partition resizing. That’s not much of an issue as long as the radio firmware is appropriate for your needs (some custom ROMS suggest specific radio firmware works better). You can still set up a custom recovery and therefore root the device and install custom ROMs.

My Progress

Ok, so there’s all the background information I managed to acquire. The following echoes what I did to root the phone and install a custom ROM.

My HTC Sensation XE shipped with a locked bootloader (HBOOT v1.27.0.0) meaning that full S-OFF is potentially a bit tricky to achieve so I opted to just use the HTC unlock route.

Unlocking the bootloader

In order to do any of this you need to be able to communicate with the phone from a computer using a USB cable. Assuming you’re using a Windows PC then you will need the HTC drivers installed. This is done as part of the process of installing HTC Sync, so if that’s installed then communication can be established.

To control the phone you need two Android SDK tools: adb.exe and fastboot.exe. If you’re a developer then it should be no big deal to install the JDK (Java Development Kit) and Android SDK, run the SDK Manager.exe and tell it to install the Android SDK Tools and Android SDK Platform-tools. This should give you the files you need in the Android SDK’s platform-tools directory, though I gather you can also find and download the required files (adb.exe, fastboot.exe and AdbWinApi.dll) online. Specifically, on a fresh install of the current SDK fastboot.exe may well be absent but you can get it from an older revision: android-sdk_r13-windows.zip.

adb.exe is the Android Debug Bridge, used to communicate with an Android phone over USB when Android is running normally. fastboot.exe can be used to communicate with the phone when in HBOOT mode. Both tools are command-line tools, so you’ll need a command prompt: WinKey+R, cmd, <Enter>.

Warning: this procedure will wipe all apps and data from your phone leaving it just like when you received it (albeit with any OTA updates still applied). In other words a factory reset will be applied. If there is anything you are able to save/backup/note down, do it now!

Note: the factory reset above will not affect your SD card, but it’s a good plan to periodically backup the SD card anyway.

Note: ensure you have a full battery before starting this whole process.

To use HTC’s bootloader unlocker you’ll need to register on htcdev.com, then select your device in the list on the start page, accept all the disclaimers and follow the instructions, which amount to this list of steps:

Reboot phone into bootloader mode. You can either do that manually as described above, or with these commands (note I prefix all adb commands with a kill command simply because I was having trouble maintaining a connection with the adb daemon):
adb kill-server
adb reboot bootloader
When the phone boots into HBOOT, get your OEM identifier token:
fastboot oem get_identifier_token
This presents a lengthy token over several lines enclosed by two sets of chevrons or angle brackets. Copy this whole token into the clipboard, including the chevrons by right-clicking the command prompt window, choosing Mark and selecting a rectangle from the top left chevron on the Identifier Token Start line to the bottom right chevron on the Identifier Token End line.
Paste token into the box in Step 10 on the appropriate HTC dev page and press Submit.
When you receive the Unlock_code.bin file in an email shortly after, save that file to disk and flash it to your phone:
fastboot flash unlocktoken Unlock_code.bin
Accept the phone’s warning that the phone will get a factory reset and press the Power button to apply.

Now you have an unlocked bootloader and what is in effect a fresh-from-the-box phone.

Install custom recovery

You are now able to install a custom recovery. In the case of CWM (ClockworkMod) you can choose either a menu-driven one that operates much like HBOOT or a newer touch-driven one. I opted for the latter and downloaded a suitable version from the available collection on offer. I also noted the MD5 sum of the image so I could check the downloaded file had come down intact. To calculate the MD5 sum of the download I used md5sum, a command-line utility that you can readily find various implementations of with a quick Internet search.

Once I’d confirmed that the custom recovery image was intact I flashed it with these commands (the fastboot flash command spills over onto two lines with the width of this blog thanks to the long custom recovery image file name):

adb kill-server
adb reboot bootloader
fastboot flash recovery recovery-clockwork-touch-
5.8.0.9-pyramid.img
fastboot reboot

With a custom recovery installed the phone can now be rooted. You need to locate a suitable update file containing an Android super user app and binary – I found one attached to this XDA Developers thread as the file CWM-SuperSU-v0.89.zip. This needs to be copied to your SD card’s root directory (the file name is, apparently, irrelevant) and then installed by the custom recovery (i.e. by CWM). So:

run these commands:
adb kill-server
adb push CWM-SuperSU-v0.88.zip /sdcard/SuperSU.zip
adb reboot recovery
Choose ‘install zip from sdcard’
Choose ‘choose zip from sdcard’
Scroll down and select SuperSU.zip
Select ‘Yes – Install SuperSU.zip’
Once the file has been installed select ‘+++++Go Back+++++’
Select ‘reboot system now’
When Android has started, verify SuperSU is in the apps list and launch it to ensure it is set up to work

Any apps that require root access will now cause SuperSu to ask for confirmation and, if given, will grant root access. For example:

Installing a custom ROM

There are many custom ROMS out there for various Android phones and quite a few for any individual phone. For a HTC phone like the Sensation XE some ROMs will include HTC Sense (some using version 3.6 as shipped with the Sensation ICS update and some working with Sense 4.0) and some will not (classed as AOSP). A good list of them for the Sensation and Sensation XE can be found at htcsensationroms.com.

I decided to start with Android Revolution HD by XDA user mike1986 as this one uses HTC Sense 3.0 and ICS, but has been highly optimised to hopefully provide better battery usage and not look wildly different from what I have been used to with the stock ICS update I’ve had for a month.

Looking down the message thread for this ROM I noted that it requires a firmware update to get a specific radio firmware version: “With 6.6.x ROMs I strongly recommend flashing 11.69.3504.00U_11.22.3504.07_M radio. Using a "non matched" radio could have a negative effect on performance, battery life & signal strength.”

Hmm. Alas, without S-OFF this will be impossible, so should I try the JuopunutBear wire trick? Well, in my case it wasn’t necessary. I checked my versions (Menu, About phone, Software information, More) and my Baseband version was already listed as being 11.69.3504.00U_11.22.3504.07_M – so that was a lucky break! I am additionally considering testing Jonny’s DarkForest Sensation XE ROM. This one states I would need to upgrade my firmware (which includes radio firmware & config, power management and so forth) to 3.32.401.105 – again, a quick check shows my Build number to be 3.32.401.105.

It seems that the recent OTA update to ICS has given me what is required for these two custom ROMs, so I can proceed!

I downloaded Android Revolution HD for the Sensation XE from the link in this thread, which gave me Android_Revolution_HD-Sensation_6.6.3_XE.zip. This needs to be copied to the SD card, but this time I found that keeping the file name intact was a requirement.

Before installing a completely different ROM it is mandatory to wipe all pertinent partitions, usually through a script (called SuperWipe) that can be invoked from recovery. There is a link to a SuperWipe script on the Android Revolution HD thread, however I encountered a problem when I ran it from recovery – it baulked at wiping the first partition. I’m thinking this is probably due to my renaming the file but I haven’t tested this theory. Instead I found a link to another version on this AndroidMe thread and used that, which worked fine (I didn’t rename that one).

So, the steps are:

Download ROM: Android_Revolution_HD-Sensation_6.6.3_XE.zip
Download SuperWipe (the archive name may vary): SuperWipe.zip
Copy both files to the SD card’s root directory. Note the trailing slash on the sdcard destination is important! It tells the command to copy the file into the specified directory, as opposed to over the SD card mount point, thereby hiding the directory – I made that mistake whilst trying to set up a custom ROM on my HTC Desire and took some while to resolve):
adb kill-server
adb push Android_Revolution_HD-
Sensation_6.6.3_XE.zip /sdcard/
adb push SuperWipe.zip /sdcard/
Boot to recovery:
adb reboot recovery
Choose ‘install zip from sdcard’
Choose ‘choose zip from sdcard’
Scroll down and select SuperWipe.zip
Select ‘Yes – Install SuperWipe.zip’
Once the script has run choose ‘choose zip from sdcard’
Scroll down and select Android_Revolution_HD-Sensation_6.6.3_XE.zip
Select ‘Yes – Install Android_Revolution_HD-Sensation_6.6.3_XE.zip’
Once the ROM is installed select ‘reboot system now’

And that gives Android Revolution HD:

So I’ve now installed Titanium Backup and have a regular backup being made of all apps and data on my phone, and my battery life seems to have improved noticeably!

I’ll be making NANDroid backups (to allow easy reversion to a given custom ROM) and testing out some other ROMs in the near future.

Custom ROM mods

After a custom ROM you may have the opportunity to install additional mods to add or change functionality. The second post at the Android Revolution HD thread lists a bunch of available mods. Custom mods are installed in the same way as rooting, SuperWipe and custom ROMs. Download the mod update file, copy it to the SD card, reboot to custom recovery and install the ZIP.

I wanted an enhanced QuickSettings area and an enhanced Power Menu so I chose Advanced APM by j4n87, described here and Extended QuickSettings by baadnewz.