It's a blong, blong, blong road...: Android from the command line

Whilst working with Mono for Android I was pretty much confined to the IDE with regard to building apps and deploying them to the emulator or to a real Android device. That was quite important as Mono for Android had various non-standard additional steps to ensure all the pieces were in place, what with there being a Mono layer, and dependencies (at least with debug builds) on the Mono shared runtime and a platform-specific additional runtime.

Now that I’m working with Project Cooper, building native Android apps, I have the same luxury of choice as a Java developer. I can do everything in the development environment or drop to a command prompt (if in Windows, or a terminal on Linux or on a Mac, though to simplify things I’ll be assuming Windows) and do any amount of stuff from there.

I figured it may be an idea to run through the steps involved in turning a .jar file into an Android app, getting on a device or emulator, and starting it. In other words show a variety of Android-related commands that I’ve been using recently.

Before kicking off, I should ensure we are all on the same page and point out that a .jar file is just a zip file. It contains one or more .class files, where a .class file is a binary files containing Java VM byte code. The Java compiler, javac.exe, emits .class files that can be packaged up in .jar files using the jar.exe command, although Eclipse typically does this for you. The Project Cooper compiler emits .jar files as a matter of course.

So, ignoring the somewhat crucial topic of how to write an Android application (I’ll loop back round to this subject with Project Cooper in time, having actually done this for Mono for Android here), I’ll work on the basis that you have an Android app all coded up and successfully compiled into a .jar file. There will be a directory hierarchy full of resources (images, layout files, strings and so on), probably called res, and also an Android manifest file called AndroidManifest.xml, which among other things, lists all the key application components and identifies the activity to launch when the app is started.

First things first. Using the Android SDK dictates that you are also using the JDK (Java Development Kit). To simplify the command lines it will be useful to put 3 directories onto the system search path. Two of these are from the Android SDK, which if you downloaded the executable installer, will by default be in your Program Files directory tree.

Note: If you’re working with Windows Vista or Windows 7 you will need to remember to run the Android SDK Manager as Administrator to enable it to have permission to modify the content of its Program Files subdirectory.

Anyway, the directories to add to the PATH:

the JDK bin directory, something like: C:\Program Files\Java\jdk1.6.0_25\bin
the Android SDK tools directory, e.g.: C:\Program Files\Android\android-sdk-windows\tools
the Android SDK platform tools directory, e.g. C:\Program Files\Android\android-sdk-windows\platform-tools

You can either add these to the system PATH or make a simple batch file that adds these to a local PATH each time you invoke it from a command prompt.

There are several steps to creating a usable Android application from a .jar and some resources. They are summarised for the Java developer at http://d.android.com/guide/developing/building/index.html#detailed-build in this image:

In the steps below, I’ll assume I have a project in a directory called C:\Cooper\com.blong.Torch and in a command prompt I’ve already changed to that directory. In this case, the project has been compiled in the Debug configuration and so the compiled .jar is C:\Cooper\com.blong.Torch\obj\Java\Debug\com.blong.Torch.jar, the resources are in the res subdirectory and the manifest file is C:\Cooper\com.blong.Torch\Properties\AndroidManifest. Temporary build files can be placed in the C:\Cooper\com.blong.Torch\obj\Java\Debug directory.

The project is being compiled against the Android SDK platform release 8 (the Android FroYo release, or version 2.2), so again it will be assumed I’ve used the Android SDK and AVD Manager to ensure I have that SDK platform installed. In my case I’m running on 64-bit Windows so any Program Files references will actually be Program Files (x86).

Along the way I’ll set up some environment variables from the prompt just to hopefully ‘simplify’ the command-lines.

set TARGET_ANDROID_API=8

set ANDROID=C:\Program Files (x86)\Android\android-sdk-windows

set ANDROID_PLATFORM=%ANDROID%\platforms\android-%TARGET_ANDROID_API%

set ANDROID_JAR="%ANDROID_PLATFORM%\android.jar"

set MANIFEST=Properties\AndroidManifest.xml

set RESOURCE_DIR=res

set TEMP=obj\Java

set BUILD=Debug

Step 1: Compiling the resources using the Android Asset Packaging Tool (aapt.exe)

The aapt command can now be run to collate all the resources from the res and output them to a designated zip file.

set RESOURCE_PACKAGE=%TEMP%\%BUILD%\resources.zip

aapt package -f -M %MANIFEST% -S %RESOURCE_DIR% -I %ANDROID_JAR% -F %RESOURCE_PACKAGE%

Adding in a -v flag shows you what aapt is up to:

C:\Cooper\com.blong.Torch>aapt package -v -f -M Properties\AndroidManifest.xml -S res -I "C:\Program Files (x86)\Android\android-sdk-windows\platforms\android-8
\android.jar" -F obj\Java\Debug\resources.zip

Locale/Vendor pairs:
   /
   /
   /
   /

Files:
  drawable-hdpi\icon.png
      Src: res\drawable-hdpi\icon.png
  drawable-ldpi\icon.png
      Src: res\drawable-ldpi\icon.png
  drawable-mdpi\icon.png
      Src: res\drawable-mdpi\icon.png
  layout\main.xml
      Src: res\layout\main.xml
  values\strings.xml
      Src: res\values\strings.xml
  AndroidManifest.xml
      Src: Properties\AndroidManifest.xml
Including resources from package: C:\Program Files (x86)\Android\android-sdk-windows\platforms\android-8\android.jar
applyFileOverlay for drawable
applyFileOverlay for layout
applyFileOverlay for anim
applyFileOverlay for animator
applyFileOverlay for interpolator
applyFileOverlay for xml
applyFileOverlay for raw
applyFileOverlay for color
applyFileOverlay for menu
applyFileOverlay for mipmap
    (processed image res\drawable-hdpi\icon.png: 74% size of source)
    (processed image res\drawable-ldpi\icon.png: 56% size of source)
    (processed image res\drawable-mdpi\icon.png: 64% size of source)
    (new resource id icon from drawable-hdpi\icon.png #generated)
    (new resource id icon from drawable-hdpi\icon.png #generated)
    (new resource id icon from drawable-hdpi\icon.png #generated)
    (new resource id main from res\layout\main.xml)
Opening 'obj\Java\Debug\resources.zip'
Writing all files...
      'res/layout/main.xml' (compressed 61%)
      'AndroidManifest.xml' (compressed 64%)
      'resources.arsc' (not compressed)
      'res/drawable-hdpi/icon.png' (not compressed)
      'res/drawable-ldpi/icon.png' (not compressed)
      'res/drawable-mdpi/icon.png' (not compressed)
Generated 6 files
Included 0 files from jar/zip files.
Checking for deleted files
Done!

One thing is missing from a typical aapt invocation though. aapt is normally run before the compilation and this allows aapt to generate a simple class containing ids for all the resources, including strings, found during the creation of the resources package. For the Android code to refer to any resource it uses the resource id, and so needs this class to be available. aapt is happy to create a Java resource class in a file called R.java and will put it in the directory specified with the -J switch. So:

set RESOURCE_PACKAGE=%TEMP%\%BUILD%\resources.zip

aapt package -v -f -M %MANIFEST% -S %RESOURCE_DIR% -I %ANDROID_JAR% -J %TEMP%\%BUILD% -F %RESOURCE_PACKAGE%

This generates a helper class obj\Java\Debug\R.java thus:

/* AUTO-GENERATED FILE.  DO NOT MODIFY.

 * This class was automatically generated by the

 * aapt tool from the resource data it found.  It

 * should not be modified by hand.

*/

package com.blong.Torch;

public final class R {

    public static final class attr {

    public static final class drawable {

        public static final int icon=0x7f020000;

    public static final class id {

        public static final int torchButton=0x7f050000;

    public static final class layout {

        public static final int main=0x7f030000;

    public static final class string {

        public static final int app_name=0x7f040000;

        public static final int main_activity_name=0x7f040001;

        public static final int torchButton_Text=0x7f040003;

        public static final int torchButton_Toast=0x7f040004;

        public static final int torchDialog_Message=0x7f040005;

        public static final int torchMenuItem_Text=0x7f040006;

        public static final int torch_activity_name=0x7f040002;

Step 2: Building the final .jar file for your application

So now you can finish any coding that refers to new resources and, in the case of a Java project, can include this R.java source file in the compilation and get a .jar out the end. In the case of Project Cooper this won't work as it is not a Java compiler (it just targets the Java VM with its generated code). To work around this we can compile R.java into R.class then compress R.class into R.jar:

pushd %TEMP%\%BUILD%

javac -d Java R.java

jar cvf R.jar -C Java .

popd

This code temporarily changes directory into the obj\Java\Debug directory where R.java lives and uses the Java compiler, javac.exe, to compile it into a set of binary .class files in a directory hierarchy placed in a further Java subdirectory. jar.exe is then invoked and told to create a new .jar file called R.jar containing all the classes found in this new Java subdirectory with verbose output. The result of this is R.jar sitting next to R.java.

Whilst this works just fine and dandy, it should be pointed out that the normal compilation process with project Cooper, using the IDE or MSBuild, will automatically generate a suitable Pascal file, equivalent to R.java, for consumption in a Project Cooper project. I'm not sure that will be invokable when doing a step-by-step manual build process as we are here, however, so adding R.jar into the project is fine in that case.

At this point any remaining compilation tasks can be taken care of to produce the application .jar, and as mentioned, this application's .jar file is sitting in obj\Java\Debug\com.blong.Torch.jar.

Step 3: Generating Dalvik code from the application classes in the .jar

The dx.bat batch file has the role of invoking some process to analyse all the code in your application's .jar file and generate compact .dex code suitable for running on the Dalvik VM (the Java VM that Android uses). Annoyingly, and this may be due to dx being a batch file and also may be due to ignorance on my part for not working out how to get around it, dx.bat needs fully qualified names for the files you pass it.

set PROJ_NAME=com.blong.Torch

set PROJ_DIR=C:\Cooper\%PROJ_NAME%

set DEX_FILE=%TEMP%\%BUILD%\classes.dex

set DEX_FILE_FULL=%PROJ_DIR%\%DEX_FILE%

set PROJECT_JAR=%TEMP%\%BUILD%\%PROJ_NAME%.jar

set PROJECT_JAR_FULL=%PROJ_DIR%\%PROJECT_JAR%

call dx --dex --verbose --output=%DEX_FILE_FULL% --positions=lines %PROJECT_JAR_FULL%

dx.bat can turn Java classes into DEX classes, but can also disassemble .dex files etc., so --dex tells it what its job is. The --positions option is used in the standard build scripts so is also used here. I suspect it relates to how debug information is generated, though haven't found a good explanation yet. So my dx call looks like this

C:\Cooper\com.blong.Torch>call dx --dex --verbose --output=C:\Cooper\com.blong.Torch\obj\Java\Debug\classes.dex --positions=lines C:\Cooper\com.blong.Torch\obj\Java\Debug\com.blong.Torch.jar

processing archive C:\Cooper\com.blong.Torch\obj\Java\Debug\com.blong.Torch.jar.
..
processing com/blong/torch/MainActivity.class...
processing com/blong/torch/MainActivity$1.class...
processing com/blong/torch/MainActivity$2.class...
processing com/blong/torch/MainActivity$3.class...
processing com/blong/torch/TorchActivity.class...
processing com/blong/torch/R.class...
processing com/blong/torch/R$attr.class...
processing com/blong/torch/R$drawable.class...
processing com/blong/torch/R$id.class...
processing com/blong/torch/R$layout.class...
processing com/blong/torch/R$string.class...
ignored resource META-INF/MANIFEST.MF

Step 4: Generating an Android package

The next step is to combine the resources package with the DEX classes to create an Android package. This is done using another batch file, apkbuilder.bat, so again full paths are required. It's worth mentioning that when invoked this batch file makes it very clear that it is deprecated and may stop functioning at any point. There isn't an alternative command-line option, but the suggestion is to build a new tool using the com.android.sdklib.build.ApkBuilder class. anyway, to invoke apkbuilder with verbose output use the following:

set PKG_SIGNED=%TEMP%\%BUILD%\%PROJ_NAME%.apk

set PKG_SIGNED_FULL=%PROJ_DIR%\%PKG_SIGNED%

set RESOURCE_PACKAGE_FULL=%PROJ_DIR%\%RESOURCE_PACKAGE%

call apkbuilder %PKG_SIGNED_FULL% -v -z %RESOURCE_PACKAGE_FULL% -f %DEX_FILE_FULL%

The output package name is specified first. -z lets you pass in a compressed zip file and -f passes in a regular file. This gives output like:

C:\Cooper\com.blong.Torch>call apkbuilder C:\Cooper\com.blong.Torch\obj\Java\Debug\com.blong.Torch.apk -v -z C:\Cooper\com.blong.Torch\obj\Java\Debug\resources.zip -f C:\Cooper\com.blong.Torch\obj\Java\Debug\classes.dex

THIS TOOL IS DEPRECATED. See --help for more information.

Using keystore: C:\Users\Brian\.android\debug.keystore
Packaging com.blong.Torch.apk
C:\Cooper\com.blong.Torch\obj\Java\Debug\resources.zip:
=> res/layout/main.xml
=> AndroidManifest.xml
=> resources.arsc
=> res/drawable-hdpi/icon.png
=> res/drawable-ldpi/icon.png
=> res/drawable-mdpi/icon.png
C:\Cooper\com.blong.Torch\obj\Java\Debug\classes.dex => classes.dex

You might notice that it has sought out a keystore - a file containing a certificate - and by default apkbuilder will sign your package with this default debug certificate. This is handy as it saves you having to sign it manually.

However, there are cases where you will need to explicitly sign your package. For example, when you develop an app that you want to sumbit to the Android Market you'll need to sign it with a specific certificate. This allows you to submit updates.

Additionally, the debug certificate is only valid for 1 year from its creation point (when the Android build tools were first used). Also, if you want to use the Google Maps MapView class, you'll need to get a key from Google which will be based on the MD5 fingerprint of your application's certificate, so it makes sense to have one specifically for your app (or indeed for all your apps).

So let's tell apkbuilder not to sign the package using the -u switch. We'll do that bit ourselves.

set PKG_UNSIGNED=%TEMP%\%BUILD%\%PROJ_NAME%_unsigned.apk

set PKG_UNSIGNED_FULL=%PROJ_DIR%\%PKG_UNSIGNED%

set RESOURCE_PACKAGE_FULL=%PROJ_DIR%\%RESOURCE_PACKAGE%

call apkbuilder %PKG_UNSIGNED_FULL% -v -u -z %RESOURCE_PACKAGE_FULL% -f %DEX_FILE_FULL%

Step 5: Creating a key store

We use the Java tool keytool.exe for this job. When the Android build tools are first used, the debug certificate is created using this command-line:

keytool -genkeypair -alias androiddebugkey -dname "CN=Android Debug,O=Android,C=US" -storepass android -keypass android -keystore %USERPROFILE%\.android\debug.keystore

To create your own certificate store use something like this, obviously substituting bits to be more appropriate for you:

set KEY_STORE=%TEMP%\%BUILD%\certificate.keystore

set KEY_ALIAS=androiddebugkey

set KEY_PWD=key_password

set STORE_PWD=store_password

set DNAME="CN=Brian Long,O=blong,C=UK"

if not exist %KEY_STORE% keytool -genkey -v -keystore %KEY_STORE% -alias %KEY_ALIAS% -keyalg RSA -keysize 2048 -storepass %STORE_PWD% -keypass %KEY_PWD% -dname %DNAME% -validity 10000

if you omit the -storepass and -keypass switches you will be prompted for the password to the key store and the password to the key in the key store. Similarly, omitting the -dname switch will prompt you for all the parts required to make up an X.500 distinguished name.

C:\Cooper\com.blong.Torch>keytool -genkey -v -keystore obj\Java\Debug\certificate.keystore -alias androiddebugkey -keyalg RSA -keysize 2048 -storepass store_password -keypass key_password -dname "CN=Brian Long,O=blong,C=UK" -validity 10000

Generating 2,048 bit RSA key pair and self-signed certificate (SHA1withRSA) with
 a validity of 10,000 days
        for: CN=Brian Long, O=blong, C=UK
[Storing obj\Java\Debug\certificate.keystore]

The idea is just to create a keystore once and then use that to sign your applications, making them all consistently signed.

Step 6: Signing your Android package

Android packages are signed using the Java jarsigner.exe utility. You feed it the unsigned package, a keystore and a keystore alias and it will output a signed package. If you don't specify the signed package name it will assume the same name as the unsigned package and overwrite it. The keystore password and key password can be passed on the command-line, or if not you will be prompted for them.

set FINAL=bin

set PKG_SIGNED=%FINAL%\%BUILD%\%PROJ_NAME%.apk

jarsigner -verbose -keystore %KEY_STORE% -storepass %STORE_PWD% -keypass %KEY_PWD% -digestalg SHA1 -signedjar %PKG_SIGNED% %PKG_UNSIGNED% %KEY_ALIAS%

Update: the -digestalg SHA1 parameter is there to keep things working if you have JDK 7 installed. JDK 6's jarsigner defaults to SHA1 for its digest algorithm, which is what Android wants, but JDK 7's jarsigner defaults to SHA-256. So specifying the right one is a good idea.

Running the command (as usual verbose output has been requested) gives this:

C:\Cooper\com.blong.Torch>jarsigner -verbose -keystore obj\Java\Debug\certificate.keystore -storepass store_password -keypass key_password -digestalg SHA1 -signedjar bin\Debug\com.blong.Torch.apk obj\Java\Debug\com.blong.Torch_unsigned.apk androiddebugkey

   adding: META-INF/MANIFEST.MF
   adding: META-INF/ANDROIDD.SF
   adding: META-INF/ANDROIDD.RSA
  signing: res/layout/main.xml
  signing: AndroidManifest.xml
  signing: resources.arsc
  signing: res/drawable-hdpi/icon.png
  signing: res/drawable-ldpi/icon.png
  signing: res/drawable-mdpi/icon.png
  signing: classes.dex

After all that we now have a self-signed Android application package in bin\Debug\com.blong.Torch.apk

Step 7: Optimising the package

For release builds there is one extra step you ought to make. To optimise memory use and performance you should run the signed package through the zipalign.exe tool, which aligns uncompressed data on suitable byte boundaries (4 is the value you should always use):

set PKG_SIGNED=%TEMP%\%BUILD%\%PROJ_NAME%.apk

set FINAL=bin

set PKG_SIGNED_ALIGNED=%FINAL%\%BUILD%\%PROJ_NAME%.apk

zipalign -f -v 4 %PKG_SIGNED% %PKG_SIGNED_ALIGNED%

Step 8: Installing the package on an Android device

Having built an Android application it now needs to be deployed. This can be done using adb (the Android Debug Bridge), which has a multitude of capabilities. The simplest installation command-line would be:

adb install %PKG_SIGNED_ALIGNED%

This attempts to install the specified package file onto the Android device connected to the machine, be it a phone or an emulator. Given that you often rebuild apps and want to reinstall them, a better option might be:

adb install -r %PKG_SIGNED_ALIGNED%

This reinstalls the application, keeping any data intact from the previous version. However it is common to have an emulator running and also have a real device connected to the computer. You can use -d or -e to distinguish between a device or an emulator:

adb -d install -r %PKG_SIGNED_ALIGNED%

This works fine until you have, say, several emulators running. To pick a specific Android target you can specify its serial number with -s. For example to target a particular emulator you might use:

adb -s emulator-5554 install -r %PKG_SIGNED_ALIGNED%

You can get a list of all the attached devices, their serial numbers and indications of whether they are emulators or real devices using:

adb devices

Step 9: Launching the application

To start the application from the command-line you need to know the package name as defined in the app's Android manifest, and also the name of the main activity class. Given these two pieces of identification you can use this command to launch the app:

adb –d shell am start –a android.intent.action.MAIN -n package_name/class_name

This application I've been using with these commands has an AndroidManifest.xml file as follows:

<?xml version="1.0" encoding="utf-8"?>

<manifest xmlns:android="http://schemas.android.com/apk/res/android"

    package="com.blong.Torch">

  <application android:persistent="true"

      android:label="@string/app_name"

      android:icon="@drawable/icon">

    <activity android:label="@string/main_activity_name" android:name="com.blong.torch.MainActivity">

      <intent-filter>

        <action android:name="android.intent.action.MAIN" />

        <category android:name="android.intent.category.LAUNCHER" />

      </intent-filter>

    </activity>

    <activity android:label="@string/torch_activity_name" android:name="com.blong.torch.TorchActivity" />

  </application>

  <uses-sdk android:minSdkVersion="4" />

</manifest>

This states the package name is com.blong.Torch. You can also see two activities defined, one of which has an intent filter defined to respond to an intent defined to have an action android.intent.action.MAIN and a category android.intent.category.LAUNCHER. This is the launch activity for the app and the android:name element specifies its class name is com.blong.torch.MainActivity. These are the two pieces of information required so we can launch it from the command line with:

adb –d shell am start –a android.intent.action.MAIN -n com.blong.Torch/com.blong.torch.MainActivity

I'll explain this form of adb command in more detail another time, but for now you just need to know that it works.

Finally, for this summary of Android command-line possibilities, to uninstall an android app you again need to know the package name, as per the Android manifest file. To uninstall this sample Torch application use:

adb –d uninstall com.blong.Torch

There are plenty more possibilities for Android at the command-line. In another post I'll look at some other useful things we can do.

Portions of this page (well, just the build workflow image really) are reproduced from work created and shared by the Android Open Source Project and used according to terms described in the Creative Commons 2.5 Attribution License.

It's a blong, blong, blong road...

Sunday 5 June 2011

Android from the command line