tewdew software

The Android Texture Decision

An early decision faced by any Android OpenGL ES 2.0 developer is whether to use texture compression, and if so which formats to support.

The first question has but one legitimate answer: Yes, of course you should.

On devices with limited memory, and even more restricted memory bandwidth (usually with no dedicated video memory to share the load), compressed textures can significantly reduce the footprint of assets (in transit, storage and in memory), and can theoretically lead to improved fill-rates.

ETC1 compression yields textures that are 1/6th the size of the original, as does DXT compression (aka S3TC). PVRTC offers similar performance. The result is imperceptibly changed from the original.

Only in a couple of specific cases is it reasonable to skip compression. If you have a dynamic texture where you algorithmically generate or alter the texels, for instance, texture compression is not an option.

Alas, Google did developers a big disservice when they published a tutorial on game development and simply avoided the whole question (going with RGBA). That sort of deferred basic optimization is exactly what is wrong with many applications on Android. Over in iPhone land — helped by a simplified hardware target — PVRTC is basically the default decision.

So why do so many Android developers avoid the question? Why do some apps not even bother?

Having faced and answered this question, I have an inkling of an answer.

1. The official texture-compression format of OpenGL ES 2.0 is ETC1. A gaping-chasm in the functionality of which is the lack of support for an alpha channel. This is fine for traditionally wrapped 3D objects, but represents a problem when you’re essentially using OpenGL ES 2.0 as a 2D compositing system, as many (if not most) are.
2. The formats that support an alpha-channel are the uncompressed RGBA, which is incredibly wasteful, or the non-standard formats of PVRTC, ATITC, and S3TC (DXT).
   
   On Honeycomb tablets today you could technically settle on S3TC, at least for a couple of months. Then the OMAP 4 tablets, lacking support for S3TC (but adding support for PVRTC), start hitting shelves and your app will be horribly broken, as will many in the current “assume Android tablets run the Tegra2” era.
3. You could support all three of the non-standard formats to cover all bases, however then you need to segregate assets by device (and accommodate the extra complexity in your tool-chain and bundling process), or redundantly pull all formats on all devices, wasting transfer and storage capacity in the process. It adds complexity to your project.

If ETC1 had an alpha channel and didn’t represent a significant compromise over the other formats, it would be the only choice.

What do I mean when I mention the possibility of a significant compromise? I am referring to the purported advantage (one that I have never seen demonstrated or benchmarked) of using the “native” texture compression format on these devices.

That a PowerVR GPU will do best with PVRTC, the Tegra2 will shine with S3TC, and the Adreno-GPU powered Snapdragon prefers ATITC. There is a reasonable logic to such a supposition.

After doing significant testing of my own, however, I can state that I’ve found a negligible performance difference between S3TC versus ETC1 on the Tegra2. This doesn’t necessarily hold true for different GPUs and devices, but nvidia seems to have done a great job of treating ETC1 as a first-class compression format, with little penalty while yielding the benefits of a cross-hardware format.

I went into the benchmark expecting to see a significant advantage for S3TC, so the results actually refuted my bias.

What of the lack of alpha functionality, however?

For that I drew inspiration from the post by erique under the Unity3D thread. In that thread the author suggests to use two textures for ETC1 textures where you need an alpha channel, mixing them in the fragment shader. I followed their advice and the performance barely changed — the second texture source coming at almost no cost. The alternative would be to use an RGBA for those specific textures, but the dual-ETC textures still beats that by 3x.

void main() {
    gl_FragColor.rgb = texture2D(sTexture1, vv2texCoord).rgb;
    gl_FragColor.a = texture2D(sTexture2, vv2texCoord).g;
}

Load an RGBA image in Gimp, pick Colors/Components/Decompose, pick Alpha, and you’ll have a perfect second image to run through ETC1 compression (the single best compression tool I’ve found is PVRTexTool by Imagination Technologies. Despite their seeming conflict of interest, it has proven excellent at compressing against all formats), used as specified above.

Clearly it adds to the memory bandwidth and the original download, but for those specific cases where I absolutely need an alpha channel it is a golden solution that makes for a single set of assets that work across all OpenGL ES 2.0 devices, present and in the future.

Alternately, where only a binary opacity was needed I could swap out true-black for no opacity in the fragment shader.

EDIT: I tested this theory, evaluating the pixel color and if black setting the opacity to 0.0. ala-

fragColor = texture2D(sTexture1, vv2texCoord);
if (fragColor.rgb == 0.0)
{
     fragColor.a = 0.0;
}

To my surprise this absolutely destroyed the performance, cutting the framerate to 1/3rd.

So at tewdew we’re throwing in with ETC1 for now, using a second compressed texture where an alpha channel is needed. This may change in the future if specific devices benefit from specialization, but for now I think this is the right approach.

Using JNI from a Native Activity

I’m a big fan of the new native activity functionality available for Android NDK users.

While it does limit your target audience to Android 2.3 and above (currently comprising less than 10% of the Android smartphone market, albeit covering 100% of the Android tablet market), it can represent a significant efficiency improvement for heavy users of the native development kit, removing most expensive JNI boundary crossings. Native activities empower and speed up native event processing.

I liberally use the NDK, and as it has been iteratively empowered by the Android developers I’ve found my motivation to build Android projects growing.

Despite years primarily dealing with higher languages and managed runtimes, I really love C++ and unmanaged code. Maybe it’s some early professional life trauma that has given me a C++ Stockholm syndrome.

The NDK importantly allows me to effectively share code across platforms, which is especially important given the state of the Android emulator: Aside from notoriously poor performance, the emulator is missing large swaths of functionality, including significant Open GL ES 2.0 functionality, ruling out its use for many graphically advanced applications.

So instead I build an application where most of the code is shared between an Android application NDK library, and a Win32 application. Conceivably an iOS port would not be that far removed, which was an important consideration.

On the Win32 front you just need to add an OpenGL ES 2.0 emulator to intercept and rewrite graphics calls and you have a world where you can build out most of your core functionality in Visual Studio, appropriately stubbing out platform specific code like mouse/touchscreen inputs, only occasionally building in the Android NDK and running it on an actual Android device just to make sure everything still works.

It is a solution that has worked wonders for my enjoyment of non-trivial Android development.

However the NDK remains incomplete. While each iteration has seen a much richer, more self-sustaining NDK, there are certain tasks that are better done, if not only possible from, Java in the Dalvik VM.

So you need to make a JNI call from the Native Activity to Java code, whether to the system class library, or to your own custom code.

The former is easy: From the ANativeActivity instance (the definition can be found in native_activity.h), which if you use the native app glue can be found in the activity member of the android_app structure you’re passed, use the vm member to obtain a thread-attached JNIEnv instance.

JNIEnv *jni;

state->activity->vm->AttachCurrentThread(&jni, NULL);

Attaching the thread is necessary given that — again if you use the native glue — your code is running in a separate thread.

Now that you have the JNIEnv you can use the normal FindClass functionality to get system classes, and normal JNI functionality to instantiate and call Java functionality.

Things get a little dicier when you want to use the class loader to load your own classes, however, which will often be the case given that normally you’ll want to limit to jumps between the NDK and the SDK, with entrypoints that do sets of functionality and then return finalized results after lots of chatter between Java objects. It would be a serious pain to do all of that from the NDK.

FindClass off of the JNIEnv will not give satisfaction. It won’t find your classes. The reasons are because the context of the JNIEnv FindClass has no awareness that your classes exist.

This seriously vexed me last night. It is one of those frustrating problems that I’m sure have a simple answer, but many searches offered no clues. I found one guy who had the same problem, apparently finding a solution, however they left the question open with an obtuse “found my solution” claim.

I believe that people in such situations often are so irritated by the lack of assistance that they decide to do the same in return to the collective. Alternately if the answer isn’t easily found, I think some are motivated by the perception that it’s a competitive advantage for it to remain so.

In the end it turned out to be painfully obvious. You need to call the getClassLoader method of the android.app.NativeActivity instance that you were given in the clazz member of the activity member previously discussed. This will give you a class loader that has contextual awareness of your own custom code.

In pseudo-code-

JNIEnv *jni;

state->vm->AttachCurrentThread(&jni, NULL);

jclass activityClass = jni->FindClass("android/app/NativeActivity");

jmethodID getClassLoader = jni->GetMethodID(activityClass,"getClassLoader", "()Ljava/lang/ClassLoader;");

jobject cls = jni->CallObjectMethod(state->activity->clazz, getClassLoader);

jclass classLoader = jni->FindClass("java/lang/ClassLoader");

jmethodID findClass = jni->GetMethodID(classLoader, "loadClass", "(Ljava/lang/String;)Ljava/lang/Class;");

jstring strClassName = jni->NewStringUTF("com/tewdew/ClassIWant");

jclass classIWant = (jclass)jni->CallObjectMethod(cls, findClass, strClassName);

Add in error handling, free the string and detach the thread, and you have a class loader that empowers your NDK code.

tl;dr;  — you need to get the class loader from your native activity. The default class loader you get otherwise is effectively just a system class loader.

I hope this proves helpful for future searchers.

Welcome to the tewdew software blog

The experiment begins.

I’m Dennis Forbes and this is the blog of tewdew software.

We’re a “startup” focused on polished, innovative applications for the tablet and smartphone markets.

I use the plural “we’re” not to pretend that there’s some big team of developers, but rather that this enterprise is a combined effort of my wife, my four children (my sons aged 0.8, 4, 6, and my eight year old daughter), and myself.

The programming department consists of me, part time. I have a successful career that I enjoy and find very rewarding, so this is generally a late night enterprise.

tewdew is an amalgamation of the first initial of the six members of my household, and is pronounced “to do”. It is a subtle reminder that this is something I need to do.

While I’m a big fan of first person shooters, I want tewdew software to be a creator of software that allows creative outlet for both adults and children, while enlightening and educating. Where the experience is enjoyable.

I’ll let other companies fill the shooter space, and I’ll undoubtedly partake of their wares. But we have a wholesome little niche we’ve chosen for ours.

Ideally this turns out to be a very lucrative adventure. I’m already making plans for that large country estate we’ll move to with all of the extra income this will yield.

In reality I’ll be pleased just to give my children the experience of contributing in the creation of software products, learning all of the stages of the process: from white-boarding to rational planning to market metrics.

This will also provide justification for buying all of the latest mobile and tablet gadgets. Those upcoming OMAP4460 and Tegra 3 tablets look enticing.

Our first product will be available on the Android market in the coming weeks, targeting Honeycomb tablets. The Android tablet market was chosen because it is so under-served right now. We look to fill some of that void, while understanding that there’s a risk that the platform might not take off and the market base simply won’t be there.

Later we will port it to iOS (geek speak: it’s worth noting that I generally develop using C/C++, targeting the Android NDK in a platform agnostic fashion. Most of my development and testing is actually built against an OpenGL ES 2.0 Win32 application, and the Android build is just a separate target with minor changes. The same porting benefits hold for iOS, aside from some edge custom code to deal with platform nuances such as touchscreen inputs).

I look forward to the adventure, and hope that the products we bring to the market meet our very lofty expectations.

To great things!