Thursday 18 July 2013

UnityPlayer as a Subview with Transparent Background (in Android)

Note: for iOS you can check this post
 
In the past couple of days I was trying and add 3D scene created with Unity into an existing Android app where most views are made in native android and make the camera background transparent.

After searching I found these posts in Unity forum which were helpful in finding most the info I needed:
http://forum.unity3d.com/threads/98315-Using-Unity-Android-In-a-Sub-View
http://forum.unity3d.com/threads/80405-Android-Augmented-Reality-Question

I tried to put everything together and ended up with the following steps for future reference  (using Unity 4.1.5):

1. Create Android Unity Project

The first step is to create a Unity scene (if you don't have a project already)
in this case I had a simple cube spinning around its Y axis, I also set the camera background color RGBA to 0,0,0,0 (alpha 0).



2. Creating eclipse project from Unity Project

Having not done much of Android native development (most my work is with iOS) it was a little challenging at first to get my head around back into eclipse but thankfully Unity 4's option to export to eclipse saved me a lot of time: simply tick create eclipse project layout in the build settings of then export.
Import the exported project inside eclipse then go to properties->Android and tick "Is Library" check box inside Library section then apply.

3. Create new Android project (or import an existing one) which has at least one view.

Once you have your project do the following 5 steps:

     a. Go to properties-> Android and inside the Library section click Add and select Unity generated project

     b. Go to properties->Java Build Path->Add External JARs... and select classes.jar from inside the generated eclipse project from Unity

     c. Copy the contents of assets folder inside Unity Eclipse project (bin & libs folders) to your targeted Android project assets folder

     d. Modify AndroidManifest.xml by copying the activities and any features from the AndroidManifest.xml that is found inside Unity Eclipse project  (make sure to remove the intent filter for the Main launcher) . The following is an example of how the end result should look like:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    installLocation="auto"
    package="com.example.testnative"
    android:versionCode="1"
    android:versionName="1.0" >

    <uses-feature android:glEsVersion="0x00020000" />

    <supports-screens
        android:largeScreens="true"
        android:normalScreens="true"
        android:smallScreens="false"
        android:xlargeScreens="false" />

    <uses-sdk
        android:minSdkVersion="8"
        android:targetSdkVersion="15" />

    <application
        android:debuggable="false"
        android:icon="@drawable/app_icon"
        android:label="@string/app_name" >
        <activity
            android:name="com.example.testnative.FullscreenActivity"
            android:configChanges="orientation|keyboardHidden|screenSize"
            android:label="@string/app_name"
            android:theme="@style/FullscreenTheme" >
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
   
        <activity
            android:name="com.unity3d.player.UnityPlayerProxyActivity"
            android:configChanges="fontScale|keyboard|keyboardHidden|locale|mnc|mcc|navigation|orientation|screenLayout|screenSize|smallestScreenSize|uiMode|touchscreen"
            android:label="@string/app_name"
            android:launchMode="singleTask"
            android:screenOrientation="portrait" >
        </activity>
        <activity
            android:name="com.unity3d.player.UnityPlayerActivity"
            android:configChanges="fontScale|keyboard|keyboardHidden|locale|mnc|mcc|navigation|orientation|screenLayout|screenSize|smallestScreenSize|uiMode|touchscreen"
            android:label="@string/app_name"
            android:launchMode="singleTask"
            android:screenOrientation="portrait" >
        </activity>
        <activity
            android:name="com.unity3d.player.UnityPlayerNativeActivity"
            android:configChanges="fontScale|keyboard|keyboardHidden|locale|mnc|mcc|navigation|orientation|screenLayout|screenSize|smallestScreenSize|uiMode|touchscreen"
            android:label="@string/app_name"
            android:launchMode="singleTask"
            android:screenOrientation="portrait" >
            <meta-data
                android:name="android.app.lib_name"
                android:value="unity" />
            <meta-data
                android:name="unityplayer.ForwardNativeEventsToDalvik"
                android:value="false" />
        </activity>
    </application>

    <uses-feature android:name="android.hardware.touchscreen" />
    <uses-feature
        android:name="android.hardware.touchscreen.multitouch"
        android:required="false" />
    <uses-feature
        android:name="android.hardware.touchscreen.multitouch.distinct"
        android:required="false" />

    <uses-permission android:name="android.permission.WAKE_LOCK" />

</manifest> 

    e. Inside the main layout file (or any view you want to use) add a FrameLayout and give it an id of your choice (in my case its "UnityView").

4. Basic Usage

With bit of luck now you should have everything setup and you can render your Unity project to the frame layout you've created as follows:
        
// you need to declare m_UnityPlayer as variable of type UnityPlayer obviously... 
// also include any missing references by pressing shift+ctrl+o (or cmd+shift+o for Mac)
 m_UnityPlayer = new UnityPlayer(this);
 int glesMode = m_UnityPlayer.getSettings().getInt("gles_mode", 1);
 m_UnityPlayer.init(glesMode, false);
 FrameLayout layout = (FrameLayout) findViewById(R.id.UnityView);
 LayoutParams lp = new LayoutParams(LayoutParams.MATCH_PARENT,
 LayoutParams.MATCH_PARENT);
 layout.addView(m_UnityPlayer.getView(), 0, lp);

5. Add Transparent Background (not working in Unity 4.3 and above since Unity changed the implementation for UnityPlayer)

For the transparent background, as mentioned in the second post referenced above you will have to extend UnityPlayer class and implement GLSurfaceView renderer. This wasn't straight forward and it required some trial and error but this worked for me at the end by replacing setZOrderMediaOverlay(true) with setZOrderOnTop(true)
 
    // the class extending unity player
    class CustomUnityPlayer extends UnityPlayer implements GLSurfaceView.Renderer {
      public CustomUnityPlayer(ContextWrapper context) {
          super(context);
      }
      public void onDrawFrame(GL10 gl) {
          super.onDrawFrame(gl);
      }
    }
     
    // inside OnCreate function:
    m_UnityPlayer = new CustomUnityPlayer(this);
    int glesMode = m_UnityPlayer.getSettings().getInt("gles_mode", 1);
    m_UnityPlayer.init(glesMode, false);
    
    //Create GLSurfaceView with transparent background 
    mUnityView = new GLSurfaceView(getApplication());
    mUnityView.setEGLContextClientVersion(2);
    mUnityView.setZOrderOnTop(true);
    mUnityView.setEGLConfigChooser(8, 8, 8, 8, 16, 0);
    mUnityView.getHolder().setFormat(PixelFormat.TRANSLUCENT);
    mUnityView.setRenderer(m_UnityPlayer);
     
    FrameLayout layout = (FrameLayout) findViewById(R.id.UnityView);
    LayoutParams lp = new LayoutParams(LayoutParams.MATCH_PARENT,
    LayoutParams.MATCH_PARENT);
    layout.addView(mUnityView, 0, lp); 

Once you made this step you will notice that UnityPlayer is not receiving Input events such as touch events any more which means that you will need to pass it yourself depending own your requirements. Also I noticed that when using a shader with color such as Diffuse the objects render with semi transparency while it works well when using mobile diffuse so there might be somethings I missed in this approach

This was my first experiment of this kind with Android and Unity & I hope it will help someone... 
My next step is to do the same thing but for iOS once I have the time.

==UPDATE==
If the Android project you are trying to integrate into uses maven I wrote a little post that might be helpful

==UPDATE 2==
I finally was able to post about my attempt with iOS