Thursday, 18 July 2013

UnityPlayer as a Subview with Transparent Background (in Android)

Note: for iOS you can check this post
 
In the past couple of days I was trying and add 3D scene created with Unity into an existing Android app where most views are made in native android and make the camera background transparent.

After searching I found these posts in Unity forum which were helpful in finding most the info I needed:
http://forum.unity3d.com/threads/98315-Using-Unity-Android-In-a-Sub-View
http://forum.unity3d.com/threads/80405-Android-Augmented-Reality-Question

I tried to put everything together and ended up with the following steps for future reference  (using Unity 4.1.5):

1. Create Android Unity Project

The first step is to create a Unity scene (if you don't have a project already)
in this case I had a simple cube spinning around its Y axis, I also set the camera background color RGBA to 0,0,0,0 (alpha 0).



2. Creating eclipse project from Unity Project

Having not done much of Android native development (most my work is with iOS) it was a little challenging at first to get my head around back into eclipse but thankfully Unity 4's option to export to eclipse saved me a lot of time: simply tick create eclipse project layout in the build settings of then export.
Import the exported project inside eclipse then go to properties->Android and tick "Is Library" check box inside Library section then apply.

3. Create new Android project (or import an existing one) which has at least one view.

Once you have your project do the following 5 steps:

     a. Go to properties-> Android and inside the Library section click Add and select Unity generated project

     b. Go to properties->Java Build Path->Add External JARs... and select classes.jar from inside the generated eclipse project from Unity

     c. Copy the contents of assets folder inside Unity Eclipse project (bin & libs folders) to your targeted Android project assets folder

     d. Modify AndroidManifest.xml by copying the activities and any features from the AndroidManifest.xml that is found inside Unity Eclipse project  (make sure to remove the intent filter for the Main launcher) . The following is an example of how the end result should look like:

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
    installLocation="auto"
    package="com.example.testnative"
    android:versionCode="1"
    android:versionName="1.0" >

    <uses-feature android:glEsVersion="0x00020000" />

    <supports-screens
        android:largeScreens="true"
        android:normalScreens="true"
        android:smallScreens="false"
        android:xlargeScreens="false" />

    <uses-sdk
        android:minSdkVersion="8"
        android:targetSdkVersion="15" />

    <application
        android:debuggable="false"
        android:icon="@drawable/app_icon"
        android:label="@string/app_name" >
        <activity
            android:name="com.example.testnative.FullscreenActivity"
            android:configChanges="orientation|keyboardHidden|screenSize"
            android:label="@string/app_name"
            android:theme="@style/FullscreenTheme" >
            <intent-filter>
                <action android:name="android.intent.action.MAIN" />

                <category android:name="android.intent.category.LAUNCHER" />
            </intent-filter>
        </activity>
   
        <activity
            android:name="com.unity3d.player.UnityPlayerProxyActivity"
            android:configChanges="fontScale|keyboard|keyboardHidden|locale|mnc|mcc|navigation|orientation|screenLayout|screenSize|smallestScreenSize|uiMode|touchscreen"
            android:label="@string/app_name"
            android:launchMode="singleTask"
            android:screenOrientation="portrait" >
        </activity>
        <activity
            android:name="com.unity3d.player.UnityPlayerActivity"
            android:configChanges="fontScale|keyboard|keyboardHidden|locale|mnc|mcc|navigation|orientation|screenLayout|screenSize|smallestScreenSize|uiMode|touchscreen"
            android:label="@string/app_name"
            android:launchMode="singleTask"
            android:screenOrientation="portrait" >
        </activity>
        <activity
            android:name="com.unity3d.player.UnityPlayerNativeActivity"
            android:configChanges="fontScale|keyboard|keyboardHidden|locale|mnc|mcc|navigation|orientation|screenLayout|screenSize|smallestScreenSize|uiMode|touchscreen"
            android:label="@string/app_name"
            android:launchMode="singleTask"
            android:screenOrientation="portrait" >
            <meta-data
                android:name="android.app.lib_name"
                android:value="unity" />
            <meta-data
                android:name="unityplayer.ForwardNativeEventsToDalvik"
                android:value="false" />
        </activity>
    </application>

    <uses-feature android:name="android.hardware.touchscreen" />
    <uses-feature
        android:name="android.hardware.touchscreen.multitouch"
        android:required="false" />
    <uses-feature
        android:name="android.hardware.touchscreen.multitouch.distinct"
        android:required="false" />

    <uses-permission android:name="android.permission.WAKE_LOCK" />

</manifest> 

    e. Inside the main layout file (or any view you want to use) add a FrameLayout and give it an id of your choice (in my case its "UnityView").

4. Basic Usage

With bit of luck now you should have everything setup and you can render your Unity project to the frame layout you've created as follows:
        
// you need to declare m_UnityPlayer as variable of type UnityPlayer obviously... 
// also include any missing references by pressing shift+ctrl+o (or cmd+shift+o for Mac)
 m_UnityPlayer = new UnityPlayer(this);
 int glesMode = m_UnityPlayer.getSettings().getInt("gles_mode", 1);
 m_UnityPlayer.init(glesMode, false);
 FrameLayout layout = (FrameLayout) findViewById(R.id.UnityView);
 LayoutParams lp = new LayoutParams(LayoutParams.MATCH_PARENT,
 LayoutParams.MATCH_PARENT);
 layout.addView(m_UnityPlayer.getView(), 0, lp);

5. Add Transparent Background (not working in Unity 4.3 and above since Unity changed the implementation for UnityPlayer)

For the transparent background, as mentioned in the second post referenced above you will have to extend UnityPlayer class and implement GLSurfaceView renderer. This wasn't straight forward and it required some trial and error but this worked for me at the end by replacing setZOrderMediaOverlay(true) with setZOrderOnTop(true)
 
    // the class extending unity player
    class CustomUnityPlayer extends UnityPlayer implements GLSurfaceView.Renderer {
      public CustomUnityPlayer(ContextWrapper context) {
          super(context);
      }
      public void onDrawFrame(GL10 gl) {
          super.onDrawFrame(gl);
      }
    }
     
    // inside OnCreate function:
    m_UnityPlayer = new CustomUnityPlayer(this);
    int glesMode = m_UnityPlayer.getSettings().getInt("gles_mode", 1);
    m_UnityPlayer.init(glesMode, false);
    
    //Create GLSurfaceView with transparent background 
    mUnityView = new GLSurfaceView(getApplication());
    mUnityView.setEGLContextClientVersion(2);
    mUnityView.setZOrderOnTop(true);
    mUnityView.setEGLConfigChooser(8, 8, 8, 8, 16, 0);
    mUnityView.getHolder().setFormat(PixelFormat.TRANSLUCENT);
    mUnityView.setRenderer(m_UnityPlayer);
     
    FrameLayout layout = (FrameLayout) findViewById(R.id.UnityView);
    LayoutParams lp = new LayoutParams(LayoutParams.MATCH_PARENT,
    LayoutParams.MATCH_PARENT);
    layout.addView(mUnityView, 0, lp); 

Once you made this step you will notice that UnityPlayer is not receiving Input events such as touch events any more which means that you will need to pass it yourself depending own your requirements. Also I noticed that when using a shader with color such as Diffuse the objects render with semi transparency while it works well when using mobile diffuse so there might be somethings I missed in this approach

This was my first experiment of this kind with Android and Unity & I hope it will help someone... 
My next step is to do the same thing but for iOS once I have the time.

==UPDATE==
If the Android project you are trying to integrate into uses maven I wrote a little post that might be helpful

==UPDATE 2==
I finally was able to post about my attempt with iOS

20 comments:

  1. It stops and doesn't work on my Sharp Aquos 102SH phone.
    My Eclipse project for this for your debugging is in https://skydrive.live.com/redir?resid=5646F440E18EDD57!644&authkey=!AFwNiF2TAS76pCs

    Please email me at bjonreyes@gmail.com if you have fixed this Eclipse project.

    ReplyDelete
    Replies
    1. Hello Bj,

      I will try to have a look on your project when I have the time

      Delete
    2. I had a quick look and everything seemed fine... I managed to use a genereted unity project with your unityview project..

      what is exactly your error?
      couple of things:
      Did you follow the exact steps? I noticed that you copied the .so files inside the libs and you hadn't copied the assets folder contents across to UnityView assets folder

      Also in Unity 4.2 the generated project has package name with the same name as unity player library make sure to change the package to something else

      Delete
    3. This comment has been removed by the author.

      Delete
    4. Ok. It now works on my Sharp Aquos 102SH phone and sometimes works on my rooted Gadmei T883. I also made a jar library for Unity using your code. I'm on to making a post-processing shader for Unity on its Android jar library code. Thanks. Also, what if the cube has black, white, all the shades and all the colors?

      Delete
    5. Hello bj,

      I haven't tried colorful cube on Android but it should work normally

      Delete
  2. Hi,

    Thanks for the tutorial, it was very helpful.

    Have you had any luck doing the same thing on iOS?

    ReplyDelete
    Replies
    1. Hi Daan,

      Thanks for that.

      On iOS I managed to do something similar but for a full screen mode. (Unity view is overlaying the full screen but with transparent background) but its slightly messier workflow.

      I was planning to make a post once I have the time.

      Delete
    2. I was having troubles myself with getting the unity view transparent, but finally managed to find a solution for it.

      I'm adding my solution here for other people who are having the same problem:

      I added the following lines to the method
      - (void)recreateSurface:(BOOL)use32bitColor use24bitDepth:(BOOL)use24bitDepth msaaSampleCount:(int)msaaSampleCount renderW:(int)renderW renderH:(int)renderH :
      (in DisplayManager.mm, in the ios project that was generated by unity):

      surface.layer.opaque = NO;

      surface.layer.drawableProperties = [NSDictionary dictionaryWithObjectsAndKeys:

      [NSNumber numberWithBool:FALSE], kEAGLDrawablePropertyRetainedBacking,

      kEAGLColorFormatRGBA8, kEAGLDrawablePropertyColorFormat,

      nil

      ];

      (camera background should be set to (0,0,0,0) just as in Android)

      Delete
    3. Yes thats exactly how I did it as well

      Delete
    4. But then I simply set the surface->layer.opaque = NO; inside CreateSystemRenderingSurface function (GlesHelper.mm)

      Delete
  3. Yaser,

    "surface->layer.opaque = NO;"

    Is this the solution for IOS ?

    ReplyDelete
    Replies
    1. Fantasia, This is to make Unity's layer with transparent background.

      Delete
    2. I've just wrote a post about my attempt with iOS

      Delete
  4. Any idea how to do this using Unity 4.5? So far I haven't found any workaround for this...

    ReplyDelete
  5. http://forum.unity3d.com/threads/rendering-to-custom-java-surfaces-exposed-in-unity-as-additional-displays.318396/

    ReplyDelete
  6. Any idea about do the same on unity 5.1.1f

    ReplyDelete
  7. I want to do exact same thing in unity 5.2 can anyone help please? Problem is they do not use GLsurface anymore.

    ReplyDelete
  8. Any idea about do the same on unity 5.6.1f1 ?
    Thank you!

    ReplyDelete