User Account Management In XR

By Kuldeep Singh

April 26, 2020 - 10 minutes read - 2064 words

In the previous article, we described the importance of interoperability in while building Enterprise XR solutions, In the article, we will discuss how to manage user accounts in the XR device, and implement single sign-on across the XR apps. We will use Android’s AccountManger approach to login into active directory and integrated it into Unity. Please read my earlier article on setting up the app in active directory, log in using android webview, considering a web-based company login form should appear for login, and no custom login form is allowed to capture the credentials.

Implement Android Library with custom AccountAuthenticator

We will continue from the last section of the previous article, which has WebViewActivity which launch webview for logon.

Let’s add a method to launch a webview activity.

public class WebViewActivity extends AppCompatActivity {
...
private WebView webView;

public static void launchActivity(Activity activity, String url, String redirectURI, IResponseCallBack callBack){
    Intent intent = new Intent(activity, WebViewActivity.class);
    intent.putExtra(INTENT_INPUT_INTERCEPT_SCHEME, redirectURI);
    intent.putExtra(INTENT_INPUT_URL, url);
    intent.putExtra(INTENT_RESPONSE_CALLBACK, new CustomReceiver(callBack));
    activity.startActivity(intent);
  }
private static class CustomReceiver extends ResultReceiver{
 ...
  @Override
  protected void onReceiveResult(int resultCode, Bundle resultData) { 
     String code = resultData.getString(INTENT_RESULT_PARAM_NAME);
     callBack.OnSuccess(code);
  }
}

Create binding service for custom Account Type

Create a custom authenticator, extending AbstractAccountAuthenticator, implement all abstract methods as follows.

public class AccountAuthenticator extends AbstractAccountAuthenticator {

  public AccountAuthenticator(Context context){
    super(context);
  }
 
  @Override
  public Bundle addAccount(AccountAuthenticatorResponse accountAuthenticatorResponse, String s, String s1, String[] strings, Bundle bundle) throws NetworkErrorException {
    // do all the validations, call AccountAuthenticatorResponse for   
    //any failures.
    return bundle;
  }
 ...
  @Override
  public Bundle getAuthToken(AccountAuthenticatorResponse accountAuthenticatorResponse, Account account, String s, Bundle bundle) throws NetworkErrorException {
    return bundle;
  }
  ...
}

Create a service to bind the custom authenticator

public class MyAuthenticatorService extends Service {
  @Override
  public IBinder onBind(Intent intent) {
    AccountAuthenticator authenticator = new AccountAuthenticator(this);
    return authenticator.getIBinder();
  }
}

Add the service in Android Manifest in the application tag

<service android:name="package.MyAuthenticatorService">
    <intent-filter>
      <action android:name="android.accounts.AccountAuthenticator" />
    </intent-filter>
    <meta-data android:name="android.accounts.AccountAuthenticator"
      android:resource="@xml/authenticator" />
  </service>

It refers to an XML file that defines the type of account the authenticator service will bind.

<?xml version="1.0" encoding="utf-8"?>
<account-authenticator xmlns:android="http://schemas.android.com/apk/res/android"
         android:accountType="com.tw.user"
         android:label="MyAccount"
         android:icon="@drawable/ic_launcher"
         android:smallIcon="@drawable/ic_launcher" />

This adds support for account type “com.tw.user”. The app which includes this library will intercept the request for this type.

Manage Accounts of Custom Type

Let’s add class to manage user accounts.

Now let’s add methods to this class, to manage account

Add User Account — This method adds an account to android AccountManager. Since we are trying to add an account of type “com.tw.user”, binder service will call our AccountAuthenticator.addAccount. We may do data validation there, however, we are currently returning the same bundle there. After this, it will call AccountManagerCallback which we have provided in the addAccount call below.

public static void addAccount(Context context, String bundleData, final IResponseCallBack callback) {
  final AccountManager accountManager = AccountManager.get(context);
  Bundle bundle = new Bundle();

  try {
    JSONObject jsonObject = new JSONObject(bundleData);
    bundle.putString(KEY_ACCOUNT_TYPE, ACCOUNT_TYPE);
    bundle.putString(KEY_ACCOUNT_NAME, "MyAccount");
   ... //prepare bundle

   accountManager.addAccount(ACCOUNT_TYPE, authTokenType, null,
      bundle,
      (Activity) context,
      new AccountManagerCallback<Bundle>() {
        @Override
        public void run(AccountManagerFuture<Bundle> future) {
          Bundle udata = null;
          try {
            udata = future != null ? future.getResult() : null;
            if (udata != null) {
              String accountName =udata.getString(KEY_ACCOUNT_NAME);
              String accountType= udata.getString(KEY_ACCOUNT_TYPE);
              Account account= new Account(accountName,accountType);
              boolean result= accountManager.addAccountExplicitly(             
                    account, null, udata);
              if(result){
               callback.OnSuccess("Account Added Successfully");
             } else {
               callback.OnFailure("Account can not be added");
             }
            } else {
              callback.OnFailure("Error! No user added");
            }
          } catch (Exception e) {
            callback.OnFailure("Error! " + e.getMessage());
          }
        }
      }, new Handler(Looper.getMainLooper()));
  } catch (Exception e) {
    callback.OnFailure("Error!  " + e.getMessage());
  }

}

Remove User Account — This method removes an account from android AccountManager.

public static void removeAccount(Context context, final IResponseCallBack callnack) {
  final AccountManager accountManager = AccountManager.get(context);
  Account[] accounts = accountManager.getAccountsByType(ACCOUNT_TYPE);
  if(accounts.length > 0) {
    Account account = accounts[0];
    try {
      boolean result = accountManager.removeAccountExplicitly( 
           account);
      if(result){
        callnack.OnSuccess("Account removed successfully!");
      } else {
        callnack.OnFailure("Error! Could not remove account");
      }
    } catch (Exception e) {
      callnack.OnFailure("Error! +" + e.getMessage());
    }
  } else {
    callnack.OnFailure("Error! No account found");
  }
}

Get Login In User Account — This method fetches logged in user data.

public static void getLoggedInUser(Context context, final IResponseCallBack responseCallBack) {
  final AccountManager accountManager = AccountManager.get(context);
  Account[] accounts = accountManager.getAccountsByType(ACCOUNT_TYPE);
  try {
    if(accounts.length > 0) {
      Account account = accounts[0];
      final JSONObject jsonObject = new JSONObject();
      jsonObject.put(KEY_USER_NAME, accountManager.getUserData(account, KEY_USER_NAME));
     ... prepare the response.
      responseCallBack.OnSuccess(jsonObject.toString());
    } else {
      responseCallBack.OnFailure("Error! No logged user of the type" + ACCOUNT_TYPE);
    }
  } catch (Exception e) {
    responseCallBack.OnFailure("Error!" + e);
  }
}

Source code for android library can be found at https://github.com/thinkuldeep/xr-user-authentication

Unity Authenticator App

Let’s customize the unity app that we have created in the last article, as follows, build the android library and include it in the unity project.

fddf

Add methods to ButtonController.cs

On clicking Login button, the following logic opens the WebViewActivity implemented in the Android library and load the provided URL.

private void OnLoginClicked()
{
  AndroidJavaClass WebViewActivity = new AndroidJavaClass("package.WebViewActivity");
  _statusLabel = "";
  AndroidJavaObject context = getUnityContext();
  WebViewActivity.CallStatic("launchActivity", context, authUri, redirectUri,
    new LoginResponseCallback( this));
}

make sure of the redirectURI. The webview will intercept this redirectURI and return code, which we use for getting the access_token.

private class LoginResponseCallback : AndroidJavaProxy
{
  private ButtonController _controller;
  public LoginResponseCallback(ButtonController controller) : base("package.IResponseCallBack")
  {
    _controller = controller;
  }
  public void OnSuccess(string result)
  {
    _statusLabel = "Code received";
    _statusColor = Color.black;
    _controller.startCoroutineGetToken(result);
  }
  public void OnFailure(string errorMessage)
  {
    _statusLabel = errorMessage;
    _statusColor = Color.red;
  }
}

Once we receive the authorization code, we need to fetch access_token by the HTTP token URL.

void startCoroutineGetToken(string code)
{
  StartCoroutine(getToken(code));
}

IEnumerator getToken(string code)
{
  WWWForm form = new WWWForm();
  form.headers["Content-Type"] = "application/x-www-form-urlencoded";
  form.AddField("code", code);
  form.AddField("client_id", clientId);
  form.AddField("grant_type", "authorization_code");
  form.AddField("redirect_uri", redirectUri);
  var tokenRequest = UnityWebRequest.Post(tokenUri, form);

  yield return tokenRequest.SendWebRequest();
  if (tokenRequest.isNetworkError || tokenRequest.isHttpError)
  {
    _statusLabel = "Login failed - " + tokenRequest.error;
    _statusColor = Color.red;
  }
  else
  {
    var response = JsonUtility.FromJson<TokenResponse>(tokenRequest.downloadHandler.text);
    _statusLabel = "Login successful! ";
    _statusColor = Color.black;
     TokenUserData userData = decodeToken(response.access_token);
     AddAccount(userData, tokenResponse);
  }
}

Once we get the access_token, we can decode access_token get some user details from it.

private class TokenUserData
{
  public string name;
  public string given_name;
  public string family_name;
  public string subscriptionId;
}
private TokenUserData decodeToken(String token) {
  AndroidJavaClass Base64 = new AndroidJavaClass("java.util.Base64");
  AndroidJavaObject decoder = Base64.CallStatic<AndroidJavaObject>("getUrlDecoder");
  var splitString = token.Split('.');
  var base64EncodedBody = splitString[1];

  string body = System.Text.Encoding.UTF8.GetString(decoder.Call<byte[]>("decode", base64EncodedBody));
  return JsonUtility.FromJson<TokenUserData>(body);
}

Once we have all the detail for the account, we call the add account to android accounts using the library APIs.

private void AddAccount(TokenUserData userData, TokenResponse tokenResponse)
{
  AndroidJavaClass AccountManager = new AndroidJavaClass("pakcage.UserAccountManager");

  UserAccountData accountData = new UserAccountData();
  accountData.userName = userData.given_name +" " + userData.family_name ;
  accountData.subscriptionId = userData.subscriptionId;
  accountData.tokenType = tokenResponse.token_type;
  accountData.authToken = tokenResponse.access_token;

  AccountManager.CallStatic("addAccount", getUnityContext(),   
   JsonUtility.ToJson(accountData), new AddAccountResponseCallback( 
   this));
}

Get Logged-in User

On Get LoggedIn User, call a respective method from the android library.

private void OnGetLoggedUserClicked()
{
  AndroidJavaClass AccountManager = new AndroidJavaClass("package.UserAccountManager");
  AccountManager.CallStatic("getLoggedInUser", getUnityContext(), new LoginResponseCallback(false, this));
}

Logout — Remove Account to Android Accounts

On logout, call remove the account.

private void OnLogoutClicked()
{
    AndroidJavaClass AccountManager = new AndroidJavaClass("poackage.UserAccountManager");
    AccountManager.CallStatic("removeAccount", getUnityContext(),   
   new LogoutResponseCallback());
}

Run and test

Lets now test all this. Export the unity project/run in the device. Try login, get logged in user, and logout API. In the next section, we will try to access the logged-in user outside of this authenticator app.

Once you logged into the XR Authenticator App and added an account in android accounts, it can be accessed in any other app using Android APIs. Let’s create another unity project on similar lines. AccountViewer XR App.

The cube is supposed to rotate only if a user is logged in the android system, and see the logged-in user details.

Get User Account from Android Accounts

It does not require any native plugin to be included. you can get the shared account directly via android APIs - accountManager.getAccountsByType(“com.tw.user”)

private UserAccountData getLoggedInAccount()
{
  UserAccountData userAccountData = null;
  var AccountManager = new 
       AndroidJavaClass("android.accounts.AccountManager");
  var accountManager = AccountManager.CallStatic<AndroidJavaObject>
             ("get", getUnityContext());
  var accounts = accountManager.Call<AndroidJavaObject>
            ("getAccountsByType", "com.tw.user");
  var accountArray = 
          AndroidJNIHelper.ConvertFromJNIArray<AndroidJavaObject[]>
         (accounts.GetRawObject());
  if (accountArray.Length > 0)
  {
    userAccountData = new UserAccountData();
    userAccountData.userName = accountManager.Call<string>
     ("getUserData", accountArray[0], "userName");
 ...
    userAccountData.tokenType = accountManager.Call<string>
     ("getUserData", accountArray[0], "tokenType");
    userAccountData.authToken = accountManager.Call<string    \.    
     ("getUserData", accountArray[0], "authToken");
  }
  return userAccountData;
}

Get profile using logged in user’s token

Read the token and get the profile data from the HTTP URL https://graph.microsoft.com/beta/me/profile/names with JWT token.

IEnumerator getProfile(UserAccountData data)
{
  UnityWebRequest request = UnityWebRequest.Get(getProfileUri);
  request.SetRequestHeader("Content-Type", "application/json");
  request.SetRequestHeader("Authorization", data.tokenType + " " + data.authToken);

  yield return request.SendWebRequest();
  if (request.isNetworkError || request.isHttpError)
  {
    _labels[indexGetProfile] = "Failed to get profile" + request.error;
    _colors[indexGetProfile] = Color.red;
  }
  else
  {
    var userProfile = JsonUtility.FromJson<UserProfileData>(request.downloadHandler.text);

    _labels[indexGetProfile] = "DisplayName: " + userProfile.displayName + "\nMail: " + userProfile.mail
+ "\nDepartment: " + userProfile.department
+ "\nCreated Date: " + userProfile.createdDateTime;
    _colors[indexGetProfile] = Color.black;
  }
}

Check if a user is logged

Even application focus, check if a user exists in android accounts. if the user exists then load the profile data.

private void OnApplicationFocus(bool hasFocus)
{
  if (hasFocus)
  {
    CheckIfUserIsLoggedIn();
  }
}

private void CheckIfUserIsLoggedIn()
{
  UserAccountData data = GetLoggedInAccount();
  if (data == null)
  {
    _isUserLoggedIn = false;
    _labels[indexWelcome] = "Welcome Guest!";
    _labels[indexGetProfile] = "";
  }
  else
  {
    _isUserLoggedIn = true;
    _labels[indexWelcome] = "Welcome " + data.userName + "!";
    StartCoroutine(getProfile(data));
  }
}

On update make sure you rotate the cube only if a user is logged in.

private void Update()
{
  if (_isUserLoggedIn)
  {
      cube.transform.Rotate(Time.deltaTime * 2 * _rotateAmount);
  }
  requestLoginButton.gameObject.SetActive(!_isUserLoggedIn);
  welcomeText.text = _labels[indexWelcome];
  getProfileResponse.text = _labels[indexGetProfile];
  getProfileResponse.color = _colors[indexGetProfile];
}

Run and test

Login on the Account Authenticator XR App, as in the last step. Open the Account Viewer XR App, you should see the Cube Rotating and Profile details of the logged-in user.

Let’s implement the request login from the external unity app

Add button for request login in the Account Viewer app. It will find an intent for the Account Authenticator app running in package com.tw.threed.authenticator. If the intent is available, means the XR authenticator app is installed in the device, then it will start the intent with a string parameter external_request.

private void OnRequestLoginClicked()
{
  AndroidJavaObject currentActivity = getUnityContext();
  AndroidJavaObject pm = currentActivity.Call<AndroidJavaObject>("getPackageManager");
  AndroidJavaObject intent = pm.Call<AndroidJavaObject>("getLaunchIntentForPackage", "com.tw.threed.authenticator");
  if (intent != null)
  {
    intent.Call<AndroidJavaObject>("putExtra","external_request", "true");
    currentActivity.Call("startActivity", intent);
    labels[requestlogin] = "Send Intent";
    colors[requestlogin] = Color.black;
  }
  else
  {
    labels[requestlogin] = "No Intent found";
    colors[requestlogin] = Color.red;
  }
}

There are multiple ways to receive externally initiated intent in Unity, but most of them need a native plugin to extend UnityPlayerActivity class. Here is another way, handle on focus event of any GameObject and access the current activity and check if intent has the external_request parameter. Call the OnLoginClicked action if it found the parameter.

private bool inProcessExternalLogin = false;
private void OnApplicationFocus(bool hasFocus)
{
  if (hasFocus)
  {
    AndroidJavaObject ca = getUnityContext();
    AndroidJavaObject intent = ca.Call<AndroidJavaObject>("getIntent");
    String text = intent.Call<String>("getStringExtra", "external_request");
    if (text != null && !inProcessExternalLogin)
    {
      _statusLabel = "Login action received externally";
      _statusColor = Color.black;
      inProcessExternalLogin = true;
      OnLoginClicked();
    }
  }
}

We need to quit the app after successfully adding the account, so it can go in. background.

private class AddAccountResponseCallback : AndroidJavaProxy
{
  ...
  void OnSuccess(string result)
  {
   _statusLabel = result;
  _statusColor = Color.black;
    if (_controller.inProcessExternalLogin)
    {
      Application.Quit();
    }
  }
  ...
}

Run and test

Open Account Viewer App and request login (it will be visible only if the user is not logged in)
It will open the Account Authenticator XR app and invoke web login.
Once you log in it will add a user account in accounts, and close the Account Authenticator XR app.
On returning to the Account Viewer app, you will the rotating cube rotating and user profile details.
Now go back to the Account Authenticator app, and logout (ie remove the account).
On the Account Viewer app, the cube is no more rotating, and no user profile data is shown.

Test

This completes the tests. You can found complete source code here :

Android Library: https://github.com/thinkuldeep/xr-user-authentication
Account Authenticator App: https://github.com/thinkuldeep/3DUserAuthenticator
Account Viewer App: https://github.com/thinkuldeep/3DAccountViewer

Conclusion

In this article, we have learned how to do single sign-on in the android based XR device, and share the user context with other XR applications. Single sign-on is one of the key requirements for the enterprise-ready XR device and applications.

It was originally published at XR Practices Medium Publication

＃xr ＃ar ＃vr ＃mr ＃enterprise ＃android ＃unity ＃tutorial ＃technology

Implement Android Library with custom AccountAuthenticator

Webview Login Activity

Create binding service for custom Account Type

Manage Accounts of Custom Type

Unity Authenticator App

Login — Launch WebView for Login

Login — Fetch the Access Token

Login — Add Account to Android Accounts

Get Logged-in User

Logout — Remove Account to Android Accounts

Run and test

Sharing Account with other XR apps

Get User Account from Android Accounts

Get profile using logged in user’s token

Check if a user is logged

Run and test

Request Login from external XR app

Add Request Login

Handle external login request in the Account Authenticator App

Run and test

Conclusion