Proposal for Voice Calls in Wonderland

This proposal covers voice calling in Wonderland. This includes incoming and outgoing calls to PBX phones as well as in-world voice chat using voice-over-IP (VoIP). The concept in this proposal is to unify PBX calling and voice chat with a single user interface. Wonderland will support two types of virtual phones - conference phones and personal phones. Either of these types of phones will be able to place and receive either PBX calls (assuming a PBX gateway is configured) or VoIP calls.

Use Cases

• Private conversation - A person in-world may wish to have a private conversation with another person, who may or may not be in-world at the moment. Sometimes the person initiating the call may not wish others to know they are talking to someone else. Other times, it's helpful for others to know that a remote conversation is taking place because it explains why that person is not completely engaged.

• Backchannel conversation - One or more people attending an in-world event together may wish to privately confer with one another. For example, if a group of people from a company are meeting with customers, the company employees may all wish to be on a private chat to confer about strategy without the customer knowing about the conversation. In another example, a technically savvy user may volunteer to help a colleague troubleshoot a problem in the background so that the rest of the people in the current meeting are not disturbed. In this case, it's appropriate for everyone in the space to know that the troubleshooting conversation is happening.

• Sidebar conversations - One or more groups of people may opt to have sidebar conversations. These conversations may be private or public, depending on the situation. For example, a professor might ask his in-world class to divide up into groups and work on a problem together and then come back and report on the results to the whole class.

• Combating noise - A group of people in a noisy world may wish to converse with one another more easily. For example, a group visiting a virtual trade show together may wish to have a private chat with one another as they navigate around the space so they can hear one another more clearly.

• Keeping in touch while navigating - Some group of people may want to stay in voice contact whether or not they can see one another. Following on from the previous example, the group visiting the trade show may want to stay in touch with one another as they scatter to see different exhibits. If one finds something of interest to the others, they can simply speak to call them over to look.

• Remote consultation - During a lecture, class, or meeting, there may be a need to consult with one or more remote people who may or may not also be in-world. In this case, one attendee can initiate a public call to the remote person or people and everyone in the room can hear and participate in the conversation.

• Remote participation - One or more remote people may wish to listen into an event such as a class or meeting taking place in a virtual space. The event may be in a single space, or the participants may be navigating around the virtual world. In this case, they can take the remote participants with them.

User Interface

This section outlines a proposed user interface for voice calling. The illustrations are a rough mock-up and don't necessarily include all the features described in the notes.

Outgoing calls from virtual personal phone

• Click on phone icon in HUD to place call

Figure 1: Phone icon in Wonderland HUD

• Call can be to PBX phone or in-world phone
  • Include "favorites" list for selecting people to call
    • Contacts can have multiple contacts points (eg, in-world, home phone, work phone, cell phone)
    • Highlight favorite people who are currently in-world
    • If person is in-world, calling them in-world would be default
    • Show symbol for people who can only be reached by PBX phone
      • If there are multiple contact points, tagged one as the "preferred" number, so this can be used as the default
    • Also include list of all in-world users?
    • Need to figure out how to add to favorites
  • Option to type in a PBX phone number
  • Option to type in WL user name??
    • Need human-understandable "phone number" for WL users, perhaps user-name + server address?
    • Allow voice chat calls to WL users on other servers?
• Set privacy slider (default is "private")
  • Slider options:
    • secret - no visual and audio indication you are talking to anyone else
    • private - visual indicator you are talking to someone else (eg, show avatar talking on a cell phone), but no one hears the conversation
    • public/silent - visual indicator of who you are talking to (eg, orb or ghost avatar), but no one hears the conversation
    • public/quiet - visual indicator of who you are talking to (eg, orb or ghost avatar), and others hear the conversation at a soft (attenuated) volume
    • public - visual indicator of who you are talking to (eg, orb or ghost avatar) and others hear the conversation at full volume
  • Initial implementation should just include "secret," "private," and "public"
• Avatar is automatically put on mute when secret or private call is placed

Figure 2: Place a call

• Secret and private calls
  • HUD controls appear prominently in CENTER of screen (as in Figure 2) to indicate that user's audio is going to private call and not to world
• Public calls
  • Orbs appear for remote participants
  • HUD controls appear when orb is clicked

Figure 3: Public call

• Placing additional outgoing calls
  • Click on phone icon again to place another separate call
    • Option to place first call on hold or disconnect and call
    • Separate HUD box for each call
  • Click on "Add person" button in HUD to add user to initial call (see Figure 5)

Incoming calls to personal phone

• Personal phone "rings" - HUD and optional audio notification
• HUD notification pane provides options:
  • Caller ID
  • Answer / Ignore / Send "busy" message
  • Public --> Private slider (default is "secret")
  • If you're currently on a call, options are:
    • Put current call on hold and answer
    • Disconnect current call and answer
    • Ignore
    • If caller is in-world, send "busy" message

Figure 4: Incoming call

In a call

• For secret and private calls:
  • Meeting Suite style display in middle of HUD with options
    • List of names with speaking indicators
    • Keypad
    • Call Quality slider to dynamically change quality
    • ... (check iPhone UI for other possible options)
    • Add person button
    • Hold button
      • HUD animates to bottom when call is on hold
      • Can be collapsed or expanded
      • HUD window is named to differentiate calls
    • Public --> Private slider
      • HUD animates to side so only tab is showing

Figure 5: In a call (image should include privacy slider)

Figure 6: In a conference call (image should include privacy slider)

Figure 7: Call on hold

• For public calls:
  • HUD animates to side or is invisible unless an orb is clicked
  • Can be collapsed to save space
  • May need "Return to World" button?
  • Does it make sense to place a public call on hold?

Mockup Source (photoshop)

Virtual Conference Phones

• Click on in-world conference phone to place a call (same as 0.4)
• HUD UI modified to be the same as shown above
• Add ability to call personal in-world phones as well as PBX phones

Issues with Public Calls

There are a variety of issues that must be resolved with public calls.

• Avoid interrupting others - the person placing a call may wish for the call to be public, but there is no way for the person to know the situation of the recipient of the call. It Alice calls Bob, for example, and Bob is in the middle of attending a lecture, Bob would not want orbs popping up over his head, distracting the speaker.
  • Solution is to make all incoming calls "secret" by default. It's perfectly fine for the call to be public on the side of the initiator and secret or private on the side of the recipient
  • If appropriate, the recipient can use the privacy slider to make the call public at any time during the call

• Knowing who can hear you - from a privacy point of view, it's important that when someone speaks, they know who is hearing their voice. For example, if Alice and Bob are in a public call and Bob is standing with Jane, then Alice needs to know that Jane, a by-stander, can hear her. Jane already knows that Alice can hear her because she sees Alice's orb over Bob's head.
  • One possible solution is to create a by-stander orb - an orb that represents one or more by-standers
    • By-stander orb would appear as soon as one or more by-standers comes into audio range of a public call
    • By-stander orb (yellow in the illustration below) would include the number of by-standers
    • Number would continually update as by-standers navigate in and out of audio range of the public call
      
      

• Knowing who is speaking - If possible, it is always helpful to know the identify of a remote person speaking. In the scenario above, Jane is not "officially" part of the call between Alice and Bob, she's a by-stander, so when she speaks, Bob knows Jane's identity, but Alice does not.
  • The by-stander orb solves this problem as well
    • When a by-stander speaks - say Larry has just walked up and joined Jane, his name would appear above the orb while he's speaking. When Jane speaks, her name would appear.
      
      
    • Clicking on a by-stander orb would open a HUD component that lists all the names of the by-standers along with speaking indicators
      • If the list is large, then it will be sorted with the most recent speakers at the top of the list, as is done successfully in the Meeting Suite UI

• Echo - Continuing with the example, If Alice's avatar (along with Bob's orb and the by-stander orb) walks up to Bob and the by-standers, there will be bad echo introduced because Alice will hear Bob and the by-standers' voices coming from both their avatars and from the orbs. Likewise, Bob and Jane will hear Alice's voice twice.
  • Since this situation is analogous to walking up to a person who you are currently talking to on a cell phone, the thought is to not worry about this situation. The solution is for Bob and Alice to end the call!

Implementation

The implementation of the backend functionality to support this user interface design will be done in the voice bridge. Preliminary documentation is here:

jVoiceBridge Voice Manager Design