Multi-modal communication through modal-specific interfaces

FIELD: radio engineering, communication.

SUBSTANCE: invention relates to the technology of providing multi-modal communication through modal-specific interfaces. In one method, a transmission in a first modality is received from a first interface via a computer network. The transmission is converted into a second modality. Upon converting the transmission into the second modality, the transmission is provided to a second interface via the computer network. The first interface is configured to transmit and receive a transmission only in the first modality and the second interface is configured to transmit and receive a transmission only in the second modality.

EFFECT: communication between one user interface operating under one modality and communication with another user interface operating under a different modality.

5 cl, 4 dwg

PRIOR art

The rapid growth of computer networks has created new modalities of communication on these networks. These new modalities include, among other things, e-mail, interactive correspondence (i.e. instant messaging, short message service (SMS, SMS) and voice over Internet Protocol (VoIP). In traditional implementations for each modality provides a specialized user interface. For example, there may be provided first user interface, giving the user the ability to send e-mail, then how could be provided by the second user interface, giving the user the ability to have an interactive communication via send and receive instant messages. In many cases, the first user interface is provided in a completely separate window from the second user interface, thus forcing the user to navigate, sometimes uncomfortable, between the Windows, to send emails and conduct interactive correspondence at the same time.

Additionally, transmission via the first interface are usually mutually exclusive from the transmission via the second interface. For example, the exchange of electronic mail between two users is not related to the exchange of interact the main correspondence between the same two users, even if the e-mail and sharing interactive correspondence occur simultaneously. As such users are usually responsible for maintaining communications between the many modalities, thus creating additional problems in relation to the implementation of communication through many modalities.

For these and other reasons presented made this disclosure.

The INVENTION

The paper describes technologies for multimodal communication through the specific modality for user interfaces. In particular, it provides communication controller, which allows you to have one user interface, operating under the same modality, communicated with another user interface running under a different modality. For example, the communication controller may enable the first user using the email interface, to communicate with a second user interface of the interactive communication.

To display the gear formed in one modality, the interface designed for the other modalities, the communication controller may be configured to convert the transmission from one modality to another modality. Examples of modalities may include, but not about reicived specified, e-mail, interactive correspondence (i.e. instant messaging, short message service (SMS) and voice mail. The communication controller may be further configured to run in the background communication system between two or more users. Thus, the user can smoothly transmit and receive transmissions via their preferred modality, regardless of the selected interface or modality to another user. Because the user can transmit and receive in a single modality, the responsibility to maintain any information or interests of the other modality can be charged to the user.

According to one aspect presented herein, a computer program provides multimodal communication through specific modality interfaces. The computer program receives data from the first interface through a computer network, the transmission of the first modality. A computer program converts the transmission of the second modality. After conversion, transfer the second modality of the computer program provides transmission to the second interface via a computer network.

This entity is shown that in a simplified form to introduce a selection of concepts that advanced opisyvayut is below in the detailed description. This entity is not intended to identify key features or essential features of the claimed subject matter of the invention and is not intended to imply that this entity is used to limit the amount of the declared object of the invention. In addition, the stated object of the invention is not limited to implementations that solve problems of any or all disadvantages noted in any part of this disclosure.

BRIEF DESCRIPTION of DRAWINGS

Figure 1 is a block diagram showing a network architecture configured with the ability to provide multimodal communication through specific modality interfaces in accordance with one embodiment;

Figure 2 - dual schema "snapshot" of the screen, showing an illustrative relationship between individual-specific modality interfaces in accordance with one embodiment;

Figure 3 is a block diagram showing an illustrative method of providing specific modality interfaces in accordance with one embodiment; and

4 is an architectural diagram of a computer system showing aspects of an illustrative architecture of PC hardware for a computer system capable of implementing aspects of the embodiments presented in the document.

DETAILED DESCRIPTION the ESSENTIAL substance of THE TEE of the INVENTION

The following detailed description relates to technologies for multimodal communication through the specific modality of the user interface. Although described in the document object of the invention is presented in the General context of program modules that are executed together with the execution of the operating system and application programs on a computer system, specialists in the art will recognize that there may be other implementations in combination with other types of software modules. In General, program modules include procedures, programs, components, data structures, and other types of structures that perform particular tasks or implement particular abstract data types. In addition, specialists in the art will appreciate that the object of the invention is described in the document, may be implemented with other configurations of computing systems, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, mainframes and the like.

In the following detailed description referring to the accompanying drawings, which form part of it and show to illustrate specific embodiments of, or examples. Now with reference on the drawings, in which on the be otherness numeric reference positions are the same elements in the several figures of the drawings, will be described aspects of a computing system and methodology for providing multimodal communication through the specific modality of the user interface. Figure 1 shows an illustrative network architecture 100, including the first user 102A and the second user 102B. The first user 102A controls the first computer 104A, which includes the interface 106A e-mail. The second user 102B controls the second computer 104B, which includes the interface 106B interactive correspondence. The first user 102A and the second user 102B can collectively be called by the users 102. Interface 106A and e-mail interface 106B online chat can collectively be called interfaces 106.

Interface 106A e-mail may be any suitable interface that can enable the first user 102A to send email to other users and receive e-mail from them. Examples of interfaces e-mail include those provided by GROUP WISE™ from NOVELL, INC. and designed for messaging and collaboration works with OUTLOOK® from MICROSOFT CORPORATION. In one embodiment, the interface 106A e-mail is a specialized e-mail application. Interface 106A e is octy may be a standalone application, executing on the first computer 104A. Alternatively, the interface 106A e-mail can be a Web application accessible through a Web browser executing on the first computer 104A. In another embodiment, the interface 106A e-mail is provided through any suitable application.

Interface 106B interactive correspondence may be any suitable interface that can enable the second user 102B to send instant messages to other users and receive instant messages from them. Examples of interactive interfaces correspondence include those provided by the software AOL® INSTANT MESSENGER™, AOL LLC and the client WINDOWS MESSENGER® messaging from MICROSOFT CORPORATION. In one embodiment, the interface 106B online chat is a specialized application for instant messaging. Interface 106B online chat can be a stand-alone application executing on the second computer 104B. In alternative interface 106B online chat can be a Web application accessible through a Web browser (not shown)executing on the second computer 104B. In another embodiment, the interface 106B interactive correspondence is provided in any suitable application.

Situational, the interface 106A and e-mail interface 106B online chat are just illustrative examples of communication interfaces. They may also use other suitable communication interfaces that allow users to communicate through text, images, audio, video, and combinations thereof. You should additionally assess what each communication interface can be precisely defined for a particular modality. Each communication interface may differ in their structure (appearance), which can be optimized for specific modalities, and specific modality for the opportunities that are provided. In many cases, even if the interfaces 106 are provided in a single application, interfaces 106 still usually differ from each drop-down menu or other suitable interface selection mechanism. In the alternative, the application can automatically switch interfaces 106 for users.

As illustrated in figure 1, the network architecture 100 additionally includes a controller 108 communication. In accordance with the variants of implementation, the controller 108 connection enables the first user 102A using the interface 106A e-mail, and the second user, using the mu interface 106B online chat, easily exchange information. In particular, the controller 108 connection enables the first user 102A to communicate, using the interface 106A e-mail, without information or leads on a specific modality used by the second user 102B or any other user. Similarly, the controller 108 connection allows the second user 102B to communicate, using the interface 106B, online chat without information or leads about modality or communication interface used by the first user 102A or any other user.

For example, when the first user 102A sends e-mail to the second user 102B, the controller 108 converts communication e-mail instant message and delivers an instant message to the second user 102B via the interface 106B interactive correspondence. Because the second user 102B deliver an instant message instead of email, the second user 102B may not have the information that the first user 102A uses the interface 106A e-mail. Regarding the second user 102B, the first user 102A may even use a different interface of the interactive communication.

Taking an instant message from the first user 102A via the controller 108 communication, the second uses the user 102B can answer an instant message through the interface 106B interactive correspondence. When sending the second user 102B answer an instant message to the first user 102A, the controller 108 converts the communication response to the instant message in e-mail and delivers the email to the first user 102A via the interface 106A e-mail. Because the first user 102A deliver e-mail instead of an instant message, the first user 102A may not have the information that the second user 102B uses the interface 106B interactive correspondence. As to the first user 102A, the second user 102A may even use a different e-mail interface.

The ability for the first user 102A and the second user 102B to communicate through separate interfaces and through different modalities without any information or leads on the other modality provides an additional level of abstraction over traditional implementations. The controller 108 due essentially hides the possibility that many users can use the interfaces that are running under different modalities. Accordingly, the user can use his preferred communication interface, even if the communication interface is specific to the modality and operates under a different modality than other users.

E.g. the motif, traditional implementation that provides multimodal communication, usually does not allow the user to select a preferred communication interface independently of other users. In one example, the sender may be limited to using only the interface 106B online chat, when the recipient is online (that is, information about the presence in the network). In another example, a recipient who receives e-mail from an interface 106A e-mail, may be limited to viewing e-mail to another e-mail interface. That is, the recipient cannot choose whether to accept e-mails in a different format, such as SMS or voice mail messages, by default.

In one embodiment, users 102 can configure the controller 108 to convert the incoming beams under one modality to one or more gears of the other modality. For example, the first user 102A may be configured controller 108 connection to convert incoming transfer messages voice mail and e-mail. If the incoming message is a text (text interface), voice mail messages can be generated by a suitable transducer "text-to-speech.

Users 102 can is advanced to configure the controller 108 to select the appropriate modality on the basis of predefined comply with conditions or business logic. For example, the second user 102B may be configured controller 108, to convert incoming transmission of the instant message if the second user 102B is online, or in an SMS message, if the second user 102B is offline. Enabling the user 102 to specify the logic that defines how users 102 will accept incoming transmission, the controller 108 connection provides improved flexibility over traditional implementations.

Now with treatment figure 2 will be presented for additional details on the relationship between the interfaces 106. In particular, Figure 2 includes diagrams "snapshot" of the screen 200A-200B showing illustrative conversation between the first user 102A and the second user 102B. As mentioned above relative to Figure 1, the first user 102A transmits and receives transmission through the interface 106A e-mail, and the second user 102B transmits and receives transmission through the interface 106B interactive correspondence. Also, as mentioned previously, the controller 108 communication gives the opportunity to users 102 communicated through their respective interfaces 106 without knowledge of a specific modality or communication interface of another user.

As illustrated in figure 2, the first circuit 200A screenshot displays Ajeet interface 106A e-mail, used by the first user 102A. Interface 106A e-mail includes a composite window 202 e-mail, where the first user 102A was 204 e-mail to the second user 102B. Interface 106A e-mail includes a field 206 of the recipient indicated by the word "to" (to), as well as field 208 topics. Making the e-mail message 204 in the composition window 202 e-mail, the first user 102A accesses (for example, performs a click to the button 210 parcels to send e-mail 204 to the second user 102B.

As additionally illustrated in figure 2, the second circuit 200B screenshot displays the interface 106B interactive correspondence used by the second user 102B. Interface 106B interactive correspondence includes panel 212 reading and panel 214 entries. Panel 212 reading displays incoming instant messages. In particular, figure 2 panel 212 reading displays an instant message 216, the controller 108 communication has been transformed from an e-mail 204. Instant message 216 includes information about the sender 218, indicating that the instant message 216 sent the first user 102A, and a time stamp 220 that indicates the time at which the first user 102A has sent an instant message 216. It should be noted that nowhere in the panel 212 canine specified that the first user 102A sent the email interface 106A e-mail. The controller 108 effective abstracts this information, thereby providing a more specialized practice for the second user 102B.

Panel 214 record enables the second user 102B to enter a response to an instant message 216 in the form of an instant message 222. Once the second user 102B provides access to the button 224 submission to submit an instant message 222 to the first user 102A, the controller 108 converts communication instant message 222 to e-mail. For example, e-mail, converted from an instant message 222 may be displayed in the Inbox (not shown) or other suitable notification interface within the interface 106A e-mail.

Returning now to Figure 3, will be presented further details concerning the actions of the controller 108 communication. In particular, figure 3 shows a block diagram illustrating aspects of the method presented in the document, to ensure multimodal communication through the specific modality interfaces. It should be appreciate that the logical operations described in the document, may be implemented (1) as a sequence implemented in a computer or a software module is th, executing in the computing system, and/or (2) as interconnected machine diagrams of the logic or circuit modules within the computing system. Implementation is a matter of choice, dependent on operating characteristics and other requirements of the computing system. Accordingly, the logical operations described in the document, variously referred to as: States, operations, structural devices, acts or modules. These operations, structural devices, acts and modules may be implemented as software, firmware, in the form of dedicated digital logic, and any combination thereof. You should also assess what can be done more or less operations than shown in the figures and described in the document. These operations can also be performed sequentially, in parallel or in sequence, other than those described in the document.

With regard to Figure 3, the procedure 300 begins at step 302, where the controller 108 communication takes first gear in the first modality from the first interface. For example, if the first interface is an interface 106A e-mail, by first modality of transmission is e-mail. The procedure 300 then proceeds to step 304, where the communication controller converts the first transmission from the first modality to the second m the range, associated with the second interface. For example, if the second interface is an interface 106B interactive correspondence, the controller 108 converts communication e-mail 204, adopted from the interface 106A e-mail, instant message 216. After converting the first transmission to the second modality procedure 300 proceeds to step 306.

At step 306, the controller 108 connection supplies the converted first transmission to the second interface. For example, after converting the electronic mail 204 in an instant message 216, the controller 108 may deliver an instant message 216 to interface 106B interactive correspondence. Having converted the first transmission from the controller 108 communication, the second interface may display the first gear in the second modality, as indicated by the second user 102B. The procedure 300 then proceeds to step 308.

At step 308, the controller 108 communication takes a second transmission of the second modality to the second interface. For example, if the second interface is an interface 106B interactive correspondence, the modality of the second gear is an interactive correspondence. The second transmission may be the answer, such as instant message 222, the first transmission. The procedure 300 then proceeds to step 310, where the controller 108 connection converts the second transmission from the second modality in parvo the modality, associated with the first interface. For example, if the first interface is an interface 106A e-mail, the controller 108 converts communication instant message 222, taken from the interface V online chat, email and more. After converting the second gear in the first modality procedure 300 proceeds to step 312.

At step 312, the controller 108 connection supplies the converted second transmission on the first interface. For example, after converting the instant message 222 to e-mail, the controller 108 may provide the email interface 106A e-mail. Having converted the second transmission from the controller 108, one interface may display the second gear in the first modality, as defined by the first user 102A.

Now with treatment figure 4 illustrates an exemplary architectural diagram of a computer system showing aspects of the computer 400. Examples of the computer 400 includes first computer 104A and the second computer 104B. The computer 400 includes a processor 402 (CPU, CPU), system memory 404 and the system bus 406 that connects the memory 404 to the CPU 402. The computer 400 additionally includes a mass storage device 412 large capacity to store one or more program modules 414 and one or more databases 416 data. Examples of prog is amnah modules 414 include an interface 106A e-mail, interface 106B interactive correspondence and a controller 108 communication. The storage device 412 large capacity is connected to the CPU 402 via the storage controller high capacity (not shown)connected to the bus 406. The storage device 412 large capacity and the associated computer readable media provide non-volatile memory for a computer 400. Although the description of computer-readable media contained herein refers to a device of a large capacity such as a hard disk drive or CD-ROM, specialists in the art will appreciate that computer-readable media can be any available media computer, which can be accessed by the computer 400.

As an example, and not limitation, computer readable media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. For example, computer readable media include, but are not limited to these, random access memory (RAM, RAM), a persistent storage device (RAM, ROM), erasable programmable ROM (EPROM, EPROM), electrically with iremote programmable ROM (EEPROM, EEPROM), flash memory, or other technology of solid state memory, a ROM on the CD-ROM (CD-ROM), digital multi discs (DVD), HD DVD (high density)disks for BLU-RAY, or other optical memory, memory, magnetic cassettes, magnetic tape, magnetic disk or other magnetic memory, or any other medium that can be used to store the desired information and which can be accessed by the computer 400.

According to different variants of implementation, the computer 400 may operate in a networked environment using logical connections to remote computers through a network 418. The computer 400 may connect to a network 418 via the block 410 of the network interface connected to the bus 406. It should be appreciate that the block 410, the network interface can also be used to connect with other types of networks and remote computer systems. The computer 400 may also include a controller 408 I/o for receiving and processing input from many input devices (not shown), including a keyboard, mouse, microphone, and game controller. Similarly, the controller 408 I/o can provide a display or other type of output device (not shown).

On the basis of the foregoing, we can estimate that the document presented technology to ensure the Oia multimodal communication through the specific modality interfaces. Although presented in the document object of the invention has been described in language specific to structural features of computer, methodological acts, and computer-readable media, it should be understood that the invention defined in the attached claims is not necessarily limited to the specific features, acts, or media described in the document. Preferable specific indications, actions, and media are disclosed as exemplary forms of implementing the claims.

The above object of the invention is presented only as an illustration and should not be construed as restrictive. You can run various modifications and changes with respect to the object of the invention is described in the document, not following the illustrated and described exemplary implementation options and applications, and without going beyond the true essence and scope of the present invention, which is set forth in the following claims.

1. How multimodal communication, comprising stages, which are:
take over a computer network from the first adapted for the specific modality of the user interface implemented with the ability to send and receive transfers only the first modality, the first transmission in a first modality from the first what about the user, moreover, the first transmission addressed to the second user;
on reception of the first transmission in a first modality from the first adapted for the specific modality of the user interface through a computer network determines that the second user is associated with the second modality, so that the second user can send and receive transmission only in the second modality;
on determining that the second user is associated with the second modality, convert the first gear to the second modality based on the second preference specified by the second user when the second user is online, and convert the first gear to the third modality, when the second user is offline;
after converting the first transmission to the second modality serves first gear in the second adapted for the specific modality of the user interface through a computer network;
take the second adapted for a specific modality of the user interface through a computer network, the second transmission of the second modality from the second user, the second transmission addressed to the first user;
receiving a second transmission of the second modality of the second adapted for a specific modality polzovatelya the interface through a computer network to determine that the first user is associated with the first modality;
on determining that the first user is associated with the first modality, convert the second transmission in the first modality based on the first preference specified by the first user; and
after converting the second gear in the first modality serves the second gear in the first adapted for the specific modality of the user interface through a computer network, the first adapted to the specific modality of the user interface is configured to represent gears in the first modality and not made with a view of the transmission in the second modality, and the second adapted to the specific modality of the user interface is configured to represent gears in the second modality, and not made with a view of the transmission in the first modality.

2. The method according to claim 1, wherein the first modality is one of the e-mail, chat, short message service (SMS) and voice mail.

3. The method according to claim 1, wherein the first modality is chatting, and the second modality is e-mail, while the second is adapted to the specific modality of the user interface displays a notice which begins on the first transmission in the Inbox.

4. Machine-readable media on which is stored Mashinostroenie commands that, when performed by a computer, instruct the computer to:
to take over a computer network from the first adapted for the specific modality of the user interface running on the first computer, the first transmission in a first modality from the first user, and the first transmission addressed to the second user, the first adapted to the specific modality of the user interface is configured to represent gears in the first modality and not made with a view of the transmission in the second modality, the first adapted to the specific modality of the user interface is configured to send and receive transfers only the first modality;
on reception of the first transmission in a first modality from the first adapted for the specific modality of the user interface through a computer network to determine that the second user is associated with a second tailored to a specific modality user interface running on the second computer, the second adapted for the specific modality of the user interface is configured to represent gears in the second modal is t and not made with a view of the transmission in the first modality, and the second adapted to the specific modality of the user interface is configured to send and receive transmission only in the second modality;
on determining that the second user is associated with a second tailored to a specific modality of the user interface, to convert the first transmission to the second modality based on the second preference specified by the second user when the second user in the online mode, and to convert the first transmission in a third modality, when the second user is offline;
after converting the first transmission to the second modality to submit the first transmission over the computer network to the second adapted for a specific modality user interface running on the second computer;
to take over a computer network from the second adapted for a specific modality of the user interface running on the second computer, the second transmission of the second modality from the second user, the second transmission addressed to the first user;
receiving a second transmission of the second modality of the second adapted for a specific modality of the user interface through a computer network to determine that the first user is the motor associated with the first modality;
on determining that the first user is associated with the first modality, to convert the second transmission in the first modality based on the first preference specified by the first user; and
after converting the second gear in the first modality to apply through a computer network, the second transmission in first adapted for the specific modality user interface running on the first computer.

5. Machine-readable medium according to claim 4, the first modality is one of the e-mail, chat, short message service (SMS) and voice mail.