Interaction Rectangle | Rectangular geometry collecting and sending interaction events via UDP or ANC channel |
The Interaction Rectangle node renders a rectangle geometry, collects all interaction (mouse, touch, keyboard) and sends this data in an Open Sound Control based protocol via UDP or the VIO ANC channel for further processing by the Live Video texture provider.
This node was basically introduced to provide the possibility of interaction with a Live Video texture source. Possible scenarios could be:
All these applications of course require software development and custom modifications on side of the 'video texture provider'.
The Interaction Rectangle provides some input properties to control the behavior of the node.
The Mouse and Multi-Touch events can only be interpreted correctly if the Interaction Rectangle is not rotated at all!
The node is only sending if the Cluster Networking is configured to Single or Master!
Tracking Mode:
Via the Custom Verbs of the Property Editor it is possible to add output property sets which provide information about active Mouse and Multi-Touch events. A description of the fixed output properties and events, that all Interaction nodes have in common, can be found here.
The interaction events collected by the node are formatted as OSC bundles. For details about the OSC format see the Remoting 4 Protocol Specification. The message format may look complicated. The reason for this is that it is based on our internal Input Subsystem format that is used for input synchronisation in a multi-machine Cluster.
The message structure looks like follows:
OscBundle Address Arguments Array OscElement /ventuz/<i_ReceiverID>/InputTimecode <d_Timecode> OscElement[n] /ventuz/<i_ReceiverID>/<s_DeviceType>/<i_DeviceID>/<s_ID>/<i_TouchGroup> <i_TouchType>, <s_TouchID>, <f_X>, <f_Y>, <f_Z>, <f_Pressure>, <f_Angle> - <i_Value> <f_X>, <f_Y>, <f_Z>
The OscBundle contains at least two OscElements. The first OscElement carries the timecode message. Its address is /ventuz/<i_ReceiverID>/InputTimecode where <i_ReceiverID> is an integer number representing the receiver ID. The argument carried by this OscElement is a double value representing the timecode of the frame in which the interaction data was recorded. All subsequent OscElements carry interaction data; the address is always /ventuz/<i_ReceiverID>/<s_DeviceType>/<i_DeviceID>/<s_ID>/<i_TouchGroup>.
The address parameters have following meaning and content:
Address Parameter | Meaning | Possible Values |
<i_ReceiverID> | identifies the receiver of the OscElement | integer >= 0 |
<s_DeviceType> | string specifying the type of the interaction data | MultiTouch, Mouse, Keyboard |
<i_DeviceID> | ID of the interaction device this information was created by | any integer |
<s_ID> | string ID representing the type of input information | ID_TouchAdd, ID_TouchUpdate, ID_TouchRemove, ID_MouseButtonDown, ID_MousePosition, ID_MouseButtonUp, ID_MouseWheel, ID_MouseHover, ID_MouseDoubleClick, ID_MouseDoubleClick2 |
<i_TouchGroup> | integer value representing a bitmask for four different touch groups | integer 0-31 |
The different argument types and values are explained here:
Argument Parameter | Meaning | Possible Values |
<d_Timecode> | timecode of the interaction data | double >= 0.0 |
<i_TouchType> | type of touch | 0 = Cursor, 1 = Marker, 2 = VR |
<s_TouchID> | string representing numeric ID of the touch (e.g. finger) | integer >= 0 |
<f_X> | X position of touch or mouse | float in range [0.0; 1.0] |
<f_Y> | Y position of touch or mouse | float in range [0.0; 1.0] |
<f_Z> | Z position of touch or mouse | float in range [0.0; 1.0] |
<f_Pressure> | pressure value of the touch | float in range [-1.0; 1.0] |
<f_Angle> | rotation angle of a touch marker | float in range [0.0; 360.0] |
<i_Value> | mouse button or key code | integer |
You can use the Ventuz OSC library to receive the interaction data via UDP.
To receive the interaction events in the VIO API use the vAncFromVentuz() method with ITAC as FourCC code.