API Reference

Index

Initialize the Javascript Library within your Spatial Tool

Include the Spatial Toolbox functionality by instancing as following in your JavaScript code:

var spatialInterface = new SpatialInterface();

Communication with other Spatial Tools

The Vuforia Toolbox allows each Tool to communicate with other tools that are currently visible in the Spatial Toolbox.

sendGlobalMessage(message)

  • message (String) message to be sent

Send messages that broadcast to all other spatial tools currently visible in the Spatial Toolbox.

spatialInterface.sendGlobalMessage("Hello World");

sendMessageToFrame(toolUUID, message)

  • toolUUID (String) Unique ID for a spatial tool
  • message (String) Message to be sent

Message to a specific tool via frameKey

spatialInterface.sendGlobalMessage("destinationToolUUID", "Hello World");

addGlobalMessageListener(callback)

  • callback (Function)
  • Returns message (String) Message recived

It allows a spatial tool to listen to messages broadcasted by other spatial tools.

spatialInterface.addGlobalMessageListener(function(message){
  console.log(message);
});

addFrameMessageListener(callback)

  • callback (Function)
  • Returns message (String) Message recived

Only listen to messages that are addressed to this specific spatial Tool.

spatialInterface.addFrameMessageListener(function(message){
  console.log(message);
});

Subscribe to Matrices

The Spatial Toolbox allows every spatial Tool to subscribe to a variety of matrices. These matrices can be used, for example, to calculate the distance to other spatial tools, or they allow the use of webGL for spatial tools that require 3D content.

subscribeToMatrix()

Subscribe to the modelView and projectionMatrix. These matrices locate the spatial origin point for your Tool relative to the Object the Tool is attached to. This can be an object or world object.

spatialInterface.subscribeToMatrix();

subscribeToScreenPosition()

Subscribe to the 2D screen position for a 3D spatial tool position in space.

spatialInterface.subscribeToScreenPosition();

subscribeToDevicePoseMatrix()

Subscribe to the devicePoseMatrix. This Matrix describes where the device is located in space relative to the world object origin.

spatialInterface.subscribeToDevicePoseMatrix();

subscribeToAllMatrices()

Subscribe to the modelView matrices for all visible tools.

spatialInterface.subscribeToAllMatrices();

subscribeToGroundPlaneMatrix()

Subscribe to the ground-plane Matrix. This Matrix will match the world object origin once the device sees the world object.

spatialInterface.subscribeToGroundPlaneMatrix();

addMatrixListener(callback)

  • callback (Function) This callback is syncronized with video frame rate.
  • Returns modelview (Number[16]) ModelViewMatrix
  • Returns projection (Number[16]) ProjectionMatrix

Listen to modelView Matrix updates for your single spatial Tool. It only works once subscribeToMatrix() is called. This listener is synchronized with the video update rate. Use the callback of this listener to synchronize animations.

spatialInterface.addMatrixListener(function(modelview, projection){
  var modelViewMatrix = modelview;
  var projectionMatrix = projection;
});

addAllObjectMatricesListener(callback)

  • callback (Function) This callback is syncronized with video frame rate.
  • Returns allModel (Object) ModelViewMatrices for all visible Tools with the format toolUUID (String): modelViewMatrix (Number[16]).
  • Returns projection (Number[16]) ProjectionMatrix

Listen to modelView Matrix updates for ALL visible spatial tools. It only works once subscribeToAllMatrices() is called. This listener is synchronized with the video update rate. You can use the callback of this listener to synchronize animations.

spatialInterface.addAllObjectMatricesListener(function(allModel,preojection){
 var allModelviewMatrices = allModel; // object of matrices arrays
 var projectionMatrix = preojection;
});

addDevicePoseMatrixListener(callback)

  • callback (Function) This callback is syncronized with video frame rate.
  • Returns devicePose (Number[16]) DevicePoseMatrix
  • Returns projection (Number[16]) ProjectionMatrix

Listen to device pose Matrix updates from your device. It only works once the related subscription is called. This listener is synchronized with the video update rate.

spatialInterface.addDevicePoseMatrixListener(function(devicePose, projection){
  var devicePoseMatrix = devicePose;
  var projectionMatrix = projection;
});

addScreenPositionListener(callback)

  • callback (Function) This callback is synchronized with the video frame rate.
  • Returns devicePose (Object) Object with x (Number) and y (Number) coordinates

Listen to the 2D screen position for any spatial tool. It only works once the related subscription is called. This listener is synchronized with the video update rate.

spatialInterface.addScreenPositionListener(function(position){
    var x = position.x;
    var y = position.y;
});

cancelScreenPositionListener()

Remove all matrix listeners.

spatialInterface.cancelScreenPositionListener();
});

getPositionX(), getPositionY(), getPositionZ()

  • Returns (Number) x,y or z

Returns a number for translation distance and position between the spatial Tool and the device.

x =  spatialInterface.getPositionX();
y =  spatialInterface.getPositionY();
z =  spatialInterface.getPositionZ();

getProjectionMatrix()

  • Returns (Number[16]) projection matrix

Returns the current projection Matrix in the format m[16]

projectionMatrix =  spatialInterface.getProjectionMatrix();

getModelViewMatrix()

  • Returns (Number[16]) model-view matrix

Returns the current modelView Matrix in the format m[16]

modelViewMatrix =  spatialInterface.getModelViewMatrix();

getGroundPlaneMatrix()

  • Returns (Number[16]) ground-plane matrix

Returns the current ground plane Matrix in the format m[16]

groundPlane =  spatialInterface.getGroundPlaneMatrix();

getDevicePoseMatrix()

  • Returns (Number[16]) device-pose matrix

Returns the current device pose Matrix in the format m[16]

devicePose =  spatialInterface.getDevicePoseMatrix();

getAllObjectMatrices()

  • Returns (Object) All matrices

Returns all current modelView Matrices in the format {"objectUuid": number[16], ...}

allMatrices =  spatialInterface.getAllObjectMatrices();

Control AR Screen Position

Spatial Tools can exist of a simple HTML page or webGL content. Some tools require fullscreen modes or persist in space, and even the related Object is not visible. WebGL content always requires a fullscreen mode since the WebGL context takes over the spatial transformations.

setFullScreenOn()

Set your tool to fullscreen mode. Use this non spatial 2D UIs or any WebGL context.

spatialInterface.setFullScreenOn();

setFullScreenOff()

Switches the full screen mode off and attaches the tool back to its spatial position.

spatialInterface.setFullScreenOff();

setStickyFullScreenOn()

Set your tool to permanent fullscreen mode. Use this mode for 2D UI screens, so that the UI does not dissapear when the attached object is out of view.

spatialInterface.setStickyFullScreenOn();

setStickinessOff()

Remove the stickyness from the Fullscreen mode.

spatialInterface.setStickinessOff();

setExclusiveFullScreenOn()

Set your Tool to permanent fullscreen mode. This mode will remove any other fullscreen mode that is called with the same exclusivity. The callback is called when the fullscreen mode is killed.

spatialInterface.setExclusiveFullScreenOn(callback);

setExclusiveFullScreenOff()

Switches the exclusive full-screen mode off and attaches the Tool back to its spatial position.

spatialInterface.setExclusiveFullScreenOff();

Media Content

These functions help to record videos, images, or use the device build-in keyboard.

startVideoRecording()

The Spatial Toolbox will start recording the screen in the background. The final video will automatically be saved once the video is stoped and attached to your spatial Tool.

spatialInterface.startVideoRecording();

stopVideoRecording(callback)

  • callback (Function) The callback provides the video URL
  • Returns url (String) The URL for the video

Stop the video recording and attach it to your Tool.

spatialInterface.startVideoRecording(function(url){
    var videoURL = e;
});

announceVideoPlay()

On certain Mobile Devices, the playback of multiple videos in space can be difficult. Use this function to announce that a single video is played.

spatialInterface.announceVideoPlay();

subscribeToVideoPauseEvents(callback)

  • callback (Function)

Use this event to program your video player to pause when other videos are playing. In the best case, try to unload your video and replace it with a screenshot to save resources for other Spatial Tools.

spatialInterface.subscribeToVideoPauseEvents(function(){
    // called when pause is requested.
});

getScreenshotBase64(callback)

  • callback (Function)
  • Returns image (String) Image encoded in base64 string.

Screenshot of the Vuforia Camera. A single frame returned as a base64 string.

spatialInterface.getScreenshotBase64(function(image){
    var imageBase64String = image;
});

openKeyboard()

Open the Keyboard

spatialInterface.openKeyboard();

closeKeyboard()

Close the Keyboard

spatialInterface.closeKeyboard();

onKeyboardClosed(callback)

  • callback (Function)

Callback if the keyboard is closed

spatialInterface.onKeyboardClosed(function(){
    // called on keyboard closed
});

onKeyUp(callback)

  • callback (Function)
  • Returns key (Object) Keybord event

Use this to listen to keyboard entries. The callback returns the entire keyboard event.

spatialInterface.onKeyUp(function(key){
    var keyboardEvent = key;
});

Tool Behavior

setVisibilityDistance(distance)

  • distance (Number) number in meter to define distance

Define the distance (in meter). Your spatial Tool is visible in the Spatial Toolbox. A user can change this number via the Spatial Toolbox UI.

spatialInterface.setVisibilityDistance(2);

addVisibilityListener(callback)

  • callback (Function)
  • Returns key (Object) Keybord event

A callback to read the visibility of a spatial tool. The Interface stays active for 3 seconds after it becomes invisible.

spatialInterface.addVisibilityListener(function(visible){
  var isVisible = visible;
});

getVisibility()

  • Returns (Boolean)

Returns true or false if the spatial Tool is visible.

spatialInterface.getVisibility();

setMoveDelay(delay)

  • delay (Number) number in milliseconds

Set how long it takes (in ms) for a user to tap and hold a spatial tool until it becomes moveable.

spatialInterface.setMoveDelay(10);

addIsMovingListener(callback)

  • callback (Function)
  • Returns move (Boolean)

Returns true if a tool is moved and false once it is released.

spatialInterface.addIsMovingListener(function(move){
  var isMoving = move;
});

enableCustomInteractionMode()

One way to define when a Tool becomes moveable is the moveDelay described above. Another way is to define specific areas that are interactive elements, and as a consequence touching all other areas makes the Tool instantly movable. Call this function to invoke this functionality. Assign all your interactive elements the class spatial interaction to make them interactive elements.

spatialInterface.enableCustomInteractionMode();

enableCustomInteractionModeInverted()

This does the invert to the function above. In this case, the elements defined by the next function become the item that allows the spatial moving.

spatialInterface.enableCustomInteractionModeInverted()

setInteractableDivs(divList)

  • divList (String[]) Array of interactive html elements

Once enableCustomInteractionModeor enableCustomInteractionModeInverteduse this function to define the interactive elements.

  spatialInterface.enableCustomInteractionMode();
  spatialInterface.setInteractableDivs([touchInteractiveElement]);

subscribeToFrameCreatedEvents(callback)

  • callback (Function)
  • Returns toolUuid (String) The Uuid of the tool created
  • Returns type (String) the type of the tool (e.g., graph, slider, etc.)

Triggers a callback anytime another new tool is created while this tool is loaded in the DOM.

spatialInterface.subscribeToFrameCreatedEvents(function(toolUuid, type){
var thisToolIsNew = toolUuid;
var itsTypeIs = type;
});

subscribeToFrameDeletedEvents(callback)

  • callback (Function)
  • Returns toolUuid (String) The Uuid of the tool deleted
  • Returns type (String) the type of the frame (e.g., graph, slider, etc.)

Triggers a callback anytime a tool is deleted while this tool is loaded in the DOM.

spatialInterface.subscribeToFrameDeletedEvents(function(toolUuid, type){
var thisToolIsDeleted= toolUuid;
var itsTypeWas = type;
});

ignoreAllTouches(newValue)

  • newValue (Boolean)

Pass in true (or omit the argument) to make the Tool set the pointer-events to none, so all touches pass through un-altered

spatialInterface.ignoreAllTouches(false);

registerTouchDecider(callback)

  • callback (Function) A callback function that decides if our Tool is touched or not.
  • Returns eventData (Object) touch event includes x and y of screen touch.
  • Returns (Boolean) Your callback should return a boolean to decide if a touch is inside an element or outside.

Use this function in a fullscreen tool for when you use, for example, uses threejs or similar. You need to pass a function that decides if a touch event is consumed with your experience.

spatialInterface.registerTouchDecider(function(eventData){
  return doSomethingThatDecides(eventData.x, eventData.y);
});

unregisterTouchDecider()

Cancel the touch decider callback.

spatialInterface.unregisterTouchDecider();

changeFrameSize(newWidth, newHeight)

  • newWidth (Number) A new touch overlay width
  • newHeight (Number) A new touch overlay height

Adjust the size of the Tool’s touch overlay element to match the current size of this tool.

spatialInterface.changeFrameSize(640, 480);

getScreenDimensions(callback)

  • callback (Function)
  • Returns (Number) screen dimension width
  • Returns (Number) screen dimension height

Asynchronously query the screen width and height from the parent application, as the iframe itself can’t access that.

spatialInterface.registerTouchDecider(function(width, height){
    screenDimensionWidth = width;
    screenDimensionHeight = height;
});

Spatial Edge Server Communication

The previous function handles the communication and behavior within the Spatial Toolbox. The following functions handle the communication with the Spatial Edge Server that owns the Object in which the Spatial Tool is registered.

write(node, value, mode, unit, unitMin, unitMax, forceWrite)

  • node (String) Name of Node
  • value (Number) The new value for the Node. The range must be between 0.0 and 1.0.
  • mode (String) [optional] Data mode. Currently, f for float is the only supported choice.
  • unit (String) [optional] The unit of your data such as ‘kg’, ‘minutes’, ‘liter’, …
  • unitMin (String) [optional] value range minimum
  • unitMax (String) [optional] data range maximum
  • forceWrite (Boolean) [optional] Sends the Value even if it hasn’t changed since the last write.

Write flow data to a node owned by the Spatial Tool. This flow data is of a simple floating data type with values between 0.0 and 1.0. The example below, writes 0.5 to the node = “nodeName”, unit = kg, min = 0, max = 10 and the last entry tells that the Value is only updated on change. The resulting value will be 5 kg.

spatialInterface.write("nodeName", 0.5, "f", "kg", 0, 10, false);

getUnitValue(flowDataObject)

  • flowDataObject (String) flow data object
  • Returns (Object) real value (mapped 0 - 1 to min - max) and unit

Returns the real Value and Unit of a node flow data object. The Value is mapped to the minimum and maximum. For example a data package with a value of 0.5, a min = 0, max = 10 and unit = “kg” will return a value of 5 and the unit “kg”.

valueAndUnitObject = spatialInterface.getUnitValue(flowDataObject)

spatial.addReadListener(node, callback)

  • node (String) name of Node
    • callback (Function) is called anytime the server provides a new data object for the Node.
  • Returns flowDataObject (Object) flow data object for the Node

Add a read listener to read value changes from a node. These changes are synchronized with the Edge Server for as long as the Spatial Tool active. The Tool becomes inactive 3 seconds after it’s not visible anymore.

spatialInterface.addReadListener("nodeName", function(flowDataObject){
    var realValues = spatialInterface.getUnitValue(flowDataObject);
    value = realValues.value;
    unit = realValues.unit;
});

readRequest(node)

  • node (String) name of Node

This forces the server to send the latest Value so that the readListener can pick it up. You can use this in combination with addReadListener to initialize your spatial Tool if needed.

spatialInterface.readRequest("nodeName");

spatialInterface.addReadListener("nodeName", function(e){
    var package = spatialInterface.getUnitValue(e);
    value = package.value;
    unit = package.unit;
});

writePublicData(node, valueName, value, realtimeOnly)

  • node (String) name of Node
  • valueName (String) public data is defined by the Node itself. This is a key to select an object within public Data.
  • value (Object/String/Number) Data you want to store with a key.
  • realtimeOnly (Boolean) doesn’t send a post message to reload Object, allows rapid stream.

PublicData is a JSON object that contains data objects specified by the server-side node definition. You need to finetune your Tool via specific node types. The publicData allows you to handle complex data among the server-side node program, and the Spatial Tool and the Value can contain any JSON encoded data. The last parameter is optional and if set to true, forces the server not to store the data but handle it as realtime flow.

spatialInterface.writePublicData("nodeName", "key", anyValue, false);

writePrivateData(node, valueName, value)

  • node (String) name of Node
  • valueName (String) writePrivateData is defined by the Node itself. This is a key to select an object within private Data.
  • value (Object/String/Number) Data you want to store with the key.

Unlike publicData, privateData is write-only, and its content is only read access to the edge server. Spatial Tools and the Spatial Toolbox will never have read access.

spatialInterface.writePublicData("nodeName", "key", anyValue);

readPublicData(node, valueName, value)

  • node (String) name of Node
  • valueName (String) This is a key to select an object within publicData.
  • value (Number) [Optional] If Value is undefined, this default value is used.
  • Returns (Object/String/Number) returns the stored data.

Read data from publicData once. Use this function if you need to fill your spatial Tool with all current publicData values. The last parameter is optional and defines a default value in case the key does not yet exist in the publicData.

value = spatialInterface.readPublicData("nodeName", "key", value);

addReadPublicDataListener(node, valueName, callback)

  • node (String) name of Node
  • valueName (String) This is a key to select an object within publicData.
  • callback (Function)
  • Returns value (Object/String/Number) returns the stored data.

Add a read listener for publicData to read value changes for a defined Key. These changes are synchronized with the Edge Server for as long as the Spatial Tool active. The Tool becomes inactive 3 seconds after it’s not visible anymore.

spatialInterface.addReadPublicDataListener("nodeName", "key", function(value){
var publicDataValue = value;
});

reloadPublicData()

Sends a message to the edge server to send the most recent state of publicData. You can use this to initialize your spatial Tool, since the publicDataListener only reacts to updates.

spatialInterface.reloadPublicData();
spatialInterface.addReadPublicDataListener("nodeName", "key", function(value){
    var publicDataValue = value;
});