Programming Patterns

Building Real-Time Collaboration Applications in Three.js


In this post we’ll look into creating a real-time collaboration applications in Three.js. If you’re working on a real-time web application or a multi-player game, I think you’ll find the approach below interesting. This approach builds on the MVC technique described in this previous post. If you’re not familiar with it, I recommend to take a look at that one before you continue reading on here.

In order to showcase the ideas in this post, I’ve created a simple real-time collaboration application. This application is based on the Three.js voxel painter example. Try opening the demo in several windows in parallel and see how the different clients get updated in real time.

demo1


Screenshot of the Demo

View demo Download source

Architecture

The architecture for this application consists of a web server, running in nodejs, and any number of web clients. The main responsibility of the server is to receive messages from each client, and forward them to the other clients. We’ll look into the server in more detail below. All the application logic resides in the client. The communication between server and client is done via websocket, which allows us to have full duplex communication. For this example I’m using the standard websocket API. If you’re working on something more professional that should support all browsers and platforms, you might want to use something like socket.io.

Server-Client Interaction

Server-Client Interaction

Web Client

The client application is structured using the MVC pattern, which allows us to decouple the user interface from the state of the application. This decoupling is key to being able to trigger changes in the state of the application from places other than the user interface, for example, from commands received from our remote server. It also makes the applications easier to test and modify.

Model

Our client has only two model classes. The first one is Voxel, which represents a voxel or cube that can be placed on a grid. It has a unique id, the cell coordinates where it should be located, type and color.

class Voxel {
    constructor(id, x, y, z, type, color) {
        this.id = id;
        this.x = x;
        this.y = y;
        this.z = z;
        this.type = type;
        this.color = color;
        this.className = 'Voxel';
    }
}

The second model is called VoxelGrid, which represents a collection of voxels. Note that this class is Observable and emits events when voxels are added, moved or removed. These events will be used by the view component to keep the user interface always up to date. The numCells parameter indicates how many cells per side the grid will have, while cellSize indicates the size of each cell.

const Observable = require('../Observable');
const Voxel = require('./Voxel');

class VoxelGrid extends Observable {
    constructor(numCells, cellSize) {
        super();
        this.numCells = numCells;
        this.cellSize = cellSize;
        this.voxels = new Map();
        this.className = 'VoxelGrid';
    }

    addVoxel(voxel) {
        this.voxels.set(voxel.id, voxel);
        this.emit('VoxelAdded', { voxel });
    }

    moveVoxel(voxel, x, y, z) {
        voxel.x = x;
        voxel.y = y;
        voxel.z = z;
        this.emit('VoxelMoved', { voxel });
    }

    removeVoxel(voxel) {
        this.voxels.delete(voxel.id);
        this.emit('VoxelRemoved', { voxel });
    }

    getVoxelById(id) {
        return this.voxels.get(id);
    }

    getNonPointerVoxelByPosition(x, y, z) {
        for (const voxel of this.voxels.values()) {
            if (voxel.type !== Voxel.Pointer && voxel.x === x && voxel.y === y && voxel.z === z) {
                return voxel;
            }
        }

        return null;
    }
}

Note that the model classes are very simple and they contain no logic related to Three.js or how things will be rendered. This not only makes for a cleaner architecture, but it will also allow us to use those very same classes on our server script.

View

The main class in our view component is MainView, which is responsible for setting up the Three.js scene and user interface, and forwarding user actions to the controller.

Main View Handling User Input

Main View Handling User Input

In addition, for each Model class that has to be rendered, we have a View Mediator class that encapsulates the interaction between model instances and their corresponding object3D in the scene. The first mediator is the VoxelViewMediator, which takes an instance of a Voxel and defines how the voxel will be rendered. If you were to decide that the voxels should be rendered using a different material or have a different shape, this would be the only part of the code that you would need to change.

import ViewMediator from './ViewMediator';
import Voxel from '../../model/Voxel';

export default class VoxelViewMediator extends ViewMediator {
    constructor(voxel) {
        super(voxel);
    }

    makeObject3D() {
        const geometry = new THREE.BoxGeometry( this.model.size, this.model.size, this.model.size );
        const mesh = new THREE.Mesh(geometry, this.getMaterialForVoxel());

        return mesh;
    }

    getMaterialForVoxel() {
        if (this.model.type === Voxel.Pointer) {
            return new THREE.MeshBasicMaterial( { color: this.model.color, opacity: 0.5, transparent: true } );
        } else {
            return new THREE.MeshPhongMaterial( { map: new THREE.TextureLoader().load( "images/" + this.getTextureForType(this.model.type)) } );
        }
    }

    getTextureForType(type) {
        switch (type) {
            case Voxel.Brick:
                return 'brick.jpg';
            case Voxel.Crate:
                return 'crate.jpg';
            case Voxel.Water:
                return 'water.jpg';
            case Voxel.Stone:
                return 'stone.jpg';
            case Voxel.Grass:
                return 'grass.jpg';
        }

        return 'brick.jpg';
    }
}

The second mediator is the VoxelGridViewMediator, which is responsible for rendering the voxel grid. It takes an instance of a VoxelGrid and registers itself as an observer of the grid object. Every time a voxel in the grid is added, moved or removed, this mediator will be notified and will perform the corresponding action to make sure that the Three.js scene reflects the latest state of the grid. This is the decoupling that I mentioned before: any part of the code can make changes on the voxel grid model and these changes will be reflected in the Three.js scene, regardless of how the changes were originated.

import ViewMediator from './ViewMediator';
import VoxelViewMediator from './VoxelViewMediator';
import Voxel from '../../model/Voxel';

export default class VoxelGridViewMediator extends ViewMediator {
    constructor(voxelGrid) {
        super(voxelGrid);
        voxelGrid.addObserver("VoxelAdded", (e) => this.onVoxelAdded(e));
        voxelGrid.addObserver("VoxelRemoved", (e) => this.VoxelRemoved(e));
        voxelGrid.addObserver("VoxelMoved", (e) => this.onVoxelMoved(e));

        const grid = this.getGridObject(voxelGrid);

        this.object3D.add(grid);
        this.objects = [];

        this.plane = this.getGridPlane();

        this.object3D.add(this.plane);
        this.objects.push(this.plane);
    }

    onVoxelMoved(e) {
        const voxel = e.voxel;

        this.setVoxelPosition(e.voxel, this.childMediators.get(voxel));
    }

    VoxelRemoved(e) {
        const voxel = e.voxel;
        const mediator = this.childMediators.get(voxel);

        this.object3D.remove(mediator.object3D);
    }

    onVoxelAdded(e) {
        const voxel = e.voxel;

        voxel.size = this.model.cellSize;

        const mediator = new VoxelViewMediator(voxel);

        this.childMediators.set(voxel, mediator);

        this.setVoxelPosition(voxel, mediator);

        this.object3D.add(mediator.object3D);

        if (voxel.type != Voxel.Pointer) {
            this.objects.push(mediator.object3D);
            mediator.object3D.cell = [voxel.x, voxel.y, voxel.z];
        }
    }

    setVoxelPosition(voxel, mediator) {
        const cube = mediator.object3D;
        const origin =  - (this.model.cellSize * this.model.numCells) / 2 + this.model.cellSize / 2;

        cube.position.x = origin + voxel.x * this.model.cellSize;
        cube.position.z = origin + voxel.y * this.model.cellSize;
        cube.position.y = this.model.cellSize / 2 + voxel.z * this.model.cellSize;
    }

    getGridCellFromWorldPosition(position) {
        const result = [];
        const origin =  - (this.model.cellSize * this.model.numCells) / 2 + this.model.cellSize / 2;

        result[0] = Math.round((position.x - origin) / this.model.cellSize);
        result[1] = Math.round((position.z - origin) / this.model.cellSize);
        result[2] = Math.round((position.y - this.model.cellSize / 2) / this.model.cellSize);

        if (result[0] >=0 && result[1] >=0 && result[2] >=0 && result[0] < this.model.numCells && result[1] < this.model.numCells && result[2] < this.model.numCells) {
            return result;
        } else {
            return null;
        }
    }

    getGridObject(voxelGrid) {
        const step = voxelGrid.cellSize;
        const size = step * voxelGrid.numCells / 2;
        const geometry = new THREE.Geometry();

        console.log(size);
        for ( let i = - size; i <= size; i += step ) {

            geometry.vertices.push( new THREE.Vector3( - size, 0, i ) );
            geometry.vertices.push( new THREE.Vector3(   size, 0, i ) );

            geometry.vertices.push( new THREE.Vector3( i, 0, - size ) );
            geometry.vertices.push( new THREE.Vector3( i, 0,   size ) );

        }

        const material = new THREE.LineBasicMaterial( { color: 0x000000, opacity: 0.2, transparent: true } );
        const lines = new THREE.LineSegments( geometry, material );

        return lines
    }

    getGridPlane() {
        const geometry = new THREE.PlaneBufferGeometry(2000, 2000);
        geometry.rotateX(-Math.PI / 2);

        const plane = new THREE.Mesh(geometry, new THREE.MeshBasicMaterial({visible: false}));

        return plane;
    }

}

Controller

The controller’s main responsibility is to receive the user input (forwarded by the view) and perform the corresponding action on the grid model. We’ll come back to the controller down below.

import MainView from '../view/MainView';
import Voxel from '../model/Voxel';
import AddVoxelCommand from '../command/AddVoxelCommand';
import MoveVoxelCommand from '../command/MoveVoxelCommand';
import RemoveVoxelCommand from '../command/RemoveVoxelCommand';
import { generateUUID } from '../util';
const randomColor = require('randomcolor');

export default class VoxelGridController {
    constructor(voxelGrid, voxelGridRemoteMediator) {
        this.voxelGrid = voxelGrid;
        this.voxelGridRemoteMediator = voxelGridRemoteMediator;
        this.view = new MainView(this, voxelGrid);
        this.view.initialize();
        this.addVoxelPointer();
        this.voxelGridRemoteMediator.initialize();
    }

    addVoxelPointer() {
        const voxelPointerCommand = new AddVoxelCommand(this.voxelGrid, generateUUID(), 0, 0, 0, Voxel.Pointer, randomColor());

        this.voxelGrid.voxelPointer = voxelPointerCommand.execute();
        this.voxelGridRemoteMediator.onCommandExecuted(voxelPointerCommand);
    }

    onCellHover(cell) {
        this.executeCommand(new MoveVoxelCommand(this.voxelGrid, this.voxelGrid.voxelPointer, cell[0], cell[1], cell[2]));
    }

    onCellClicked(cell, isShiftDown, uiSettings) {
        if (isShiftDown) {
            const voxel = this.voxelGrid.getNonPointerVoxelByPosition(cell[0], cell[1], cell[2]);

            if (voxel) {
                this.executeCommand(new RemoveVoxelCommand(this.voxelGrid, voxel));
            }
        } else {
            this.executeCommand(new AddVoxelCommand(this.voxelGrid, generateUUID(), cell[0], cell[1], cell[2], parseInt(uiSettings.type)));
        }
    }

    executeCommand(command) {
        command.execute(command);
        this.voxelGridRemoteMediator.onCommandExecuted(command);
    }
}

Real-Time Collaboration

Up to this point we have a simple, well-structured and functional Three.js application. In this section we’ll explore how we can extend this code to make the application collaborative in real time. The way to do this is surprisingly simple: every time our controller objects perform an action on the voxel grid, it will also send the action to the server. The server will forward the action to all the other clients, which will, once they receive it, run it locally. This way, as long as every action a client runs is executed locally by its peers, everyone will be in sync, and will have real-time updates and collaboration.

Command Pattern

The first thing we need to work out is how to send actions from the client to the server and vice versa. We’ll do this using the Command Pattern. Every change that a client can make on its voxel grid will be represented by a command Class. In our example we have three commands: AddVoxelCommand, MoveVoxelCommand and RemoveVoxelCommand. Each command class has all the necessary parameters in order to be able to execute the required action. Here is the code for the AddVoxelCommand class:

const Voxel = require('../model/Voxel');

class AddVoxelCommand {
    constructor(voxelGrid, id, x, y, z, type, color) {
        this.voxelGrid = voxelGrid;
        this.id = id;
        this.x = x;
        this.y = y;
        this.z = z;
        this.type = type;
        this.color = color;
        this.className = 'AddVoxelCommand';
    }

    execute() {
        const voxel = new Voxel(this.id, this.x, this.y, this.z, this.type, this.color);

        this.voxelGrid.addVoxel(voxel);
        return voxel;
    }
}

module.exports = AddVoxelCommand;

If you go back to the Controller class, you’ll see that all the changes made on the voxel grid are done through commands. This is very important for making sure that all local changes are sent to the server.

Serialization

The websocket interface can only send and receive strings. So if we want to be able to send and receive commands through our websocket channel, we need to have a way of serializing and deserializing them to and from strings. For this, I’ve defined a class called CommandSerializer, that knows how to serialize and deserialize all our commands. If you look at the code of the class, you’ll see that it doesn’t follow the Open/Closed Principle: every time we add or remove a command, we’ll need to make changes to this class. If you’re working on a more complex application, with many different types of commands, you’ll want to have a separate serializer class for each command, and use the Open/Closed Principle.

Since we want to keep our Controller class easy to understand and testable, we’ll encapsulate all the logic required for sending and receiving commands into a few new classes. The first one is called RemoteClient, whose responsibility is setting up the websocket channel and sending and receiving commands. By having only one class in the entire project interacting directly with the websocket interface, our code remains easy to test.

import Observable from '../Observable';

export default class RemoteClient extends Observable {
    constructor(uri, commandSerializer) {
        super();
        this.uri = uri;
        this.commandSerializer = commandSerializer;
    }

    connect() {
        this.ws = new WebSocket(this.uri);

        this.ws.onmessage = (event) => {
            const serializedCommand = JSON.parse(event.data);
            const command = this.commandSerializer.deserialize(serializedCommand);

            if (command) {
                this.emit('CommandReceived', command);
            }

        };

        this.ws.onopen = (event) => {
            this.emit('Connected', event);
        };
    }

    runCommand(command) {
        this.sendCommand('RUN', command);
    }

    setTerminateCommand(command) {
        this.sendCommand('ON_DISCONNECT', command);
    }

    sendCommand(type, command) {
        const serializedCommand = this.commandSerializer.serialize(command);
        const payload = { type, command: serializedCommand};

        this.ws.send(JSON.stringify(payload));
    }
}

Next we have the VoxelGridRemoteMediator class. This class acts as a bridge between the controller and remoteClient. It receives commands from the controller and forwards them to the remoteClient. It also receives commands from the remoteClient, using the Observer Pattern, and executes them.

import RemoveVoxelCommand from '../command/RemoveVoxelCommand';

export default class VoxelGridRemoteMediator {
    constructor(voxelGrid, remoteClient) {
        this.voxelGrid = voxelGrid;
        this.remoteClient = remoteClient;
        this.remoteClient.addObserver("CommandReceived", (e) => this.onCommandReceived(e));
    }

    initialize() {
        const terminationCommand = new RemoveVoxelCommand(this.voxelGrid, this.voxelGrid.voxelPointer);

        this.remoteClient.setTerminateCommand(terminationCommand);
    }

    onCommandExecuted(command) {
        this.remoteClient.runCommand(command);
    }

    onCommandReceived(command) {
        command.execute();
    }
}
Life Cycle of Commands

Life Cycle of Commands

And that’s it. Here is a diagram summarizing how all the components in the application work together. You’ll see that the MVC part of our application is mostly unaware of the real-time collaboration feature, which makes our code easy to understand, test and extend.

Flow of Data in our Application

Flow of Data in our Application

Server

The server side of the application is written in nodejs. It uses websocketserver for creating a websocket server. The main responsibility of the server is to forward commands between clients. In addition, the server has its own copy of a VoxelGrid instance, which it uses for initializing new clients. When a new client joins the server, this will send the client a list of commands for the client to run and make sure its voxel grid is sync with those of its peers. The server keeps its voxel grid instance in sync by just executing every command it receives, before forwarding it.

The approach we used is based on commands, in which our application code has to explicitly send the commands it has run to the server. This is fine for simple applications like the one we’ve created. If you’re working on a more complex application, you might notice that this approach doesn’t scale well. For such cases, an alternative approach would be to use a shared data model similar to how the Google Realtime API works.

As always, let me know if you have questions or comments.

Software Engineering
Building a Serverless Mesh Processing Microservice in Node.js
Tutorials
Mesh Manipulation Using Mean Values Coordinates in Three.js
Software Engineering
4 Top Tips For Successful Software Development