multiplayer concepts §

this is an introductory guide designed to shows you the basics of cyxth colab starting from why you need colab in the first place, the various collaborative data types available at your disposal with real world examples and use cases. it also goes indepth on the basics user and state management and sharing presence data to elevate the colab user experience. We highly recommend reading this through before the other colab tutorials in this series.

First let’s dive into why we need colab in the first place.

Essence of colab §

You may ask ,Why do i need cyxth in the first place when i can just send and receive changes using websockets or something?.

Good question.

the short answer — consistency.

when you have multiple users editing the same piece of state conflict are guaranteed to happen and you have to ensure that the state is always consitent for all users. generally to ensure a consistent state users have to agree on what the state should look like. this is similar to what happens in git where a user has to manually merge and resolve conflicts.

For real-time multiplayer applications we need a way to automatically resolve conflicts when they arise and ensure the state is always consistent using a traditional pr and merge method will not suffice here. to ensure consistency you’ll ask.

What happens when two or more users edit text at the same index concurrently whose change will be in front of the other?, What happens when two users concurrently update and delete the same element in an array? or what happens when a user disconnects or goes offline while editing?, will there changes be reflected? what if the document has been modified when offline?. How well does my solution scale with more that two users or even how do i ensure the state shown to all users is consistent at all times?.

colab automatically resolves conflicts and ensures shared state is always consitent.

on top of that colab will handle real-time networking for you, cyxth is fast and scalable allowing upto 256 concurrent editors and any number of viewers on the same state. with offline editing capabilities ensuring no change is lost and more.

cyxth colab is based on conflict free reprecated data types (CRDTs) to ensure the state is always consitent. you dont have to know how CRDTs work or real-time networking to use cyxth. colab provides a nice API to work with various data types that looks like regular javascript objects and just works.

Getting Started §

to get started with colab first ensure you have a cyxth account. and that you have created atleast one instance with collaboration features enabled.

then install @cyxth/colab and @cyxth/core packages

bashpnpm install @cyxth/core @cyxth/colab

Authorization §

only users authorized are allowed to access your various colab states. to authorize users first generate a EdDSA key pair with openssl set the public key in cyxth console and use your private key to sign a user id that connects to your cyxth instance. Follow this quick guide to setup authorization in your backend.

once you have auth running in your backend you will be able to generate user tokens for example. assuming you have the an auth service running on local port 8902.

tsconst AUTH_URL = "http://localhost:8902/authorize";

const getToken = async (userId: string) => {
 let stream = await fetch(AUTH_URL, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ id: userId }),
  });

  let token = await stream.json();
  return token;
}

colab setup §

after getting an authorization token we can connect to cyxth and prepare our colab instance.

tsimport Cyxth from '@cyxth/core';
import Colab from '@cyxth/colab';

const APP_URL = "my-app.apps.cyxth.com";
const stateId = "our-docs";
let colab: Colab;

const multiplayer = async (token: TokenData) => {
    const cyxth = await new Cyxth(APP_URL, [Colab]).connect(token);
    const colab = await cyxth.colab();
    await colab.createOrJoin(stateId);
}

going over the code. First we get our app url from the cyxth console, the state id is analogus to a channel users on the same state can modify and interact with the state. you can have predefined states where users can only join or request to join or allow users to create any number of states. we’ll talk more about state management later in this guide.

using the auth token obtained from a backend service we connect() to cyxth and then get a colab instance with cyxth.colab. with this colab instance we are ready to add multiplayer features to our apps.

Modifying State §

to interact with the collaborative data type we use the changeContext function in colab. this returns a context that we use to access to text, counter and tree data types, more about that in few.

ts// get change context
let context = colab.changeContext();

// get a text node
let doc = context.text("doc");

// modify text
doc.insert(0,"hello world");

// listen for changes
doc.handlers.insert = (index: number, value: string, ctx: Hctx) => {
    console.log(ctx);
    // update text
}

a single state can have multiple nodes which are accessed using top level keys. in the snippet above we get a text node at the top level key “doc”, using the node we can do actions like insert, delete and replace and handle their corresponding incoming changes. handlers are generally called for both local and remote changes. each of the data type has actions and handlers for the same actions.

Data Types §

colab comes with 3 data types. text for collaborative text editing, counter an increment decrement counter and tree a json like data structure to represent any structured state. operations made to these data types are always guaranteed to be consistent across all editors. we discuss more on each below with examples.

Text §

the text node allows for collaborative text editing capabilities similar to those found in google docs. Users can modify the same piece of text by insert, delete or replace without worrying about any conflicts. this node also allows for offline editing unlike google docs so users can have bad connections or be offline and all their changes will be reflected when they are back online.

our multiplayer monaco-editor guide goes into depth of implementing a text editor with this node.

to get a text node from the change context

tslet doc: Text = context.text("doc");

the text node has only three actions

• insert(index,value) => insert text
• delete(index,length) => delete text
• replace(index, length, value) => replace text basically combining delete and insert
• value() => get value of the text node

ts// insert "hello world"
doc.insert(0, "hello world");

// get value
console.log(doc.value()); // "hello world"

// delete world
doc.delete(6,5);

// replace hello 
doc.replace(0, 6, "hi");

and the corresponding handlers are shown below.

tstype DelFragment = {index: number, length: number};

doc.handlers = {
    insert(index: number, value: string, ctx: Hctx) {
        // handle insert
    },
    delete(fragments: DelFragment[], ctx: Hctx){
        // handle delete
    },
    replace(
        fragments: DelFragment[],
        index?: number,
        value?: string,
        ctx: Hctx
    ){
        // handle replace
    }
}

handlers are triggered for both remote and local changes to the underlying text node. to know if a change is remote check isLocal flag in the handler context. when a user executes an insert it triggers the insert handler, a delete triggers the delete handler and a replace basically combines both insert and delete handlers.

you can choose to only use the replace handler or insert and delete handlers separately. You will notice that the delete handler contains multiple fragments, this happens when a user deletes text while another user inserts in between the same text concurrently. our monaco-editor example goes in depth on why.

Counter §

this is the simplest colab node. Most reactive javascript framework have a counter example to show basic reactivity this node serves to show the same but with multiplayer functionality.

a counter node has only two actions increment and decrement that do exactly that respectively. it has only one handler update that is triggered for both operations. on top of that the value method gets the counter value.

here is the simple counter example in svelte that works with multiple users and is always consistent across all users.

svelte<script>
    // ...

    let context = colab.changeContex();
    let clicks: Counter = context.counter("clicks")
    let clicksValue = clicks.value();
    clicks.handlers.update = (value:number, ctx: Hctx) => {
        clicksValue = value;
    }
</script>

<button onclick={() => clicks.decrement()}> - </button>
<p>{clicksValue}</p>
<button onclick={() => clicks.increment()}> + </button>

Tree §

the tree node allows for nested maps and lists basically allowing you to turn any application state collaborative with ease. check out our great multiplayer svg editor example for a real world example after this.

the tree node has a defination similar to the one shown below, this shows that any JSON data can be represented with the tree with any nesting levels.

tsvalue := number | string | boolean; 
element := value | list | tree ;
list := element[];
tree := Map<string, element>;

to get the tree node simply use the text() method on the change context.

tslet tasks : {state: Tasks} = context.tree("tasks");

let’s explore the tree node by creating a simple multiplayer tasks app.

Tasks App Example §

first lets define the app state. our state is simply a list of tasks.

tsinterface Task {
    title: string;
    description: string;
    done: boolean;
    date: number;
    tags: string[]
}

type Tasks = Task[];

and the actions done on the state are as follows.

tsinterface TaskActions {
    createTask(index: number, task: Task): void;
    deleteTask(index: number): void;
    updateTask(index: number, task: Partial<Task>): void;
    markAsDone(index: number, done: boolean): void;
    addTags(index: number, tags: string[]): void;
    delTag(index: number, tagIndex: number): void;
}

we are not going to create yet another todo app atleast not the ui, we assume (and believe) you are able to create one in a few minutes with all these actions in a framework of your choice or even vanilla js. we are going to focus on colab features.

mutate state §

to mutate the state we can do so directly as you could with javascript.

tslet tasks: Tasks = context.tree("tasks").state;

// add a task
tasks.push({
    title: "write docs",
    description: "write the colab docs",
    done: false,
    date: new Date().getTime(),
    tags: ["docs"]
});

// mark as done
tasks[0].done = true;

// delete a task
delete tasks[0];

// add tag 
tasks[0].tags.push("colab");

// delete a tag 
delete tasks[0].tags[1];

// update multiple keys 
tasks[0].updte({
    title: "update docs",
    description: "update the calls docs"
})

note we have to get the state before we can modify it by invoking get state() method in Tree. and modifying it directly like we could regular js object. the tree is not a regular js object though and most methods for objects will fail. for example Tree adds an insert method for lists and implements only push and pop methods.

tstasks.insert(0, {
    title: "write docs",
    description: "write the colab docs",
    done: false,
    date: new Date().getTime(),
    tags: ["docs"]
});

you can call .value() on any key to retrieve the inner value. if the key has concurrent values that have not been overwritten yet they will be shown in the concurrent key. the value key is always consistent with all users. you can still choose show the concurrent values in the ui or use them to update state.

handle incoming changes §

tree changes can be summarized by on three actions insert, update and delete. for example a list push is just an insertion at last index. a pop is delete …etc. insert is only triggered for list inserts as it exist only for lists otherwise all changes are updates and deletes.

tslet tree = context.tree("tasks");

tree.handlers = {
    insert(path: KeyPath, value: any, ctx: Hctx) {
        // handle insert 
    },
    update(path: KeyPath, value: any, ctx: Hctx) {
        // handle updates
    },
    delete(path: KeyPath,ctx: Hctx){
        // handle delete
    }
}

type KeyPath = (string|number)[];

to handle some of our actions

ts// in handlers.insert 

if (path.length === 1){
    // add new task to ui
    localTasks[path[0]] = value;
} else if (path.length === 3){
    // add a tag
    let [taskIndex,_ ,tagIndex] = path;
    localTasks[taskIndex].tags[tagIndex] = value
}

using this handlers can get messy especially for deeply nested states. would it be nice if we had predefined actions like we do in counter and text. the next section explores how we can have that instead.

named actions & handlers §

unlike counter and text where we have predefined actions and handlers for trees we have to create our own to fit our specific needs to avoid the messy built-in handlers where we have to walk the path in order to update the ui.

we start by defining our actions

tslet tree = context.tree("tasks");
let myTasks = tree.setActions((tasks: Task[]) => {
    return  {
        createTask(task: Task){
            tasks.push(task);
        },
        deleteTask(index: number){
            delete tasks[index];
        },
        updateTask(index:number, task: Partial<Task>){
            tasks[index].update(task)  
        },
        markAsDone(index: number, done: boolean){
            tasks[index].done = done;
        },
        addTag(index: number, tag: string){
            tasks[index].tags.push(tag);
        },
        delTag(index: number, tagIndex: number){
            delete tasks[index].tags[tagIndex];
        }
    }
})

the function setActions returns the same actions that you pass into it so you can now modify state like shown below. note all actions must only contain one change to state and have no side effects i.e you can not interact with the ui in setActions.

tsmyTasks.createTask({
    title: "write docs",
    description: "write the colab docs",
    done: false,
    date: new Date().getTime(),
    tags: ["docs"]
});


myTasks.markAsDone(0, true);

myTasks.deleteTask(0);

// ...

this all looks straight forward. it helps alot when defining our handlers, the handlers will look exactly similar to actions but with the added handler context. these handlers are triggered for local and remote changes with the handler context containing information about who triggered the events.

now lets define our handlers.

tstree.setHandlers({
    createTask(task: Task, ctx: Hctx){
        // add task to ui
    },
    deleteTask(index: number, ctx: Hctx){
        // remove task from ui
    },

    // ...
})

creating actions allows us to have handlers similar to the ones in counter and text. they automatically handle path filtering for you. note for list push and pop there is always a pushIndex and popIndex in the handler context. you should not directly push if the action had a push. the context also contains concurrent values if any, the userId and isLocal flag for local events.

named handlers are a great option if your application has some predefined actions i.e state is only modified in a given way. you can still combine this with the default handlers. if a change does not match any actions the default handlers are still called.

the recommended way to ensure consistency is to only update the ui and user facing data in handlers only and apply all incoming events. if you change the ui directly without passing through colab you may end up with a mismatch in the underlying tree and ui. for example if you add a task without passing through colab you will have incorrect indices when modifying the tasks further. you can use .value() on any key to get latest values.

all local changes are instant and will always be shown first ensuring instant feedback, if a remote has been applied and not handled in between edits the local change will be shown briefly before the other changes are shown.

Sharing Presence §

presence is an integral part for any real-time collaboration application. it serves to improve user experience avoid unnecessary conflicts by convey user intent. having consistency alone will render your application unusable or a pain to use. For a great collaboration experience users need to know what the other user is doing or what they are about to do and anticipate the actions.

For a real multiplayer application presence is key. when users think of multiplayer applications they think multiple cursors on the screen showing each user’s name or avatar, multiple selections and such

sharing presence with colab is done with the presence method. you can use this method to share binary, text or json data with all users in the same state. listen for presence events with the on(presence) method, the presence event will contain the userId and the sent data.

here is an example from the monaco editor tutorial.

ts// send cursors and selections
writer.onDidChangeCursorSelection((e) => {
  let {
    selectionStartColumn: selCol,
    selectionStartLineNumber: selLin,
    positionColumn: posCol,
    positionLineNumber: posLin,
  } = e.selection;

  colab.presence({ selection: [selLin, selCol, posLin, posCol] });
});

// ..
// and listen for the events
colab.on("presence", (e) => {
    if (!e.data.selection) return;
    users[e.userId].selection = e.data.selection;
})

presence does not stop with just sending simple data, you can add real-time audio or video chat, add real-time text messaging and much more, cyxth has calls and chat modules that will integrate seamlessly with your colab instance.

check out our monaco editor and svg editor tutorial on how we use presence to share cursors and selections.

Users & Channels §

colab provides the channel interface to create and manage states. the most frequent used methods are explained herecreate method will create a new state, join allows users to join an existing state, the load function is similar to join but works with offline users who are rejoining, modUsers for moderation of users you can set the level of access per user i.e editor, viewer or no access.

a full list of all channel and user management methods is available in the reference all with examples.

conclusion §

in this guide we explored colab concepts, starting from its main goal to “automatically resolve conflicts and ensure consistent state” we then looked at the three data types text, tree and counter and how they are used to modify the shared state and finally sharing user presence and user management. after this we recommend checking our monaco editor tutorial for a real world example use of text and the svg editor tutorial for tree.

-- ✌🏾