deepstream.io

deepstream.io is an open-source server that helps applications share data and communicate in real time. It was built drawing on concepts from financial trading systems, where speed and reliability across many simultaneous connections are critical. Clients and backend services connect to a deepstream server to sync data, exchange messages, and call remote functions at high speed.

The server has three core building blocks. Records are persistent, schema-less documents: any connected client can read or update a record, and every change is pushed to all other clients and services within milliseconds. Records can reference each other and be organized into lists, which makes it possible to model relational data without a fixed schema. Events are the publish/subscribe layer: a client publishes a message on a topic, and any subscriber on that topic receives it. RPCs (remote procedure calls) let one part of a system ask another to perform a task and wait for a result; deepstream handles load balancing and routing so the caller does not need to know which server or process handles the request.

Security is built in. The server supports multiple authentication mechanisms and ships a permission language called Valve, which lets developers write rules specifying which users can read, write, or subscribe to which data.

The server is written in TypeScript and runs on Node.js. It can be installed as an npm package or run from a Docker image. Contributing to the project requires Node.js 10 or newer and TypeScript 3 or newer; cloning the repository and running npm install followed by npm start is enough to get a local server running. Unit tests and a separate end-to-end test suite are both available.

The README notes the design is inspired by financial trading technology, and the core focus is low-latency synchronization across a large number of simultaneous connections.