Visual Stream Processing System

Add

I’ve added a new tldraw tool called Stream Component (left-most icon: </> in the toolbar, shortcut key S). You can click or draw a rectangle to add a stream component to the canvas.

The component is not initialized at first - you need to select the component type from the drop-down menu. This UI will be improved to allow more discoverability, show descriptions of components, etc.

After selecting the component type, it will be initialized (started in the backend) and begin running immediately, waiting for inputs on its input port(s). There is no separate step for starting the component; it is always running.

I want the designer to have the feeling of working with regular drawings - so normal actions like moving, resizing, deleting, copying, pasting, undo, redo, etc., should work as expected.

Connect

You can connect two components by adding arrows from an output port to an input port using regular tldraw arrows.

Type check on connect

The backend where the streaming components are started is a Scala program¹. This means that the connections between components can be type-checked. If the connection can be made, the arrow turns green; otherwise, it turns red.

This can be an advantage over other visual programming systems that are usually dynamically typed.

The system still allows the connection to be made, but you can expect a runtime error if the types are not compatible. I chose to allow a more dynamic feel and enable the designer to do whatever they want (allowing rough scribbles or scrappy fiddles), even if that means entering a temporary state where the types don’t match. It is up to the designer to fix this later.

Edit

The component’s code can be edited in any text editor. I want to make it easy for the programmer to open the code directly from the canvas, so I added a context button that opens the code in a default editor.

Currently, changing the code requires a restart of the backend - that is not ideal and will be improved. However, the restart only takes a few seconds, so it is not a significant issue for now.

Logs

Instead of adding a custom view for showing logs, I decided to use an existing solution (ELK stack - ElasticSearch, Logstash, Kibana) and embed it in the tool. After all, tldraw allows embedding of any web content with iframes.

Proof of concept: Fibonacci

This is a simple example of a Fibonacci stream component. It generates the Fibonacci sequence and prints it.

What’s the big idea, anyway?

This image depicts how much of software is built today. Someone - architect, designer, senior dev - starts by drawing some diagrams. These are representations of what the code for an application should do, not the actual code.

Then developers take those diagrams and start implementing code. Most of the time, the diagrams become outdated as new information comes along.

What if the diagram can start as a sketch, but can be gradually “filled out” with the actual implemented code?

And at the same time, the diagram can be run as a program.

This is the idea behind Visual Stream Processing System (VSPS).

I’m trying to capture a feeling I have when developing streaming applications. I usually start with a high-level component graph (using Pekko Graph DSL). The components are just mocks - only the types are defined, but the graph actually runs. Then I start implementing the components one by one, slowly filling out the graph with actual code, and refactoring the shape as I gather new information about what I actually need to build. The implementation also guides the graph design by surfacing some paths I haven’t thought of before - especially error cases, edge cases, etc. But I always have a running system, even if it is not fully implemented.

At the same time I want to combine this with the feeling of freedom and speed of drawing diagrams like I do for my day job as a software architect. I work a lot with DrawIO and I like having the ability to quickly iterate, duplicate parts of the diagram, mix and match various ideas to show competing options for designs. I want to be able to do the same but with real, running software.

This is a highly experimental project that I’ve been working on in my (very limited) spare time, but I think it has potential.

If you are interested in this project, please reach out to me. I would love to hear your thoughts on it.