Unified text-based and visual programming

I’ve tried quite a few graphical programming languages over the years, such as Pure Data (pd), but having experience with more traditional text-based languages, was always left frustrated by the seemingly roundabout way of data entry. The same things that attract me to vim and Dvorak made me long for more convenient methods.

At the same time, there are some things which are much better suited to graphical programming environments. It’s easier to keep track of variables, since they just sit right in front of you. Controls can model traditional interfaces such as knobs for tweaking values, making real-time manipulation much more friendly.

Obviously, both systems have their advantages and disadvantages. While typing ‘x = y + z*2’ is much quicker and more concise than navigating multilevel menus to create discrete operators and operands in a visual system, finding exactly that right shade of indigo is much nicer with a typical palette finder rather than guess and checking with RGB triplets. Both systems are equally capable, but some tasks lend themselves more naturally to one system than the other. Being able to pick and choose which to use at any given time led me to attempt a parallel model.

In Canvasthesia I’m using Python to implement much of the higher level functionality. It’s quite a versatile language, and has the somewhat unusual ability to interpret code from an interactive console. I have a few classes which represent various types of entities, such as a Renderable entity, which startlingly enough, renders something in a scene. Adding a custom Python descriptor (EntityConnection) to a class derived from one of these base entity classes lets the system know that a particular type of entity can be connected to it.

class Test(vj.Renderable):
testlink = vj.EntityConnection(vj.EntityType.Renderable)
def __init__(self):
def Render(self):

This simple renderable entity does nothing of interest, as evidenced by its minimal Render method. However, it does contain a link to another renderable entity via its testlink attribute. Because it is an EntityConnection descriptor, it is automatically added to any instance’s corresponding visual control.

A few lines typed into the console creates a couple objects and adds them to the scene:

test = Test()
test.testlink = Test()

Although only a single Test instance was directly attached to the scene, a simple depth-first search reveals that a second actually exists. Additionally, it is evident that there is a connection between them via the first’s testlink attribute. The patchbay shows this intuitively:Patchbay test

Likewise, a new entity could be created in the visual editor and subsequently accessed by the Python console. They are simply parallel frontends manipulating the same backing objects, allowing whichever is convenient to be used at any time. It’s the best of both worlds.