News: We have just released the authoring system, Field, programmed by Downie, that has underpinned the majority of our artworks since 2002. Documentation and download available here.
Opening up
The technical frontier of digital art is no longer what computers can do, but how we get them to do it.
Today, digital artists can choose to work with a profusion of advanced hardware options (fast processors, graphics cards, sensors, etc) — but when it comes to software for authoring their artworks, their choices are remarkably few. With this open source release, we hope to expand the possibilities open to our field.
Marc Downie developed Field over the course of many years, always in a collaborative context — where the collaborators were not only other digital artists, but also choreographers, composers, musicians, and lighting designers. For such projects, the algorithms had to be created simultaneously with the ideas and the aesthetics, and usually under the typical time constraints of collaboration (the cost of theater time, the pressure of live performance, etc).
This meant that Downie had to devise an authoring system as flexible as it is powerful: flexible so that the artwork can be revised, recast, and reconsidered as rapidly and as radically as creative collaborations usually demand; and powerful so that new algorithms and control structures can be created from scratch rather than simply selected from a limited menu of preset options.
More than an authoring system, Field is in fact a system for creating authoring systems (a “meta authoring system,” if you like). It allows artists to fashion their own authoring environments for any given domain they are working within; and as they work they can continue to adjust and even recast their environments as the need arises.
This software demands more of you than so-called visual programming systems like MaxMSP. You must at a minimum be able to write code in the scripting language Python and have a good working knowledge of Java. To guide your way, we plan to release mini-implementations for specific kinds of projects (described below), which should provide useful examples of how the system can work for you. But we also hope that those of you in the open source community will contribute more such examples for others — while adding powerful extensions to the codebase itself.
We ourselves rely on open source software to create our works, which is why we’ve seized on this chance to give back to that community. We wish to give back to the arts community as well by offering a radically different way of conceiving, implementing, and running digital artworks. We look forward to what you make of it — or rather what you make with it.
Technical information
This — this code, this environment for writing code, and some of the code written in this environment — is what we intend to release as open source in November. We plan on using GPL 3 and we’re currently reviewing the licenses of the code that we are dependent on and have incorporated over the years. Cleaning up this highly personal codebase is taking some time, but we are committed to making the code available for November, and a few people outside of our group are currently already using and extending it.
Most of Field is written in Java, with Python (specifically Jython) used as a way to interact with it in rapid iterations — thus permitting the quick assembly and improvisatory collaborative workflow so central to our work.
Our target users may thus be closer to those for the Processing program — which we applaud as perhaps one of the first real contributions to the field for a decade or more. However, our system offers considerably more in its development interface than Processing’s conventional text editor — it includes embedded user interface elements, node-based editing, timelines, advanced language constructs, and fine-grained Mercurial integration, and more. It also features, in our opinion, a more mature idea of what programming languages need to be in order to be used for making art — keeping a small amount of highly malleable python in a nimble, open development environment, atop a large amount of Java. Finally, for those invested in Processing’s extensive libraries (which, we confess, we know very little about), there’s a standard plugin to Field that allows you write Python code that runs inside Processing’s environment. We’re committed to making the use of Processing libraries at least as technically easy as possible.
A detailed technical overview of the system, at least as it stood in 2005, may be found in the last chapter of Downie’s doctoral thesis. (There it goes by the name of “Fluid”).
More specific, and up to date, material on the system will be appearing here over the next few weeks and months. The first posting in this series , a leisurely walk through some of the features of the text editor, can be found here.
Other components
Also included in this open-source release, by choice and by necessity, are a number of relatively self contained projects that Fluid depends upon, that will also be of interest to others:
- geometry — a fast, flexible java-based OpenGL 2.0-style scenegraph library based on JOGL (from which we created the renderer for Forest and Recovered Light and many other works); A number of non-photorealistic shading techniques are found here as well.
- linear — a set of plugins for Fluid that add geometrically complex 2d parametric drawing tools, used to create the material for Breath, together with SVG and PDF import and export.
- jfbx — a library for reading FBX animations and geometry into java (and out again as xml)
- diagram — a (rather personal) java platform for software that assists in the creation of AI agents that incorporate both “reactive” and “planning” AI techniques. This is specifically useful for creating real-time computer music; but we’ve used it, and developed it, for making abstract graphics that share the concerns of music or linkages to music as well. Implementations, often highly generalized, for number of the more primitive machine-learning techniques are also to be found in this package. (Used in Loops and Enlightenment).
- context — an annotation tag library, and annotation processing factory, with classloader based bytecode augmentation, for extending the semantics of the Java programming language with a little of the flavor of dynamic programming languages. This includes implementations of lisp “advice” style “aspect orientation” and java continuations.
Example implementations
The release will contain several examples. The largest will be the open-source release of Loops. Additionally, there will be a library for 2D parametric drawing (developed for both Breath and Other Bodies, and useful for pedagogy); time-line examples (drawn from Point A –> B); and DMX lightboard control (used to control the dynamic theatrical lighting of Breath).
With the exception of the upcoming version of Loops, we’re not specifically releasing the source code to our artworks — not because it’s precious, but because it’s the code that’s typically the messiest, most fragile, most dependent on large assets and above all the first to become out of date (as lessons learned from the previous project are folded into the main codebase). In short it’s the material that is of least use to anybody else.