p5.strands Overview

p5.strands is a shader programming environment that p5.js provides, starting from p5 version 2. It aims to put the visual opportunities uniquely afforded by shaders within reach of people familiar with p5 without a huge learning curve. It does so by letting you write shader code in JavaScript, and by letting you focus on small changes rather than requiring a full rendering pipeline. The specifics of these APIs are still changing: a valuable contribution to the project is to give us feedback on what parts are still awkward or confusing and need smoothing out!

If you’re looking to start writing p5.strands shaders yourself, take a look at our p5.strands tutorial or the examples in the reference for the p5.js base shaders. The rest of this document will describe how p5.strands works behind the scenes. If you are interested in contributing to the p5.strands, read on!

Project goals

p5.strands aims to address a number of issues at once:

• Learning shaders: traditional shader programming is notoriously difficult to get into, requiring you to learn a new language, a parallel programming paradigm, and an execution pipeline before you can truly get started. p5.strands aims to create an environment as familiar as possible for those used to non-shader programming in p5.js, and requiring you to learn incrementally only what is relevant for your task at hand. It should do so without being just a “toy” environment; you should be able to use p5.strands in your work or art in p5.
• Teaching shaders: to teach shader programming, one often ends up building scaffolding so that you may abstract away the many concepts and boilerplate code required to get to an output, and then slowly remove the scaffolding as you teach. p5.strands is built with this in mind as an alternative to building this yourself, with further development of code educational materials built in.
• Using custom shaders: while one could always provide custom shaders to extend the rendering capabilities of p5, it was difficult to integrate new shaders into the rest of p5’s systems (working with spatial transformations, lighting, etc.) without both an intimate knowledge of internal p5 code, and the time to keep up to changes to those internals between versions to keep integrations working. p5.strands as a system lets you make updates to the built-in shaders without replacing them from scratch or knowing about anything beyond the part of the pipeline you are taking over.
• Futureproofing code: today, most shaders are written for WebGL. Development on WebGL as a technology is decelerating as momentum on WebGPU, its successor, increases, where new functionality and programming paradigms are being introduced in a new shader programming language. p5.strands offers a way to target both systems with the same code to smooth the transition between technologies, and offer an openings to experiment with the new parts of WebGPU without having to completely start from scratch.

What needs feedback

The main thing you can do to help out is to test out p5.strands and let us know what parts are the hardest to understand, and what parts are the most difficult to use: what took longest to figure out, where did you get tripped up along the way, etc. We continue to iterate on the programming interface to try to make this easier.

At this point, the general technical framework has been set up (see the technical overview below), but may still include implementation bugs. Testing and reporting these bugs (and helping fix them if you are up for it!) helps the project reach maturity.

Finally, since the system is still so new and is still changing, it lacks extensive tutorials and examples. If you want to write or make videos to help people understand how to use p5.strands, or show examples illustrating the kinds of things you can make with it, reach out!

Transforms and matrices

p5.strands includes a set of functions for building and applying transformation matrices inside a shader. They are newer than the rest of p5.strands, so using them logs a console warning that points here: they are the part of the API most likely to still change, and we want to hear how they work out for you.

The functions look like the transforms you already use on the canvas:

• transform2D() creates a 3x3 matrix that can move, turn, scale, and slant 2D points, such as the texture coordinates in a filter shader. transform3D() creates a 4x4 matrix that does the same for 3D positions. Both start out as the identity transform, which leaves points exactly where they are. They are the same functions as mat3() and mat4(), and mat2() exists for plain 2x2 math, but a 2x2 matrix has no room to store a move, so there is no transform alias for it.
• translate(), rotate(), scale(), shearX(), and shearY() add a step to a transform, and rotateX(), rotateY(), rotateZ(), and rotateAxisAngle() do the same for 3D transforms. Unlike their canvas counterparts, they don’t change anything in place: each returns a new transform, so the result has to be assigned back, as in t = translate(t, 20, 0, 0). Steps combine the way they do on the canvas, so translating and then rotating turns a shape in place before moving it.
• transformPoint() applies a transform to a position, and transformNormal() applies it to a surface direction, which has to ignore the move part of the transform to stay correct.

translate(), rotate(), scale(), shearX(), shearY(), and rotateX()/rotateY()/rotateZ() share their names with the regular p5 functions, so p5.strands only takes over when their first argument is a transform created inside a shader callback, and translate(20, 0) on the canvas works the same as it always has. The rest of these functions only exist inside a shader callback. Angles inside a shader are always in radians, since angleMode() doesn’t reach inside the shader.

A typical use is placing many copies of a shape at once, giving each instance its own transform based on instanceIndex:

let myShader;
let count = 10;

function setup() {
  createCanvas(200, 200, WEBGL);
  myShader = buildMaterialShader(placeOnRing);
}

function placeOnRing() {
  worldInputs.begin();
  let angle = instanceIndex * TWO_PI / count;
  let t = transform3D();
  t = rotateY(t, angle);
  t = translate(t, 65, 0, 0);
  worldInputs.position = transformPoint(t, worldInputs.position);
  worldInputs.normal = transformNormal(t, worldInputs.normal);
  worldInputs.end();
}

function draw() {
  background(220);
  lights();
  noStroke();
  fill('red');
  shader(myShader);
  instances(count).box(18);
}

The reference pages for transform2D() and transform3D() have more examples.

What we want feedback on

The most useful thing you can do is to try the transforms out in a real sketch and let us know how it went: what you were trying to make, what took longest to figure out, and where the names or the way the functions fit together tripped you up. These functions are the newest part of p5.strands, so now is the best time to tell us what feels awkward, before the API settles.

Bugs are just as welcome. If something renders differently from what the same transforms would do on the canvas, or an error message leaves you stuck, please report it (and help fix it if you are up for it!).

Finally, since these functions are so new, there are very few examples of them in use. Sketches, write-ups, or videos showing what you have made with them help others get started, and help us see which parts need more explanation. For any of this, file an issue on GitHub or talk to us on Discord.

Helper state for addons

Addons can register helpers whose shader state follows branches and loops. This API is experimental and may change.

p5.registerStrandsState(name, { init, update, read }) returns a factory. Assign it to fn; registration installs nothing. name identifies errors. update and read are required but may be empty objects.

• init(...args) receives the factory arguments and returns { config, state }. Any config value is passed unchanged to methods. state contains scoped and global objects with at least one field overall. Fields must be Strands nodes holding float, int or bool scalars or vectors, or mat2, mat3 or mat4 matrices, for example from p5.strandsNode() or this.mat3(). In WebGPU, ivec2() and vector equal() currently generate WGSL that doesn’t compile, so int and bool vector fields have only been tested in WebGL.
• update methods receive (state, config, ...args) and return { scoped, global } with matching field names, types and sizes. Calling an update on a helper returns that helper.
• read methods receive the same arguments and may return any value. Changes to their state snapshot don’t update stored state.
• All callbacks receive the factory’s sketch as this, allowing calls such as this.mix().

Method names cannot overlap between update and read. _installGlobal is reserved, and so are push and pop: installed globally, they would hide the built-in push() and pop() that save and restore helper state. set is reserved too: p5.strands rewrites .set() calls inside if and for as hook setters, which would break a helper’s own set().

Factory calls create independent helpers. Branches merge each field from the branch taken; loop updates carry into later iterations and after the loop.

Inside a hook, push() saves all existing helpers’ scoped fields and pop() restores them. Global fields are unaffected. Balance pushes and pops within every branch, loop body and hook. Outside a build, push() and pop() save and restore drawing state as usual.

Use scoped fields for settings that apply to what is drawn next, such as a transform or material, and global fields for results that build up, such as the color drawn so far:

function sceneAddon(p5, fn) {
  fn.createScene = p5.registerStrandsState('createScene', {
    init(uv) {
      return {
        config: { uv },
        state: {
          scoped: { offset: p5.strandsNode([0, 0]), tint: p5.strandsNode([1, 1, 1]) },
          global: { color: p5.strandsNode([0, 0, 0]) }
        }
      };
    },
    update: {
      shift(state, config, x, y) {
        const offset = state.scoped.offset.add(p5.strandsNode([x, y]));
        return { scoped: { ...state.scoped, offset }, global: state.global };
      },
      tint(state, config, rgb) {
        return { scoped: { ...state.scoped, tint: p5.strandsNode(rgb) }, global: state.global };
      },
      circle(state, config, radius) {
        const d = this.length(config.uv.sub(0.5).sub(state.scoped.offset));
        const drawn = this.mix(state.global.color, state.scoped.tint, this.step(d, radius));
        return { scoped: state.scoped, global: { color: drawn } };
      }
    },
    read: { sceneColor: state => state.global.color }
  });
}
p5.registerAddon(sceneAddon);

let myShader;

function setup() {
  createCanvas(200, 200, WEBGL);
  myShader = buildMaterialShader(drawScene);
}

function drawScene() {
  pixelInputs.begin();
  let scene = createScene(pixelInputs.texCoord);
  push();
  scene.shift(0.25, 0).tint([1, 0, 0]);
  scene.circle(0.2); // red, to the right
  pop();
  scene.circle(0.1); // white, centred: pop() undid the shift and tint
  pixelInputs.color = [scene.sceneColor(), 1];
  pixelInputs.end();
}

function draw() {
  background(255);
  noStroke();
  shader(myShader);
  plane(width, height);
}

Restrictions:

• Create helpers inside a hook, before branches or loops. Use them only in that hook invocation.
• Helpers and push/pop cannot run in loop initializers, conditions or updates.
• Loop bodies cannot use break or return early while the hook has helpers.
• Both sides of ?: and logical expressions are built. Ternary branches, &&/|| operands and the RHS of &&=/||= allow reads, but not helper creation, updates or push/pop.

Calling modify() while another shader is being built throws. Errors escaping hook callbacks or thrown by begin()/end() are recorded. If caught inside modify(), the first is rethrown after its callback returns, before code generation. This does not cover errors caught between manual begin() and end() calls.

This addon defines a brush; the sketch uses two independent instances:

function brushAddon(p5, fn) {
  fn.createBrush = p5.registerStrandsState('createBrush', {
    init(start) {
      return {
        config: {},
        state: { scoped: {}, global: { color: p5.strandsNode(start) } }
      };
    },
    update: {
      paint(state, config, target, amount) {
        const mixed = this.mix(state.global.color, target, amount);
        return { scoped: {}, global: { color: mixed } };
      }
    },
    read: {
      brushColor: state => state.global.color
    }
  });
}
p5.registerAddon(brushAddon);

let myShader;

function setup() {
  createCanvas(200, 200, WEBGL);
  myShader = buildMaterialShader(twoBrushes);
}

function twoBrushes() {
  pixelInputs.begin();
  let warm = createBrush([1, 0, 0]);
  let cool = createBrush([0, 0, 1]);
  if (pixelInputs.texCoord.x > 0.5) {
    warm.paint([1, 1, 0], 0.5);
  }
  cool.paint([0, 1, 1], 0.5);
  let t = pixelInputs.texCoord.y;
  pixelInputs.color = [mix(warm.brushColor(), cool.brushColor(), t), 1];
  pixelInputs.end();
}

function draw() {
  background(255);
  noStroke();
  shader(myShader);
  plane(width, height);
}

A shared instance

modify(callback, scope, { hook, globalInstance }) begins the named hook, exposes its properties by name, runs the callback and ends the hook. globalInstance is a function requiring hook. It runs after begin(), with the sketch as this, and must return a registered helper. Its methods are installed through _installGlobal(); other factory-created helpers remain independent.

Keep brushAddon above, then replace the sketch with this shared-brush builder and sketch:

function brushShaderAddon(p5, fn) {
  fn.buildBrushShader = function (callback) {
    return this.baseMaterialShader().modify(callback, {}, {
      hook: 'getFinalColor',
      globalInstance() {
        return this.createBrush([0, 0, 0]);
      }
    });
  };
}
p5.registerAddon(brushShaderAddon);

let myShader;

function setup() {
  createCanvas(200, 200, WEBGL);
  myShader = buildBrushShader(() => {
    paint([1, 0, 0], 0.5);
    finalColor.set([brushColor(), 1]);
  });
}

function draw() {
  background(255);
  noStroke();
  shader(myShader);
  plane(width, height);
}

helper._installGlobal() exposes update/read methods under their own names until the build ends, and returns the helper. Call it inside a hook, before branches or loops. The update-method restrictions also apply.

Temporary hooks, aliases, options.hook properties and shared methods replace matching names on window, p5.prototype and p5.Graphics.prototype. Cleanup restores exact descriptors, removes newly added properties and never calls setters, even after a failed build. Non-replaceable properties, such as non-configurable getters, throw a TypeError before being written. Cleanup errors are reported only if the build otherwise succeeded.

Technical overview

At its core, p5.strands works in four steps:

1. The user writes a function in pseudo-JavaScript.
2. p5.strands transpiles that into actual JavaScript and rewrites aspects of your code.
3. The transpiled code is run. Variable modification function calls are tracked in a graph data structure.
4. p5.strands generates GLSL code from that graph.

What functionality is available in JavaScript?

We expose JavaScript versions of the following GLSL functions in strands_builtins.js:

• Trigonometry
  • acos()
  • acosh()
  • asin()
  • asinh()
  • atan()
  • atanh()
  • cos()
  • cosh()
  • degrees()
  • radians()
  • sin()
  • sinh()
  • tan()
  • tanh()
• Math
  • abs()
  • ceil()
  • clamp()
  • dFdx()
  • dFdy()
  • exp()
  • exp2()
  • floor()
  • fma()
  • fract()
  • fwidth()
  • inversesqrt()
  • log()
  • log2()
  • max()
  • min()
  • mix()
  • mod()
  • pow()
  • round()
  • roundEven()
  • sign()
  • smoothstep()
  • sqrt()
  • step()
  • trunc()
• Vector
  • cross()
  • distance()
  • dot()
  • equal()
  • faceForward()
  • length()
  • normalize()
  • notEqual()
  • reflect()
  • refract()
• Color
  • texture()
• State
  • gl_InstanceID via instanceIndex

In strands_api.js, we make the following p5 functionality available in strands shaders:

• Environment
  • width
  • height
  • mouseX
  • mouseY
  • pmouseX
  • pmouseY
  • winMouseX
  • winMouseY
  • pwinMouseX
  • pwinMouseY
  • frameCount
  • deltaTime
  • displayWidth
  • displayHeight
  • windowWidth
  • windowHeight
  • mouseIsPressed
  • millis()
• Math
  • lerp()
  • map()
• Color
  • color()
  • lerpColor()
  • red()
  • green()
  • blue()
  • alpha()
  • saturation()
  • brightness()
  • lightness()
• Random
  • random()
  • randomSeed()
  • randomGaussian()
  • noise()
  • noiseDetail()

Why pseudo-JavaScript?

The code the user writes when using p5.strands is mostly JavaScript, with some extensions. Shader code heavily encourages use of vectors, and the extensions all make this as easy in JavaScript as in GLSL.

• In JavaScript, there is not a vector data type. In p5.strands, you create vectors by creating array, e.g. myVec = [1, 0, 0]. You can’t use actual arrays in p5.strands; all arrays are fixed-size vectors.
• In JavaScript, you can only use mathematical operators like + between numbers and strings, not with vectors. In p5.strands, we allow use of these operators between vectors.
• In GLSL, you can do something called swizzling, where you can create new vectors out of the components of an existing vector, e.g. myvec.xy, myvec.bgr, or even myvec.zzzz. p5.strands adds support for this on its vectors.

When we transpile the input code, we rewrite these into valid JavaScript. Array literals are turned into function calls like vec3(1, 0, 0) which return vector class instances. These instances are wrapped in a Proxy that handles property accesses that look like swizzles, and converts them into sub-vector references. Operators between vectors like a + b are rewritten into method calls, like a.add(b).

If a user writes something like this:

baseMaterialShader().modify(() => {
  const t = uniformFloat(() => millis());
  getWorldInputs(inputs => {
    inputs.position += [20, 25, 20] * sin(inputs.position.y * 0.05 + t * 0.004);
    return inputs;
  });
});

…it gets transpiled to something like this:

baseMaterialShader().modify(() => {
  const t = uniformFloat('t', () => millis());
  getWorldInputs(inputs => {
    inputs.position = inputs.position.add(
      strandsNode([20, 25, 20]).mult(
        sin(inputs.position.y.mult(0.05).add(strandsNode(t).mult(0.004)))
      )
    );
    return inputs;
  });
});

The program graph

The overall structure of a shader program is represented by a control-flow graph (CFG). This divides up a program into chunks that need to be outputted in linear order based on control flow. A program like the one below would get chunked up around the if statement:

// Start chunk 1
let a = 0;
let b = 1;
// End chunk 1

// Start chunk 2
if (a < 2) {
  b = 10;
}
// End chunk 2

// Start chunk 3
b += 2;
return b;
// End chunk 3

flowchart TD

subgraph chunk1
  a0[let a = 0]
  b0[let b = 1]
end

subgraph chunk2
  ifstart[if a < 2]
  b1[b = 10]
  ifend[end if]
end

subgraph chunk3
  b2[b += 2]
  ret[return b]
end

chunk1-->chunk2
chunk2-->chunk3

We store the individual states that variables can be in as nodes in a directed acyclic graph (DAG). This is a fancy name that basically means each of these variable states may depend on previous variable states, and outputs can’t feed back into inputs. Each time you modify a variable, that represents a new state of that variable. For example, below, it is not sufficient to know that c depends on a and b; you also need to know which version of b it branched off from:

let a = 0;
let b = 1;
b += 1;
let c = a + b;
return c;

We can imagine giving each of these states a separate name to make it clearer. In fact, that’s what we do when we output GLSL, because we don’t need to preserve variable names.

let a_0 = 0;
let b_0 = 1;
let b_1 = b_0 + 1;
let c_0 = b_1 + a_0;
return c_0;

When we generate GLSL from the graph, we start from the variables we need to output, the return values of the function (e.g. c_0 in the example above.) From there, we can track dependencies through the DAG (in this case, b_1 and a_0). Each dependency has their own dependencies. We make sure we output the dependencies for a node before the node itself.

flowchart TB

c_0-->c_0_plus((+))
c_0_plus-->b_1
c_0_plus-->a_0
b_1-->b_1_plus((+))
b_1_plus-->b_0
b_1_plus-->n1_0[1]
b_0-->b1_1[1]
a_0-->n0[0]

Each node in the DAG belongs to a chunk in the CFG. This helps us keep track of key points in the code. If we need to, for example, generate a temporary variable at the end of an if statement, we can refer to that CFG chunk rather than whatever the last value node in the if statement happens to be.

Control flow

p5.strands has to convert any control flow that should show up in GLSL into function calls instead of JavaScript keywords. If we don’t, they run in JavaScript, and are invisible to GLSL generation. For example, if you had a loop that runs 10 times that adds 1 each time, it would output the add 1 line 10 times rather than outputting a for loop.

let a = 0;
for (let i = 0; i < 10; i++) {
  a += 2;
}
return a;

float a = 0.0;
a += 2.0;
a += 2.0;
a += 2.0;
a += 2.0;
a += 2.0;
a += 2.0;
a += 2.0;
a += 2.0;
a += 2.0;
a += 2.0;
return a;

However, once we have a function call instead of real control flow, we also need a way to make sure that when the users’ javascript subsequently references nodes that were updated in the control flow, they properly reference the modified value after the if or for and not the original value.

let a = 0;
for (let i = 0; i < 10; i++) {
  a += 2;
}
let b = a + 1;
return b;

let a = 0;
p5.strandsFor(
  () => 0,
  i => i.lessThan(10),
  i => i.add(1),

  () => {
    a = a.add(2);
  }
);
let b = a.add(1);
return b;

let a_0 = 0;

p5.strandsFor(
  // ...
);
// At this point, the final state of a is a_n

// ...but since we didn't actually run the loop,
// b still refers to the initial state of a!
let b_0 = a_0.add(1);
return b;

For that, we make the function calls return updated values, and we generate JS code that assigns these updated values back to the original JS variables. So for loops end up transpiled to something like this, inspired by the JavaScript reduce function:

let a = 0;
for (let i = 0; i < 10; i++) {
  a += 2;
}
let b = a + 1;
return b;

let a = 0;

const outputState = p5.strandsFor(
  () => 0,
  i => i.lessThan(10),
  i => i.add(1),

  // Explicitly output new state based on prev state
  (i, prevState) => {
    return { a: prevState.a.add(2) };
  },

  { a } // Pass in initial state
);
a = outputState.a; // Update reference

// b now correctly is based off of the final state of a
let b = a.add(1);
return b;

We use a special kind of node in the DAG called a phi node, something used in compilers to refer to the result of some conditional execution. In the example above, the state of a in the output state is represented by a phi node.

In the CFG, we surround chunks producing phi nodes by a BRANCH and a MERGE chunk. In the BRANCH chunk, we can initialize phi nodes, sometimes giving them initial values. In the MERGE chunk, the value of the phi node has stabilized, and other nodes can use them as a dependency.

Phi nodes in a loop work a little differently: each one holds the loop’s current state until the next state is written into it. The body callback isn’t given these phi nodes directly. At the start of each iteration, we read each one into a snapshot node, and prevState refers to those values.

For example, a swap comes out something like this:

for (...) {
  float prev_a = a;
  float prev_b = b;
  a = prev_b;
  b = prev_a;
}

A snapshot always gets its own variable, even if it’s only used once, so it keeps the value read at the start of the iteration.

GLSL generation

GLSL is currently the only output format we support, but p5.strands is designed to be able to generate multiple formats. Specifically, in WebGPU, they use the WebGPU Shading Language (WGSL). Our goal is that your same JavaScript p5.strands code can be used in WebGL or WebGPU without you having to do any modifications.

To support this, p5.strands separates out code generation into backends. A backend is responsible for converting each type of CFG chunk into a string of shader source code. We currently have a GLSL backend, but in the future we’ll have a WGSL backend too!