Writing a plugin

A plugin is a JavaScript file that talks to jumpcut.GG through one frozen API called jc.v1. It runs sandboxed, with no app DOM, no globals and no network of its own, so jc.v1 is the whole of what it can do.

Hello world

(() => {
  const jc = window.jc && window.jc.v1;
  if (!jc) return;

  jc.addMenuItem("toolbar", {
    label: "+ Hello",
    onClick: async () => {
      const ref = await jc.addText("hello", { x: 0.3, y: 0.4, size: 0.06 });
      await jc.setTextStyle(ref, { color: "#ffd54a", weight: 800 });
      jc.status("added a layer");
    },
  });
})();

Every call is await-able. Each one crosses the sandbox boundary as a message, so treat them all as async.

Folder plugins

A single file is fine when all you have is code. A folder adds a manifest, translations and assets:

my-plugin/
  plugin.json      name, id, version, declared hosts
  main.js          the entry point
  lang/de.json     one file per language you translate into
  lang/_dynamic.json   keys you pass to t() through a variable
{
  "id": "cards.skelly.gg",
  "name": "skelly.cards",
  "description": "Look up trading cards and put them in the scene.",
  "author": "skelly.cards",
  "version": "1.2.0",
  "api": 1,
  "minApp": "1.2.0",
  "hosts": ["api.scryfall.com", "cards.scryfall.io"]
}

Everything you own is filed under id: layer claims, stored settings, per-project data. Declare one (reverse-DNS shape, a dot is required) and you can rename the plugin later without losing a user's work. Without one the file name is used, and renaming the file loses all of it.

Your description is translated by your own lang/ files, with the English text as the key. It is the only string someone reads before enabling you.

Reading the project

await jc.canvas()        // {w, h}
await jc.texts()         // [{ref, text}]
await jc.images()        // [{ref, name, x, y, w, start, end}]
await jc.clips()         // [{start, end, title, source, keep}]
await jc.transcript()    // {segments: [{start, end, text, words}]}, null if none
await jc.sourceId()      // which recording is open

The two clocks

The most common plugin bug is reading one and writing the other.

They are different numbers. While the user works in the Composer the source clock often sits at 0:00, so a plugin that times a layer from it stacks everything at the start. Cross between them:

const sceneAt  = await jc.sceneTimeOf(sourceSeconds); // null if the scene omits it
const sourceAt = await jc.sourceTimeAt(sceneSeconds); // null outside the cut

Reading and moving each clock:

await jc.sourceTime();   await jc.sourceSeek(t);   // the recording's clock
await jc.sceneTime();    await jc.sceneSeek(t);    // the composition's
await jc.sceneDuration();                          // how long the scene runs
await jc.playhead();     await jc.seek(t);         // aliases of the SOURCE pair

playhead() and seek() are the source clock under shorter names, not a view-aware pair - they read and write the same value sourceTime() and sourceSeek() do. A plugin working in the Composer that reaches for seek() because it sounds general is writing the wrong clock, which is the bug this section exists to prevent and has shipped once. Prefer the named pair; the short names suit plugins that only ever touch the source.

A scene cut from a two-hour recording contains almost none of it, so null is the ordinary answer rather than the error case. Store moments in source seconds with the sourceId() they belong to, and convert on the way out. A re-cut then moves your display without touching your data.

Making and changing layers

const t = await jc.addText("words", { x, y, size, above });   // or below: another layer you hold
const i = await jc.addImage(url, { x, y, w, name, like, above });
const s = await jc.addShape("rect", { x, y, w, h, color, above });   // or "ellipse"; w and h are shares of the canvas

await jc.setText(ref, "new words");        // content only; styling stays the user's
await jc.setImage(ref, url, { name });     // swaps the file, keeps the transform
await jc.setTransform(ref, { x, y, rot, opacity, z, w, size, origin });   // w = image width / text box; size = text; a shape takes h too
await jc.setTextStyle(ref, { size, color, font, weight, outline, shadow, lineHeight,
                             tracking, shadowColor, shadowDist, shadowAngle,
                             shadowBlur, shadowOpacity, bg, align, reveal });
await jc.setTiming(ref, { start, end, fadeIn, fadeOut });   // scene seconds; end 0 = to the end
await jc.setAnim(ref, { x: [{t, v, ease}, …] });
await jc.remove(ref);

Positions are canvas fractions, so the same numbers hold at any output resolution. Text size is a fraction of canvas height.

Give a text layer bounds and it wraps itself. Never insert line breaks by hand, or the user cannot reflow what you wrote:

await jc.setTransform(textRef, { w: 0.42 });   // 0 clears the bounds

align is "left", "center" or "right": how the lines sit in the text's own width. That width is its bounds, or with no bounds its longest line, so one line with no bounds looks the same in all three. It never moves the layer. For a line centred in the frame, give the text bounds as wide as the frame:

await jc.setTransform(ref, { x: 0, w: 1 });
await jc.setTextStyle(ref, { align: "center" });

layers.get(id).align reads it back, and layers.copyLook carries it.

reveal writes a text out a letter or a word at a time, over a number of seconds or at a steady rate. A revealed text plays no keyframes, so use one or the other:

await jc.setTextStyle(ref, { reveal: { mode: "letter", dur: 1.2 } });   // or { mode: "word", rate: 4 }
await jc.setTextStyle(ref, { reveal: null });                           // off

origin is the point a layer scales and turns about, one of "tl" "t" "tr" "l" "c" "r" "bl" "b" "br". A text's is its top left until you say otherwise, so a centred title that grows needs "c":

await jc.setTransform(ref, { origin: "c" });   // null puts the default back

A text can carry its own background: a filled rectangle behind it that is part of the same layer, so it moves, fades, trims and duplicates with the words. Do not put a shape under a text to get one.

await jc.setTextStyle(ref, { bg: { color: "#101010", opacity: 0.8, pad: 0.35, radius: 0.2 } });
await jc.setTextStyle(ref, { bg: { canvas: true } });   // the whole frame: one text is a title card
await jc.setTextStyle(ref, { bg: { on: false } });      // off, its settings kept
await jc.setTextStyle(ref, { bg: null });               // gone

pad and radius are in font sizes. It merges like every other style: name what changes. A text that had no background gets it switched on. layers.get(id).bg reads it back whole, or null.

Smart objects

Claim a layer and it stays yours across saves, reloads and undo:

const id = await jc.layers.claim(ref, {
  kind: "card",
  data: { name: "Black Lotus", label: "Black Lotus", laneColor: "#b57cf2", group: "card:1" },
});
await jc.layers.find({ kind: "card" });   // only ever YOUR layers
await jc.layers.get(id);                  // the whole state of one
await jc.layers.getData(id);              // just your payload
await jc.layers.setData(id, { … });       // REPLACES your payload, whole
await jc.layers.copyLook(from, to);       // styling only, both layers yours
await jc.layers.order(ref, { below: other });   // or above; two of yours in one group, or in none
await jc.selection();                     // null when nothing is selected
await jc.select(ref);                     // put the user on one of yours

For a text, layers.get(id).lines is how many lines it is once the app has wrapped it in its box, and pieceLines is the same for each piece of content over time. A text is burned lines × lineHeight × round(size × canvas height) pixels tall, which is what a scroll needs to know how far to travel. textPx is its widest line as the app wraps it, in the canvas's pixels: how wide the words are, where w is the box they wrap in.

selection() answers {kind, ref, locked, id, mine, pluginKind, data}, and id, pluginKind and data are null unless mine - you never see another plugin's payload. Check locked before writing: a locked layer refuses everything with a 423, and that arrives as a throw rather than a false.

The two writers differ, and the difference bites. claim() merges - name only what changed and the rest of your payload survives, the same as setTransform, setTextStyle and setTiming. Pass replace: true when you genuinely mean "this is the whole payload now". setData() writes it whole, so anything you do not name is gone. Claim was the odd one out until a partial re-claim quietly dropped the flag marking an object as auto-built; reach for it by default and for setData only when replacing is the intent.

copyLook takes styling across and leaves identity, content and life in time alone - it is "look like that one", not "become that one". Both layers have to be yours and of the same kind.

Three keys in data the app itself reads:

A claimed layer shows a ◈ naming you. That is how the user knows it is rebuilt rather than hand-edited.

Groups

When one thing you build is several layers, make them a group. It is the app's own group: one row on the timeline that folds, moves every member in time, and has a position and an opacity of its own on top of its members'.

const id = await jc.groups.make([back, title, subtitle], {
  name: "Intro",          // the row's name; the user can rename it
  collapsed: true,        // start folded
  kind: "intro", data: { … },
  asOne: true,            // folded, a drag on any member's bar moves the group
});
await jc.groups.find({ kind: "intro" });        // [{ id, kind, data, name, n }]
await jc.groups.get(id);                        // the whole state, or null once it is gone
await jc.groups.set(id, { name, collapsed, hidden, locked, data });
await jc.groups.add(id, ref, { below: title }); // one more of your layers, placed
await jc.groups.remove(ref);                    // the layer leaves and stays in the project
await jc.groups.shift(id, 2.5);                 // seconds; answers how far it really moved
await jc.groups.setTransform(id, { x, y, opacity });   // an OFFSET on every member; opacity multiplies
await jc.groups.setAnim(id, { opacity: [{ t: 8, v: 1 }, { t: 9, v: 0 }] });   // t in scene seconds
await jc.groups.select(id);
await jc.groups.order(id, { below: otherId });  // the whole row under another group of yours; or { belowLayer: ref }
await jc.groups.ungroup(id);                    // the group goes, the layers stay
await jc.groups.delete(id);                     // the group and your layers in it go

The older group string in a claim's data (above) only makes layers select together. It gives no row, and a copy of one of them is not a second object. Prefer a real group for anything new.

Content over time

One layer can show different things at different moments: a card that changes partway through, a total that ticks down. Use segments for that, not a layer per appearance.

await jc.setSegments(ref, [
  { at: 0,  file: "a.png", label: "Black Lotus", exit:  { type: "flip", dur: 0.22 } },
  { at: 8,  file: "b.png", label: "Rancor",      enter: { type: "flip", dur: 0.22 } },
  { at: 20, blank: true },                       // a gap: nothing on screen
]);
await jc.moveSegments(ref, [0, 6, 18]);          // only the times changed
await jc.segments(ref);                          // [] when it has none, null if not a layer

Clips, markers, transcript

await jc.clips.add({ start, end, title, excerpt });  // an ordinary manual clip
await jc.clips.clear();                             // only clips YOU added
await jc.markers.add(12.5);                         // or an array
await jc.markers.list();                            // [{t, source}], everyone's
await jc.markers.clear(from, to);                   // no range = the whole source
await jc.transcriptMarks.set([{ start, end, color: "#ffd54a", label: "…" }]);
await jc.transcriptMarks.clear();                   // only yours, as with clips

Markers are the app's own, not a plugin surface bolted beside it: they click to jump and double-click to seed a clip, they save with the project, and they stay after your panel closes. A detector that only lists its findings inside its own panel is asking the user to keep it open. add() de-dupes within 0.3s of an existing marker and returns how many actually landed, so running a scan twice is harmless.

The detector's other half is the loudness envelope the app already computes for its own peak markers:

const { peaks, duration, secondsPerBucket } = await jc.audio.peaks({ n: 3000 });

peaks is a normalised 0-1 array, one bucket per secondsPerBucket. Ask for the resolution you need and do the arithmetic yourself; dead-air.js is the worked example.

Transcript marks are data, not DOM. The transcript is virtualised, so you name the stretches and the app paints them as rows scroll into view. The same marks burn into the captions.

Right-click actions get the row and the selection:

jc.addTranscriptAction({ label: "Mark as card", onClick: (d) => {
  // d = {index, start, end, text, speaker, selection}
  // d.selection is {start, end, text} when words are selected, else null
} });

Your panel

jc.setPanel({
  title: "My plugin",
  width: 320, height: 420,
  surface: "source",   // "source" | "composer" | "both" (default)
  render(root) { root.appendChild(jc.ui.button("Do it", run)); },
});
jc.showPanel();  jc.closePanel();  jc.panelOpen();  jc.dockPanel(true);
jc.addMenuItem("toolbar", { label: "My plugin", onClick: () => jc.showPanel() });
jc.addMenuItem("plugins", { label: "Do the thing", onClick: run });

jc.on("selection", () => { … });   // also: "scene", "playhead"

addMenuItem takes four targets, and three of them are contextual: "toolbar" is a real ＋ button in the composer's add group, "image" hangs off the ＋Image menu, and "canvas" is the right-click menu on empty canvas. Each of those only makes sense when the user is already somewhere specific.

"plugins" is the exception: it puts your command in the menu bar's Plugins menu, where it is reachable at any time. Use it for the commands that are not about the current selection - importing something, running a pass over the whole project, opening a report. Your items are grouped under your plugin's name, so pick labels that read as commands ("Import chapter markers"), not as your plugin's name again. Labels there are shown exactly as you write them - the app does not translate them, so if you support several languages, translate them yourself before you register.

jc.on is how a panel follows the user instead of polling. The callback takes no argument - it is a nudge, not a payload, so read back whatever you need (jc.selection(), jc.playhead()) when it fires. That is also why it survives an undo your plugin did not perform: you are re-reading the app rather than trusting a copy.

surface says where your button belongs. The app has two work surfaces and they are not interchangeable: source is the recording, its audio, its transcript and its markers; composer is the canvas, layers and the timeline. Declare the one you act on and your button appears in that panel's header. both is the default, and is right for a plugin that genuinely does both: skelly.cards places cards on the canvas and marks the transcript, so it takes a button in each.

Worth declaring even when the default would do. A plugin that reads audio and moves the source playhead, offered from the composer header, tells the user it belongs somewhere it does not.

Inside your panel the DOM is yours, and jc.ui.button/input/row build the app's real components. Docking is the app's call: it grants the rail while one of your own layers is selected, so a plugin that owns no layers (a detector, say) never docks, by design.

Remembering things

await jc.prefs.get();  await jc.prefs.set({ … });          // yours everywhere
await jc.projectData.get();  await jc.projectData.set({ … }); // yours HERE, in the project file

Both are namespaced to you and invisible to other plugins. projectData is saved in the project file and tracked by undo, so it moves with the edit and rewinds with it. Spread what is already there rather than replacing it: set({ ...raw, mine: next }).

One more kind of memory is for your panel. The user can open a panel in a window of its own, and bring it back; either way your frame is built again and your code starts from the top. Everything above survives that, since it is kept by the app. What you hold only in memory does not - so say what your panel was showing, and ask for it when you start:

jc.panelState.onSave(() => ({ view: "results", query: lastQuery }));   // plain data; asked for whenever the panel changes
const was = await jc.panelState.get();                                   // what the frame before you returned, or null
if (was && was.query) search(was.query);

The text in your fields and how far each box was scrolled are put back for you, without events being fired, so there is nothing to do for those. Use panelState for what is not on screen as text: which view you were on, what was last searched. It lasts for the session and is never saved to disk.

Network

await jc.fetchJson(url);   // parsed JSON, through the app
await jc.thumb(url);       // a data: URL you can put in an <img>
await jc.fetchImage(url);  // fetched into the project cache, returns the file name

A picture the user chooses

const pic = await jc.pickImage();          // {file, width, height, name}, or null
const cv = await jc.canvas();
const bg = await jc.fitImage(pic.file, { fit: "cover", w: cv.w, h: cv.h });   // "cover", "stretch" or "tile"
const ref = await jc.addImage({ file: bg.file }, { x: 0, y: 0, w: 1 });
await jc.setImage(ref, { file: pic.file }); // the same layer, another cached picture

pickImage() opens the app's own file chooser and stores the picture in the project's cache. Call it straight from a press in your panel. fitImage(file, {fit, w, h, tile}) draws a cached picture again into a frame of that many pixels and answers with a new cached file, which is an ordinary picture. No host is asked for: nothing is fetched.

A sound the user chooses

const snd = await jc.pickAudio();          // {file, name}, or null
const ref = await jc.addAudio({ file: snd.file }, { start: 12.5, volume: 0.8 });
await jc.layers.claim(ref, { kind: "sting" });
await jc.setAudio(ref, { start: 14 });      // moved, its length kept

pickAudio() stores a sound (mp3, wav, m4a, aac or ogg) in the project. addAudio({file}, {start, volume, name, above, below}) places it as an audio layer that plays its whole length. Claimed, it is one of your layers: layers.find lists it with layer: "audio", a group takes it, and remove(ref) removes it. It has no place or look on the canvas.

A colour the user picks off the screen

const hex = await jc.pickColor();            // "#rrggbb", or null
const swatch = jc.ui.color("#ff8a2e");        // the app's own colour control
swatch.addEventListener("change", () => console.log(swatch.value));

pickColor() starts the app's eyedropper: a crosshair, a swatch that follows the pointer with the colour under it, and the next click anywhere in the app's windows answers with the colour of the screen there, read as it is drawn. Esc answers null. jc.ui.color(value) is the control the app's own Properties use: it reads and writes value, fires input while a colour is being chosen and change when the picker closes on a new one, and its own eyedropper button calls pickColor() for you. It works from a panel in a window of its own.

A frame of the footage, as a picture

const now = await jc.sourceAt(29.96);        // {source, t}: what is on screen then, or null
const pic = await jc.grabFrame(now);         // {file, width, height}, or null
const ref = await jc.addImage({ file: pic.file }, { x: 0, y: 0, w: 1 });

sourceAt(t) says which footage the topmost video layer shows at composition time t and where in it; unlike sourceTimeAt it answers for any footage of the project. grabFrame({source, t}) stores one full-size frame of a footage in the project's cache as a picture, read from the original file. It is an ordinary picture after that: fitImage makes one the frame's shape from it.

A layer of the project's footage

const all = await jc.footage();             // [{id, name, duration, width, height, active}]
const ref = await jc.addVideo({ source: all[0].id }, { from: 62, start: 12, end: 17 });
await jc.layers.claim(ref, { kind: "backdrop" });
await jc.setVideo(ref, { from: 90, volume: 0.3 });   // another part of it, with some of its sound

footage() lists the footage in the project; active marks the one the Footage viewer shows, which sourceTime() is the clock of. addVideo({source}, {from, start, end, x, y, w, h, opacity, volume, fadeIn, fadeOut, above, below}) places a video layer of it: from seconds into the footage, on the composition from start to end. It fills the frame and is silent unless you say otherwise, and a range that runs past the footage's end is moved back to fit. Claimed, it is one of your layers, listed with layer: "video". enter and exit take one of the app's own animations, {type, dur} (slideU, pop, glitch and the rest of Properties > Effects), or null for none; a plain fade is fadeIn and fadeOut. loop is how long one play of the range is, in seconds: the layer plays it again from from until end, as a layer of pieces (0 plays once). A clip from clips() carries its footage as source, so {from: c.start, loop: c.end - c.start} plays that clip for as long as the layer lasts. setVideo(ref, {...}) is its one setter: setTransform, setTiming and setAnim leave a video layer alone.

Your frame cannot originate a request. These go through the app, and the first time you reach a host it did not ship with, the user is asked, by host and for your plugin only. Declare the hosts you need in plugin.json. Grants are per plugin: two plugins reaching the same API are two separate permissions.

Speaking the user's language

jc.strings({ de: { "Insert": "Einfügen" }, ja: { "Insert": "挿入" } });
jc.t("Insert");
jc.t("added {name}", { name: card.name });
await jc.locale();   // "de", "pt-BR", …

The English string is the key. Keep {tokens} intact in every translation; a dropped token puts a literal brace on screen. Never build a sentence by joining pieces. Translate the whole thing and substitute.

Traps

Each one has a symptom that points somewhere else.

Reading the wrong clock sourceTime() looks like "the playhead" and is wrong for anything you place in the scene. Symptom: everything lands at 0:00, or nothing appears to happen. See the two clocks.
Rebuilding a list instead of carrying it forward setSegments replaces the whole list, and a piece holds more than its content: its transition, its label. Re-timing by hand with {at, text} drops the rest, and the symptom is a transition vanishing from a layer nobody edited. Use moveSegments when only the times change.
Assuming a moment belongs to the project A source second means nothing without the recording it came from. Store sourceId() alongside it, or your work lands on the wrong footage the moment the user switches sources.
Keys the checker cannot see jc.t(someVariable) is invisible to a static scan, so a string can ship English in every language while coverage reads 100%. List those keys in lang/_dynamic.json.
Hand-editing what you rebuild If your object is derived from stored state, a user's edit to it is replaced on your next rebuild. The ◈ marker warns them, but decide which layers are yours to own and leave the rest alone.

Limits and denials

The plugins folder ships a README.txt with the same reference in plain text, and the Plugins window opens it. Both bundled plugins are meant to be read: skelly.cards is a card lookup with panels, network and smart objects; dead air reads the loudness envelope, lists where the action is, and sends those peaks to the ruler, in about two hundred lines.