Welcome to the plugin development guide for MSE (Music Sheet Editor)!

This guide will help you make your own plugins for MSE (Music Sheet Editor). This wiki contains the best practices, handy tips, and detailed examples ranging from simple setups to more advanced concepts.

• Plugin Structure
• Getting Started
• Writing Your First Plugin
  • Types of Plugins
  • Background-Based Plugin
  • Menu Based Plugins
• Advanced Topics
  • Important Concepts
  • Integrating JavaScript Libraries
  • Custom Sound Effects
  • Post and Receive Messages
  • Updating an Object Texture
• Object Names

Each plugin should follow this folder structure:

my-plugin/
├── script.js         # Main plugin logic
├── README.md         # The readme file of your plugin
├── settings.json     # Custom plugin settings
├── LICENSE           # The license of you plugin. If you use external libraries then you **must** have a license that comply to it's license.
└── info.txt          # This file should contain the version of your plugin usually in a format like `{"version":1.0}`

To know how to customize settings.json for your requirements, check out: https://github.com/Michaelgde/MSE-for-gdevelop/wiki/Plugin-Settings

1. Get a copy of MSE. (You can get one here: https://michaelgd.itch.io/mse and the web version is fine too)
2. Create a new folder in your plugins/ directory.
3. Add the required files (script.js, README.md and info.txt).
4. Open your app, and open a project. Navigate to the 3 vertical dots on the top-right of the screen and click it and then click script. Copy-paste your script.js code into script textarea and close it. Now your plugin should be loaded into the tool.

NOTE: Most of the things done in this documentation can be easily implemented by using this MSE (Music Sheet Editor) plugin.

While writing your first plugin, you gotta make sure that you know GDJS (GDevelop JavaScript API) since this software is built using GDevelop.

Check out the GDJS docs here:

• https://docs.gdevelop.io/GDJS%20Runtime%20Documentation/modules/gdjs.html

Even if you don't know much about the API, you should at least grasp the basics of JavaScript, HTML and CSS.

When you're making a plugin, it usually falls into one of these two categories:

• Menu-Based Plugin
  Plugins that show up in the interface (like buttons or tools) and perform tasks when the user interacts with them.

• Background-Based Plugin
  These plugins run silently in the background and perform tasks automatically, like analyzing the chart or modifying data behind the scenes.

You can also combine both approaches to make a hybrid plugin that uses interface elements while also doing background processing. However, it’s helpful to first understand each type separately, since they require different implementations, and the menu-based plugins are typically more complex to build.

Let's start off with the Background Plugin since it's the easier one to implement. It only runs in the background and performs a specific task when the conditions are met.

Here is an example of a background plugin made by Michael:

if(runtimeScene.getOnceTriggers().triggerOnce(1)){
    registerTxt('event_display',true,false,30)
    runtimeScene.createObject('event_display').setPosition(415,34)
    runtimeScene.getObjects('event_display')[0].setLayer('Layer')
}
runtimeScene.getObjects('music_container').forEach(obj=>{if(gdjs.RuntimeObject.collisionTest(obj,runtimeScene.getObjects('play_node')[0],true)){
    runtimeScene.getObjects('event_display')[0].setString(obj.getVariables().get('data').getAsString())
}})

see events plugin source code made by Michael

Let's understand this code piece by piece

The code is wrapped in a triggers once so that it won't execute repeatedly in a loop

Note: GDevelop executes the code in a loop so if we want something to only trigger once, wrap it in a trigger once as shown in the example.

So after making sure that it only executes once, we create a text input at the x position: 415, and y position: 34.

• This is the text in which we will display the event name of the node.

runtimeScene.getObjects('music_container').forEach(obj=>{if(gdjs.RuntimeObject.collisionTest(obj,runtimeScene.getObjects('play_node')[0],true)){
    runtimeScene.getObjects('event_display')[0].setString(obj.getVariables().get('data').getAsString())
}})

This code first gets all objects named music_container (the playhead) and then for each music_container object on the screen, we would check if any of them collides with the play_node object (the node), and if it collides then we would change the string value of the text that we created before to the name of the node.

If you are wondering what the names of specific objects are, then you can skip over to Object Names to find it out

This is the more complex part. GDJS doesn't directly support creating sprites or objects in the screen other than texts and other minor things.

But we can make a menu by tricking GDJS into loading an [iframe](https://www.w3schools.com/TAGS/tag_iframe.asp) with HTML5 code by injecting HTML code into an iframe.

Here is a simple example:

document.addEventListener("keydown", function (event) {
  if (event.ctrlKey && event.shiftKey && event.key.toLowerCase() === "s") { // add a custom keybind so that you can open your menu
    if (!document.getElementById("add-your-own-unqie-iframe-id")) {
      const iframe = document.createElement("iframe");
      iframe.id = "add-your-own-unqie-iframe-id";
      iframe.style.width = "340px"; // add your own width
      iframe.style.height = "280px"; // add your own height
      iframe.style.border = "2px solid #8A2BE2"; // style, you can choose your own
      iframe.style.borderRadius = "10px";
      iframe.style.position = "fixed";
      iframe.style.top = "50%";
      iframe.style.left = "50%";
      iframe.style.transform = "translate(-50%, -50%)";
      iframe.style.background = "#2D1B5A"; // style, you can choose your own
      iframe.style.zIndex = "1000";
      document.body.appendChild(iframe);

      iframe.srcdoc = `
                <html>
                <head>
                    <title>Example</title>
                </head>
                <style>
                   /* css style */
                </style>
                <body>
                    <script>
                       // your script
                    </script>
                </body>
                </html>
          `;
      /* const iframeHtml = `` // IF USING THIS PASTE THE HTML IN HERE.

      const blob = new Blob([iframeHtml], { type: "text/html" });
      iframe.src = URL.createObjectURL(blob); 

      This also works, you can use this if you dont want to use srcdoc.

      IF YOU NEED TO SAVE SOMETHING TO LOCALSTORAGE OR OTHER SAVING SYSTEMS, THEN USE SCRDOC
      IF YOU NEED TO POST/RECEIVE MESSAGES BETWEEN PARENT (THE MAIN DOCUMENT) AND THE CHILD (THE IFRAME)
      */
    }
  }
});

This example opens up a menu when we press ctrl+shift+s. If you press it, nothing will appear other than the iframe since the HTML is just bare bones and has no elements or style.

      iframe.style.width = "340px"; // add your own width
      iframe.style.height = "280px"; // add your own height
      iframe.style.border = "2px solid #8A2BE2"; // style, you can choose your own
      iframe.style.borderRadius = "10px";
      iframe.style.position = "fixed";
      iframe.style.top = "50%";
      iframe.style.left = "50%";
      iframe.style.transform = "translate(-50%, -50%)";
      iframe.style.background = "#2D1B5A"; // style, you can choose your own
      iframe.style.zIndex = "1000";

These are the default styles that we applied to the iframe while creating it.

The basics of creating a plugin is not always helpful in more complex plugin's so, the advanced topics will help you the most in making a complex plugin.

When developing more advanced plugins for MSE (Music Sheet Editor), understanding the following key concepts will greatly help you:

• Base64 Encoding/Decoding
  • Key concept in loading external data
• DOM Manipulation
  • Important in creating User Interface
• PostMessage Communication
  • Important concept when you have to transfer data between iframe and main script.
• Resource Management and Memory Optimization
  • Crucial for a plugin to preserver stability

Sometimes you might not always be able to do complex functionalities on your own, so you would reside to using JavaScript Libraries.

Integrating libraries is an easy task because we can just do <script stc="https://cdn.deliveringsite/library.min.js"></script> and start using the library! But it requires internet connection. MSE (Music Sheet Editor) is an offline software and user's might not always be connected to the internet, so if you want to make your plugin offline, you can try to convert the library source code into a base64 data uri format and then integrate it into your project. This can be a bit hard for some people so let's understand on how to access the library faster and offline!

• First of all download the javascript file of the library. It should be in the .js file extension.

NOTE: If your library relies on multiple files to function then unfortunately you wont be able to run your library offline!

Now after you have the JavaScript source code of the library, convert it into the base64 format by either using CyberChef or Base64 Guru. If both doesn't work, then try searching for a JavaScript to Base64 converter online.

const CodeBase64 = "<your base64 encoded string>"
const decodedCode = atob(CodeBase64);
const CodeBlob = new Blob([decodedCode], {
  type: "application/javascript",
});
const CodeBlobUrl = URL.createObjectURL(CodeBlob);

This is the example code that you should have in your plugin script.

Now load the library by loading the blob in the html.

    <script src="${CodeBlobUrl}"></script>

This will load the library if everything went right.

Sometimes you might need to be able to play sound effects in your plugin since MSE (Music Sheet Editor) is about music and rythm.

Playing sound effects is not as straightforward as playing a sound in JavaScript since you don't have access to the root folder where sound and musics are located. Moreover you cant change the soundtrack of any audio in the root as well.

So to play a sound effect you need to utilize base64 here as well.

You can check out the doc where we discussed about base64 here:

• Integrating JavaScript Libraries

Here is an example plugin made by Byson94 for playing sound effects

• https://github.com/Michaelgde/music-sheet/blob/main/plugins/Node-Soundboard/script.js

To play a sound effect, first you would need an audio file like .mp3, .ogg etc.

Then convert the audio file into base64 as data uri. You can use: Base64 Guru Encode Audio for converting the audio file into base64 as data uri output format.

After you have the base64 data uri format, copy it and go to the script.js of your plugin and create a variable and store the base64 as a string in it.

For example:

const soundEffects = {
   soundefffectname = "data:audio/restofyourbase64data..."
}

Now we just have to play the sound effect!

So create this function in your code:

function playSound(soundName) {
  const sound = soundEffects[soundName]; // soundEffects is the json object and we are searching for the value name that matches soundName argument.
  if (!sound) {
    console.warn(`Sound '${soundName}' not found in soundEffects.`);
    return;
  }

  const audio = new Audio(sound); // sound will play since its in the data uri format (THE SOUND WONT PLAY IF THE BASE64 IS NOT IN DATA URI FORMAT)
  audio.play().catch((err) => {
    console.error("Audio play failed:", err);
  });
}

Now trigger playSound() and pass your sound name in for it play the sound effect.

Sometimes you might need to send messages from the iframe to the main script or vice versa.

You might need to do this when you want to send data that can only be obtained from the iframe or main script and make it accessible to the other.

Posting Messages from the Iframe to the Main Script

You can use the window.parent.postMessage() method to send a message from the iframe to the parent document:

// Inside iframe script
window.parent.postMessage({ myData: "hello from iframe" }, "*");

This sends a plain JavaScript object as a message to the parent window.

Receiving Messages in the Main Script

In your main script, add an event listener for the message event:

window.addEventListener("message", (event) => {
  // Optional: validate the origin
  // if (event.origin !== "https://your-plugin-domain.com") return;

  console.log("Message from iframe:", event.data);
  // Do something with event.data
});

To send messages to a specific iframe, you need a reference to its contentWindow:

const pluginIframe = document.getElementById("plugin-iframe");
pluginIframe.contentWindow.postMessage("hello from main script", "*");

Receiving Messages in the Iframe

Inside the iframe, listen for incoming messages like this:

window.addEventListener("message", (event) => {
  console.log("Message from parent:", event.data);
  // Do something with event.data
});

• You can send any data that can be safely cloned (e.g., strings, numbers, objects, arrays) using this method.
• For security, it's a good idea to validate the origin of incoming messages.
• The second argument to postMessage() should usually be the expected origin (like "https://example.com") instead of "*", especially in production environments.

You can update an objects texture using gdjs by modifying the texture via a data uri.

    const base64DataURI = "data:image/restofyourbase64data..."
    const image = new Image();
    image.onload = () => {
      const texture = new PIXI.Texture(new PIXI.BaseTexture(image));

      const objects = runtimeScene.getObjects("thespritename")

      objects.forEach((sprite) => {
        const width = sprite.getWidth();
        const height = sprite.getHeight();
        const zOrder = sprite.getZOrder();

        sprite.getRendererObject().texture = texture;

        sprite.setWidth(width);
        sprite.setHeight(height);
        sprite.setZOrder(zOrder);
      });
    };
    image.onerror = (e) => {
      console.error("Failed to load texture:", e);
    };
    image.src = base64DataURI;

How it works:

• const image = new Image();
  Creates a new in-memory HTML image element. It's used to load the base64-encoded image data.

• image.src = base64DataURI;
  Starts loading the image from a base64 string. This is asynchronous.

• image.onload = () => { ... }
  Once the image is fully loaded, the following code runs.

• const texture = new PIXI.Texture(new PIXI.BaseTexture(image));
  After the image is loaded, it’s turned into a PIXI BaseTexture, which holds the raw image data. That base texture is wrapped in a Texture, which can be applied to a sprite.

• sprite.getRendererObject().texture = texture;
  Replaces the sprite’s current texture with the new one you just created. This causes the sprite to visually update and show your base64 image.

• image.src = base64DataURI;
  Sets the image source to a base64-encoded data URI, which starts loading the image in memory.
  This is what actually triggers the image to begin loading. Once it's done, the onload event will fire, allowing you to use the image to create a texture.