JavaScript Modules And Vendoring Explained - edmozley/freeitsm GitHub Wiki
A plain-English guide to words that come up whenever FreeITSM adds or updates a library: dependency, npm, bundle, module, vendoring. It was written when PDF exports moved from jsPDF to PDFKit (#254), because that was the first library FreeITSM took that is a module rather than a plain script - but it applies to every library in the repository.
No knowledge of JavaScript tooling is assumed.
- Why PDFKit: PDF exports moved to PDFKit
- How FreeITSM's PDF code works: PDF writer - Developer Guide
Nobody writes everything. To make PDFs, FreeITSM uses PDFKit, a library other people wrote and maintain. FreeITSM depends on it, so PDFKit is a dependency.
Dependencies have dependencies of their own. PDFKit uses:
| Package | What it does for PDFKit |
|---|---|
| fontkit | Reads font files, and shapes text - joins Hindi letters, reorders vowel signs |
| linebreak | Knows where a line may break, in any language |
| fflate | Compresses the PDF so it is small |
| png-js | Reads PNG pictures |
| @noble/hashes, @noble/ciphers | PDF passwords and encryption |
| restructure, dfa, unicode-trie, ... | Small helpers fontkit uses |
So "add PDFKit" really means "add PDFKit and about fifteen packages it stands on". That chain is the dependency tree.
Most JavaScript libraries are published on npm (npmjs.com) - think of it as a public warehouse. Each library is a package, with a name (pdfkit), versions (0.20.2), and a list of the packages it needs.
Most JavaScript projects never copy libraries by hand. They keep a list (package.json) and a tool (npm install) fetches the whole tree from the warehouse. A second tool - a bundler - then joins it all into files a browser can load.
FreeITSM does neither, on purpose. There is no package.json, no npm install, no build step. Installing FreeITSM is: copy the files, point a web server at them, done. That is a big part of why it runs on shared hosting, on Windows with WAMP, and on servers with no internet. The price is that libraries must be prepared once, by hand, and kept in the repository. That is vendoring.
Vendoring means copying a library's files into your own repository and shipping them with your code, instead of fetching them from somewhere else when the app runs.
FreeITSM vendors everything: TinyMCE, Chart.js, pdf.js, Prism - and now PDFKit and the Noto fonts. They live under assets/js/vendor/ and assets/fonts/.
What that buys:
| Vendored (FreeITSM) | Fetched from the internet (a CDN) at run time | |
|---|---|---|
| Air-gapped / offline installs | โ Works - every file comes from the FreeITSM server | โ Breaks the moment there is no internet |
| Privacy | โ Nobody else sees who uses FreeITSM | โ Every page view tells a third party |
| Stability | โ The exact files that were tested | โ The CDN can change, move or vanish |
| Install | โ Copy the files | โ Needs outbound internet from every browser |
What it costs:
- The repository is bigger. The PDF fonts are about 10 MB.
-
Updates are manual. No tool warns that a newer version exists - which is why
VENDOR.mdlists every vendored library, its version and where it came from, and checking it is part of every release. -
Security scanners cannot see it. Tools that read
package.jsonfind nothing and report "no dependencies".VENDOR.mdis how a person can still check.
Air-gapped installs: because everything is vendored, a FreeITSM server with no internet at all makes PDFs exactly as one with internet does. The PDF writer's test checks it: every file the browser loaded must come from the FreeITSM server itself.
This is the part that changed with PDFKit.
<script src="assets/js/vendor/chart.umd.min.js"></script>The file runs and leaves its library on the page as a global - for example window.Chart. Any later script just uses Chart. Simple, and how FreeITSM loaded jsPDF (window.jspdf). Libraries built for this are often called UMD builds.
A module is a JavaScript file that says exactly what it needs and what it offers:
import { create } from 'fontkit'; // what it needs
export { PDFDocument, registerFile }; // what it offersNothing leaks onto window. Whoever wants the library asks for it by name. Modules are tidier, and most libraries are now published only this way - PDFKit's browser version among them. There is no window.PDFDocument file to drop into a <script> tag.
import { create } from 'fontkit' is a name, not a file. A bundler or npm knows where fontkit lives; a browser does not. So PDFKit's own module file, loaded as it is, fails at the first import.
A bundle is one file containing a library and its whole dependency tree, with every import already resolved inside it. Nothing is left to look up.
FreeITSM's PDFKit is exactly that: assets/js/vendor/pdfkit/pdfkit.bundle.js, about 550 KB. It was made once by esm.sh, a public service that builds such bundles from npm packages, downloaded from:
https://esm.sh/[email protected]/es2020/pdfkit.bundle.mjs
and committed. esm.sh was used once, to build the file - FreeITSM never contacts it while running.
Two details:
-
Renamed
.mjsto.js..mjsmeans "module", but IIS and older nginx do not know the extension and serve it with the wrong type, which a browser refuses to run. A browser decides whether a file is a module from how it is loaded, not from its name, so.jsis safe. pdf.js in FreeITSM was renamed for the same reason. - It is checked to be self-contained. If the bundle still imported anything from a URL, an air-gapped install would break. The update steps (section 6) include checking that.
FreeITSM's own code is plain scripts. A plain script can still load a module, with import() - a function that fetches a module and gives back what it exports:
const lib = await import('/freeitsm/assets/js/vendor/pdfkit/pdfkit.bundle.js');
const doc = new lib.PDFDocument();That is what assets/js/pdf-writer.js does, the first time somebody makes a PDF. Pages only include the writer, as a plain script; PDFKit is fetched only when it is actually needed, so a page where nobody exports a PDF never loads it.
Copying someone's code into your repository is only allowed if its licence says so. Licences fall into two families:
| Family | Examples | What it means for FreeITSM |
|---|---|---|
| Permissive | MIT, BSD, Apache 2.0, SIL Open Font License | Use, change and ship it - even commercially - as long as the copyright notice and licence text go with it |
| Copyleft | GPL, AGPL | Shipping it can require releasing your own code under the same terms |
FreeITSM is MIT. Everything in the PDFKit bundle is MIT, except one tiny helper under Apache 2.0 - both permissive. The fonts are under the SIL Open Font License, which exists for bundling fonts with software (the only real rule: the fonts may not be sold on their own). So:
-
assets/js/vendor/pdfkit/LICENSE.txtlists every package inside the bundle with its copyright line, and the licence texts. - Each font folder has its
OFL.txt.
Checked when PDFKit was chosen: PDF exports moved to PDFKit - licensing.
Because PDFKit is a bundle of many packages, updating it is not "replace one file":
-
Find the new version:
curl -s https://registry.npmjs.org/pdfkit/latest- readversion, and readdependenciesfor anything new. -
Check every licence is still permissive: for each dependency,
curl -s https://registry.npmjs.org/<name>/latestand readlicense. A copyleft one is a stop. -
Download the bundle:
https://esm.sh/pdfkit@<version>/es2020/pdfkit.bundle.mjs. -
Check it is the version you asked for (its first line is
/* esm.sh - pdfkit@<version> */) and self-contained - search it forimport(orfrom "pointing at a URL; there must be none. -
Save it as
assets/js/vendor/pdfkit/pdfkit.bundle.js, and updateLICENSE.txtand the row inVENDOR.md. -
Test:
php tests/pdf-writer.php(20 checks), then open the PDF it saves and look - Polish, Hindi, Tamil and Bengali, a table over three pages. Thenphp tests/security-findings/run.php(section F6 checks the version matchesVENDOR.md). -
Bump the writer's
?v=on the pages so browsers fetch it fresh, and export a PDF from Knowledge and from a Report Pack by hand.
TRAP: line endings. The bundle is minified but has line breaks inside its strings. Git on Windows (
core.autocrlf) converts line endings on checkout, which would change what PDFKit writes into PDFs..gitattributesmarks the file-text- never convert - so every checkout gets the exact bytes that were tested. Keep that line when the file is replaced, and check withgit show HEAD:assets/js/vendor/pdfkit/pdfkit.bundle.js | cmp - <the download>.
TRAP:
assets/js/pdf-writer.jsreads one PDFKit internal (_font.font, to ask whether a font has a character). A PDFKit update could rename it. If it does, the writer stops noticing missing characters and they show as empty boxes - the "colon in Bengali" and "เฅค" checks intests/pdf-writer.phpcatch that.
| Word | Meaning |
|---|---|
| Dependency | Code your code needs, written by somebody else |
| Dependency tree | Your dependencies, their dependencies, and so on |
| npm | The public warehouse of JavaScript packages |
| Package | One library on npm, with versions |
| Bundler | A tool that joins a library and its tree into files a browser can load |
| Bundle | The result: one file, nothing left to look up |
| Plain script / UMD | A file that puts its library on window
|
| Module / ESM | A file that says what it imports and exports; nothing on window
|
import() |
How any script loads a module, when it needs it |
| CDN | A server on the internet that hands out library files |
| Vendoring | Keeping a copy of a library in your own repository |
| Air-gapped | A server with no internet connection at all |
| Permissive / copyleft | The two families of open-source licence |
- PDF exports moved to PDFKit ยท PDF writer - Developer Guide
- Offline and air-gapped installs
-
VENDOR.md- every vendored library