Using git - Underrout/LunarHelper GitHub Wiki

What is git?

Git is a very popular version control tool. It is used to keep track of changes in files in an organized way, which can be incredibly helpful in the context of a hacking project.

If you're curious what using git for hacking might look like in practice, I made a video of me setting up and using my project template for summer C3 2022 which you can check out if you want to see things in action. Fair warning, since the video was made a bit ago, there have been changes since then, but overall it should still give you a good idea of the general workflow.

Benefits of using git instead of Lunar Magic's restore system or manual backups include:

  • being able to restore the entire project instead of just the ROM
  • being able to push the project and its entire history to git repository hosting services for remote "backups"
  • smaller overall backup file size
  • figuring out which changes introduced a bug via git bisect
  • being able to decide which changes to include in a "restore point" and what additional information to include with it
  • being able to work on different aspects of your hack in isolation
  • being able to work together with other people in a more organized way than, for example, on google drive or dropbox
  • if using a git repository host, ability to create issues, pull requests, wiki pages, etc. for your hack

Git can take some time to get used to, but it is, in my opinion, very worthwhile to learn due to the sheer amount of benefits it offers.

I will not give a detailed guide to git here, as there are already plenty of tutorials and courses available online. Here is a simple guide you can read in a few minutes and here is the de facto book on anything you could possibly want to know about git. There is also a more playful and practical tutorial available in multiple languages that I found which looks useful to me. Feel free to learn using any of these or any other tutorials for that matter and keep in mind that for essentially any issue you have with git, using a search engine will almost assuredly lead you to an answer.

The rest of this chapter will assume you are somewhat familiar with git and discuss issues specific to hacking using git as opposed to using it for more general purposes.

Using git for hacking with Lunar Helper

.gitignore file

.gitignore files specify files and folders that git should not keep track of. This is important in the context of Lunar Helper, because we want git to ignore any files that are generated during Lunar Helper's builds, which includes ROMs and auxiliary files such as SPC, MW2, MWT and many other files generated by the underlying tools.

Which folders and files should be ignored depends a bit on your project's structure, for examples of .gitignore files, feel free to check out the RHR baserom's as well as my template project's. You can base your own .gitingore file on these if you want to.

In general, your goal should be to ignore any file that may change or be generated during the build process, while not ignoring any file that plays a direct role in determining what the built ROM contains (i.e. patches, sprites, blocks, levels, map16, etc. almost never make sense to ignore).

Note that ignoring specific config_*.txt files can make sense in some cases. For example, in a collab it's probably a good idea to have a config file specifically for emulator preferences which is ignored by git, so that every collaborator can specify their own emulator preferences in that untracked config file.

For most other config variables, I would recommend keeping them under version control, including the output and temp variables, since the temp variable can affect PIXI list path resolving and output should be in the same folder as lunar-monitor-config.txt, meaning their locations actually do matter in many cases.

clean and package config variables should be fine to ignore, though there's not really a huge need to do so.

Any other config variables should probably be kept under version control.

What to put in the git repository?

I would recommend keeping as much of your project as possible in the git repository, including all used tools (yes, that includes Lunar Helper/Lunar Monitor), patches, sprites, blocks, music, etc., so that when you check out a specific commit/branch, you can be sure that everything will be built exactly as it was at the time.

You can keep things like the clean SMW ROM outside the repository if you want, but there's not a huge reason to do so since you should be ignoring all ROM files via your .gitignore file anyway.

Minimizing merge conflicts

Merge conflicts are probably the scariest part of git. While they are plenty annoying in other situations, they can be especially annoying when it comes to hacking, as resolving merge conflicts between binary files is quite painful in many cases.

In a solo project, it is fairly easy to avoid pretty much all potential for merge conflicts by staying on a single branch. But since branches can be quite nice to use and you may wish to collab with other people, we have to accept some potential for merge conflicts in these cases.

Luckily, most files in an average project are likely to be text files. Merge conflicts between text files are significantly more simple to resolve than those between binary files. Resources that are usually text-based in the average project include:

  • patches
  • sprites
  • blocks
  • uberasm code
  • AddMusicK song files

You can just refer to one of the numerous guides on how to resolve text file merge conflicts in these cases.

Most other relevant files are unfortunately likely to be in a binary format, this includes:

  • levels
  • map16
  • shared palettes
  • GFX
  • ExGFX
  • overworld
  • title screen
  • credits
  • AddMusicK BRR files

Note that for map16, there is actually a conversion tool from the binary format to a text format available, see Human readable map16 format for details.

For these binary files, merge conflicts are significantly more annoying to resolve. Usually, you will have to open the conflicting versions side by side and manually merge them by hand, unless you know that you want to keep only one or the other version and don't care about discarding changes from one of the conflicting files, in which case you can just pick whichever version you want to keep and resolve the conflict that way.

In general, I would try to avoid editing any of these files in more than one branch at a time. Meaning you should not work on the same level, GFX, ExGFX, the overworld, credits, etc. in different branches at the same time. As long as you follow this rule, all merges should be pretty much automatic.

sysLMRestore

I would recommend actually disabling Lunar Magic's restore system completely if you're using git setups for your projects, if possible, since you should not need it with this workflow. To do so, open Lunar Magic and go to Options -> Restore Point Options and uncheck every box in the dialog window.

Note that if you're using git only for some of your projects, you may still want to keep the restore point system enabled for those other projects, if you would prefer not to port them over to this workflow.

Legal considerations

Note: I am not a lawyer, this is not legal advice!

Given that sharing ROMs is illegal, I again recommend ignoring them via .gitignore files. Furthermore, if you are using a remote git repository, I would recommend keeping it private, since many hacking tools and nearly all hacking resources (sprites, patches, etc.) don't come with a license and are thus theoretically illegal to publicly redistribute in any way, unless you directly get the author's permission. It would be nice if this weren't the case and we had permissive licenses for resources and tools or didn't have to worry about licensing at all, but right now this is just how it is. If you want to avoid legal liability, keep your repositories private. Of course you can always host them publicly if you want, but you are doing so at your own risk (which is probably not that big, admittedly).