TODO Ideas - bryc/mempak GitHub Wiki
TODO
- (Very High priority) Option to put timestamp in filenames instead of hash. I find it actually more useful.
- (Very High priority) Add option to use Basic note file.
- (Very High priority) Make comment box larger, or at least resizable; feels too small to read anything of value.
- (Very High priority) Add timestamp support to the MPK Block format.
- (Medium priority) Fix bug in pixicon - top right pixel is missing.
- (Medium priority) More robust NoteName editing (deal with extension issue.)
- (Medium priority) Do a direct libultra port for comparing parsing results. Could be very insightful.
- Design test cases that libultra can repair but not MPKEdit. Improve
checkIndexes
.
- Design test cases that libultra can repair but not MPKEdit. Improve
- (High priority) Add way to change region of a save file.
- (Low priority)
IndexTable
Defrag (Rebuild) and Randomize (for debugging purposes/fun). - (Low priority) Improve the
IndexTable
diagrams a bit, with opacity and mouseover stuff. - (Low priority) Attempt 'emulator detection' of MPK files. i.e. display a message if the file originates from emulator. DexDrive too maybe.
- Just for fun. I abandoned the idea of using flags to store that info in note files, and its not reliable anyway.
Honestly everything below here is pretty unimportant or not good enough to do.
TODO (Old)
- (Medium priority) Develop heuristic detection ASCII GameCode/Company Code. Typically check if bit 7 is set. If check fails, display the hex as monospace text instead, e.g. DEADBEEF. This is simply to allow undocumented homebrew stuff to not look ugly.
- (High priority) Add Note export to the detail modal. Not hugely important but seems like quality of life improvement.
- (High priority) Incorporate deviceid bit check, and '01' bank check. This is possibly unnecessary if I do the libultra port first.
- Think about skipping the ID check entirely. Some MPK files can have a corrupt ID, but are recoverable. Perhaps a prompt such as "This file appears corrupt, but may be repairable. Attempt repair?"
TODO (Low priority)
- (Low priority) GameTests battery: Go through games and see how they react to changes to ID header bits. Possibly unnecessary now.
- (High priority) GameTests battery: Game note tests (Data Constraints, Duplicate data use between notes, etc.)
- (Low priority) Heuristics for mismatched DexDrive comments. Some files may have mismatched comments, such as wwf_no_mercy_caw_b.N64. It may be possible to find unused comments and apply them to existing save data. I'll need to first find all the affected files.
Idea box (Super low priority)
- (Low priority but interesting) Note name beautify database, e.g.
ASB '99 ROSTER
>All-Star Baseball 99: Roster
orDKRACING-TIMES
>Diddy Kong Racing: Track records
. This would be tedious. - (Low priority) Drag (or copy/paste) notes between MPKEdit instances. Could be useful, but is it even possible?
- (Low priority) Hex view: Display save data as raw hex/ascii. Currently done for note table data
- Per-game data parsing engine for displaying save variable contents.
- (Medium priority) Allow user to specify/modify output filename. Not really an issue if the user already supplies a MPK file. Perhaps, specify the name for "New.mpk" on launch?
Main Tasks
Consider grouping Delete / Export into a "..." menu
For added flexibility, I could convert the two buttons, Delete and Export, into its own dropdown menu.
- Export
- Export (Raw)
- Add to Archive
- Delete
This could facilitate certain new options like 'Add to Archive' more cleanly.
Archive / Database pane
Alright. I've had this idea for a while but it felt almost out of scope, too ambitious, but thinking about it more, I think some people could find it useful. So I want to do an outline.
The idea is a browser-based database of your save files, not bound to the 16-note limit of normal MPK paks.
The main idea is to roughly simulate the workflow of saving notes externally, but do it internally, and provide a more dedicated interface, while also allowing backup and restoring of the whole database as a ZIP file.
Other finer points:
- IndexedDB for storage (more resilient, handles binary data better)
- Store 'date archived' metadata - should be handled similarly to how date-modified is handled when saving note files.
-
- Basically, the interface switches to let you sort by date, perhaps with a 'timeago' scheme, grouping stuff
- In addition to sorting by date, filtering by game should be possible, so prior revisions of the same game are shown.
- Comments should be highly visible and perhaps allow inline-editing.
- Import/Export is extremely important - use ZIP format with zero compression(?)
-
- Ensure that all metadata is losslessly retained. A root JSON file might be required to store some stuff.
- A nice to have would be browser sync, but if this cannot be done in Brave/Opera/Chromium I will possibly scrap it.
Automatic ID repair
More insight to this issue might be obtained by a full reveng of the mempak lib.
Two users have shown examples of MPK files which do not load in MPKEdit, yet are able to repair the data through a real game. This happens because MPKEdit rejects any MPK files which fail all four checksums, while the mempak library code prompts the user to attempt a repair, with a warning to indicate that data may be lost.
MPKEdit aims to do everything as correctly as possible, and so blindly taking a risk like that would be against my design philosophy. Not to mention, I don't want to concern the user with making such a choice unless absolutely necessary. There are already parts of MPKEdit that silently repair minor issues; the user doesn't need to be concerned about operations that are necessary and 100% risk-free.
In any case, this issue appears relatively common, and it's not even the first time I've heard of a corrupt ID block getting in the way of valid data. So it looks like this issue should be fixed.
The question then becomes, how can I detect this situation, and silently repair the ID block? It can't be too complicated or deeply nested in parsing logic; this is still the initial line of defense; and it has to be rock solid, as we don't want to let any invalid files in. So what are our options?
I previously considered the parity of the ID blocks, but this doesn't appear universally reliable. The solution would have to be 100% reliable. I do have one possibility:
IndexTable sanity check
The IndexTable's structure is specific enough that it follows certain rules, that we may be able to easily exploit here.
All the 00
s that are highlighted green must be zero. The adjacent values must be either 1, 3 or 5-127. And because there's a backup of the IndexTable, we can do this test twice. Testing for the existence of 246 zero bytes alone, is potentially a useful sanity check.
Although there is one buggy DexDrive file that will fail this (1 out of ~500, so it's quite rare), so I'd have to look into that as well. Not to mention, if higher capacity MPK files ever catch on, these 'zero bytes' stop being zero. The rule is only possible when all MPK files are 32 KiB, but there's a hidden unused feature that breaks this rule, sadly.
OK so, it's not perfect, but it could be just a few extra lines.. but what about option #2?
checkIndexes
to be reusable.
Re-engineer The fact is, my checkIndexes
function already checks for zero bytes, etc. But it goes beyond that, crossreferencing with note data. While it too, will need to be changed if support for higher capacity paks is ever added, re-using this function might be the best way to help us solve this ID repair issue.
The logic is pretty simple.... If the ID area is completely corrupt, but the indexes/notes indicate valid data..., we should probably allow the file, and I guess, forcibly repair the checksum.
Mupen64Plus-Next .SRM Support
Status: Finished, pending one potential minor enhancement. A possible concern of comments being lost within this mode should also be investigated.
MPKEdit can now detect these files, and import the P1 MPK data only. I have abandoned any possibility of outputting in this format. Some games (BattleTanx) allow you to save to a different port. It might be worthwhile to add a simple "port detector", which can check certain offsets to determine which port contains the actual note. However I have no intention to support the case where multiple ports contain notes, it will be handled as an error. Again, this implementation is strictly import-only, as the code and work required for a "full" implementation is not worth it.
RetroArch's Mupen64Plus-Next emulator saves a single 290 KiB .srm
file for each game. This file allocates memory for all possible save data, even if the game does not support one or the other. Here's the structure:
Type | Offset | Size |
---|---|---|
EEPROM | 0 | 2048 |
MPK P1 | 2048 | 32768 |
MPK P2 | 34816 | 32768 |
MPK P3 | 67584 | 32768 |
MPK P4 | 100352 | 32768 |
SRAM | 133120 | 32768 |
FLASH | 165888 | 131072 |
A note about Raphnet MPK4 format: Raphnet's N64 USB adapter defines a format called MPK4, which is just four 32,768 byte MPK files concatenated together. I believe the intention is that these represent controller ports 1 to 4. This system is similar to the above RetroArch scheme, but without the additional save types. Currently MPKEdit supports this by simply loading only P1 data. I have no clue if it's possible to have data in ports 2-4, considering the design of the adapter only has one or two controller ports.
Luckily, detection is not an impossible task.
- If a file's size is exactly 131072 bytes, we assume it is MPK4.
- If a file's size is exactly 165888 bytes, we assume it is .SRM.