Introduction - AlexAltea/nucleus GitHub Wiki

Disclaimer: At this stage this project could only be interesting for developers, not end-users. Not everything described below is fully implemented, parts of it are just features under early development that I consider interesting and worthy of sharing.

I started Nucleus about a year and a half ago, with the goal of providing fast retargetable emulation across different host and guest platforms.

  • Host platforms: Windows, Linux, Android, OS X, iOS.
  • Guest platforms: PlayStation 3 and PlayStation 4.

Before continuing: I am aware of the technical limitations of matching a cheap Android-based smartphone host with a PlayStation 4 guest. I simply consider that fact irrelevant at the moment. Also, at this point there are some issues with many host platforms (anything but Windows, as only the Direct3D12 backend is reliable) and the PlayStation 4 frontends aren't barely able to load ELF files (more information on this below).

CPU

The distinction between interpreters, dynamic recompilers and static recompilers that is commonly found in current emulators is, in my opinion, unnecessary. Any of these three paradigms could be used in an emulator by writing single binary translator capable of processing translation units of different size. The difference between these translation strategies (interpreters, recompilers) would arise from the different translation unit sizes. For instance:

  • Instruction: Equivalent to an interpreter.
  • Block: Equivalent to fine-grained dynamic recompiler, suitable for cases where oddly handwritten machine code might make function detection hard.
  • Function: Equivalent to a dynamic recompiler.
  • Module: Equivalent to a static recompiler.

The compiler is heavily inspired by Xenia's backends and some other books (kudos to benvanik for his awesome tips). There are not many differences aside from slightly different opcodes and some base compiler/assembler classes that are going to be reused in other compiler backends. Some parts in the emitter (e.g. constant formation) have been slightly optimized as well.

At the moment, there is no frontend that generates enough information to make a per-module translation possible. But some essential parts are, like frontend analyzers. The way in which such a translation would be implemented involves the detection of all functions, followed by an analysis pass on each function monitoring register usage and comparing it with the guest ABI to determine the argument/return types of the function. More information about this:

The frontends (e.g. PowerPC) also includes an JIT assembler to generate guest machine code required for testing. Unlike testing in most emulators, these are actual unit tests: An initial guest state is given, a single instruction is translated and run, and the resulting state is compared to an expected state described by the guest ISA specs.

An example of how these unit tests look like can be found here: https://github.com/AlexAltea/nucleus/blob/master/tests/cpu/ppc/ppc_integer.cpp

GPU

Emulating N graphics cards (RSX for PS3, R10XX for PS4, etc.) using M different backends (Direct3D12, Vulkan, Metal, etc.), will imply writing N¡M translators. Translating through an intermediate representation (e.g. SPIR-V) like with the CPU binary translator could reduce this to just N+M translators. However, there are other tasks remaining like managing the host GPU pipeline, resources and command submission. This is solved in Nucleus with an abstract graphics library that covers all this functionality.

Thus, any guest GPU only need to interact with this library (e.g. creating abstract resources, abstract command buffers, SPIR-V based shaders) and the backends will ensure all these inputs are properly translated to something the host system can actually work with.

UI

Writing an UI is as simple as using the abstract graphics library mentioned above and handling events each platform fires (mouse, keyboard, touch, gamepad). That way we don't need big UI libraries such as Qt/wxWidgets. Aside from avoiding unnecessary bloatware, it would allow us to apply advanced UI effects. All that we need for the UI to be visible is creating a window (Win32, Xlib, Wayland, etc.) and passing its handle to the abstract graphics initializator.

Specifically, in Nucleus, the UI is made of a stack multiple screens: independent trees of widgets that are rendered to the Nucleus window. In this stack only the top screen receives the event signals. The tree of Widgets is similar to HTML, where the internal nodes are Container, and the leaves are Text or Image. Other widgets (Button, List, etc.) are defined trees of the three previously mentioned Widget types.

The Widget properties can be easily modified in a CSS fashion thanks to some user-defined literals and Length types, for instance: button.style.width = 100_px; and paragraph.style.margin.top = 2_cm;. The actual viewport coordinates are later computed while traversing down the tree with information from the window width, height, DPI, etc.

Debugger

Even with said UI library, writing a fully functional debugger that covers could be quite a complex task. The UI library is designed to provide simple interaction with installed guest apps, not to design complex development environments. To have more freedom while designing an emulator debugger, I decided to detach the debugger server (i.e. the emulator itself) from the debugger client (i.e. the application that allows accessing the emulator state).

Previously, the debugger server implemented a REST API via Mongoose (HTML server) which I documented here: http://docs.nerve.apiary.io/ Some changes that I have yet to publish, including the replacement of parts where low-latency communication is required with WebSockets (e.g. guest memory management).

The debugger client is written in HTML5 as I considered the most flexible way to go, however with the documentation anyone could write a Qt/wxWidgets frontend. The debugger client takes care of pre-processing most of the data the server transmits, even instruction disassembling via http://alexaltea.github.io/capstone.js/. The previous version was written in AngularJS 1.x and now I'm transitioning it to Angular2 + TypeScript.

Testing

Aside from the unit tests mentioned before. Following repository has been created to test the functionality of the PS3 frontend via integration tests. Each of these tests is compiled to an ELF file that is run both on the emulator and target console resulting in two output files that are compared to look for differences. Link: https://github.com/AlexAltea/ps3autotests.

Frontends

  • PlayStation 3: The PowerPC frontend is finished, tested and working perfectly (aside from Altivec extensions), the SPU frontend is at the time a work in progress. The decoder does not generate enough information at the moment to make a per-module translation possible, but it works fine on a per-function basis. The RSX frontend (GPU) also works for translating simple vertex/fragment shaders and managing the guest GPU resources. At the moment many syscalls are yet to be reversed, although the ones implemented at the moment are enough to run simple demos.

  • PlayStation 4: Having access to decrypted kernel, decrypted modules and the release of the free and open-source SDK's at https://github.com/CTurt/PS4-SDK and https://github.com/ps4dev/ps4sdk all the information required to write a PS4 emulator is there. The first attempt at Nucleus was to load the PS4 executable files as Wine does. All the technical challenges have been studied before, and it has been proven to work. The remaining part is to reverse-engineer and reimplement OS syscalls and implement a GPU frontend for the Liverpool/R10XX guest GPU. More information:

PlayStation 3 homebrew demo running in Nucleus:

Other

For further docs please check: https://github.com/AlexAltea/nucleus/tree/master/docs I probably have forgotten to mention many other things. For further discussion, please join #nucleus-emu at irc.freenode.org:6667.

TL;DR: Next-gen guests need next-gen emulators. Writing emulators from scratch everytime is a fun and instructive task, but feels like reinventing the wheel 90% of the time. Having a common infrastructure that simplifies the creation of modern platform emulators would be nice.