message_recorder - ryzom/ryzomcore GitHub Wiki

When creating client-server applications, it is generally hard to debug one program, because you need one or more external processes to run it. Moreover, these other external programs should work perfectly in order to debug the first program, but to test them you need the first program to work perfectly too! Eventually, when several processes run in parallel and interact with each other, a bug behaviour is difficult to reproduce.

The Message Recorder allows to record every message and network event (connection, disconnection...) that is managed by the NeL network engine. Then, it is easy to replay the sequence later, even if the peers (clients or servers) are not running, and to trace it with a debugger.

If your program is a service based on the NeL service framework (see NLNET::IService), all you have to do is to add the following line in the config file of the service (this is also where you type the address of the Naming Service):

Rec = "Record";

or

Rec = "Replay";

Note that in the first case, you lauch the service in the same way as usual, launching other services such as the Naming Service before, but in the second case, the service process is the only one required.

To disable recording/replaying, remove or comment out the line in the config file, or set it to:

Rec = "Off";

Besides, if your service creates some client connections using layer 3, creating CCallbackClient or CCallbackServer objects, you need to change the code. For example:

MyConnection = new CCallbackClient("OneService");

becomes

MyConnection = new CCallbackClient("OneService", IService::recordingState(), "one_service.nmr");

If your service uses layer 4 to create additionnal connections, using CNetManager::addClient() or CNetManager::addServer(), you don't need to change the code.

If your program is not based on the service framework, there is no standard config file. Consequently, you need to change the code. For example, using layer 3:

MyConnection = new CCallbackClient("OneService");

becomes

MyConnection = new CCallbackClient("OneService", CCallbackNetBase::Record, "one_service.nmr");

The type of the second argument is CCallbackNetBase::TRecordingState and can take one of these values: Off, Record, Replay.

Using layer 4, all you have to do is to pass the recording state to CNetManager::init().

In the examples above, we set the output/input file to "one_service.nmr". When using layer 4 or the service framework, each client connection or server gets a short service name, such as NS for Naming Service. The output/input filenames are built upon this short service name and the extension ".nmr". NMR stands for NeL Message Record.

Consequently, do not run several programs located in the same directory in record or replay mode at the same time. The filenames would conflict.

The messages and events are stored in plain text, so that anyone can read them.

In nel/net/callback_net_base.h, the macro USE_MESSAGE_RECORDER must be defined. In nel/net/message.h, the macro MESSAGES_PLAIN_TEXT must be defined.

• Why don't I see exactly the same outputs in the logs, in replay mode?

In replay mode, the sockets operations are only simulated, the implementation of the network engine is very different; there are no additional threads; the sending of data does not simulate the buffering delays. That is why some outputs are not displayed. Keep in mind that the message recorder's goal is to replay a sequence of users' messages to debug users' code, and not to debug NeL itself.

• When replaying a sequence, what does the following warning mean? WRN <thread_id> : L3C: No disconnection event in the replay data

It means that your program is actively disconnecting a socket, while the disconnection has not been recorded at this moment in the replay data.

• How are handled socket descriptors and socket ids in replay mode?

When replaying, the sockets ids are not identical to the recorded ones, but they work as usual. The socket descriptors (displayed in logs for information purpose) are neither valid in replay mode, nor are the "CLT" or "SRV" tags.

• Why does the replayed sequence halt and restart after a short while?

The recorder stores the number of updates between the network events. If an event occured after a moment while recording (for example in an interactive session), it will be the same when replaying it. Because it is not based on the time but on the update counter, it allows to step into the code, using a debugger.

• Why does the replayed sequence not exit at the end?

When recording, no user input is recorded. Therefore, if you exit the program by Ctrl-C, the replayed sequence will remain running.