APrime uses multicast to broadcast messages to multiple nodes at once when used on an internal network. Any node can create tasks (in our case, initial numbers) that can be processed by others nodes.

Joining

Joining should be as simple as connecting to the multicast socket and announcing that you have arrived, just like it should be done elsewhere.

The joining node would simply need to send a JOINED message, which notifies all other nodes that it has joined.

JOINED

All other nodes should then add the IP to a local registry.

In addition, we should add an acknowledgement message of sorts that contains the number of nodes that are in the cluster. This way, since there is no actual guarantee that the multicast packet will reach all the available nodes, we can keep sending multicast packets until we've contacted all of the nodes. In addition, each node should only ever acknowledge a new node once to ensure things don't get all funky.

ACK <number of available nodes in cluster>

The node joining the cluster should open up a TCP socket for every node that replies to the JOINED message, and the acknowledgement should be send via the respective TCP socket (to ensure that we don't get the wrong number of nodes and break everything). An alternative to using the TCP socket method would just be to also include a CRC checksum with every UDP packet, and request a resend to any nodes that send bad packets.

Leaving

In the event that a node knows that it will have to leave (and it is not unexpected), the node should notify all other nodes.

The leaving node should iterate through it's list of known nodes and send a LEAVING message to each node.

LEAVING

All other nodes should then remove the IP from their table of known nodes.

Pinging

Because it is impossible to know what other nodes are attached to a socket, nodes should ping each other every few minutes.

The initial node should iterate through all of the known nodes with PING through their respective sockets.

PING

All other nodes should reply with a PONG message, which allows those listening to the socket to rebuild their internal registries. Nodes which do not respond to a ping should be considered unreachable after a specified timeout.

PONG

In the event that the network becomes crowded, a separate socket should be created on the primary node and the PONG messages should be sent directly to that node. (Chandler: I don't really know if this would actually become an issue, with the design it looks like we're leaning toward the data being sent to each node should be rather minimal)

Tasks

APrime depends on individual tasks to be created which are distributed throughout the nodes. All tasks have a unique ID and task status that is associated with them.

All task information should be sent using TASK messages:

TASK [id] [status]

Task status

Each task should have a status attached to it.

1. New - The task was just created
2. Started - The task has been started and does not need any more nodes to handle it.
3. Aborted - The task generated an error and had to be aborted.
4. Finished - The task finished.

New tasks

New tasks should be broadcast from one node to all of the other node's respective TCP sockets. The only information they should carry is a unique task ID and the socket which the rest of the information will be broadcast on. The creators of the task are responsible for distributing the task information.

The initial creation message should be a TASK message.

TASK [id] 1 [socket]

Nodes interested in taking the task should connect to the sender on the specified socket via TCP. It is up to the sender to determine how many nodes should be able to handle a task. Tasks are sent as serialized Task objects via an Object[Input/Output]Stream.

Important: New task notices should be limited once nodes stop responding in a timely manner. The exact time is up to the sender, but tasks should not be created twice as fast as they are being completed.

Task started

When a task is marked as started, the socket which was created for distributing the task should be closed. At this point, no other nodes should attempt to take the task.

TASK [id] 2

Task aborted

In the event that the task generates an error, a notification should be broadcasted which notifies all nodes of the issue.

TASK [id] 3

Task finished

When a task finishes, the carrying node should broadcast over the network that the task has been completed. A socket should be opened which allows a TaskResult to be taken by anyone who is interested in the result.

TASK [id] 4 [socket]

Nodes interested in receiving the result should connect to the sender on the specified socket via TCP. The result would then be distributed through serialized TaskResult objects using an Object[Input/Output]Stream.