Block Pipelines - laforge49/JFile GitHub Wiki
A Block Pipeline is a series of actors which process a Block and then pass it on to the next actor in the pipeline. For efficiency, BlockFlowBuffer actors are placed between these actors to add a degree of parallelism.
All but the last actor in the pipeline have a setNext method that is used in constructing the pipeline, and it is this method which inserts the intervening BlockFlowBuffer's into the pipeline.
protected BlockFlowBuffer blockFlowBuffer;
public void setNext(BlockProcessor nextInPipeline)
throws Exception {
blockFlowBuffer = new BlockFlowBuffer(getMailboxFactory().createMailbox());
blockFlowBuffer.next = nextInPipeline;
}
The BlockSource class extends JLPCActor and can be used as a base class for actors in the pipeline, though its use is not required. The setNext method is implemented by BlockSource.
The actors in the pipeline need to use an asynchronous mailbox, to ensure that they are running on separate threads. In contrast, the BlockFlowBuffer actors must use a synchronous mailbox. This greatly improves throughput, as commandeering can be used between 50% and 75% of the time to pass requests and responses to it--and commandeering is ever so much faster than passing messages between threads.
##The Finish Request
When working with a pipeline, sometimes you need to know when the pipeline is empty. You do this by sending the Finish request to the first actor in the pipeline. When you get a response, you know the pipeline is empty. (Unless, of course, you've put something in the pipeline after you sent the Finish request!)
The Finish request is targeted at the Finisher interface, and the request is passed all the way through the pipeline, so all the actors must implement the Finisher interface and the last actor in the pipeline is responsible for sending the (null) result back.
##Transactions
A transaction encapsulates a durable action, allowing you to log them and subsequently recover their effect. Transactions must implement the Transacton interface, which indirectly extends _Jid.
Eval is a Request which, when passed to a transaction, invokes the encapsulated action.
Some transactions may return a result, but this is only done when the encapsulated action is invoked on-line. When recovering from a log file, no result is returned. The TransactionResult request is used to manage this. When a transaction receives a TransactionResult request, it saves the RP object but does not persist it. Later when the transaction receives a Eval request, it returns the result using the saved RP object, if present.
_TransactionJid is a convenience class which extends Jid and implements Transaction. You can extend _TransactionJid to define your own transaction classes, so long as those transactions have no persistent data. Two other convenience classes are provided, _StringTransactionJid and _TupleTransactionJid, which extend StringJid and TupleJid respectively.
##EvaluatorActorJid and EvaluatorListJid
Transactions are grouped into a block for logging, processing and recovery. EvaluatorActorJid and EvaluatorListJid make this easy. Both actors implement the Evaluator interface, which the Transaction interface extends. And the Eval request targets Evaluator, rather than Transaction. This way the transaction processor can evaluate an entire collection with one request.
The RootJid of a transaction block holds a EvaluatorListJid, which is a list of EvaluatorActorJid's. Each EvaluatorActorJid holds a reference to a Transaction in turn. By this means we can create a list of diverse types of transactions.
##On-Line Transaction Pipeline
The on-line transaction pipeline is used to log and process transactions. If it does not already exist, the DB actor creates the on-line transaction pipeline when either the getTransactionAggregator method or the getDurableTransactionLogger method on DB is called.
This pipeline is composed with the following actors:
- TransactionAggregator,
- Serializer,
- DurableTransactionLogger and
- TransactionProcessor.
Transactions are created by sending AggregateTransaction requests to the TransactionAggregator actor. The result from processing the transaction is the result returned by this request.
##Recovery Transaction Pipeline
The recovery transaction pipeline is used to reprocess a log file. If it does not already exist, the DB actor creates the recovery transaction pipeline when the getLogReader method is called.
This pipeline is composed of the following actors:
- LogReader,
- Deserializer and
- TransactionProcessor.
##JFile Classes Referenced on this Page
BlockFlowBuffer
BlockSource
Finish
Finisher
Transaction
TransactionEval
TransactionResult
_TransactionJid
_StringTransactionJid
_TupleTransactionJid
Evaluator
EvaluatorActorJid
EvaluatorListJid
TransactionAggregator
Serializer
DurableTransactionLogger
TransactionProcessor
LogReader
Deserializer