ApplicationDetails - WASdev/sample.batch.bonuspayout GitHub Wiki

Bonus Payout in detail

The BonusPayoutJob.xml is structured in 3 steps:

  1. The first step, generate, is a batchlet step which generates some random values (representing account balances), and writes them into a text file in CSV format.

  2. The second step, addBonus, is a chunk step which reads these values from the text file, adds an amount representing a 'bonus' to each record, and writes the updated value into a database table.

  3. The third step, validation, is a chunk step which loops through the database table updated in step 2 as well as the text file generated in step 1 validating the calculation in the second step. For each record, in confirms that the value now read from the database table corresponds to the value in the generated text file, plus the bonus amount. It also confirms that steps 1 and 2 have written the same number of records.

SimpleBonusPayoutJob - A simplified sample

We provide a simplified job definition SimpleBonusPayoutJob.xml which only includes the first two steps of the BonusPayoutJob.

Though the third step, validation, of BonusPayoutJob provides a way to force a failure and demonstrate restart, and makes the job more interesting overall, it does make the sample signficantly more complicated.

If you just want to see a simple example of a chunk step without as much application detail to filter through, the SimpleBonusPayoutJob may be a better place to start.

Note there is no separate application for SimpleBonusPayoutJob, it's just another JSL within the same WAR file as BonusPayoutJob, which would be submitted via commands like:

```
$ ./wlp/bin/batchManager submit --batchManager=localhost:9443 --trustSslCertificates --user=bob --password=bobpwd --applicationName=BonusPayout-1.0 --jobXMLName=SimpleBonusPayoutJob --jobPropertiesFile=$WLP_USER_DIR/shared/resources/runToCompletionParms.txt --wait --pollingInterval_s=2 
```

Application database tables

The application table, BONUSPAYOUT.ACCOUNT, used in the 2nd and 3rd steps, is defined by the DDL in the ddls directory, e.g. BonusPayout.derby.ddl

Job Parameters - detailed look

Parameter Default Description
numRecords 1000 Total number of records generated in step 1 and processed later
chunkSize 100 The chunk (transaction) size used in steps 2 and 3
generateFileNameRoot <None> Directory+prefix of file generated in step 1. No default defined in JSL, but there is a default in the Java logic.
dsJNDI java:comp/env/jdbc/BonusPayoutDS DataSource JNDI location for application table (the sample could be refactored so this isn't also used for container persistence as it is right now).
bonusAmount 100 Amount the “account balance” will be incremented by in step 2
tableName BONUSPAYOUT.ACCOUNT Application database table
fileEncoding <None> Char encoding used to write text file generated in step 1 and read in step 3

Important BonusPayout flows/constructs

What is the logic flow of this application?

Note:

One thing to keep in mind looking at the BonusPayout source is that some of the constructs are structured to allow for later inheritance/extension. In some cases the logic could be captured a bit more compactly if this were not the goal, so do keep this in mind in case it looks like the sample is unnecessarily complex.

For example: there was a plan was to include a variant of this job with a partitioned third step, validation. Though we haven't gotten around to it, still, this is part of the explanation for the class structure.

Feel free to use simpler constructs then, if you find the opportunity.

Some key constructs:

The number of records validated

As part of the validation performed in validate, the app checks that the number of records read from the DB (written in the 2nd step), is equal to value of the numRecords job parameter.
We leverage the checkpoint capabilities of the batch chunk step by ensuring the validation count will pick up where it left off in case of restart.

In more detail:

  • We use the step's persistent user data to hold this count of validated records.
  • We update this count with the listener ValidationCountUpdatingWriteListener, updating the persistent user data (and also the exit status) with the updated count value.
  • Finally, at the end of the step we use listener ValidationCountAfterStepListener to compare the count from the exit status (and persistent user data) with the original numRecords value.
  • Note: We decouple the reader checkpoint logic from this flow by not trying to also directly use the persistent user data in checkpointing (even though the values may be the same after each checkpoint).

Mechanism for DB query in the validate step

In this sample we issue a separate query (and get a separate ResultSet) per chunk. We use the StepContext transient user data to pass query parameters from the reader to the ChunkListener, and we also use the transient user data to pass the ResultSet back from the ChunkListener to the reader.

The ItemReader stores values into the transient user data both inopen() as well as in each checkpointInfo() invocation. The ChunkListener's beforeChunk() executes the query, and sets the ResultSet into the transient user data. The ItemReader's readItem() then iterates through this chunk's ResultSet.

CDI integration

In general, it is recommended that you use and leverage CDI in your batch applications. Besides being useful today, we intend to continue to move in the direction of better integrating with CDI and better leveraging CDI in the future.

However, since there are some gotchas here, we provide these constructs within this sample, to both show what is needed to use CDI, and to show a pattern for debugging whether CDI is or isn't being used when loading a given artifact.

Source

The sample shows that a CDI bean can be injected into a batch artifact.

First, since CDI will not inject other managed beans into your batch artifact unless the artifact itself is loaded as a CDI-managed bean, the GenerateDataBatchlet is annotated with the @Dependent "bean-definining annotation".

This tells CDI to recognize this class as a CDI-managed bean (batch artifacts are not automatically treated as managed beans otherwise).

@Dependent
public class GenerateDataBatchlet implements Batchlet, BonusPayoutConstants {
...

Now, the sample defines a field AccountType acctType, which is initialized like so:

/*
 * For CDI version of sample this will be injectable.
 */
private AccountType acctType = new CheckingAccountType();

In addition, we also define a setter injection

/*
 * Included for CDI version of sample.
 */
@Inject
public void setAccountType(@PriorityAccount AccountType acctType) {
    this.acctType = acctType;
}

and we also define within this sample a PreferredAccountType bean marked with this @PriorityAccount qualifier, (the qualifier is also defined also within this sample).

Observing effects

Since the server configuration included in the sample includes the cdi-1.2 feature, we can work through the following sequence to observe the different paths through the sample, with and without CDI:

  1. submit job with predefined server config (including cdi-1.2)

    ./wlp/bin/batchManager submit  ....
    
  2. Look in messages.log (to see account code = PREF) from PreferredAccountType

    $ tail $WLP_USER_DIR/servers/BonusPayout/logs/messages.log
    ...
    ...
    [5/23/15 18:45:46:633 EDT] 0000001e BonusPayout   I In GenerateDataBatchlet, using account code = PREF
    
  3. Edit server.xml, comment out cdi-1.2 feature, save, and wait for app to be restarted

    $ tail $WLP_USER_DIR/servers/BonusPayout/logs/messages.log
    ...
    [5/23/15 22:20:46:306 EDT] 00000028 com.ibm.ws.app.manager.AppMessageHelper    A CWWKZ0003I: The application batch-bonuspayout-application updated in 0.081 seconds.
    
  4. submit job again

    ./wlp/bin/batchManager submit  ....
    
  5. Look in messages.log again (to see account code = CHK) from CheckingAccountType

    $ tail $WLP_USER_DIR/servers/BonusPayout/logs/messages.log
    ...
    ...
    [5/23/15 22:21:27:546 EDT] 00000033 BonusPayout   I In GenerateDataBatchlet, using account code = CHK
    

Links

⚠️ **GitHub.com Fallback** ⚠️