Integration between XAP (Java) and Scala language is possible, since both use JVM, which makes pieces of compiled Scala code (except a few examples) usable in XAP. The main goal of the XAP-Scala project is to provide certain mechanisms to make XAP-Scala cooperation even easier.

1. Requirements

2. [Features] (#features)

3. [Constructor Based Properties] (#constructor_based_properties)

3.1. [Scala space class example] (#scala_space_class_example)

3.2. [Considerations] (#considerations)

4. [Enhanced Scala REPL] (#enhanced_scala_repl)

4.1. [Demo setup] (#demo_setup)

4.2. [The demo] (#the_demo)

4.3. [Configuration] (#configuration)

5. [Predicate Based Queries] (#predicate_based_queries)

5.1. [Retrieve object with predicate based query] (#retrieve_object_with_pbq)

5.2. [Supported queries] (#supported_queries)

5.3. [Available operations] (#available_operations)

6. [Scripting Executor] (#scripting_executor)

6.1. [Scripting configuration] (#scripting_configuration)

6.2. [Types of scripts] (#types_of_scripts)

6.3. [Scripting example] (#scripting_example)

7. [Scala Task Execution] (#scala_task_execution)

7.1. [Convenient task methods] (#convenient_task_methods)

7.2. [Task usage] (#task_usage)

8. [Exemplary project] (#exemplary_project)

8.1. [OpenSpaces Maven plugin project] (#openspaces_maven_plugin_project)

8.2. [Build & run] (#build_run)

8.3. [Used XAP Scala features] (#used_xap_scala_features)

8.4. [Building Scala and mixed Java/Scala modules] (#building_scala_mixed_modules)

XAP integrates with scala 2.11.x, current version of scala is 2.11.6. All versions from 2.11.0 up to 2.11.6 are supported by XAP. Scala basic JARs (e.g. scala-library.jar, scala-compiler.jar) that are installed along with Scala have to be located by XAP. This can be achieved in two ways: either copy the scala JARs to the $GS_HOME/lib/platform/scala/lib directory or set $SCALA_JARS variable in $GS_HOME/bin/setenv.sh (or setenv.bat) to point to directory with mentioned scala JARs.

XAP-scala runs on JDK 1.6, 1.7 and 1.8.

XAP-scala introduces the following features:

• constructor based properties - annotations that enable defining immutable properties of space objects,
• predicate based queries - more natural way of writing queries to the space,
• enhanced Scala REPL - extension of Scala REPL,
• scripting executor - feature allowing to run Scala code as a script,
• Scala task execution - simplified method of defining Scala tasks and distributed tasks.

All of them will be described in more detail in the next paragraphs.

Space classes are annotated so that analysis on their properties can be performed. By default, the properties are bean properties (the ones with corresponding getters and setters methods). The com.gigaspaces.annotation.pojo.SpaceClassConstructor annotation modifies this behavior so that properties are based on parameters defined in constructor. This allows data class properties to be immutable.

package org.openspaces.scala.example.data
import scala.beans.BeanProperty

/*
 * This imports enhanced space annotations such as @SpaceId, @SpaceRouting, etc...
 * with a @beanGetter annotation attached to them so annotations will be attached to the generated getter method.
 */
import org.openspaces.scala.core.aliases.annotation._

/**
 * Data properties are inferred from getters and setters. The pojo properties are mutable.
 */
case class MutablePropsData(@BeanProperty @SpaceId var id: String, @BeanProperty var content: String) {
    def this() = this(null, null)
}

/**
 * Data properties are inferred from constructor properties.
 * This allows the pojo properties to remain immutable as demonstrated below.
 */
case class ImmutablePropsData @SpaceClassConstructor() (
  @BeanProperty
  @SpaceId
  id: String = null,

  @BeanProperty
  content: String = null
)


val mutableData = MutablePropsData("id1", "content")
mutableData.setContent("new-content")

val immutableData = ImmutablePropsData("id2", "content")

// error -> value setContent is not a member of ImmutablePropsData
// immutableData.setContent("new-content")

// error -> reassignment to val
// immutableData.data = "new-content"

• Each property defined in the specified constructor must have a matching bean getter method. This method can be generated automatically by the compiler by using the @BeanProperty annotation. If custom behavior is required, the getter method can be written explicitly.
• To extract the names from the constructor parameters, it is required that the classes will be compiled with debug information.
• Space metadata annotations (such as @SpaceId) should be placed on the getter method. If the getter method is generated by the compiler using @BeanProperty, the annotation aliases under org.openspaces.scala.core.aliases.annotation can be used which include the @beanGetter annotation.

Following is a short demo of what can be done with the XAP scala shell. It should be noted that this shell is a regular Scala REPL with some initial imports and initialization code.

• Run $GS_HOME/bin/gs-agent.sh (or .bat)
• Start the shell $GS_HOME/tools/scala/shell.sh (or .bat)

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Initializing... This may take a few seconds.
Welcome to Scala version 2.11.6 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_45).
Type in expressions to have them evaluated.
Type :help for more information.
Please enjoy the predefined 'admin' val.

xap>

We'll start by deploying a single space to the service grid, notice we already have an admin instance in scope to simplify this process:

xap> val gsm = admin.getGridServiceManagers.waitForAtLeastOne
gsm: org.openspaces.admin.gsm.GridServiceManager = org.openspaces.admin.internal.gsm.DefaultGridServiceManager@ca43aa97

xap> gsm.deploy(new org.openspaces.admin.space.SpaceDeployment("mySpace"))
res0: org.openspaces.admin.pu.ProcessingUnit = org.openspaces.admin.internal.pu.DefaultProcessingUnit@59479eba

We'll use some helper method that is imported into the session scope (from org.openspaces.scala.repl.GigaSpacesScalaReplUtils) to get a GigaSpace proxy:

xap> val Some(gigaSpace) = getGigaSpace("mySpace")
gigaSpace: org.openspaces.core.GigaSpace = mySpace_container:mySpace

Now we'll execute a task using another helper method:

xap> execute(gigaSpace) { holder =>
     | holder.clusterInfo.getNumberOfInstances
     | }
res1: com.gigaspaces.async.AsyncFuture[Integer] = org.openspaces.core.transaction.internal.InternalAsyncFuture@f1423ba

xap> val numberOfInstances = res1.get
numberOfInstances: Integer = 1

Let's define a new case class and write an entry to the space:

xap> case class Data @SpaceClassConstructor() (@BeanProperty @SpaceId id: String = null, @BeanProperty content: String = null)
defined class Data

xap> gigaSpace.write(Data(id = "id1", content = "my data content"))
res2: com.j_spaces.core.LeaseContext[Data] = SpaceEntryLease[uid=-792314720^58^id1^0^0,typeName=Data,routingValue=id1,expirationTime=9223372036854775807]

Now execute a task that reads this entry and returns content property:

xap> execute(gigaSpace) { holder =>
     |   holder.gigaSpace.read(Data()).content
     | }
res3: com.gigaspaces.async.AsyncFuture[String] = org.openspaces.core.transaction.internal.InternalAsyncFuture@7c767c0d

xap> val dataContent = res3.get
dataContent: String = my data content

It is possible to customize the initialization code, the shutdown code and the initial imports. XAP 10.2 introduced alternative method to customize initialization code along with initial imports.

XAP REPL during initialization phase will load imports from initial imports file and then execute init code. The end of shell session will be preceded by execution of shutdown code.

By default the initial imports will be loaded from $GS_HOME/tools/scala/conf/repl-imports.conf. This location can be overridden by the system property: org.os.scala.repl.imports. Each import should be in its own line. (empty lines and lines beginning with # are ignored)

By default the initialization code will be loaded from $GS_HOME/tools/scala/conf/init-code.scala. This location can be overridden by the system property: org.os.scala.repl.initcode.

By default the shutdown code will be loaded from $GS_HOME/tools/scala/conf/shutdown-code.scala. This location can be overridden by the system property: org.os.scala.repl.shutdowncode.

The difference between the two methods is that the new one does not use a file with initial imports. Those imports should be placed in initialization code script (with import prefix added). Alternative method is used if system property org.os.scala.repl.newinitstyle is set as 'yes', 'on' or 'true'.

By default the initialization code will be loaded from $GS_HOME/tools/scala/conf/new-init-code.scala. This location can be overridden by the system property: org.os.scala.repl.initcode. Please note that new init code script should also contain necessary imports.

Please refer to Shutdown code described above in the standard method.

Predicate is a term that originates from mathematical logic. Basically it means a function that takes one ore more arguments and returns true or false. Predicates are natural way in functional languages to filter objects that satisfy a certain condition. Predicate based queries use one argument predicates which describe constraints that are met by data to retrieve from the space.

Support for predicate based queries on the GigaSpace proxy has been in added. It is based on the new macros feature introduced in Scala 2.10. Each predicate based query is transformed during compilation into an equivalent SQLQuery.

To use predicate based queries, import import org.openspaces.scala.core.ScalaGigaSpacesImplicits._ into scope. Then call the predicate method on the GigaSpace instance as demonstrated:

/* Import GigaSpace implicits into scope */
import org.openspaces.scala.core.ScalaGigaSpacesImplicits._

/* implicit conversion on gigaspace returns a wrapper class with predicate based query methods */
val predicateGigaSpace = gigaSpace.predicate

/* return person with name 'John' */
val john = predicateGigaSpace read { person: Person => person.name == "John" }

This query will return objects of class Person with name 'John'. ScalaGigaSpacesImplicits contains wrapper class that is implicitly constructed from a GigaSpace instance. It adds predicate based queries, among other features.

For the purpose of demonstration, we will use the following example data class

case class Person @SpaceClassConstructor() (

  @BeanProperty
  @SpaceId
  id: String = null,

  @BeanProperty
  name: String = null,

  @BeanProperty
  @SpaceProperty(nullValue = "-1")
  @SpaceIndex(`type` = SpaceIndexType.EXTENDED)
  height: Int = -1,

  @BeanProperty
  birthday: Date = null,

  @BeanProperty
  son: Person = null

)

The previous paragraph has shown usage of predicate based query along with read operation. However, other gigaSpace standard operations have corresponding predicate based query operation. Thy can be customized by parameters depending on type of method. The table below presents all available operations with possible customization arguments:

Dynamic Language Tasks feature has been extended and now supports Scala based script execution.

ScalaLocalScriptExecutor handles task of compiling local scripts. It is worth noting that this executor cannot compile scripts in parallel because of Scala internal mechanisms. Before script gets compiled, it will be wrapped by code that creates parameters used in script, imports certain scala specific classes and executes script with the passed arguments defined in its closure.

Here is how processing unit is configured to run a scripting executor with scala support and use it from a client proxy. For detailed information on the Scripting Executor framework, see Dynamic Language Tasks.

Below is presented configuration of a processing unit executing scripts. This piece should be inserted into pu.xml file.

<os-core:embedded-space id="space" name="mySpace"/>

<os-core:giga-space id="gigaSpace" space="space"/>

<bean id="scriptingExecutorImpl" class="org.openspaces.remoting.scripting.DefaultScriptingExecutor">
  <property name="executors">
    <map>
      <entry key="scala">
        <bean class="org.openspaces.remoting.scripting.ScalaLocalScriptExecutor">
        </bean>
      </entry>
    </map>
  </property>
</bean>

<os-remoting:service-exporter id="serviceExporter">
  <os-remoting:service ref="scriptingExecutorImpl"/>
</os-remoting:service-exporter>

<os-events:polling-container id="remotingContainer" giga-space="gigaSpace">
  <os-events:listener ref="serviceExporter"/>
</os-events:polling-container>

Please note that scriptingExecutorImpl bean should contain a mapping for 'Scala' type script. Scripts of unknown types are handled by default executor, what is not desired for Scala scripts.

Client side configuration is no different than in other scripting languages. For more details please refer to Dynamic Language Tasks.

<os-core:space-proxy  id="space" name="mySpace"/>

<os-core:giga-space id="gigaSpace" space="space"/>

<os-remoting:executor-proxy id="executorScriptingExecutor" giga-space="gigaSpace"
	interface="org.openspaces.remoting.scripting.ScriptingExecutor">
  <os-remoting:aspect>
    <bean class="org.openspaces.remoting.scripting.LazyLoadingRemoteInvocationAspect" />
  </os-remoting:aspect>
  <os-remoting:routing-handler>
	<bean class="org.openspaces.remoting.scripting.ScriptingRemoteRoutingHandler" />
  </os-remoting:routing-handler>
  <os-remoting:meta-arguments-handler>
	<bean class="org.openspaces.remoting.scripting.ScriptingMetaArgumentsHandler" />
  </os-remoting:meta-arguments-handler>
</os-remoting:executor-proxy>

3 new Script implementations have been added to support compilation and caching of compiled scala scripts. These provide the ability to explicitly set the static type for script parameters which is required when the runtime type is not public. In most cases, there is no need to define these as they can be deduced to use the parameter runtime type.

• org.openspaces.remoting.scripting.ScalaTypedStaticScript which extends StaticScript.
• org.openspaces.remoting.scripting.ScalaTypedStaticResourceScript which extends StaticResourceScript.
• org.openspaces.remoting.scripting.ScalaTypedResourceLazyLoadingScript which extends ResourceLazyLoadingScript.

The following example is pretty straightforward. Parameters used inside script are injected by parameter method.

val code = """
val readData: Any = gigaSpace.read(null)
val numberAsString = someNumber.toString
val setAsString = someSet.toString
numberAsString + " " + someString + " " + setAsString + " " + readData
"""

val script = new ScalaTypedStaticScript("myScript", "scala", code)
  .parameter("someNumber", 1)
  .parameter("someString", "str")
  // explicit type is requierd because the runtime type of the generated
  // set is not public
  .parameter("someSet", Set(1,2,3), classOf[Set[_]])

val result = executor.execute(script)

println("Script execution result: " + result)

A wrapper around the XAP API provides some sugaring on top of the GigaSpace#execute methods. For more information on Tasks, please refer to documentation.

GigaSpace wrapper defines a few task related methods that simplify execution of tasks:

• tasks:

• execute(mapper)

• execute(mapper, routing)

• execute(mapper, routing, listener)

• distributed tasks:

• execute(mapper, reducer)

• execute(mapper, reducer, routing)

Where:

• mapper - function executing task
• reducer - function gathering results from mappers and producing final result
• routing - value that decides on which partition should the task be executed
• listener - handler fired after task has been executed

Import org.openspaces.scala.core.ScalaGigaSpacesImplicits.ScalaEnhancedGigaSpaceWrapper into scope to use the methods demonstrated below.

Some examples:

/** Import GigaSpace implicits into scope */
import org.openspaces.scala.core.ScalaGigaSpacesImplicits._

...

gigaSpace write Data(id = 1, routing = 2, data = "some data")
gigaSpace write Data(id = 2, routing = 3, data = "some other data")

// execution of a task
val asyncFuture1 = gigaSpace.execute { gigaSpace: GigaSpace =>
  gigaSpace.readById(classOf[Data], 1l)
}

println("Execute1 result: " + asyncFuture1.get())

// execution of a distributed task
val asyncFuture2 = gigaSpace.execute(
  { gigaSpace: GigaSpace => gigaSpace.read(Data()).data } /* map */,
  { results: Seq[AsyncResult[String]] => results.map { _.getResult() }.mkString } /* reduce */
)

println("Map reduce result: " + asyncFuture2.get())

There has been created a project that shows how certain features of XAP Scala can be used in real project and how Scala and Java code might be integrated.

The project is based on a template project of basic type from the OpenSpaces Maven plugin. Obviously, a few changes were introduced:

• Common module, which implements spaces classes, has been rewritten in Scala and takes advantage of constructor based properties.
• A new module - verifier - has been created. It uses a class with constructor based properties and predicate based queries to obtain objects from space.
• Build process of common and verifier modules has been modified to handle Scala and mixed Java/Scala modules, respectively.

1. JDK in version at least 1.6 is required to build the project.
2. The project uses maven build tool.
3. To run the project, Scala libraries have to be a accessible for XAP - to achieve this please follow steps in requirements pararaph.

Please note that Scala is not required to build the project, since requried libraries will be downloaded by maven.

1. From the project's main directory $XAP_SCALA_MASTER/example/gs-openspaces-scala-example run command mvn clean package - necessary JAR files to deploy on a grid will be created.
2. Start XAP Grid Service by running command: $GS_HOME/bin/gs-agent.sh
3. Deploy the project on the grid (from $XAP_SCALA_MASTER/example/gs-openspaces-scala-example): mvn os:deploy -Dgroups=$LOOKUPGROUPS.

Common module defines space classes used by other modules. Please note, that the classes are written in Scala, and are used in other Scala and Java modules as well. This is caused by the fact that all of them are translated to a common code (bytecode) and therefore, can be used interchangeably.

Sometimes, having immutable state is very desired feature. This requirement is covered in XAP Scala by classes that use constructor based properties - in case of the common module it is the Verification class. It is written only once to the Space and never changed (eventually instance can be removed), because the goal of a single object is to remember appearance of a certain, unchangeable:

case class Verification @SpaceClassConstructor() (
  @BeanProperty
  @SpaceId
  id: String,

  @BeanProperty
  dataId: String) extends scala.Serializable {

  override def toString: String = s"id[$id] dataId[$dataId]"
}

The other class (Data) has been rewritten to Scala. However, its behavior has not been modified (apart from a adding new field needed by the verifier module):

case class Data (
  @BeanProperty @SpaceId(autoGenerate = true) var id: String = null,
  @BeanProperty @SpaceRouting @SpaceProperty(nullValue = "-1") var `type`: Long = -1,
  @BeanProperty var rawData: String = null,
  @BeanProperty var data: String = null,
  @BooleanBeanProperty var processed: Boolean = false,
  @BooleanBeanProperty var verified: Boolean = false) {

  def this(`type`: Long, rawData: String) = this(null, `type`, rawData, null, false, false)

  def this() = this(-1, null)

  override def toString: String = s"id[${id}] type[${`type`}] rawData[${rawData}] data[${data}] processed[${processed}] verified[${verified}]"
}

The verifier module extends the pipeline presented in the baseline project (the one created by the OpenSpaces Maven plugin). Verifier picks up processed Data instances and tries to verify them. The objects that pass the verification process are then modified (verified set to true) and saved along with a new, immutable Verification object. The objects that failed during verification process are removed from the space. The verifier uses the new feature - predicate based queries - to access the space in a more readable and natural way (especially for functional languages such as Scala):

@GigaSpaceContext private var gigaSpace: GigaSpace = _ // injected
// ...

// data instances to process further are obtained in the following way
val unverifiedData = gigaSpace.predicate.readMultiple { data: Data => data.processed == true && data.verified == false }

Pu.xml contains a standard description of gigaSpace:

...
<os-core:giga-space-context/>

<os-core:space id="space" url="jini://*/*/space" />

<os-core:giga-space id="gigaSpace" space="space"/>
...

Please note that gigaSpace from the code above is an instance of ScalaEnhancedGigaSpaceWrapper - a wrapper around GigaSpace introduced in XAP Scala.

The build configuration in Scala or Java/Scala modules is almost as simple in case of pure Java modules.

The common module is a pure Scala module. The maven-compiler-plugin has been replaced by scala-maven-plugin. The build configuration from the pom.xml for the common has the following form:

<build>
        <sourceDirectory>src/main/scala</sourceDirectory>
        <plugins>
            <plugin>
                <groupId>net.alchim31.maven</groupId>
                <artifactId>scala-maven-plugin</artifactId>
                <version>3.2.0</version>
                <executions>
                    <execution>
                        <goals>
                            <goal>compile</goal>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
                <configuration>
                    <scalaCompatVersion>${scalaBinaryVersion}</scalaCompatVersion>
                </configuration>
            </plugin>
        </plugins>


        <finalName>gs-openspaces-scala-example-common</finalName>
    </build>

where scalaBinaryVersion is a property defined in a parent pom file (in this case it is 2.11).

The verifier module is a mixed Java-Scala module, where Scala classes call Java classes. This configuration can be used when a separate task is implemented in Java and it only needs to be called from other parts of application. In case of this project, Java module is simulated by VerifierEngine class and, for ease of use, it is executed by Scala verifier.

In such a configuration, Scala compiler has to 'somehow' reach Java compiled classes. This is where a build-helper-maven-plugin is used - it adds Java classes to the source, then they are compiled and finally Scala compiler uses them during Scala code compilation. The build configuration of the verifier module is as follows:

<build>
    <sourceDirectory>src/main/scala</sourceDirectory>

    <plugins>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>build-helper-maven-plugin</artifactId>
            <version>1.4</version>
            <executions>
                <execution>
                    <id>add-java-source</id>
                    <phase>generate-sources</phase>
                    <goals>
                        <goal>add-source</goal>
                    </goals>
                    <configuration>
                        <sources>
                            <source>${basedir}/src/main/java</source>
                        </sources>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>net.alchim31.maven</groupId>
            <artifactId>scala-maven-plugin</artifactId>
            <version>3.2.0</version>
            <executions>
                <execution>
                    <goals>
                        <goal>compile</goal>
                        <goal>testCompile</goal>
                    </goals>
                </execution>
            </executions>
            <configuration>
                <scalaCompatVersion>${scalaBinaryVersion}</scalaCompatVersion>
            </configuration>
        </plugin>
    </plugins>
</build>