Required software
In this tutorial, we will be writing a sample JPPF application, and we will run it on a small grid. To this effect, we will need to download and install the following JPPF components:- JPPF application template: this is the JPPF-x.y.z-application-template.zip file
- JPPF driver: this is the JPPF-x.y.z-driver.zip file
- JPPF node: this is the JPPF-x.y.z-node.zip file
- JPPF administration console: this is the JPPF-x.y.z-admin-ui.zip file
Note: “x.y.z” designates the latest version of JPPF (major.minor.update). Generally, “x.y.0” is abbreviated into “x.y”.
These files are all available from the JPPF installer and/or from the JPPF download page.
In addition to this, Java 1.6 or later and Apache Ant 1.7.0 or later should already be installed on your machine.
We will assume the creation of a new folder called "JPPF-Tutorial", in which all these components are unzipped. Thus, we should have the following folder structure:
» JPPF-Tutorial » JPPF-x.y.z-admin-ui » JPPF-x.y.z-application-template » JPPF-x.y.z-driver » JPPF-x.y.z-node
Overview
Tutorial organization
We will base this tutorial on a pre-existing application template, which is one of the components of the JPPF distribution. The advantage is that most of the low-level wiring is already written for us, and we can thus focus on the steps to put together a JPPF application. The template is a very simple, but fully working, JPPF application, and contains fully commented source code, configuration files and scripts to build and run it.It is organized with the following directory structure:
- root directory: contains the scripts to build and run the application
- src: this is where the sources of the application are located
- classes: the location where the Java compiler will place the built sources
- config: contains the JPPF and logging configuration files
- lib: contains the required libraries to build and run the application
Expectations
We will learn how to:- write a JPPF task
- create a job and execute it
- process the execution results
- manage JPPF jobs
- run a JPPF application
The features of JPPF that we will use:
- JPPF task and job APIs
- local code changes automatically accounted for
- JPPF client APIs
- management and monitoring console
- configuring JPPF
By the end of this tutorial, we will have a full-fledged JPPF application that we can build, run, monitor and manage in a JPPF grid. We will also have gained knowledge of the workings of a typical JPPF application and we will be ready to write real-life, grid-enabled applications.
Writing a JPPF task
A JPPF task is the smallest unit of code that can be executed on a JPPF grid. From a JPPF perspective, it is thus defined as an atomic code unit. A task is always defined as a subclass of the class JPPFTask. JPPFTask is an abstract class that implements the Runnable interface. The part of a task that will be executed on the grid is whatever is written in its run() method.From a design point of view, writing a JPPF task will comprise 2 major steps:
- create a subclass of JPPFTask.
- implement the run() method.
In the editor you will see a full-fledged JPPF task declared as follows:
public class TemplateJPPFTask extends JPPFTaskBelow this, you will find a run() method declared as:
public void run() { // write your task code here. System.out.println("Hello, this is the node executing a template JPPF task"); // ... // eventually set the execution results setResult("the execution was performed successfully"); }We can guess that this task will first print a "Hello …" message to the console, then set the execution result by calling the setResult() method with a string message. The setResult() method actually takes any object, and is provided as a convenience to store the results of the task execution, for later retrieval.
In this method, to show that we have customized the template, let's replace the line "// …" with a statement printing a second message, for instance "In fact, this is more than the standard template". The run() method becomes:
public void run() { // write your task code here. System.out.println("Hello, this is the node executing a template JPPF task"); System.out.println("In fact, this is more than the standard template"); // eventually set the execution results setResult("the execution was performed successfully"); }Do not forget to save the file for this change to be taken into account.
The next step is to create a JPPF job from one or multiple tasks, and execute this job on the grid.
Creating and executing a job
A job is a grouping of tasks with a common set of characteristics and a common SLA. These characteristics include:- common data shared between tasks
- a priority
- a maximum number of nodes a job can be executed on
- an execution policy describing which nodes it can run on
- a suspended indicator, that enables submitting a job in suspended state, waiting for an external command to resume or start its execution
- a blocking/non-blocking indicator, specifying whether the job execution is synchronous or asynchronous from the application's point of view
Creating and populating a job
In the JPPF APIs, a job is represented as an instance of the class JPPFJob.To see how a job is created, let's open the source file "TemplateApplicationRunner.java" in the folder JPPF-x.y.z-application-template/src/org/jppf/application/template. In this file, navigate to the method createJob().
This method is written as follows:
public JPPFJob createJob() throws Exception {
// create a JPPF job
JPPFJob job = new JPPFJob();
// give this job a readable unique id that we can use to
// monitor and manage it.
job.setName("Template Job Id");
// add a task to the job.
job.addTask(new TemplateJPPFTask());
// add more tasks here ...
// there is no guarantee on the order of execution of the tasks,
// however the results are guaranteed to be returned in the same
// order as the tasks.
return job;
}
|
We can see that creating a job is done by calling the default constructor of class JPPFJob. The call to the method job.setName(String) is used to give the job a meaningful name and readable that we can use later to manage it. If this method is not called, an id is automatically generated, as a string of 32 hexadecimal characters.
Adding a task to the job is done by calling the method addTask(Object task, Object...args). The optional arguments are used when we want to execute other forms of tasks, that are not subclasses of JPPFTask. We will see their use in the more advanced sections of the JPPF user manual. As we can see, all the work is already done in the template file, so there is no need to modify the createJob() method for now.
Executing a job and processing the results
Now that we have learned how to create a job and populate it with tasks, we still need to execute this job on the grid, and process the results of this execution. Still in the source file "TemplateApplicationRunner.java", let's navigate to the main(String...args) method. we will first take a closer look at the try block, which contains a very important initialization statement:jppfClient = new JPPFClient();This single statement initializes the JPPF framework in your application. When it is executed JPPF will do several things:
- read the configuration file
- establish a connection with one or multiple servers for job execution
- establish a monitoring and management connection with each connected server
- register listeners to monitor the status of each connection
private static JPPFClient jppfClient = null;It is also a good practice to release the resources used by the JPPF client when they are not used anymore. We actually recommend to do this by calling its close() method within a finally{} block:
try {
jppfCLient = new JPPFClient();
// ...
} finally {
if (jppfClient != null) jppfClient.close();
}
|
// create a runner instance. TemplateApplicationRunner runner = new TemplateApplicationRunner(); // Create a job JPPFJob job = runner.createJob(); // execute a blocking job runner.executeBlockingJob(job);The call to runner.createJob() is exactly what we saw in the previous section. What remains to do is to execute the job and process the results, which is the intent of the call to executeBlockingJob(JPPFJob job):
/** * Execute a job in blocking mode. The application will be blocked until the job * execution is complete. * @param job the JPPF job to execute. * @throws Exception if an error occurs while executing the job. */ public void executeBlockingJob(JPPFJob job) throws Exception { // set the job in blocking mode. job.setBlocking(true); // Submit the job and wait until the results are returned. // The results are returned as a list of JPPFTask instances, // in the same order as the one in which the tasks where initially added the job. List<JPPFTask> results = jppfClient.submit(job); // process the results processExecutionResults(results); }The first statement of this method ensures that the job will be submitted in blocking mode, meaning that the application will block until the job is executed:
job.setBlocking(true);This is, in fact, optional since submission in blocking mode is the default behavior in JPPF.
The second statement is the one that will send the job to the server and wait until it has been executed and the results are returned:
List<JPPFTask> results = jppfClient.submit(job);We can see that the results are returned as a list of JPPFTask objects. It is guaranteed that each task in this list has the same position as the corresponding task that was added to the job. In other words, the results are always in the same order as the tasks in the the job.
The last step is to interpret and process the results. From the JPPF point of view, there are two possible outcomes of the execution of a task: one that raised a Throwable, and one that did not. When an uncaught Throwable (i.e. generally an instance of a subclass of java.lang.Error or java.lang.Exception) is raised, JPPF will catch it and set it as the outcome of the task. To do so, the method JPPFTask.setException(Exception) is called. You will note that the parameter is an instance of Exception or of one of its subclasses. Thus, any uncaught Error will be wrapped in a JPPFException. JPPF considers that exception processing is part of the life cycle of a task and provides the means to capture that information accordingly.
This explains why, in our template code, we have separated the result processing of each task in 2 blocks:
public void processExecutionResults(final List<JPPFTask> results) {
// process the results
for (JPPFTask task: results) {
if (task.getException() != null) {
// process the exception here ...
} else {
// process the result here ...
}
}
}
|
As an example for this tutorial, let's modify this part of the code to display the exception message if an exception was raised, and to display the result otherwise:
if (task.getException() != null) {
System.out.println("An exception was raised: " + task.getException().getMessage());
} else {
System.out.println("Execution result: " + task.getResult());
}
|
Running the application
We are now ready to test our JPPF application. To this effect, we will need to first start a JPPF grid, as follows:Step 1: start a server
Go to the JPPF-x.y.z-driver folder and open a command prompt or shell console. Type "startDriver.bat" on Windows or “./startDriver.sh.” on Linux/Unix. You should see the following lines printed to the console:
driver process id: 2612 management initialized and listening on port 11191 ClientClassServer initialized NodeClassServer initialized ClientServer initialized TasksServer initialized Acceptor initialized - accepting plain connections on port 11111 - accepting secure connections on port 11443 JPPF Driver initialization completeThe server is now ready to process job requests.
Step 2: start a node
Go to the JPPF-x.y.z-node folder and open a command prompt or shell console. Type "startNode.bat" on Windows or “./startNode.sh.” on Linux/Unix. You will then see the following lines printed to the console:
node process id: 3336 Attempting connection to the class server at localhost:11111 Reconnected to the class server JPPF Node management initialized Attempting connection to the node server at localhost:11111 Reconnected to the node server Node successfully initializedTogether, this node and the server constitute the smallest JPPF grid that you can have.
Step 3: run the application
Go to the JPPF-x.y.z-application-template folder and open a command prompt or shell console. Type "ant". This time, the Ant script will first compile our application, then run it. You should see these lines printed to the console:
[client: driver-1] Attempting connection to the class server at localhost:11111 [client: driver-1] Reconnected to the class server [client: driver-1] Attempting connection to the JPPF task server at localhost:11111 [client: driver-1] Reconnected to the JPPF task server Execution result: the execution was performed successfullywhere <ip_address> corresponds to the IP address of your computer.
You will notice that the last printed line is the same message that we used in our task in the run() method, to set the result of the execution in the statement:
setResult("the execution was performed successfully");
Now, if you switch back to the node console, you should see that 2 new messages have been printed:
[java] Hello, this is the node executing a template JPPF task
[java] In fact, this is more than the standard template
These 2 lines are those that we actually coded at the beginning of the task's run() method:
System.out.println("Hello, this is the node executing a template JPPF task");
System.out.println("In fact, this is more than the standard template");
From these messages, we can conclude that our application was run successfully. Congratulations!
At this point, there is however one aspect that we have not yet addressed: since the node is a separate process from our application, how does it know to execute our task? Remember that we have not even attempted to deploy the application classes to any specific location. We have simply compiled them so that we can execute our application locally. This topic is the object of the next section of this tutorial.
Dynamic deployment
One of the greatest features of JPPF is its ability to dynamically load the code of an application that was deployed only locally. JPPF extends the standard Java class loading mechanism so that, by simply using the JPPF APIs, the classes of an application are loaded to any remote node that needs them. The benefit is that no deployment of the application is required to have it run on a JPPF grid, no matter how many nodes or servers are present in the grid. Furthermore, this mechanism is totally transparent to the application developer.A second major benefit is that code changes are automatically taken into account, without any need to restart the nodes or the server. This means that, when you change any part of the code executed on a node, all you have to do is recompile the code and run the application again, and the changes will take effect immediately, on all the nodes that execute the application.
We will now demonstrate this by making a small, but visible, code change and running it against the server and node we have already started, If you have stopped them already, just perform again all the steps described in the previous section (2.5 ), before continuing.
Let's open again the source file "TemplateJPPFTask.java" in JPPF-x.y.z-application-template/src/org/jppf/application/template/, and navigate to the run() method. Let's replace the first two lines with the following:
System.out.println("*** We are now running a modified version of the code ***");
The run() method should now look like this:
public void run() {
// write your task code here.
System.out.println("*** We are now running a modified version of the code ***");
// eventually set the execution results
setResult("the execution was performed successfully");
}
Save the changes to the file, and open or go back to a command prompt
or shell console in the JPPF-x.y.z-application-template folder. From
there, type "ant" to run the application again. You should now see the
same messages as in the initial run displayed in the console. This is
what we expected. On the other hand, if you switch back to the node
console, you should now see a new message displayed:
[java] *** We are now running a modified version of the code ***Success! We have successfully executed our new code without any explicit redeployment.
Job Management
Now that we are able to create, submit and execute a job, we can start thinking about monitoring and eventually controlling its life cycle on the grid. To do that, we will use the JPPF administration and monitoring console. The JPPF console is a standalone graphical tool that provides user-friendly interfaces to:- obtain statistics on server performance
- define, customize and visualize server performance charts
- monitor and control the status and health of servers and nodes
- monitor and control the execution of the jobs on the grid
- manage the workload and load-balancing behavior
Preparing the job for management
In our application template, the job that we execute on the grid has a single task. As we have seen, this task is very short-live, since it executes in no more than a few milliseconds. This definitely will not allow us us to monitor or manage it with our bare human reaction time. For the purpose of this tutorial, we will now adapt the template to something more realistic from this perspective.Step 1: make the tasks last longer
What we will do here is add a delay to each task, before it terminates. It will do nothing during this time, only wait for a specified duration. Let's edit again the source file "TemplateJPPFTask.java" in JPPF-x.y.z-application-template/src/org/jppf/application/template/ and modify the run() method as follows:
public void run() {
// write your task code here.
System.out.println("*** We are now running a modified version of the code ***");
// simply wait for 3 seconds
try {
Thread.sleep(3000L);
} catch(InterruptedException e) {
setException(e);
return;
}
// eventually set the execution results
setResult("the execution was performed successfully");
}
Note that here, we make an explicit call to setException(), in case an InterruptedException
is raised. Since the exception would be occurring in the node,
capturing it will allow us to know what happened from the application
side.
Step 2: add more tasks to the job, submit it as suspended
This time, our job will contain more than one task. In order for us to have the time to manipulate it from the administration console, we will also start it in suspended mode. To this effect, we will modify the method createJob() of the application runner "TemplateApplicationRunner.java" as follows:
public JPPFJob createJob() throws Exception {
// create a JPPF job
JPPFJob job = new JPPFJob();
// give this job a readable unique id that we can use to monitor and manage it.
job.setName("Template Job Id");
// add 10 tasks to the job.
for (int i=0; i<10; i++) job.addTask(new TemplateJPPFTask());
// start the job in suspended mode
job.getSLA().setSuspended(true);
return job;
}
Step 3: start the JPPF components
If you have stopped the server and node, simply start them again as described in the first two step of section 2.5 of this tutorial.
We will also start the administration console:
Go to the JPPF-x.y.z-admin-ui folder and open a command prompt or shell console. Type "ant".
When the console is started, you will see a panel named "Topology" displaying the servers and the nodes attached to them. It should look like this:
We can see here that a server is started on machine "lolo-quad" and that it has a node attached to it. The color for the server is a health indicator, green meaning that it is running normally and red meaning that it is down.
Let's switch to the "Job Data" panel, which should look like this:
We also see the color-coded driver health information in this panel. There is currently no other element displayed, because we haven't submitted a job yet.
Step 4: start a job
We will now start a job by running our application: go to the JPPF-x.y.z-application-template folder and open a command prompt or shell console. Type "ant". Switch back to the administration console. We should now see some change in the display:
We now see that a job is present in the server's queue, in suspended state (yellow highlighting). Here is an explanation of the columns in the table:
- "Driver / Job / Node" : displays an identifier for a server, for a job submitted to that server, or for a node to which some of the tasks in the job have been dispatched for execution
- "State" : the current state of a job, either "Suspended" or "Executing"
- "Initial task count" : the number of tasks in the job at the time it was submitted by the application
- "Current task count": the number of tasks remaining in the job, that haven't been executed
- "Priority" : this is the priority, of the job, the default value is 0.
- "Max nodes" : the maximum number of nodes a job can be executed on. By default, there is no limit, which is represented as the infinity symbol
Step 5: resuming the job execution
Since the job was submitted in suspended state, we will resume its execution manually from the console. Select the line where the job "Template Job Id" is displayed. You should see that some buttons are now activated. Click on the resume button (marked by the icon
) to resume the job execution, as shown below:
As soon as we resume the job, the server starts distributing tasks to the node, and we can see that the current task count starts decreasing accordingly, and the job status has been changed to "Executing":
You are encouraged to experiment with the tool and the code. For example you can add more tasks to the job, make them last longer, suspend, resume or terminate the job while it is executing, etc...
0 comments:
Post a Comment