[Download] | [Documentation Home] | [Release Note]
This document describes the WSAS clustering functionality and demonstrates it using examples.
Figure 2: WSAS high availability cluster
Session replication in WSAS can be used to achieve high availability for stateful services as shown in figure 2. All the client requests are always directed to Node 1, which is elected as the primary server by the load balancing system. Node 2 and Node 3 are kept as back up servers and none of the requests will be directed to them under normal operations. All three nodes are configured as a WSAS cluster, so that session state changes in Node 1 will be replicated to Node 2 and Node 3.
Now if Node 1 fails, the load balancing system elects Node 2 as the primary server and the other two are considered as backups. From that point, client requests are directed to Node 2, but the client does not notice any change or data loss, as all the session data is available in Node 2 as well.
Figure 3: WSAS scalability cluster
By combining the WSAS clustering and a smart load balancing system, it is possible to achieve high availability as well as scalability. Such deployment is shown in figure 3. Two WSAS clusters are created in this scenario. Node 1 and Node 2 belong to the first cluster and Node 3 and Node 4 belong to the second cluster. Once a client sends a request through the load balancing system, it binds that client to a particular cluster. From that point onwards, all the requests from that client will always be directed to the same cluster. If a primary node of a cluster failed, the load balancing system should elect a backup node of that cluster as the primary node and direct requests to that. Thus, scalability is achieved as a growing number of clients can be handled simply by introducing new clusters, and the high availability is achieved as each cluster can have backup nodes with an identical configuration and session state.
WSAS clustering currently does not support distributed locking for session data replication. Therefore, we have to deploy primary backup clusters for stateful services as mentioned above. This restriction does not apply to stateless services and you can direct client requests to any node in the cluster for such services.
WSAS clustering is configured using the axis2.xml file. As all instances of a WSAS cluster can be configured to load this file from the shared repository, initial clustering configuration can be done by editing a single file.
For more details about WSAS clustering, please see WSO Carbon Clustering Configuration Language
Now we have explored enough background information about WSAS clustering. It is time to see it in action.
Download the WSAS binary distribution by following the installation guide. As we are going to set up a WSAS cluster for the demonstration purpose, we are going to host all the instances in the same computer. Extract the WSAS distribution to two directories, so that we can start two instances. We refer to these two instances as Node 1 and Node 2. The next step is to configure the URL repository for both nodes. Open the server.xml of Node 1 and change the repository location to http://localhost/repository/. Please make sure that the WSAS repository is accessible from the given URL. Then change the configuration file location to point to the axis2.xml of Node 1. The section of the server.xml file containing the changed locations is shown below:
<Axis2Config> <!-- Location of the Axis2 Services & Modules repository This can be a directory in the local file system, or a URL. e.g. 1. /home/wso2wsas/repository/ - An absolute path 2. repository - In this case, the path is relative to CARBON_HOME 3. file:///home/wso2wsas/repository/ 4. http://wso2wsas/repository/ --> <RepositoryLocation>http://localhost/repository/</RepositoryLocation> <!-- Location of the main Axis2 configuration descriptor file, a.k.a. axis2.xml file This can be a file on the local file system, or a URL e.g. 1. /home/wso2wsas/conf/axis2.xml - An absolute path 2. conf/axis2.xml - In this case, the path is relative to CARBON_HOME 3. file:///home/wso2wsas/conf/axis2.xml 4. http://wso2wsas/conf/axis2.xml --> <ConfigurationFile>/home/wsas/node1/conf/axis2.xml</ConfigurationFile> <!-- ServiceGroupContextIdleTime, which will be set in ConfigurationContex for multiple clients which are going to access the same ServiceGroupContext Default Value is 30 Sec. --> <ServiceGroupContextIdleTime>30000</ServiceGroupContextIdleTime> </Axis2Config>
Now make the same changes for the server.xml file of Node 2. Make sure to set the configuration file location to the axis2.xml file of Node 2. Please note that we are setting different configuration file locations to the nodes to avoid port conflicts mentioned later. In production deployments, all nodes can be pointed to the same axis2.xml file located in the HTTP repository.
Now we are done with configuring the repository. Next we have to enable clustering for both nodes. By default, clustering is turned off to avoid additional overhead for individual deployments. Open the axis2.xml of both nodes and uncomment the clustering section. You may also change clustering properties to suit your deployment. However, the default configuration is sufficient for the demonstration.
There is one more step left as we are trying to run both nodes in the same machine. That is, we have to change the various ports opened by WSAS to avoid conflicts. As some of these ports are configured in the axis2.xml file, we have to use two axis2.xml files for the two WSAS instances, instead of sharing it from the central repository. However, we can share service archives from the central repository between both nodes.
Open the axis2.xml of Node 2 and change the HTTP transport receiver port to 9763. Then change the HTTPS transport receiver port to 9444. A portion of the axis2.xml file after changing the ports is shown below:
<transportReceiver name="http" class="org.wso2.wsas.transport.http.HttpTransportListener"> <parameter name="port">9763</parameter> </transportReceiver> <transportReceiver name="https" class="org.wso2.wsas.transport.http.HttpsTransportListener"> <parameter name="port">9444</parameter> <parameter name="sslProtocol">TLS</parameter>
Now open the server.xml file of Node 2 and change the command listener port to 6667. Portion of the server.xml file containing the command listener port is shown below:
<CommandListener> <Port>6667</Port> </CommandListener>
We have completed configuring the WSAS nodes for clustered deployment. Now start both WSAS nodes using the wso2wsas.sh from the bin directory of Node 1 and Node 2.
Let's write a small service for the demonstration. This service has two methods called setValue() and getValue(). The setValue() method stores the value passed as the parameter in the service group context. The getValue() method retrieves that value from the service group context and returns it. If this service is deployed in any other scope than the request scope, this value is available between method invocations, making it a stateful service. Code of the service class is listed below:
public class Service1 { public OMElement setValue(OMElement param) { param.build(); param.detach(); String value = param.getText(); MessageContext. getCurrentMessageContext().getServiceGroupContext().setProperty("myValue", value); System.out.println("=============================================================="); System.out.println("Value: " + value + " is set."); System.out.println("=============================================================="); param.setText("Value: " + value + " is set."); return param; } public OMElement getValue(OMElement param) { param.build(); param.detach(); String value = (String) MessageContext. getCurrentMessageContext().getServiceGroupContext().getProperty("myValue"); System.out.println("=============================================================="); System.out.println("Value: " + value + " is retrieved."); System.out.println("=============================================================="); param.setText(value); return param; } }
We are going to deploy this in the SOAP session scope. Therefore, set the scope attribute to soapsession in the services.xml file.
<service name="service1" scope="soapsession"> ... </service>
Now compile the service class and make a service archive named service1.aar from it.
Deploy the service1 in the cluster as mentioned in the previous section. Now we have a stateful service deployed in the cluster. We can verify the session state replication by setting some session data in Node 1 and accessing them from Node 2. This is possible as WSAS clustering layer replicates session data added to any node to all other nodes. We have to write a Web services client to set the value in Node 1 and access it from Node 2. This client code is listed below:
public class SessionClient { public static void main(String[] args) { new SessionClient().invoke(); } private void invoke() { String repository = "/home/wso2/products/axis2/repository"; String axis2xml_path = "/home/wso2/products/axis2/conf/axis2-client.xml"; OMFactory fac = OMAbstractFactory.getOMFactory(); OMNamespace ns = fac.createOMNamespace("http://services", "ns"); OMElement value = fac.createOMElement("Value", ns); value.setText("Sample session data"); try { ConfigurationContext configContext = ConfigurationContextFactory. createConfigurationContextFromFileSystem(repository, axis2xml_path); ServiceClient client = new ServiceClient(configContext, null); client.engageModule("addressing"); Options options = new Options(); options.setTimeOutInMilliSeconds(10000000); options.setManageSession(true); client.setOptions(options); // Set some session data in Node 1 options.setTo(new EndpointReference("http://192.168.1.3:9762/services/service1")); options.setAction("setValue"); OMElement response1 = client.sendReceive(value); System.out.println( "Server response for setting the sample session data in Node 1: " + response1.getText()); // Access the session data from Node 2 options.setTo(new EndpointReference("http://192.168.1.3:9763/services/service1")); options.setAction("getValue"); OMElement response2 = client.sendReceive(value); System.out.println("Retrieved sample session data from Node 2: " + response2.getText()); } catch (AxisFault axisFault) { axisFault.printStackTrace(); } } }
The above code first invokes the setValue() method of our sample service in Node 1 with the string "Sample session data". Then it invokes the getValue() method from Node 2 and displays the retrieved value. Compile and execute this code with required Axis2 jars in the class path. Please make sure that the "repository" string contains a path of a Axis2 repository and "axis2xml_path" contains a path for an axis2.xml file. You can see the following output in the console:
Server response for setting the sample session data in Node 1: Value: Sample session data is set. Retrieved sample session data from Node 2: Sample session data
The session data "Sample session data" was set in Node 1 and retrieved from Node 2. You may also disable the clustering in the nodes and perform the above test again. You will see that this time session data cannot be accessed from Node 2.
We have completed the WSAS clustering guide. If you have more questions on WSAS clustering functionality, please feel free to ask them on WSAS user list: wsas-java-user@wso2.org.