Tuesday 14 July 2009

Getting Started with StreamHub and Comet

This article has now been superseded by the similar: Getting Started with Reverse Ajax and Comet.

This article refers to version 2.0.2. Download 2.0.2 here.

Introduction

This is a quick getting started guide for StreamHub. StreamHub is a lightweight Comet server used for streaming real-time data to web browsers and other clients. In this article I'll take you through how to create a simple 'Hello World' Comet application using the StreamHub Java SDK for Streaming and the AJAX Client API. By the end of the guide we'll have a cross-browser webpage that will be receiving real-time data pushed from the server. This guide assumes you know what Comet is and have some basic JavaScript and Java knowledge but it doesn't get too deep so don't worry.

Step 1 - Download the Free Community Edition

First of all we need to download the StreamHub SDK from the website. Go to the StreamHub download page and grab yourself a copy of the Community Edition. At time of writing the latest version was 2.0 but any version should work. There isn't any real difference between the .ZIP version and the .TAR.GZ version - so grab which ever one you prefer. After downloading, extract the archive somewhere you can store it permanently. I put mine in C:\streamhub. Don't worry at this point if you're using Linux or Mac - the StreamHub server is completely cross platform.

Step 2 - Creating the 'Hello World' web page

Let's create a simple web page to host our Comet application. Create a new folder in your StreamHub directory called HelloWorld, so now you'll have something like C:\streamhub\HelloWorld. Create a new file in the HelloWorld directory called index.html. This will be our Comet web page. Edit that file, adding the following:
<html>
<head>
<title>Hello World Comet Application</title>
</head>
<body>
<h1>Hello World Comet Application</h1>
<input type="button" value="Start streaming" onclick="startStreaming()">
<div id="streamingData"></div>
<script>
function startStreaming() {
// empty for now
}
</script>
</body>
</html>
That's it - all we've done so far is create a very basic HTML page with a single button that when clicked triggers an empty JavaScript function. We'll need to add the StreamHub JavaScript API in order to connect to the server, so lets grab that from the Community Edition we downloaded earlier and include it in our script. The JavaScript API comes as a single mini-fied file called streamhub-min.js. Its in the archive we extracted in step 1, mine was in the directory C:\streamhub\streamhub-java-adapter-sdk-1.0\examples\lib\js.

Copy this file and paste it in the HelloWorld folder. Now we just need to include it in our web page, the new index.html should look something like this:
<html>
<head>
<title>Hello World Comet Application</title>
<script src="streamhub-min.js" type="text/javascript"></script>
</head>
<body>
<h1>Hello World Comet Application</h1>
<input type="button" value="Start streaming" onclick="startStreaming()">
<div id="streamingData"></div>
<script>
function startStreaming() {
// empty for now
}
</script>
</body>
</html>
Step 3 - Creating a Java Adapter for Streaming

Now we've got a basic webpage we need to provide some sort of data to it before we connect to the Comet server. So in this section we'll be creating a HelloWorldStreamer Java class to stream data to our webpage.

Create a new file in the HelloWorld directory called HelloWorldStreamer.java. First we need to add the code to create and start the Comet server. Edit HelloWorldStreamer.java, adding the following:
import com.streamhub.api.PushServer;
import com.streamhub.nio.NIOServer;

public class HelloWorldStreamer {
public static void main(String[] args) throws Exception {
new HelloWorldStreamer();
}

public HelloWorldStreamer() throws Exception {
// Create a new server on port 7878
PushServer server = new NIOServer(7878);
server.start();
System.out.println("Comet server started at http://localhost:7878/.");
System.out.println("Press any key to stop...");
System.in.read();
server.stop();
}
}
This creates the Comet server and starts it. To actually stream some data to our webpage we'll need to add ourselves as a listener by changing the code to:
import com.streamhub.api.Client;
import com.streamhub.api.SubscriptionListener;
import com.streamhub.api.PushServer;
import com.streamhub.nio.NIOServer;

public class HelloWorldStreamer implements SubscriptionListener {
public static void main(String[] args) throws Exception {
new HelloWorldStreamer();
}

public HelloWorldStreamer() throws Exception {
// Create a new server on port 7878
PushServer server = new NIOServer(7878);
server.start();
server.getSubscriptionManager().addSubscriptionListener(this);
System.out.println("Comet server started at http://localhost:7878/.");
System.out.println("Press any key to stop...");
System.in.read();
server.stop();
}

public void onSubscribe(String topic, Client client) {

}

public void onUnSubscribe(String topic, Client client) {

}
}
Notice we now implement SubscriptionListener which requires us to implement the onSubscribe and onUnSubscribe methods. These methods will be called when a client, such as our HelloWorld web page, subscribes or unsubscribes to a particular topic. For now lets just send a synchronous response back to the client when they subscribe:
import com.streamhub.api.Client;
import com.streamhub.api.SubscriptionListener;
import com.streamhub.api.PushServer;
import com.streamhub.nio.NIOServer;
import com.streamhub.api.JsonPayload;

public class HelloWorldStreamer implements SubscriptionListener {
public static void main(String[] args) throws Exception {
new HelloWorldStreamer();
}

public HelloWorldStreamer() throws Exception {
// Create a new server on port 7878
PushServer server = new NIOServer(7878);
server.start();
server.getSubscriptionManager().addSubscriptionListener(this);
System.out.println("Comet server started at http://localhost:7878/.");
System.out.println("Press any key to stop...");
System.in.read();
server.stop();
}

public void onSubscribe(String topic, Client client) {
JsonPayload payload = new JsonPayload(topic);
payload.addField("Response", "Hello World!");
client.send(topic, payload);
}

public void onUnSubscribe(String topic, Client client) {

}
}
All we do here is create a new Payload (or message), add a field with a key of "Response" and value of "Hello World!", then send it to the client that subscribed. We'll see how to subscribe in the next step.

Step 4 - Connecting to the Comet Server and Subscribing to a Topic

Go back and edit the index.html file in the HelloWorld directory. We're going to add the code to connect and subscribe:
<html>
<head>
<title>Hello World Comet Application</title>
<script src="streamhub-min.js" type="text/javascript"></script>
</head>
<body>
<h1>Hello World Comet Application</h1>
<input type="button" value="Start streaming" onclick="startStreaming()">
<div id="streamingData"></div>
<script>
function topicUpdated(sTopic, oData) {
var newDiv = document.createElement("DIV");
newDiv.innerHTML = "Update for topic '" + sTopic + "' Response: '" + oData.Response + "'";
document.getElementById('streamingData').appendChild(newDiv);
}

function startStreaming() {
var hub = new StreamHub();
hub.connect("http://localhost:7878/");
hub.subscribe("HelloWorld", topicUpdated);
}
</script>
</body>
</html>

Connecting is really easy. We just create a new StreamHub object and connect to the Comet server we created in HelloWorldStreamer.java:
    var hub = new StreamHub();
hub.connect("http://localhost:7878/");
To subscribe we need to specify a topic and an update listener.
    hub.subscribe("HelloWorld", topicUpdated);
The topic will be passed to the onSubscribe method in our HelloWorldStreamer. The update listener is just a JavaScript function with the arguments sTopic, the topic that has been updated, and oData, a JSON object containing the Payload we sent in our onSubscribe method. You don't need to understand too much about JSON for this guide, just know that any field we put in the Payload in our HelloWorldStreamer, we can get that same field out of oData using the key. In our listener we get the 'Response' field put it inside a new div and append it to the streamingData div:
function topicUpdated(sTopic, oData) {
var newDiv = document.createElement("DIV");
newDiv.innerHTML = "Update for topic '" + sTopic + "' Response: '" + oData.Response + "'";
document.getElmentById('streamingData').appendChild(newDiv);
}
If you don't understand too much about JavaScript - this will just display every message we receive on our webpage inside the streamingData HTML element. Note: update listeners don't need to have a specific name, we've just opted to call this one topicUpdated. OK, now we're almost ready to try it out.

Step 5 - Compiling HelloWorldStreamer and Starting the Server

Just before we compile HelloWorldStreamer we need to make a couple of changes. Using addStaticContentDir we will make the HelloWorld folder available via the Comet server. Setting the file to "." means we're adding the current directory. Although StreamHub is capable of serving normal static content, for example HTML pages, it was primarily designed as a Comet server. For a live website I would recommend serving static content from a standard HTTP Web server such as Apache or IIS. However, for our 'Hello World' application this will do just fine.

To compile and run HelloWorldStreamer we just need to copy a few Java JAR libraries and the license file. The JARs can be found in the Community Edition downloaded in step one. You will need streamhub-2.0.jar, json-20080701.jar and log4j-1.2.14.jar. Mine were in the folder C:\streamhub\streamhub-java-adapter-sdk-1.0\examples\lib\jar. Copy the JARs to the HelloWorld directory. You will also need a license file, if you have an evaluation or development license copy this over to the HelloWorld directory, otherwise copy the Community Edition license over. I used the Community Edition license at C:\streamhub\streamhub-java-adapter-sdk-1.0\license.txt. You also might want to copy over the conf directory containing the log4j configuration. Now, open a command prompt, navigate to the HelloWorld directory and execute the following command to compile:
javac -cp streamhub-2.0.2.jar HelloWorldStreamer.java
Now start it up:
java -cp .;streamhub-2.0.2.jar;json-20080701.jar;log4j-1.2.14.jar HelloWorldStreamer
On Unix systems you'll need to replace the ';' with a ':', like so:
java -cp .:streamhub-2.0.2.jar:json-20080701.jar:log4j-1.2.14.jar HelloWorldStreamer
You should now see a message along the lines of:
Comet server started at http://localhost:7878/.
Press any key to stop...
Open a browser and go to the address http://localhost:7878/. You should now see the HelloWorld web page we created earlier. Click the "Start streaming" button. You should now see a message on the page saying:
Update for topic 'HelloWorld'. Response: 'Hello World!'
That's it - your first Comet message! However, one message is pretty boring, let's change the HelloWorldStreamer to continually stream messages in true Comet fashion.

Step 6 - Updating HelloWorldStreamer to Continuously Stream

Stop the server if it's still running and open HelloWorldStreamer.java for editing. First of all we're going to add a new ContinuousStreamer inner class which streams data until you stop the server. The final HelloWorldStreamer.java should look like this:
import java.io.File;
import com.streamhub.api.PushServer;
import com.streamhub.nio.NIOServer;
import com.streamhub.api.Client;
import com.streamhub.api.SubscriptionListener;
import com.streamhub.api.JsonPayload;

public class HelloWorldStreamer implements SubscriptionListener {
public static void main(String[] args) throws Exception {
new HelloWorldStreamer();
}

public HelloWorldStreamer() throws Exception {
PushServer server = new NIOServer(7878);
server.addStaticContent(new File("."));
server.start();
server.getSubscriptionManager().addSubscriptionListener(this);
ContinuousStreamer continuousStreamer = new ContinuousStreamer(server);
new Thread(continuousStreamer).start();
System.out.println("Comet server started at http://localhost:7878/.");
System.out.println("Press any key to stop...");
System.in.read();
continuousStreamer.stop();
server.stop();
}

public void onSubscribe(String topic, Client client) {
JsonPayload payload = new JsonPayload(topic);
payload.addField("Response", "Hello World! Initial Response");
client.send(topic, payload);

}

public void onUnSubscribe(String topic, Client client) {

}

private static class ContinuousStreamer implements Runnable {
private boolean isStopped = false;
private int updateNumber = 0;
private final PushServer server;

public ContinuousStreamer(PushServer server) {
this.server = server;
}

public void run() {
while (!isStopped) {
JsonPayload payload = new JsonPayload("HelloWorld");
payload.addField("Response", "Hello World! Response number " + updateNumber++);
server.publish("HelloWorld", payload);

try {
// Sleep for one second (1000ms)
Thread.sleep(1000);
} catch (InterruptedException e) {}
}
}

public void stop() {
isStopped = true;
}
}
}
Notice in the ContinuousUpdateStreamer we don't use client.send(), but instead we use server.publish(). Using client.send() is a way of sending something to an individual client - whatever you send using this method will go to this client and this client only. Whereas, using server.publish(topic, payload) will send that payload to every client who is subscribed to that topic. In most cases you should use server.publish() as it knows best who is subscribed and who isn't. Using client.send() to send an initial message inside onSubscribe is a common and good use. The only other time to prefer client.send() is when each client will be receiving unique data. Lets compile and run one more time. Now when you click the "Start streaming" button you should see something like the following:
Update for topic 'HelloWorld' Response: 'Hello World! Initial Response'
Update for topic 'HelloWorld' Response: 'Hello World! Response number 8'
Update for topic 'HelloWorld' Response: 'Hello World! Response number 9'
Update for topic 'HelloWorld' Response: 'Hello World! Response number 10'
...

The updates will keep streaming until you stop the server. That's the guide finished. The source code is available to download here: CometHelloWorld-2.0.2.zip. A good next step is to explore the examples that come with the Community Edition. We hope you found this useful - if you have any problems or suggestions for other guides and tutorials please comment.

6 comments:

  1. Hi
    Very intresting topic.
    I have one question. What happens if the hub streamer fail and is not working after some time after client subscribing. The publishing of payload stops?
    And the client stop beeing updated.
    Is there anything like load balance mechanism to set secondary server if the first is crached?

    ReplyDelete
  2. @dimitar: There are several different mechanisms available. StreamHub has sophisticated reconnection and failover features built-in. For a guide on how to use them see the posts on Configuring Failover and Reconnection in the JS API and New failover features in the GWT Adapter.

    ReplyDelete
  3. Hi,
    This clearly explains evreything. This seems like exacatly the same thing that I have been looking for. But my data streaming source is not written in Java, but in C#. So does it mean that I cannot use the comet server in my application? This is for some academic research work and we cannot afford commercial products.

    ReplyDelete
  4. Hi,

    When I run this application, rather start the server, i get a 404, what am I doing wrong?

    ReplyDelete
  5. Hi Su, We have a .NET Streaming Adapter SDK which could be exactly what you need to stream data from C#. Please contact us via our website or support@stream-hub.com and we can look into issuing you an academic license.

    ReplyDelete
  6. @Vivek: The best place to ask these kind of questions is in the StreamHub Community Group. http://groups.google.co.uk/group/streamhub-comet-server-community

    ReplyDelete