Using DataSift with Codenvy

Instantly create a DataSift project off a Codenvy Factory:


DATASIFT is mainly used to track the information from the Social Networks like Facebook, Twitter and YouTube etc. DATASIFT provides different Client API’s to track this information from the DATASIFT. It provides two different APIs

  • REST API – Provides the core functionality of the DataSift Services
  • STREAM API – provides the information in Real Time

Before using the Datasift API User need to be registered at http://www.datasift.com. Where user can get the Authentication details for using the APIs. After Login into DATASIFT site user will get loginID and API Key to use in Client Applications.

DataSift Dashboard Provides information about the Streams created, API Usage Details, Billing and other information. Using some specific services like twitter DATASIFT will charge for the User.
datasift
Datasift Provides live API Usage page at https://console.datasift.com/datasift where user can test TheDatasift Core API.
coreapi
In this Tutorial we are going to use the Core API to get the DPU cost for using the DataSift Stream API.
For this we need the Stream Hash Value. Which user need to provide in the below screen:

api

Stream Hash Value can be get from Dashboard. Where user can create his own streams.

companywebsite

The result of the calling the API we’ll get the DPI Cost as below:

callapi callapi

Create a Project in Codenvy

project_structure

Create Servlet

package com.codenvy;

import org.apache.http.Header;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.BasicResponseHandler;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.util.EntityUtils;

import java.io.IOException;

import javax.servlet.RequestDispatcher;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class DataSiftServlet extends HttpServlet{

private static final long serialVersionUID = 1L;
private HttpClient client = new DefaultHttpClient();

  protected void doGet(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {

        StringBuffer sb = null;
        try {
            String getURL = "http://api.datasift.com/dpu?hash=42b401b822b147d9950a9d8ddb4e62e5&api_key=a98bd731b76db1b1220e67e1acbea8ce&&username=anil14349";

            HttpGet req = new HttpGet(getURL);

            HttpResponse res = client.execute(req);
            HttpEntity entity = res.getEntity();
            String responseString = new BasicResponseHandler().handleResponse(res);
            if (entity != null) {
                EntityUtils.consume(entity);
            }

            sb = new StringBuffer("<br/>Simple GET request to url : "+ getURL +"<br/><br/>");
            sb.append("*** Request headers ***").append("<br/>");
            Header[] requestHeaders = req.getAllHeaders();
            for (Header header : requestHeaders) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response headers ***").append("<br/>");
            sb.append(res.getStatusLine()).append("<br/>");
            Header[] headers = res.getAllHeaders();
            for (Header header : headers) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response Body ***").append("<br/>");
           for(String s:str){
              sb.append(s).append("<br/>");
            }
           // sb.append(responseString).append("<br/>");

        } catch (Exception ex) {
            sb = new StringBuffer(ex.toString());
        }

        request.setAttribute("data", sb.toString());
        RequestDispatcher rd = request.getRequestDispatcher("/success.jsp");
        rd.forward(request, response);
   }
}

In this servlet we are calling the Rest API of Datasift for retrieving the DPI cost of a stream.In response we’ll receive information in JSON Objects as shown below.

Create index.jsp and success.jsp Files

We’ll call the Servlet from a Jsp.
Index.jsp:

<%
    response.sendRedirect("/datasift");
%>

We’ll retrieve the information in success.jsp
Success.jsp:

<html>
  <body>
    <div>
      <p>Data : <br/><%=request.getAttribute("data") %></p>
      <br/>
    </div>
 </body>
</html>

Create pom.xml File

We need to specify the dependencies in pom.xml as shown below:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>com.codenvy</groupId>
  <artifactId>DataSift-web</artifactId>
  <packaging>war</packaging>
  <version>1.0-SNAPSHOT</version>
  <name>java-web-sample</name>
  <dependencies>
  <dependency>
    <groupId>javax.servlet</groupId>
    <artifactId>servlet-api</artifactId>
    <version>2.5</version>
  </dependency>
  <dependency>
    <groupId>org.apache.httpcomponents</groupId>
    <artifactId>httpclient</artifactId>
    <version>4.2.5</version>
  </dependency>
  <dependency>
    <groupId>javax</groupId>
    <artifactId>javaee-web-api</artifactId>
    <version>6.0</version>
  </dependency>
  </dependencies>
  <build>
    <finalName>java-web-sample</finalName>
  </build>
</project>