Using BigML with Codenvy

Below are various different projects talking to BigML API that you can create with Codenvy Factories:
Sample 1 :

Sample 2 :


This document will explain the details of integrating the BigML API into Codenvy. BigML API is a set of powerful APIs that assists users in creating and managing BigML resources such as Sources, Datasets, Models, Ensembles, Predictions, and Evaluations programmatically. Using BigML.io users can create, retrieve, update and delete Sources, Datasets, Models, Ensembles, Predictions, and Evaluations using standard HTTP methods.. The explanation has been done with 3 very basic Java servlet projects that make use of the BigML API to explain some basic features provided by the API.

Before proceeding to the creation of the Java projects, please register at https://bigml.com/accounts/register/ with your email id and this will generate your API key and user name which will be used for the integration. You can see your keys at https://bigml.com/account/apikey :

bigml_apikeys

The BigML API has various methods to integrate with your application for all the BigML resources.

You can get more information about the API methods at https://bigml.com/developers/.

The first factory workspace source code is to explain the process of retrieving the details of a Source using the BigML API. The input will be the unique source id and the output will be in JSON format displayed on the browser.

Create the first Project

Login to your Codenvy workspace and create a WAR project named as ‘BigMLSample’. This will have the following structure:

bigml_structure

Create pom.xml File

To make use of the BigML API through HttpClient as a servlet, please update the pom.xml to include the HttpClient and Java EE Web API jars and the servletapi jar and the Json jar into your project as Maven dependencies :

<dependencies>
     <dependency>
      <groupId>javax.servlet</groupId>
      <artifactId>servlet-api</artifactId>
      <version>2.5</version>
    </dependency>
    <dependency>
      <groupId>org.apache.httpcomponents</groupId>
      <artifactId>httpclient</artifactId>
      <version>4.2.5</version>
    </dependency>
    <dependency>
      <groupId>javax</groupId>
      <artifactId>javaee-web-api</artifactId>
      <version>6.0</version>
    </dependency>
    <dependency>
      <groupId>com.googlecode.json-simple</groupId>
      <artifactId>json-simple</artifactId>
      <version>1.1.1</version>
    </dependency>
  </dependencies>

This will include these jars in your project as follows:

bigml_dependencies

Create Java class

Now create a Java servlet class named ‘BigMLServlet.java’ that will talk to the BigML website with the Http request URL that will invoke the relevant BigML resource and then get a Http response back with the response header and the response body in the JSON format which will be then displayed on the browser through a JSP. All this is done inside the method doGet() in the sample source Java class:

  protected void doGet(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {

        StringBuffer sb = null;
        try {
            String getURL = "https://bigml.io/andromeda/source/5277b1bd035d074e940056e0?username=dasn82;api_key=7df509b00ff73d37bafcbff9ad540f039e6a92e5";

            HttpGet req = new HttpGet(getURL);
            req.addHeader("X-Originating-Ip", "80.254.147.148");
            HttpResponse res = client.execute(req);
            HttpEntity entity = res.getEntity();
            String responseString = new BasicResponseHandler().handleResponse(res);
            if (entity != null) {
                EntityUtils.consume(entity);
            }

            sb = new StringBuffer("<br/>Simple GET request to url : "+ getURL +"<br/><br/>");
            sb.append("*** Request headers ***").append("<br/>");
            Header[] requestHeaders = req.getAllHeaders();
            for (Header header : requestHeaders) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response headers ***").append("<br/>");
            sb.append(res.getStatusLine()).append("<br/>");
            Header[] headers = res.getAllHeaders();
            for (Header header : headers) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response Body ***").append("<br/>");
            sb.append(responseString).append("<br/>");

        } catch (Exception ex) {
            sb = new StringBuffer(ex.toString());
        }

        request.setAttribute("data", sb.toString());
        RequestDispatcher rd = request.getRequestDispatcher("/success.jsp");
        rd.forward(request, response);
   }

This code retrieves the details of the Source data that you request in the HTTP url. In a similar way, you can retrieve details of any other BigML resource.
The HTTP request URL contains your BigML username, your BigML api key, the BigML resource name that you want to invoke and the unique identifier of the resource such as the Source Id. You can see the Id of the particular entity in your account dashboard under the appropriate section :

bigml_dashboard

Create index.jsp File

Your Codenvy project will have an index.jsp which will invoke the Servlet class:

<%
    response.sendRedirect("/hello");
%>

Your Codenvy project will have a success.jsp which will be invoked from the BigMLSample class to display the response on the browser:

<html>
  <body>
    <div>
      <p>Data : <br/><%=request.getAttribute("data") %></p>
      <br/>
    </div>
 </body>
</html>

Create web.xml

The servlet mapping ‘/hello’ is defined in the web.xml as follows:

<!DOCTYPE web-app PUBLIC
 "-//Sun Microsystems, Inc.//DTD Web Application 2.3//EN"
 "http://java.sun.com/dtd/web-app_2_3.dtd" >

<web-app>
  <display-name>BigML Servlet App</display-name>
  <servlet>
    <servlet-name>hello</servlet-name>
    <servlet-class>com.codenvy.BigMLServlet</servlet-class>
  </servlet>

  <servlet-mapping>
    <servlet-name>hello</servlet-name>
    <url-pattern>/hello</url-pattern>
  </servlet-mapping>
</web-app>

Build and Run Your first Project

Now you build and run your Codenvy project and the Http JSON response from the BigML website with the details for the Source supplied as input will be displayed in the browser:

bigml_runapp

The second factory workspace source code is to explain the process of creating a Dataset in your BigML account from your Source that you retrieved in the first sample code :

Create the second Project

Login to your Codenvy workspace and create a WAR project named as ‘BigMLSample1’. This will have the same structure as the first project and the pom.xml will define the same dependencies as the first project.

Create Java class

Now create a Java servlet class named ‘BigMLServlet.java’ that will talk to the BigML website with the Http POST request URL that will invoke the BigML Dataset resource to create a Dataset for the supplied Source Id and then get a Http response back with the response header and the response body in the JSON format which will be then displayed on the browser through a JSP. All this is done inside the method doGet() and doPost() in the sample source Java class. On successful completion of the request, the newly created Dataset would be visible in your BigML account dashboard:

   protected void doGet(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {

doPost(request, response);

   }

   protected void doPost(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {

        StringBuffer sb = null;
        try {
            String postURL = "https://bigml.io/andromeda/dataset?username=dasn82;api_key=7df509b00ff73d37bafcbff9ad540f039e6a92e5;";

            HttpPost req = new HttpPost(postURL);
            req.setHeader("Content-Type", "application/json");

            JSONObject requestObject = new JSONObject();
            requestObject.put("source",  "source/5277b1bd035d074e940056e0");

            StringEntity reqEntity = new StringEntity(requestObject.toJSONString());
            req.setEntity(reqEntity);

            HttpResponse res = client.execute(req);
            HttpEntity entity = res.getEntity();
            String responseString = new BasicResponseHandler().handleResponse(res);
            if (entity != null) {
                EntityUtils.consume(entity);
            }

            sb = new StringBuffer("<br/>Simple POST request to url : "+ postURL +"<br/><br/>");
            sb.append("*** Request headers ***").append("<br/>");
            Header[] requestHeaders = req.getAllHeaders();
            for (Header header : requestHeaders) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response headers ***").append("<br/>");
            sb.append(res.getStatusLine()).append("<br/>");
            Header[] headers = res.getAllHeaders();
            for (Header header : headers) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response Body ***").append("<br/>");
            sb.append(responseString).append("<br/>");

        } catch (Exception ex) {
            ex.printStackTrace();
            sb = new StringBuffer(ex.toString());
        }

        request.setAttribute("data", sb.toString());
        RequestDispatcher rd = request.getRequestDispatcher("/success.jsp");
        rd.forward(request, response);
   }

This code invokes the BigML Dataset resource with the HTTP POST url to create the dataset.
The HTTP POST request URL contains your BigML username, your BigML api key, the BigML resource name that you want to invoke which is Dataset. The Source Id from which the Dataset will be created will be passed as a JSON string parameter to the HTTP POST request.

Create index.jsp File

Your Codenvy project will have an index.jsp similar to the first project:

Your Codenvy project will have a success.jsp which will be invoked from the BigMLSample class to display the response on the browser

The web.xml will be same as the first project.

Build and Run Your second Project

Now you build and run your Codenvy project and the Http JSON response from the BigML website with the details for the created Dataset will be displayed in the browser:

bigml1_runapp

The newly created Dataset will now be visible in your BigML dashboard:

bigml1_dashboard

The third factory workspace source code is to explain the process of creating a Model in your BigML account from your Dataset that you created in the second sample code :

Create the third Project

Login to your Codenvy workspace and create a WAR project named as ‘BigMLSample2’. This will have the same structure as the first project and the pom.xml will define the same dependencies as the first project.

Create Java class

Now create a Java servlet class named ‘BigMLServlet.java’ that will talk to the BigML website with the Http POST request URL that will invoke the BigML Model resource to create a Model for the supplied Dataset Id and then get a Http response back with the response header and the response body in the JSON format which will be then displayed on the browser through a JSP. All this is done inside the method doGet() and doPost() in the sample source Java class. On successful completion of the request, the newly created Model would be visible in your BigML account dashboard:

    protected void doGet(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {

doPost(request, response);

   }

  protected void doPost(HttpServletRequest request, HttpServletResponse response)
        throws ServletException, IOException {

        StringBuffer sb = null;
        try {
            String postURL = "https://bigml.io/andromeda/model?username=dasn82;api_key=7df509b00ff73d37bafcbff9ad540f039e6a92e5";

            HttpPost req = new HttpPost(postURL);
            req.setHeader("Content-Type", "application/json");

            JSONObject requestObject = new JSONObject();
            requestObject.put("dataset",  "dataset/527866f0035d074e9100ed06");

            StringEntity reqEntity = new StringEntity(requestObject.toJSONString());
            req.setEntity(reqEntity);

            HttpResponse res = client.execute(req);
            HttpEntity entity = res.getEntity();
            String responseString = new BasicResponseHandler().handleResponse(res);
            if (entity != null) {
                EntityUtils.consume(entity);
            }

            sb = new StringBuffer("<br/>Simple POST request to url : "+ postURL +"<br/><br/>");
            sb.append("*** Request headers ***").append("<br/>");
            Header[] requestHeaders = req.getAllHeaders();
            for (Header header : requestHeaders) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response headers ***").append("<br/>");
            sb.append(res.getStatusLine()).append("<br/>");
            Header[] headers = res.getAllHeaders();
            for (Header header : headers) {
                sb.append(header.toString()).append("<br/>");
            }
            sb.append("***********************").append("<br/><br/>");

            sb.append("*** Response Body ***").append("<br/>");
            sb.append(responseString).append("<br/>");

        } catch (Exception ex) {
            ex.printStackTrace();
            sb = new StringBuffer(ex.toString());
        }

        request.setAttribute("data", sb.toString());
        RequestDispatcher rd = request.getRequestDispatcher("/success.jsp");
        rd.forward(request, response);
   }

This code invokes the BigML Model resource with the HTTP POST url to create the Model.
The HTTP POST request URL contains your BigML username, your BigML api key, the BigML resource name that you want to invoke which is Model. The Dataset Id from which the Model will be created will be passed as a JSON string parameter to the HTTP POST request.

Create index.jsp File

Your Codenvy project will have an index.jsp similar to the first project:

Your Codenvy project will have a success.jsp which will be invoked from the BigMLSample class to display the response on the browser

The web.xml will be same as the first project.

Build and Run Your third Project

Now you build and run your Codenvy project and the Http JSON response from the BigML website with the details for the created Model will be displayed in the browser:

bigml2_runapp

The newly created Model will now be visible in your BigML dashboard:

bigml2_dashboard

In a similar way, you can use various API methods of BigML to invoke the various BigML recources that may be required within your application.