[Part 2] Implementing Deep Learning for WSO2-ML

This section is about the changes made to WSO2-ML during my project. First I will be discussing about the modules and classes that play a major role in the system. This knowledge will be important for understanding changes I’ve done.

Key Concepts of WSO2-ML

There are 5 concepts you should understand to grasp WSO2-ML well. They are,

  • Dataset – A Dataset can be uploaded as a CSV or TSV
  • Dataset version – A Dataset can have multiple versions but will have the same schema
  • Analysis – An analysis is a single task which comprises of pre-processed feature set, a selected algorithm and its hyper-parameters, and the results(e.g. accuracy)
  • Project – A Project is a collection of Analyses performed on a single dataset
  • Model – A representation of an analysis for storage

I would not go into details about these concepts here. WSO2 website has a very good explanation in their website.

Core (org.wso2.carbon.ml.core)

carbon-ml contains this module along with several other modules. However, in my project I did not have to worry about other modules. Furthermore, org.wso2.carbon.ml.core contains all the core implementations where rest of the modules mostly act as infrastructure.

So, let’s dive in!

The following diagram shows an architectural breakdown of the modules and sub-modules that were involved in my project.

High-level Architecture of WSO2-ML
Here, blue blocks are the components affected during my project.

A Technical Description of My Project

As said many times, my project was implementing deep learning for WSO2-ML. For this we used H2O Server an underlying server which provides deep learning as a service. Therefore, I only had to worry about the client side of that and merge it seamlessly with the existing WSO2-ML.

In my project I did the following
Introduce Deep Hyper Parameters (Jaggery UI,
Implementing a Deep Classifier which can send data to the server and retrieve the results
Implement a Deep Learning Model Builder which creates a classifier retrieve the results and transform the results and data accordingly
Implement Utility classes for transforming data for compatibility (Feature set and Test results)
Implement Serialization/Deserialization for Deep Learning Model (This in particular was very challenging as H2O uses a custom serialization mechanism)
Implement Visualization for Deep Networks in Jaggery

Jaggery UI

WSO2-ML has a web UI developed using Jaggery.

REST API (org.wso2.carbon.ml.rest.api)

The UI and underlying Java business logic communicate via the REST API. REST API has following components.

  • Analysis API
  • Configuration API
  • Dataset API
  • LoginLogout API
  • ML Rest API
  • Model API
  • Project API