Blueweave Machine Learning

Well this time I have done it, what have I gotten myself into? I have released my own deep learning framework called blueweave. This really isn't a ground up implementation, I am standing on the shoulders of the giants over at skymind.io who have created the deep learning platform deeplearning4j. This framework is intended to accelerate and simplify the creation of deep learning networks by making configurations and data ingress and egress easier.

Deep learning is helpful to train computers on how they should look at information, then to allow them to predict how new data should be classified. This framework (so far) is focused on time series based classifications. Doing these types of classification is such a giant thing that if it was just done well it would be very valuable. Key "real world" use cases for deep learning for time series include.

• Network Monitoring and Healing
• Political and Sports Predictions
• Financial Opportunity Screening

How To Setup A Network

By using a builder pattern it is possible to create a network, and evaluate which returns a classification result.

//get the config from the classpath
Config conf = ConfigFactory.load();

//build the network
ClassifierNetwork network = new TimeseriesClassifierNetwork.TimeseriesClassifierNetworkBuilder()
        .setNetworkClasses(trainingModel.getNetworkClasses())
        .setTrainClassifications(trainingModel.getNetworkClassifications())
        .setTrainTable(trainingTable)
        .setTestTable(testingTable)
        .setConfig(conf,"TimeseriesClassifierNetwork")
        .build();

//the classification results as a table        
Table<Integer, String, Object> result = network.evaluate();

As shown in this example you need a more complex model for training because it needs the classification mappings so I think of the builder as having four key elements: classifiers, trainingClassifications, trainingData and testingData. To get a result you can properly correlate back you need all of that not just the network output. It is my opinion that the framework should do the correlation for the classification result as well as generate the network output.

Configuring Your Network

For configuration we chose to use typesafe. A simple JSON configuration framework that allows for mapping your network for different environments, or having multiple configurations by name.

{
  "TimeseriesClassifierNetwork": {
    "layers": [{
      "number":0,
      "type":"GravesLSTM",
      "activation": "tanh",
      "nIn": 1,
      "nOut": 10
    }, {
      "number":1,
      "type":"RnnOutputLayer",
      "lossFunction": "MCXENT",
      "activation": "softmax",
      "nIn": 10,
      "nOut": 5
    }],
    "optimizationAlgo": {
      "type": "STOCHASTIC_GRADIENT_DESCENT",
      "iterations": 1
    },
    "seed": 123,
    "learningRate": 0.02,
    "gradientNormalization": {
      "threshold": 0.45,
      "type": "ClipElementWiseAbsoluteValue"
    },
    "updater": {
      "type": "NESTEROVS",
      "momentum": 0.85
    },
    "weightInit": "XAVIER"
  }
}

Training Data

Training a neural network is about teaching it how to interpret new data that it sees based on data it has already experienced. To do this you need three elements:

1. The classifications you want to describe different data types.
2. Your mapping of those classifications to each series.
3. The data in each series.

Once you have established this making a training model is as easy as creating two model types. The classification model and the training data.

The training model is used to define the mapping of each series to the possible classification types that will be used to train the network. This is an example of training a network to classify a set of number series.

{
  "startDate":"2016-04-01",
  "endDate":"2016-08-01",
  "networkClasses":[
    {"id":0, "name":"cyclic"},
    {"id":1, "name":"upward-trend"},
    {"id":2, "name":"downward-trend"},
    {"id":3, "name":"upward-shift"},
    {"id":4, "name":"downward-shift"}
  ],
  "networkClassifications":[
    {"name":"PACW", "classId":0},
    {"name":"PAG",  "classId":0},
    {"name":"PAHC", "classId":4},
    {"name":"PANW", "classId":2},
    {"name":"PATK", "classId":3},
    {"name":"PATR", "classId":3},
    {"name":"PAY",  "classId":4},
    {"name":"PAYC",  "classId":1},
    {"name":"PAYX",  "classId":3}
  ]
}

And to deserialize that model for use by the builder:

//make the training 
modelSequenceNetworkModel trainingModel = GsonFactory.fromJson(
        IOUtils.toString(
                TimeseriesClassifierTest.class.getResourceAsStream("/data/01/train/trainModel01.json"), "UTF-8"),
        SequenceNetworkModel.class, GsonFactory.Type.DEFAULT);

Data Models

The data model for both training and testing are simple guava tables. An example of a serialized model for training could be:

{
  "columns": [
    "DATE",
    "PACW",
    "PAG",
    "PAHC",
    "PANW",
    "PATK",
    "PAY",
    "PAYC",
    "PAYX"
  ],
  "data": [
    [
      "2016-04-01",
      37.07,
      36.7,
      27.39,
      161.12,
      45.92,
      28.2,
      35.43,
      54.17
    ],
    [
      "2016-04-04",
      37.11,
      35.82,
      27.29,
      161.59,
      45.37,
      27.78,
      35.8,
      53.45
    ],
    [
      "2016-04-05",
      36.25,
      35.68,
      26.66,
      151.92,
      46.05,
      27.76,
      34.43,
      53.11
    ],
    [
      "2016-04-06",
      36.76,
      35.74,
      27.26,
      158.22,
      46.64,
      28.23,
      35.52,
      53.64
    ]
  ]
}

And deserializing it is as simple as using the provided factory:

Table<Date, String, Double> trainingTable = GsonFactory.fromJson(
    IOUtils.toString( TimeseriesClassifierTest.class.getResourceAsStream(
               "/data/01/train/trainTable01.json"), "UTF-8"),
    TreeBasedTable.class, GsonFactory.Type.DEFAULT);

Where Next?

This framework is in its infancy. I would love to hear from people willing to contribute and improve its coverage.