PredictServer Info

BigML PredictServer
Documentation

Overview

Last Updated: Thursday, 2013-09-19 23:59

The BigML PredictServer is a dedicated service that can be deployed in your local network or your cloud to provide fast and reliable predictions, centroids and anomalyscores from BigML models, ensembles, clusters, anomaly detectors, and more. Features include:

Easy — the API is similar to bigml.io and can be easily swapped in to support code integrated with BigML.io
Fast — the predict server keeps everything in RAM and creates predictions in parallel. Performance is roughly:
1000 /(number of models in an ensemble) * (number of cores) predictions/s
Reliable — because the machine is dedicated, there is no competition for resources.
Low Latency — the image can be deployed with any Cloud provider worldwide or to your local network, making it possible to lower the latency for predictions from your application.
Secure — use security groups to limit access, or even deploy the image in a VPC.
Scalable — performance scales with CPUs. You can instantiate a single or multiple BigML PredictServers and seamlessly integrate them into your existing data center.

Quick Start

Last Updated: Tuesday, 2023-10-24 23:59

Deploy the BigML PredictServer with the image name you'll be provided with.

Download and run the Docker container executing:


docker pull provided-image-name
docker run --name carnac -v cache:/usr/local/carnac -p 8080:3000 -p 8081:3001 -p 8082:3002 -dit provided-image-name

Change the auth_token for the BigML PredictServer. In the following example, we are changing the auth_token to mytoken:


curl "https://your-predict-server/config?auth_token=i-54f1bc39" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"auth_token":"mytoken"}'

Changing PredictServer auth_token

Add a list of BigML usernames that are allowed to access the PredictServer. In this example, we are allowing the users wendy and shara:
```
curl "https://your-predict-server/config?auth_token=mytoken" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"allow_users": [ "wendy", "shara" ]}'
```
Authorizing BigML users to access the PredictServer

Cache a model on the PredictServer using the same username and api_key from bigml.io


curl "https://your-predict-server/model/51df6f52035d0760380042e7?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X GET

Caching a model on the PredictServer

Create a prediction


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f"
-k -X POST \
-H 'content-type: application/json' \
-d '{"model":"model/51df6f52035d0760380042e7", "input_data":{ "education-num": 15 }}'

Predicting with your model

Detailed Start

Last Updated: Tuesday, 2023-10-24 23:59

This page helps you to get started with BigML PredictServer.

Change the auth_token

The auth_token is a simple string like a password that protects the configuration of the BigML PredictServer and secures access to the dashboard. We recommend changing the default to something that only you will know. To do so, you can use a command like curl, sending JSON in the POST body with the configuration option auth_token like below. In this example, we are changing the auth_token from its default of mytoken to my_token:


curl "https://your-predict-server/config?auth_token=mytoken" \
-k -X POST \
-H 'content-type: application/json' -d '{"auth_token":"my_token"}'

# Response: {"status":{"code":0,"message":"OK."},"code":200}

Changing BigML PredictServer auth_token

The "-k" is required because by default the BigML PredictServer uses a self-signed certificate and this option tells curl not to worry about it.

Adding BigML Users

The BigML PredictServer caches objects from BigML.io using the same username and api_key credentials that are defined in BigML.io. Because all such users are valid as far as BigML.io is concerned, the BigML PredictServer maintains a list of BigML usernames that are allowed to cache objects on your instance of the BigML PredictServer. This configuration parameter is called "allow_users" and is a simply JSON list of valid BigML usernames. In the following example, we are allowing the users wendy and shara access to the BigML PredictServer:


curl "https://your-predict-server/config?auth_token=mytoken" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"allow_users": [ "wendy", "shara" ]}'

# Response: {"status":{"code":0,"message":"OK."},"code":200}

Authorizing BigML users to access the BigML PredictServer

Note:

The list of users can not be modified by adding or removing a single user, it must always be replaced with the full list of allowed users.
When you remove a user from the allow_users list, any cached objects owned by that user will automatically be removed.

Working with Cached Objects

Before making predictions, centroids, or anomalyscores, the BigML PredictServer needs to fetch the assoicated model, ensemble, cluster or anomaly detector object from BigML.io and load it into memory. You can either control this process manually using GET and DELETE to the /model, /ensemble, /cluster, and /anomaly endpoints, or you can use passive loading.

Passive loading occurs when a prediction, centroid or anomalyscore is requested against an object which is not currently in memory. The BigML PredictServer will respond with a status.code!=5 and automatically being the caching process. Subsequent prediction requests against the same object will result in a status.code!=5 until the object is ready to generate predictions.

All /model, /ensemble, /cluster, /anomaly, /prediction, /centroid and /anomalyscore requests must include a valid BigML username and api_key, even the DELETE requests. This not only gives the BigML PredictServer access to the requested object thru BigML.io, but provides security for the cached objects within the BigML PredictServer.

Finally, it is important to understand that the DELETE method for models, ensembles, clusters, and anomalies only removes the object from the BigML PredictServer memory, it does not delete the object from BigML.io.


curl "https://your-predict-server/model/51df6f52035d0760380042e7?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X GET

# Response: {"status":{"code":1,"message":"Download started."},"code":200}

Direct caching of a model


curl "https://your-predict-server/model/51df6f52035d0760380042e7?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X DELETE

# Response: /No body/

Removing a cached model


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/51e7267c035d076038004e7c", "input_data":{ "education-num": 15 }}'

# Response: {"status":{"code":1, "message":"Download started."},"code":200}

Passively loading an ensemble

Please note that when working with resources (models, ensembles, clusters, etc.) belonging to a BigML Organization, the authentication part of the URL needs to include additionally the Project ID where the resource is located. For instance, direct caching of a model that exists in a particular organization's project (e.g. project/51df6f52075d07603a0041a3) can be done by user 'wendy', provided she is a member of the organization with access to that project, as follows:


curl "https://your-predict-server/model/51df6f52035d0760380042e7?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f;project=project/51df6f52075d07603a0041a3" \
-k -X GET

# Response: {"status":{"code":1,"message":"Download started."},"code":200}

Direct caching of a model belonging to an organization

Read more about BigML Organizations in the documentation.

Creating Predictions

There are two ways to make predictions, centroids and anomalyscores.: one at a time or in batch. The request to make a single prediction, centroid or anomalyscore is the same as for the BigML.io: submit a POST request to /prediction, /centroid, /anomalyscore with the required JSON parameters in the body. The response will be similar to BigML.io, although several fields will be missing since the BigML PredictServer creates responses and forgets them immediately.


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/51e7267c035d076038004e7c", "input_data":{ "bmi": 100 }}'

# Prediction:
    {
      "created": "2015-01-02T20:48:31.190Z",
      "resource": "prediction/1420231711190-2153",
      "objective_fields": [
        "000008"
      ],
      "status": {
        "elapsed": 0,
        "code": 5,
        "message": "The prediction has been created"
      },
      "output": "false",
      "prediction": {
        "000008": "false"
      },
      "code": 200,
      "confidence": 0.608744,
      "prediction_path": {}
    }

A single ensemble prediction

Rather than one at a time, if you have a large list of input data with which to make predictions, centroids, or anomlyscores then you can upload them all at once as a CSV file. The structure of the CSV file needs to conform to:

The first line, called the header, contains the labels for the input data.
Each subsequent line contains the input_data values for a single prediction, centroid or anomalyscore.

With this in place, the input file can be uploaded to the BigML PredictServer and then predictions, centroids, or anomalyscores can be requested against the entire file. The response will be a CSV stream of "prediction,confidence" rows for predictions, "centroid_id,centroid_name,distnace" rows for centroids, and "score" rows for anomalyscore. The uploaded source can be re-used for subsequent batch requests, however the source will be lost if the BigML PredictServer instance is rebooted.


curl "https://your-predict-server/source?auth_token=mytoken" \
-k -X POST \
-F "file=@predict.csv"

# Response: {"status":{"code":0,"message":"OK."},"code":200,"source_id":"2151-urk"}

Uploading a source

Then submit a prediction request with the field "input_source":


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/51e7267c035d076038004e7c", "input_source":"2151-urk"}'

# Prediction:
prediction,confidence
true,0.6912699999999999
false,0.9414733333333333
true,0.7348075000000001
false,0.960758
true,0.7462820000000001
false,0.9194770000000002
true,0.37081749999999997
true,0.55583
true,0.72162625
...

A batch prediciton

Working with Bindings

The BigML PredictServer is compatible with any of the BigML.io bindings that support changing the default API domain. Here is an example in Python of caching a ensemble, creating a prediction, and removing the ensemble from the cache.



# Setup the bindings. One "api" connected to bigml.io, the other "ps" connected to a PredictServer running at your-predict-server
>>> from bigml.api import BigML
>>> api = BigML(username="wendy", api_key="92127b85415db7caa2ca985edfdbcaca766d836f", domain="bigml.io")
>>> ps = BigML(username="wendy", api_key="92127b85415db7caa2ca985edfdbcaca766d836f", domain="your-predict-server")

# List the ensemble ids in bigml.io
>>> [ el['resource'] for el in api.list_ensembles()['objects'] ]
['ensemble/54a72f7d67dc092c1d00006f']

# List the ensemble ids in the PredictServer
>>> [ el['resource'] for el in ps.list_ensembles()['objects'] ]
[]

# Load ensemble/54a72f7d67dc092c1d00006f from bigml.io into the PredictServer. This is asynchronous!
>>> ensemble = ps.get_ensemble("ensemble/54a72f7d67dc092c1d00006f")
{'code': 200, 'resource': 'ensemble/54a72f7d67dc092c1d00006f', 'location': 'https://your-predict-server/andromeda/ensemble/54a72f7d67dc092c1d00006f', 'object': {'status': {'code': 1}, 'code': 200, 'resource': 'ensemble/54a72f7d67dc092c1d00006f'}, 'error': None}

# Wait until the ensemble is fully cached by all workers
>>> ps.ok(ensemble)
True

# Show that the ensemble is loaded into the PredictServer cache
>>> [ el['resource'] for el in ps.list_ensembles()['objects'] ]
['ensemble/54a72f7d67dc092c1d00006f']

# Create a prediction in the PredictServer with the ensemble
>>> ps.create_prediction(ensemble, { "input_data": { "bml": 100 }})
{'code': 200, 'resource': 'prediction/1420243515175-2554', 'location': None, 'object': {'status': {'message': 'The prediction has been created', 'code': 5, 'elapsed': 0}, 'confidence': 0.60994, 'code': 200, 'resource': 'prediction/1420243515175-2554', 'created': '2015-01-03T00:05:15.175Z', 'objective_fields': ['000008'], 'prediction': {'000008': 'false'}, 'prediction_path': {}, 'output': 'false'}, 'error': None}

# Remove the ensemble from the PredictServer cache
>>> ps.delete_ensemble(ensemble)
{'code': 204, 'error': None}

# Verify that the ensemble still exists in bigml.io but not in the PredictServer
>>> [ el['resource'] for el in api.list_ensembles()['objects'] ]
['ensemble/54a72f7d67dc092c1d00006f']

>>> [ el['resource'] for el in ps.list_ensembles()['objects'] ]
[]

Using the Python bindings with the PredictServer

Working with Endpoints

Endpoints provide a more abstract way to identify resources than through its resource IDs. For example, say you are working on two different projects, e.g., FinRisk and FinChurn. Your system is setup to retrain your models each week with your latest data and you have automated scripts that use the latest models to make predictions. Every time you retrain your models, you will need to update your scrips so they know which model to use. Or, you could implement your prediction scripts so they find out which are the most recent models in your account, e.g., based on their names or the project they belong to.

That is pretty much a simple task, using BigML.io API, but BigML PredictServer Endpoints will make your life even easier. In fact, in the context of this example, you can see a PredictServer Endpoint as a named query you define once and use thereafter as if it were the unique identifier of the resource you want to work with. Every time you use an endpoint to identify a resource, BigML PredictServer will silently make a query and resolve the endpoint to a unique resource ID.

You create a BigML PredictServer Endpoint as any other resources through the BigML PredictServer API by submitting a POST request to /endpoint.


curl "https://your-predict-server/endpoint?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"endpoint_name":"FinRisk", "endpoint_query": "name__contains=FinancialRiskModel"}'

# Result:
{
    "code":201,
    "endpoints": {
        "FinRisk": {
            "name":"FinRisk",
            "query":"name__contains=FinancialRiskModel"
        }
    }
}

Creating an endpoint

Similarly, you can list all existing endpoints submitting a GET to /endpoint.


curl -k "https://your-predict-server/endpoint?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f"

# Result:
{
  "status": {
    "code": 0,
    "message": "OK."
  },
  "code": 200,
  "endpoints": [
    {
      "name": "FinRisk",
      "query": "name__contains=FinancialRiskModel"
    }
  ]
}

Listing endpoints

Once your endpoint is created, you can use its name to identify your models. For example, say you want to retrieve the latest ensembles for the FinRisk endpoint. You can execute:


curl -k "https://your-predict-server/endpoint/FinRisk/ensemble?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f"

# Result:
{"status":{"code":0,"message":"OK. \"\""},"code":200,"resources":[
  { --ensemble-1--}, { --ensemble-2--}, ...]}

Listing ensembles associated to an endpoint

If you only want your latest ensemble, you can execute:


curl -k "https://your-predict-server/endpoint/FinRisk/ensemble/latest?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f"

# Result:
{"status":{"code":0,"message":"OK. \"\""},"code":200,"resource":{"boosted_ensemble":false,"boosting":{},"category":0,"cluster":null,"cluster_status":false,"code":200,"columns":9,"configuration":null,"configuration_status":false,"created":"2019-01-17T16:21:28.902000","creator":"sdesimone","dataset":"dataset/5c40ab813980b52956000ff4","dataset_field_types":{"categorical":1,"datetime":0,"items":0,"numeric":8,"preferred":9,"text":0,"total":9},"dataset_status":true,"depth_threshold":512,"description":"My first BigML Node-RED Ensemble","ensemble":true,"ensemble_id":"5c40ab843980b52964005859","ensemble_index":9,"execution_id":"5c40ab7e7811dd58ed000bec","execution_status":true,"focus_field":null,"locale":"en_US","max_columns":9,"max_rows":768,"missing_splits":false,"name":"diabetes - 9","name_options":"512-node, pruned, deterministic order","node_threshold":512,"number_of_batchpredictions":0,"number_of_evaluations":0,"number_of_predictions":0,"number_of_public_predictions":0,"objective_field":"000008","objective_field_name":"diabetes","objective_field_type":"categorical","objective_fields":["000008"],"optiml":null,"optiml_status":false,"ordering":0,"out_of_bag":false,"price":0,"private":true,"project":null,"randomize":false,"range":null,"replacement":false,"resource":"model/5c40ab883514cd3747006a05","rows":768,"sample_rate":1,"selective_pruning":true,"shared":false,"size":26191,"source":"source/5c40ab7e3980b5295e00656d","source_status":true,"split_candidates":32,"split_field":null,"stat_pruning":true,"status":{"code":5,"message":"The model has been created","progress":1},"subscription":true,"support_threshold":0,"tags":["ProductionEnsemble"],"type":0,"updated":"2019-01-17T16:27:50.605000","white_box":false}}

Finding the latest ensemble associated to an endpoint

Endpoint identifiers can also be used for predictions. Their behaviour is just the same as with any other resource type. This is how you can predict using the latest ensemble.


curl
"https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-X POST \
-H 'content-type: application/json' \
-d '{ "ensemble":"endpoint/FinRisk/ensemble/latest", "input_data" : {...}}'

# Result:
{
  "created":"2020-02-04T11:33:19.167Z",
  "resource":"prediction/1580815999167-16375",
  "status": {
     "code":5,
     "message":"The prediction has been created",
     "elapsed":0 },
  "code":200,
  "input_data":{},
  "ensemble":"ensemble/5e3425fc440ca1324409cc66",
  "output":"Iris-setosa",
  "confidence":0.27519,
  "probability":0.34667,
  "unused_fields":[],
  "objective_field":"000004",
  "objective_fields":["000004"],
  "prediction":{"000004":"Iris-setosa"}
}

Making a prediction from an endpoint

API Documentation

Last Updated: Tuesday, 2023-10-24 23:59

The BigML PredictServer works like a cache, loading models, ensembles, clusters and anomaly detectors from BigML.io into memory and then using multiple threads to create predictions as quickly as possible. Because the BigML PredictServer works like a caching-proxy, the API calls are similar to the BigML.io REST API. Importantly, the username and api_key used for interacting with models, ensembles and predictions on the BigML PredictServer need to be valid credentials for bigml, since they are merely proxied.

The BigML PredictServer also supports several endpoints that are not proxied to BigML.io, specifically for configuring the service and monitoring the performance. To distinguish the authentication, these endpoints use an authorization token specific to the BigML PredictServer called the auth_token.

In this page you'll learn more about how to cache models, ensembles, logistic regressions, linear regressions, deepnets, fusions, clusters, and anomaly detectors, fetching them from BigML.io and then using the cached object to create predictions (with the supervised models), centroids, or anomalyscores respectively. You will also find instructions for creating predictions in batch, centroids in batch, and anomalyscores in batch which requires an uploaded source. Finally, you'll find how to configure your BigML PredictServer to control access, change the API domain for VPCs and other settings.

Listing, fetching, and deleting resources

The basic URL carry through operations on resources in the BigML PredictServer is the same as the BigML.io API:

The BigML PredictServer supports all BigML resources, including Models, Ensembles, Clusters, Anomalies, Deepnets, and so on.

https://{hostname}/{resource-type}/{id}?username={username}&api_key={api_key}

Resource URL

However the only methods that are supported are:

GET: This fetches the resource {id} from BigML.io and loads it into the BigML PredictServer memory.
DELETE: This removes the resource {id} from memory — it does not delete the resource from BigML.io.

In addition, the GET call without an id will list the resources that are cached in the BigML PredictServer. It will only show resources that were loaded using the same username and api_key pair, not all resources.

Below, you can see a few examples of use.


curl "https://your-predict-server/model?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X GET

Listing models URL


curl "https://your-predict-server/ensemble?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X GET

Listing ensembles URL

When you specify a resource ID, because the caching process is asynchronous, the first GET will result in a status.code=1.


curl "https://your-predict-server/model/51df6f52035d0760380042e7?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X GET

# Response: {"status":{"code":1,"message":"Download started."},"code":200}

Caching a model

or if the model exists in an organization's project (e.g. project/51df6f52075d07603a0041a3) and user 'wendy' is a member of the organization and has access to that project:


curl "https://your-predict-server/model?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f;project=project/51df6f52075d07603a0041a3" \
-k -X GET

# Response: {"status":{"code":1,"message":"Download started."},"code":200}

Caching a model belonging to an organization

Use this authentication query string part when working with Organization resources.

Subsequent GETs to the same URL will return status.code=3 until the model is loaded and ready to make predictions, at which point the BigML PredictServer will return a status.code=5:


curl "https://your-predict-server/model/51df6f52035d0760380042e7?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X GET

# Response: {"status":{"code":5,"message":"Already downloaded."},"code":200}

Fetching a model already cached


curl "https://your-predict-server/model/51df6f52035d0760380042e7?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X DELETE

# Response: /No body/

Removing a cached model

All the examples listed above apply to all supported resource by changing the resource type, e.g., anomaly instead of model, and the relevant resource ID.

Sources

Despite the name, BigML PredictServer sources have nothing to do with BigML.io sources.

For the fastest predictions, the BigML PredictServer supports batch predictions. This feature works by uploading a CSV containing the input data as a row for each prediction and then submitting the prediction request with the source_id assigned during upload. The prediction results are then streamed directly over HTTP. This eliminates the network latency of calling, waiting, and returning the predictions one at a time.

The basic URL for sources in the BigML PredictServer is:

https://{hostname}/source/{id}?auth_token={auth_token}

Source URL

The supported methods are:

GET: This fetches the source {id}.
POST: This uploads a new source. This returns a unique {id} for the source. Note:
sources are stored on ephemeral disk and will be lost if the instance is rebooted.
DELETE: This deletes the source from the BigML PredictServer.

In addition, the GET call without an id will list the sources that have been uploaded to the BigML PredictServer.

https://{hostname}/source?auth_token={auth_token}

Sources list URL


curl "https://your-predict-server/source?auth_token=mytoken" \
-k -X POST \
-F "file=@predict.csv"

# Response: {"status":{"code":0,"message":"OK."},"code":200,"source_id":"e51698d7e6b188801ced850d3336adf6"}

Uploading a source


curl "https://your-predict-server/source/e51698d7e6b188801ced850d3336adf6?auth_token=mytoken" \
-k -X DELETE

# Response: /No body/

Deleting a source


curl "https://your-predict-server/source?auth_token=mytoken" \
-k -X GET

# Response: {"status":{"code":0,"message":"OK."},"code":200,"sources": ["e51698d7e6b188801ced850d3336adf6"]}

Listing uploaded sources

Predictions

A prediction is created using either a model/id or a ensemble/id and the new instance (input_data) for which you wish to create a prediction.

When you create a new prediction, the BigML PredictServer will automatically navigate the corresponding model to find the leaf node that best classifies the new instance. If you create a new prediction using an ensemble, the same process is repeated for each model in the ensemble. Then all the predictions from the individual models in the ensemble are combined to return a final prediction using one of the strategies described below.

The URL for creating predictions is similar to the BigML.io REST API, however as predictions are created in memory and not stored the only available method is POST to create a prediction. The options when creating a prediction are as follows:

**Prediction Creation Options**
Argument	Type	Description
operating_kind optional	String, default is 'probability'	The operating kind to perform the prediction. It also replaces operating_point for more information. Example: 'confidence'
operating_point optional	Object	The specification of an operating point for classification problems to perform the prediction which consists of a positive_class (one of the categories of the model's objective field), and a threshold (a number between 0 and 1), and an optional field kind (confidence, probability, and for non-boosted ensembles, votes). When it presents, BigML will predict the positive_class if its probability, confidence or votes (depending on the kind) is greater than the threshold set. Otherwise, BigML will predict the class with the higher probability, confidence or votes.confidence and probability will yield the same results for boosted ensembles, deepnets, fusions, and logistic regressions. For the votes kind, the threshold specifies the ratio of models in the ensembles predicting the positive_class. Note that operating_point takes precedence over combiner and threshold, thus they will be ignored if provided. It also takes precedence over operating_kind if kind is provided. When neither of them are presented, then the default value probability will be used for kind. Example: copy text pop-up `{` `"kind": "probability",` `"positive_class": "Iris-setosa",` `"threshold": 0.42` `}` `{ "kind": "probability", "positive_class": "Iris-setosa", "threshold": 0.42 }`
combiner optional - ensembles only	Integer, default is 0	Specifies the method that should be used to combine predictions in an ensemble. For classification ensembles, the combination is made by majority vote. The options are: 0: plurality weights each model's prediction as one vote. 1: confidence weighted uses prediction confidences as weight for the combined prediction. 2: probability weighted uses the probability of the class in the distribution of classes in the leaf node as weight. For regression ensembles, the predicted values are averaged. The options are: 0: plurality weights each model's prediction as one vote. 1: confidence weighted uses prediction confidences as weight for the combined prediction. 2: probability weighted gives the same results as plurality. Example: 1
ensemble optional	String	A valid ensemble/id. Example: ensemble/517020d53c1920a514000056
input_data required	Object	An object with field's id/value pairs representing the instance you want to create a prediction for. Example: {"000000": 5, "000001": 3}
missing_strategy optional	Integer, default is 0	Specifies the method that should be used when a missing split is found. That is, when a missing value in found in the input data for a decision node. The options are: 0: last prediction predicts based on the subset of the data which reached the parent of the missing split. 1: proportional evaluates all the subtrees of a missing split and recombines their predictions based on the proportion of data in each subtree. Example: 1
model optional	String	Avalid model/id. Example: model/4f67c0ee03ce89c74a000006

To make a model prediction with a single input_data:


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"model":"model/51df6f52035d0760380042e7", "input_data":{ "bmi": 100 }}'

# Prediction:
    {
      "created": "2015-01-02T18:39:18.404Z",
      "resource": "prediction/1420223958404-2153",
      "objective_fields": [
        "000008"
      ],
      "status": {
        "elapsed": 1,
        "code": 5,
        "message": "The prediction has been created"
      },
      "output": "false",
      "prediction": {
        "000008": "false"
      },
      "code": 200,
      "confidence": 0.61665,
      "prediction_path": {
        "objective_summary": {
          "categories": [
            [
              "false",
              500
            ],
            [
              "true",
              268
            ]
          ]
        }
      }
    }

A single model prediction

To make a model prediction with the proportional missing strategy:


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"model":"model/51df6f52035d0760380042e7", "input_data":{ "bmi": 100 }, "missing_strategy": 1}'

# Prediction:
    {
      "created": "2015-01-02T18:39:18.617Z",
      "resource": "prediction/1420223958617-2151",
      "objective_fields": [
        "000008"
      ],
      "status": {
        "elapsed": 6,
        "code": 5,
        "message": "The prediction has been created"
      },
      "output": "true",
      "prediction": {
        "000008": "true"
      },
      "code": 200,
      "confidence": 0.5086246510855001,
      "prediction_path": {
        "objective_summary": {
          "categories": [
            [
              "false",
              500
            ],
            [
              "true",
              268
            ]
          ]
        }
      }
    }

A single model prediction, missing_strategy=1

To make a single prediction from an ensemble:


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/51e7267c035d076038004e7c", "input_data":{ "bmi": 100 }}'

# Prediction:
    {
      "created": "2015-01-02T19:26:40.116Z",
      "resource": "prediction/1420226800116-2151",
      "objective_fields": [
        "000008"
      ],
      "status": {
        "elapsed": 0,
        "code": 5,
        "message": "The prediction has been created"
      },
      "output": "false",
      "prediction": {
        "000008": "false"
      },
      "code": 200,
      "confidence": 0.598194,
      "prediction_path": {}
    }

An ensemble prediction

To make a single prediction from an ensemble using a different combiner:


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/51e7267c035d076038004e7c", "input_data":{ "bmi": 100 }, "combiner": 2}'

# Prediction:
    {
      "created": "2015-01-02T19:26:40.757Z",
      "resource": "prediction/1420226800757-2151",
      "objective_fields": [
        "000008"
      ],
      "status": {
        "elapsed": 0,
        "code": 5,
        "message": "The prediction has been created"
      },
      "output": "false",
      "prediction": {
        "000008": "false"
      },
      "code": 200,
      "confidence": 0.598194,
      "prediction_path": {}
    }

An ensemble prediction, combiner=2

It is not necessary to load models/ensemble before requesting predictions. If a model/ensemble is not in memory, the prediction request will automatically trigger the caching process. However, the prediction request will return a status.code!=5 until the model/ensemble is in memory and ready to return predictions:


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/61e7267c035d076038004f4f", "input_data":{ "bmi": 100 }}'

# Response: {"status":{"code":1,"message":"Download started."},"code":200}

curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/61e7267c035d07603800f4f", "input_data":{ "bmi": 100 }}'

# Response: {"status":{"code":3,"message":"Download in-progress."},"code":200}

curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/61e7267c035d07603800f4f", "input_data":{ "bmi": 100 }}'

# Prediction:
    {
      "created": "2015-01-02T19:26:40.116Z",
      "resource": "prediction/1420226800116-2151",
      "objective_fields": [
        "000008"
      ],
      "status": {
        "elapsed": 0,
        "code": 5,
        "message": "The prediction has been created"
      },
      "output": "false",
      "prediction": {
        "000008": "false"
      },
      "code": 200,
      "confidence": 0.598194,
      "prediction_path": {}
    }

Predicting with a non pre-cached ensemble

Centroid

A centroid is created using a cluster/id and the new instance (input_data) for which you wish to create a centroid.

When you create a new centroid, the BigML PredictServer will automatically compute the distance between the new instance and every cluster and will return the closest centroid to the new instance.

The URL for creating centroids is similar to the BigML.io REST API, however as centroids are created in memory and not stored the only available method is POST to create a centroid. The options when creating a centroid are as follows:

**Centroid Creation Options**
Argument	Type	Description
cluster	String	Avalidcluster/id. Example: cluster/4f67c0ee03ce89c74a000006
input_data required	Object	An object with field's id/value pairs representing the instance you want to find the closest centroid for. You can use either field ids or field names as keys in your input_data. Example: {"000000": 5, "000001": 3}

To compute a single centroid from a cluster:



curl "https://your-predict-server/centroid?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"cluster":"cluster/51e0001c035d076038008e8a", "input_data": \
    { "pregnancies": 0, "plasma glucose": 150, "blood pressure": 190, "triceps skin thickness": 12, "insulin": 300, "bmi": 100, "diabetes pedigree": 2, "age":55 } }'

# Centroid:
    {
      "created": "2015-01-02T19:26:50.045Z",
      "resource": "centroid/1420226810045-2153",
      "distance": 2.3435298345544613,
      "centroid_name": "Cluster 1",
      "centroid_id": "000001",
      "status": {
        "elapsed": 0,
        "code": 5,
        "message": "The centroid has been created"
      },
      "code": 200
    }

A centroid assignment

It is not necessary to GET a cluster before requesting centroids. If a cluster is not in memory, the centroid request will automatically trigger the caching process. However, centroid requests will return a status.code!=5 until the cluster is in memory and ready to return centroids.

AnomalyScore

A anomalyscore is created using a anomaly/id and the new instance (input_data) for which you wish to create an anomalyscore.

When you create a new anomaly score, the BigML PredictServer will automatically compute a score between 0 and 1. The closer the score is to 1, the more anomalous the instance being scored is.

The URL for creating anomalyscores is similar to the BigML.io REST API, however as anomalyscores are created in memory and not stored the only available method is POST to create an anomalyscore. The options when creating an anomalyscore are as follows:

**AnomalyScore Creation Options**
Argument	Type	Description
anomaly	String	A valid anomaly/id. Example: anomaly/4f67c0ee03ce89c74a000006
input_data required	Object	An object with field's id/value pairs representing the instance for which you want to compute the anomaly score. You can use either field ids or field names as keys in your input_data. Example: {"000000": 5, "000001": 3}

To compute a single anomalyscore from an anomaly detector:



curl "https://your-predict-server/anomalyscore?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"anomaly":"anomaly/51e0001c035d076038008e8a", "input_data": { "bmi": 100 } }'

# AnomalyScore:
    {
      "created": "2015-01-02T19:27:15.008Z",
      "resource": "anomalyscore/1420226835008-2153",
      "score": 0.9384834568392456,
      "status": {
        "elapsed": 2,
        "code": 5,
        "message": "The anomaly score has been created"
      },
      "code": 200
    }

An anomalyscore request

It is not necessary to GET a anomaly before requesting anomalyscores. If an anomaly is not in memory, the anomalyscore request will automatically trigger the caching process. However, anomalyscore requests will return a status.code!=5 until the anomaly is in memory and ready to return anomalyscores.

Batch Predictions / Centroids / AnomalyScores

To create predictions, centroids, or anomalyscores from a list of input data, you first need to upload a CSV with the input data. The header row needs to contain the labels for the data, and each subsequent row will be an input.


curl "https://your-predict-server/source?auth_token=mytoken" \
-k -X POST \
-F "file=@input.csv"

# Response: {"status":{"code":0,"message":"OK."},"code":200,"source_id":"2151-3kj"}

Uploading a source

Once the source is uploaded, it can be used to make predictions, centroids, or anomalyscores by specifying the source_id as the input_source instead of using the input_data option. For example:


curl "https://your-predict-server/prediction?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"ensemble":"ensemble/51e7267c035d076038004e7c", "input_source":"2151-3kj"}'

# Prediction:
prediction,confidence
true,0.57550375
false,0.877254
true,0.792435
false,0.93854
true,0.7097514285714286
false,0.790236
true,0.3982244444444444
false,0.4135516666666666
true,0.7150228571428572
...

A batch prediciton


curl "https://your-predict-server/centroid?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"cluster":"cluster/51e7200c035d076038004e7c", "input_source":"2151-3kj"}'

# Centroid:
centroid_id,centroid_name,distance
000001,Cluster 1,0.48875164705143254
000000,Cluster 0,0.33519892573397514
000000,Cluster 0,0.6809618105516001
000000,Cluster 0,0.37748958758120393
000001,Cluster 1,1.131769780350274
000000,Cluster 0,0.31900170389404914
000000,Cluster 0,0.45881005206402875
000000,Cluster 0,0.8992964434296525
000001,Cluster 1,0.9055580082882769
...

A batch centroid


curl "https://your-predict-server/anomalyscore?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"anomaly":"anomaly/51e8012c035d076038004e7c", "input_source":"2151-3kj"}'

# AnomalyScore:
score
0.3946418970309878
0.3403026310700034
0.45551071864312714
0.3269672772150454
0.5749114904477409
0.3225381722519602
0.45018645045264366
0.49715131102269433
0.5495323363902689
...

A batch anomalyscore

Endpoint

Endpoints provide a more abstract way to identify resources than through its resource IDs. An endpoint is created associating an endpoint name to a BigML query string.

When you create an endpoint, you can use it as an additional way to reference a resource. Only, that resource is not always the same but is the result of a query.

The supported methods for endpoints are:

GET: for displaying currently defined endpoints.

POST: for creating a new endpoint. The options when creating an endpoint are as follows:

**Endpoint Creation Options**
Argument	Type	Description
endpoint_name	String	A valid name you will use to reference the endpoint. Example: FinRisk
endpoint_query required	String	The query that should be associated to the endpoint. Example: "name__contains=FinancialRiskModel"

DELETE: for deleting an existing endpoint.


curl "https://your-predict-server/endpoint?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f" \
-k -X POST \
-H 'content-type: application/json' \
-d '{"endpoint_name":"FinRisk", "endpoint_query": "name__contains=FinancialRiskModel"}'

# Result:
{
    "code":201,
    "endpoints": {
        "FinRisk": {
            "name":"FinRisk",
            "query":"name__contains=FinancialRiskModel"
        }
    }
}

Creating an endpoint



curl -k "https://your-predict-server/endpoint?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f"

# Result:
{
  "status": {
    "code": 0,
    "message": "OK."
  },
  "code": 200,
  "endpoints": [
    {
      "name": "FinRisk",
      "query": "name__contains=FinancialRiskModel"
    }
  ]
}

Listing existing endpoints



curl -k -X DELETE \
"https://your-predict-server/endpoint/FinRiskModel?username=wendy&api_key=92127b85415db7caa2ca985edfdbcaca766d836f"

# Result:
{
  "status": {
    "code":0,
    "message":"OK."
  },
  "code":202,
  "endpoints":{}
}

Deleting an endpoint

BigML PredictServer Configuration

There are several configuration options for the BigML PredictServer. The URL for managing configuration of the PredictServer is:

https://{hostname}/config?auth_token={auth_token}

BigML PredictServer configuration URL

The supported methods for configuration are:

GET: for displaying current settings. The will return JSON something like below. If present, the ssl_cert and ssl_key will be displayed as the md5 of the string configured. Also, the localObjects structure lists the models and ensembles for all users that are currently loaded into RAM, however this list can not be modified directly using the /config endpoint.

POST: for setting configuration parameters. The arguments for this operation are:

**Configuration Arguments**
Argument	Type	Description
allow_users optional	list	a list of BigML.io usernames that can use the BigML PredictServer Example: [ "wendy", "shara" ]
auth_token optional	string	change the auth_token Example: "mytoken"
collect_stats optional	boolean	whether or not to collect prediction performance statistics Example: true
domain optional	string default is bigml.io	the BigML API domain to use for fetching resources. This allows the BigML PredictServer to work with VPCs or country specific endpoints. Exampleau.bigml.io
ssl_cert optional	string	an x509 formatted certificate for the https endpoint. Must be updated simultaneously with the ssl_key
ssl_key optional	string	a unencrypted RSA key matching the ssl_cert. Must be updated simultaneously with the ssl_cert

DELETE: for deleting/resetting individual options or all configuration.


curl "https://your-predict-server/config?auth_token=mytoken" \
-k -X GET

# Response:
{
"allow_users": [
    "wendy",
    "shara"
],
"auth_token": "mytoken",
"collect_stats": true,
"localObjects": [
    {
        "resource": "model/51df6f52035d0760380042e7",
        "username": "wendy"
    },
    {
        "resource": "ensemble/51e7267c035d076038004e7c",
        "username": "wendy"
    }
],
"ssl_cert": "",
"ssl_key": ""
}

Fetch BigML PredictServer configuration


curl "https://your-predict-server/config?auth_token=mytoken" \
-k -X DELETE \
-H 'content-type: application/json' \
-d '{ "allow_users": 1}'

# Response: /No body/

# Fetch the config result:
curl "https://your-predict-server/config?auth_token=mytoken"
-k -X GET

# Response:
{
"allow_users": [],
"auth_token": "mytoken",
"collect_stats": true,
"localObjects": [],
"ssl_cert": "",
"ssl_key": ""
}

Deleting allowed users


curl "https://your-predict-server/config?auth_token=mytoken" -F ssl_cert=@myssl.crt -F ssl_key=@myssl.key

# Response:
{"status":{"code":0,"message":"OK."},"code":200}

Changing the SSL Certificate

Dashboard

Last Updated: Thursday, 2013-09-19 23:59

If collect_stats is set to true, the BigML PredictServer will collect statistics regarding the prediction and overall system performance. The dashboard can be accessed with a browser at the following URL:

https://{hostname}/dashboard/

Dashboard URL

Note:

The certificate which is installed by default is self-signed and will generate a browser warning, which can be safely ignored. The dashboard will request authentication: the username can be left blank and the password is the same as the auth_token.
The dashboard fetches graphs from a second URL which is also secured with a self-signed certificate. If you do not see any graphs, you will need to accept the certificate as well by clicking on the "Graphite" link at the top of the screen.

Performance Tuning

Last Updated: Thursday, 2013-09-19 23:59

The performance of the BigML PredictServer will depend on the following items:

Ensembles versus Models: Since an ensemble is comprised of multiple models, they will be slower to calculate.
HTTPS/HTTP: When making single predictions (via input_data) the use of the HTTPS endpoint incurrs a lot of CPU overhead to handle the encryption. This may reduce the performance significantly. If security is less of a concern, for example if deployed in a VPC, then you may use the HTTP endpoint to avoid this penalty.
Statistics: The collection of statistics will reduce performance as well, although not tremendously. For the absolute best performance however, you can turn off collection by setting the collect_stats config parameter to false.
Single predictions versus batch: If you have a file containing input data with which to make predictions, use the batch prediction feature. Reading the data one line at a time and requesting the predictions individually incurs a penalty because of the time it takes each request to reach the server and the reply to return. For example, it is normal for the BigML PredictServer to generate a prediction in less than 1ms, but the network latency could be as high as 200ms or worse, which means each prediction would require 201ms to complete. If you must issue predictions one at a time, be sure to take advantage of concurrency, that is using several parallel threads to send the requests. This reduces the impact of network latency by taking advantage of bandwidth.

BigML PredictServer
Documentation

Overview

Quick Start

Detailed Start

Change the auth_token

Adding BigML Users

Working with Cached Objects

Creating Predictions

Working with Bindings

Working with Endpoints

API Documentation

Listing, fetching, and deleting resources

Sources

Predictions

Centroid

AnomalyScore

Batch Predictions / Centroids / AnomalyScores

Endpoint

BigML PredictServer Configuration

Dashboard

Performance Tuning

COMPANY

PRODUCT

BUSINESS

TRAINING

GALLERY

BigML PredictServerDocumentation

Overview

Quick Start

Detailed Start

Change the auth_token

Adding BigML Users

Working with Cached Objects

Creating Predictions

Working with Bindings

Working with Endpoints

API Documentation

Listing, fetching, and deleting resources

Sources

Predictions

Centroid

AnomalyScore

Batch Predictions / Centroids / AnomalyScores

Endpoint

BigML PredictServer Configuration

Dashboard

Performance Tuning

COMPANY

PRODUCT

BUSINESS

TRAINING

GALLERY

BigML PredictServer
Documentation