The KNIME Server REST API

The KNIME® Server REST API has already been covered from a design perspective in this blog post by Thorsten. My blog post aims to give you a hands on guide to getting started with the REST API. I’ll show some tools that you can use to get started and point you in the direction of some libraries that might help. But first I’ll discuss why you might want to use the REST API.

REST is a design pattern used for building networked applications. Practically, it is increasingly used in modern IT applications to allow developers and architects to build robust, flexible applications by integrating tools from various sources. With KNIME Server, you can use REST to give access to your workflows to external applications. That might mean that you build a workflow to predict something, and then an external application can trigger that workflow to predict on the data that it is interested in.

A typical use case for a REST service would be to integrate the results from KNIME workflows into an existing, often complex IT infrastructure. More concretely, you might have a KNIME workflow that predicts whether your customers are feeling good, or bad about your company today. Useful information, but only if this can be supplied to your colleagues in a timely manner. In the example of a call centre, it would allow colleagues to follow a script that will help to find the reason for the negative sentiment, and hopefully turn an unhappy customer into a happy customer.

Tools

Thorsten used the curl tool, which has the benefit of being available under Linux, and being relatively simple and lightweight. For those of you who prefer something a bit more point-and-click, you could consider the chrome extension Postman, the Firefox extension RESTClient, or  the desktop application SoapUI. The above isn’t a comprehensive list of tools, just enough to give you a couple of options. It should be possible to adapt the instructions in the following sections to work with your favorite tools.

Getting started

Navigating the API

One of the design decisions that Thorsten discusses is that the KNIME Server has a ‘self-documenting’ API. This means that as you add workflows to KNIME Server, the description of how to use these workflows is automatically generated. The main benefit of that is that you don’t need to maintain a separate API reference. The first call that we will try allows us to get a list of the options available to us on the KNIME Server.

curl -u 'knime-rest-user:knockknock' -i http://localhost:8080/com.knime.enterprise.server/rest

Note that we specify the username: knime-rest-user and the password knockknock, the only other parameter is the URL to the REST API, which is the address to the KNIME Server WebPortal, plus /rest.

Note that the response shows us some information about the API version, and also gives us a list of links that we could follow to further explore the API. e.g.

curl -u 'knime-rest-user:knockknock' -i http://localhost:8080/com.knime.enterprise.server/rest/v4/

Following the above link reveals the next possibility:

curl -u 'knime-rest-user:knockknock' -i ​http://localhost:8080/com.knime.enterprise.server/rest/v4/repository/

There is an optional parameter ?deep which instructs the server to return the list of workflows in a recursive manner. Normally the server will only return the workflows in the specified workflow group.

Before we go on, lets do the exact same as above, but this time using Postman.

Finding our workflow

As we learnt previously, each API call gives us some information about the additional calls that we can do. If we keep going, we can find a workflow. In this example it is located at: http://localhost:8080/com.knime.enterprise.server/rest/v4/repository/Basic%20KNIME%20Examples/REST%20Demo/workflows/Predictor%20Flow

If we were to browse to the workflow via the WebPortal, it would look something like this:

and the results of the call would look something like this:

Multi-step method

Now that we’ve found a workflow, how do we learn more about it? The first call that we make allows us to get information about the workflow jobs that already exist.

http://localhost:8080/com.knime.enterprise.server/rest/v4/repository/Basic%20KNIME%20Examples/REST%20Demo/workflows/Predictor%20Flow:jobs

We can see that there is the option to create a new job. Let’s do that:

http://localhost:8080/com.knime.enterprise.server/rest/v4/repository/Basic%20KNIME%20Examples/REST%20Demo/workflows/Predictor%20Flow:jobs

Most importantly here, we see that the “state” is “IDLE”, meaning that the new job is ready to be executed, and we also see the “inputParameters” variable, that describes the JSON format required by this workflow.

One important point at this stage is to check that any workflows you intend to execute using the REST API are reset before being uploaded to KNIME Server. If you haven’t done this, you might see slightly different results to those described here. This has already been done for the workflow accompanying this blog post.

But how exactly is this “inputParameters” variable defined? To understand this (and the “outputValues”) variables, we can best look at the KNIME workflow itself. Notice that there is a JSON Input and a JSON Output node. The JSON Input node defines the “inputParameters” variable. If there are multiple JSON Input nodes, the ‘data-record’ field (which can be edited), will specify which JSON the parameters refer to. The same is true for the JSON Output node.

 

Next we can execute the workflow using our input parameters as a template. In our case we’ll change the default parameters. Once we’ve designed the input parameters to our taste, we’ll need to include them as part of an HTTP POST request. To construct the request we’ll also need the job “id” from the previous REST call.

curl –x POST  -u 'knime-rest-user:knockknock' -k -H "Content-Type: application/json" -d '
    {"data-record-1" : {
          "jsonvalue" : {
            "Col0" : "A12",
            "Col1" : 3,
            "Col2" : "A32",
            "Col3" : "A41",
            "Col4" : 1164,
            "Col5" : "A63",
            "Col6" : "A78",
            "Col7" : 3,
            "Col8" : "A96",
            "Col9" : "A102",
            "Col10" : 2,
            "Col11" : "A221",
            "Col12" : 64,
            "Col13" : "A142",
            "Col14" : "A151",
            "Col15" : 3,
            "Col16" : "A133",
            "Col17" : 11,
            "Col18" : "A191",
            "Col19" : "A202"
          }
    }}' http://localhost:8080/com.knime.enterprise.server/rest/v4/jobs/8993d16c-a7f9-4437-85d6-28f989c8f965

after executing the query, we get the results:

Notice that the results are contained in the “outputValues” parameter. This parameter is defined by the JSON Output node (or nodes) that are present in the workflow. Here we see that the prediction value is 1, and the confidence in the prediction is 0.83.

Once we are finished with a job, we can choose to delete it:

curl –i –X DELETE –u ‘knime-rest-user:knockknock’ http://localhost:8080/com.knime.enterprise.server/rest/v4/jobs/8993d16c-a7f9-4437-85d6-28f989c8f9

Single step method

In some cases, this multistep approach can be cumbersome, so starting the v4.2 of the KNIME Server REST API, you can consolidate queries into a single call. This works as follows:

curl -X POST -u ‘knime-rest-user:knockknock' -k -H "Content-Type: application/json" -d '
    {"data-record-1" : {
          "jsonvalue" : {
            "Col0" : "A12",
            "Col1" : 3,
            "Col2" : "A32",
            "Col3" : "A41",
            "Col4" : 1164,
            "Col5" : "A63",
            "Col6" : "A78",
            "Col7" : 3,
            "Col8" : "A96",
            "Col9" : "A102",
            "Col10" : 2,
            "Col11" : "A221",
            "Col12" : 64,
            "Col13" : "A142",
            "Col14" : "A151",
            "Col15" : 3,
            "Col16" : "A133",
            "Col17" : 11,
            "Col18" : "A191",
            "Col19" : "A202"
          }
}}' -i http://localhost:8080/com.knime.enterprise.server/rest/v4/repository/Basic%20KNIME%20Examples/REST%20Demo/workflows/Predictor%20Flow:job-pool

You could also adjust only the parameters that you want to change, e.g.:

curl -X POST -u knime-rest-user:knockknock' -k -H "Content-Type: application/json" -d '
        {"data-record-1" : {
          "jsonvalue" : {
            "Col0" : "A11",
            "Col1" : 76,
            "Col4" : 1173,
            "Col5" : "A64",
            "Col19" : "A201"
          }
}}' -i http://localhost:8080/com.knime.enterprise.server/rest/v4/repository/Basic%20KNIME%20Examples/REST%20Demo/workflows/Predictor%20Flow:job-pool

Summary

We learnt how to use the KNIME Server REST API to execute a KNIME workflow with parameters defined in a JSON format, and returned the results in a JSON format. The KNIME Server REST API is only available as part of KNIME Server (and KNIME Cloud Server). For more information or to arrange a trial, contact sales@knime.com.

Resources

Programming Libraries

Once you’ve followed the steps above you are probably ready to get started integrating this into an existing tool. For reference I’ve listed a couple of libraries for some commonly used programming languages.

Java library: https://cxf.apache.org/
Python library: http://requests.readthedocs.org/en/latest/
R library: https://cran.r-project.org/web/packages/jsonlite/vignettes/json-apis.html
JavaScript: https://spring.io/guides/gs/consuming-rest-jquery/

AttachmentSize
rest_demo.zip337.32 KB