Pregel HTTP API

See Distributed Iterative Graph Processing (Pregel) for details.

Start Pregel job execution

Start the execution of a Pregel algorithm

POST /_api/control_pregel

A JSON object with these properties is required:

  • algorithm: Name of the algorithm. One of:
  • "pagerank" - Page Rank
  • "sssp" - Single-Source Shortest Path
  • "connectedcomponents" - Connected Components
  • "wcc" - Weakly Connected Components
  • "scc" - Strongly Connected Components
  • "hits" - Hyperlink-Induced Topic Search
  • "effectivecloseness" - Effective Closeness
  • "linerank" - LineRank
  • "labelpropagation" - Label Propagation
  • "slpa" - Speaker-Listener Label Propagation

    • graphName: Name of a graph. Either this or the parameters vertexCollections and edgeCollections are required. Please note that there are special sharding requirements for graphs in order to be used with Pregel.

    • vertexCollections: List of vertex collection names. Please note that there are special sharding requirements for collections in order to be used with Pregel.

    • edgeCollections: List of edge collection names. Please note that there are special sharding requirements for collections in order to be used with Pregel.

    • params: General as well as algorithm-specific options.

The most important general option is “store”, which controls whether the results computed by the Pregel job are written back into the source collections or not.

To start an execution you need to specify the algorithm name and a named graph (SmartGraph in cluster). Alternatively you can specify the vertex and edge collections. Additionally you can specify custom parameters which vary for each algorithm, see Pregel - Available Algorithms.

Return codes

  • 200: HTTP 200 is returned in case the Pregel was successfully created and the reply body is a string with the id to query for the status or to cancel the execution.

  • 400: An HTTP 400 error is returned if the set of collections for the Pregel job includes a system collection, or if the collections to not conform to the sharding requirements for Pregel jobs.

  • 403: An HTTP 403 error is returned if there are not sufficient privileges to access the collections specified for the Pregel job.

  • 404: An HTTP 404 error is returned if the specified “algorithm” is not found, or the graph specified in “graphName” is not found, or at least one the collections specified in “vertexCollections” or “edgeCollections” is not found.

Examples

Run the Weakly Connected Components (WCC) algorithm against a graph and store the results in the vertices as attribute component:

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/control_pregel <<EOF
{ 
  "algorithm" : "wcc", 
  "graphName" : "connectedComponentsGraph", 
  "params" : { 
    "maxGSS" : 36, 
    "resultField" : "component" 
  } 
}
EOF

HTTP/1.1 200 OK
content-type: application/json
connection: Keep-Alive
content-length: 7
server: ArangoDB
x-content-type-options: nosniff

"71922"

Get Pregel job execution status

Get the status of a Pregel execution

GET /_api/control_pregel/{id}

Query Parameters

  • id (required): Pregel execution identifier.

Returns the current state of the execution, the current global superstep, the runtime, the global aggregator values as well as the number of sent and received messages.

HTTP 200 HTTP 200 will be returned in case the job execution id was valid and the state is returned along with the response.

  • state: State of the execution. The following values can be returned:
  • "running": Algorithm is executing normally.
  • "storing": The algorithm finished, but the results are still being written back into the collections. Occurs only if the store parameter is set to true.
  • "done": The execution is done. In version 3.7.1 and later, this means that storing is also done. In earlier versions, the results may not be written back into the collections yet. This event is announced in the server log (requires at least info log level for the pregel log topic).
  • "canceled": The execution was permanently canceled, either by the user or by an error.
  • "fatal error": The execution has failed and cannot recover.
  • "in error" (currently unused): The execution is in an error state. This can be caused by DB-Servers being not reachable or being non responsive. The execution might recover later, or switch to "canceled" if it was not able to recover successfully.
  • "recovering" (currently unused): The execution is actively recovering, will switch back to running if the recovery was successful.

    • gss: The number of global supersteps executed.

    • totalRuntime: Total runtime of the execution up to now (if the execution is still ongoing).

    • startupTime: Startup runtime of the execution. The startup time includes the data loading time and can be substantial. The startup time will be reported as 0 if the startup is still ongoing.

    • computationTime: Algorithm execution time. The computation time will be reported as 0 if the computation still ongoing.

    • storageTime: Time for storing the results if the job includes results storage. The storage time be reported as 0 if storing the results is still ongoing.

    • reports: Statistics about the Pregel execution. The value will only be populated once the algorithm has finished.

      • vertexCount: Total number of vertices processed.

      • edgeCount: Total number of edges processed.

    • 404: An HTTP 404 error is returned if no Pregel job with the specified execution number is found or the execution number is invalid.

Examples

Get the execution status of a Pregel job:

shell> curl --header 'accept: application/json' --dump - http://localhost:8529/_api/control_pregel/72069

HTTP/1.1 200 OK
content-type: application/json
connection: Keep-Alive
content-length: 270
server: ArangoDB
x-content-type-options: nosniff

Show response body

Cancel Pregel job execution

Cancel an ongoing Pregel execution

DELETE /_api/control_pregel/{id}

Query Parameters

  • id (required): Pregel execution identifier.

Cancel an execution which is still running, and discard any intermediate results. This will immediately free all memory taken up by the execution, and will make you lose all intermediary data.

You might get inconsistent results if you requested to store the results and then cancel an execution when it is already in its "storing" state (or "done" state in versions prior to 3.7.1). The data is written multi-threaded into all collection shards at once. This means there are multiple transactions simultaneously. A transaction might already be committed when you cancel the execution job. Therefore, you might see some updated documents, while other documents have no or stale results from a previous execution.

Return codes

  • 200: HTTP 200 will be returned in case the job execution id was valid.

  • 404: An HTTP 404 error is returned if no Pregel job with the specified execution number is found or the execution number is invalid.

Examples

Cancel a Pregel job to stop the execution or to free up the results if it was started with "store": false and is in the done state:

shell> curl -X DELETE --header 'accept: application/json' --dump - http://localhost:8529/_api/control_pregel/71776

HTTP/1.1 200 OK
content-type: application/json
connection: Keep-Alive
content-length: 2
server: ArangoDB
x-content-type-options: nosniff

""