Accessing Cursors via HTTP

Create cursor

create a cursor and return the first results

POST /_api/cursor

A JSON object describing the query and query parameters.

A JSON object with these properties is required:

  • query: contains the query string to be executed

  • count: indicates whether the number of documents in the result set should be returned in the “count” attribute of the result. Calculating the “count” attribute might have a performance impact for some queries in the future so this option is turned off by default, and “count” is only returned when requested.

  • batchSize: maximum number of result documents to be transferred from the server to the client in one roundtrip. If this attribute is not set, a server-controlled default value will be used. A batchSize value of 0 is disallowed.

  • ttl: The time-to-live for the cursor (in seconds). The cursor will be removed on the server automatically after the specified amount of time. This is useful to ensure garbage collection of cursors that are not fully fetched by clients. If not set, a server-defined value will be used (default: 30 seconds).

  • cache: flag to determine whether the AQL query results cache shall be used. If set to false, then any query cache lookup will be skipped for the query. If set to true, it will lead to the query cache being checked for the query if the query cache mode is either on or demand.

  • memoryLimit: the maximum number of memory (measured in bytes) that the query is allowed to use. If set, then the query will fail with error “resource limit exceeded” in case it allocates too much memory. A value of 0 indicates that there is no memory limit.

  • bindVars: key/value pairs representing the bind parameters.

  • options: key/value object with extra options for the query.

    • fullCount: if set to true and the query contains a LIMIT clause, then the result will have an extra attribute with the sub-attributes stats and fullCount, { ... , "extra": { "stats": { "fullCount": 123 } } }. The fullCount attribute will contain the number of documents in the result before the last top-level LIMIT in the query was applied. It can be used to count the number of documents that match certain filter criteria, but only return a subset of them, in one go. It is thus similar to MySQL’s SQL_CALC_FOUND_ROWS hint. Note that setting the option will disable a few LIMIT optimizations and may lead to more documents being processed, and thus make queries run longer. Note that the fullCount attribute may only be present in the result if the query has a top-level LIMIT clause and the LIMIT clause is actually used in the query.

    • maxPlans: Limits the maximum number of plans that are created by the AQL query optimizer.

    • maxWarningCount: Limits the maximum number of warnings a query will return. The number of warnings a query will return is limited to 10 by default, but that number can be increased or decreased by setting this attribute.

    • failOnWarning: When set to true, the query will throw an exception and abort instead of producing a warning. This option should be used during development to catch potential issues early. When the attribute is set to false, warnings will not be propagated to exceptions and will be returned with the query result. There is also a server configuration option --query.fail-on-warning for setting the default value for failOnWarning so it does not need to be set on a per-query level.

    • stream: Specify true and the query will be executed in a streaming fashion. The query result is not stored on the server, but calculated on the fly. Beware: long-running queries will need to hold the collection locks for as long as the query cursor exists. When set to false a query will be executed right away in its entirety. In that case query results are either returned right away (if the result set is small enough), or stored on the arangod instance and accessible via the cursor API (with respect to the ttl). It is advisable to only use this option on short-running queries or without exclusive locks (write-locks on MMFiles). Please note that the query options cache, count and fullCount will not work on streaming queries. Additionally query statistics, warnings and profiling data will only be available after the query is finished. The default value is false

    • optimizer.rules: A list of to-be-included or to-be-excluded optimizer rules can be put into this attribute, telling the optimizer to include or exclude specific rules. To disable a rule, prefix its name with a -, to enable a rule, prefix it with a +. There is also a pseudo-rule all, which will match all optimizer rules.

    • profile: If set to true or 1, then the additional query profiling information will be returned in the sub-attribute profile of the extra return attribute, if the query result is not served from the query cache. Set to 2 the query will include execution stats per query plan node in sub-attribute stats.nodes of the extra return attribute. Additionally the query plan is returned in the sub-attribute extra.plan.

    • satelliteSyncWait: This Enterprise Edition parameter allows to configure how long a DBServer will have time to bring the satellite collections involved in the query into sync. The default value is 60.0 (seconds). When the max time has been reached the query will be stopped.

    • maxTransactionSize: Transaction size limit in bytes. Honored by the RocksDB storage engine only.

    • intermediateCommitSize: Maximum total size of operations after which an intermediate commit is performed automatically. Honored by the RocksDB storage engine only.

    • intermediateCommitCount: Maximum number of operations after which an intermediate commit is performed automatically. Honored by the RocksDB storage engine only.

    • skipInaccessibleCollections: AQL queries (especially graph traversals) will treat collection to which a user has no access rights as if these collections were empty. Instead of returning a forbidden access error, your queries will execute normally. This is intended to help with certain use-cases: A graph contains several collections and different users execute AQL queries on that graph. You can now naturally limit the accessible results by changing the access rights of users on collections. This feature is only available in the Enterprise Edition.

The query details include the query string plus optional query options and bind parameters. These values need to be passed in a JSON representation in the body of the POST request.

HTTP 201 is returned if the result set can be created by the server.

  • error: A flag to indicate that an error occurred (false in this case)

  • code: the HTTP status code

  • result: an array of result documents (might be empty if query has no results)

  • hasMore: A boolean indicator whether there are more results available for the cursor on the server

  • count: the total number of result documents available (only available if the query was executed with the count attribute set)

  • id: id of temporary cursor created on the server (optional, see above)

  • extra: an optional JSON object with extra information about the query result contained in its stats sub-attribute. For data-modification queries, the extra.stats sub-attribute will contain the number of modified documents and the number of documents that could not be modified due to an error (if ignoreErrors query option is specified)

  • cached: a boolean flag indicating whether the query result was served from the query cache or not. If the query result is served from the query cache, the extra return attribute will not contain any stats sub-attribute and no profile sub-attribute.

HTTP 400 is returned if the JSON representation is malformed or the query specification is missing from the request.

If the JSON representation is malformed or the query specification is missing from the request, the server will respond with HTTP 400.

The body of the response will contain a JSON object with additional error details. The object has the following attributes:

  • error: boolean flag to indicate that an error occurred (true in this case)

  • code: the HTTP status code

  • errorNum: the server error number

  • errorMessage: a descriptive error message

If the query specification is complete, the server will process the query. If an error occurs during query processing, the server will respond with HTTP 400. Again, the body of the response will contain details about the error.

A list of query errors can be found here.

  • 404: The server will respond with HTTP 404 in case a non-existing collection is accessed in the query.

  • 405: The server will respond with HTTP 405 if an unsupported HTTP method is used.

Examples

Execute a query and extract the result in a single go

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 2 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Execute a query and extract a part of the result

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 5 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Using the query option “fullCount”

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR i IN 1..1000 FILTER i > 500 LIMIT 10 RETURN i", 
  "count" : true, 
  "options" : { 
    "fullCount" : true 
  } 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Enabling and disabling optimizer rules

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR i IN 1..10 LET a = 1 LET b = 2 FILTER a + b == 3 RETURN i", 
  "count" : true, 
  "options" : { 
    "maxPlans" : 1, 
    "optimizer" : { 
      "rules" : [ 
        "-all", 
        "+remove-unnecessary-filters" 
      ] 
    } 
  } 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Execute instrumented query and return result together with execution plan and profiling information

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "LET s = SLEEP(0.25) LET t = SLEEP(0.5) RETURN 1", 
  "count" : true, 
  "options" : { 
    "profile" : 2 
  } 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Execute a data-modification query and retrieve the number of modified documents

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products REMOVE p IN products" 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Execute a data-modification query with option ignoreErrors

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "REMOVE 'bar' IN products OPTIONS { ignoreErrors: true }" 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Bad query - Missing body

shell> curl -X POST --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor

HTTP/1.1 Bad Request
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Bad query - Unknown collection

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR u IN unknowncoll LIMIT 2 RETURN u", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 Not Found
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Bad query - Execute a data-modification query that attempts to remove a non-existing document

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "REMOVE 'foo' IN products" 
}
EOF

HTTP/1.1 Not Found
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

return the next results from an existing cursor

PUT /_api/cursor/{cursor-identifier}

Path Parameters

  • cursor-identifier (required): The name of the cursor

If the cursor is still alive, returns an object with the following attributes:

  • id: the cursor-identifier
  • result: a list of documents for the current batch
  • hasMore: false if this was the last batch
  • count: if present the total number of elements

Note that even if hasMore returns true, the next call might still return no documents. If, however, hasMore is false, then the cursor is exhausted. Once the hasMore attribute has a value of false, the client can stop.

Return codes

  • 200: The server will respond with HTTP 200 in case of success.

  • 400: If the cursor identifier is omitted, the server will respond with HTTP 404.

  • 404: If no cursor with the specified identifier can be found, the server will respond with HTTP 404.

Examples

Valid request for next batch

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 5 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

shell> curl -X PUT --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/103972

HTTP/1.1 OK
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Missing identifier

shell> curl -X PUT --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor

HTTP/1.1 Bad Request
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Unknown identifier

shell> curl -X PUT --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/123123

HTTP/1.1 Not Found
content-type: application/json; charset=utf-8
x-content-type-options: nosniff
Show response body

Delete cursor

dispose an existing cursor

DELETE /_api/cursor/{cursor-identifier}

Path Parameters

  • cursor-identifier (required): The id of the cursor

Deletes the cursor and frees the resources associated with it.

The cursor will automatically be destroyed on the server when the client has retrieved all documents from it. The client can also explicitly destroy the cursor at any earlier time using an HTTP DELETE request. The cursor id must be included as part of the URL.

Note: the server will also destroy abandoned cursors automatically after a certain server-controlled timeout to avoid resource leakage.

Return codes

  • 202: is returned if the server is aware of the cursor.

  • 404: is returned if the server is not aware of the cursor. It is also returned if a cursor is used after it has been destroyed.

Examples

shell> curl -X POST --header 'accept: application/json' --data-binary @- --dump - http://localhost:8529/_api/cursor <<EOF
{ 
  "query" : "FOR p IN products LIMIT 5 RETURN p", 
  "count" : true, 
  "batchSize" : 2 
}
EOF

HTTP/1.1 Created
content-type: application/json; charset=utf-8
x-content-type-options: nosniff

shell> curl -X DELETE --header 'accept: application/json' --dump - http://localhost:8529/_api/cursor/103889
Show response body
❮  Query Results

© ArangoDB - the native multi-model NoSQL database