Copyright © 2022 MarkLogic Corporation. All rights reserved.
MarkLogic Server
Node.js Application Developer’s
Guide
1
MarkLogic 10
June, 2019
Last Revised: 10.0-9, February, 2022
MarkLogic Server
Table of Contents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 2
Table of Contents
Node.js Application Developer’s Guide
1.0
Introduction to the Node.js Client API ..........................................................9
1.1
Getting Started ........................................................................................................9
1.2
Required Software ................................................................................................14
1.3
Security Requirements ..........................................................................................15
1.3.1 Basic Security Requirements ....................................................................15
1.3.2 Controlling Document Access ..................................................................16
1.3.3 Evaluating Requests Against a Different Database ..................................16
1.3.4 Evaluating or Invoking Server-Side Code ................................................16
1.4
Terms and Definitions ..........................................................................................17
1.5
Key Concepts and Conventions ............................................................................18
1.5.1 MarkLogic Namespace .............................................................................18
1.5.2 Parameter Passing Conventions ................................................................18
1.5.3 Document Descriptor ................................................................................19
1.5.4 Supported Result Handling Techniques ...................................................19
1.5.5 Promise Result Handling Pattern ..............................................................20
1.5.6 Stream Result Handling Pattern ................................................................21
1.5.7 Streaming Into the Database .....................................................................22
1.5.8 Performing Point-in-Time Operations ......................................................23
1.5.9 Error Handling ..........................................................................................24
1.6
Creating a Database Client ...................................................................................25
1.7
Authentication and Connection Security ..............................................................26
1.7.1 Connecting to MarkLogic with SSL .........................................................27
1.7.2 Using SAML Authentication ....................................................................27
1.7.3 Using Certificate-Based Authentication ...................................................28
1.7.3.1 Obtaining a Client Certificate ...................................................28
1.7.3.2 Configuring Your App Server ...................................................29
1.7.3.3 Examples: Database Client Configuration ................................29
1.7.4 Using Kerberos Authentication ................................................................30
1.7.4.1 Configuring MarkLogic to Use Kerberos .................................30
1.7.4.2 Configuring Your Client Host for Kerberos .............................31
1.7.4.3 Creating a Database Client That Uses Kerberos .......................31
1.8
Using the Examples in This Guide .......................................................................31
2.0 Manipulating Documents .............................................................................33
2.1
Introduction to Document Operations ..................................................................33
2.2
Loading Documents into the Database .................................................................36
2.2.1 Overview ...................................................................................................36
2.2.2
Input Document Descriptors .....................................................................37
MarkLogic Server
Table of Contents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 3
2.2.3 Calling Convention ...................................................................................38
2.2.4 Example: Loading A Single Document ....................................................39
2.2.5 Example: Loading Multiple Documents ...................................................40
2.2.6
Inserting or Updating Metadata for One Document .................................42
2.2.7 Automatically Generating Document URIs ..............................................43
2.2.8 Transforming Content During Ingestion ...................................................43
2.3
Reading Documents from the Database ................................................................44
2.3.1 Retrieving the Contents of a Document By URI ......................................45
2.3.2 Retrieving Metadata About a Document ..................................................46
2.3.3 Example: Retrieving Content and Metadata .............................................48
2.3.4 Transforming Content During Retrieval ...................................................50
2.4
Removing Content from the Database ..................................................................51
2.4.1 Removing Documents By URI .................................................................51
2.4.2 Removing Sets of Documents ...................................................................52
2.4.3 Removing All Documents ........................................................................53
2.5
Managing Collections of Objects and Documents ...............................................54
2.6
Performing a Lightweight Document Check ........................................................56
2.7
Conditional Updates Using Optimistic Locking ...................................................57
2.7.1 Understanding Optimistic Locking ...........................................................57
2.7.2 Enable Optimistic Locking .......................................................................58
2.7.3 Obtain a Version Id ...................................................................................59
2.7.4 Apply a Conditional Update .....................................................................60
2.8
Working with Binary Documents .........................................................................61
2.8.1 Type of Binary Documents .......................................................................61
2.8.2 Streaming Binary Content ........................................................................62
2.8.3 Retrieving Binary Content with Range Requests .....................................62
2.9
Working with Temporal Documents ....................................................................63
2.10 Working with Metadata ........................................................................................64
2.10.1 Metadata Categories .................................................................................64
2.10.2 Metadata Format .......................................................................................65
2.10.3 Working with Document Properties .........................................................67
2.10.4 Disabling Metadata Merging ....................................................................68
2.10.4.1 When to Consider Disabling Metadata Merging .......................68
2.10.4.2 How to Disable Metadata Merging ...........................................69
3.0
Patching Document Content or Metadata ....................................................70
3.1
Introduction to Content and Metadata Patching ...................................................70
3.2
Example: Adding a JSON Property ......................................................................72
3.3
Patch Reference ....................................................................................................73
3.3.1
insert ..........................................................................................................75
3.3.2
replace .......................................................................................................76
3.3.3
replaceInsert ..............................................................................................78
3.3.4
remove ......................................................................................................80
3.3.5
apply ..........................................................................................................81
3.3.6
library ........................................................................................................82
3.3.7 pathLanguage ............................................................................................82
MarkLogic Server
Table of Contents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 4
3.3.8
collections .................................................................................................82
3.3.9 permissions ...............................................................................................83
3.3.10 properties ..................................................................................................83
3.3.11 quality .......................................................................................................83
3.3.12 metadataValues .........................................................................................83
3.4
Defining the Context for a Patch Operation .........................................................84
3.5
How Position Affects the Insertion Point .............................................................84
3.6
Patch Examples .....................................................................................................86
3.6.1 Preparing to Run the Examples ................................................................86
3.6.2 Example: Insert .........................................................................................87
3.6.3 Example: Replace .....................................................................................90
3.6.4 Example: ReplaceInsert ............................................................................93
3.6.5 Example: Remove .....................................................................................96
3.6.6 Example: Patching Metadata ....................................................................99
3.7
Creating a Patch Without a Builder ....................................................................102
3.8
Patching XML Documents .................................................................................103
3.9
Constructing Replacement Data on MarkLogic Server ......................................104
3.9.1 Overview of Replacement Constructor Functions ..................................105
3.9.2 Using a Builtin Replacement Constructor ..............................................106
3.9.3 Passing Parameters to a Replacement Constructor .................................107
3.9.4 Using a Custom Replacement Constructor .............................................107
3.9.5 Writing a Custom Replacement Constructor ..........................................108
3.9.6
Installing or Updating a Custom Replace Library ..................................109
3.9.7 Uninstalling a Custom Replace Library ..................................................110
3.9.8 Example: Custom Replacement Constructors ........................................111
3.9.9 Additional Operations .............................................................................116
4.0
Querying Documents and Metadata ...........................................................117
4.1
Query Interface Overview ..................................................................................117
4.2
Introduction to Search Concepts .........................................................................118
4.2.1 Search Overview .....................................................................................118
4.2.2 Query Styles ............................................................................................119
4.2.3 Types of Query .......................................................................................120
4.2.4
Indexing ..................................................................................................122
4.3
Understanding the queryBuilder Interface ..........................................................122
4.4
Searching with String Queries ............................................................................125
4.4.1
Introduction to String Query ...................................................................125
4.4.2 Example: Basic String Query .................................................................126
4.4.3 Using Constraints in a String Query .......................................................128
4.4.4 Example: Using Constraints in a String Query .......................................129
4.4.5 Using a Custom Constraint Parser ..........................................................131
4.4.6 Example: Custom Constraint Parser .......................................................132
4.4.6.1
Implementing the Constraint Parser ........................................132
4.4.6.2
Installing the Constraint Parser ...............................................133
4.4.6.3 Using the Custom Constraint in a String Query ......................133
4.4.7 Additional Information ...........................................................................135
MarkLogic Server
Table of Contents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 5
4.5
Searching with Query By Example ....................................................................135
4.5.1
Introduction to QBE ................................................................................135
4.5.2 Creating a QBE with queryBuilder .........................................................136
4.5.3 Querying XML Content With QBE ........................................................138
4.5.4 Additional Information ...........................................................................139
4.6
Searching with Structured Queries .....................................................................140
4.6.1 Basic Usage .............................................................................................140
4.6.2 Example: Using Structured Query ..........................................................140
4.6.3 Builder Methods Taxonomy Reference ..................................................142
4.6.3.1 Basic Content Queries .............................................................143
4.6.3.2 Logical Composers ..................................................................145
4.6.3.3 Location Qualifiers ..................................................................145
4.6.3.4 Document Selectors .................................................................147
4.6.4 Query Parameter Helper Functions .........................................................147
4.6.5 Search Result Refiners ............................................................................149
4.7
Searching with Combined Query ........................................................................150
4.8
Searching Values Metadata Fields ......................................................................152
4.9
Querying Lexicons and Range Indexes ..............................................................152
4.9.1 Querying Values in a Lexicon or Range Index .......................................153
4.9.2 Finding Value Co-Occurrences in Lexicons ...........................................155
4.9.3 Building an Index Reference ..................................................................157
4.9.4 Refining the Results of a Values or Co-Occurrence Query ....................158
4.9.5 Analyzing Lexicons and Range Indexes with Aggregate Functions ......159
4.9.5.1 Aggregate Function Overview ................................................159
4.9.5.2 Using Builtin Aggregate Functions .........................................159
4.9.5.3 Using User-Defined Aggregate Functions ..............................160
4.10 Generating Search Facets ....................................................................................161
4.10.1 Defining a Simple Facet .........................................................................161
4.10.2 Naming a Facet .......................................................................................163
4.10.3 Including Facet Options ..........................................................................163
4.10.4 Defining Bucket Ranges .........................................................................163
4.10.5 Creating and Using Custom Constraint Facets .......................................164
4.11 Refining Query Results .......................................................................................165
4.11.1 Available Refinements ............................................................................165
4.11.2 Paginating Query Results ........................................................................166
4.11.3 Returning Metadata .................................................................................167
4.11.4 Excluding Document Descriptors or Values From Search Results ........167
4.11.5 Generating Search Snippets ....................................................................168
4.11.6 Transforming the Search Results ............................................................169
4.11.7 Extracting a Portion of Each Matching Document .................................170
4.12 Generating Search Term Completion Suggestions .............................................173
4.12.1 Understanding the Suggestion Interface .................................................173
4.12.2 Example: Generating Search Term Suggestions ....................................176
4.13
Loading the Example Data .................................................................................179
5.0
Using the Optic API for Relational Operations .........................................183
MarkLogic Server
Table of Contents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 6
5.1
Introduction to the Optic Interfaces ....................................................................183
5.2
Interface Summary ..............................................................................................184
5.3
Preparing to Run the Examples ..........................................................................184
5.4
Generating a Plan ................................................................................................185
5.5
Invoking a Plan ...................................................................................................186
5.6
Configuring Row Set Format ..............................................................................189
5.6.1 Configuration Options ............................................................................189
5.6.2 Layout Examples ....................................................................................189
5.7
Streaming Row Data ...........................................................................................193
5.7.1 Object Mode Streaming ..........................................................................193
5.7.2 Chunked Mode Streaming ......................................................................195
5.7.3 Sequence Mode Streaming .....................................................................195
5.8
Passing Parameters into a Plan ...........................................................................197
5.9
Handling Complex Column Values ....................................................................197
5.10 Generating an Execution Plan .............................................................................198
5.11
Serializing a Plan ................................................................................................199
6.0 Working With Semantic Data ....................................................................201
6.1
Overview of Common Semantics Tasks .............................................................201
6.2
Loading Triples ...................................................................................................202
6.3
Querying Semantic Triples With SPARQL ........................................................204
6.4
Example: SPARQL Query ..................................................................................205
6.5
Managing Graphs ................................................................................................206
6.5.1 Creating or Replacing a Graph ...............................................................207
6.5.2 Adding Triples to an Existing Graph ......................................................207
6.5.3 Removing a Graph ..................................................................................208
6.5.4 Retrieving the Contents, Metadata, or Permissions of a Graph ..............209
6.5.5 Testing for Graph Existence ...................................................................210
6.5.6 Retrieving a List of Graphs .....................................................................211
6.6
Using SPARQL Update to Manage Graphs and Graph Data .............................211
6.7
Applying Inferencing Rules to a SPARQL Query or Update .............................213
6.7.1 Basic Inference Ruleset Usage ...............................................................213
6.7.2 Example: SPARQL Query With Inference Ruleset ................................214
6.7.3 Example: SPARQL Update With Inference Rulesets .............................214
6.7.4 Controlling the Default Database Ruleset ..............................................214
7.0 Managing Transactions ..............................................................................216
7.1
Transaction Overview .........................................................................................216
7.2
Creating a Transaction ........................................................................................217
7.3
Associating a Transaction with an Operation .....................................................218
7.4
Committing a Transaction ..................................................................................219
7.5
Rolling Back a Transaction .................................................................................219
7.6
Example: Using Promises With a Multi-Statement Transaction ........................220
7.7
Checking Transaction Status ..............................................................................220
7.8
Managing Transactions When Using a Load Balancer ......................................220
MarkLogic Server
Table of Contents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 7
8.0
Extensions, Transformations, and Server-Side Code Execution ...............223
8.1
Ways to Extend and Customize the API .............................................................223
8.2
Working with Resource Service Extensions .......................................................224
8.2.1 What is a Resource Service Extension? ..................................................224
8.2.2 Creating a Resource Service Extension ..................................................225
8.2.3
Installing a Resource Service Extension .................................................225
8.2.4 Using a Resource Service Extension ......................................................227
8.2.5 Example: Installing and Using a Resource Service Extension ...............228
8.2.6 Retrieving the Implementation of a Resource Service Extension ..........231
8.2.7 Discovering Resource Service Extensions .............................................231
8.2.8 Deleting Resource Service Extensions ...................................................232
8.3
Working with Content Transformations .............................................................233
8.3.1 What is a Content Transformation? ........................................................233
8.3.2 Creating a Transformation ......................................................................234
8.3.3
Installing a Transformation .....................................................................234
8.3.4 Using a Transformation ..........................................................................235
8.3.5 Example: Read, Write, and Query Transforms .......................................237
8.3.5.1
Install the Transforms ..............................................................237
8.3.5.2 Use the Write Transform .........................................................238
8.3.5.3 Use the Read Transform ..........................................................240
8.3.5.4 Use the Query Transform ........................................................241
8.3.5.5 Read Transform Source Code .................................................243
8.3.5.6 Write Transform Source Code ................................................244
8.3.5.7 Query Transform Source Code ...............................................245
8.3.6 Discovering Installed Transforms ...........................................................246
8.3.7 Deleting a Transformation ......................................................................246
8.4
Error Reporting in Extensions and Transformations ..........................................247
8.4.1 Example: Reporting Errors in JavaScript ...............................................247
8.4.2 Example: Reporting Errors in XQuery ...................................................249
8.5
Evaluating Ad-Hoc Code and Server-Side Modules ..........................................250
8.5.1 Required Privileges .................................................................................250
8.5.2 Evaluating a Ad-Hoc Query ...................................................................251
8.5.3
Invoking a Module Installed on MarkLogic Server ................................253
8.5.4
Interpreting the Results of Eval or Invoke ..............................................255
8.5.5 Specifying External Variable Values ......................................................256
8.6
Managing Assets in the Modules Database ........................................................257
8.6.1 Overview of Asset Management .............................................................258
8.6.2
Installing or Updating an Asset ..............................................................260
8.6.3 Referencing an Asset from Server-Side Code ........................................260
8.6.4 Removing an Asset .................................................................................261
8.6.5 Retrieving an Asset List ..........................................................................261
8.6.6 Retrieving an Asset .................................................................................262
9.0
Administering REST API Instances ..........................................................263
9.1
What Is a REST API Instance? ...........................................................................263
MarkLogic Server
Table of Contents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 8
9.2
Creating an Instance ............................................................................................264
9.3
Configuring Instance Properties .........................................................................264
9.4
Retrieving Configuration Information ................................................................266
9.5
Removing an Instance .........................................................................................266
10.0 Creating Data Services and Developer Actions in Node.js .......................267
10.1 Node.js Annotations for Declarations .................................................................268
10.2 Using Gulp to Generate Models .........................................................................269
10.3 Generated Modules .............................................................................................270
10.4
Expected Pattern of Use ......................................................................................270
11.0 Data Movement in the Node.js API ...........................................................272
11.1
Concurrency and Large Data Sets in Node.js .....................................................272
11.1.1 Optimal Concurrency ..............................................................................272
11.1.2 Detection of Server Factors ....................................................................273
11.1.3 IO With Node.js Streams ........................................................................273
11.1.4 Data Movement Functions ......................................................................274
11.2 Node-client-api - 2.8.0 ........................................................................................274
11.2.1 Ingesting Documents using - writeAll API .............................................274
11.2.1.1 writeAll (options) ....................................................................275
11.3 Node-client-api - 2.9.0 ........................................................................................279
11.3.1 Collecting Document uris - queryAll API ..............................................280
11.3.2 Exporting Documents - readAll API ......................................................284
12.0 Technical Support ......................................................................................289
13.0 Copyright ...................................................................................................291
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 9
1.0 Introduction to the Node.js Client API
32
The Node.js Client API enables you to create Node.js applications that can read, write, and query
documents and semantic data in a MarkLogic database.
• Getting Started
• Required Software
• Security Requirements
• Terms and Definitions
• Key Concepts and Conventions
• Creating a Database Client
• Authentication and Connection Security
• Using the Examples in This Guide
The Node.js API is an open source project maintained on GitHub. To access the sources, report or
review issues, or contribute to the project, go to http://github.com/marklogic/node-client-api.
1.1 Getting Started
This section demonstrates loading documents into the database, querying the documents,
updating a portion of a document, and reading documents from the database. The basic features
demonstrated here have many more capabilities. The end of this section contains pointers to
resources for exploring the Node.js Client API in more detail.
Before you begin, make sure you have installed the software listed in “Required Software” on
page 14. You should also have the node and npm commands on your path.
Note: If you are working on Microsoft Windows, you should use a DOS command shell
rather than a Cygwin shell. Cygwin is not a supported environment for node and
npm.
The following procedure walks you through installing the Node.js Client API, loading some
simple JSON documents into the database, and then searching and modifying the documents.
1.
If you have not already done so, download, install, and start MarkLogic Server from
http://developer.marklogic.com.
2.
Create or select a project directory from which to exercise the examples in this walk
through. The rest of the instructions assume you are in this directory.
3.
Download and install the latest version of the Node.js Client API from the public npm
repository into your project directory. For example:
npm install marklogic
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 10
4.
Configure your MarkLogic connection information: Copy the following code to a file
named my-connection.js. Modify the MarkLogic Server connection information to match
your environment. You must change at least the user and password values. Select a
MarkLogic user that has at least the rest-reader and rest-writer roles or equivalent
privileges; for details, see “Security Requirements” on page 15.
module.exports = {
connInfo: {
host: 'localhost',
port: 8000,
user: 'user',
password: 'password'
}
};
The rest of the examples in this guide assume this connection configuration module exists
with the path ./my-connection.js.
5.
Load the example documents into the database: Copy the following script to a file and run
it using the node command. Several JSON documents are inserted into the database using
DatabaseClient.documents.write.
// Load documents into the database.

const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

// Document descriptors to pass to write().
const documents = [
{ uri: '/gs/aardvark.json',
content: {
name: 'aardvark',
kind: 'mammal',
desc: 'The aardvark is a medium-sized burrowing, nocturnal mammal.'
}
},
{ uri: '/gs/bluebird.json',
content: {
name: 'bluebird',
kind: 'bird',
desc: 'The bluebird is a medium-sized, mostly insectivorous bird.'
}
},
{ uri: '/gs/cobra.json',
content: {
name: 'cobra',
kind: 'mammal',
desc: 'The cobra is a venomous, hooded snake of the family Elapidae.'
}
},
];
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 11

// Load the example documents into the database
db.documents.write(documents).result(
function(response) {
console.log('Loaded the following documents:');
response.documents.forEach( function(document) {
console.log(' ' + document.uri);
});
},
function(error) {
console.log(JSON.stringify(error, null, 2));
}
);
You should see output similar to the following:
Loaded the following documents:
/gs/aardvark.json
/gs/bluebird.json
/gs/cobra.json
6.
Search the database: Copy the following script to a file and run it using the node
command. The script retrieves documents from the database that contain the JSON
property kind with the value 'mammal'.
// Search for documents about mammals, using Query By Example.
// The query returns an array of document descriptors, one per
// matching document. The descriptor includes the URI and the
// contents of each document.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);
const qb = marklogic.queryBuilder;

db.documents.query(
qb.where(qb.byExample({kind: 'mammal'}))
).result( function(documents) {
console.log('Matches for kind=mammal:')
documents.forEach( function(document) {
console.log('\nURI: ' + document.uri);
console.log('Name: ' + document.content.name);
});
}, function(error) {
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 12
console.log(JSON.stringify(error, null, 2));
});
You should see output similar to the following. Notice that cobra is incorrectly labeled as
a mammal. The next step will correct this error in the content.
Matches for kind=mammal:

URI: /gs/cobra.json
Name: cobra

URI: /gs/aardvark.json
Name: aardvark
7.
Patch a document: Recall from the previous step that cobra is incorrectly labeled as a
mammal. This step changes the kind property for /gs/cobra.json from 'mammal' to
'reptile'. Copy the following script to a file and run it using the node command.
// Use the patch feature to update just a portion of a document,
// rather than replacing the entire contents.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);
const pb = marklogic.patchBuilder;

db.documents.patch(
'/gs/cobra.json',
pb.replace('/kind', 'reptile')
).result( function(response) {
console.log('Patched ' + response.uri);
}, function(error) {
console.log(JSON.stringify(error, null, 2));
});
You should see output similar to the following:
Patched /gs/cobra.json
8.
Confirm the change by re-running the search or retrieving the document by URI. To
retrieve /gs/cobra.json by URI, copy the following script to a file and run it using the
node command.
// Read documents from the database by URI.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.read(
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 13
'/gs/cobra.json'
).result( function(documents) {
documents.forEach( function(document) {
console.log(JSON.stringify(document, null, 2) + '\n');
});
}, function(error) {
console.log(JSON.stringify(error, null, 2));
});
You should see output similar to the following:
{
"uri": "/gs/cobra.json",
"category": "content",
"format": "json",
"contentType": "application/json",
"contentLength": "106",
"content": {
"name": "cobra",
"kind": "reptile",
"desc": "The cobra is a venomous, hooded snake of the family Elapidae."
}
}
9.
Optionally, delete the example documents: Copy the following script to a file and run it
using the node command. To confirm deletion of the documents, you can re-run the script
from Step 8.
// Remove the example documents from the database.
// This example removes all the documents in the directory
// /gs/. You can also remove documents by document URI.

const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.removeAll(
{directory: '/gs/'}
).result( function(response) {
console.log(response);
});
You should see output similar to the following:
{ exists: false, directory: '/gs/' }
Document removal is an idempotent operation. Running the script again produces the
same output.
To explore the API further, see the following resources:
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 14
1.2
Required Software
To use the Node.js Client API, you must have the following software:
• MarkLogic 8 or later. Features in version 2.0.x of the Node.js Client API can only be used
with MarkLogic 9 or later.
• Node.js, version 6.3.1 or later. Node.js is available from http://nodejs.org.
• The Node.js Package Manager tool, npm. The latest version compatible with a supported
Node.js version is recommended.
•
If you plan to use Kerberos for authentication, you must have the MIT Kerberos software.
For details, see “Using Kerberos Authentication” on page 30.
The examples in this guide assume you have the node and npm commands on your path.
If You Want To
Then See
Explore more examples
The examples and tests that are distributed with the
API. Sources are available from
http://github.com/marklogic/node-client-api or in your
node_modules/marklogic directory after you install
the API.
Learn about reading and writing
documents and metadata
“Manipulating Documents” on page 33.
Learn about searching documents and
querying lexicons and indexes
“Querying Documents and Metadata” on page 117.
The Search Developer’s Guide
Learn about extension points such as
content transformations and resource
service extensions
“Extensions, Transformations, and Server-Side
Code Execution” on page 223
Explore the low level API
documentation.
The Node.js API Reference.
You can also generate a local copy of the API
reference. For details, see the project page on
GitHub: http://github.com/marklogic/node-client-api
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 15
1.3
Security Requirements
This describes the basic security model used by the Node.js Client API, and some common
situations in which you might need to change or extend it. The following topics are covered:
• Basic Security Requirements
• Controlling Document Access
• Evaluating Requests Against a Different Database
• Evaluating or Invoking Server-Side Code
1.3.1
Basic Security Requirements
The user you specify when creating a DatabaseClient object must have appropriate URI
privileges for the content accessed by the operations performed, such as permission to read or
update documents in the target database.
The Node.js Client uses the MarkLogic REST Client API to communicate with MarkLogic
Server, so it uses the same security model. In addition to proper URI privileges, the user must
have one of the pre-defined roles listed below, or the equivalent privileges. The capabilities of
each role in the table is subsumed in the roles below it.
Some operations require additional privileges, such as using a database other than the default
database associated with the REST instance and using eval or invoke methods of DatabaseClient.
These requirements are detailed below.
Role
Description
rest-extension-user Enables access to resource service extension methods. This role is
implicit in the other pre-defined REST API roles, but you may need to
explicitly include it when defining custom roles.
rest-reader
Enables read operations, such as retrieving documents and metadata.
This role does not grant any other privileges, so the user might still
require additional privileges to read content.
rest-writer
Enables write operations, such as creating documents, metadata, or
configuration information. This role does not grant any other
privileges, so the user might still require additional privileges to write
content.
rest-admin
Enables administrative operations, such as creating an instance and
managing instance configuration. This role does not grant any other
privileges, so the user might still require additional privileges.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 16
1.3.2
Controlling Document Access
Documents you create using the Node.js Client API default roles have a read permission for the
rest-reader role and an update permission for the rest-writer role. By default, users with the
rest-reader role can read all documents created as rest-reader and users with the rest-writer
role can write all documents created as rest-writer. You can override this behavior using
document permissions and/or custom roles.
To restrict access to particular users, create custom roles rather than assigning users to the default
rest-* roles. For example, you can use a custom role to restrict users in one group from seeing
documents created by another.
For details, see Controlling Access to Documents and Other Artifacts in the REST Application
Developer’s Guide.
1.3.3
Evaluating Requests Against a Different Database
When you connect to a MarkLogic Server instance by creating a DatabaseClient, the REST
instance you connect to has a default content database associated with it. You can specify an
alternative database when you create the DatabaseClient, but to perform operations against an
alternative database requires the http://marklogic.com/xdmp/privileges/xdmp-eval-in privilege
or equivalent.
To enable your application to use a different database:
1.
Create a role with the xdmp:eval-in execution privilege, in addition to appropriate mix of
rest-* roles. (You can also add the privileges to an existing role.)
2.
Assign the role from Step 1 to a user.
3.
Create a DatabaseClient with the user from Step 2.
One simple way to achieve this is to inherit from one of the predefined rest-* roles and then
addin the eval-in privileges.
For details about roles and privileges, see the Security Guide. To learn more about managing
REST API instances, see “Administering REST API Instances” on page 263.
1.3.4
Evaluating or Invoking Server-Side Code
You can use the DatabaseClient.eval and DatabaseClient.invoke operations to evaluate arbitrary
code on MarkLogic Server. These operations require special privileges instead of (or in addition
to) the normal REST API roles like rest-reader and rest-writer.
For details, see “Required Privileges” on page 250.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 17
1.4
Terms and Definitions
This guide uses the following terms and definitions:
Term
Definition
REST Client API
A MarkLogic API for developing applications that communicate with
MarkLogic using RESTful HTTP requests. The Node.js Client API is
built on top of the REST Client API.
REST API instance
A MarkLogic HTTP App Server specially configured to service
REST Client API requests. The Node.js Client API requires a REST
API instance. One is available on port 8000 as soon as you install
MarkLogic. For details, see “What Is a REST API Instance?” on
page 263.
npm
The Node.js package manager. Use npm to download and install the
Node.js Client API and its dependencies.
builder
An interface in the Node.js Client API that exposes functions for
building potentially complex data structures such as queries
(marklogic.queryBuilder) and document patches
(marklogic.patchBuilder).
Promise
A Promise is a JavaScript interface for interacting with the outcome of
an asynchronous event. For details, see “Promise Result Handling
Pattern” on page 20.
MarkLogic module
The module that encapsulates the Node.js Client API. Include the
module in your application using require(). For details, see
“MarkLogic Namespace” on page 18.
document descriptor
An object that encapsulates document content and metadata as named
JavaScript object properties. For details, see “Document Descriptor”
on page 19.
database client
A special object that encapsulates your connection to MarkLogic
Server through a REST API instance. Almost all Node.js Client API
operations take place through a database client object. For details, see
“Creating a Database Client” on page 25.
git
A source control management system. You will need a git client if you
want to checkout and use the Node.js Client API sources.
GitHub
The open source project repository that hosts the Node.js Client API
project. For details, see http://github.com/.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 18
1.5
Key Concepts and Conventions
• MarkLogic Namespace
• Parameter Passing Conventions
• Document Descriptor
• Supported Result Handling Techniques
• Promise Result Handling Pattern
• Stream Result Handling Pattern
• Streaming Into the Database
• Performing Point-in-Time Operations
• Error Handling
1.5.1
MarkLogic Namespace
The Node.js Client API library exports a namespace that provides a database client factory
method and access to builders such as queryBuilder (search), valuesBuilder (values queries), and
patchBuilder (partial document updates).
To include the MarkLogic module in your code, use require() and bind the result to a variable.
For example, you can include it by the name “marklogic” if you have installed in the module
under your own Node.js project:
const ml = require('marklogic');
You can use any variable name, but the examples in this guide assume ml.
1.5.2
Parameter Passing Conventions
Node.js Client API functions that require many input parameter values accept these values as
named properties of a call object. For example, you can specify a hostname, port, database name,
and several other connection properties when calling the createDatabaseClient() method. Do so
by encapsulating these values in a single object, such as the following:
ml.createDatabaseClient({host: 'some-host', port: 8003, ...});
Where a parameter value can have one or more values, the value of the property can be either a
single value or an array of values. Some functions support either an array or a list. For example:
db.documents.write(docDescriptor)
db.documents.write([docDescriptor1, docDescriptor2, ...])
db.documents.write(docDescriptor1, docDescriptor2, ...)
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 19
Where a function has a parameter that is frequently used without other parameters, you can pass
the parameter directly as a convenient alternative to encapsulating it in a call object. For example,
DatabaseClient.documents.remove accepts either a call object that can have several properties, or
a single URI string:
db.documents.remove('/my/doc.json')
db.documents.remove({uri: '/my/doc.json', txid: ...})
For details on a particular operation, see the Node.js API Reference.
1.5.3
Document Descriptor
A document descriptor is an object that encapsulates document content and metadata as named
JavaScript object properties. Node.js Client API document operations such as
DatabaseClient.documents.read and DatabaseClient.documents.write accept and return
document descriptors.
A document descriptor usually includes at least the database URI and properties representing
document content, document metadata, or both. For example, the following is a document
descriptor for a document with URI /doc/example.json. Since the document is a JSON document,
its contents can be expressed as a JavaScript object.
{ uri : 'example.json', content : {some : 'data'} }
Not all properties are always present. For example if you read just the contents of a document,
there will be no metadata-related properties in the resulting document descriptor. Similarly, if you
insert just content and the collections metadata property, the input descriptor will not include
permissions or quality properties.
{ uri : 'example.json',
content : {some : 'data'},
collections : ['my-collection']
}
The content property can be an object, string, Buffer, or ReadableStream.
See DocumentDescriptor in the Node.js API Reference for a complete list of property names.
1.5.4
Supported Result Handling Techniques
Most functions in the Node.js Client API support the following ways of processing results
returned by MarkLogic Server:
• Callback: Call the result function, passing in a success and/or error callback function.
Use this pattern when you don’t need to synchronize results. For example:
db.documents.read(...).result(function(response) {...})
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 20
• Promise: Call the result function and process the results through a Promise. Use Promises
to chain interactions together, such as writing documents to the database, followed by a
search. Your success callback is not invoked until all the requested data has been returned
by MarkLogic Server. For example:
db.documents.read(...).result().then(function(response) {...})...
For details, see “Promise Result Handling Pattern” on page 20.
• Object Mode Streaming: Call the stream function and process the results through a
Readable stream. Your code gets control each time a document or other discrete part is
received in full. If you’re reading a JSON document, it is converted to a JavaScript object
before invoking your callback. For example:
db.documents.read(...).stream().pipe(...)
For details, see “Stream Result Handling Pattern” on page 21.
• Chunked Mode Streaming: Call the stream function with a 'chunked' argument and
process the results through a Readable stream. Your code gets control each time a
sufficient number of bytes are accumulated, and the input to your callback is a byte
stream.
db.documents.read(...).stream('chunked').pipe(...)
For details, see “Stream Result Handling Pattern” on page 21.
When you use the classic callback or promise pattern, your code does not get control until all
results are returned by MarkLogic. This is suitable for operations that do not return a large amount
of data, such as a read operation that returns a small number of documents or a write. Streaming is
better suited to handling large files or a large number documents because it allows you to process
results incrementally.
Note: Errors in the user code of a success callback are handled in the next error callback.
Therefore, you should include a catch clause to handle such errors. For details, see
“Error Handling” on page 24.
1.5.5
Promise Result Handling Pattern
Node.js Client API functions return an object with a result() method that returns a Promise
object. A Promise is a JavaScript interface for interacting with the outcome of an asynchronous
event. A Promise has then, catch, and finally methods. For details, see http://promisesaplus.com/.
Promises can be chained together to synchronize multiple operations.
The success callback you pass to the Promise then method is not invoked until your interaction
with MarkLogic completes and all results are received. The Promise pattern is well suited to
synchronizing operations.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 21
For example, you can use a sequence such as the following to insert documents into the database,
query them after the insertion completes, and then work with the query results.
db.documents.write(...).result().then(
function(response) {
// search the documents after insertion
return db.documents.query(...).result();
}).then( function(documents) {
// work with the documents matched by the query
});
For a more complete example, see “Example: Using Promises With a Multi-Statement
Transaction” on page 220.
Note: You should include a catch clause in your promise chain to handle errors raised in
user code in your success callbacks. For details, see “Error Handling” on page 24.
The Node.js Client API also supports a stream pattern for processing results. A stream is better
suited to handling very large amounts of data than a Promise. For details, see “Stream Result
Handling Pattern” on page 21.
1.5.6
Stream Result Handling Pattern
Node.js Client API functions return an object with a stream method that returns a Readable stream
on the results from MarkLogic. Streams enable you to process results incrementally. Consider
using streaming if you’re reading a large number of documents or if your documents are large.
Streams can provide better throughput at lower memory overhead than the Promises when you’re
working with large amounts of data because result data can be processed as it is received from
MarkLogic Server.
Two stream modes are supported:
• Object Mode: Your code gets control each time a complete document or other discrete
part is received. A Document Descriptor is the unit of interaction. For a JSON document, the
content in the descriptor is converted into JavaScript object for ease of use. Object mode is
the default streaming mode.
• Chunked mode: Your code gets control each time a certain number of bytes is received.
An opaque byte stream is the unit of interaction. Enable chunked mode by passing the
value 'chunked' to the stream method.
Object mode is best when you need to handle each portion of the result as a document or object.
For example, if you persist a collection of domain objects in the database as JSON documents and
then want to restore them as JavaScript objects in your application. Chunked mode is best for
handling large amounts of data opaquely, such as reading a large binary file from the database and
saving it out to file.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 22
The following code snippet uses a stream in object mode to process multiple documents as they
are fetched from the database. Each time a complete document is received, the stream on('data')
callback is invoked with a document descriptor. When all documents are received, the on('end')
callback is called.
db.documents.read(uri1, uri2, uriN).stream()
.on('data', function(document) {
// process one document
}).on('end', function() {
//wrap it up
}).on('error', function(error) {
// handle errors
});
The following code snippet uses a stream in chunked mode to stream a large binary file from the
database into a file using pipe.
const fs = require('fs');
const ostrm = fs.createWriteStream(outFilePath);

db.document.read(largeFileUri).stream('chunked').pipe(ostrm);
The Promise pattern is usually more convenient if you are not processing a large amount of data.
For details, see “Promise Result Handling Pattern” on page 20.
1.5.7
Streaming Into the Database
Most Node.js methods that deal with potentially large input datasets support using a
ReadableStream to pass in the data. For example , the content property of a document descriptor
passed to DatabaseClient.documents.write can be an object, a string, a Buffer, or a Readable
stream. If you’re simply streaming data from a source such as a file, this interface is all you need.
For example, the following call uses a Readable stream to stream an image from a file into the
database:
db.documents.write({
uri: '/my/image.png',
contentType: 'image/png',
content: fs.createReadStream(pathToImage)
})
If you are assembling the stream on the fly, or otherwise need to have fine grained control, you
can use the createWriteStream method of the documents and graphs interfaces. For example, if
you use DatabaseClient.documents.createWriteStream instead of
DatabaseClient.documents.write, you can control the calls to write so you can assemble the
documents yourself, as shown below:
const ws = db.documents.createWriteStream({
uri: '/my/data.json',
contentType: 'application/json',
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 23
});
// Resulting doc contains {"key":"value"}
ws.write('"{key"', 'utf8');
ws.write(': "value"}', 'utf8');
ws.end();
You can use the writeable stream interface to load documents and semantic graphs. For details,
see documents.createWriteStream and graphs.createWriteStream in the Node.js API Reference.
1.5.8
Performing Point-in-Time Operations
If you need to perform read-only operations spanning multiple requests that must all return results
based on a consistent snapshot of the database, you can use the “point-in-time query” feature of
the Node.js Client API. In this context, “query” means a read-only operation, such as a search or
document read.
Most read-only operations accept an optional Timestamp object, created by calling
DatabaseClient.createTimestamp. If no explicit timestamp value is set on the object, then the
timestamp is set during execution of the read-only operation.
Alternatively, you can supply an explicit timestamp when creating a timestamp. This must be a
timestamp generated by MarkLogic, not an arbitrary value you create. To learn more about
point-in-time queries (reads) and timestamps, see Point-In-Time Queries in the Application
Developer’s Guide.
When you pass a Timestamp object whose timestamp is set to subsequent supporting operations,
these operations see the same snapshot of the database.
For example, suppose you are incrementally fetching search results in a context in which the
database is changing and consistency of results is important. If you pass a Timestamp object on the
search, then the effective query timestamp is captured in the Timestamp object and you can pass
the object in to subsequent searches.
const marklogic = require('marklogic');
const my = require('./my-connection.js');

const db = marklogic.createDatabaseClient(my.connInfo);
const qb = marklogic.queryBuilder;
let timestamp = db.createTimestamp();

// First search sets the timestamp value
db.documents.query(
qb.where(qb.parsedFrom('cat AND dog')).slice(0,5),
timestamp
).result().then( function(results) {
console.log(JSON.stringify(results, null, 2));
});

// ...perform subsequent searches, re-using the same timestamp object
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 24
Another example use case is reading a large number of documents from the database by URI (or
search query) in batches. If you need a consistent snapshot of the documents, use the
point-in-time feature.
You can use this feature across different kinds of operations. For example you might get the initial
timestamp from a search, and then use it to perform a SPARQL query at the same point-in-time.
This capability is supported on any operation that accepts a timestamp parameter, including the
following:
• Document read: DatabaseClient.documents.read
• Document search: DatabaseClient.documents.query
• Values Query: DatabaseClient.values.read
• Semantic Search: DatabaseClient.graphs.sparql
• Semantic Update: DatabaseClient.graphs.sparqlUpdate
• Semantic Graph Access: DatabaseClient.graphs.read and DatabaseClient.graphs.list
• Rows Query: DatabaseClient.rows.query
For more details, see the Node.js Client API Reference.
1.5.9
Error Handling
When using the callback or promise pattern, errors in your success callback are handled in the
next error callback. If you want to trap such errors, you should include a catch clause at the end of
your promise chain (or after your result handler, in the case of the callback pattern). Simply
wrapping a try-catch block around your call(s) will not trap such errors.
For example, in the case of the classic callback pattern, if you made a call to
DatabaseClient.documents.write, you should end with a catch similar to the following. The
onError function executes if the onSuccess callback throws an exception.
db.documents.write(...)
.result(function onSuccess(response) {...})
.catch(function onError(err) {...});
Similarly if you’re chaining requests together using thePromise pattern, then you should terminate
the chain with a similar handler:
db.documents.write(...).result()
.then(function onSuccess1(response) {...})
.then(function onSuccess2(response) {...})
.catch(function onError(err) {...});
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 25
1.6
Creating a Database Client
All the interactions of your application with MarkLogic Server are through a
marklogic.DatabaseClient object. Each database client manages a connection by one user to a
REST API instance and a particular database. Your application can create multiple database
clients for connecting to different REST API instances, connecting to different databases, or
connecting as different users.
Note: If you use multi-statement transactions and multiple databases, note that the
database context in which you perform an operation as part of a multi-statement
transaction must be the same as the database context in which the transaction was
created. The same restriction applies to committing or rolling back a
multi-statement transaction.
To create a database client, call marklogic.createDatabaseClient with a parameter that describes
the connection details. For example, the following code creates a database client attached to the
REST API instance listening on the default host and port (localhost:8000), using the default
database associated with the instance, and digest authentication. The connection authenticates as
user “me” with password “mypwd”.
const ml = require('marklogic');
const db = ml.createDatabaseClient({user:'me', password:'mypwd'});
The connection details must include a username and password if you are not using certificate
based authentication or Kerberos. You can include additional properties. The following table lists
key properties you can include in the connection object passed to createDatabaseClient.
Property
Name
DefaultValue
Description
host
localhost
A MarkLogic Server host with a configured
REST API instance.
port
8000
The port on which the REST API instance
listens.
database
the default database associated
with the REST instance
The database against which document operations
and queries are performed. Specifying a
database other than the REST API instance
default requires the xdmp-eval-in privilege. For
details, see “Evaluating Requests Against a
Different Database” on page 16.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 26
For details, see marklogic.createDatabaseClient in the Node.js API Reference and “Administering
REST API Instances” on page 263.
1.7
Authentication and Connection Security
This section provides an overview of how to configure authentication and SSL when creating a
database client and establishing a connection to MarkLogic. This section covers the following
topics:
• Connecting to MarkLogic with SSL
• Using SAML Authentication
• Using Certificate-Based Authentication
• Using Kerberos Authentication
authType digest
The authentication method to use in establishing
the connection. Allowed values: basic, digest,
digestbasic, application-level, or
kerberos-ticket. This must match the
authentication method configured on the REST
API instance. For details, see the Security Guide.
ssl
false
Whether or not to establish an SSL connection.
For details, see Configuring SSL on App Servers in
the Security Guide. When set to true, you can
include additional SSL properties on the
connection object. These are passed through to
the agent. For a list of these properties, see
http://nodejs.org/api/https.html#https_https_request_
options_callback
agent
max of 10 free sockets; total of
50 sockets kept alive for 60
seconds
A connection pooling agent.
Property
Name
DefaultValue
Description
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 27
1.7.1
Connecting to MarkLogic with SSL
You can combine an ssl property with any of the authentication methods so that your database
client object uses a secure connection to MarkLogic. For example:
ml.createDatabaseClient({
user: 'me',
password: 'mypassword',
authType: 'digest',
ssl: true
})
Your App Server must be SSL-enabled. For details, see Configuring SSL on App Servers in the
Security Guide.
The Node.js Client API must be able to verify that MarkLogic is sending a legitimate certificate
when first establishing the connection. If the certificate is signed by an authority other than one of
the established authorities like VeriSign, then you must include a certificate from the certification
authority in your database client configuration. Use the ca property to specify a certificate
authority. For example:
ml.createDatabaseClient({
authType: 'certificate',
ssl: true
ca: fs.readFileSync('ca.crt')
})
For more details, see Procedures for Obtaining a Signed Certificate in the Security Guide.
1.7.2
Using SAML Authentication
Your client application is responsible for acquiring a SAML assertions token from the SAML
Identity Provider (IDP). You can then use the SAML assertions received from a SAML IDP as
well as HTTP access to a MarkLogic cluster configured to verify SAML assertions from the IDP.
Your client application sends the SAML assertions to the MarkLogic enode to invoke MarkLogic
operations that are authorized for the user until the SAML assertions expire.
The MarkLogic.createDatabaseClient function uses the property values shown in the table below
to authenticate using SAML.
Property Name
Purpose
authType
Specify the SAML as the authentication type.
token
The SAML assertions token to make requests to MarkLogic. This is
required if the authType is SAML.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 28
For example, after obtaining an authorization token (base64 encoded) from an IDP, the
MarkLogic.createDatabaseClient function to create a client might look like the following.
const db = marklogic.createDatabaseClient({
host: appserverHost,
port: appserverPort,
authType: 'SAML',
token: authorizationToken,
... other configuration such as SSL ...
});
In addition, the database client object uses the setAuthToken function that takes a SAML
assertions token. Requests made after the token is set use the new SAML assertions token.
Note: Unlike the Java API, the Node.js API doesn't support a reauthorizer or renewer
callback. In Node.js, calls run to completion instead of blocking. Consequently,
your client application can change the SAML assertions token without affecting
requests that are about to be sent to the server.
1.7.3
Using Certificate-Based Authentication
When using certificate-based authentication, your client application obtains a certificate signed by
a certificate authority, along with the certificate’s private key. The certificate contains a public key
and other information required to establish a connection.
See the following topics for details:
• Obtaining a Client Certificate
• Configuring Your App Server
• Examples: Database Client Configuration
Note: You can only use certificate-based authentication with the Node.js Client API
when you connect to MarkLogic using SSL. For details, see “Connecting to
MarkLogic with SSL” on page 27.
1.7.3.1 Obtaining a Client Certificate
You can use either a client certificate signed by an established certificate authority or a
self-signed certificate. Choose one of the following options:
• Obtain a client certificate from an established certificate authority such as Verisign.
• Create a self-signed certificate.
To obtain a client certificate signed by an established certificate authority, create a certificate
signing request (CSR) using OpenSSL software or a similar tool, then send the CSR to the
certificate authority. For details, see http://openssl.org and man page for the openssl req command.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 29
To create a self-signed certificate, install your own certificate authority in MarkLogic, and then
use that certificate authority to self-sign your client certificate. For details, see Creating a Certificate
Authority in the Security Guide.
To obtain a client certificate and the associated key by self-signing, use the
xdmp.x509CertificateGenerate Server-Side JavaScript function or the
xdmp:x509-certificate-generate XQuery function. Set the private-key parameter to null, and
set the credentialId option to correspond to your certificate authority. For example:
const x509Config = ...;
const cert = xdmp.x509CertificateGenerate(
x509Config, null, {credentialId: xdmp.credentialId('ca-cred')});
1.7.3.2 Configuring Your App Server
Your App Server must also be configured for certificate-based authentication and SSL. For more
details, see Configuring an App Server for External Authentication and Procedures for Enabling SSL on
App Servers in the Security Guide. When configuring the App Server for SSL, include the
following steps; for more details, see Enabling SSL for an App Server in the Security Guide.
1.
Set “ssl require client certificate” to true.
2.
Click Show under “SSL Client Certificate Authorities”, and then select the certificate
authorities that can be used to sign client certificates for the server
1.7.3.3 Examples: Database Client Configuration
For example, if you have a certificate in a file named “client.crt” and a private key in a file named
“clientpriv.pem”, you can use them in your database client configuration as follows,:
ml.createDatabaseClient({
authType: 'certificate',
cert: fs.readFileSync('client.crt'),
key: fs.readFileSync('clientpriv.pem'),
ssl: true
})
For enhanced security, the client certificate and private key can also be combined into a single
PKCS12 file that can be protected with a passphrase. For details, see http://openssl.org and the man
page for the “openssl pkcs12” command.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 30
For example, if you have a PKCS12 file named “credentials.pfx”, then you can use the file and
your passphrase in your database client configuration as follows:
ml.createDatabaseClient({
authType: 'certificate',
pfx: fs.readFileSync('credentials.pfx'),
key: 'yourPassphrase',
ssl: true
})
You can also use a certificate with basic or digest authentication to enhance the security of these
methods. For example, the following code uses a certificate with digest authentication:
ml.createDatabaseClient({
user: 'me',
password: 'mypassword',
authType: 'digest',
cert: fs.readFileSync('client.crt'),
key: fs.readFileSync('clientpriv.pem'),
ssl: true
})
1.7.4
Using Kerberos Authentication
Use the following steps to configure your MarkLogic installation and client application
environment for Kerberos authentication:
• Configuring MarkLogic to Use Kerberos
• Configuring Your Client Host for Kerberos
• Creating a Database Client That Uses Kerberos
1.7.4.1 Configuring MarkLogic to Use Kerberos
Before you can use Kerberos authentication, you must configure MarkLogic to use external
security. If your installation is not already configured for Kerberos, you must perform at least the
following steps:
1.
Create a Kerberos external security configuration object. For details, see Creating an
External Authentication Configuration Object in the Security Guide.
2.
Create a Kerberos keytab file and install it in your MarkLogic installation. For details, see
Creating a Kerberos keytab File in the Security Guide.
3.
Create one or more users associated with an external name. For details, see Assigning an
External Name to a User in the Security Guide.
4.
Configure your App Server to use “kerberos-ticket” authentication. For details, see
Configuring an App Server for External Authentication in the Security Guide.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 31
For more details, see External Security in the Security Guide.
1.7.4.2 Configuring Your Client Host for Kerberos
On the client, the Node.js Client API must be able to access a Ticket-Granting Ticket (TGT) from
the Kerberos Key Distribution Center. The API uses the TGT to obtain a Kerberos service ticket.
Follow these steps to make a TGT available to the client application:
1.
Install MIT Kerberos in your client environment if it is not already installed. You can
download this software from http://www.kerberos.org/software/index.html.
2.
If this is a new installation of MIT Kerberos, configure your installation by editing the
krb5.conf file. On Linux, this file is located in /etc/ by default. For details, see
https://web.mit.edu/kerberos/krb5-1.15/doc/admin/conf_files/krb5_conf.html.
3.
Use kinit on your client host to create and cache a TGT with the Kerberos Key
Distribution Center. The principal supplied to kinit must be one you associated with a
MarkLogic user when performing the steps in Configuring MarkLogic to Use Kerberos.
For more details, see the following topics:
• https://web.mit.edu/kerberos/krb5-1.15/doc/user/user_commands/kinit.html
• http://web.mit.edu/kerberos/krb5-current/doc/user/tkt_mgmt.html#obtaining-tickets-with-kinit
1.7.4.3 Creating a Database Client That Uses Kerberos
In your client application, set the authType property to 'kerberos' when creating a database client.
For example, assuming you’re connecting to localhost on port 8000 and therefore don’t need to
explicitly specify host and port, then the following call creates a database client object that
connects to localhost:8000 using kerberos authentication:
ml.createDatabaseClient({authType: 'kerberos'});
1.8
Using the Examples in This Guide
All requests to MarkLogic Server using the Node.js Client API go through a DatabaseClient
object. Therefore, all the examples begin by creating such an object. Creating a DatabaseClient
requires you to specify MarkLogic Server connection information such as host, port, user, and
password.
Most of the examples in this guide abstract away the connection details by require’ing a module
named my-connection.js that exports a connection object suitable for use with
marklogic.createDatabaseClient. This encapsulation is only done for convenience. You are not
required to do likewise in your application.
MarkLogic Server
Introduction to the Node.js Client API
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 32
For example, the following statements appear near the top of each example in this guide:
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);
To use the examples you should first create a file named my-connection.js with the following
contents. This file should be co-located with any scripts you create by copying the examples in
this guide.
module.exports = {
connInfo: {
host: 'localhost',
port: 8000,
user: your-ml-username,
password: your-ml-user-password
}
};
Modify the connection details to match your environment. You must modify at least the user and
password properties. Most examples require a user with the rest-reader and/or rest-writer role
or equivalent, but some operations require additional privileges. For details, see “Security
Requirements” on page 15.
If you do not create my-connection.js, modify the calls to marklogic.createDatabaseClient in
the examples to provide connection details in another way.
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 33
2.0 Manipulating Documents
69
This chapter discusses the following topics related to using the Node.js Client API to create, read,
update and delete documents and metadata:
•
Introduction to Document Operations
• Loading Documents into the Database
• Reading Documents from the Database
• Removing Content from the Database
• Managing Collections of Objects and Documents
• Performing a Lightweight Document Check
• Conditional Updates Using Optimistic Locking
• Working with Binary Documents
• Working with Temporal Documents
• Working with Metadata
2.1
Introduction to Document Operations
The Node.js Client API exposes functions for creating, reading, updating and deleting documents
and document metadata.
Most document manipulation functions are provided through the DatabaseClient.documents
interface. For example, the following code snippet reads a document by creating a database client
object and invoking its documents.read() method:
const ml = require('marklogic');
const db = ml.createDatabaseClient({'user':'me','password':'mypwd'});
db.documents.read('/doc/example.json'). ...;
The DatabaseClient interface includes read and write operations for binding JavaScript objects to
Database documents, such as DatabaseClient.read and DatabaseClient.createCollection.
Generally, these operations provide a simpler but less powerful capability than the equivalent
method of DatabaseClient.documents. For example, you cannot specify a transaction id or read
document metadata using DatabaseClient.read.
Several of the DatabaseClient.documents interfaces accept or return document descriptors that
encapsulate data such as the URI and document content. For details, see “Input Document
Descriptors” on page 37 and “Document Descriptor” on page 19.
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 34
When loading data into the database, the DatabaseClient.documents.write method provides the
most control and richest feature set. However, if you do not need that level control, one of the
other interfaces may be simpler to use. For example, if you just want to save JavaScript domain
objects in the database, DatabaseClient.createCollection enables you to do so without creating
document descriptors or constructing document URIs.
By default, each Node.js Client API call that interacts with the database represents a complete
transactional operation. For example, if you use a single call to DatabaseClient.Documents.write
to update multiple documents, then all the updates are applied as part of the same transaction, and
the transaction is committed when the operation completes on the server. You can use
multi-statement transactions to have multiple client-side operations span a single transaction. For
details, see “Managing Transactions” on page 216.
The following table lists some common tasks related to writing to the databases, along with the
method best suited for the completing the task. For a complete list of interfaces, see the Node.js
API Reference.
If you want to
Then use
Save a collection of JavaScript objects in the
database as JSON documents.
DatabaseClient.createCollection
For details, see “Managing Collections of
Objects and Documents” on page 54.
Update a collection of JavaScript objects
created using
DatabaseClient.createCollection.
DatabaseClient.writeCollection
For details, see “Managing Collections of
Objects and Documents” on page 54.
Insert or update a collection of documents by
URI.
DatabaseClient.writeCollection
For details, see “Managing Collections of
Objects and Documents” on page 54.
Insert or update document metadata, with or
without accompanying content.
DatabaseClient.documents.write
For details, see “Inserting or Updating
Metadata for One Document” on page 42.
Insert or update documents and/or metadata in
the context of a multi-statement transaction.
DatabaseClient.documents.write
For details, see “Loading Documents into the
Database” on page 36.
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 35
The following table lists some common tasks related to reading data from the database, along with
the functions best suited for each task. For a complete list of interfaces, see the Node.js API
Reference.
Apply a content transformation while loading
documents.
DatabaseClient.documents.write
For details, see “Loading Documents into the
Database” on page 36 and “Transforming
Content During Ingestion” on page 43.
Update a portion of a document or its
metadata, rather than replacing the entire
document.
DatabaseClient.documents.patch
For details, see “Patching Document Content
or Metadata” on page 70.
If you want to
Then use
Read the contents of one or more documents
by URI.
DatabaseClient.read
Restore a collection of JavaScript objects
previously saved in the in database using
DatabaseClient.createCollection.
DatabaseClient.documents.query
For details, see “Querying Documents and
Metadata” on page 117 and “Managing
Collections of Objects and Documents” on
page 54.
Read one or more documents and/or metadata
by URI.
DatabaseClient.documents.read
For details, see “Reading Documents from the
Database” on page 44.
Read the contents of one or more documents
and/or metadata by URI and apply a read
transformation.
DatabaseClient.documents.read
For details, see “Reading Documents from the
Database” on page 44 and “Transforming
Content During Retrieval” on page 50.
Read one or more documents and/or metadata
by URI in the context of a multi-statement
transaction.
DatabaseClient.documents.read
For details, see “Reading Documents from the
Database” on page 44.
If you want to
Then use
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 36
2.2
Loading Documents into the Database
Use the DatabaseClient.documents.write or DatabaseClient.documents.createWriteStream
methods to insert document content and metadata into the database. The stream interface is
primarily intended for writing large documents such as binaries.
• Overview
•
Input Document Descriptors
• Calling Convention
• Example: Loading A Single Document
• Example: Loading Multiple Documents
•
Inserting or Updating Metadata for One Document
• Automatically Generating Document URIs
• Transforming Content During Ingestion
2.2.1
Overview
Use DatabaseClient.documents.write to insert or update whole documents and/or metadata. To
update only a portion of a document or its metadata, use DatabaseClient.documents.patch; for
details, see “Patching Document Content or Metadata” on page 70.
The primary input to the write function is one or more document descriptors. Each descriptor
encapsulates a document URI with the content and/or metadata to be written. For details, see
“Input Document Descriptors” on page 37.
For example, the following call writes a single document with the URI /doc/example.json:
const db = marklogic.createDatabaseClient(...);
db.documents.write(
{ uri: '/doc/example.json',
Read documents and/or metadata that match a
query.
DatabaseClient.documents.query
For details, see “Querying Documents and
Metadata” on page 117.
Query and analyze values in lexicons and
range indexes. For details, see “Querying
Lexicons and Range Indexes” on page 152.
DatabaseClient.values.read
Read a semantic graph from the database. For
details, see Node.js API Reference.
DatabaseClient.graphs.read
If you want to
Then use
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 37
contentType: 'application/json',
content: { some: 'data' }
}
);
Write multiple documents by passing in multiple descriptors. For example, the following call
writes 2 documents:
db.documents.write(
{ uri: '/doc/example1.json',
contentType: 'application/json',
content: { data: 'one' }
},
{ uri: '/doc/example2.json',
contentType: 'application/json',
content: { data: 'two' }
},
);
Descriptors can be passed as individual parameters, in an array, or in an encapsulating call object.
For details, see “Calling Convention” on page 38.
You can take action based on the success or failure of a write operation by calling the result()
function on the return value. For details, see “Supported Result Handling Techniques” on page 19.
For example, the following code snippet prints an error message to the console if the write fails:
db.documents.write(
{ uri: '/doc/example.json',
contentType: 'application/json',
content: { some: 'data' }
}
).result(null, function(error) {
console.log(
})
2.2.2
Input Document Descriptors
Each document to be written is described by a document descriptor. The document descriptor
must include a URI and either content, metadata, or both content and metadata. For details, see
“Document Descriptor” on page 19.
For example, the following is a document descriptor for a document with URI
/doc/example.json. The document contents are expressed as a JavaScript object containing a
single property.
{ uri: '/doc/example.json', content: {'key': 'value'} }
The content property in a document descriptor can be an object, a string, a Buffer, or a
ReadableStream.
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 38
Metadata is expressed as properties of a document descriptor when it applies to a specific
document. Metadata is expressed as properties of a call object when it applies to multiple
documents; for details, see “Inserting or Updating Metadata for One Document” on page 42.
For example, the following document descriptor includes collection and document quality
metadata for the document with URI /doc/example.json:
{ uri: '/doc/example.json’,
content: {'key': 'value'},
collections: [ 'collection1', 'collection2' ],
quality: 2
}
2.2.3
Calling Convention
You must pass at least one document descriptor to DatabaseClient.documents.write. You can
also include additional properties such as a transform name or a transaction id. The parameters
passed to documents.write can take one of the following forms:
• One or more document descriptors: db.documents.write(desc1, desc2,...).
• An array of one or more document descriptors: db.documents.write([desc1, desc2,
...]).
• A call object that encapsulates a document descriptor array and additional optional
properties: db.documents.write({documents: [desc1, desc2, ...], txid: ..., ...}).
The following calls are equivalent:
// passing document descriptors as parameters
db.documents.write(
{uri: '/doc/example1.json', content: {...}},
{uri: '/doc/example2.json', content: {...}}
);
// passing document descriptors in an array
db.documents.write([
{uri: '/doc/example1.json', content: {...}},
{uri: '/doc/example1.json', content: {...}}
]);
// passing document descriptors in a call object
db.documents.write({
documents: [
{uri: '/doc/example1.json', content: {...}},
{uri: '/doc/example2.json', content: {...}}
],
additional optional properties
});
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 39
The additional optional properties can include a transform specification, transaction id, or
temporal collection name; for details, see the Node.js API Reference. You can always specify such
properties as properties of a call object.
For example, the following call includes a transaction id (txid) as an additional property of the
call object:
// passing a transaction id as a call object property
db.documents.write({
documents: [
{uri: '/doc/example1.json', content: {...}},
{uri: '/doc/example2.json', content: {...}}
],
txid: '1234567890'
});
For convenience, if and only if there is a single document descriptor, the additional optional
properties can be passed as properties of the document descriptors, as an alternative to using a call
object. For example, the following call includes a transaction id inside the single document
descriptor:
// passing a transaction id as a document descriptor property
db.documents.write(
{ uri: '/doc/example1.json',
content: {...},
txid: '1234567890'
}
);
2.2.4
Example: Loading A Single Document
This example inserts a single document into the database using DatabaseClient.documents.write.
The document to load is identified by a document descriptor. The following document descriptor
describes a JSON document with the URI /doc/example.json. The document content is expressed
as a JavaScript object here, but it can also be a string, Buffer, or ReadableStream.
{ uri: '/doc/example.json',
contentType: 'application/json',
content: { some: 'data' }
})
The code below creates a database client and calls DatabaseClient.documents.write to load the
document. The example checks for a write failure by calling the result function and passing in an
error handler. In this example, no action is taken if the write succeeds, so null is passed as the first
parameter to result().
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 40

db.documents.write(
{ uri: '/doc/example.json',
contentType: 'application/json',
content: { some: 'data' }
})
.result(null, function(error) {
console.log(JSON.stringify(error));
});
For additional examples, see examples/before-load.js and examples/write-remove.js in the
node-client-api source directory.
To include metadata, add metadata properties to the document descriptor. For example, to add the
document to a collection, you can add a collections property to the descriptor:
db.documents.write(
{ uri: '/doc/example.json',
contentType: 'application/json',
content: { some: 'data' },
collections: [ 'collection1', 'collection2' ]
})
You can include optional additional parameters such as a transaction id or a write transform by
using a call object. For details, see “Calling Convention” on page 38.
2.2.5
Example: Loading Multiple Documents
This example builds on “Example: Loading A Single Document” on page 39 to insert multiple
documents into the database using DatabaseClient.documents.write.
To insert or update multiple documents in a single request to MarkLogic Server, pass multiple
document descriptors to DatabaseClient.documents.write.
The following code inserts 2 documents into the database with URIs /doc/example1.json and
/doc/example2.json:
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.write(
{ uri: '/doc/example1.json',
contentType: 'application/json',
content: { data: 'one' }
},
{ uri: '/doc/example2.json',
contentType: 'application/json',
content: { data: 'two' }
}
).result(null, function(error) {
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 41
console.log(JSON.stringify(error));
});
A multi-document write returns an object that contains a descriptor for each document written.
The descriptor includes the URI, the MIME type the contents were interpreted as, and whether the
write updated content, metadata, or both.
For example, the return value of the above call is as follows:
{ documents: [
{ uri: '/doc/example1.json',
mime-type: 'application/json',
category: ['metadata','content']
},{
uri: '/doc/example2.json',
mime-type: 'application/json',
category: ['metadata','content']
}
]}
Note that the category property indicates both content and metadata were updated even though no
metadata was explicitly specified. This is because system default metadata values were implicitly
assigned to the documents.
To include metadata for a document when you load multiple documents, include
document-specific metadata in the descriptor for that document. To specify metadata that applies
to multiple documents include a metadata descriptor in the parameter list or documents property.
For example, to add the two documents to the collection “examples”, add a metadata descriptor
before the document descriptors, as shown below. The order of the descriptors matters as the set
of descriptors is processed in the order it appears. A metadata descriptor only affects document
descriptors that appear after it in the parameter list or documents array.
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.write({
documents: [
{ contentType: 'application/json',
collections: [ 'examples' ]
},
{ uri: '/doc/example1.json',
contentType: 'application/json',
content: { data: 'one' }
},
{ uri: '/doc/example2.json',
contentType: 'application/json',
content: { data: 'two' }
}
]
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 42
}).result(null, function(error) {
console.log(JSON.stringify(error));
});
2.2.6
Inserting or Updating Metadata for One Document
To insert or update metadata for a specific document, include one or more metadata properties in
the document descriptor passed to DatabaseClient.documents.write. To insert or update the same
metadata for multiple documents, you can include a metadata descriptor in a multi-document
write; for details, see “Working with Metadata” on page 64.
Note: When setting permissions, at least one update permission must be included.
Metadata is replaced on update, not merged. For example, if your document descriptor includes a
collections property, then calling DatabaseClient.documents.write replaces all existing
collection associations for the document.
The following example inserts a document with URI /doc/example.json and adds it to the
collections “examples” and “metadata-examples”. If the document already exists and is part of
other collections, it is removed from those collections.
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.write(
{ uri: '/doc/example.json',
collections: ['examples', 'metadata-examples'],
contentType: 'application/json',
content: { some: 'data' }
})
.result(null, function(error) {
console.log(JSON.stringify(error));
});
To insert or update just metadata for a document, omit the content property. For example, the
following code sets the quality to 2 and the collections to “some-collection”, without changing the
document contents:
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.write(
{ uri: '/doc/example.json',
collections: ['some-collection'],
quality: 2,
})
.result(null, function(error) {
console.log(JSON.stringify(error));
});
MarkLogic Server
Manipulating Documents
MarkLogic 10—June, 2019
Node.js Application Developer’s Guide—Page 43
2.2.7
Automatically Generating Document URIs
You can have document URIs automatically generated during insertion by replacing the uri
property in your document descriptor with an extension property, as described below.
Note: You can only use this feature to create new documents. To update an existing
document, you must know its URI.
To use this feature, construct a document descriptor with the following characteristics:
• Omit the uri property.
•
Include an extension property that specifies the generated URI extension, such as “xml”
or “json”. Do not include a “dot” (.) prefix. That is, specify “json”, not “.json”.
• Optionally, include a directory property that specifies a database directory prefix for the
generated URI. The directory prefix must end in a forward slash (/).
The following example inserts a document into the database with a URI of the form
/my/directory/auto-generated.json.
const marklogic = require('marklogic');
const my = require('./my-connection.js');
const db = marklogic.createDatabaseClient(my.connInfo);

db.documents.write(
{ extension: 'json',
directory: '/my/directory/',
content: { some: 'data' },
contentType: 'application/json'
}
).result(
function(response) {
console.log('Loaded ' + response.documents[0].uri);
},
function(error) {
console.log(JSON.stringify(error));
}
);
Running the above script results in output similar to the following upon success:
Loaded /my/directory/16764526972136717799.json
2.2.8
Transforming Content During Ingestion
You can transform content during ingestion by applying a custom write transform. A transform is
server-side XQuery, JavaScript, or XSLT that you install in the modules database associated with
your REST API instance. You can install transforms using the config.transforms functions. This
topic describes how to apply a transform during ingestion. For more details and examples, see
“Working with Content Transformations” on page 233.