caf_redis/plug_checkpoint

A plug to access an external checkpointing service that supports LUA scripting (Redis version >= 2.6).

The goal is to enable full pipelining of requests while still guaranteeing ownership of leases during operations.

Lua scripts are executed atomically by the server, and therefore, we can combine the check and the update in the same script.

We also implement coalescing of state updates across different CAs in a single script, to amortize the setup cost of LUA with several updates.

This improves dramatically Redis throughput, and also reduces client overhead.

The drawback is an increase in latency when the system is not under heavy load, and we need a cron to bound that increase.

Typical coalescing config values for a fast server are a maximum of 10 requests or 10 extra msec. Note that cron scheduling is not very accurate under load, and we also need a limit on the number of pending requests.

Source:

Extends

Methods

checkBits(bitfieldName, bits, cb0)

Source:

Check bits in a bitfield. If any is not set it returns false.

Parameters:
Name Type Description
bitfieldName string

The name of the bitfield.
bits Array.<number>

The index of bits to be checked.
cb0 cbType

A callback to return true if all set or null otherwise (or an error).

clearBits(bitfieldName, cb0)

Source:

Clear all bits in a bitfield.

Parameters:
Name Type Description
bitfieldName string

The name of the bitfield.
cb0 cbType

A callback to notify an error.

createMap(mapName, initialValue, cb0)

Source:

Creates a Map or returns the contents of an existing one.

Parameters:
Name Type Description
mapName string

The name of the map.
initialValue Object.<string, jsonType> | null

An optional initial value. This value is ignored if the map was already initialized.
cb0 cbType

A callback with the contents of the map.

deleteMap(mapName, cb0)

Source:

Deletes a Map.

Parameters:
Name Type Description
mapName string

The name of the map.
cb0 cbType

A callback with an error or null.

deleteState(id, cb0)

Source:

Removes the state of a CA in the checkpointing service.

Parameters:
Name Type Description
id string

An identifier for the CA.
cb0 cbType

A callback to notify an error deleting or its succesful completion if the argument is a falsy.

getCache(key, cb0)

Source:

Looks up a cache entry.

Parameters:
Name Type Description
key string

The cache entry key.
cb0 cbType

A callback returning the value (or null).

getState(id, cb0)

Source:

Gets the state of a CA from the checkpointing service.

Note that only the current (lease) owner can read this state.

Parameters:
Name Type Description
id string

An identifier for the CA.
cb0 function

A callback to notify an error getting the state or (in a second argument) the serialized state of that CA.

grabLease(id, leaseTimeout, cb0)

Source:

Grabs a lease that guarantees exclusive ownership of a CA by this node.

Parameters:
Name Type Description
id string

An identifier for the CA.
leaseTimeout number

Duration of the lease in seconds.
cb0 cbType

A callback with optional (error) argument containing the current owner in a remoteNode:string error field, if we failed to acquire the lease. Null error argument if we succeeded.

grabNodeLease(allPublicNodeIds, privateNodeId, leaseTimeout, cb0)

Source:

Picks an available public nodeId for this process and binds it with a lease to the private node identifier.

Parameters:
Name Type Description
allPublicNodeIds Array.<string>

A list of candidates for a public nodeId for this process.
privateNodeId string

A private unique identifier for the current node.js process.
leaseTimeout number

Duration of the lease in seconds.
cb0 function

A callback with an error if we cannot renew lease or the new external nodeId.

listNodes(allPublicNodeIds, cb0)

Source:

Lists status of nodes registered with this redis instance.

Parameters:
Name Type Description
allPublicNodeIds Array.<string> | null

A list of public nodeIds that we are interested on, or null if we want them all.
cb0 function

A callback with an error or an object with external nodeIds to private nodeIds (containing only the ones in use).

publishPubSub(topic, message, cb0)

Source:

Publishes a message to a channel using the PubSub service.

Parameters:
Name Type Description
topic string

The channel name.
message string

The message to be published.
cb0 cbType

A callback to return a publish error.

readMap(mapName, cb0)

Source:

Reads all the contents of a map.

Parameters:
Name Type Description
mapName string

The name of the map.
cb0 cbType

A callback with an error if it does not exist or an object (second argument) of type caf.changes : {version:number, remove:[], add:Array (i.e., flattened key value pairs)}

renewLeases(ids, leaseTimeout, cb0)

Source:

Renews a list of leases currently owned by this node.

Parameters:
Name Type Description
ids Array.<string>

A list of identifiers for local CAs.
leaseTimeout number

Duration of the lease in seconds.
cb0 function

A callback with either an error (first) argument or a (second) argument with a list of CA Ids that we failed to renew.

renewNodeLease(privateNodeId, leaseTimeout, cb0)

Source:

Renews a lease associated with this node.js process.

Parameters:
Name Type Description
privateNodeId string

A private unique identifier for the current node.js process.
leaseTimeout number

Duration of the lease in seconds.
cb0 function

A callback with an error if we cannot renew lease.

setBits(bitfieldName, bits, cb0)

Source:

Sets bits in a bitfield.

Parameters:
Name Type Description
bitfieldName string

The name of the bitfield.
bits Array.<number>

The index of bits to be set.
cb0 cbType

A callback to notify an error.

subscribeMap(mapName, handler, cb0)

Source:

Listens to Map updates.

Parameters:
Name Type Description
mapName string

The name of the map.
handler cbType

A handler to listen to updates or errors.
cb0 cbType

A callback with an error or null.

subscribePubSub(topic, deliverMsgF, cb0)

Source:

Subscribes to a channel in the PubSub service.

Parameters:
Name Type Description
topic string

The channel name.
deliverMsgF function

Handler of received messages. First argument is the topic, second the message.
cb0 cbType

A callback to return a subscription error.

unsubscribeMap(mapName, cb0)

Source:

Unregisters a Map update listener.

Parameters:
Name Type Description
mapName string

The name of the map.
cb0 cbType

A callback with an error or null.

unsubscribePubSub(topic, cb0)

Source:

Unsubscribes to a channel in the PubSub service.

Parameters:
Name Type Description
topic string

The channel name.
cb0 cbType

A callback to return a subscription error.

updateCache(key, value, timeout, cb0)

Source:

Updates a cache entry.

Parameters:
Name Type Description
key string

The cache entry key.
value string

The cache entry value.
timeout number

An expire time in seconds.
cb0 cbType

A callback to return an error.

updateMap(mapName, changes, cb0)

Source:

Updates a Map.

The type changesType is a normalized update (no duplicated keys) of the form:

{version: number, remove : Array.<string>, add : Array<Object>}

It publishes in a channel with id mapName the JSON-encoded changes contents.

Parameters:
Name Type Description
mapName string

The name of the map.
changes changesType

A delta with changes.
cb0 cbType

A callback with an error if it does not exist or the number of subscribed clients.

updateState(id, newValue, cb0)

Source:

Updates the state of a CA in the checkpointing service.

Parameters:
Name Type Description
id string

An identifier for the CA.
newValue string

A serialized new state for this CA.
cb0 cbType

A callback to notify of an error updating or succesful completion if falsy argument.