Xirsys Partners (Beta) – A Walkthrough

Overview

Xirsys Partners is a means of leveraging the Xirsys infrastructure for customers who want better integration with Xirsys or would like to provide Xirsys-like services of their own:

The Partners API provides for:

  • The ability to administer domains for other users. That is, you may add your own customers, and then administer their domains.
  • Generate tokens for our signal servers.
  • Use of KV4, a 4 dimensional key value store.
  • List subscribers and publish to given node.
  • More than 1 authorized user. As well as a username/secret combination, partners can add admins that are authorized by email/pw within the Partner’s own dbs.

The API comes as a self describing JSON download here.

Partner Client

Xirsys Partners comes with an associated command line client.

The client provides prompted usage, history, curl history and other useful goodies. The client provides access by downloading the latest API from the link above, and interpreting it for command line purposes. So, the API link is extremely important, and current.

The rest of this document will describe these services in detail.

Getting Started with the Client

The easiest way to get started with the Xirsys Partners API is to use the command line client. It’s a node.js client and available on NPM.

So please install node.js first, and then:

The client will download the API first, and then provide all the commands currently defined in the API. When a new version of the API is available it will instruct you on how to upgrade the cached version.

By default, the client stores 2 files in your home directory:

1. ~/.xirsys api A cache of current version of the API
2. ~./xirsys history   Your usage history minus authentication details.

Help

To find the list of all available commands, just type ‘xirsys’ with no parameters, to get:

As can be seen the Partner API provides a synthesized view of Xirsys services.

Versioning

Because the API is self describing, we hope we won’t need to update the client very often, rather we will update the API. However, and probably especially in the early days, if you do need to update your client, then:

If you have issues with an upgrade remove the client completely with:

and reinstall with:

Command line Credentials

Each command takes a “partner” parameter always as the first parameter. The partner parameter is a string with the following format:

These are your standard Xirsys account details and are required on every invocation of the Partner API.

Because the command credentials are added to every command we’ve abstracted them out.

Will automatically insert the details on each command.

*Note: The xirsys history does not store the credentials in ~/.xirsys history, rather they will be substituted in from XIRSYS CRED on rewind

Useful Basic Commands

xirsys help Display a table of all available commands.
xirsys help [command]     Display raw API info from the API file.
xirsys history-list   Show command history, this is a listing of the ./xirsys history file.
xirsys history-curl   Show history as curl commands.
xirsys api-version   Query server for latest version. Typically you’ll be prompted when a new version is posted.
xirsys api-refresh   Get latest version and save in ~/.xirsys api

Examining a Call

Let’s examine the structure of an API call, by getting the API specification for room-add.

Note: This is not a strict REST api, it does not take into account DELETE/GET and so on. Rather it maps parameters directly onto our internal Partner API.

This type of meta data describes what you should provide for the call as “args” and what you can expect back as “ret”. There’s the self explanatory “doc” and the endpoint you should POST to.

Args

As described previously, you can see the “partner” arg is the first given argument, followed by 4 other string arguments.

Ret

The API always follows the convention:

So typically you’ll have status “ok” or “error” and either a single string value or an array of objects.

Partner Accounts

One of the main features of the Partners API is the ability for a Xirsys Partner to sign up their own customers. That is, a Partner can create their own customers, add and remove domains/apps/rooms on behalf of their customers, query traffic statistics, so on and so forth.

Any existing Xirsys account, could become a Partners account, the original account is maintained within the primary Xirsys database, but Partners customer accounts are stored in their own database. What’s more, the Partner’s database could be external to the Xirsys system, this allows you to add value in ways we can’t imagine.

Note: Your original Xirsys account and domains remain intact.

In Beta, this feature is available to all accounts, but remember it will be restricted to Standard/Pro Accounts when we go live.

This feature is enabled via the use of the account-* commands:

xirsys help account-create provides the full API info on the command , however just the arguments are pertinent to the discussion here.

As ever the “partner” param is required, but we also have a second parameter called “details”. “details” is actually an object.

To facilitate the entry of objects, the command line will prompt you field by field for each item if you omit typing the parameters yourself, e.g.:

This is a good way of entering data first time around, as thereafter you may use history-curl to find out how it was posted, and then copy/paste the curl command for your own purposes.

Notice the return value of the account-create command corresponds to the type data given for “ret” in xirsys help account-create:

Remember that “s” stands for status. If you get an “error” for the status, the “v” will be the message.

If the “s” is “ok”, then the return value will take the shape of the object indicated in “v” of the the type specification.

Inspection

Once you have created your account, check to see if it exists.

account-list

account-get

Retrieve by username:

account-by-email

Or retrieve by email:

Again, you can query the API to know what is expected on return:

And if you wanted to repeat the process with curl, just use xirsys history-curl:

What’s going on behind the scenes?

When you create a new account, Xirsys will create a new “table” behind the scenes for your customers. As we use CouchDB, a JSON document database, a better word than “table” is “bucket”, but table and bucket are equivalent for our purposes.

By saving your data in your bucket, we enable private replication going forward for all your data. * Note replication is not part of this BETA release.

How can I use this new account?

Step 1 – Add a domain.

The first thing you need to do is get a domain name. Typically, you would provide a front end service to get this information from your customers, just as Xirsys got the information when you signed up with us. However, for illustrative purposes, we can do:

We have just added a new domain, for customer ‘un’, whom we created above with account-create.

As we’ve described, you have a private customer bucket, wherein you may create customers unique to that bucket, however, Xirsys maintains a global (x-data-center) tree of all domains – thus domains are unique system wide.

Step 2 – Get a Token.

Now that you have created a sub account, you’ll need a token so you can login into the Xirsys Signal Servers.

Existing API

The existing Xirsys API and Javascript SDK may use the new user directly by specifying the composite identifier of sub-account-name#main-account-name and the secret of the sub-account.

Where async is my main account. More examples of this later.

More Domain Manipulation and Me!

Very similar to the services provided by the existing API, the Partners API provides the ability to add to your tree with the following commands:

domain-add Add a new domain
domain-delete Delete a domain
domain-list List domains
application-add Add an app to a domain
application-delete     Delete an app from a domain
application-list List apps in a domain
room-added Add a room to an app
room-delete Delete a room from an app
room-list List rooms in an app

Each of these commands is performed in the context of “username” where username is your customer.

Above we added a domain for customer ‘un’. Let’s see if there are other domains:

This is the first example of an array returned from the api, let’s see how it’s represented in the type specification:

So, on a status “ok”, we will get a “v” which is a representation of an array of strings.

Me!

The domain manipulation commands above are quite useful on a general basis. For example, if you’re administrating your own domains as user:secret, then these commands and others could be quite handy for your own Xirsys account.

So, we’ve provided the special user Me! which will allow you to administer your own domains:

Live Features

The live features of the Partner API provide direct access to the signaling services of Xirsys.

live-chat

As already mentioned:

allows you to retrieve a token that enables a specific user to attach to our signal servers.

As a convenience and really just for testing purposes, the client bundles this behavior up and provides a simple interface for chatting in a specific room.

live-subscribers

live-broadcast

Metrics

To find out usage on given domains over time, use these commands.

usage-signals

This command provides signal traffic statistics by domain or path within domain with a date range:

Return Value

The value returned, provides the total number of bytes transferred in “sum”, the number of packets that have passed through the system in “count”, and an average packet size in “mean”.

From and To

The [from] date and [to] date parameters are both optional, just hit enter to ignore them. If they are both empty, then the system will run a full query over all existing records.

If [from] is filled but [to] is empty the query will run from [from] until now.

Username

Should be a sub account you have created or your own account me!.

Path

Path can be a path or partial path to a room, e.g.:

/my.supercool.com/default/default

Or you can find the aggregation of your data over applications, e.g.:

/my.supercool.com/default

Or just domain, my.supercool.com. Minimally, you must provide at least the domain.

usage-turn

This command provides insight into turn traffic:

Introducing KV4 – 4D Key/Value Store

Status: EXPERIMENTAL, just for evaluation and testing purposes.

We’re releasing KV4 early to get your feedback. Again, an integral part of Xirsys 3.0, KV4 is simply a means of providing data integration with our existing services – **and our services are based around a global tree**.

The Model

Dimension Name Description
1D key This a hash table, a 1D k/v store
2D path KV4 indexes first by tree path then by key. Thus we can put/get to a hash table via the second dimension index of path [*d/a/r*]
3D layer In addition we offer a 3rd dimension – layering
4D time As a fourth dimension we offer, not surprisingly time. Each Kv4.put never destroys the old value, you add the latest value. Thus, you may always retain a time series

Storage

Because you may now administer your customer’s domains as well as your own, and the tree structure of each domain is different, we’ve opted to store KV4 data by domain. That is each domain created gets it’s own bucket. This in turn means that at any stage this data is replicable (not available in BETA).

As a Partner you can add data into the KV4 stores of your customers allowing you to add value.

Important

You may add/remove domains arbitrarily, and thus the KV4 stored per domain, will become obsolete on removal of the domain. In BETA we’re currently removing the bucket – for live we will need a somewhat more nuanced strategy to prevent disaster 😉

Integration

KV4 puts are baked into the signaling services and analytics. You can receive event notifications via a soon to be released Xirsys 3.0 client API.

So how does it work already?

The great thing about k/v stores, 4 dimensional or not, is you’re just working with 3 coordinates, [path,key,layer], as a reference into your space. Usefully the client will prompt us for everything it needs:

What happened?

You added a value at coordinate:

[“/my.supercool.com/default/default”,”accessor”,”default”]

for the customer ‘un’.

Notice, the layer was omitted and defaulted to “default”.

If you want to add data to your own KV4 store for your own domain, just use me! for the username.

Let’s get it back:

Lets get some history and add a new value using curl for a change:

Did it stick?:

Yes, the new value is safely in place … however …

Time Series

One of the interesting features of KV4, is it doesn’t delete your old value. When you do a “get” you see the latest value, but the old value is still there.

There they are, the items associated with this [path,key,layer], with their time of creation as an integer.

Inspection

Use data-keys to get a list of keys at a given node.

Purging

For the live version we will provide system purge options for your KV4 databases. For example, I want my key to retain only 1 hour’s worth of data or 1 month’s worth of data etc.

For now however, we provide some commands:

data-delete-key

data-delete-layer

Let’s add a meta data layer:

Does it exist?:

Yes, there’s m1, so can we delete the layer:

And it’s gone:

Future: Private Data (Not in Beta)

As we push toward Xirsys 3.0 (Q1 2016), we will provide the ability to use your own external dbs for storage and provide replication services for your data.

This allows you to add value in ways we have not imagined, this could include:

  • better privacy (data on our servers is ephemeral)
  • backup strategies
  • other means of indexing/mining the data
  • integration of other services

Thus, you could use a cloud data service like IBM’s Cloudant (recommended) or host your own CouchDB document storage. We will look to provide other 3rd party storage going forward.

Please contact us if you feel this feature would be useful to you.