Incapture Technologies

Inside the Cloud

Incapture Technologies Blog

 

Rapture API

Published:
November 3, 2014
Author:

In earlier posts in this Rapture Series we have shown the APIs for Document, Series, Blob and Sheet Repositories and how those APIs can be called from the Reflex scripting language. This post is intended to give some more “insider” information into how the Rapture APIs are structured and produced, and how Rapture developers can use the same technique to create their own APIs that have the same non-functional features.

API Positioning

The core Rapture kernel is in some ways simply the implementation of a set of APIs. Server side applications that embed the Rapture Kernel (such as the Schedule Server or Exec Server) use these APIs in an active way to provide higher level functionality as part of the Rapture “platform” but the main coordination of work is performed through the implementation of the APIs. The fundamental idea is that the API forms one of the lowest levels of the Rapture stack (implementation of repositories is the only lower part) and that applications layer themselves above this — there is no “undocumented” APIs used by applications and any Rapture developer can reproduce some of the higher level Rapture applications (such as the Dev Ops web front end) entirely by making calls of the Rapture API. This layering is shown in the diagram below:

APILayers

In this way a user of the API uses an API transport to communicate with a Rapture Kernel environment. In some cases – when Rapture is embedded in an application – the transport is completely internal and it is in effect a “null” transport. In other cases the transport is either an internal HTTP based RPC-style transport or something like Thrift.

API code generation

Internally in Rapture each API is defined in a file that conforms to a Rapture interface definition format. A part of the document repository api is reproduced below:


[ The Blob api is used to manipulate large opaque objects that do have names (displaynames) like other data but do not
have any insight to be gained from their contents from within Rapture. The RESTful API can be used to efficiently download
a blob as a stream (or upload it)]

api(Doc) {

    [This api call can be used to determine whether a given type exists in a given authority.]
    @entitle=/repo/list
    @public Boolean doesDocumentExist(String documentURI);

    [Retrieve the meta data associated with a document - the meta data includes version and user information.]
    @entitle=/data/get/$f(docURI)
    @public DocumentWithMeta getMetaContent(String docURI);

    [Retrieve just the meta data for a given document.]
    @entitle=/data/list/$f(docURI)
    @public DocumentMetadata getMetaData(String docURI);

    [Revert this document back to the previous version, by taking the previous version and making a new version]
    @entitle=/data/write/$f(docURI)
    @public DocumentWithMeta revertDocument(String docURI);

    [Retrieve the content for a document.]
    @entitle=/data/read/$f(docURI)
    @public String getContent(String docURI);

    [Store a document in the Rapture system. The new URI name is returned; this allows parts of the new name to be automatically generated]
    @entitle=/data/write/$f(docURI)
    @public String putContent(String docURI, String content);

    [Attempt to put the content into the repository, but fail if the repository supports versioning and the current version of the
    document stored does not match the version passed. A version of zero implies that the document should not exist. The idea of
    this call is that a client can call getMetaContentP to retrieve an existing document, modify it, and save the content back,
    using the version number in the meta data of the document. If another client has modified the data since it was loaded this
    call will return false, indicating that the save was not possible.]
    @entitle=/data/write/$f(docURI)
    @public Boolean putContentWithVersion(String docURI, String content, int currentVersion);

    [Remove a document from the system. Note that the actual implementation is dependent on the repository;
    the document may simply be tagged as deleted rather than permanently removed.]
    @entitle=/data/write/$f(docURI)
    @public Boolean deleteContent(String docURI);

    [Return a list of full display names of the paths below this one. Ideally optimized depending on the repo.]
    @entitle=/data/read/$f(docURI)
    @public List(RaptureFolderInfo) getChildren(String docURI);
}

There are some key parts to the definition of an API that are important to call attention to. The API definition includes some documentation – this will be reflected in comments in any generated code or in any generated documentation. The entitlement section is important in that it defines the “entitlement path” that a user must possess to invoke this API call. In some cases the entitlement definition includes a reference to a parameter – the docURI in these cases. So the actual entitlement path needed by a user can be varied based on what parameter (document) they are interacting with. Entitlements are the subject of a future blog post but at this point it’s important to realize that entitlements are checked at the API call level and can be made very broad or very specific depending on the use required of the platform.

These api input files are then used as part of a general build process by a Rapture tool called “APIGen”. This tool parses the input files and generates a set of implementations of the API (transport and client side bindings) for a number of languages – Java, C#, Python, Ruby, Go and Javascript. The production is template driven and is straightforward to extend.

APIGen

This same process generates the implementations of stubs used by the Rapture Kernel internally for the main forms of API transport.

The same process can be used by a developer to generate and embed “SDK APIs” – custom APIs defined by a developer but embedded within Rapture, with the same client side language support.

Rapture API areas

We’ve already seen some of the areas of the Rapture API in previous blog posts. Here we’ll simply list the areas with a brief description, with the details behind the APIs covered within this blog series.
Admin
The admin api manages Rapture from a fundamental level. Deleting and creating users, retrieving properties of the platform as a whole, managing the message of the day and so on.
Async
The async API is used as a means to “fire and forget” certain long running activities. If you choose to remember again the API gives you a handle to query the status of an asynchronous job.
Audit
The audit API is used to both write and read audit records.
Blob
The blob API is used to manipulate blobs and their repositories.
Bootstrap
The bootstrap API is used to define the initial document based repositories that store the configuration information for a Rapture environment (users, repositories, entitlements, sessions, etc.)
Decision
The decision API is used to define and manage workflows in Rapture.
Doc
The doc API is used to manipulate documents and their repositories.
Entitlements
The entitlements API is used to manage how users are members of entitlement groups and how entitlement groups map to entitlements in the system.
Environment
The environment API gives the caller the ability to find out about the environment in which Rapture is hosted – license information, server lists and status.
Event
The event API is used to drive an Event Driven Architectural approach. Callers can associate scripts, workflows or messages to named events which are then activated (fired) using a different call. Internally Rapture defines some system events (such as updating data) that can be hooked into.
Feature
The feature API is used to install features in Rapture. Features are collections of scripts, repository definitions and content, events and so on that can be considered to be a single well defined installable concept.
Fountain
A fountain is simply a means to autogenerate ids (such as “order ids”). The fountain api gives access to this ability.
Index
The index API is used to manage indices on document repositories.
Lock
The lock API is used to coordinate complex distributed tasks.
Notification
The notification API is used to separate the publishing of an internal notification on a channel with the receiver or subscriber of that information.
Pipeline
The pipeline API is used to manage message queues used both internally by Rapture and as a means for connecting external environments into and out of Rapture through messaging.
Relationship
The relationship API manages the directed graph repositories used to manage relationships and the ability to query and publish the relationship between entities in Rapture.
Runner
The runner API is used to manage the services and systems that form a Rapture environment.
Schedule
The schedule API manages schedules (similar to cron) that invoke services or scripts in a Rapture environment.
Script
The script API manages scripts (Reflex scripts) and their execution.
Series
The series API manages series repositories and their content.
Sheet
The sheet API manages sheet repositories and their content.
Sys
The sys API is used to manipulate raw data within Rapture’s configuration – bypassing the more specific APIs for that purpose.
User
The user API is used by users to manage their account and presence on Rapture.

Summary

The Rapture API is pretty wide with calls protected using entitlements. An automatic and repeatable process is used to convert API definitions into code that connects clients and the Rapture server and this process can be used by developers for their own APIs.


Subscribe for updates