How to add....

Adding a Class to the API

This document describes how to add a new class to the data model that defines the Xen Server API. It complements two other documents that describe how to extend an existing class:

As a running example, we will use the addition of a class that is part of the design for the PVS Direct feature. PVS Direct introduces proxies that serve VMs with disk images. This class was added via commit CP-16939 to Xen API.

Example: PVS_server

In the world of Xen Server, each important concept like a virtual machine, interface, or users is represented by a class in the data model. A class defines methods and instance variables. At runtime, all class instances are held in an in-memory database. For example, part of [PVS Direct] is a class PVS_server, representing a resource that provides block-level data for virtual machines. The design document defines it to have the following important properties:

Fields

(string set) addresses (RO/constructor) IPv4 addresses of the server.
(int) first_port (RO/constructor) First UDP port accepted by the server.
(int) last_port (RO/constructor) Last UDP port accepted by the server.
(PVS_farm ref) farm (RO/constructor) Link to the farm that this server is included in. A PVS_server object must always have a valid farm reference; the PVS_server will be automatically GC’ed by xapi if the associated PVS_farm object is removed.
(string) uuid (R0/runtime) Unique identifier/object reference. Allocated by the server.

Methods (or Functions)

(PVS_server ref) introduce (string set addresses, int first_port, int last_port, PVS_farm ref farm) Introduce a new PVS server into the farm. Allowed at any time, even when proxies are in use. The proxies will be updated automatically.
(void) forget (PVS_server ref self) Remove a PVS server from the farm. Allowed at any time, even when proxies are in use. The proxies will be updated automatically.

Implementation Overview

The implementation of a class is distributed over several files:

ocaml/idl/datamodel.ml – central class definition
ocaml/idl/datamodel_types.ml – definition of releases
ocaml/xapi/cli_frontend.ml – declaration of CLI operations
ocaml/xapi/cli_operations.ml – implementation of CLI operations
ocaml/xapi/records.ml – getters and setters
ocaml/xapi/OMakefile – refers to xapi_pvs_farm.ml
ocaml/xapi/api_server.ml – refers to xapi_pvs_farm.ml
ocaml/xapi/message_forwarding.ml
ocaml/xapi/xapi_pvs_farm.ml – implementation of methods, new file

Data Model

The data model ocaml/idl/datamodel.ml defines the class. To keep the name space tidy, most helper functions are grouped into an internal module:

(* datamodel.ml *)

let schema_minor_vsn = 103 (* line 21 -- increment this *)
let _pvs_farm = "PVS_farm" (* line 153 *)

module PVS_farm = struct (* line 8658 *)
  let lifecycle = [Prototyped, rel_dundee_plus, ""]

  let introduce = call
    ~name:"introduce"
    ~doc:"Introduce new PVS farm"
    ~result:(Ref _pvs_farm, "the new PVS farm")
    ~params:
    [ String,"name","name of the PVS farm"
    ]
    ~lifecycle
    ~allowed_roles:_R_POOL_OP
    ()

  let forget = call
    ~name:"forget"
    ~doc:"Remove a farm's meta data"
    ~params:
    [ Ref _pvs_farm, "self", "this PVS farm"
    ]
    ~errs:[
      Api_errors.pvs_farm_contains_running_proxies;
      Api_errors.pvs_farm_contains_servers;
    ]
    ~lifecycle
    ~allowed_roles:_R_POOL_OP
    ()


  let set_name = call
    ~name:"set_name"
    ~doc:"Update the name of the PVS farm"
    ~params:
    [ Ref _pvs_farm, "self", "this PVS farm"
    ; String, "value", "name to be used"
    ]
    ~lifecycle
    ~allowed_roles:_R_POOL_OP
    ()

  let add_cache_storage = call
    ~name:"add_cache_storage"
    ~doc:"Add a cache SR for the proxies on the farm"
    ~params:
    [ Ref _pvs_farm, "self", "this PVS farm"
    ; Ref _sr, "value", "SR to be used"
    ]
    ~lifecycle
    ~allowed_roles:_R_POOL_OP
    ()

  let remove_cache_storage = call
    ~name:"remove_cache_storage"
    ~doc:"Remove a cache SR for the proxies on the farm"
    ~params:
    [ Ref _pvs_farm, "self", "this PVS farm"
    ; Ref _sr, "value", "SR to be removed"
    ]
    ~lifecycle
    ~allowed_roles:_R_POOL_OP
    ()

  let obj =
    let null_str = Some (VString "") in
    let null_set = Some (VSet []) in
    create_obj (* <---- creates class *)
    ~name: _pvs_farm
    ~descr:"machines serving blocks of data for provisioning VMs"
    ~doccomments:[]
    ~gen_constructor_destructor:false
    ~gen_events:true
    ~in_db:true
    ~lifecycle
    ~persist:PersistEverything
    ~in_oss_since:None
    ~messages_default_allowed_roles:_R_POOL_OP
    ~contents:
    [ uid     _pvs_farm ~lifecycle

    ; field   ~qualifier:StaticRO ~lifecycle
              ~ty:String "name" ~default_value:null_str
              "Name of the PVS farm. Must match name configured in PVS"

    ; field   ~qualifier:DynamicRO ~lifecycle
              ~ty:(Set (Ref _sr)) "cache_storage" ~default_value:null_set
              ~ignore_foreign_key:true
              "The SR used by PVS proxy for the cache"

    ; field   ~qualifier:DynamicRO ~lifecycle
              ~ty:(Set (Ref _pvs_server)) "servers"
              "The set of PVS servers in the farm"


    ; field   ~qualifier:DynamicRO ~lifecycle
              ~ty:(Set (Ref _pvs_proxy)) "proxies"
              "The set of proxies associated with the farm"
    ]
    ~messages:
    [ introduce
    ; forget
    ; set_name
    ; add_cache_storage
    ; remove_cache_storage
    ]
    ()
end
let pvs_farm = PVS_farm.obj

The class is defined by a call to create_obj and it defines the fields and messages (methods) belonging to the class. Each field has a name, a type, and some meta information. Likewise, each message (or method) is created by call that describes its parameters.

The PVS_farm has additional getter and setter methods for accessing its fields. These are not declared here as part of the messages but are automatically generated.

To make sure the new class is actually used, it is important to enter it into two lists:

(* datamodel.ml *)
let all_system = (* line 8917 *)
  [
    ...
    vgpu_type;
    pvs_farm;
    ...
  ]

let expose_get_all_messages_for = [ (* line 9097 *)
  ...
  _pvs_farm;
  _pvs_server;
  _pvs_proxy;

When a field refers to another object that itself refers back to it, these two need to be entered into the all_relations list. For example, _pvs_server refers to a _pvs_farm value via "farm", which, in turn, refers to the _pvs_server value via its "servers" field.

let all_relations =
  [
    (* ... *)
    (_sr, "introduced_by"), (_dr_task, "introduced_SRs");
    (_pvs_server, "farm"), (_pvs_farm, "servers");
    (_pvs_proxy,  "farm"), (_pvs_farm, "proxies");
  ]

CLI Conventions

The CLI provides access to objects from the command line. The following conventions exist for naming fields:

A field in the data model uses an underscore (_) but a hyphen (-) in the CLI: what is cache_storage in the data model becomes cache-storage in the CLI.
When a field contains a reference or multiple, like proxies, it becomes proxy-uuids in the CLI because references are always referred to by their UUID.

CLI Getters and Setters

All fields can be read from the CLI and some fields can also be set via the CLI. These getters and setters are mostly generated automatically and need to be connected to the CLI through a function in ocaml/xapi/records.ml. Note that field names here use the naming convention for the CLI:

(* ocaml/xapi/records.ml *)
let pvs_farm_record rpc session_id pvs_farm =
  let _ref = ref pvs_farm in
  let empty_record =
    ToGet (fun () -> Client.PVS_farm.get_record rpc session_id !_ref) in
  let record = ref empty_record in
  let x () = lzy_get record in
    { setref    = (fun r -> _ref := r ; record := empty_record)
    ; setrefrec = (fun (a,b) -> _ref := a; record := Got b)
    ; record    = x
    ; getref    = (fun () -> !_ref)
    ; fields=
      [ make_field ~name:"uuid"
        ~get:(fun () -> (x ()).API.pVS_farm_uuid) ()
      ; make_field ~name:"name"
        ~get:(fun () -> (x ()).API.pVS_farm_name)
        ~set:(fun name ->
          Client.PVS_farm.set_name rpc session_id !_ref name) ()
      ; make_field ~name:"cache-storage"
        ~get:(fun () -> (x ()).API.pVS_farm_cache_storage
          |> List.map get_uuid_from_ref |> String.concat "; ")
        ~add_to_set:(fun sr_uuid ->
          let sr = Client.SR.get_by_uuid rpc session_id sr_uuid in
          Client.PVS_farm.add_cache_storage rpc session_id !_ref sr)
        ~remove_from_set:(fun sr_uuid ->
          let sr = Client.SR.get_by_uuid rpc session_id sr_uuid in
          Client.PVS_farm.remove_cache_storage rpc session_id !_ref sr)
        ()
      ; make_field ~name:"server-uuids"
        ~get:(fun () -> (x ()).API.pVS_farm_servers
          |> List.map get_uuid_from_ref |> String.concat "; ")
        ~get_set:(fun () -> (x ()).API.pVS_farm_servers
          |> List.map get_uuid_from_ref)
        ()
      ; make_field ~name:"proxy-uuids"
        ~get:(fun () -> (x ()).API.pVS_farm_proxies
          |> List.map get_uuid_from_ref |> String.concat "; ")
        ~get_set:(fun () -> (x ()).API.pVS_farm_proxies
          |> List.map get_uuid_from_ref)
        ()
      ]
    }

CLI Interface to Methods

Methods accessible from the CLI are declared in ocaml/xapi/cli_frontend.ml. Each declaration refers to the real implementation of the method, like Cli_operations.PVS_far.introduce:

(* cli_frontend.ml *)
let rec cmdtable_data : (string*cmd_spec) list =
  (* ... *)
  "pvs-farm-introduce",
  {
    reqd=["name"];
    optn=[];
    help="Introduce new PVS farm";
    implementation=No_fd Cli_operations.PVS_farm.introduce;
    flags=[];
  };
  "pvs-farm-forget",
  {
    reqd=["uuid"];
    optn=[];
    help="Forget a PVS farm";
    implementation=No_fd Cli_operations.PVS_farm.forget;
    flags=[];
  };

CLI Implementation of Methods

Each CLI operation that is not a getter or setter has an implementation in cli_operations.ml which is implemented in terms of the real implementation:

(* cli_operations.ml *)
module PVS_farm = struct
  let introduce printer rpc session_id params =
    let name  = List.assoc "name" params in
    let ref   = Client.PVS_farm.introduce ~rpc ~session_id ~name in
    let uuid  = Client.PVS_farm.get_uuid rpc session_id ref in
    printer (Cli_printer.PList [uuid])

  let forget printer rpc session_id params =
    let uuid  = List.assoc "uuid" params in
    let ref   = Client.PVS_farm.get_by_uuid ~rpc ~session_id ~uuid in
    Client.PVS_farm.forget rpc session_id ref
end

Fields that should show up in the CLI interface by default are declared in the gen_cmds value:

(* cli_operations.ml *)
let gen_cmds rpc session_id =
  let mk = make_param_funs in
  List.concat
  [ (*...*)
  ; Client.Pool.(mk get_all get_all_records_where
    get_by_uuid pool_record "pool" []
    ["uuid";"name-label";"name-description";"master"
    ;"default-SR"] rpc session_id)
  ; Client.PVS_farm.(mk get_all get_all_records_where
    get_by_uuid pvs_farm_record "pvs-farm" []
    ["uuid";"name";"cache-storage";"server-uuids"] rpc session_id)

Error messages

Error messages used by an implementation are introduced in two files:

(* ocaml/xapi-consts/api_errors.ml *)
let pvs_farm_contains_running_proxies = "PVS_FARM_CONTAINS_RUNNING_PROXIES"
let pvs_farm_contains_servers = "PVS_FARM_CONTAINS_SERVERS"
let pvs_farm_sr_already_added = "PVS_FARM_SR_ALREADY_ADDED"
let pvs_farm_sr_is_in_use = "PVS_FARM_SR_IS_IN_USE"
let sr_not_in_pvs_farm = "SR_NOT_IN_PVS_FARM"
let pvs_farm_cant_set_name = "PVS_FARM_CANT_SET_NAME"

(* ocaml/idl/datamodel.ml *)
  (* PVS errors *)
  error Api_errors.pvs_farm_contains_running_proxies ["proxies"]
    ~doc:"The PVS farm contains running proxies and cannot be forgotten." ();

  error Api_errors.pvs_farm_contains_servers ["servers"]
    ~doc:"The PVS farm contains servers and cannot be forgotten."
    ();

  error Api_errors.pvs_farm_sr_already_added ["farm"; "SR"]
    ~doc:"Trying to add a cache SR that is already associated with the farm"
    ();

  error Api_errors.sr_not_in_pvs_farm ["farm"; "SR"]
    ~doc:"The SR is not associated with the farm."
    ();

  error Api_errors.pvs_farm_sr_is_in_use ["farm"; "SR"]
    ~doc:"The SR is in use by the farm and cannot be removed."
    ();

  error Api_errors.pvs_farm_cant_set_name ["farm"]
    ~doc:"The name of the farm can't be set while proxies are active."
    ()

Method Implementation

The implementation of methods lives in a module in ocaml/xapi:

(* ocaml/xapi/api_server.ml *)
  module PVS_farm = Xapi_pvs_farm

The file below is typically a new file and needs to be added to ocaml/xapi/OMakefile.

(* ocaml/xapi/xapi_pvs_farm.ml *)
module D = Debug.Make(struct let name = "xapi_pvs_farm" end)
module E = Api_errors

let api_error msg xs = raise (E.Server_error (msg, xs))

let introduce ~__context ~name =
  let pvs_farm = Ref.make () in
  let uuid = Uuid.to_string (Uuid.make_uuid ()) in
  Db.PVS_farm.create ~__context
    ~ref:pvs_farm ~uuid ~name ~cache_storage:[];
  pvs_farm

(* ... *)

Messages received on a slave host may or may not be executed there. In the simple case, each methods executes locally:

(* ocaml/xapi/message_forwarding.ml *)
module PVS_farm = struct
  let introduce ~__context ~name =
    info "PVS_farm.introduce %s" name;
    Local.PVS_farm.introduce ~__context ~name

  let forget ~__context ~self =
    info "PVS_farm.forget";
    Local.PVS_farm.forget ~__context ~self

  let set_name ~__context ~self ~value =
    info "PVS_farm.set_name %s" value;
    Local.PVS_farm.set_name ~__context ~self ~value

  let add_cache_storage ~__context ~self ~value =
    info "PVS_farm.add_cache_storage";
    Local.PVS_farm.add_cache_storage ~__context ~self ~value

  let remove_cache_storage ~__context ~self ~value =
    info "PVS_farm.remove_cache_storage";
    Local.PVS_farm.remove_cache_storage ~__context ~self ~value
end

Adding a field to the API

This page describes how to add a field to XenAPI. A field is a parameter of a class that can be used in functions and read from the API.

Bumping the database schema version

Whenever a field is added to or removed from the API, its schema version needs to be increased. XAPI needs this fundamental procedure in order to be able to detect that an automatic database upgrade is necessary or to find out that the new schema is incompatible with the existing database. If the schema version is not bumped, XAPI will start failing in unpredictable ways. Note that bumping the version is not necessary when adding functions, only when adding fields.

The current version number is kept at the top of the file ocaml/idl/datamodel_common.ml in the variables schema_major_vsn and schema_minor_vsn, of which only the latter should be incremented (the major version only exists for historical reasons). When moving to a new XenServer release, also update the variable last_release_schema_minor_vsn to the schema version of the last release. To keep track of the schema versions of recent XenServer releases, the file contains variables for these, such as miami_release_schema_minor_vsn. After starting a new version of Xapi on an existing server, the database is automatically upgraded if the schema version of the existing database matches the value of last_release_schema_*_vsn in the new Xapi.

As an example, the patch below shows how the schema version was bumped when the new API fields used for ActiveDirectory integration were added:

--- a/ocaml/idl/datamodel.ml  Tue Nov 11 16:17:48 2008 +0000
+++ b/ocaml/idl/datamodel.ml  Tue Nov 11 15:53:29 2008 +0000
@@ -15,17 +15,20 @@ open Datamodel_types
  open Datamodel_types

  (* IMPORTANT: Please bump schema vsn if you change/add/remove a _field_.
     You do not have to dump vsn if you change/add/remove a message *)

  let schema_major_vsn = 5
 -let schema_minor_vsn = 55
 +let schema_minor_vsn = 56

  (* Historical schema versions just in case this is useful later *)
  let rio_schema_major_vsn = 5
  let rio_schema_minor_vsn = 19

 +let miami_release_schema_major_vsn = 5
 +let miami_release_schema_minor_vsn = 35
 +
  (* the schema vsn of the last release: used to determine whether we can
     upgrade or not.. *)
  let last_release_schema_major_vsn = 5
 -let last_release_schema_minor_vsn = 35
 +let last_release_schema_minor_vsn = 55

Setting the schema hash

In the ocaml/idl/schematest.ml there is the last_known_schema_hash This needs to be updated to be the next hash after the schema version was bumped. Get the new hash by running make test and you will receive the correct hash in the error message.

Adding the new field to some existing class

ocaml/idl/datamodel.ml

Add a new “field” line to the class in the file ocaml/idl/datamodel.ml or ocaml/idl/datamodel_[class].ml. The new field might require a suitable default value. This default value is used in case the user does not provide a value for the field.

A field has a number of parameters:

The lifecycle parameter, which shows how the field has evolved over time.
The qualifier parameter, which controls access to the field. The following values are possible:

Value	Meaning
StaticRO	Field is set statically at install-time.
DynamicRO	Field is computed dynamically at run time.
RW	Field is read/write.

The ty parameter for the type of the field.
The default_value parameter.
The name of the field.
A documentation string.

Example of a field in the pool class:

field ~lifecycle:[Published, rel_orlando, "Controls whether HA is enabled"]
      ~qualifier:DynamicRO ~ty:Bool
      ~default_value:(Some (VBool false)) "ha_enabled" "true if HA is enabled on the pool, false otherwise";

See datamodel_types.ml for information about other parameters.

Changing Constructors

Adding a field would change the constructors for the class – functions Db.*.create – and therefore, any references to these in the code need to be updated. In the example, the argument ~ha_enabled:false should be added to any call to Db.Pool.create.

Examples of where these calls can be found is in ocaml/tests/common/test_common.ml and ocaml/xapi/xapi_[class].ml.

CLI Records

If you want this field to show up in the CLI (which you probably do), you will also need to modify the Records module, in the file ocaml/xapi-cli-server/records.ml. Find the record function for the class which you have modified, add a new entry to the fields list using make_field. This type can be found in the same file.

The only required parameters are name and get (and unit, of course ). If your field is a map or set, then you will need to pass in get_{map,set}, and optionally set_{map,set}, if it is a RW field. The hidden parameter is useful if you don’t want this field to show up in a *_params_list call. As an example, here is a field that we’ve just added to the SM class:

make_field ~name:"versioned-capabilities"
           ~get:(fun () -> get_from_map (x ()).API.sM_versioned_capabilities)
           ~get_map:(fun () -> (x ()).API.sM_versioned_capabilities)
           ~hidden:true ();

Testing

The new fields can be tested by copying the newly compiled xapi binary to a test box. After the new xapi service is started, the file /var/log/xensource.log in the test box should contain a few lines reporting the successful upgrade of the metadata schema in the test box:

[...|xapi] Db has schema major_vsn=5, minor_vsn=57 (current is 5 58) (last is 5 57)
[...|xapi] Database schema version is that of last release: attempting upgrade
[...|sql] attempting to restore database from /var/xapi/state.db
[...|sql] finished parsing xml
[...|sql] writing db as xml to file '/var/xapi/state.db'.
[...|xapi] Database upgrade complete, restarting to use new db

Making this field accessible as a CLI attribute

XenAPI functions to get and set the value of the new field are generated automatically. It requires some extra work, however, to enable such operations in the CLI.

The CLI has commands such as host-param-list and host-param-get. To make a new field accessible by these commands, the file xapi-cli-server/records.ml needs to be edited. For the pool.ha-enabled field, the pool_record function in this file contains the following (note the convention to replace underscores by hyphens in the CLI):

let pool_record rpc session_id pool =
  ...
[
  ...
  make_field ~name:"ha-enabled" ~get:(fun () -> string_of_bool (x ()).API.pool_ha_enabled) ();
  ...
]}

NB: the ~get parameter must return a string so include a relevant function to convert the type of the field into a string i.e. string_of_bool

See xapi-cli-server/records.ml for examples of handling field types other than Bool.

Adding a function to the API

This page describes how to add a function to XenAPI.

Add message to API

The file idl/datamodel.ml is a description of the API, from which the marshalling and handler code is generated.

In this file, the create_obj function is used to define a class which may contain fields and support operations (known as “messages”). For example, the identifier host is defined using create_obj to encapsulate the operations which can be performed on a host.

In order to add a function to the API, we need to add a message to an existing class. This entails adding a function in idl/datamodel.ml or one of the other datamodel files to describe the new message and adding it to the class’s list of messages. In this example, we are adding to idl/datamodel_host.ml.

The function to describe the new message will look something like the following:

let host_price_of = call ~flags:[`Session]
    ~name:"price_of"
    ~in_oss_since:None
    ~in_product_since:rel_orlando
    ~params:[(Ref _host, "host", "The host containing the price information");
             (String, "item", "The item whose price is queried")]
    ~result:(Float, "The price of the item")
    ~doc:"Returns the price of a named item."
    ~allowed_roles:_R_POOL_OP
    ()

By convention, the name of the function is formed from the name of the class and the name of the message: host and price_of, in the example. An entry for host_price_of is added to the messages of the host class:

let host =
    create_obj ...
        ~messages: [...
                    host_price_of;
                   ]
...

The parameters passed to call are all optional (except ~name and ~in_product_since).

The ~flags parameter is used to set conditions for the use of the message. For example, `Session is used to indicate that the call must be made in the presence of an existing session.
The value of the ~in_product_since parameter is a string taken from idl/datamodel_types.ml indicates the XenServer release in which this message was first introduced.
The ~params parameter describes a list of the formal parameters of the message. Each parameter is described by a triple. The first component of the triple is the type (from type ty in idl/datamodel_types.ml); the second is the name of the parameter, and the third is a human-readable description of the parameter. The first triple in the list is conventionally the instance of the class on which the message will operate. In the example, this is a reference to the host.
Similarly, the ~result describes the message’s return type, although this is permitted to merely be a single value rather than a list of values. If no ~result is specified, the default is unit.
The ~doc parameter describes what the message is doing.
The bool ~hide_from_docs parameter prevents the message from being included in the documentation when generated.
The bool ~pool_internal parameter is used to indicate if the message should be callable by external systems or only internal hosts.
The ~errs parameter is a list of possible exceptions that the message can raise.
The parameter ~lifecycle takes in an array of (Status, version, doc) to indicate the lifecycle of the message type. This takes over from ~in_oss_since which indicated the release that the message type was introduced. NOTE: Leave this parameter empty, it will be populated on build.
The ~allowed_roles parameter is used for access control (see below).

Compiling xen-api.(hg|git) will cause the code corresponding to this message to be generated and output in ocaml/xapi/server.ml. In the example above, a section handling an incoming call host.price_of appeared in ocaml/xapi/server.ml. However, after this was generated, the rest of the build failed because this call expects a price_of function in the Host object.

Expected values in parameter ~in_product_since

In the example above, the value of the parameter ~in_product_since informs that the message host_price_of was added during the rel_orlando release cycle. If a new release cycle is required, then it needs to be added in the file idl/datamodel_types.ml. The patch below shows how the new rel_george release identifier was added. Any class, message, etc. added during the rel_george release cycle should contain ~in_product_since:rel_george entries. (obs: the release and upgrade infrastructure can handle only one new rel_* identifier – in this case, rel_george – in each release)

--- a/ocaml/idl/datamodel_types.ml Tue Nov 11 15:17:48 2008 +0000
+++ b/ocaml/idl/datamodel_types.ml Tue Nov 11 15:53:29 2008 +0000
@@ -27,14 +27,13 @@
 (* useful constants for product vsn tracking *)
 let oss_since_303 = Some "3.0.3"
+let rel_george = "george"
 let rel_orlando = "orlando"
 let rel_orlando_update_1 = "orlando-update-1"
 let rel_symc = "symc"
 let rel_miami = "miami"
 let rel_rio = "rio"
-let release_order = [engp:rel_rio; rel_miami; rel_symc; rel_orlando; rel_orlando_update_1]
+let release_order = [engp:rel_rio; rel_miami; rel_symc; rel_orlando; rel_orlando_update_1; rel_george]

Update expose_get_all_messages_for list

If you are adding a new class, do not forget to add your new class _name to the expose_get_all_messages_for list, at the bottom of datamodel.ml, in order to have automatically generated get_all and get_all_records functions attached to it.

Update the RBAC field containing the roles expected to use the new API call

After the RBAC integration, Xapi provides by default a set of static roles associated to the most common subject tasks.

The api calls associated with each role are defined by a new ~allowed_roles parameter in each api call, which specifies the list of static roles that should be able to execute the call. The possible roles for this list is one of the following names, defined in datamodel.ml:

role_pool_admin
role_pool_operator
role_vm_power_admin
role_vm_admin
role_vm_operator
role_read_only

So, for instance,

~allowed_roles:[role_pool_admin,role_pool_operator] (* this is not the recommended usage, see example below *)

would be a valid list (though it is not the recommended way of using allowed_roles, see below), meaning that subjects belonging to either role_pool_admin or role_pool_operator can execute the api call.

The RBAC requirements define a policy where the roles in the list above are supposed to be totally-ordered by the set of api-calls associated with each of them. That means that any api-call allowed to role_pool_operator should also be in role_pool_admin; any api-call allowed to role_vm_power_admin should also be in role_pool_operator and also in role_pool_admin; and so on. Datamodel.ml provides shortcuts for expressing these totally-ordered set of roles policy associated with each api-call:

_R_POOL_ADMIN, equivalent to [role_pool_admin]
_R_POOL_OP, equivalent to [role_pool_admin,role_pool_operator]
_R_VM_POWER_ADMIN, equivalent to [role_pool_admin,role_pool_operator,role_vm_power_admin]
_R_VM_ADMIN, equivalent to [role_pool_admin,role_pool_operator,role_vm_power_admin,role_vm_admin]
_R_VM_OP, equivalent to [role_pool_admin,role_pool_operator,role_vm_power_admin,role_vm_admin,role_vm_op]
_R_READ_ONLY, equivalent to [role_pool_admin,role_pool_operator,role_vm_power_admin,role_vm_admin,role_vm_op,role_read_only]

The ~allowed_roles parameter should use one of the shortcuts in the list above, instead of directly using a list of roles, because the shortcuts above make sure that the roles in the list are in a total order regarding the api-calls permission sets. Creating an api-call with e.g. allowed_roles:[role_pool_admin,role_vm_admin] would be wrong, because that would mean that a pool_operator cannot execute the api-call that a vm_admin can, breaking the total-order policy expected in the RBAC 1.0 implementation. In the future, this requirement might be relaxed.

So, the example above should instead be used as:

~allowed_roles:_R_POOL_OP  (* recommended usage via pre-defined totally-ordered role lists *)

and so on.

How to determine the correct role of a new api-call:

if only xapi should execute the api-call, ie. it is an internal call: _R_POOL_ADMIN
if it is related to subject, role, external-authentication: _R_POOL_ADMIN
if it is related to accessing Dom0 (via console, ssh, whatever): _R_POOL_ADMIN
if it is related to the pool object: R_POOL_OP
if it is related to the host object, licenses, backups, physical devices: _R_POOL_OP
if it is related to managing VM memory, snapshot/checkpoint, migration: _R_VM_POWER_ADMIN
if it is related to creating, destroying, cloning, importing/exporting VMs: _R_VM_ADMIN
if it is related to starting, stopping, pausing etc VMs or otherwise accessing/manipulating VMs: _R_VM_OP
if it is related to being able to login, manipulate own tasks and read values only: _R_READ_ONLY

Update message forwarding

The “message forwarding” layer describes the policy of whether an incoming API call should be forwarded to another host (such as another member of the pool) or processed on the host which receives the call. This policy may be non-trivial to describe and so cannot be auto-generated from the data model.

In xapi/message_forwarding.ml, add a function to the relevant module to describe this policy. In the running example, we add the following function to the Host module:

let price_of ~__context ~host ~item =
    info "Host.price_of for item %s" item;
    let local_fn = Local.Host.price_of ~host ~item in
    do_op_on ~local_fn ~__context ~host
      (fun session_id rpc -> Client.Host.price_of ~rpc ~session_id ~host ~item)

After the ~__context parameter, the parameters of this new function should match the parameters we specified for the message. In this case, that is the host and the item to query the price of.

The do_op_on function takes a function to execute locally and a function to execute remotely and performs one of these operations depending on whether the given host is the local host.

The local function references Local.Host.price_of, which is a function we will write in the next step.

Implement the function

Now we write the function to perform the logic behind the new API call. For a host-based call, this will reside in xapi/xapi_host.ml. For other classes, other files with similar names are used.

We add the following function to xapi/xapi_host.ml:

let price_of ~__context ~host ~item =
    if item = "fish" then 3.14 else 0.00

We also need to add the function to the interface xapi/xapi_host.mli:

val price_of :
    __context:Context.t -> host:API.ref_host -> item:string -> float

Congratulations, you’ve added a function to the API!

Add the operation to the CLI

Edit xapi-cli-server/cli_frontend.ml. Add a block to the definition of cmdtable_data as in the following example:

"host-price-of",
{
  reqd=["host-uuid"; "item"];
  optn=[];
  help="Find out the price of an item on a certain host.";
  implementation= No_fd Cli_operations.host_price_of;
  flags=[];
};

Include here the following:

The names of required (reqd) and optional (optn) parameters.
A description to be displayed when calling xe help <cmd> in the help field.
The implementation should use With_fd if any communication with the client is necessary (for example, showing the user a warning, sending the contents of a file, etc.) Otherwise, No_fd can be used as above.
The flags field can be used to set special options:
- Vm_selectors: adds a “vm” parameter for the name of a VM (rather than a UUID)
- Host_selectors: adds a “host” parameter for the name of a host (rather than a UUID)
- Standard: includes the command in the list of common commands displayed by xe help
- Neverforward:
- Hidden:
- Deprecated of string list:

Now we must implement Cli_operations.host_price_of. This is done in xapi-cli-server/cli_operations.ml. This function typically extracts the parameters and forwards them to the internal implementation of the function. Other arbitrary code is permitted. For example:

let host_price_of printer rpc session_id params =
  let host = Client.Host.get_by_uuid rpc session_id (List.assoc "host-uuid" params) in
  let item = List.assoc "item" params in
  let price = string_of_float (Client.Host.price_of ~rpc ~session_id ~host ~item) in
  printer (Cli_printer.PList [price])

Tab Completion in the CLI

The CLI features tab completion for many of its commands’ parameters. Tab completion is implemented in the file ocaml/xe-cli/bash-completion, which is installed on the host as /etc/bash_completion.d/cli, and is done on a parameter-name rather than on a command-name basis. The main portion of the bash-completion file is a case statement that contains a section for each of the parameters that benefit from completion. There is also an entry that catches all parameter names ending at -uuid, and performs an automatic lookup of suitable UUIDs. The host-uuid parameter of our new host-price-of command therefore automatically gains completion capabilities.

Executing the CLI operation

Recompile xapi with the changes described above and install it on a test machine.

Execute the following command to see if the function exists:

xe help host-price-of

Invoke the function itself with the following command:

xe host-price-of host-uuid=<tab> item=fish

and you should find out the price of fish.

Adding a XenAPI extension

A XenAPI extension is a new RPC which is implemented as a separate executable (i.e. it is not part of xapi) but which still benefits from xapi parameter type-checking, multi-language stub generation, documentation generation, authentication etc. An extension can be backported to previous versions by simply adding the implementation, without having to recompile xapi itself.

A XenAPI extension is in two parts:

a declaration in the xapi datamodel. This must use the ~forward_to:(Extension "filename") parameter. The filename must be unique, and should be the same as the XenAPI call name.
an implementation executable in the dom0 filesystem with path /etc/xapi.d/extensions/filename

To define an extension

First write the declaration in the datamodel. The act of specifying the types and writing the documentation will help clarify the intended meaning of the call.

Second create a prototype of your implementation and put an executable file in /etc/xapi.d/extensions/filename. The calling convention is:

the file must be executable
xapi will parse the XMLRPC call arguments received over the network and check the session_id is valid
xapi will execute the named executable
the XMLRPC call arguments will be sent to the executable on stdin and stdin will be closed afterwards
the executable will run and print an XMLRPC response on stdout
xapi will read the response and return it to the client.

See the basic example.

Second make a pull request containing only the datamodel definitions (it is not necessary to include the prototype too). This will attract review comments which will help you improve your API further. Once the pull request is merged, then the API call name and extension are officially yours and you may use them on any xapi version which supports the extension mechanism.

Packaging your extension

Your extension /etc/xapi.d/extensions/filename (and dependencies) should be packaged for your target distribution (for XenServer dom0 this would be a CentOS RPM). Once the package is unpacked on the target machine, the extension should be immediately callable via the XenAPI, provided the xapi version supports the extension mechanism. Note the xapi version does not need to know about the specific extension in advance: it will always look in /etc/xapi.d/extensions/ for all RPC calls whose name it does not recognise.

Limitations

On type-checking

if the xapi version is new enough to know about your specific extension: xapi will type-check the call arguments for you
if the xapi version is too old to know about your specific extension: the extension will still be callable but the arguments will not be type-checked.

On access control

if the xapi version is new enough to know about your specific extension: you can declare that a user must have a particular role (e.g. ‘VM admin’)
if the xapi version is too old to know about your specific extension: the extension will still be callable but the client must have the ‘Pool admin’ role.

Since a xapi which knows about your specific extension is stricter than an older xapi, it’s a good idea to develop against the new xapi and then test older xapi versions later.