ZPM Core Functions

zpmlib.zpm.EXEC_TABLE_HEADER = ['Node', 'Status', 'Retcode', 'NodeT', 'SysT', 'UserT', 'DiskReads', 'DiskBytesR', 'DiskWrites', 'DiskBytesW', 'NetworkReads', 'NetworkBytesR', 'NetworkWrites', 'NetworkBytesW']

Column labels for the execution summary table

zpmlib.zpm.NO_AUTH_MSG = 'Auth version 1.0 requires ST_AUTH, ST_USER, and ST_KEY environment variables\nto be set or overridden with -A, -U, or -K.\n\nAuth version 2.0 requires OS_AUTH_URL, OS_USERNAME, OS_PASSWORD, and\nOS_TENANT_NAME OS_TENANT_ID to be set or overridden with --os-auth-url,\n--os-username, --os-password, --os-tenant-name or os-tenant-id. Note:\nadding "-V 2" is necessary for this.'

Message displayed if insufficient auth settings are specified, either on the command line or in environment variables. Shamelessly copied from python-swiftclient.

zpmlib.zpm.SYSTEM_MAP_ZAPP_PATH = 'boot/system.map'

path/filename of the system.map (job description) in every zapp

class zpmlib.zpm.ZeroCloudConnection(authurl=None, user=None, key=None, retries=5, preauthurl=None, preauthtoken=None, snet=False, starting_backoff=1, max_backoff=64, tenant_name=None, os_options=None, auth_version='1', cacert=None, insecure=False, ssl_compression=True, retry_on_ratelimit=False)

An extension of the swiftclient.Connection which has the capability of posting ZeroVM jobs to an instance of ZeroCloud (running on Swift).

authenticate()

Authenticate with the provided credentials and cache the storage URL and auth token as self.url and self.token, respectively.

post_job(job, response_dict=None, response_body_buffer=None)

Start a ZeroVM job, using a pre-uploaded zapp

Parameters:job (object) – Job description. This will be encoded as JSON and sent to ZeroCloud.
zpmlib.zpm._add_file_to_tar(root, path, tar, arcname=None)
Parameters:
  • root – Root working directory.
  • path – File path.
  • tar – Open tarfile.TarFile object to add the files to.
zpmlib.zpm._deploy_zapp(conn, target, zapp_path, auth_opts, force=False)

Upload all of the necessary files for a zapp.

Returns the name an uploaded index file, or the target if no index.html file was uploaded.

Parameters:force (bool) – Force deployment, even if the target container is not empty. This means that files could be overwritten and could cause consistency problems with these objects in Swift.
zpmlib.zpm._generate_job_desc(zapp)

Generate the boot/system.map file contents from the zapp config file.

Parameters:zappdict of the contents of a zapp.yaml file.
Returns:dict of the job description
zpmlib.zpm._generate_uploads(conn, target, zapp_path, auth_opts)

Generate sequence of (container-and-file-path, data, content-type) tuples.

zpmlib.zpm._get_exec_table(resp)

Build an execution summary table from a job execution response.

Parameters:resp (dict) – Response dictionary from job execution. Must contain a headers key at least (and will typically contain status and reason as well).
Returns:Tuple of total execution time (str), prettytable.PrettyTable containing the summary of all node executions in the job.
zpmlib.zpm._get_exec_table_data(headers)

Extract a stats table from execution HTTP response headers.

Stats include things like node name, execution time, number of reads/writes, bytes read/written, etc.

Parameters:headers (dict) – dict of response headers from a job execution request. It must contain at least x-nexe-system, x-nexe-status, x-nexe-retcode, x-nexe-cdr-line.
Returns:Tuple of two items. The first is the total time for the executed job (as a str). The second is a table (2d list) of execution data extracted from X-Nexe-System and X-Nexe-Cdr-Line headers.

Each row in the table consists of the following data:

  • node name
  • node time
  • system time
  • user time
  • number of disk reads
  • number of bytes read from disk
  • number of disk writes
  • number of bytes written to disk
  • number of network reads
  • number of bytes read from network
  • number of network writes
  • number of bytes written to network
zpmlib.zpm._get_swift_zapp_url(swift_service_url, zapp_path)
Parameters:

Here’s a typical usage example, with typical input and output:

>>> swift_service_url = ('http://localhost:8080/v1/'
...                      'AUTH_469a9cd20b5a4fc5be9438f66bb5ee04')
>>> zapp_path = 'test_container/myapp.zapp'
>>> _get_swift_zapp_url(swift_service_url, zapp_path)
'swift://AUTH_469a9cd20b5a4fc5be9438f66bb5ee04/test_container/myapp.zapp'
zpmlib.zpm._guess_auth_version(args)

Guess the auth version from first the command line args and/or envvars.

Command line arguments override environment variables, so we check those first.

Auth v1 arguments:

  • --auth
  • --user
  • --key

Auth v2 arguments:

  • --os-auth-url
  • --os-username
  • --os-password
  • --os-tenant-name

If all of the v1 and v2 arguments are specified, default to 1.0 (this is how python-swiftclient behaves).

If no auth version can be determined from the command line args, we check environment variables.

Auth v1 vars:

  • ST_AUTH
  • ST_USER
  • ST_KEY

Auth v2 vars:

  • OS_AUTH_URL
  • OS_USERNAME
  • OS_PASSWORD
  • OS_TENANT_NAME

The same rule above applies; if both sets of variables are specified, default to 1.0.

If no auth version can be determined, return None.

Parameters:argsargparse.Namespace, representing the args specified on the command line.
Returns:‘1.0’, ‘2.0’, or None
zpmlib.zpm._prepare_auth(version, args, conn)
Parameters:
  • version (str) – Auth version: “0.0”, “1.0”, or “2.0”. “0.0” indicates “no auth”.
  • argsargparse.Namespace instance, with attributes representing the various authentication parameters
  • connZeroCloudConnection instance.
zpmlib.zpm._prepare_job(tar, zapp, zapp_swift_url)
Parameters:
  • tar – The application .zapp file, as a tarfile.TarFile object.
  • zapp (dict) – Parsed contents of the application zapp.yaml specification, as a dict.
  • zapp_swift_url (str) –

    Path of the .zapp in Swift, which looks like this:

    'swift://AUTH_abcdef123/test_container/hello.zapp'
    

    See _get_swift_zapp_url().

Returns:

Extracted contents of the boot/system.map with the swift path to the .zapp added to the devices for each group.

So if the job looks like this:

[{'exec': {'args': 'hello.py', 'path': 'file://python2.7:python'},
  'devices': [{'name': 'python2.7'}, {'name': 'stdout'}],
  'name': 'hello'}]

the output will look like something like this:

[{'exec': {u'args': 'hello.py', 'path': 'file://python2.7:python'},
  'devices': [
    {'name': 'python2.7'},
    {'name': 'stdout'},
    {'name': 'image',
     'path': 'swift://AUTH_abcdef123/test_container/hello.zapp'},
  ],
  'name': 'hello'}]

zpmlib.zpm.bundle_project(root, refresh_deps=False)

Bundle the project under root.

zpmlib.zpm.create_project(location, with_ui=False, template=None)

Create a ZeroVM application project by writing a default zapp.yaml in the specified directory location.

Parameters:
  • location – Directory location to place project files.
  • with_ui – Defaults to False. If True, add basic UI template files as well to location.
  • template – Default: None. If no template is specified, use the default project template. (See zpmlib.zapptemplate.)
Returns:

List of created project files.

zpmlib.zpm.execute(args, response_body_buffer=None)

Execute a zapp remotely on a ZeroCloud deployment.

Returns:A dict with response data, including the keys ‘status’, ‘reason’, and ‘headers’.
zpmlib.zpm.find_project_root()

Starting from the cwd, search up the file system hierarchy until a zapp.yaml file is found. Once the file is found, return the directory containing it. If no file is found, raise a RuntimeError.

Previous topic

Glossary

This Page