searcch.importer.util package

Subpackages

• searcch.importer.util.retrieve package
  • Module contents
• searcch.importer.util.unpack package
  • Module contents

Submodules

searcch.importer.util.applicable module

class searcch.importer.util.applicable.Applicable

Bases: object

static add_subparsers(subparsers)

A user should call this method to have ApplicableMethod add subcommand parsers for all the methods/functions it knows of (i.e, that were decorated).

Parameters:subparsers – the argparse action object returned by argparse.ArgumentParser.add_subparsers()

static apply(method_name, options)

Apply some argparse’d options to a function, and return its result.

Parameters:

• method_name – the function name to invoke (i.e., the argparse subcommand that the parser found was invoked).
• options – the argparse options Namespace object

Returns:

the result of applying the function to the given options

static register_function(applicable)

Register a function as an Applicable function. Typically this is invoked by the decorator; user should never do this directly.

(See the parameter documentation for __init__().)

static register_object(applicable, name=None)

Register an instance object (which should be of a type that has already been decorated with ApplicableMethod).

Parameters:applicable – an instance object, some of whose methods have been decorated with ApplicableMethod.

class searcch.importer.util.applicable.ApplicableClass

Bases: object

__init__()

Initialize self. See help(type(self)) for accurate signature.

class searcch.importer.util.applicable.ApplicableFormatter(help=None, largs=None, kwargs=None, autofill=True, excluded=None)

Bases: object

__init__(help=None, largs=None, kwargs=None, autofill=True, excluded=None)

Initialize self. See help(type(self)) for accurate signature.

class searcch.importer.util.applicable.ApplicableMethod(help=None, largs=None, kwargs=None, alias=None, formatter=True, fmtkwargs=None, autofill=True, excluded=None)

Bases: object

A decorator class that transforms class methods into argparse subcommands that can be invoked from a command-line client.

It allows you to expose methods to application to arguments parsed by argparse subcommand parsers. You decorate functions like:

@ApplicableMethod(
    help="Get a self credential",
    kwargs=[dict(name='force',help='Bypass the cache',
                 action='store_true',default=False)])
def get_self_credential(self,force=False,cache_notifier=None)

Finally, once you’ve decorated all the functions you want to expose, you need to bind some actual objects to these decorated definitions, and create an ApplicableMethod, subcommand-enabled argparse.ArgumentParser parser. Here’s an example:

import argparse
import elasticslice.rpc.protogeni as pgrpc
from elasticslice.util.applicable import Applicable

class DefaultSubcommandArgumentParser(argparse.ArgumentParser):
    __default_subparser = None

    def set_default_subparser(self, name):
        self.__default_subparser = name

    def _parse_known_args(self, arg_strings, *args, **kwargs):
        in_args = set(arg_strings)
        dsp = self.__default_subparser
        if dsp is not None and not {'-h', '--help'}.intersection(in_args):
        for x in self._subparsers._actions:
            subparser_found = (
            isinstance(x, argparse._SubParsersAction) and
            in_args.intersection(x._name_parser_map.keys())
        )
        if subparser_found:
            break
        else:
            # insert default in first position, this implies no
            # global options without a sub_parsers specified
            arg_strings = [dsp] + arg_strings
    return super(DefaultSubcommandArgumentParser,self)._parse_known_args(
        arg_strings, *args, **kwargs)

    pass

def parse_options():
    parser = DefaultSubcommandArgumentParser()
    parser.add_argument("-d", "--debug", dest="debug", action='store_true',
                        help="Enable debugging log level")
    parser.add_argument("--config", dest="config_file",
                        default="~/.protogeni/elasticslice.conf",

    # Add in subparsers.
    subparsers = parser.add_subparsers(help="Subcommands",dest='subcommand')
    subparsers.add_parser('interactive',help="Run in interactive mode")
    Applicable.add_subparsers(subparsers)
    parser.set_default_subparser('interactive')

    (options, args) = parser.parse_known_args(sys.argv[1:])
    return (options, args)

(options, args) = parse_options()
config = Config(options)
server = pgrpc.ProtoGeniServer(config=config)
retval = Applicable.apply(options.subcommand,options)
if type(retval) in [str,int,float]:
    print retval
else:
    print json.dumps(retval,indent=2)
pass

__init__(help=None, largs=None, kwargs=None, alias=None, formatter=True, fmtkwargs=None, autofill=True, excluded=None)

Python class decorators are a bit funny. First, the class of the decorator is instantiated, and is passed all arguments given to the @decorator() invocation. Second, after decorator instantiation, the __call__() method is called with the decorated function object.

Parameters:

• help – the help string for the subcommand created for this function
• largs – list of dicts describing the function’s list arguments
• kwargs – list of dicts describing the function’s keyword arguments

The largs and kwargs parameters are used to create an argparse subcommand parser that parses parameters you want to expose to invocation of the method via CLI/argparse. largs describes your function’s list arguments; and kwargs describes your function’s keyword arguments. largs and kwargs are both lists of dicts, where each dict describes a function argument.

These dictionaries must have at least one key (‘name’); the value of the ‘name’ key must be the actual name of the parameter. If you specify a ‘help’ key/value pair, that is used as the argparse help for for the parameter. Anything else in the dictionary is basically passed to parser.add_argument() as a keyword arg. There are a few exceptions. Typically, the first two arguments to parser.add_argument() are a short option name and a long option name (i.e, '-h','--help'). You can customize those by setting a ‘parser_largs’ key in the dict to something like ['-h','--help'] (then that list will be passed to parser.add_arguments(*args)). If you specify ‘parser_largs’ in this way, it must conform to parser.add_argument()’s requirements. If you do not supply the ‘parser_largs’ key in your parameter dict, we will automatically generate the short/long option names by taking the first character from the parameter name, and by using the entire parameter name, respectively. If the short option name has already been used in the argparser for this function, we will not add a short name option for that argument. Note that for the largs parameter dictionaries, we also add a ‘required’ kwarg when creating the argument parser. Also note that we add a ‘dest’ kwarg when creating the argument parser, since we have to be able to retrieve the parameter value deterministically to pass it to the function. Again, any other keyword in the dictionary is passed directly to parser.add_argument() as a kwarg. This allows the user substantial control over argument processing.

You will want to be familiar with the argparse.ArgumentParser documentation.

For example, if you are decorating the elasticslice.rpc.protogeni.ProtoGeniServer.get_self_credential() instance method:

def get_self_credential(self,force=False,cache_notifier=None)

you would invoke the decorator like:

@ApplicableMethod(
    'ProtoGeniServer',
    help="Get a self credential",
    kwargs=[dict(name='force',help='Bypass the cache',
                 action='store_true',default=False)])

Another example: if you are decorating the elasticslice.rpc.protogeni.ProtoGeniServer.create_sliver() instance method:

def create_sliver(self,rspec,cm=None,slicename=None,dokeys=True,
                  selfcredential=None,gen_random_key=False):

you would invoke the decorator like:

@ApplicableMethod(
    help="Create a sliver at a CM",
    largs=[dict(name='rspec',help='An RSpec as an XML string'),],
    kwargs=[dict(name='slicename',help='Specify a slice URN'),
            dict(name='cm',
                 help='Specify a CM; otherwise the default CM will be used'),
            dict(name='dokeys',help='Install your pubkeys in the sliver',
                 default=True,type=bool),
            dict(name='gen_random_key',action='store_true',default=False,
                 help='Generate an unencrypted RSA key that will be installed on each node.')],
)

searcch.importer.util.applicable.DEFAULT_FORMATTER(result, text=None, json=None)

Format a value as a string or JSON object string.

Parameters:

• text – display value as plaintext
• json – display value as a JSON object

class searcch.importer.util.applicable.DefaultSubcommandArgumentParser(prog=None, usage=None, description=None, epilog=None, parents=[], formatter_class=<class 'argparse.HelpFormatter'>, prefix_chars='-', fromfile_prefix_chars=None, argument_default=None, conflict_handler='error', add_help=True, allow_abbrev=True)

Bases: argparse.ArgumentParser

set_default_subparser(name)

searcch.importer.util.applicable.DocHelpTuple

Attribute params:  a dict of param name to help string Attribute returns:  the help string for the return value, if any Attribute help:the remainder of the function help, sans n r t

alias of searcch.importer.util.applicable.DocTuple

class searcch.importer.util.applicable.InspectedFunction(func)

Bases: object

Conveniently group info extracted from a function’s argspec with information extracted from its docstring.

__init__(func)

Initialize self. See help(type(self)) for accurate signature.

searcch.importer.util.applicable.docparse(doc)

Parses a Sphinxy Python function’s doc string.

Returns:a DocHelpTuple of a parse of a Sphinxy Python function’s doc string.

We only find ‘:param <foo>: …’, ‘:return(s): <blah>’, and the rest is assumed to be generic help text. In the generic help text, we convert n, r, t to spaces; then we convert any sequence of multiple spaces to a single space.

searcch.importer.util.applicable.get_default_formatter()

searcch.importer.util.config module

class searcch.importer.util.config.ConfigSection

Bases: object

classmethod section_defaults()

classmethod section_name()

class searcch.importer.util.config.MyConfigParser(*args, **kwargs)

Bases: configparser.ConfigParser

__init__(*args, **kwargs)

Initialize self. See help(type(self)) for accurate signature.

read_env()

searcch.importer.util.config.add_default_configfile(fp)

searcch.importer.util.config.config_section(cls)

searcch.importer.util.config.find_configfile()

searcch.importer.util.config.get_config_parser(*args, **kwargs)

searcch.importer.util.inspect module

class searcch.importer.util.inspect.FileTypeInspector

Bases: object

This class attempts to return one or more mime types and classes (the kind of file, e.g. plain, compressed, archive, image, etc) that are associated with a file. It returns a list of mime types, from outermost encapsulation to innermost. It first attempts to use python-magic, then falls back to python-filetype.

classmethod inspect(filepath)

Returns a list of mime types, from outermost encapsulation to innermost. Each list item is a tuple (<mime-type>,<description>).

searcch.importer.util.log module

searcch.importer.util.log.configure_logging(level=30)

Module contents

searcch.importer.util.bytes2str(maybe_bytes)

searcch.importer.util.str2bytes(maybe_str)