eidolon.Concurrency Namespace Reference

Classes
class	AlgorithmProcess
class	DynamicProxy
class	MethodProxy
class	ObjectServer
class	ObjectSharer
class	ProcessServer

Functions
def	chooseProcCount (numelems, refine, threshold)
def	checkResultMap (result)
def	sumResultMap (result)
def	listResults (result)
def	concurrent (func)
def	concurrentFuncExec (process, mcode, cglobals, args, kwargs)
def	concurrentExec (process, execcode, clocals={}, cglobals={}, returnName=None)
def	concurrencyTestRange (process, values)
	Routines used by unit tests, these have to be here to be defined in the module namespace. More...
def	concurrencyTestProcessValues (process)
def	concurrencyTestReturnArg (process, values)
def	concurrencyTestShareObjects (process)

Function Documentation

checkResultMap()

def eidolon.Concurrency.checkResultMap

(

result

)

Checks the results from a concurrent operation, throwing the first returned exception.

Here is the caller graph for this function:

chooseProcCount()

def eidolon.Concurrency.chooseProcCount	(		numelems,
			refine,
			threshold
	)

Determine a process count to use when running concurrent routines. If numelems*(1+refine)>=threshold then 0 is
returned to indicate the number of processes used will be however many processes exist (usually the number of
physical cores). A value of 1 is returned otherwise to clue in the system to compute sequentially in the calling
process. If `numelems' is less than the number of processes used, only that many will actually be used.

Here is the caller graph for this function:

concurrencyTestProcessValues()

def eidolon.Concurrency.concurrencyTestProcessValues

(

process

)

Returns the index, PID, startval, and endval for the given `process' object.

concurrencyTestRange()

def eidolon.Concurrency.concurrencyTestRange	(		process,
			values
	)

Routines used by unit tests, these have to be here to be defined in the module namespace.

For each range index value, appends the value from `values' to the result. At every process.total index, prints each 
indexed value in `values' to stdout then syncs with other processes. Returns the process' index range of `values'.

Here is the call graph for this function:

concurrencyTestReturnArg()

def eidolon.Concurrency.concurrencyTestReturnArg	(		process,
			values
	)

Returns the `process' index and `values'.

concurrencyTestShareObjects()

def eidolon.Concurrency.concurrencyTestShareObjects

(

process

)

Test sharing objects bween processes using ShareObject().

Here is the call graph for this function:

concurrent()

def eidolon.Concurrency.concurrent

(

func

)

Replaces `func' with a wrapper which will call `func' in parallel using processes. The first argument of `func'
must be the AlgorithmProcess instance corresponding to the subprocess it is being called in. When calling the
decorated form of `func' the first three arguments provided must be the `valrange', `numprocs', and `task' values
expected by ProcessServer.callProcessFunc(), followed by the arguments normally passed to `func'.

Applying this decorator to a function whose code can be marshalled into a code blob using marshal will produce a
wrapper which executes that blob in subprocesses using concurrentFuncExec(). This in general allows code defined
in script files or in interpreter shells to be executabled concurrently. When the wrapped function is called the 
method 'ProcessServer.globalServer.callProcessFunc' is called to execute the marshalled function.

Applying this decorator to a function which cannot be marshalled, ie. builtin or compiled Cython function,
creates a global variable with the name '__local__'+func.__name__ referencing a lambda function which calls `func' 
when evaluated. This is passed to 'ProcessServer.globalServer.callProcessFunc' when the call is made and is needed 
for picklability. Do not call this created function directly. This is necessary since pickling a builtin function 
only wraps up its name in the output which is looked up in the global namespace when unpickled.

This decorator can only be applied to a builtin function declared in a module which is loaded by each process when 
the app starts up. Since functions are looked up in the global namespace, any functions declared after module import 
time are not present in the processes global namespace and thus can only be called if marshallable. 

Example:
    @concurrent 
    def testfunc(process,values):
        return (process.index,values)
    
    values=list(range(10))
    result=testfunc(len(values),3,None,values,partitionArgs=(values,))
    printFlush(listResults(result))
    
Output: [(0, [0, 1, 2]), (1, [3, 4, 5]), (2, [6, 7, 8, 9])]

concurrentExec()

def eidolon.Concurrency.concurrentExec	(		process,
			execcode,
			clocals = {},
			cglobals = {},
			returnName = None
	)

Execute the code string `execcode' in separate processes using the `clocals' and `cglobals' environment dictionaries.
If `returnName' is given this is queried first from `clocals', then `cglobals' if not found, and returned.

concurrentFuncExec()

def eidolon.Concurrency.concurrentFuncExec	(		process,
			mcode,
			cglobals,
			args,
			kwargs
	)

Execute the marshalled code segment `mcode' by converting it to a function using types.FunctionType. The `cglobals'
dictionary and globals() initialize the global variables supplied to the function, which is called with arguments
`args' and `kwargs'. This function is meant to be called indirectly through concurrent() in subprocesses.

listResults()

def eidolon.Concurrency.listResults

(

result

)

Returns a list of the results from the given result map in process order.

sumResultMap()

def eidolon.Concurrency.sumResultMap

(

result

)

Given a result dict mapping process indices to lists, returns the summed lists in order.

Here is the call graph for this function:

Here is the caller graph for this function: