cdxcore.verbose#

This module contains the cdxcore.verbose.Context manager class which supports printing hierarchical verbose progress reports.

Overview#

The key point of this class is to implement an easy-to-use method to print indented progress which can also be turned off easily without untidy code constructs such as excessive if blocks. In this case, we also avoid formatting any strings.

Here is an example:

from cdxcore.verbose import Context

def f_sub( num=3, context = Context.quiet ):
        context.write("Entering loop")
        for i in range(num):
            context.report(1, "Number %ld", i)

def f_main( context = Context.quiet ):
    context.write( "First step" )
    # ... do something
    context.report( 1, "Intermediate step 1" )
    context.report( 1, "Intermediate step 2\\n with newlines" )
    # ... do something
    f_sub( context=context(2) ) # call function f_sub with a sub-context
    # ... do something
    context.write( "Final step" )

print("Verbose=1")
context = Context(1)
f_main(context)

print("\\nVerbose=2")
context = Context(2)
f_main(context)

print("\\nVerbose='all'")
context = Context('all')
f_main(context)

print("\\nVerbose='quiet'")
context = Context('quiet')
f_main(context)

print("\\ndone")

Returns:

Verbose=1
00: First step
01:   Intermediate step 1
01:   Intermediate step 2
01:    with newlines
00: Final step

Verbose=2
00: First step
01:   Intermediate step 1
01:   Intermediate step 2
01:    with newlines
02:     Entering loop
00: Final step

Verbose='all'
00: First step
01:   Intermediate step 1
01:   Intermediate step 2
01:    with newlines
02:     Entering loop
03:       Number 0
03:       Number 1
03:       Number 2
00: Final step

Verbose='quiet'

done

Workflow#

The basic idea is that the root context has level 0, with increasing levels for sub-contexts. When printing information, we can limit printing up to a given level and automatically indent the output to reflect the current level of detail.

Workflow:

Lazy Formatting#

cdxcore.verbose.Context message formattting is meant to be lazy and only executed if a message is actually written. This means that if the Contex is "quiet" no string formatting takes place.

Consider a naive example:

from cdxcore.verbose import Context
import numpy as np

def f( data : np.ndarray, verbose : Context = Context.quiet ):
    verbose.write(f"'f' called; data has mean {np.mean(data)} and variance {np.var(data)}")
    # ...
f( verbose = Context.quiet )

In this case f will compute np.mean(data) and np.var(data) even though the use of the quiet Context means that the formatted string will not be printed.

To alleviate this, cdxcore.verbose.Context.write() supports a number of alternatives which are leveraging cdxcore.err.fmt(). In above example, the most efficient use case is the use of a lambda function:

def f( data : np.ndarray,  verbose : Context ):
    verbose.write(lambda : f"'f' called; data has mean {np.mean(data)} and variance {np.var(data)}")

The lambda function is only called when the message is about to be printed.

Providing Updates#

In many applications we wish to provide progress updates in a single line, and not clutter the output. In the example from the beginng, the long lists of output are not informative.

cdxcore.verbose.Context supports the use of “\r” and “\n for simple output formatting. Under the hood it uses cdxcore.crman.CRMan.

Consider the following change to f_sub in above code example:

 def f_sub( num=3, context = Context.quiet ):
     context.write("Entering loop")
     for i in range(num):
         context.report(1, ":emphasis:`\\r`Number %ld", i, end='')   # Notice use of \\r and end=''
     context.write("\\rLoop done")                       # Notice use of \\r `

 context = Context('all')
 f_main(context)

During execution this prints, for example at step i==1:

00: First step
01:   Intermediate step 1
01:   Intermediate step 2
01:    with newlines
02:     Entering loop
03:       Number 1

But once the loop finished the update per i is overwitten:

00: First step
01:   Intermediate step 1
01:    with newlines
02:     Entering loop
02:     Loop done 
00: Final step

Composing Line Output and Timing#

For lengthy operations it is often considerate to provide the user with an update on how long an operation takes. cdxcore.verbose.Context provides some simple tooling:

from cdxcore.verbose import Context
import time as time

def takes_long( n : int, verbose : context = Context.quiet ):    
    with verbose.write_t("About to start... ", end='') as tme:  
        for t in range(n):
            verbose.write(lambda : f"\\rTakes long {int(100.*(t+1)/n)}%... ", end='') 
            time.sleep(0.22)
        verbose.write(lambda : f"done; this took {tme}.", head=False)

takes_long(5, Context.all)

During execution prints

00: Takes long 80%...

The example finishes with

00: Takes long 100%... done; this took 1.1s.

Import#

from cdxcore.verbose import Context

Documentation#

Classes

Context([init, indent, fmt_level, level, ...])

Class for printing indented messages, filtered by overall level of visibility.
class cdxcore.verbose.Context(init=None, *, indent=2, fmt_level='%02ld: ', level=None, channel=None)[source]#

Bases: object

Class for printing indented messages, filtered by overall level of visibility.

  • Construction with keywords:

    Context( "all" )` or
Context( "quiet" )

  • Display everything:

    Context( None )

  • Display only up to level 2 (top level is 0) e.g.:

    Context( 2 )

  • Copy constructor:

    Context( context )

Example:

from cdxcore.verbose import Context

def f_2( verbose : Context = Context.quiet ):
    verbose.write( "Running 'f_2'")
    for i in range(5):
        verbose.report(1, "Sub-task {i}", i=i)
        # do something

def f_1( verbose : Context = Context.quiet ):
    verbose.write( "Running 'f_1'")
    f_2( verbose(1) )
    # do something

verbose = Context("all")
verbose.write("Starting:")
f_1(verbose(1))   
verbose.write("Done.")

prints

00: Starting:
01:   Running 'f_1'
02:     Running 'f_2'
03:       Sub-task 0
03:       Sub-task 1
03:       Sub-task 2
03:       Sub-task 3
03:       Sub-task 4
00: Done.

If we set visibility to 2

verbose = Context(2)
verbose.write("Starting:")
f_1(verbose(1))   # <-- make it a level higher
verbose.write("Done.")

we get the reduced

00: Starting:
01:   Running 'f_1'
02:     Running 'f_2'
00: Done.

Lazy Formatting

The cdxcore.verbose.Context.write() and cdxcore.verbose.Context.report() functions provide string formatting capabilities. If used, then a message will only be formatted if the current level grants it visibility. This avoids unnecessary string operations when no output is required.

In the second example above, the format string verbose.report(1, "Sub-task {i}", i=i) in f_2 will not be evaluated as that reporting level is turned off.

Parameters:
initstr | int | cdxcore.verbose.Context

  • If a string is provided: must be "all" or "quiet".

  • If an integer is privided it represents the visibility level up to which to print. Set to 0 to print only top level messages. Any negative number will turn off any messages and is equivalent to "quiet".

  • If set to None display everything.

  • A Context is copied.

indentint, optional

How much to indent strings per level. Default 2.

fmt_levelstr, optional

A format string containing %d for the current indentation. Default is "%02ld: ".

levelint, optional

Current level. If init is another context, and level is specified, it overwrites the level from the other context.

If level is None:

  • If init is another Context object, use that object’s level.

  • If init is an integer or one of the keywords above, use the default, 0.

channelCallable, optional

Advanced parameter.

A callable which is called to print text. The call signature is:

channel( msg : str, flush : bool )`

which is meant to mirror print( msg, end='', flush ) for the provided channel. In particular do not terminate msg automatically with a new line.

Illustration:

class Collector:
    def __init__(self):
        self.messages = []
    def __call__(self, msg, flush ):
        self.messages.append( msg )

collect  = Collector()
verbose  = Context( channel = collect )

verbose.write("Write at 0")
verbose.report(1,"Report at 1")

print(collect.messages)

prints

['00: Write at 0\\n', '01:   Report at 1\\n']
ALL = 'all'#

Constant for the keyword "all"

QUIET = 'quiet'#

Constant for the keyword "quiet"

__call__(add_level=1, message=None, end='\\n', head=True, *args, **kwargs)[source]#

Create and return a sub Context at current level plus add_level.

If a message is provided, cdxcore.verbose.Context.write() is called before the new Context is created.

Example:

from cdxcore.verbose import Context
def f( verbose : Context = Context.quiet ):
    # ...
    verbose.write("'f'' usuing a sub-context.")
verbose = Context.all
verbose.write("Main")
f( verbose=verbose(1) )   # create sub-context

prints

00: Main
01:   'f'' usuing a sub-context.
Parameters:
add_levelint

Level to add to the current level. Set to 0 for the same level.

messagestr | Callable, optional.

Text containing format characters, or None to not print a message.

The following alternatives are suppoted:

  • Python 3 `{parameter:d}`, in which case message.fmt(kwargs) for str.format() is used to obtain the output message.

  • Python 2 `%(parameter)d` in which case message % kwargs is used to obtain the output message.

  • Classic C-stype `%d, %s, %f` in which case message % args is used to obtain the output message.

  • If message is a Callable such as a lambda function, then message( *args, **kwargs ) is called to obtain the output message.

    Note that a common use case is using an f-string wrapped in a lambda function. In this case you do not need args or kwargs:

    x = 1
verbose.write(lambda : f"Delayed f-string formatting {x}")
headbool, optional;

Whether this message needs a header (i.e. the 01 and spacing). Typically False if the previous call to write() used end=’’. See examples above.

*args, **kwargs:

See above

Returns:
verboseContext

Sub context with new level equal to current level plus add_level.

all = <cdxcore.verbose.Context object>#
apply_channel(channel)[source]#

Advanced Use

Returns a new `Context object with the same currrent state as self, but pointing to channel.

property as_quiet: Context#

Return a Context at the same current reporting level as self with zero visibility

property as_verbose: Context#

Return a Context at the same current reporting level as self with full visibility

fmt(level, message, *args, head=True, **kwargs)[source]#

Formats message with the formattting arguments at curent context level plus level.

This function returns None if current level plus level is not visible. In that case no string formatting takes place.

Parameters:
levelint

Level to add to current level.

messagestr | Callable

Text containing format characters.

Supported formats:

  • Python 3 {parameter:d} (uses message.format(*args, **kwargs); see str.format()).

  • Python 2 %(parameter)d (uses message % kwargs).

  • Classic C-style %d, %s, %f (uses message % args).

  • If message is a callable (e.g. a lambda), then message(*args, **kwargs) is called.

A common use case is an f-string wrapped in a lambda to delay formatting. In this case you do not need args or kwargs:

x = 1
verbose.write(lambda: f"Delayed f-string formatting {x}")
headbool, optional;

Whether this message needs a header (i.e. the 01 and spacing). Typically False if the previous call to write() used end=''. See examples above.

*args, **kwargs:

See above

Returns:
Stringstr

Formatted string, or None if the current level plus level is not visible.

property is_quiet: bool#

Whether the current context is "quiet"

quiet = <cdxcore.verbose.Context object>#
report(level, message, *args, end='\\n', head=True, **kwargs)[source]#

Report message at current level plus level.

The message will be formatted using cdxcore.err.fmt() is the current level plus level is visible.

The parameter end matches end in print() e.g. end='' avoids a newline at the end of the message.

  • If head is True, then the first line of the text will be preceeded by proper indentation.

  • If head is False, the first line will be printed without preamble.

This means the following is a valid pattern:

from cdxcore.verbose import Context
verbose = Context()
verbose.report(1, "Doing something... ", end='')
# ... do something
verbose.report(1, "done.", head=False)

which prints

01: Doing something... done.

Another use case is updates per line, for example::

from cdxcore.verbose import Context
verbose = Context()
N  = 1000
for i in range(N):                
    verbose.report(1,f"\\rStatus {int(float(i+1)/float(N)*100)}%... ", end='')
    # do something
verbose.report(1,"done.", head=False)

will provide progress information in the current line as the loop is processed.

Implementation notice: The use of \\r is managed using cdxcore.crman.CRMan.

Parameters:
levelint

Level to add to current level.

messagestr | Callable

Text containing format characters.

The following alternatives are suppoted:

  • Python 3 `{parameter:d}`, in which case message.fmt(kwargs) for str.format() is used to obtain the output message.

  • Python 2 `%(parameter)d` in which case message % kwargs is used to obtain the output message.

  • Classic C-stype `%d, %s, %f` in which case message % args is used to obtain the output message.

  • If message is a Callable such as a lambda function, then message( *args, **kwargs ) is called to obtain the output message.

    Note that a common use case is using an f-string wrapped in a lambda function. In this case you do not need args or kwargs:

    x = 1
verbose.write(lambda : f"Delayed f-string formatting {x}")
endstr, optional

Terminating string akin to end in print(). Use '' to not print a newline. See example above for a use case.

headbool, optional;

Whether this message needs a header (i.e. the 01 and spacing). Typically False if the previous call to write() used end=’’. See examples above.

*args, **kwargs:

See above

shall_report(add_level=0)[source]#

Returns whether to print something at current level plus add_level.

str_indent(add_level=0)[source]#

Returns the string indentation for the current level plus add_level

timer()[source]#

Returns a new cdxcore.util.Timer object to measure time spent in a block of code.

Example:

import time as time
from cdxcore.verbose import Context

verbose = Context("all")
with verbose.Timer() as tme:
    verbose.write("Starting job... ", end='')
    time.sleep(1)
    verbose.write(f"done; this took {tme}.", head=False)

produces

00: Starting job... done; this took 1s.
write(message, *args, end='\\n', head=True, **kwargs)[source]#

Report message at current level.

The message will be formatted using cdxcore.err.fmt() if the current level is visible. If the current level is not visible no message formatting will take place.

The parameter end matches end in print() e.g. end='' avoids a newline at the end of the message.

  • If head is True, then the first line of the text will be preceeded by proper indentation.

  • If head is False, the first line will be printed without preamble.

This means the following is a valid pattern:

from cdxcore.verbose import Context
verbose = Context()
verbose.write("Doing something... ", end='')
# ... do something
verbose.write("done.", head=False)

which prints

00: Doing something... done.

Another use case is updates per line, for example:

from cdxcore.verbose import Context
verbose = Context()
N  = 1000
for i in range(N):                
    verbose.write(f"\\rDoing something {int(float(i+1)/float(N)*100)}%... ", end='')
    # do something
verbose.write("done.", head=False)

which will provide progress information in a given line.

Implementation notice: the use of \r is managed using cdxcore.crman.CRMan.

Parameters:
messagestr | Callable

Text containing format characters.

The following alternatives are suppoted:

  • Python 3 `{parameter:d}`, in which case message.fmt(kwargs) for str.format() is used to obtain the output message.

  • Python 2 `%(parameter)d` in which case message % kwargs is used to obtain the output message.

  • Classic C-stype `%d, %s, %f` in which case message % args is used to obtain the output message.

  • If message is a Callable such as a lambda function, then message( *args, **kwargs ) is called to obtain the output message.

    Note that a common use case is using an f-string wrapped in a lambda function. In this case you do not need args or kwargs:

    x = 1
verbose.write(lambda : f"Delayed f-string formatting {x}")
endstr, optional

Terminating string akin to end in print(). Use '' to not print a newline. See example above for a use case.

headbool, optional;

Whether this message needs a header (i.e. the 01 and spacing). Typically False if the previous call to write() used end=’’. See examples above.

*args, **kwargs:

See above

write_t(message, *args, end='\\n', head=True, **kwargs)[source]#

Reports message subject to string formatting at current level if visible and returns a cdxcore.util.Timer object which can be used to measure time elapsed since write_t() was called:

from cdxcore.verbose import Context
verbose = Context()
with verbose.write_t("Doing something... ", end='') as tme:
    # do something
    verbose.write("done; this took {tme}.", head=False)

produces

00: Doing something... done; this took 1s.

Equivalent to using cdxcore.verbose.Context.write() first followed by cdxcore.verbose.Context.timer().