Lib Copy Section

Home
<a id="jc.lib"></a>

jc.lib

Table of Contents Copy Chapter

* jc.lib
* all_parser_info
* get_help
* get_parser
* parse
* parser_info
* parser_mod_list
* plugin_parser_mod_list
* slurpable_parser_mod_list
* standard_parser_mod_list
* streaming_parser_mod_list

jc - JSON Convert lib module

<a id="jc.lib.all_parser_info"></a>

all_parser_info Copy Section

def all_parser_info(
        documentation: bool = False,
        show_hidden: bool = False,
        show_deprecated: bool = False) -> List[jc.jc_types.ParserInfoType]

Returns a list of dictionaries that includes metadata for all parser
modules. By default only non-hidden, non-deprecated parsers are
returned.

Parameters:

documentation: (boolean) include parser docstrings if True
show_hidden: (boolean) also show parsers marked as hidden
in their info metadata.
show_deprecated: (boolean) also show parsers marked as
deprecated in their info metadata.

<a id="jc.lib.get_help"></a>

get_help Copy Section

def get_help(parser_mod_name: Union[str, module]) -> None

Show help screen for the selected parser.

This function will accept module_name, cli-name, and
--argument-name variants of the module name string as well as a
parser module object.

<a id="jc.lib.get_parser"></a>

get_parser Copy Section

def get_parser(parser_mod_name: Union[str, module]) -> module

Return the parser module object and check that the module is a valid
parser module.

Parameters:

parser_mod_name: (string or Name of the parser module. This
Module) function will accept module_name,
cli-name, and --argument-name
variants of the module name.

If a Module is given and the Module
is a valid parser Module, then the
same Module is returned.

Returns:

Module: the parser Module object

Raises:

ModuleNotFoundError: If the Module is not found or is not a valid
parser Module, then a ModuleNotFoundError
exception is raised.

<a id="jc.lib.parse"></a>

parse Copy Section

def parse(
    parser_mod_name: Union[str, module],
    data: Union[str, bytes, Iterable[str]],
    quiet: bool = False,
    raw: bool = False,
    ignore_exceptions: Optional[bool] = None,
    kwargs
) -> Union[Dict[str, Any], List[Dict[str, Any]], Iterator[Dict[str, Any]]]

Parse the data (string or bytes) using the supplied parser (string or
module object).

This function provides a high-level API to simplify parser use. This
function will call built-in parsers and custom plugin parsers.

Example (standard parsers):

>>> import jc
>>> date_obj = jc.parse('date', 'Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022

Example (streaming parsers):

>>> import jc
>>> ping_gen = jc.parse('ping_s', ping_output.splitlines())
>>> for item in ping_gen:
>>> print(f'Response time: {item["time_ms"]} ms')
Response time: 102 ms
Response time: 109 ms
...

To get a list of available parser module names, use parser_mod_list().

Alternatively, a parser module object can be supplied:

>>> import jc
>>> jc_date = jc.get_parser('date')
>>> date_obj = jc.parse(jc_date, 'Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022

You can also use the parser modules directly via get_parser():

>>> import jc
>>> jc_date = jc.get_parser('date')
>>> date_obj = jc_date.parse('Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022

Finally, you can access the low-level parser modules manually:

>>> import jc.parsers.date
>>> date_obj = jc.parsers.date.parse('Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022

Though, accessing plugin parsers directly is a bit more cumbersome, so
this higher-level API is recommended. Here is how you can access plugin
parsers without this API:

>>> import os
>>> import sys
>>> import jc.appdirs
>>> data_dir = jc.appdirs.user_data_dir('jc', 'jc')
>>> local_parsers_dir = os.path.join(data_dir, 'jcparsers')
>>> sys.path.append(local_parsers_dir)
>>> import my_custom_parser
>>> my_custom_parser.parse('command_data')

Parameters:

parser_mod_name: (string or name of the parser module. This
Module) function will accept module_name,
cli-name, and --argument-name
variants of the module name.

A Module object can also be passed
directly or via get_parser()

data: (string or data to parse (string or bytes for
bytes or standard parsers, iterable of
iterable) strings for streaming parsers)

raw: (boolean) output preprocessed JSON if True

quiet: (boolean) suppress warning messages if True

ignore_exceptions: (boolean) ignore parsing exceptions if True
(streaming parsers only)

Returns:

Standard Parsers: Dictionary or List of Dictionaries
Streaming Parsers: Generator Object containing Dictionaries

<a id="jc.lib.parser_info"></a>

parser_info Copy Section

def parser_info(parser_mod_name: Union[str, module],
                documentation: bool = False) -> jc.jc_types.ParserInfoType

Returns a dictionary that includes the parser module metadata.

Parameters:

parser_mod_name: (string or name of the parser module. This
Module) function will accept module_name,
cli-name, and --argument-name
variants of the module name as well
as a parser module object.

documentation: (boolean) include parser docstring if True

<a id="jc.lib.parser_mod_list"></a>

parser_mod_list Copy Section

def parser_mod_list(show_hidden: bool = False,
                    show_deprecated: bool = False) -> List[str]

Returns a list of all available parser module names.

<a id="jc.lib.plugin_parser_mod_list"></a>

plugin_parser_mod_list Copy Section

def plugin_parser_mod_list(show_hidden: bool = False,
                           show_deprecated: bool = False) -> List[str]

Returns a list of plugin parser module names. This function is a
subset of parser_mod_list().

<a id="jc.lib.slurpable_parser_mod_list"></a>

slurpable_parser_mod_list Copy Section

def slurpable_parser_mod_list(show_hidden: bool = False,
                              show_deprecated: bool = False) -> List[str]

Returns a list of slurpable parser module names. This function is a
subset of parser_mod_list().

<a id="jc.lib.standard_parser_mod_list"></a>

standard_parser_mod_list Copy Section

def standard_parser_mod_list(show_hidden: bool = False,
                             show_deprecated: bool = False) -> List[str]

Returns a list of standard parser module names. This function is a
subset of parser_mod_list() and does not contain any streaming
parsers.

<a id="jc.lib.streaming_parser_mod_list"></a>

streaming_parser_mod_list Copy Section

def streaming_parser_mod_list(show_hidden: bool = False,
                              show_deprecated: bool = False) -> List[str]

Returns a list of streaming parser module names. This function is a
subset of parser_mod_list().

---

Readme Copy Section

Home
<a id="jc"></a>

jc

JC - JSON Convert

* [email protected]

This package converts the output of many standard unix command line tools
and file-types to dictionaries and lists of dictionaries.

Interactive Documentation Copy Chapter

Using jc in your python programs:

>>> help('jc')
>>> help('jc.lib')
>>> jc.get_help('parser_module_name')

Developing jc parsers:

>>> help('jc.utils')
>>> help('jc.streaming')
>>> help('jc.parsers.universal')

Online Documentation Copy Chapter

Latest Copy Section

https://github.com/kellyjonbrazil/jc/tree/master/docs

Specific Version Copy Section

https://github.com/kellyjonbrazil/jc/tree/v<full_version_number>/docs

Replace <full_version_number> - e.g. 1.18.0:

Specific versions can also be selected by tag in the Github branch dropdown
menu.

Usage Example Copy Chapter

>>> import subprocess
>>> import jc
>>>
>>> cmd_output = subprocess.check_output(['dig', 'example.com'],
text=True)
>>> data = jc.parse('dig', cmd_output)
>>> data
[{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', ...}]

Alternatively, you can bypass the high-level API and call the parser
modules directly:

>>> import subprocess
>>> import jc
>>>
>>> jc_dig = jc.get_parser('dig')
>>> cmd_output = subprocess.check_output(['dig', 'example.com'],
text=True)
>>> data = jc_dig.parse(cmd_output)
>>> data
[{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', ...}]

or

>>> import subprocess
>>> import jc.parsers.dig
>>>
>>> cmd_output = subprocess.check_output(['dig', 'example.com'],
text=True)
>>> data = jc.parsers.dig.parse(cmd_output)
>>> data
[{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', ...}]

Available Functions Copy Chapter

Use help(jc.lib) for details.

parse Copy Section

parse(
parser_module_name: str,
data: str | bytes | Iterable
) -> dict | list[dict] | Iterable[dict]

High-level API to easily access the parser. This API will find both
built-in parsers and local plugin parsers.

get_parser Copy Section

get_parser(
parser_module_name: str
) -> ModuleType

Get a parser Module object so you can use it directly.

parser_info Copy Section

parser_info(
parser_module_name: str,
documentation: bool = False
) -> dict

Get the metadata for a particular parser.

all_parser_info Copy Section

all_parser_info(documentation: bool = False) -> list[dict]

Get the metadata for all parsers.

get_help Copy Section

get_help(parser_module_name: str) -> None

Convenience function to display the help screen for a parser using
its module name.

parser_mod_list Copy Section

parser_mod_list() -> list[str]

Get a list of all available parser module names to be used in
parse(), parser_info(), and get_help().

plugin_parser_mod_list Copy Section

plugin_parser_mod_list() -> list[str]

Get a list of plugin parser module names to be used in
parse(), parser_info(), and get_help(). This list is a subset of
parser_mod_list().

standard_parser_mod_list Copy Section

standard_parser_mod_list() -> list[str]

Get a list of standard parser module names to be used in
parse(), parser_info(), and get_help(). This list is a subset of
parser_mod_list() and does not contain any streaming parsers.

streaming_parser_mod_list Copy Section

streaming_parser_mod_list() -> list[str]

Get a list of streaming parser module names to be used in
parse(), parser_info(), and get_help(). This list is a subset of
parser_mod_list().

---

Streaming Copy Section

Home
<a id="jc.streaming"></a>

jc.streaming

Table of Contents Copy Chapter

* jc.streaming
* add_jc_meta
* raise_or_yield
* stream_error
* stream_success
* streaming_input_type_check
* streaming_line_input_type_check

jc - JSON Convert streaming utils

<a id="jc.streaming.add_jc_meta"></a>

add_jc_meta Copy Section

def add_jc_meta(func: ~F) -> ~F

Decorator for streaming parsers to add stream_success and
stream_error objects. This simplifies the yield lines in the
streaming parsers.

With the decorator on parse():

# successfully parsed line:
yield output_line if raw else _process(output_line)

# unsuccessfully parsed line:
except Exception as e:
yield raise_or_yield(ignore_exceptions, e, line)

Without the decorator on parse():

# successfully parsed line:
if raw:
yield stream_success(output_line, ignore_exceptions)
else:
stream_success(_process(output_line), ignore_exceptions)

# unsuccessfully parsed line:
except Exception as e:
yield stream_error(raise_or_yield(ignore_exceptions, e, line))

In all cases above:

output_line: (Dict) successfully parsed line yielded as a dict

e: (BaseException) exception object as the first value
of the tuple if the line was not successfully parsed.

line: (str) string of the original line that did not
successfully parse.

ignore_exceptions: (bool) continue processing lines and ignore
exceptions if True.

<a id="jc.streaming.raise_or_yield"></a>

raise_or_yield Copy Section

def raise_or_yield(ignore_exceptions: bool, e: BaseException,
                   line: str) -> Tuple[BaseException, str]

Return the exception object and line string if ignore_exceptions is
True. Otherwise, re-raise the exception from the exception object with
an annotation.

<a id="jc.streaming.stream_error"></a>

stream_error Copy Section

def stream_error(e: BaseException, line: str) -> Dict[str, Any]

Return an error _jc_meta field.

<a id="jc.streaming.stream_success"></a>

stream_success Copy Section

def stream_success(output_line: Dict[str, Any],
                   ignore_exceptions: bool) -> Dict[str, Any]

Add _jc_meta object to output line if ignore_exceptions=True

<a id="jc.streaming.streaming_input_type_check"></a>

streaming_input_type_check Copy Section

def streaming_input_type_check(data: Iterable[Union[str, bytes]]) -> None

Ensure input data is an iterable, but not a string or bytes. Raises
TypeError if not.

<a id="jc.streaming.streaming_line_input_type_check"></a>

streaming_line_input_type_check Copy Section

def streaming_line_input_type_check(line: str) -> None

Ensure each line is a string. Raises TypeError if not.

---

Utils Copy Section

Home
<a id="jc.utils"></a>

jc.utils

Table of Contents Copy Chapter

* jc.utils
* compatibility
* convert_size_to_int
* convert_to_bool
* convert_to_float
* convert_to_int
* error_message
* has_data
* input_type_check
* is_compatible
* line_slice
* normalize_key
* remove_quotes
* warning_message

jc - JSON Convert utils

<a id="jc.utils.compatibility"></a>

compatibility Copy Section

def compatibility(mod_name: str,
                  compatible: List[str],
                  quiet: bool = False) -> None

Checks for the parser's compatibility with the running OS platform and
prints a warning message to STDERR if not compatible and
quiet=False.

Parameters:

mod_name: (string) __name__ of the calling module

compatible: (list) sys.platform name(s) compatible with
the parser. compatible options:
linux, darwin, cygwin, win32, aix, freebsd

quiet: (bool) suppress compatibility message if True

Returns:

None - just prints output to STDERR

<a id="jc.utils.convert_size_to_int"></a>

convert_size_to_int Copy Section

def convert_size_to_int(size: str,
                        binary: bool = False,
                        posix_mode: bool = False,
                        decimal_bias: bool = False) -> Optional[int]

Parse a human readable data size and return the number of bytes.

Parameters:

size: (string) The human readable file size to parse.
binary: (boolean) True to use binary multiples of bytes
(base-2) for ambiguous unit symbols and names,
False to use decimal multiples of bytes (base-10).
posix_mode: (boolean) Treat one-letter units (k, m, g, etc.) as
binary.
decimal_bias: (boolean) True to treat slightly ambiguous two-
letter unit symbols ending in "i" (e.g. Ki, Gi) to
use decimal multiples of bytes (base-10). False
(default) to use binary multiples of bytes.
Returns:

integer/None Integer if successful conversion, otherwise None

This function knows how to parse sizes in bytes, kilobytes, megabytes,
gigabytes, terabytes and petabytes. Some examples:

>>> convert_size_to_int('42')
42
>>> convert_size_to_int('13b')
13
>>> convert_size_to_int('5 bytes')
5
>>> convert_size_to_int('1 KB')
1000
>>> convert_size_to_int('1 kilobyte')
1000
>>> convert_size_to_int('1 KiB')
1024
>>> convert_size_to_int('1 Ki')
1024
>>> convert_size_to_int('1 Ki', decimal_bias=True)
1000
>>> convert_size_to_int('1 KB', binary=True)
1024
>>> convert_size_to_int('1.5 GB')
1500000000
>>> convert_size_to_int('1.5 GB', binary=True)
1610612736

<a id="jc.utils.convert_to_bool"></a>

convert_to_bool Copy Section

def convert_to_bool(value: object) -> bool

Converts string, integer, or float input to boolean by checking
for 'truthy' values.

Parameters:

value: (string/integer/float) Input value

Returns:

True/False False unless a 'truthy' number or string is found
('y', 'yes', 'true', '1', 1, -1, etc.)

<a id="jc.utils.convert_to_float"></a>

convert_to_float Copy Section

def convert_to_float(value: object) -> Optional[float]

Converts string and int input to float. Strips all non-numeric
characters from strings.

Parameters:

value: (string/integer) Input value

Returns:

float/None Float if successful conversion, otherwise None

<a id="jc.utils.convert_to_int"></a>

convert_to_int Copy Section

def convert_to_int(value: object) -> Optional[int]

Converts string and float input to int. Strips all non-numeric
characters from strings.

Parameters:

value: (string/float) Input value

Returns:

integer/None Integer if successful conversion, otherwise None

<a id="jc.utils.error_message"></a>

error_message Copy Section

def error_message(message_lines: List[str]) -> None

Prints an error message to STDERR for fatal issues. The first line is
prepended with 'jc: Error - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.

Parameters:

message: (list) list of string lines

Returns:

None - just prints output to STDERR

<a id="jc.utils.has_data"></a>

has_data Copy Section

def has_data(data: Union[str, bytes]) -> bool

Checks if the string input contains data. If there are any
non-whitespace characters then return True, else return False.

For bytes, returns True if there is any data.

Parameters:

data: (string, bytes) input to check whether it contains data

Returns:

Boolean True if input string (data) contains non-whitespace
characters, otherwise False. For bytes data, returns
True if there is any data, otherwise False.

<a id="jc.utils.input_type_check"></a>

input_type_check Copy Section

def input_type_check(data: object) -> None

Ensure input data is a string. Raises TypeError if not.

<a id="jc.utils.is_compatible"></a>

is_compatible Copy Section

def is_compatible(compatible: List[str]) -> bool

Returns True if the parser is compatible with the running OS platform.

<a id="jc.utils.line_slice"></a>

line_slice Copy Section

def line_slice(
    data: Union[str, Iterable[str], TextIO, bytes, NoneType],
    slice_start: Optional[int] = None,
    slice_end: Optional[int] = None
) -> Union[str, Iterable[str], TextIO, bytes, NoneType]

Slice input data by lines - lazily, if possible.

Accepts a string (for normal parsers) or an iterable (for streaming
parsers). Uses normal start/stop slicing values, but will always slice
on lines instead of characters. Positive slices will use less memory as
the function will attempt to lazily iterate over the input. A negative
slice parameter will force the function to read in all of the data and
then slice, which will use more memory.

Parameters:

data: (string or iterable) - input to slice by lines
slice_start: (int) - starting line
slice_end: (int) - ending line

Returns:
string if input is a string.
iterable of strings if input is an iterable (for streaming parsers)

<a id="jc.utils.normalize_key"></a>

normalize_key Copy Section

def normalize_key(data: str) -> str

Normalize a key name by shifting to lower-case and converting special
characters to underscores.

Special characters are defined as space and the following:

!"#$%&'()*+,-./:;<=>?@[\]^{|}~

This is a lossy algorithm. Repeating and trailing underscores are
removed.

Parameters:

data: (string) Input value

Returns:

string

<a id="jc.utils.remove_quotes"></a>

remove_quotes Copy Section

def remove_quotes(data: str) -> str

Remove single or double quotes surrounding a string. If no quotes are
found then the string is returned unmodified.

Parameters:

data: (string) Input value

Returns:

string

<a id="jc.utils.warning_message"></a>

warning_message Copy Section

def warning_message(message_lines: List[str]) -> None

Prints warning message to STDERR` for non-fatal issues. The first line
is prepended with 'jc: Warning - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.

Parameters:

message: (list) list of string lines

Returns:

None - just prints output to STDERR

---