### Lib
[Home](https://kellyjonbrazil.github.io/jc/)
# jc.lib
## Table of Contents
* [jc.lib](#jc.lib)
* [all_parser_info](#jc.lib.all_parser_info)
* [get_help](#jc.lib.get_help)
* [get_parser](#jc.lib.get_parser)
* [parse](#jc.lib.parse)
* [parser_info](#jc.lib.parser_info)
* [parser_mod_list](#jc.lib.parser_mod_list)
* [plugin_parser_mod_list](#jc.lib.plugin_parser_mod_list)
* [slurpable_parser_mod_list](#jc.lib.slurpable_parser_mod_list)
* [standard_parser_mod_list](#jc.lib.standard_parser_mod_list)
* [streaming_parser_mod_list](#jc.lib.streaming_parser_mod_list)
jc - JSON Convert lib module
### all_parser_info
```python
def all_parser_info(
documentation: bool = False,
show_hidden: bool = False,
show_deprecated: bool = False) -> List[jc.jc_types.ParserInfoType]
```
Returns a list of dictionaries that includes metadata for all parser
modules. By default only non-hidden, non-deprecated parsers are
returned.
Parameters:
documentation: (boolean) include parser docstrings if `True`
show_hidden: (boolean) also show parsers marked as hidden
in their info metadata.
show_deprecated: (boolean) also show parsers marked as
deprecated in their info metadata.
### get_help
```python
def get_help(parser_mod_name: Union[str, module]) -> None
```
Show help screen for the selected parser.
This function will accept **module_name**, **cli-name**, and
**--argument-name** variants of the module name string as well as a
parser module object.
### get_parser
```python
def get_parser(parser_mod_name: Union[str, module]) -> module
```
Return the parser module object and check that the module is a valid
parser module.
Parameters:
parser_mod_name: (string or Name of the parser module. This
Module) function will accept module_name,
cli-name, and --argument-name
variants of the module name.
If a Module is given and the Module
is a valid parser Module, then the
same Module is returned.
Returns:
Module: the parser Module object
Raises:
ModuleNotFoundError: If the Module is not found or is not a valid
parser Module, then a ModuleNotFoundError
exception is raised.
### parse
```python
def parse(
parser_mod_name: Union[str, module],
data: Union[str, bytes, Iterable[str]],
quiet: bool = False,
raw: bool = False,
ignore_exceptions: Optional[bool] = None,
**kwargs
) -> Union[Dict[str, Any], List[Dict[str, Any]], Iterator[Dict[str, Any]]]
```
Parse the data (string or bytes) using the supplied parser (string or
module object).
This function provides a high-level API to simplify parser use. This
function will call built-in parsers and custom plugin parsers.
Example (standard parsers):
>>> import jc
>>> date_obj = jc.parse('date', 'Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022
Example (streaming parsers):
>>> import jc
>>> ping_gen = jc.parse('ping_s', ping_output.splitlines())
>>> for item in ping_gen:
>>> print(f'Response time: {item["time_ms"]} ms')
Response time: 102 ms
Response time: 109 ms
...
To get a list of available parser module names, use `parser_mod_list()`.
Alternatively, a parser module object can be supplied:
>>> import jc
>>> jc_date = jc.get_parser('date')
>>> date_obj = jc.parse(jc_date, 'Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022
You can also use the parser modules directly via `get_parser()`:
>>> import jc
>>> jc_date = jc.get_parser('date')
>>> date_obj = jc_date.parse('Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022
Finally, you can access the low-level parser modules manually:
>>> import jc.parsers.date
>>> date_obj = jc.parsers.date.parse('Tue Jan 18 10:23:07 PST 2022')
>>> print(f'The year is: {date_obj["year"]}')
The year is: 2022
Though, accessing plugin parsers directly is a bit more cumbersome, so
this higher-level API is recommended. Here is how you can access plugin
parsers without this API:
>>> import os
>>> import sys
>>> import jc.appdirs
>>> data_dir = jc.appdirs.user_data_dir('jc', 'jc')
>>> local_parsers_dir = os.path.join(data_dir, 'jcparsers')
>>> sys.path.append(local_parsers_dir)
>>> import my_custom_parser
>>> my_custom_parser.parse('command_data')
Parameters:
parser_mod_name: (string or name of the parser module. This
Module) function will accept module_name,
cli-name, and --argument-name
variants of the module name.
A Module object can also be passed
directly or via `get_parser()`
data: (string or data to parse (string or bytes for
bytes or standard parsers, iterable of
iterable) strings for streaming parsers)
raw: (boolean) output preprocessed JSON if `True`
quiet: (boolean) suppress warning messages if `True`
ignore_exceptions: (boolean) ignore parsing exceptions if `True`
(streaming parsers only)
Returns:
Standard Parsers: Dictionary or List of Dictionaries
Streaming Parsers: Generator Object containing Dictionaries
### parser_info
```python
def parser_info(parser_mod_name: Union[str, module],
documentation: bool = False) -> jc.jc_types.ParserInfoType
```
Returns a dictionary that includes the parser module metadata.
Parameters:
parser_mod_name: (string or name of the parser module. This
Module) function will accept module_name,
cli-name, and --argument-name
variants of the module name as well
as a parser module object.
documentation: (boolean) include parser docstring if `True`
### parser_mod_list
```python
def parser_mod_list(show_hidden: bool = False,
show_deprecated: bool = False) -> List[str]
```
Returns a list of all available parser module names.
### plugin_parser_mod_list
```python
def plugin_parser_mod_list(show_hidden: bool = False,
show_deprecated: bool = False) -> List[str]
```
Returns a list of plugin parser module names. This function is a
subset of `parser_mod_list()`.
### slurpable_parser_mod_list
```python
def slurpable_parser_mod_list(show_hidden: bool = False,
show_deprecated: bool = False) -> List[str]
```
Returns a list of slurpable parser module names. This function is a
subset of `parser_mod_list()`.
### standard_parser_mod_list
```python
def standard_parser_mod_list(show_hidden: bool = False,
show_deprecated: bool = False) -> List[str]
```
Returns a list of standard parser module names. This function is a
subset of `parser_mod_list()` and does not contain any streaming
parsers.
### streaming_parser_mod_list
```python
def streaming_parser_mod_list(show_hidden: bool = False,
show_deprecated: bool = False) -> List[str]
```
Returns a list of streaming parser module names. This function is a
subset of `parser_mod_list()`.
---
### Readme
[Home](https://kellyjonbrazil.github.io/jc/)
# jc
JC - JSON Convert
* kellyjonbrazil@gmail.com
This package converts the output of many standard unix command line tools
and file-types to dictionaries and lists of dictionaries.
## Interactive Documentation
Using `jc` in your python programs:
>>> help('jc')
>>> help('jc.lib')
>>> jc.get_help('parser_module_name')
Developing `jc` parsers:
>>> help('jc.utils')
>>> help('jc.streaming')
>>> help('jc.parsers.universal')
## Online Documentation
### Latest
https://github.com/kellyjonbrazil/jc/tree/master/docs
### Specific Version
`https://github.com/kellyjonbrazil/jc/tree/v/docs`
> Replace `` - e.g. `1.18.0`:
Specific versions can also be selected by tag in the Github branch dropdown
menu.
## Usage Example
>>> import subprocess
>>> import jc
>>>
>>> cmd_output = subprocess.check_output(['dig', 'example.com'],
text=True)
>>> data = jc.parse('dig', cmd_output)
>>> data
[{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', ...}]
Alternatively, you can bypass the high-level API and call the parser
modules directly:
>>> import subprocess
>>> import jc
>>>
>>> jc_dig = jc.get_parser('dig')
>>> cmd_output = subprocess.check_output(['dig', 'example.com'],
text=True)
>>> data = jc_dig.parse(cmd_output)
>>> data
[{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', ...}]
or
>>> import subprocess
>>> import jc.parsers.dig
>>>
>>> cmd_output = subprocess.check_output(['dig', 'example.com'],
text=True)
>>> data = jc.parsers.dig.parse(cmd_output)
>>> data
[{'id': 64612, 'opcode': 'QUERY', 'status': 'NOERROR', ...}]
## Available Functions
Use `help(jc.lib)` for details.
### parse
parse(
parser_module_name: str,
data: str | bytes | Iterable
) -> dict | list[dict] | Iterable[dict]
High-level API to easily access the parser. This API will find both
built-in parsers and local plugin parsers.
### get_parser
get_parser(
parser_module_name: str
) -> ModuleType
Get a parser Module object so you can use it directly.
### parser_info
parser_info(
parser_module_name: str,
documentation: bool = False
) -> dict
Get the metadata for a particular parser.
### all_parser_info
all_parser_info(documentation: bool = False) -> list[dict]
Get the metadata for all parsers.
### get_help
get_help(parser_module_name: str) -> None
Convenience function to display the help screen for a parser using
its module name.
### parser_mod_list
parser_mod_list() -> list[str]
Get a list of all available parser module names to be used in
`parse()`, `parser_info()`, and `get_help()`.
### plugin_parser_mod_list
plugin_parser_mod_list() -> list[str]
Get a list of plugin parser module names to be used in
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
`parser_mod_list()`.
### standard_parser_mod_list
standard_parser_mod_list() -> list[str]
Get a list of standard parser module names to be used in
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
`parser_mod_list()` and does not contain any streaming parsers.
### streaming_parser_mod_list
streaming_parser_mod_list() -> list[str]
Get a list of streaming parser module names to be used in
`parse()`, `parser_info()`, and `get_help()`. This list is a subset of
`parser_mod_list()`.
---
### Streaming
[Home](https://kellyjonbrazil.github.io/jc/)
# jc.streaming
## Table of Contents
* [jc.streaming](#jc.streaming)
* [add_jc_meta](#jc.streaming.add_jc_meta)
* [raise_or_yield](#jc.streaming.raise_or_yield)
* [stream_error](#jc.streaming.stream_error)
* [stream_success](#jc.streaming.stream_success)
* [streaming_input_type_check](#jc.streaming.streaming_input_type_check)
* [streaming_line_input_type_check](#jc.streaming.streaming_line_input_type_check)
jc - JSON Convert streaming utils
### add_jc_meta
```python
def add_jc_meta(func: ~F) -> ~F
```
Decorator for streaming parsers to add `stream_success` and
`stream_error` objects. This simplifies the `yield` lines in the
streaming parsers.
With the decorator on parse():
# successfully parsed line:
yield output_line if raw else _process(output_line)
# unsuccessfully parsed line:
except Exception as e:
yield raise_or_yield(ignore_exceptions, e, line)
Without the decorator on parse():
# successfully parsed line:
if raw:
yield stream_success(output_line, ignore_exceptions)
else:
stream_success(_process(output_line), ignore_exceptions)
# unsuccessfully parsed line:
except Exception as e:
yield stream_error(raise_or_yield(ignore_exceptions, e, line))
In all cases above:
output_line: (Dict) successfully parsed line yielded as a dict
e: (BaseException) exception object as the first value
of the tuple if the line was not successfully parsed.
line: (str) string of the original line that did not
successfully parse.
ignore_exceptions: (bool) continue processing lines and ignore
exceptions if `True`.
### raise_or_yield
```python
def raise_or_yield(ignore_exceptions: bool, e: BaseException,
line: str) -> Tuple[BaseException, str]
```
Return the exception object and line string if `ignore_exceptions` is
`True`. Otherwise, re-raise the exception from the exception object with
an annotation.
### stream_error
```python
def stream_error(e: BaseException, line: str) -> Dict[str, Any]
```
Return an error `_jc_meta` field.
### stream_success
```python
def stream_success(output_line: Dict[str, Any],
ignore_exceptions: bool) -> Dict[str, Any]
```
Add `_jc_meta` object to output line if `ignore_exceptions=True`
### streaming_input_type_check
```python
def streaming_input_type_check(data: Iterable[Union[str, bytes]]) -> None
```
Ensure input data is an iterable, but not a string or bytes. Raises
`TypeError` if not.
### streaming_line_input_type_check
```python
def streaming_line_input_type_check(line: str) -> None
```
Ensure each line is a string. Raises `TypeError` if not.
---
### Utils
[Home](https://kellyjonbrazil.github.io/jc/)
# jc.utils
## Table of Contents
* [jc.utils](#jc.utils)
* [compatibility](#jc.utils.compatibility)
* [convert_size_to_int](#jc.utils.convert_size_to_int)
* [convert_to_bool](#jc.utils.convert_to_bool)
* [convert_to_float](#jc.utils.convert_to_float)
* [convert_to_int](#jc.utils.convert_to_int)
* [error_message](#jc.utils.error_message)
* [has_data](#jc.utils.has_data)
* [input_type_check](#jc.utils.input_type_check)
* [is_compatible](#jc.utils.is_compatible)
* [line_slice](#jc.utils.line_slice)
* [normalize_key](#jc.utils.normalize_key)
* [remove_quotes](#jc.utils.remove_quotes)
* [warning_message](#jc.utils.warning_message)
jc - JSON Convert utils
### compatibility
```python
def compatibility(mod_name: str,
compatible: List[str],
quiet: bool = False) -> None
```
Checks for the parser's compatibility with the running OS platform and
prints a warning message to `STDERR` if not compatible and
`quiet=False.`
Parameters:
mod_name: (string) __name__ of the calling module
compatible: (list) sys.platform name(s) compatible with
the parser. compatible options:
linux, darwin, cygwin, win32, aix, freebsd
quiet: (bool) suppress compatibility message if `True`
Returns:
None - just prints output to STDERR
### convert_size_to_int
```python
def convert_size_to_int(size: str,
binary: bool = False,
posix_mode: bool = False,
decimal_bias: bool = False) -> Optional[int]
```
Parse a human readable data size and return the number of bytes.
Parameters:
size: (string) The human readable file size to parse.
binary: (boolean) `True` to use binary multiples of bytes
(base-2) for ambiguous unit symbols and names,
`False` to use decimal multiples of bytes (base-10).
posix_mode: (boolean) Treat one-letter units (k, m, g, etc.) as
binary.
decimal_bias: (boolean) `True` to treat slightly ambiguous two-
letter unit symbols ending in "i" (e.g. Ki, Gi) to
use decimal multiples of bytes (base-10). `False`
(default) to use binary multiples of bytes.
Returns:
integer/None Integer if successful conversion, otherwise None
This function knows how to parse sizes in bytes, kilobytes, megabytes,
gigabytes, terabytes and petabytes. Some examples:
>>> convert_size_to_int('42')
42
>>> convert_size_to_int('13b')
13
>>> convert_size_to_int('5 bytes')
5
>>> convert_size_to_int('1 KB')
1000
>>> convert_size_to_int('1 kilobyte')
1000
>>> convert_size_to_int('1 KiB')
1024
>>> convert_size_to_int('1 Ki')
1024
>>> convert_size_to_int('1 Ki', decimal_bias=True)
1000
>>> convert_size_to_int('1 KB', binary=True)
1024
>>> convert_size_to_int('1.5 GB')
1500000000
>>> convert_size_to_int('1.5 GB', binary=True)
1610612736
### convert_to_bool
```python
def convert_to_bool(value: object) -> bool
```
Converts string, integer, or float input to boolean by checking
for 'truthy' values.
Parameters:
value: (string/integer/float) Input value
Returns:
True/False False unless a 'truthy' number or string is found
('y', 'yes', 'true', '1', 1, -1, etc.)
### convert_to_float
```python
def convert_to_float(value: object) -> Optional[float]
```
Converts string and int input to float. Strips all non-numeric
characters from strings.
Parameters:
value: (string/integer) Input value
Returns:
float/None Float if successful conversion, otherwise None
### convert_to_int
```python
def convert_to_int(value: object) -> Optional[int]
```
Converts string and float input to int. Strips all non-numeric
characters from strings.
Parameters:
value: (string/float) Input value
Returns:
integer/None Integer if successful conversion, otherwise None
### error_message
```python
def error_message(message_lines: List[str]) -> None
```
Prints an error message to `STDERR` for fatal issues. The first line is
prepended with 'jc: Error - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.
Parameters:
message: (list) list of string lines
Returns:
None - just prints output to `STDERR`
### has_data
```python
def has_data(data: Union[str, bytes]) -> bool
```
Checks if the string input contains data. If there are any
non-whitespace characters then return `True`, else return `False`.
For bytes, returns `True` if there is any data.
Parameters:
data: (string, bytes) input to check whether it contains data
Returns:
Boolean `True` if input string (data) contains non-whitespace
characters, otherwise `False`. For bytes data, returns
`True` if there is any data, otherwise `False`.
### input_type_check
```python
def input_type_check(data: object) -> None
```
Ensure input data is a string. Raises `TypeError` if not.
### is_compatible
```python
def is_compatible(compatible: List[str]) -> bool
```
Returns True if the parser is compatible with the running OS platform.
### line_slice
```python
def line_slice(
data: Union[str, Iterable[str], TextIO, bytes, NoneType],
slice_start: Optional[int] = None,
slice_end: Optional[int] = None
) -> Union[str, Iterable[str], TextIO, bytes, NoneType]
```
Slice input data by lines - lazily, if possible.
Accepts a string (for normal parsers) or an iterable (for streaming
parsers). Uses normal start/stop slicing values, but will always slice
on lines instead of characters. Positive slices will use less memory as
the function will attempt to lazily iterate over the input. A negative
slice parameter will force the function to read in all of the data and
then slice, which will use more memory.
Parameters:
data: (string or iterable) - input to slice by lines
slice_start: (int) - starting line
slice_end: (int) - ending line
Returns:
string if input is a string.
iterable of strings if input is an iterable (for streaming parsers)
### normalize_key
```python
def normalize_key(data: str) -> str
```
Normalize a key name by shifting to lower-case and converting special
characters to underscores.
Special characters are defined as `space` and the following:
!"#$%&'()*+,-./:;<=>?@[\]^`{|}~
This is a lossy algorithm. Repeating and trailing underscores are
removed.
Parameters:
data: (string) Input value
Returns:
string
### remove_quotes
```python
def remove_quotes(data: str) -> str
```
Remove single or double quotes surrounding a string. If no quotes are
found then the string is returned unmodified.
Parameters:
data: (string) Input value
Returns:
string
### warning_message
```python
def warning_message(message_lines: List[str]) -> None
```
Prints warning message to `STDERR` for non-fatal issues. The first line
is prepended with 'jc: Warning - ' and subsequent lines are indented.
Wraps text as needed based on the terminal width.
Parameters:
message: (list) list of string lines
Returns:
None - just prints output to STDERR
---