## 1. Project Overview & Quickstart (tboox/tbox) A glib-like cross-platform C library ## Supporting the project Support this project by becoming a sponsor. Your logo will show up here with a link to your website. 🙏 [[Become a sponsor](https://docs.tboox.top/#/about/sponsor)] [](https://opencollective.com/tbox#backers) ## Introduction ([中文](/README_zh.md)) TBOX is a glib-like cross-platform C library that is simple to use yet powerful in nature. The project focuses on making C development easier and provides many modules (.e.g stream, coroutine, regex, container, algorithm ...), so that any developer can quickly pick it up and enjoy the productivity boost when developing in C language. It supports the following platforms: Windows, Macosx, Linux, Android, iOS, *BSD and etc. And it provides many compiling options using [xmake](https://github.com/xmake-io/xmake): * Release: Disable debug information, assertion, memory checking and enable optimization. * Debug: Enable debug information, assertion, memory checking and disable optimization. * Small: Disable all extensional modules and enable space optimization. * Micro: compiling micro library (~64K) for the embed system. If you want to know more, please refer to: [Documents](https://docs.tboox.top/#/getting_started), [Github](https://github.com/tboox/tbox) and [Gitee](https://gitee.com/tboox/tbox) ## Features #### The stream library - Supports file, data, http and socket source - Supports the stream filter for gzip, charset and... - Implements stream transfer - Implements the static buffer stream for parsing data - Supports coroutine and implements asynchronous operation #### The coroutine library - Provides high-performance coroutine switch - Supports arm, arm64, x86, x86_64 .. - Provides channel interfaces - Provides semaphore and lock interfaces - Supports io socket and stream operation in coroutine - Provides some io servers (http ..) using coroutine - Provides stackfull and stackless coroutines - Support epoll, kqueue, poll, select and IOCP - Support to wait pipe, socket and process in coroutine and poller at same time #### The database library - Supports mysql and sqlite3 database and enumerates data using the iterator mode #### The xml parser library - Supports DOM and SAX mode and Supports xpath #### The serialization and deserialization library - Supports xml, json, bplist, xplist, binary formats #### The memory library - Implements some memory pools for optimizing memory - Supports fast memory error detecting. it can detect the following types of bugs for the debug mode: - out-of-bounds accesses to heap and globals - use-after-free - double-free, invalid free - memory leaks #### The container library - Implements hash table, single list, double list, vector, stack, queue and min/max heap. Supports iterator mode for algorithm #### The algorithm library - Uses the iterator mode - Implements find, binary find and reverse find algorithm - Implements sort, bubble sort, quick sort, heap sort and insert sort algorithm - Implements count, walk items, reverse walk items, for_all and rfor_all #### The network library - Implements dns(cached) - Implements ssl(openssl, polarssl, mbedtls) - Implements http - Implements cookies - Supports ipv4, ipv6 - Supports coroutine #### The platform library - Implements timer, fast and low precision timer - Implements atomic and atomic64 operation - Implements spinlock, mutex, event, semaphore, thread and thread pool - Implements file, socket operation - Implements poller using epoll, poll, select, kqueue ... - Implements switch context interfaces for coroutine #### The charset library - Supports utf8, utf16, gbk, gb2312, uc2 and uc4 - Supports big endian and little endian mode #### The zip library - Supports gzip, zlibraw, zlib formats using the zlib library if exists - Implements lzsw, lz77 and rlc algorithm #### The utils library - Implements base32, base64 encoder and decoder - Implements assert and trace output for the debug mode - Implements bits operation for parsing u8, u16, u32, u64 data #### The math library - Implements random generator - Implements fast fixed-point calculation, Supports 6-bits, 16-bits, 30-bits fixed-point number #### The libc library - Implements lightweight libc library interfaces, the interface name contains `tb_xxx` prefix for avoiding conflict - Implements strixxx strrxxx wcsixxx wcsrxxx interface extension - Optimizes some frequently-used interface, .e.g. memset, memcpy, strcpy ... - Implements `memset_u16`, `memset_u32`, `memset_u64` extension interfaces #### The libm library - Implements lightweight libm library interfaces, the interface name contains `tb_xxx` prefix for avoiding conflict - Supports float and double type #### The regex library - Supports match and replace - Supports global/multiline/caseless mode - Uses pcre, pcre2 and posix regex modules #### The hash library - Implements crc32, adler32, md5 and sha1 hash algorithm - Implements some string hash algorithms (.e.g bkdr, fnv32, fnv64, sdbm, djb2, rshash, aphash ...) - Implements uuid generator ## Projects Some projects using tbox: * [gbox](https://github.com/tboox/gbox) * [vm86](https://github.com/tboox/vm86) * [xmake](http://www.xmake.io) * [itrace](https://github.com/tboox/itrace) * [more](https://github.com/tboox/tbox/wiki/tbox-projects) ## Build (xmake) Please install xmake first: [xmake](https://github.com/xmake-io/xmake) ```console # build for the host platform $ cd ./tbox $ xmake # build for the mingw platform $ cd ./tbox $ xmake f -p mingw --sdk=/home/mingwsdk $ xmake # build for the iphoneos platform $ cd ./tbox $ xmake f -p iphoneos $ xmake # build for the android platform $ cd ./tbox $ xmake f -p android --ndk=xxxxx $ xmake # build for the linux cross-platform $ cd ./tbox $ xmake f -p linux --sdk=/home/sdk # --bin=/home/sdk/bin $ xmake ``` ## Build (xmake.sh) ```console $ ./configure $ make ``` ## Example ```c #include "tbox/tbox.h" int main(int argc, char** argv) { if (!tb_init(tb_null, tb_null)) return 0; tb_vector_ref_t vector = tb_vector_init(0, tb_element_str(tb_true)); if (vector) { tb_vector_insert_tail(vector, "hello"); tb_vector_insert_tail(vector, "tbox"); tb_for_all (tb_char_t const*, cstr, vector) { tb_trace_i("%s", cstr); } tb_vector_exit(vector); } tb_exit(); return 0; } ``` ## Technical Support You can also consider sponsoring us to get technical support services, [[Become a sponsor](https://docs.tboox.top/#/about/sponsor)] ## Contacts * Email:[waruqi@gmail.com](mailto:waruqi@gmail.com) * Homepage:[tboox.top](https://tboox.top) * Community:[/r/tboox on reddit](https://www.reddit.com/r/tboox/) * ChatRoom:[Chat on telegram](https://t.me/tbooxorg), [Chat on gitter](https://gitter.im/tboox/tboox?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge) * QQ Group: 343118190(full), 662147501 * Wechat Public: tboox-os ## 2. Official Technical Reference & Guides (tboox/tbox-docs) ## File: README.md ## Introduction TBOX is a glib-like cross-platform C library that is simple to use yet powerful in nature. The project focuses on making C development easier and provides many modules (.e.g stream, coroutine, regex, container, algorithm ...), so that any developer can quickly pick it up and enjoy the productivity boost when developing in C language. It supports the following platforms: - Windows - Macosx - Linux - Android - iOS And it provides many compiling options using [xmake](https://github.com/xmake-io/xmake): * Release: Disable debug information, assertion, memory checking and enable optimization. * Debug: Enable debug information, assertion, memory checking and disable optimization. * Small: Disable all extensional modules and enable space optimization. * Micro: compiling micro library (~64K) for the embed system. If you want to know more, please refer to: * [HomePage](https://tboox.org) * [Documents](https://tboox.io/#/getting_started) * [Github](https://github.com/tboox/tbox) * [Gitee](https://gitee.com/tboox/tbox) ## Features #### The stream library - Supports file, data, http and socket source - Supports the stream filter for gzip, charset and... - Implements stream transfer - Implements the static buffer stream for parsing data - Supports coroutine and implements asynchronous operation #### The coroutine library - Provides high-performance coroutine switch(refer to [reports](https://tboox.org/2016/10/28/benchbox-coroutine/)) - Supports arm, arm64, x86, x86_64 .. - Provides channel interfaces - Provides semaphore and lock interfaces - Supports io socket and stream operation in coroutine - Provides some io servers (http ..) using coroutine - Provides stackfull and stackless coroutines - Support epoll, kqueue, poll, select and IOCP #### The database library - Supports mysql and sqlite3 database and enumerates data using the iterator mode #### The xml parser library - Supports DOM and SAX mode and Supports xpath #### The serialization and deserialization library - Supports xml, json, bplist, xplist, binary formats #### The memory library - Implements some memory pools for optimizing memory - Supports fast memory error detecting. it can detect the following types of bugs for the debug mode: - out-of-bounds accesses to heap and globals - use-after-free - double-free, invalid free - memory leaks #### The container library - Implements hash table, single list, double list, vector, stack, queue and min/max heap. Supports iterator mode for algorithm #### The algorithm library - Uses the iterator mode - Implements find, binary find and reverse find algorithm - Implements sort, bubble sort, quick sort, heap sort and insert sort algorithm - Implements count, walk items, reverse walk items, for_all and rfor_all #### The network library - Implements dns(cached) - Implements ssl(openssl, polarssl, mbedtls) - Implements http - Implements cookies - Supports ipv4, ipv6 - Supports coroutine #### The platform library - Implements timer, fast and low precision timer - Implements atomic and atomic64 operation - Implements spinlock, mutex, event, semaphore, thread and thread pool - Implements file, socket operation - Implements poller using epoll, poll, select, kqueue ... - Implements switch context interfaces for coroutine #### The charset library - Supports utf8, utf16, gbk, gb2312, uc2 and uc4 - Supports big endian and little endian mode #### The zip library - Supports gzip, zlibraw, zlib formats using the zlib library if exists - Implements lzsw, lz77 and rlc algorithm #### The utils library - Implements base32, base64 encoder and decoder - Implements assert and trace output for the debug mode - Implements bits operation for parsing u8, u16, u32, u64 data #### The math library - Implements random generator - Implements fast fixed-point calculation, Supports 6-bits, 16-bits, 30-bits fixed-point number #### The libc library - Implements lightweight libc library interfaces, the interface name contains `tb_xxx` prefix for avoiding conflict - Implements strixxx strrxxx wcsixxx wcsrxxx interface extension - Optimizes some frequently-used interface, .e.g. memset, memcpy, strcpy ... - Implements `memset_u16`, `memset_u32`, `memset_u64` extension interfaces #### The libm library - Implements lightweight libm library interfaces, the interface name contains `tb_xxx` prefix for avoiding conflict - Supports float and double type #### The regex library - Supports match and replace - Supports global/multiline/caseless mode - Uses pcre, pcre2 and posix regex modules #### The hash library - Implements crc32, adler32, md5 and sha1 hash algorithm - Implements some string hash algorithms (.e.g bkdr, fnv32, fnv64, sdbm, djb2, rshash, aphash ...) - Implements uuid generator ## Projects Some projects using tbox: * [gbox](https://github.com/tboox/gbox) * [vm86](https://github.com/tboox/vm86) * [xmake](http://www.xmake.io) * [itrace](https://github.com/tboox/itrace) * [more](https://github.com/tboox/tbox/wiki/tbox-projects) ## Build Please install xmake first: [xmake](https://github.com/xmake-io/xmake) ```console # build for the host platform $ cd ./tbox $ xmake # build for the mingw platform $ cd ./tbox $ xmake f -p mingw --sdk=/home/mingwsdk $ xmake # build for the iphoneos platform $ cd ./tbox $ xmake f -p iphoneos $ xmake # build for the android platform $ cd ./tbox $ xmake f -p android --ndk=xxxxx $ xmake # build for the linux cross-platform $ cd ./tbox $ xmake f -p linux --sdk=/home/sdk # --bin=/home/sdk/bin $ xmake ``` ## Example ```c #include "tbox/tbox.h" int main(int argc, char** argv) { // init tbox if (!tb_init(tb_null, tb_null)) return 0; // trace tb_trace_i("hello tbox"); // init vector tb_vector_ref_t vector = tb_vector_init(0, tb_element_cstr(tb_true)); if (vector) { // insert item tb_vector_insert_tail(vector, "hello"); tb_vector_insert_tail(vector, "tbox"); // dump all items tb_for_all (tb_char_t const*, cstr, vector) { // trace tb_trace_i("%s", cstr); } // exit vector tb_vector_exit(vector); } // init stream tb_stream_ref_t stream = tb_stream_init_from_url("http://www.xxx.com/file.txt"); if (stream) { // open stream if (tb_stream_open(stream)) { // read line tb_long_t size = 0; tb_char_t line[TB_STREAM_BLOCK_MAXN]; while ((size = tb_stream_bread_line(stream, line, sizeof(line))) >= 0) { // trace tb_trace_i("line: %s", line); } } // exit stream tb_stream_exit(stream); } // wait tb_getchar(); // exit tbox tb_exit(); return 0; } ``` ## Contacts * Email:[waruqi@gmail.com](mailto:waruqi@gmail.com) * Homepage:[tboox.org](https://tboox.org) * Community:[/r/tboox on reddit](https://www.reddit.com/r/tboox/) * QQ Group: 343118190 * Wechat Public: tboox-os --- ## File: guide/quickstart.md ## Ready to work ### Installing xmake To compile the tbox source, you need to install the [xmake](https://github.com/xmake-io/xmake) build tool first, because the entire tbox project is maintained by xmake, a cross-platform build tool. For how to install xmake, you can look at: [xmake installation documentation](https://xmake.io/#/guide/installation) The inside is very detailed, of course, under normal circumstances, the following installation methods can basically meet most of the installation scenarios, unless you want to compile and install the source code. #### via curl ```bash bash <(curl -fsSL https://raw.githubusercontent.com/xmake-io/xmake/master/scripts/get.sh) ``` #### via wget ```bash bash <(wget https://raw.githubusercontent.com/xmake-io/xmake/master/scripts/get.sh -O -) ``` #### via powershell ```bash Invoke-Expression (Invoke-Webrequest 'https://raw.githubusercontent.com/xmake-io/xmake/master/scripts/get.ps1' -UseBasicParsing).Content ``` ## Create an empty project with tbox Xmake provides an empty project template with tbox, so you can create a tbox-based empty project via xmake to quickly integrate and compile the tbox library. We only need to execute the command: ```bash $ xmake create -t console_tbox test ``` Our tbox-based console program is created, let's take a look at the structure inside this project: ``` . ├── src │   ├── main.c │   └── xmake.lua └── xmake.lua ``` Very simple, the general content of xmake.lua, under the simplification, it is like this: ```lua add_requires("tbox") target("test") set_kind("binary") add_files("src/*.c") ``` To put it bluntly, it is to add a reference library reference to tbox. In main.c, only the tbox.h header file is referenced, which is very simple: ```c #include "tbox/tbox.h" tb_int_t main(tb_int_t argc, tb_char_t** argv) { // init tbox if (!tb_init(tb_null, tb_null)) return -1; // trace tb_trace_i("hello tbox!"); // exit tbox tb_exit(); return 0; } ``` ## Compiling project After the project is created, the next step is to compile. This step is also very simple. You only need to execute the xmake command to compile: ```bash $ xmake checking for the architecture ... x86_64 checking for the Xcode directory ... /Applications/Xcode.app checking for the SDK version of Xcode ... 10.14 checking for the Cuda SDK directory ... /Developer/NVIDIA/CUDA-10.1 note: try installing these packages (pass -y to skip confirm)? in xmake-repo: -> tbox v1.6.3 please input: y (y/n) => install tbox v1.6.3 .. ok ruki:test ruki$ xmake -r [ 0%]: ccache compiling.release src/main.c [100%]: linking.release test build ok!👌 ``` Xmake will automatically download the tbox library dependencies, and automatically install the integrated tbox library into the current project, the user does not need to care about any other details, just trust the code to call the tbox interface to achieve their own logic. ## Run program After compiling, you can complete the run by typing the following command: ```bash $ xmake run ``` ## Debug program If you want to call the debugger such as gdb/lldb/vsjitdebugger to debug the program, just add the `-d` parameter when running, and turn on the debug compilation mode: ```bash $ xmake f -m debug $ xmake $ xmake run -d ``` Because the default compiled release mode, without debugging symbol information, so if you want to debug the program, you can first enable the debug mode to compile, then debug and run. ## Generate IDE project files ### Generate vs project We can also develop and debug more conveniently by generating a vs project. The following command also generates a vc project with two compilation modes: debug and release. ```bash xmake project -k vs2017 -m "debug,release" ``` ### Generate cmake file ```bash xmake project -k cmakelists ``` ### Generate makefile ```bash xmake project -k makefile ``` ## Compile Source code In addition to creating an empty project, using `add_requires("tbox")` in xmake.lua to quickly integrate tbox, we can also manually compile the integration through the current tbox source. First, we need to download the source code of tbox: ```bash git clone https://github.com/tboox/tbox.git ``` Then enter the project root directory and execute xmake compilation: ```bash cd tbox xmake ``` After compiling, we can get the corresponding library files and header files by installing or packaging. ### Install library Through the installation, you can install the compiled library and header files to the system directory or specify the directory. ```bash xmake install xmake install -o /xxx/installdir ``` ### Package library The above installation method can only install the corresponding arch library under one platform. If you want to switch the platform and architecture at the same time, compile and generate a series of library versions. This method is very cumbersome, so it can be completed by the packaging command. ```bash xmake f -p iphoneos -a armv7 xmake xmake package xmake f -p iphoneos -a arm64 -m debug xmake xmake package ``` The above command compiles two arch libraries under iphoneos, and the arm64 library is the debug version, which is packaged. The generated result is as follows: ``` build/tbox.pkg/ ├── iphoneos │   ├── arm64 │   │   ├── include │   │   │   └── tbox │   │   │   ├── algorithm │   │   └── lib │   │   └── debug │   │   └── libtbox.a │   └── armv7 │   ├── include │   │   └── tbox │   └── lib │   └── release │   └── libtbox.a └── xmake.lua ``` It can be seen that the package command will generate the tbox.pkg package in the build directory, which categorizes different platforms, different arches, different compilation modes, and header files. This is very helpful for outputting different libraries at a time. of. Moreover, xmake also provides an auxiliary macro command, which can simplify the packaging process and implement bulk packaging for all arches under one platform. Especially under iphoneos, it also generates a universal package: ```bash xmake macro package -p iphoneos ``` If you want to cut to debug mode, generate all the arch packages at once, you can pass the configuration into: ```bash xmake macro package -p iphoneos -f "-m debug" ``` ### Integrate library #### Integrate local packages The tbox.pkg package generated by the xmake package command can be directly referenced and integrated into our own project. Just set the corresponding header file search path and library path. And if it is in the xmake project, the integration is more convenient, just edit the xmake.lua file, plus two lines: ```lua add_packagedirs("packages") target("test")     set_kind("binary")     add_files("src/*.c")     add_packages("tbox") ``` We specify the directory where tbox.pkg is located by add_packagedirs, and then we can directly reference the integrated package through `add_packages("tbox"). When xmake compiles, it will automatically handle the search of header files and library paths. !> However, it should be noted that if you compile the tbox library using the debug version, you need to define the `__tb_debug__` macro in your own project. Therefore, we can continue to improve, plus release/debug mode support: ```lua add_packagedirs("packages") add_rules("mode.debug", "mode.release") if is_mode("debug") then add_defines("__tb_debug__") end target("test") set_kind("binary") add_files("src/*.c") add_packages("tbox") ``` #### Integrate Remote Dependency Package Of course, if you think that compiling the tbox library integration is rather cumbersome, you can use the remote dependency download mode mentioned above, xmake will handle the download compilation and integration of the tbox library. ```lua add_requires("tbox") add_rules("mode.debug", "mode.release") target("test") set_kind("binary") add_files("src/*.c") add_packages("tbox") ``` We don't need to download the source code to compile the tbox. We only need to set `add_requires("tbox") to add the required package dependencies. xmake will automatically download the description of the tbox package from the official package repository, and then automatically compile and install the integration. A bit like the way homebrew. If you need to use the debug version of the tbox library, you only need to change it to: ```lua add_requires("tbox", {debug = true}) ``` Users do not need to add the extra `__tb_debug__` macro because these will be handled automatically. If you want to use the specified version of the tbox library, or the dev/master repository, you only need to: ```lua add_requires("tbox dev") add_requires("tbox master") add_requires("tbox 1.6.3") add_requires("tbox >1.6.0") add_requires("tbox ~1.6.0") ``` The integration method is to support the semantic version dependency. For more information about the use of this block, you can look at the official documentation of xmake: [remote dependency mode] (https://xmake.io/#/zh-cn/guide/package_management ?id=%e8%bf%9c%e7%a8%8b%e4%be%9d%e8%b5%96%e6%a8%a1%e5%bc%8f) ### Module Configuration The default compiled and integrated tbox library, compiled for small mode, does not have any extension modules. If you want to use some extensions, you need to manually enable them. For source code compilation, you can enable the corresponding module to compile tbox by: ```bash xmake f --xml=y --coroutine=y xmake ``` For the remote dependency mode, it is convenient to enable the corresponding module. Change the dependency rule in xmake.lua: ```lua add_requires("tbox", {configs = {xml = true, coroutine = true}}) ``` ### Code Example In the `src/demo` source directory of tbox, the usage examples of each module are integrated. You can refer to the usage and familiarity with the use of different module interfaces. You can also refer to the interface comment description in the corresponding module header file for more information. The test run under the demo, you can enter the tbox root directory, compile and run through the following command: ```bash xmake run demo coroutine_http_server ``` The above command is to run the `http_server` example program based on coroutine in tbox. We can also list all current example program names by the following command: ```bash xmake run demo ``` The output is as follows: ``` [demo]: ====================================================================== [demo]: Usages: xmake r demo [testname] arguments ... [demo]: [demo]: .e.g [demo]: xmake r demo stream http://www.xxxxx.com /tmp/a [demo]: [demo]: testname: libc_time [demo]: testname: libc_wchar [demo]: testname: libc_string [demo]: testname: libc_stdlib [demo]: testname: libc_wcstombs [demo]: testname: coroutine_echo_client [demo]: testname: coroutine_file_client [demo]: testname: coroutine_http_server [demo]: testname: coroutine_spider ... ``` It’s too long, it’s not listed here, everyone can look at it after running it. ### Debug and analysis In addition to debugging with the debugger, the debug version of the tbox library also has a large number of assert detections built in, as well as various memory analysis and detection methods, including memory leaks, memory out-of-bounds analysis, etc., relying on the memory pool allocator that comes with the tbox library. . Therefore, under normal circumstances, using the tbox library to write the program, as long as there is no error under the debug, basically the program is relatively stable. #### Assertion detection All interfaces in tbox are implemented, and a large number of asserts are added to judge the validity of the program state and the parameters. As long as the user passes the wrong parameters, the debug mode can basically report the first time, and even some heap overflow problems. Will even report it. It is convenient for the user to obtain the error information in the first time and locate all the locations of the fault code. #### Memory out of bounds detection As long as the user finishes writing the program and enables the debug version to run the program, the program will automatically report an error when the memory is out of bounds. The detection of the out-of-boundary overflow is done in real time, and the libc is also instrumented, so the use of commonly used strcpy, memset, etc., is back to detect: ```c tb_void_t tb_demo_overflow() { tb_pointer_t data = tb_malloc0(10); if (data) { tb_memset(data, 0, 11); tb_free(data); } } ``` Output: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` #### Memory leak detection The detection of memory leaks must be executed when `tb_exit()` is called immediately before the program exits. If there is a leak, it will be output to the terminal in detail. ```c tb_void_t tb_demo_leak() { tb_pointer_t data = tb_malloc0(10); } ``` Output: ``` [tbox]: [error]: leak: 0x7f9d5b058908 at tb_static_fixed_pool_dump(): 735, memory/impl/static_fixed_pool.c [tbox]: [error]: data: from: tb_demo_leak(): 43, memory/check.c [tbox]: [error]: [0x000001050e742a]: 0 demo.b 0x00000001050e742a tb_fixed_pool_malloc0_ + 186 [tbox]: [error]: [0x000001050f972b]: 1 demo.b 0x00000001050f972b tb_small_pool_malloc0_ + 507 [tbox]: [error]: [0x000001050f593c]: 2 demo.b 0x00000001050f593c tb_pool_malloc0_ + 540 [tbox]: [error]: [0x00000105063cd7]: 3 demo.b 0x0000000105063cd7 tb_demo_leak + 55 [tbox]: [error]: [0x00000105063e44]: 4 demo.b 0x0000000105063e44 tb_demo_memory_check_main + 20 [tbox]: [error]: [0x0000010505b08e]: 5 demo.b 0x000000010505b08e main + 878 [tbox]: [error]: [0x007fff8c95a5fd]: 6 libdyld.dylib 0x00007fff8c95a5fd start + 1 [tbox]: [error]: [0x00000000000002]: 7 ??? 0x0000000000000002 0x0 + 2 [tbox]: [error]: data: 0x7f9d5b058908, size: 10, patch: cc ``` #### Memory Overlap Cover Detection If the copy of the two memories overlaps, it may overwrite some of the data, causing a bug, so TBOX has also done some detection. ```c tb_void_t tb_demo_overlap() { tb_pointer_t data = tb_malloc(10); if (data) { tb_memcpy(data, (tb_byte_t const*)data + 1, 5); tb_free(data); } } ``` Output: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` #### Memory Double Release Detection ```c tb_void_t tb_demo_free2() { tb_pointer_t data = tb_malloc0(10); if (data) { tb_free(data); tb_free(data); } } ``` Output: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ## File: zh-cn/guide/quickstart.md ## 准备工作 ### 安装xmake 编译tbox源码,需要先安装[xmake](https://github.com/xmake-io/xmake)构建工具,因为整个tbox项目都是由xmake这个跨平台的构建工具维护的。 关于如何安装xmake,可以看下:[xmake安装文档](https://xmake.io/#/zh-cn/guide/installation) 里面讲的非常详细,当然通常情况下,下面的安装方式基本上已经可以满足大部分安装场景,除非你想源码编译安装。 #### 使用curl ```bash bash <(curl -fsSL https://raw.githubusercontent.com/xmake-io/xmake/master/scripts/get.sh) ``` #### 使用wget ```bash bash <(wget https://raw.githubusercontent.com/xmake-io/xmake/master/scripts/get.sh -O -) ``` #### 使用powershell ```bash Invoke-Expression (Invoke-Webrequest 'https://raw.githubusercontent.com/xmake-io/xmake/master/scripts/get.ps1' -UseBasicParsing).Content ``` ## 创建空工程 xmake里面提供了带有tbox的空工程模板,因此可以通过xmake创建一个基于tbox的空工程,来快速集成和编译使用tbox库。 我们只需要执行命令: ```bash $ xmake create -t console_tbox test ``` 我们的基于tbox的控制台程序就创建好了,我们来看下这个工程里面的结构: ``` . ├── src │   ├── main.c │   └── xmake.lua └── xmake.lua ``` 非常简单,xmake.lua的大致内容,精简下,就长这样: ```lua add_requires("tbox") target("test") set_kind("binary") add_files("src/*.c") ``` 说白了,就是加上了tbox的依赖库引用。 而main.c里面,仅仅只引用了tbox.h头文件,非常简单: ```c #include "tbox/tbox.h" tb_int_t main(tb_int_t argc, tb_char_t** argv) { // init tbox if (!tb_init(tb_null, tb_null)) return -1; // trace tb_trace_i("hello tbox!"); // exit tbox tb_exit(); return 0; } ``` ## 编译工程 创建完工程,接下来就是编译了,这步也非常简单,只需要执行xmake命令即可完成编译: ```bash $ xmake checking for the architecture ... x86_64 checking for the Xcode directory ... /Applications/Xcode.app checking for the SDK version of Xcode ... 10.14 checking for the Cuda SDK directory ... /Developer/NVIDIA/CUDA-10.1 note: try installing these packages (pass -y to skip confirm)? in xmake-repo: -> tbox v1.6.3 please input: y (y/n) => install tbox v1.6.3 .. ok ruki:test ruki$ xmake -r [ 0%]: ccache compiling.release src/main.c [100%]: linking.release test build ok!👌 ``` xmake会去自动下载tbox库依赖,并且自动安装集成tbox库到当前项目中去,用户不需要关心其他任何细节,只管安心敲代码调用tbox接口实现自己的逻辑就行了。 ## 运行程序 编译完,可以敲下面的命令完成运行: ```bash $ xmake run ``` ## 调试程序 如果要调用gdb/lldb/vsjitdebugger等调试器来调试程序,只需要运行的时候加上`-d`参数,并且开启调试编译模式: ```bash $ xmake f -m debug $ xmake $ xmake run -d ``` 由于默认编译式release模式,不带调试符号信息,因此如果要调试程序,可以先启用debug模式编译后,再调试运行即可。 ## 生成IDE工程文件 ### 生成vs工程 我们也可以通过生成vs工程来更方便的开发和调试,下面的命令同时生成带有debug, release两个编译模式的vc工程。 ```bash xmake project -k vs2017 -m "debug,release" ``` ### 生成cmake文件 ```bash xmake project -k cmakelists ``` ### 生成makefile ```bash xmake project -k makefile ``` ## 源码编译 除了通过创建空工程,在xmake.lua中使用`add_requires("tbox")`快速集成tbox,我们也可以通过现在tbox源码手动编译集成。 首先,我们需要下载tbox的源码: ```bash git clone https://github.com/tboox/tbox.git ``` 然后进入工程根目录,执行xmake编译: ```bash cd tbox xmake ``` 编译完,我们可以通过安装或者打包的方式,获取对应的库文件和头文件 ### 安装库 通过安装,可以将编译好的库和头文件安装到系统目录或者指定目录 ```bash xmake install xmake install -o /xxx/installdir ``` ### 打包库 上面安装的方式,只能安装一个平台下对应arch的库,如果要同时切换平台和架构,编译生成一系列库版本,这种方式就很繁琐了,因此可以通过打包命令来完成。 ```bash xmake f -p iphoneos -a armv7 xmake xmake package xmake f -p iphoneos -a arm64 -m debug xmake xmake package ``` 上述命令,编译了iphoneos下,两个arch库,并且其中arm64库是debug版本的,进行打包,生成后的结果如下: ``` build/tbox.pkg/ ├── iphoneos │   ├── arm64 │   │   ├── include │   │   │   └── tbox │   │   │   ├── algorithm │   │   └── lib │   │   └── debug │   │   └── libtbox.a │   └── armv7 │   ├── include │   │   └── tbox │   └── lib │   └── release │   └── libtbox.a └── xmake.lua ``` 可以看出,打包命令,会在build目录下生成tbox.pkg包,里面对不同平台、不同arch、不同编译模式以及头文件都做了归类整理,这对于一次输出不同的库是非常有帮助的。 并且,xmake还提供了一个辅助的宏命令,可以锦衣简化打包流程,实现对一个平台下的所有arch,进行批量打包,尤其是iphoneos下,还会同时生成universal包: ```bash xmake macro package -p iphoneos ``` 如果想要切到debug模式,一次生成所有arch的包,可以传递配置进去: ```bash xmake macro package -p iphoneos -f "-m debug" ``` ### 集成库 #### 集成本地包 我们通过xmake的打包命令生成的tbox.pkg包,是可以直接引用集成到自己项目中去的,只要设置上对应的头文件搜索路径以及库路径即可 而如果是在xmake的项目中,集成就更加方便了,只需要编辑xmake.lua文件,加上两行: ```lua add_packagedirs("packages") target("test") set_kind("binary") add_files("src/*.c") add_packages("tbox") ``` 我们通过add_packagedirs指定下tbox.pkg所在的目录,然后就可以通过`add_packages("tbox")`直接引用集成对应的包,xmake在编译的时候,会自动处理头文件和库路径的搜索。 !> 不过需要注意的一点是,如果是编译使用debug版本的tbox库,还需要在自己的项目中,额外定义`__tb_debug__`宏标示下才行。 因此,我们可以继续完善下,加上release/debug模式支持: ```lua add_packagedirs("packages") add_rules("mode.debug", "mode.release") if is_mode("debug") then add_defines("__tb_debug__") end target("test") set_kind("binary") add_files("src/*.c") add_packages("tbox") ``` #### 集成远程依赖包 当然如果觉得这样编译tbox库集成还是比较繁琐,可以采用上面说的远程依赖下载模式,xmake会自己处理tbox库的下载编译和集成。 ```lua add_requires("tbox") add_rules("mode.debug", "mode.release") target("test") set_kind("binary") add_files("src/*.c") add_packages("tbox") ``` 我们不需要再自己下载源码编译tbox,只需要设置`add_requires("tbox")`添加需要的包依赖就行了,xmake会自动从官方包仓库下载tbox包的描述信息,然后自动编译和安装集成,有点类似homebrew的方式。 如果需要使用debug版本tbox库,只需要改成: ```lua add_requires("tbox", {debug = true}) ``` 用户也不需要额外加`__tb_debug__`宏了因为这些都会自动处理。 如果要使用指定版本的tbox库,或者dev/master版本库,只需要: ```lua add_requires("tbox dev") add_requires("tbox master") add_requires("tbox 1.6.3") add_requires("tbox >1.6.0") add_requires("tbox ~1.6.0") ``` 集成方式是支持语义版本依赖的哦,更多关于这块的使用描述,可以看下xmake的官方文档:[远程依赖模式](https://xmake.io/#/zh-cn/guide/package_management?id=%e8%bf%9c%e7%a8%8b%e4%be%9d%e8%b5%96%e6%a8%a1%e5%bc%8f) ### 模块配置 默认编译和集成的tbox库,为small编译模式,是不带有任何扩展模块的,如果想要使用一些扩展,需要自己手动启用。 对于源码编译方式,可以通过如下方式启用对应模块来编译tbox: ```bash xmake f --xml=y --coroutine=y xmake ``` 对于远程依赖方式,启用对应模块也很方便,在xmake.lua里面改下依赖规则就好: ```lua add_requires("tbox", {configs = {xml = true, coroutine = true}}) ``` ### 代码示例 tbox的`src/demo`源码目录下,集成了各个模块的使用例子,可以参考其中的用法也熟悉不同模块接口的使用,也可以参看对应模块头文件中的接口注释说明来获取更多信息。 而demo下的测试运行,可以进入tbox根目录,编译后通过下面的命令来运行: ```bash xmake run demo coroutine_http_server ``` 上面的命令就是运行tbox中的基于协程的`http_server`例子程序,我们也可以通过下面的命令列举当前所有的例子程序名: ```bash xmake run demo ``` 输出结果如下: ``` [demo]: ====================================================================== [demo]: Usages: xmake r demo [testname] arguments ... [demo]: [demo]: .e.g [demo]: xmake r demo stream http://www.xxxxx.com /tmp/a [demo]: [demo]: testname: libc_time [demo]: testname: libc_wchar [demo]: testname: libc_string [demo]: testname: libc_stdlib [demo]: testname: libc_wcstombs [demo]: testname: coroutine_echo_client [demo]: testname: coroutine_file_client [demo]: testname: coroutine_http_server [demo]: testname: coroutine_spider ... ``` 太长了,这里就不全列举出来,大家可以自己运行后看下。 ### 调试分析 除了用调试器运行后进行调试,tbox库的debug版本还内置的大量了assert检测,以及各种内存分析检测手段,包括内存泄露,内存越界分析等,依托于tbox库自带的内存池分配器。 因此,通常情况下,使用tbox库写完程序,只要在debug下运行没有任何报错,基本上程序算是比较稳定了。 #### 断言检测 tbox里面所有接口实现,加了大量的assert来判断程序状态和传参的有效性,只要用户传参不对,在debug模式基本上都能第一时间报出来,甚至一些堆溢出导致的问题,也会即使报出。 方便用户第一时间获取到错误信息,定位故障代码所有位置。 #### 内存越界检测 用户只要写完程序,启用debug版本运行程序,程序在出现常规对内存越界后,会自动报错提示。 越界溢出的检测,是实时完成的,而且对libc也做了插桩,所以对常用strcpy,memset等的使用,都回去检测: ```c tb_void_t tb_demo_overflow() { tb_pointer_t data = tb_malloc0(10); if (data) { tb_memset(data, 0, 11); tb_free(data); } } ``` 输出: ``` /* Detailed source-code truncated for AI context efficiency. */ ``` #### 内存泄露检测 内存泄露的检测必须在程序退出的前一刻,调用`tb_exit()`的时候,才会执行,如果有泄露,会有详细输出到终端上。 ```c tb_void_t tb_demo_leak() { tb_pointer_t data = tb_malloc0(10); } ``` 输出: ``` [tbox]: [error]: leak: 0x7f9d5b058908 at tb_static_fixed_pool_dump(): 735, memory/impl/static_fixed_pool.c [tbox]: [error]: data: from: tb_demo_leak(): 43, memory/check.c [tbox]: [error]: [0x000001050e742a]: 0 demo.b 0x00000001050e742a tb_fixed_pool_malloc0_ + 186 [tbox]: [error]: [0x000001050f972b]: 1 demo.b 0x00000001050f972b tb_small_pool_malloc0_ + 507 [tbox]: [error]: [0x000001050f593c]: 2 demo.b 0x00000001050f593c tb_pool_malloc0_ + 540 [tbox]: [error]: [0x00000105063cd7]: 3 demo.b 0x0000000105063cd7 tb_demo_leak + 55 [tbox]: [error]: [0x00000105063e44]: 4 demo.b 0x0000000105063e44 tb_demo_memory_check_main + 20 [tbox]: [error]: [0x0000010505b08e]: 5 demo.b 0x000000010505b08e main + 878 [tbox]: [error]: [0x007fff8c95a5fd]: 6 libdyld.dylib 0x00007fff8c95a5fd start + 1 [tbox]: [error]: [0x00000000000002]: 7 ??? 0x0000000000000002 0x0 + 2 [tbox]: [error]: data: 0x7f9d5b058908, size: 10, patch: cc ``` #### 内存重叠覆盖检测 如果两块内存的copy发生了重叠,有可能会覆盖掉部分数据,导致bug,因此TBOX对此也做了些检测。 ```c tb_void_t tb_demo_overlap() { tb_pointer_t data = tb_malloc(10); if (data) { tb_memcpy(data, (tb_byte_t const*)data + 1, 5); tb_free(data); } } ``` 输出 ``` /* Detailed source-code truncated for AI context efficiency. */ ``` #### 内存双重释放检测 ```c tb_void_t tb_demo_free2() { tb_pointer_t data = tb_malloc0(10); if (data) { tb_free(data); tb_free(data); } } ``` 输出 ``` /* Detailed source-code truncated for AI context efficiency. */ ``` --- ## File: zh-cn/README.md ## 简介 TBOX是一个用c语言实现的跨平台开发库。 针对各个平台,封装了统一的接口,简化了各类开发过程中常用操作,使你在开发过程中,更加关注实际应用的开发,而不是把时间浪费在琐碎的接口兼容性上面,并且充分利用了各个平台独有的一些特性进行优化。 这个项目的目的,是为了使C开发更加的简单高效。 目前支持的平台有: - Windows - Macosx - Linux - Android - iOS 通过[xmake](https://github.com/xmake-io/xmake))支持各种编译模式: * Release: 正式版编译,禁用调试信息、断言,各种检测机制,启用编译器优化 * Debug: 调试模式,默认启用详细调试信息、断言、内存越界检测、内存泄漏、锁竞争分析等检测机制 * Small: 最小化编译,默认禁用所有扩展模块,启用编译器最小化优化 * Micro: 针对嵌入式平台,仅仅编译tbox微内核,仅提供最基础的跨平台接口,生成库仅64K左右(内置轻量libc接口实现) 如果你想了解更多,请参考: * [项目主页](https://tboox.org/cn) * [在线文档](https://tboox.io/#/zh-cn/getting_started) * [Github](https://github.com/tboox/tbox) * [Gitee](https://gitee.com/tboox/tbox) ## 特性 #### 流库 针对http、file、socket、data等流数据,实现统一接口进行读写,并且支持: 阻塞、非阻塞、异步 三种读写模式。 支持中间增加多层filter流进行流过滤,实现边读取,内部边进行解压、编码转换、加密等操作,极大的减少了内存使用。 主要提供以下模块: - `stream`:通用非阻塞流,用于一般的单独io处理,同时支持协程以实现异步传输。 - `transfer`:流传输器,维护两路流的传输。 - `static_stream`:针对静态数据buffer优化的静态流,用于轻量快速的数据解析。 #### 协程库 - 快速高效的协程切换支持(具体性能参考:[基准测试报告](https://tboox.org/cn/2016/10/28/benchbox-coroutine/)) - 提供跨平台支持,核心切换算法参考boost,并且对其进行重写和优化,目前支持架构:x86, x86_64, arm, arm64, mips32 - 提供channel协程间数据通信支持,基于生产、消费者模型 - 提供信号量、协程锁支持 - socket、stream都模块原生支持协程,并且可在线程和协程间进行无缝切换 - 提供http、file等基于协程的简单服务器实例,只需几百行代码,就可以从socket开始写个高性能io服务器,代码逻辑比异步回调模式更加清晰 - 同时提供stackfull, stackless两种协程模式支持,stackless协程更加的轻量(每个协程只占用几十个bytes),切换更快(会牺牲部分易用性) - 支持epoll, kqueue, poll, select 和 IOCP #### 数据库 - 统一并简化数据库操作接口,适配各种数据源,通过统一的url来自动连接打开支持的数据库,数据的枚举采用迭代器模型。 - 目前支持sqlite3以及mysql两种关系型数据库,也可自定义扩展使用其他关系型数据库。 #### xml库 - 针对xml提供DOM和SAX两种解析模式,SAX方式采用外部迭代模式,灵活性和性能更高,并且可以选择指定路径,进行解析。 - 解析过程完全基于stream,所以是高度流化的,可以实现边下载、边解压、边转码、边解析一条龙服务,使用较低的内存也可以解析大规模数据。 - 提供xml writer以支持对xml生成 #### 内存库 - 参考linux内核内存管理机制的实现,并对其进行各种改造和优化,所实现的TBOX独有的一整套内存池管理架构。 - 调试模式下,可以轻松检测并定位内存泄露、内存越界溢出、内存重叠覆盖等常见内存问题,并对整体内存的使用进行了统计和简要分析。 - 针对大块数据、小块数据、字符串数据进行了充分的利用,避免了大量外部碎片和内部碎片的产生。分配操作进行了各种优化,96%的情况下,效率都是在O(1)。 #### 容器库 - 提供哈希、链表、数组、队列、堆栈、最小最大堆等常用容器。 - 支持各种常用成员类型,在原有的容器期初上,其成员类型还可以完全自定义扩展。 - 所有容器都支持迭代器操作。 - 大部分容器都可以支持基于stream的序列化和反序列化操作。 #### 算法库 - 提供各种排序算法:冒泡排序、堆排序、快速排序、插入排序。 - 提供各种查找算法:线性遍历、二分法搜索。 - 提供各种遍历、删除、统计算法。 - 以迭代器为接口,实现算法和容器的分离,类似stl,但是c实现的,更加轻量。 #### 网络库 - 实现http客户端模块 - 实现cookies - 实现dns解析与缓存 - 实现ssl(支持openssl, polarssl, mbedtls) - 支持ipv4、ipv6 - 支持通过协程实现异步模式 #### 数学运算库 - 提供各种精度的定点运算支持 - 提供随机数生成器 #### libc库 - libc的一个轻量级实现,完全跨平台,并且针对不同架构进行了优化。 - 支持大部分字符串、宽字符串操作。 - 扩展字符串、宽字符串的各种大小写不敏感操作接口 - 扩展`memset_u16`、`memset_u32`等接口,并对其进行高度优化,尤其适合图形渲染程序 #### libm库 - libm部分接口的一个轻量级实现,以及对常用系统接口的封装。(目前只实现了部分,之后有时间会完全实现掉) - 扩展部分常用接口,增加对sqrt、log2等常用函数的整数版本计算,进行高度优化,不涉及浮点运算,适合嵌入式环境使用。 #### object库 - 轻量级类apple的CoreFoundation库,支持object、dictionary、array、string、number、date、data等常用对象,并且可以方便扩展自定义对象的序列化。 - 支持对xml、json、binary以及apple的plist(xplist/bplist)格式序列化和反序列化。 并且实现自有的binary序列化格式, 针对明文进行了简单的加密,在不影响性能的前提下,序列化后的大小比bplist节省30%。 #### 平台库 - 提供file、directory、socket、thread、time等常用系统接口 - 提供atomic、atomic64接口 - 提供高精度、低精度定时器 - 提供高性能的线程池操作 - 提供event、mutex、semaphore、spinlock等事件、互斥、信号量、自旋锁操作 - 提供获取函数堆栈信息的接口,方便调试和错误定位 - 提供跨平台动态库加载接口(如果系统支持的话) - 提供io轮询器,针对epoll, poll, select, kqueue进行跨平台封装 - 提供跨平台上下文切换接口,主要用于协程实现,切换效率非常高 #### 压缩库 - 支持zlib/zlibraw/gzip的压缩与解压(需要第三方zlib库支持)。 #### 字符编码库 - 支持utf8、utf16、gbk、gb2312、uc2、uc4 之间的互相转码,并且支持大小端格式。 #### 实用工具库 - 实现base64/32编解码 - 实现crc32、adler32、md5、sha1等常用hash算法 - 实现日志输出、断言等辅助调试工具 - 实现url编解码 - 实现位操作相关接口,支持各种数据格式的解析,可以对8bits、16bits、32bits、64bits、float、double以及任意bits的字段进行解析操作,并且同时支持大端、小端和本地端模式,并针对部分操作进行了优化,像static_stream、stream都有相关接口对其进行了封装,方便在流上进行快速数据解析。 - 实现swap16、swap32、swap64等位交换操作,并针对各个平台进行了优化。 - 实现一些高级的位处理接口,例如:位0的快速统计、前导0和前导1的快速位计数、后导01的快速位计数 - 实现单例模块,可以对静态对象、实例对象进行快速的单例封装,实现全局线程安全 - 实现option模块,对命令行参数进行解析,提供快速方便的命令行选项建立和解析操作,对于写终端程序还是很有帮助的 #### 正则表达式库 - 支持匹配和替换操作 - 支持全局、多行、大小写不敏感等模式 - 使用pcre, pcre2和posix正则库 ## 一些使用tbox的项目: * [gbox](https://github.com/tboox/gbox) * [vm86](https://github.com/tboox/vm86) * [xmake](http://www.xmake.io/cn) * [itrace](https://github.com/tboox/itrace) * [更多项目](https://github.com/tboox/tbox/wiki/%E4%BD%BF%E7%94%A8tbox%E7%9A%84%E5%BC%80%E6%BA%90%E5%BA%93) ## 编译 请先安装: [xmake](https://github.com/xmake-io/xmake) ```console # 默认直接编译当前主机平台 $ cd ./tbox $ xmake # 编译mingw平台 $ cd ./tbox $ xmake f -p mingw --sdk=/home/mingwsdk $ xmake # 编译iphoneos平台 $ cd ./tbox $ xmake f -p iphoneos $ xmake # 编译android平台 $ cd ./tbox $ xmake f -p android --ndk=xxxxx $ xmake # 交叉编译 $ cd ./tbox $ xmake f -p linux --sdk=/home/sdk #--bin=/home/sdk/bin $ xmake ``` ## 例子 ```c #include "tbox/tbox.h" int main(int argc, char** argv) { // init tbox if (!tb_init(tb_null, tb_null)) return 0; // trace tb_trace_i("hello tbox"); // init vector tb_vector_ref_t vector = tb_vector_init(0, tb_element_cstr(tb_true)); if (vector) { // insert item tb_vector_insert_tail(vector, "hello"); tb_vector_insert_tail(vector, "tbox"); // dump all items tb_for_all (tb_char_t const*, cstr, vector) { // trace tb_trace_i("%s", cstr); } // exit vector tb_vector_exit(vector); } // init stream tb_stream_ref_t stream = tb_stream_init_from_url("http://www.xxx.com/file.txt"); if (stream) { // open stream if (tb_stream_open(stream)) { // read line tb_long_t size = 0; tb_char_t line[TB_STREAM_BLOCK_MAXN]; while ((size = tb_stream_bread_line(stream, line, sizeof(line))) >= 0) { // trace tb_trace_i("line: %s", line); } } // exit stream tb_stream_exit(stream); } // wait tb_getchar(); // exit tbox tb_exit(); return 0; } ``` ## 联系方式 * 邮箱:[waruqi@gmail.com](mailto:waruqi@gmail.com) * 主页:[TBOOX开源工程](https://tboox.org/cn) * 社区:[Reddit论坛](https://www.reddit.com/r/tboox/) * QQ群:343118190 * 微信公众号:tboox-os