# Technical Documentation: joncampbell123/dosbox-x > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/joncampbell123/dosbox-x) ### Overview (README.md) **Welcome to the DOSBox-X project homepage located on GitHub.** ## Useful links - [DOSBox-X's website](https://dosbox-x.com) ([https://dosbox-x.com](https://dosbox-x.com) or [http://dosbox-x.software](http://dosbox-x.software)) - [DOSBox-X's Wiki page](https://dosbox-x.com/wiki) - The DOSBox-X project has a Discord server. Please ask in the discussion tab for a one-time invite link to join. - [Releases](https://github.com/joncampbell123/dosbox-x/releases) - [Development/Nightly builds](https://dosbox-x.com/devel-build.html) - [Install instructions](INSTALL.md) - [Build instructions](BUILD.md) ## Table of Contents - [Introduction to DOSBox-X](#introduction-to-dosbox-x) - [Notable features in DOSBox-X](#notable-features-in-dosbox-x) - [DOSBox-X supported platforms and releases](#dosbox-x-supported-platforms-and-releases) - [Compatibility with DOS programs and games](#compatibility-with-dos-programs-and-games) - [Contributing to DOSBox-X](#contributing-to-dosbox-x) - [DOSBox-X development and release pattern](#dosbox-x-development-and-release-pattern) - [Future development experiments](#future-development-experiments) - [Software security comments](#software-security-comments) - [Features that DOSBox-X is unlikely to support at this time](#features-that-dosbox-x-is-unlikely-to-support-at-this-time) - [Origin and history of the DOSBox-X project](#origin-and-history-of-the-dosbox-x-project) - [Known DOSBox-X forks](#known-dosbox-x-forks) - [Support for international language translations and keyboard layouts](#support-for-international-language-translations-and-keyboard-layouts) ## Introduction to DOSBox-X DOSBox-X is a cross-platform DOS emulator based on the DOSBox project (www.dosbox.com). Like DOSBox, it emulates a PC necessary for running many MS-DOS games and applications that simply cannot be run on modern PCs and operating systems. However, while the main focus of DOSBox is for running DOS games, DOSBox-X goes much further than this. Started as a fork of the DOSBox project, it retains compatibility with the wide base of DOS games and DOS gaming DOSBox was designed for. But it is also a platform for running DOS applications, including emulating the environments to run Windows 3.x, 9x and ME and software written for those versions of Windows. By adding official support for Windows 95, 98, ME emulation and acceleration, we hope that those old Windows games and applications could be enjoyed or used once more. Moreover, DOSBox-X adds support for DOS/V and NEC PC-98 emulations so that you can play DOS/V and PC-98 games with it. Compared with DOSBox, DOSBox-X focuses more on general emulation and accuracy. In order to help running DOS games and applications, Windows 3.x/9x/ME, as well as for the purpose of historical preservation, testing and continued DOS developments, it is our desire to implement accurate emulation, accurate enough to [help make new DOS developments possible](https://dosbox-x.com/newdosdevelopment.html) with confidence the program will run properly on actual DOS systems. DOSBox-X includes various features for different purposes (some of them ported from other projects), which are implemented as incremental changes since it was forked from DOSBox SVN Daum. DOSBox-X provides many ways to tweak and configure the DOS virtual machine, as we believe a better way to emulate the DOS platform is to give users all the options they need to emulate everything from the original IBM PC system all the way up to late 1990's configuration, whatever it takes to get your game or software package to run. Our goal is to eventually make DOSBox-X a complete emulation package that covers all pre-2000 DOS and Windows 9x based system scenarios, including peripherals, motherboards, CPUs, and all manner of hardware that was made for PC hardware of that time. Please check out the [DOSBox-X homepage](https://dosbox-x.com) for common packages of the latest release for the supported platforms, as well as screenshots of some DOS programs and games running in DOSBox-X. Also see the [INSTALL](INSTALL.md) page for DOSBox-X installation instructions and other packages, and the [Releases](https://github.com/joncampbell123/dosbox-x/releases) page for archives of all released DOSBox-X versions. For more information about DOSBox-X, such as setting up and running DOSBox-X including its usage tips, please read the user guide in the [DOSBox-X Wiki](https://dosbox-x.com/wiki). Steps for building the source code can be found in the [BUILD](BUILD.md) page. DOSBox-X is completely open-source and free of charge to use and distribute. It is released under the [GNU General Public License, version 2](COPYING). See also the [About DOSBox-X](https://dosbox-x.com/about.html) page for more information about DOSBox-X's goals and non-goals, along with some links to other projects. This project has a [Code of Conduct](CODE_OF_CONDUCT.md), please read it for general information on contributing to or getting support from the project. Brought to you by: joncampbell123 (Jonathan Campbell) ## Legal notice If you reside in a jurisdiction that enforces OS level age verification, you are only permitted to use the main-osfree branch of this project (including builds), and are prohibited from using the main (master) branch of this project. ## Notable features in DOSBox-X Although based on the DOSBox project, DOSBox-X is now a separate project because both have their own separate schedules and development priorities. For example, the main focus of DOSBox is for running DOS games whereas DOSBox-X goes way beyond this. At this time DOSBox-X already has a great number of features that do not exist in DOSBox. Examples of such features include: * GUI drop-down menu and built-in graphical configuration tool * Save and load state support (with up to 100 save slots + save files) * NEC PC-98, AX, DOS/V emulation and Chinese/Japanese/Korean support * Fully translatable user interfaces (with language files available) * Better support and compatibility with DOS applications * Support for more DOS commands and built-in external tools * Support for different ways to customize the internal Z: drive * Support for CPU types like Pentium Pro, II, III and MMX instructions * Support for IDE interfaces and improved Windows 3.x/9x emulation * Support for long filenames and FAT32 disk images (DOS 7+ features) * Support for pixel-perfect scaling output for improved image quality * Support for TrueType font (TTF) output for text-mode DOS programs * Support for printing features, either to a real or to a virtual printer * Support for starting programs to run on the host systems (-hostrun option) * Support for 3dfx Voodoo chip and Glide emulation (including Glide wrapper) * Support for cue sheets with FLAC, MP3, WAV, OGG Vorbis and Opus CD-DA tracks * Support for FluidSynth MIDI synthesizer (with sound fonts) and MT-32 emulation * Support for NE2000 Ethernet for networking features and modem phone book mapping * Support for features such as V-Sync, overscan border and stereo swapping * Plus many more.. While the vast majority of features in DOSBox-X are cross-platform, DOSBox-X does also have several notable platform-dependent features, such as Direct3D output and support for automatic drive mounting on the Windows platform. These features cannot be easily ported to other platforms. More information about DOSBox-X's features can be found in [DOSBox-X’s Feature Highlights](https://dosbox-x.com/wiki/DOSBox%E2%80%90X%E2%80%99s-Feature-Highlights) page in the [DOSBox-X Wiki](https://dosbox-x.com/wiki). DOSBox-X officially supports both SDL 1.2 and SDL 2.0; both 32-bit and 64-bit builds are also supported. ## DOSBox-X supported platforms and releases DOSBox-X is a cross-platform DOS emulator, so all major host operating systems are officially supported, including: 1. Windows (95/NT4 or higher), 32-bit and 64-bit 2. Linux (with X11), 32-bit and 64-bit 3. macOS (Mac OS X), Intel and ARM-based 64-bit 4. DOS (MS-DOS 5.0+ or compatible) Windows binaries (both 32-bit and 64-bit), Linux Flatpak or RPM packages (64-bit), macOS packages (64-bit) and DOS versions are officially released periodically, typically on the last day of a month or the first day of the next month. Please check out the [DOSBox-X homepage](https://dosbox-x.com) and the [INSTALL](INSTALL.md) page for the latest DOSBox-X packages on these platforms and further installation instructions. You can also find ZIP packages or Windows installers for all released versions and their change logs in the [Releases](https://github.com/joncampbell123/dosbox-x/releases) page. The Window installers are intended to ease the installation process, and they allow you to start DOSBox-X as soon as the installation ends. For running DOSBox-X in a real DOS system (MS-DOS or compatible), you can find the HX-DOS package that makes use of the freely-available [HX DOS Extender](https://github.com/Baron-von-Riedesel/HX). Type DOSBOX-X to run it from a DOS system. There is also the DOS LOADLIN package which can run from within DOSBox-X itself in addition to a DOS system. Note, however, that not all features of DOSBox-X that are supported in other platforms can be supported in the real DOS environment. Development (preview) builds intended for testing purposes for various platforms are also available from the [DOSBox-X Development Builds](https://dosbox-x.com/devel-build.html) page. The full source code is officially provided with each DOSBox-X release, which may be compiled to run on the above and possibly other operating systems too. You can also get the latest development source code from the repository directly. See also the [BUILD](BUILD.md) page for information on building/compiling the DOSBox-X source code. ## Compatibility with DOS programs and games With the eventual goal of being a complete DOS emulation package that covers all pre-2000 DOS and Windows 3.x/9x based hardware scenarios, we are making efforts to ensure that the vast majority of DOS games and applications will run in DOSBox-X, and these include both text-mode and graphical-mode DOS programs. Microsoft Windows versions that are largely DOS-based (such as Windows 3.x and 9x) are officially supported by DOSBox-X as well. Note that certain config settings may need to be changed from the default ones for some of these programs to work smoothly. Take a look at the [DOSBox-X Wiki](https://dosbox-x.com/wiki) for more information. Efforts are also made to aid [continued DOS developments](https://dosbox-x.com/newdosdevelopment.html) by attempting to accurately emulate the hardware, which is why DOSBox-X used to focus on the demoscene software (especially anything prior to 1996) because that era of the MS-DOS scene tends to have all manner of weird hardware tricks, bugs, and speed-sensitive issues that make them the perfect kind of stuff to test emulation accuracy against, even more so than old DOS games. But without a doubt we are also making a lot of efforts to test DOSBox-X against other DOS games and applications, as well as PC-98 programs (most of them are games). We add new features and make other improvements in every new DOSBox-X version, so its compatibility with DOS programs and games are also improving over time. If you have some issue with a specific DOS program or game, please feel free to post it in the [issue tracker](https://github.com/joncampbell123/dosbox-x/issues). ## Contributing to DOSBox-X We encourage new contributors by removing barriers to entry. Ideas and patches are always welcome, though not necessarily accepted. If you really need that feature or change, and your changes are not accepted into this main project (or you just want to mess around with the code), feel free to fork this project and make your changes in your fork. As joncampbell123 only has limited time to work on DOSBox-X, help is greatly appreciated: - Testing - Features of DOSBox-X, such as its commands and functions - The normal operation of DOS games and applications - Windows 1.0/2.x/3.x & Windows 95/98/ME guest system support - Software or hardware emulation accuracy, helped by for example demoscene software - Write more unit tests to test various functions (see existing unit tests in [tests/](https://github.com/joncampbell123/dosbox-x/tree/master/tests)) - Developments of new DOS software (possibly aided by [DOSLIB](https://github.com/joncampbell123/doslib)/[DOSLIB2](https://github.com/joncampbell123/doslib2)) - Bug fixes, patches, improvements, refinements - Suggestions, ideas, assistance of other users, and/or general conversation - Platform support (Windows, Linux, macOS, DOS, but others are welcome) - Documentation, language file translation, and software packaging - Notes regarding DOS and Win3.x/9x games, applications, hacks or weird tricks, etc. See the [CONTRIBUTING](CONTRIBUTING.md) page for more contribution guidelines. If you want to tweak or write some code and you don't know what to work on, feel free to visit the [issue tracker](https://github.com/joncampbell123/dosbox-x/issues) to get some ideas. For more descriptions on the source code, please take a look at the [DOSBox-X source code description](README.source-code-description) page. Information on building on the source code can be found in the [BUILD](BUILD.md) page. Information about the debugger is also available in the [DOSBox-X Debugger](README.debugger) page. See also the [CREDITS](CREDITS.md) page for crediting information. ## DOSBox-X development and release pattern In order to make DOSBox-X's development process more smooth, we have implemented a general development/release pattern for DOSBox-X. The current release pattern for DOSBox-X is as follows: New DOSBox-X versions are made public at the start (typically on the first day) of each month, including the source code and binary releases. Then the DOSBox-X developments will be re-opened for new features, pull requests, etc. There will be no new features added 6 days before the end of the month, but only bug fixes. The last day of the month is DOSBox-X’s build day to compile for binary releases the first of the next month, so there will be no source code changes on this day including pull requests or bug fixes. For example, suppose August is the current month - August 25th will be the day pull requests will be ignored unless only bug fixes. August 31st (the last day of August) will be DOSBox-X build day. This is DOSBox-X’s official release pattern, although it may change later. ## Future development experiments Scattered experiments and small projects are in [experiments/](https://github.com/joncampbell123/dosbox-x/tree/master/experiments) as proving grounds for future revisions to DOSBox-X and its codebase. These experiments may or may not make it into future revisions or the next version. Comments are welcome on the experiments, to help improve the code overall. There are also patches in [patch-integration/](https://github.com/joncampbell123/dosbox-x/tree/master/patch-integration) for possible feature integrations in the future. We have already integrated many community-developed patches into DOSBox-X in the past. See also [General TODO.txt](docs/PLANS/General%20TODO.txt) for some plans of future DOSBox-X developments. ## Software security comments DOSBox-X cannot claim to be a "secure" application. It contains a lot of code designed for performance, not security. There may be vulnerabilities, bugs, and flaws in the emulation that could permit malicious DOS executables within to cause problems or exploit bugs in the emulator to cause harm. There is no guarantee of complete containment by DOSBox-X of the guest operating system or application. If security is a priority, then: Do not use DOSBox-X on a secure system. Do not run DOSBox-X as root or Administrator. If you need to use DOSBox-X, run it under a lesser privileged user, in a chroot jail or sandbox, or enable DOSBox-X's secure mode with its command-line option ``-securemode``, which disables commands that may allow access to the host system. If your Linux distribution has it enabled, consider using the auditing system to limit what the DOSBox-X executable is allowed to do. ## Features that DOSBox-X is unlikely to support at this time DOSBox-X aims to be a fully-featured DOS emulation package, but there are some things the design as implemented now cannot accommodate. * Pentium 4 or higher CPU level emulation. DOSBox-X contains code only to emulate the 8086 through the Pentium III. Real DOS systems (MS-DOS and compatibles) also work best with these CPUs. If Pentium 4 or higher emulation is desired, consider using a PC emulator like Bochs or QEMU instead. DOSBox-X may eventually develop Pentium 4 emulation, if wanted by the DOSBox-X community in general. * Emulation of PC hardware 2001 or later. The official cutoff for DOSBox-X is 2001, when updated "PC 2001" specifications from Microsoft mandated the removal of the ISA slots from motherboards. The focus is on implementing hardware emulation for hardware made before that point. Contributors are free to focus on emulating hardware within the time frame between 1980 and 2000/2001 of their choice. * Windows guest emulation, Windows Vista or later. DOSBox-X emulation, in terms of running Windows in DOSBox-X, will focus primarily on Windows 1.0 through Windows ME (Millennium Edition), and then on Windows NT through Windows XP. Windows Vista and later versions are not a priority and will not be considered at this time. These versions of Windows are not based on DOS. If you need to run Windows XP and later, please consider using QEMU, Bochs, VirtualBox, or VMware. * Any MS-DOS system other than IBM PC/XT/AT, AX, Tandy, PCjr, and PC-98. Only the above listed systems will be considered for development in DOSBox-X. This restriction prevents stretching of the codebase to an unmanageable level and helps keep the code base organized. It would be easier on myself and the open source community if developers could focus on emulating their platform of interest in parallel instead of putting everything into one project that, most likely, will do a worse job overall emulating all platforms. However, if adding emulation of the system requires only small minimal changes, then the new system in question may be considered. You are strongly encouraged to fork this project and implement your own variation if you need to develop MS-DOS emulation for any other system or console. In doing that, you gain the complete freedom to focus on implementing the particular MS-DOS based system of interest, and if desired, the ability to strip away conflicting IBM PC/XT/AT emulation and unnecessary code to keep your branch's code manageable and maintainable. If you are starting a fork, feel free to let me know where your fork is and what system it is emulating, so I can list it in this README file for others seeking emulation of that system. To help, I have added machine and video mode enumerations as "stubs" to provide a starting point for your branch's implementation of the platform. A stub implemented so far is "FM Towns emulation" (```machine=fm_towns```). * Cycle-accurate timing of x86 instructions and execution. Instructions generally run one per cycle in DOSBox-X, except for I/O and memory access. If accurate emulation of cycles per instruction is needed, please consider using PCem, 86Box, or VARCem instead. * Full precision floating point emulation. Unless using the dynamic core, DOSBox and DOSBox-X emulate the FPU registers using the "double" 64-bit floating point data type. The Intel FPU registers are 80-bit "extended precision" floating point values, not 64-bit double precision, so this is effectively 12 bits of precision loss and 5 bits of range loss (64 to 53 mantissa bits and 16 to 11 exponent bits). This slight loss of precision is perfectly fine considering DOSBox's original goal in supporting DOS games, but may cause problems in other cases that need the full precision. It is known at this time that this lack of precision is enough to cause otherwise straightforward comparisons against integers to fail in DOS applications originally written in QBasic or Turbo Basic. There are such DOS games written that check their file size using a floating point compare that will fail in this manner. To run these games, you will need to disable FPU emulation (```fpu=false```) to force the QBasic/TurboBasic runtime to use software emulation instead. ## Origin and history of the DOSBox-X project DOSBox-X started as a fork of the original DOSBox project sometime in mid-2011. It was started out of a desire to improve the emulator without having to fight with or worry about submitting patches upstream. As its developers have made it clear, DOSBox's main focus is on DOS games. This is evident by the fact that much of the code is somewhat accurate code with kludges to make DOS games run, instead of focusing on the actual behaviors of real DOS systems. Jonathan Campbell, the DOSBox-X project maintainer wanted to make various changes to the source code, but many of them were non-game related, and thus were unlikely to be accepted by the DOSBox developers. Since then, Jonathan Campbell has been modifying the source code over time to improve emulation, fix bugs, and resolve incompatibilities with Windows 95 through ME. He has added options so that DOSBox-X by default can emulate a wider variety of configurations more accurately, while allowing the user to enable various techniques or hacks if needed to run their favorite DOS games or programs. He has also been cleaning up and organizing the code to improve stability and portability where possible. The original DOSBox project was not written by one programmer. It has been under development since late 2000 with patches, fixes, and improvements from members all over the Vogons forums. Despite not having a major official release since DOSBox 0.74 over 10 years ago, the project is still in semi-active development today in the form of DOSBox SVN. Meanwhile, some of the changes themselves incorporated code from other projects. Some features and improvements in DOSBox-X also came from another branch of DOSBox known as [DOSBox SVN Daum](http://ykhwong.x-y.net) which itself incorporated features from the original DOSBox project, DOSBox-X, and many experimental patches. Although the Daum branch seems to be dead, the features borrowed from it still exists in DOSBox-X. Later on, DOSBox-X also incorporated several features and improvements from other projects such as [DOSBox ECE](https://dosboxece.yesterplay.net/), [DOSBox Staging](https://dosbox-staging.github.io/), [DOSVAX](http://radioc.web.fc2.com/soflib/dosvax/dosvax.htm)/[DOSVAXJ3](https://www.nanshiki.co.jp/software/dosvaxj3.html), and [vDosPlus](http://www.vdosplus.org/). The DOSBox-X project is also helped by its other developers and contributors such as Wengier, aybe, Allofich, and rderooy, who have done significant work to improve the DOSBox-X project, including adding new features, fixing bugs, creating the documentation, maintaining the website, and porting code from other projects. See also the [CREDITS](CREDITS.md) page for crediting of the source code. ## Known DOSBox-X forks * DOSBox-X Emscripten port (runnable in a web browser) by Yksoft1 Significant changes are made in order to run efficiently within the web browser when compiled using LLVM/Emscripten. These significant changes require dropping some useful features (including the menus) but are required for performance. URL: https://github.com/yksoft1/dosbox-x-vanilla-sdl/tree/emscripten (look for clone URL and use the emscripten branch) * DOSBox-X-App (for Windows and macOS) by emendelson DOSBox-X-App is a slightly customized version of DOSBox-X, combined with external programs and commands that make it easy to print and create PDFs from DOS applications. It is customized for use with applications, not games. URL: http://www.columbia.edu/~em36/dosboxapp.html * DOSBoxWP (for WordPerfect for DOS) by emendelson DOSBoxWP is a customized version of DOSBox-X targeted for users of WordPerfect for DOS. URL (Windows): http://www.columbia.edu/~em36/wpdos/dosboxwp.html URL (macOS): http://www.columbia.edu/~em36/wpdos/wpdosboxmac.html * Win31DOSBox (Windows 3.1 for 64-bit Windows) by emendelson Win31DOSBox aims to be an easy method of running Windows 3.x software for 64-bit Windows systems. The system uses a custom build of DOSBox-X when running Windows 3.1x. URL: http://www.columbia.edu/~em36/win31dosbox.html ## Support for international language translations and keyboard layouts ### Translations DOSBox-X displays English as the default language, and uses the U.S. code page (437) by default, just like DOSBox. All messages displayed by DOSBox-X are in English with the default setting. DOSBox-X does support the feature to change the display messages with the use of language files. The language files control all visible output of the internal commands and the internal DOS, as well as the text in DOSBox-X's drop-down menus. Language files can be found in the `languages` directory of your DOSBox-X installation or https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/ . Language files currently available are: [Chinese (Simplified)](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/zh/zh_CN.lng), [Chinese (Traditional)](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/zh/zh_TW.lng), [Dutch](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/nl/nl_NL.lng), [French](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/fr/fr_FR.lng), [German](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/de/de_DE.lng), [Hungarian](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/hu/hu_HU.lng), [Italian](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/it/it_IT.lng), [Japanese](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/ja/ja_JP.lng), [Korean](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/ko/ko_KR.lng), [Portuguese (Brazilian)](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/pt/pt_BR.lng), [Russian](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/ru/ru_RU.lng), [Spanish](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/es/es_ES.lng), [Turkish](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/tr/tr_TR.lng) If you are a speaker of any language not included above, you are encouraged to create additional language files by translating the messages to your language using the [English (Template)](https://github.com/joncampbell123/dosbox-x/blob/master/contrib/translations/en/en_US.lng) or generate one by `CONFIG -wl filename` command. ### Keyboard layouts DOSBox-X supports various keyboard layouts by `keyboardlayout` option in the .conf file or `KEYB` command. On Windows, DOSBox-X will try to match the layout with your physical keyboard when `keyboardlayout=auto`. On other platforms, the keyboard will work as US keyboard if not set otherwise. The SDL1 version additionally requires use of scancodes by enabling `usescancodes` option, when using non-US keyboards. The default setting (`usescancodes=auto`) should work in most cases, and this setting is NOT required and ignored for SDL2 versions. If you find that a keyboard layout is not yet supported by DOSBox-X, in order to add additional layouts for use with DOSBox-X, please see file [README.keyboard-layout-handling](README.keyboard-layout-handling) on how to do so as a developer. ### Codepage settings You need to switch to the appropriate codepage in order to display various characters for your language. You can set the codepage by `KEYB` or `CHCP` command. Type `KEYB /?` or `CHCP /?` for details. For further information on international support and regional settings of DOSBox-X, such as steps to create DOSBox-X language files or use external keyboard files in DOSBox-X, as well as support for the Euro symbol and country-specific date and time formats, please look at the guide [Regional settings in DOSBox-X](https://dosbox-x.com/wiki/Guide%3ARegional-settings-in-DOSBox%E2%80%90X) in the [DOSBox-X Wiki](https://dosbox-x.com/wiki). For more information on East Asian (Chinese/Japanese/Korean) language support, see the [East Asian language and system support](https://dosbox-x.com/wiki/Guide%3AEast-Asian-language-support-in-DOSBox%E2%80%90X) guide page. --- ### Vs Libpdcurses Doc README (`vs/libpdcurses/doc/README.md`) PDCurses Documentation ====================== Some plain text documents to provide a basic overview of PDCurses and details for specific platforms, along with a program to combine them with the "man page" sections from the source code files, into "PDCurses.md". Building -------- - Type "make". ("make clean" to remove the built files.) A Unix-like environment is required for the Makefile. This is called automatically when doing a top-level make of the X11 port. Distribution Status ------------------- Public Domain, except for manext.c, which is GPL 2+. --- ### Vs Libpdcurses Doc Intro (`vs/libpdcurses/doc/intro.md`) PDCurses User's Guide ===================== Curses Overview --------------- The X/Open Curses Interface Definition describes a set of C-Language functions that provide screen-handling and updating, which are collectively known as the curses library. The curses library permits manipulation of data structures called windows which may be thought of as two-dimensional arrays of characters representing all or part of a terminal's screen. The windows are manipulated using a procedural interface described elsewhere. The curses package maintains a record of what characters are on the screen. At the most basic level, manipulation is done with the routines move() and addch() which are used to "move" the curses around and add characters to the default window, stdscr, which represents the whole screen. An application may use these routines to add data to the window in any convenient order. Once all data have been added, the routine refresh() is called. The package then determines what changes have been made which affect the screen. The screen contents are then changed to reflect those characters now in the window, using a sequence of operations optimized for the type of terminal in use. At a higher level routines combining the actions of move() and addch() are defined, as are routines to add whole strings and to perform format conversions in the manner of printf(). Interfaces are also defined to erase the entire window and to specify the attributes of individual characters in the window. Attributes such as inverse video, underline and blink can be used on a per-character basis. New windows can be created by allowing the application to build several images of the screen and display the appropriate one very quickly. New windows are created using the routine newwin(). For each routine that manipulates the default window, stdscr, there is a corresponding routine prefixed with w to manipulate the contents of a specified window; for example, move() and wmove(). In fact, move(...) is functionally equivalent to wmove( stdscr, ...). This is similar to the interface offered by printf(...) and fprintf(stdout, ...). Windows do not have to correspond to the entire screen. It is possible to create smaller windows, and also to indicate that the window is only partially visible on the screen. Furthermore, large windows or pads, which are bigger than the actual screen size, may be created. Interfaces are also defined to allow input character manipulation and to disable and enable many input attributes: character echo, single character input with or without signal processing (cbreak or raw modes), carriage returns mapping to newlines, screen scrolling, etc. Data Types and the \ Header -------------------------------------- The data types supported by curses are described in this section. As the library supports a procedural interface to the data types, actual structure contents are not described. All curses data are manipulated using the routines provided. ### The \ Header The \ header defines various constants and declares the data types that are available to the application. ### Data Types The following data types are declared: WINDOW * pointer to screen representation SCREEN * pointer to terminal descriptor bool boolean data type chtype representation of a character in a window cchar_t the wide-character equivalent of chtype attr_t for WA_-style attributes The actual WINDOW and SCREEN objects used to store information are created by the corresponding routines and a pointer to them is provided. All manipulation is through that pointer. ### Variables The following variables are defined: LINES number of lines on terminal screen COLS number of columns on terminal screen stdscr pointer to the default screen window curscr pointer to the current screen image SP pointer to the current SCREEN struct Mouse_status status of the mouse COLORS number of colors available COLOR_PAIRS number of color pairs available TABSIZE size of one TAB block acs_map[] alternate character set map ttytype[] terminal name/description ### Constants The following constants are defined: #### General FALSE boolean false value TRUE boolean true value NULL zero pointer value ERR value returned on error condition OK value returned on successful completion #### Video Attributes Normally, attributes are a property of the character. For chtype: A_ALTCHARSET use the alternate character set A_BLINK bright background or blinking A_BOLD bright foreground or bold A_DIM half bright -- no effect in PDCurses A_INVIS invisible -- no effect in PDCurses A_ITALIC italic A_LEFT line along the left edge A_PROTECT protected -- no effect in PDCurses A_REVERSE reverse video A_RIGHT line along the right edge A_STANDOUT terminal's best highlighting mode A_UNDERLINE underline A_ATTRIBUTES bit-mask to extract attributes A_CHARTEXT bit-mask to extract a character A_COLOR bit-mask to extract a color-pair Not all attributes will work on all terminals. A_ITALIC is not standard, but is shared with ncurses. For attr_t: WA_ALTCHARSET same as A_ALTCHARSET WA_BLINK same as A_BLINK WA_BOLD same as A_BOLD WA_DIM same as A_DIM WA_INVIS same as A_INVIS WA_ITALIC same as A_ITALIC WA_LEFT same as A_LEFT WA_PROTECT same as A_PROTECT WA_REVERSE same as A_REVERSE WA_RIGHT same as A_RIGHT WA_STANDOUT same as A_STANDOUT WA_UNDERLINE same as A_UNDERLINE The following are also defined, for compatibility, but currently have no effect in PDCurses: A_HORIZONTAL, A_LOW, A_TOP, A_VERTICAL and their WA_* equivalents. ### The Alternate Character Set For use in chtypes and with related functions. These are a portable way to represent graphics characters on different terminals. VT100-compatible symbols -- box characters: ACS_ULCORNER upper left box corner ACS_LLCORNER lower left box corner ACS_URCORNER upper right box corner ACS_LRCORNER lower right box corner ACS_RTEE right "T" ACS_LTEE left "T" ACS_BTEE bottom "T" ACS_TTEE top "T" ACS_HLINE horizontal line ACS_VLINE vertical line ACS_PLUS plus sign, cross, or four-corner piece VT100-compatible symbols -- other: ACS_S1 scan line 1 ACS_S9 scan line 9 ACS_DIAMOND diamond ACS_CKBOARD checkerboard -- 50% grey ACS_DEGREE degree symbol ACS_PLMINUS plus/minus sign ACS_BULLET bullet Teletype 5410v1 symbols -- these are defined in SysV curses, but are not well-supported by most terminals. Stick to VT100 characters for optimum portability: ACS_LARROW left arrow ACS_RARROW right arrow ACS_DARROW down arrow ACS_UARROW up arrow ACS_BOARD checkerboard -- lighter (less dense) than ACS_CKBOARD ACS_LANTERN lantern symbol ACS_BLOCK solid block That goes double for these -- undocumented SysV symbols. Don't use them: ACS_S3 scan line 3 ACS_S7 scan line 7 ACS_LEQUAL less than or equal ACS_GEQUAL greater than or equal ACS_PI pi ACS_NEQUAL not equal ACS_STERLING pounds sterling symbol Box character aliases: ACS_BSSB same as ACS_ULCORNER ACS_SSBB same as ACS_LLCORNER ACS_BBSS same as ACS_URCORNER ACS_SBBS same as ACS_LRCORNER ACS_SBSS same as ACS_RTEE ACS_SSSB same as ACS_LTEE ACS_SSBS same as ACS_BTEE ACS_BSSS same as ACS_TTEE ACS_BSBS same as ACS_HLINE ACS_SBSB same as ACS_VLINE ACS_SSSS same as ACS_PLUS For cchar_t and wide-character functions, WACS_ equivalents are also defined. ### Colors For use with init_pair(), color_set(), etc.: COLOR_BLACK COLOR_BLUE COLOR_GREEN COLOR_CYAN COLOR_RED COLOR_MAGENTA COLOR_YELLOW COLOR_WHITE Use these instead of numeric values. The definition of the colors depends on the implementation of curses. ### Input Values The following constants might be returned by getch() if keypad() has been enabled. Note that not all of these may be supported on a particular terminal: KEY_BREAK break key KEY_DOWN the four arrow keys KEY_UP KEY_LEFT KEY_RIGHT KEY_HOME home key (upward+left arrow) KEY_BACKSPACE backspace KEY_F0 function keys; space for 64 keys is reserved KEY_F(n) (KEY_F0+(n)) KEY_DL delete line KEY_IL insert line KEY_DC delete character KEY_IC insert character KEY_EIC exit insert character mode KEY_CLEAR clear screen KEY_EOS clear to end of screen KEY_EOL clear to end of line KEY_SF scroll 1 line forwards KEY_SR scroll 1 line backwards (reverse) KEY_NPAGE next page KEY_PPAGE previous page KEY_STAB set tab KEY_CTAB clear tab KEY_CATAB clear all tabs KEY_ENTER enter or send KEY_SRESET soft (partial) reset KEY_RESET reset or hard reset KEY_PRINT print or copy KEY_LL home down or bottom (lower left) KEY_A1 upper left of virtual keypad KEY_A3 upper right of virtual keypad KEY_B2 center of virtual keypad KEY_C1 lower left of virtual keypad KEY_C3 lower right of virtual keypad KEY_BTAB Back tab key KEY_BEG Beginning key KEY_CANCEL Cancel key KEY_CLOSE Close key KEY_COMMAND Cmd (command) key KEY_COPY Copy key KEY_CREATE Create key KEY_END End key KEY_EXIT Exit key KEY_FIND Find key KEY_HELP Help key KEY_MARK Mark key KEY_MESSAGE Message key KEY_MOVE Move key KEY_NEXT Next object key KEY_OPEN Open key KEY_OPTIONS Options key KEY_PREVIOUS Previous object key KEY_REDO Redo key KEY_REFERENCE Reference key KEY_REFRESH Refresh key KEY_REPLACE Replace key KEY_RESTART Restart key KEY_RESUME Resume key KEY_SAVE Save key KEY_SBEG Shifted beginning key KEY_SCANCEL Shifted cancel key KEY_SCOMMAND Shifted command key KEY_SCOPY Shifted copy key KEY_SCREATE Shifted create key KEY_SDC Shifted delete char key KEY_SDL Shifted delete line key KEY_SELECT Select key KEY_SEND Shifted end key KEY_SEOL Shifted clear line key KEY_SEXIT Shifted exit key KEY_SFIND Shifted find key KEY_SHELP Shifted help key KEY_SHOME Shifted home key KEY_SIC Shifted input key KEY_SLEFT Shifted left arrow key KEY_SMESSAGE Shifted message key KEY_SMOVE Shifted move key KEY_SNEXT Shifted next key KEY_SOPTIONS Shifted options key KEY_SPREVIOUS Shifted prev key KEY_SPRINT Shifted print key KEY_SREDO Shifted redo key KEY_SREPLACE Shifted replace key KEY_SRIGHT Shifted right arrow KEY_SRSUME Shifted resume key KEY_SSAVE Shifted save key KEY_SSUSPEND Shifted suspend key KEY_SUNDO Shifted undo key KEY_SUSPEND Suspend key KEY_UNDO Undo key The virtual keypad is arranged like this: A1 up A3 left B2 right C1 down C3 This list is incomplete -- see curses.h for the full list, and use the testcurs demo to see what values are actually returned. The above are just the keys required by X/Open. In particular, PDCurses defines many CTL_ and ALT_ combinations; these are not portable. -------------------------------------------------------------------------- --- ### Vs Sdl2 Docs README (`vs/sdl2/docs/README.md`) # Simple DirectMedia Layer https://www.libsdl.org/ Simple DirectMedia Layer is a cross-platform development library designed to provide low level access to audio, keyboard, mouse, joystick, and graphics hardware via OpenGL and Direct3D. It is used by video playback software, emulators, and popular games including Valve's award winning catalog and many Humble Bundle games. SDL officially supports Windows, macOS, Linux, iOS, and Android. Support for other platforms may be found in the source code. SDL is written in C, works natively with C++, and there are bindings available for several other languages, including C# and Python. This library is distributed under the zlib license, which can be found in the file "LICENSE.txt". The best way to learn how to use SDL is to check out the header files in the "include" subdirectory and the programs in the "test" subdirectory. The header files and test programs are well commented and always up to date. More documentation and FAQs are available online at [the wiki](http://wiki.libsdl.org/) - [Android](README-android.md) - [CMake](README-cmake.md) - [DirectFB](README-directfb.md) - [DynAPI](README-dynapi.md) - [Emscripten](README-emscripten.md) - [GDK](README-gdk.md) - [Gesture](README-gesture.md) - [Git](README-git.md) - [iOS](README-ios.md) - [Linux](README-linux.md) - [macOS](README-macos.md) - [OS/2](README-os2.md) - [Native Client](README-nacl.md) - [Pandora](README-pandora.md) - [Supported Platforms](README-platforms.md) - [Porting information](README-porting.md) - [PSP](README-psp.md) - [PS2](README-ps2.md) - [Raspberry Pi](README-raspberrypi.md) - [Touch](README-touch.md) - [Versions](README-versions.md) - [WinCE](README-wince.md) - [Windows](README-windows.md) - [WinRT](README-winrt.md) - [PSVita](README-vita.md) - [Nokia N-Gage](README-ngage.md) If you need help with the library, or just want to discuss SDL related issues, you can join the [SDL Discourse](https://discourse.libsdl.org/), which can be used as a web forum or a mailing list, at your preference. If you want to report bugs or contribute patches, please submit them to [our bug tracker](https://github.com/libsdl-org/SDL/issues) Enjoy! Sam Lantinga --- ### BDA BIOS Data Area PC Memory Map (`docs/BDA - BIOS Data Area - PC Memory Map.txt`) http://stanislavs.org/helppc/bios_data_area.html BDA - BIOS Data Area - PC Memory Map Address Size Description 00:00 256dwords Interrupt vector table 30:00 256bytes Stack area used during post and bootstrap 40:00 word COM1 port address 40:02 word COM2 port address 40:04 word COM3 port address 40:06 word COM4 port address 40:08 word LPT1 port address 40:0A word LPT2 port address 40:0C word LPT3 port address 40:0E word LPT4 port address (except PS/2) Extended BIOS Data Area segment (PS/2, see EBDA) 40:10 2 bytes Equipment list flags (see INT 11) |7|6|5|4|3|2|1|0| 40:10 (value in INT 11 register AL) | | | | | | | `- IPL diskette installed | | | | | | `-- math coprocessor | | | | |-+--- old PC system board RAM < 256K | | | | | `-- pointing device installed (PS/2) | | | | `--- not used on PS/2 | | `------ initial video mode `--------- # of diskette drives, less 1 |7|6|5|4|3|2|1|0| 40:11 (value in INT 11 register AH) | | | | | | | `- 0 if DMA installed | | | | `------ number of serial ports | | | `------- game adapter | | `-------- not used, internal modem (PS/2) `----------- number of printer ports 40:12 byte PCjr: infrared keyboard link error count 40:13 word Memory size in Kbytes (see INT 12) 40:15 byte Reserved 40:16 byte PS/2 BIOS control flags 40:17 byte Keyboard flag byte 0 (see KB FLAGS) |7|6|5|4|3|2|1|0| keyboard flag byte 0 | | | | | | | `--- right shift key depressed | | | | | | `---- left shift key depressed | | | | | `----- CTRL key depressed | | | | `------ ALT key depressed | | | `------- scroll-lock is active | | `-------- num-lock is active | `--------- caps-lock is active `---------- insert is active 40:18 byte Keyboard flag byte 1 (see KB FLAGS) |7|6|5|4|3|2|1|0| keyboard flag byte | | | | | | | `--- left CTRL key depressed | | | | | | `---- left ALT key depressed | | | | | `----- system key depressed and held | | | | `------ suspend key has been toggled | | | `------- scroll lock key is depressed | | `-------- num-lock key is depressed | `--------- caps-lock key is depressed `---------- insert key is depressed 40:19 byte Storage for alternate keypad entry 40:1A word Offset from 40:00 to keyboard buffer head 40:1C word Offset from 40:00 to keyboard buffer tail 40:1E 32bytes Keyboard buffer (circular queue buffer) 40:3E byte Drive recalibration status |7|6|5|4|3|2|1|0| drive recalibration status | | | | | | | `-- 1=recalibrate drive 0 | | | | | | `--- 1=recalibrate drive 1 | | | | | `---- 1=recalibrate drive 2 | | | | `----- 1=recalibrate drive 3 | `---------- unused `----------- 1=working interrupt flag 40:3F byte Diskette motor status |7|6|5|4|3|2|1|0| diskette motor status | | | | | | | `-- 1=drive 0 motor on | | | | | | `--- 1=drive 1 motor on | | | | | `---- 1=drive 2 motor on | | | | `----- 1=drive 3 motor on | `---------- unused `----------- 1=write operation 40:40 byte Motor shutoff counter (decremented by INT 8) 40:41 byte Status of last diskette operation (see INT 13,1) |7|6|5|4|3|2|1|0| status of last diskette operation | | | | | | | `--- invalid diskette command | | | | | | `---- diskette address mark not found | | | | | `----- sector not found | | | | `------ diskette DMA error | | | `------- CRC check / data error | | `-------- diskette controller failure | `--------- seek to track failed `---------- diskette time-out 40:42 7 bytes NEC diskette controller status (see FDC) 40:49 byte Current video mode (see VIDEO MODE) 40:4A word Number of screen columns 40:4C word Size of current video regen buffer in bytes 40:4E word Offset of current video page in video regen buffer 40:50 8 words Cursor position of pages 1-8, high order byte=row low order byte=column; changing this data isn't reflected immediately on the display 40:60 byte Cursor ending (bottom) scan line (don't modify) 40:61 byte Cursor starting (top) scan line (don't modify) 40:62 byte Active display page number 40:63 word Base port address for active 6845 CRT controller 3B4h = mono, 3D4h = color 40:65 byte 6845 CRT mode control register value (port 3x8h) EGA/VGA values emulate those of the MDA/CGA 40:66 byte CGA current color palette mask setting (port 3d9h) EGA and VGA values emulate the CGA 40:67 dword CS:IP for 286 return from protected mode dword Temp storage for SS:SP during shutdown dword Day counter on all products after AT dword PS/2 Pointer to reset code with memory preserved 5 bytes Cassette tape control (before AT) 40:6C dword Daily timer counter, equal to zero at midnight; incremented by INT 8; read/set by INT 1A 40:70 byte Clock rollover flag, set when 40:6C exceeds 24hrs 40:71 byte BIOS break flag, bit 7 is set if Ctrl-Break was *ever* hit; set by INT 9 40:72 word Soft reset flag via Ctl-Alt-Del or JMP FFFF:0 1234h Bypass memory tests & CRT initialization 4321h Preserve memory 5678h System suspend 9ABCh Manufacturer test ABCDh Convertible POST loop ????h many other values are used during POST 40:74 byte Status of last hard disk operation (see INT 13,1) 40:75 byte Number of hard disks attached 40:76 byte XT fixed disk drive control byte 40:77 byte Port offset to current fixed disk adapter 40:78 4 bytes Time-Out value for LPT1,LPT2,LPT3(,LPT4 except PS/2) 40:7C 4 bytes Time-Out value for COM1,COM2,COM3,COM4 40:80 word Keyboard buffer start offset (seg=40h,BIOS 10-27-82) 40:82 word Keyboard buffer end offset (seg=40h,BIOS 10-27-82) 40:84 byte Rows on the screen (less 1, EGA+) 40:85 word Point height of character matrix (EGA+) byte PCjr: character to be repeated if the typematic repeat key takes effect 40:86 byte PCjr: initial delay before repeat key action begins 40:87 byte PCjr: current Fn function key number byte Video mode options (EGA+) |7|6|5|4|3|2|1|0| Video mode options (EGA+) | | | | | | | `-- 1=alphanumeric cursor emulation enabled | | | | | | `--- 1=video subsystem attached to monochrome | | | | | `---- reserved | | | | `----- 1=video subsystem is inactive | | | `------ reserved | `--------- video RAM 00-64K 10-192K 01-128K 11-256K `---------- video mode number passed to INT 10, function 0 40:88 byte PCjr: third keyboard status byte EGA feature bit switches, emulated on VGA |7|6|5|4|3|2|1|0| EGA feature bit switches (EGA+) | | | | | | | `-- EGA SW1 config (1=off) | | | | | | `--- EGA SW2 config (1=off) | | | | | `---- EGA SW3 config (1=off) | | | | `----- EGA SW4 config (1=off) | | | `------ Input FEAT0 (ISR0 bit 5) after output on FCR0 | | `------- Input FEAT0 (ISR0 bit 6) after output on FCR0 | `-------- Input FEAT1 (ISR0 bit 5) after output on FCR1 `--------- Input FEAT1 (ISR0 bit 6) after output on FCR1 40:89 byte Video display data area (MCGA and VGA) |7|6|5|4|3|2|1|0| Video display data area (MCGA and VGA) | | | | | | | `-- 1=VGA is active | | | | | | `--- 1=gray scale is enabled | | | | | `---- 1=using monochrome monitor | | | | `----- 1=default palette loading is disabled | | | `------ see table below | | `------- reserved | `-------- 1=display switching enabled `--------- alphanumeric scan lines (see table below) Bit7 Bit4 Scan Lines 0 0 350 line mode 0 1 400 line mode 1 0 200 line mode 1 1 reserved 40:8A byte Display Combination Code (DCC) table index (EGA+) 40:8B byte Last diskette data rate selected |7|6|5|4|3|2|1|0| last diskette data rate selected | | | | `--------- reserved | | `------------ last floppy drive step rate selected `-------------- last floppy data rate selected Data Rate Step Rate 00 500K bps 00 step rate time of 0C 01 300K bps 01 step rate time of 0D 10 250K bps 10 step rate time of 0A 11 reserved 11 reserved 40:8C byte Hard disk status returned by controller 40:8D byte Hard disk error returned by controller 40:8E byte Hard disk interrupt control flag(bit 7=working int) 40:8F byte Combination hard/floppy disk card when bit 0 set 40:90 4 bytes Drive 0,1,2,3 media state |7|6|5|4|3|2|1|0| drive media state (4 copies) | | | | | `------- drive/media state (see below) | | | | `------- reserved | | | `------- 1=media/drive established | | `------- double stepping required `--------- data rate: 00=500K bps 01=300K bps 10=250K bps 11=reserved Bits 210 Drive Media State 000 360Kb diskette/360Kb drive not established 001 360Kb diskette/1.2Mb drive not established 010 1.2Mb diskette/1.2Mb drive not established 011 360Kb diskette/360Kb drive established 100 360Kb diskette/1.2Mb drive established 101 1.2Mb diskette/1.2Mb drive established 110 Reserved 111 None of the above 40:94 byte Track currently seeked to on drive 0 40:95 byte Track currently seeked to on drive 1 40:96 byte Keyboard mode/type |7|6|5|4|3|2|1|0| Keyboard mode/type | | | | | | | `--- last code was the E1 hidden code | | | | | | `---- last code was the E0 hidden code | | | | | `----- right CTRL key depressed | | | | `------ right ALT key depressed | | | `------- 101/102 enhanced keyboard installed | | `-------- force num-lock if Rd ID & KBX | `--------- last char was first ID char `---------- read ID in process 40:97 byte Keyboard LED flags |7|6|5|4|3|2|1|0| Keyboard LED flags | | | | | | | `--- scroll lock indicator | | | | | | `---- num-lock indicator | | | | | `----- caps-lock indicator | | | | `------ circus system indicator | | | `------- ACK received | | `-------- re-send received flag | `--------- mode indicator update `---------- keyboard transmit error flag 40:98 dword Pointer to user wait complete flag 40:9C dword User wait Time-Out value in microseconds 40:A0 byte RTC wait function flag |7|6|5|4|3|2|1|0| INT 15,86 RTC wait function flag | | | | | | | `--- 1= wait pending | `-------------- not used `--------------- 1=INT 15,86 wait time elapsed 40:A1 byte LANA DMA channel flags 40:A2 2 bytes Status of LANA 0,1 40:A4 dword Saved hard disk interrupt vector 40:A8 dword BIOS Video Save/Override Pointer Table address (see VIDEO TABLES) 40:AC 8 bytes Reserved 40:B4 byte Keyboard NMI control flags (convertible) 40:B5 dword Keyboard break pending flags (convertible) 40:B9 byte Port 60 single byte queue (convertible) 40:BA byte Scan code of last key (convertible) 40:BB byte NMI buffer head pointer (convertible) 40:BC byte NMI buffer tail pointer (convertible) 40:BD 16bytes NMI scan code buffer (convertible) 40:CE word Day counter (convertible and after) 40:F0 16bytes Intra-Applications Communications Area (IBM Technical Reference incorrectly locates this at 50:F0-50:FF) Address Size Description (BIOS/DOS Data Area) 50:00 byte Print screen status byte 00 = PrtSc not active, 01 = PrtSc in progress FF = error 50:01 3 bytes Used by BASIC 50:04 byte DOS single diskette mode flag, 0=A:, 1=B: 50:05 10bytes POST work area 50:0F byte BASIC shell flag; set to 2 if current shell 50:10 word BASICs default DS value (DEF SEG) 50:12 dword Pointer to BASIC INT 1C interrupt handler 50:16 dword Pointer to BASIC INT 23 interrupt handler 50:1A dword Pointer to BASIC INT 24 disk error handler 50:20 word DOS dynamic storage 50:22 14bytes DOS diskette initialization table (INT 1E) 50:30 4bytes MODE command 70:00 I/O drivers from IO.SYS/IBMBIO.COM The following map varies in size and locus 07C0:0 Boot code is loaded here at startup (31k mark) A000:0 EGA/VGA RAM for graphics display mode 0Dh & above B000:0 MDA RAM, Hercules graphics display RAM B800:0 CGA display RAM C000:0 EGA/VGA BIOS ROM (thru C7FF) C400:0 Video adapter ROM space C600:0 256bytes PGA communication area C800:0 16K Hard disk adapter BIOS ROM C800:5 XT Hard disk ROM format, AH=Drive, AL=Interleave D000:0 32K Cluster adapter BIOS ROM D800:0 PCjr conventionalsoftware cartridge address E000:0 64K Expansion ROM space (hardwired on AT+) 128K PS/2 System ROM (thru F000) F000:0 System monitor ROM PCjr: software cartridge override address F400:0 System expansion ROMs F600:0 IBM ROM BASIC (AT) F800:0 PCjr software cartridge override address FC00:0 BIOS ROM FF00:0 System ROM FFA6:E ROM graphics character table FFFF:0 ROM bootstrap code FFFF:5 8 bytes ROM date (not applicable for all clones) FFFF:E byte ROM machine id (see MACHINE ID) --- ### DOS GAMES THAT NEED A HIGHER MCB BASE SEGMENT (`docs/DOS GAMES THAT NEED A HIGHER MCB BASE SEGMENT.TXT`) DOS games/applications that will not work correctly unless given a minimum MCB segment that is higher than default: recommended game/application value reason ------------------------------------------------------------------------------------------------------------------- Shamaani (1994 demo) 800 Will crash if ems=true and loaded too low after GUS I/O port selection menu Space Pigs Megademo (1990) C00 Demo will crash at startup, claim EXE corruption or files are screwed up, if too low Vicky (Space Pigs demo) 1000 Demo is compressed with EXEPACK, needs to be loaded 64KB or higher in memory to run Warlock (1992 demo) C00 Final credits part will crash if too low --- ### PC 98 Display Terminology (`docs/PC-98 display terminology.txt`) This is posted here in the source tree in the event it gets lost in Github's issue tracker. From hdk1983: ... The second part of the undocumented_mem/io_prn.txt is for ハイレゾ (hairezo, high-resolution) mode. The games that read the port for detecting system clock are for normal mode. ... It's "ハイレゾ", its resolution is super high resolution... normal mode = 15khz 200-line mode high resolution = 24/31khz 400/480-line mode super high resolution = ?khz 1120x750 mode Modes are named as: ノーマルモード, normal mode, is 15kHz/24kHz/31kHz. ハイレゾモード, hi-res mode, high resolution mode, is the 750 lines mode. However resolution of the high resolution mode is super high resolution, that term is normally not used, but only used in Wikipedia for describing it is different from the 高解像度, high resolution, of the normal mode. When ハイレゾモード was released, ハイレゾ term might be used only for PC-98 series in Japan, so many Japanese speakers using the ハイレゾ term might not think about that ハイレゾ was a abbreviation for high resolution and the same meaning as 高解像度. So if you find ハイレゾ in documentation, normally it means the 750 lines mode. If you use a translator, maybe you need to look at the original sentense of "high resolution". A display for 15kHz is called as 標準ディスプレイ、a standard display. The 15kHz mode was normally not used. That might be used in early 1980s, because its display might compatible with other PCs like Sharp X1 and Fujitsu FM-7, but I've never seen PC-98 systems that are connected with such displays in 1990s. --- ### Runtime Media Swapping (`docs/Runtime media swapping.txt`) == Floppy image swapping == If you are booting from floppy using the BOOT command (but not yet the IMGMOUNT command), you can use the SwapFloppy key combination through the mapper to swap between floppies. This key combination is F11+CTRL+D by default, but can be changed through the mapper interface. == CD-ROM image swapping == IMGMOUNT can be used to attach multiple ISO images to a CD-ROM drive. At first, the first ISO image is presented to the guest as the CD-ROM drive. To emulate the act of removing the CD and putting in the next one, use the SwapCD key combination through the mapper. This key combination is F11+CTRL+C by default, but can be changed through the mapper interface. To provide a realistic CD change delay, CD switching is emulated as first the removal of the CD-ROM media from the drive, followed by a 4 second delay, followed by the emulation of inserting the new CD and waiting for the CD-ROM drive to spin up and read it. This delay is necessary for some OSes like Windows 95 who cannot detect CD changes if the changeover happens too fast. --- ### WINDOWS DOS VM TEST RESULTS (`docs/WINDOWS DOS VM TEST RESULTS.TXT`) Windows 3.11/95 DOS VM save/restore test results machine=svga_s3 N - WIN31 WIN95 0 - PASSED PASSED 1 - PASSED PASSED 2 - PASSED PASSED 3 - PASSED PASSED 4 - PASSED PASSED 5 - PASSED PASSED 6 - PASSED PASSED 7 - FAIL[1] FAIL[1] 13 - PASSED PASSED 14 - PASSED PASSED 15 - PASSED PASSED 16 - PASSED PASSED 17 - PASSED PASSED 18 - PASSED PASSED 19 - *----- PASSED 19x- *----- PASSED [1] The DOS VM is able to draw the mode windowed correctly, but the S3 driver doesn't seem to switch it back to fullscreen properly. * NTS: Windows 3.1 will not show the DOS screen windowed for 320x200x256 color (mode 19) for some reason. Despite that, the DOS VM is able to restore the mode correctly going back to fullscreen. Windows 3.11/95 DOS VM save/restore test results machine=svga_et4000 N - WIN31 WIN95 0 - PASSED PASSED 1 - PASSED PASSED 2 - PASSED PASSED 3 - PASSED PASSED 4 - FAIL PASSED 5 - FAIL PASSED 6 - FAIL PASSED 7 - PASSED FAIL[1] 13 - FAIL PASSED 14 - FAIL PASSED 15 - FAIL PASSED 16 - FAIL PASSED 17 - FAIL PASSED 18 - FAIL PASSED 19 - FAIL PASSED 19x- FAIL PASSED Windows 95: [1] The DOS VM is able to draw the mode windowed correctly, but the driver doesn't seem to switch it back to fullscreen properly. Windows 3.1: The ET4000 driver in the test VM seems to mistake all modes 4-6 and 13-18 as some variant of 256-color mode for some weird reason, the graphics are drawn as "squeezed". And for some reason, the DOS VM refuses to run 320x200x256-color mode windowed, which is strange, I remember running that mode just fine on my ancient 386... Windows 3.11/95 DOS VM save/restore test results machine=vgaonly N - WIN31 WIN95 0 - PASSED PASSED 1 - PASSED PASSED 2 - PASSED PASSED 3 - PASSED PASSED 4 - PASSED PASSED 5 - PASSED PASSED 6 - PASSED PASSED 7 - PASSED PASSED 13 - PASSED PASSED 14 - PASSED PASSED 15 - PASSED PASSED 16 - PASSED PASSED 17 - PASSED PASSED 18 - PASSED PASSED 19 - *----- PASSED 19x- PASSED PASSED * NTS: Windows 3.1's standard VGA driver (640x480x16 mode) will not show the DOS screen windowed for 320x200x256 color (mode 19) obviously because that would require the DOS VM window to create a color palette and map it somehow to 16 colors. But... for some reason, 320x200x256 mode X is OK to run windowed, even if it allows the general palette on screen to get messed up doing so. Weird. Despite that, the DOS VM is able to restore the mode correctly going back to fullscreen. --- ### Windows 3.0 And Later EGA, VGA Driver Notes (`docs/Windows 3.0 and later EGA, VGA driver notes.txt`) Notes: - Starting with Windows 3.0, the stock EGA/VGA drivers require a 80186 or higher, they do not work on an 8086. This is due to the use of instructions not supported by the 8086. --- ### Rpc Agent Usage (`docs/rpc-agent-usage.md`) # DOSBox-X Debugger Agent 使用手册 本文面向需要调用 DOSBox-X 调试后端的 AI Agent。目标是让 Agent 通过结构化 RPC 对 DOS 程序进行动态逆向：启动目标、停在入口、设置断点、运行到断点、读取寄存器和内存、单步、收集 trace，并在有明确前置条件时修改 guest memory。 Agent 不应依赖 DOSBox-X 窗口、stdout、`LOGCPU.TXT` 或 debugger 的人类可读文本来获得核心状态。正式数据必须来自 JSON-RPC；`debugger.execute_command` 只用于有限的只读诊断。 ## 1. 运行模型 ```text Agent | | Python client 或 JSON-RPC 2.0 v DOSBox-X named pipe agent | | emulation-thread queue v DOSBox-X debugger / emulator | v DOS program ``` 一次分析通常包含两个进程角色： 1. Agent 启动一个 DOSBox-X 进程，并传入 `--agent-config `。 2. Agent 通过配置中的 Windows named pipe 发送 JSON-RPC 请求。 v1 的约束： - 仅支持 Windows。 - 默认 transport 是当前用户可访问的 named pipe，不开放未认证 TCP。 - 一个 DOSBox-X 进程只有一个活动 session 和一个受控的 `C:` mount。 - `session.start` 固定在 `entry` 停止。 - target 必须是 DOS 安全文件名，例如 `GAME.COM`；不能传宿主机路径或 shell 命令。 - `session.start` 会等待入口停止后才返回；这不是一个只返回 operation id 的后台启动调用。 - `execution.continue`、`execution.pause`、`session.stop` 返回 `operation_id`，终态通过 `execution.wait` 获取。 ## 2. 准备分析目录 建议为每次分析创建一个新的 runtime 目录，并只把待分析程序和必要的数据文件复制进去： ```text reverse/ configs/ game.env runtime/ GAME.COM DATA.DAT ``` `dosbox_workdir` 必须是受控目录。`session.start` 的 `mounts[0].host_path` 必须解析为同一个目录，不能临时切换到其他路径。目标程序和参数应使用 DOS 目录中的名字： ```python session = agent.start( "GAME.COM", ["LEVEL1", "-DEBUG"], mount_path=workdir, ) ``` `target.command` 和 `target.arguments` 只允许字母、数字、`.`、`_`、`-`，因此应在启动前完成文件准备，不要把 `C:\...`、引号、重定向或 shell 元字符传给 RPC。 ## 3. 配置文件 配置来自显式的 `.env` 文件。server 和 Python client 都不会从系统环境变量读取 endpoint、目标程序、工作目录或超时。 生产配置使用绝对路径： ```env transport=named_pipe endpoint=\\.\pipe\dosbox-agent-game-01 dosbox_executable=D:\workspace\dosbox-x\bin\x64\Agent Debug SDL2\dosbox-x.exe dosbox_workdir=D:\workspace\reverse\game-01 profile=production request_timeout_ms=5000 max_message_bytes=1048576 max_memory_read_bytes=65536 max_trace_events=10000 ``` 配置键说明： | 键 | 说明 | | --- | --- | | `transport` | v1 使用 `named_pipe`。 | | `endpoint` | Windows named pipe 名称；每个并行 DOSBox-X 进程应使用不同名称。 | | `dosbox_executable` | 要启动的 DOSBox-X 可执行文件绝对路径。 | | `dosbox_workdir` | 唯一受控的 `C:` mount 目录绝对路径。 | | `profile` | `production` 或 `test`；正式 Agent 使用 `production`。 | | `request_timeout_ms` | server 等待 emulation-thread 操作的上限。 | | `max_message_bytes` | 单条 JSON request/response 的消息上限。 | | `max_memory_read_bytes` | 单次 `memory.read` 和 `memory.write` 的最大字节数。 | | `max_trace_events` | session trace store 的最大事件数。 | `profile=test` 只用于仓库测试夹具。生产配置使用相对的 executable 或 workdir 会被拒绝；不要通过把路径放进 `tests` 目录来绕过校验。 ## 4. 启动 DOSBox-X PowerShell 示例： ```powershell $config = 'D:\workspace\reverse\configs\game.env' $process = Start-Process ` -FilePath 'D:\workspace\dosbox-x\bin\x64\Agent Debug SDL2\dosbox-x.exe' ` -ArgumentList @('--agent-config', $config) ` -WindowStyle Hidden ` -PassThru # 等待进程完成 DOS shell 和 debugger 初始化。 Start-Sleep -Seconds 2 ``` Agent 应通过进程句柄、named pipe readiness 或有限的启动等待确认 DOSBox-X 已经启动。不要无限重试 RPC，也不要把带副作用的请求自动重发。 Python Agent 通常在 `finally` 中关闭 client、停止 session，并回收 DOSBox-X 进程： ```python process = subprocess.Popen( [str(config.dosbox_executable), "--agent-config", str(config.path)], cwd=repository_root, creationflags=getattr(subprocess, "CREATE_NO_WINDOW", 0), ) try: # 连接并进行 RPC 分析 ... finally: client.close() if process.poll() is None: process.kill() process.wait() ``` ## 5. Python client 快速开始 仓库当前的 Python client 位于 `client/python`，还没有独立安装包。Agent 脚本可将该目录加入自身模块搜索路径： ```python from pathlib import Path import sys repository_root = Path(__file__).resolve().parents[2] sys.path.insert(0, str(repository_root / "client" / "python")) from dosbox_agent import AgentClient, MemoryAddress config_path = repository_root / "reverse" / "configs" / "game.env" workdir = Path(r"D:\workspace\reverse\game-01") with AgentClient.from_config(config_path) as agent: capabilities = agent.capabilities() session = agent.start("GAME.COM", mount_path=workdir) print(session.id, session.state, session.stop_reason) registers = agent.get_registers(session.id) code = agent.read_memory( session.id, MemoryAddress.segmented(registers.segments["cs"], registers.instruction_pointer), 32, ) print(registers) print(code.data.hex(), code.sha256) stop = agent.stop(session.id) stopped = agent.wait(session.id, stop.id, timeout_ms=5000) while stopped.running: stopped = agent.wait(session.id, stop.id, timeout_ms=5000) ``` 关键返回对象是 typed model，不需要 Agent 手动解析 JSON 或 base64： - `Session`：`id`、`state`、`state_revision`、`stop_reason`。 - `RegisterSnapshot`：`general`、`segments`、`instruction_pointer`、`flags`、`cpu_mode`。 - `MemoryRead`：`address`、`data: bytes`、`sha256`、`state_revision`。 - `MemoryWrite`：写入前后 SHA-256 和 `byte_count`。 - `Breakpoint`、`Operation`、`WaitResult`、`OutputPage`、`TracePage`。 ## 6. 标准动态逆向流程 ### 6.1 检查能力 先调用 `agent.capabilities()`，根据返回值决定是否使用 trace 或 memory-change breakpoint： ```python capabilities = agent.capabilities() if not capabilities["debugger"]: raise RuntimeError("current DOSBox-X build has no debugger capability") trace_enabled = capabilities["trace"]["cpu"] is True memory_breakpoint_enabled = capabilities["breakpoints"]["memory_change"] is True ``` 不要把 capability 缺失当成普通断点或空 trace；server 会返回 `CapabilityUnavailableError`。 ### 6.2 在入口建立基线 `agent.start()` 成功后 session 应为 `stopped`，`stop_reason.kind` 应为 `startup`。此时保存： - `CS:IP` 和 `DS:SI/SS:SP` 等段寄存器组合。 - 通用寄存器和 FLAGS。 - 入口附近的 code bytes。 - 关键数据区的初始 bytes 和 SHA-256。 示例： ```python session = agent.start("GAME.COM", mount_path=workdir) assert session.state == "stopped" assert session.stop_reason is not None assert session.stop_reason.kind == "startup" regs = agent.get_registers(session.id) entry = MemoryAddress.segmented(regs.segments["cs"], regs.instruction_pointer) entry_bytes = agent.read_memory(session.id, entry, 64) ``` 不要假设 `CS`、`DS` 相同，也不要把 segmented 地址拼成字符串。使用 `MemoryAddress.segmented()`、`.linear()` 或 `.physical()` 明确指定 address space。 ### 6.3 设置断点并运行 执行断点使用稳定的 `breakpoint_id`，不使用底层 debugger 的列表索引： ```python breakpoint = agent.create_execution_breakpoint( session.id, segment=regs.segments["cs"], offset="0x00001234", ) operation = agent.continue_(session.id) result = agent.wait(session.id, operation.id, timeout_ms=5000) while result.running: result = agent.wait(session.id, operation.id, timeout_ms=5000) assert result.session.stop_reason is not None print(result.session.stop_reason.kind, result.session.stop_reason.address) ``` 执行断点当前要求 `segmented` 地址。`once=True` 可创建命中后自动删除的一次性断点： ```python temporary = agent.create_execution_breakpoint( session.id, regs.segments["cs"], "0x00001400", once=True ) ``` ### 6.4 读取寄存器和内存 寄存器和内存观察只能在 `stopped` 状态进行。运行时调用会抛出 `TargetRunningError`。 ```python regs = agent.get_registers(session.id) stack = agent.read_memory( session.id, MemoryAddress.segmented(regs.segments["ss"], regs.general["esp"]), 32, ) linear_page = agent.read_memory( session.id, MemoryAddress.linear("0x00001000"), 256, ) physical = agent.read_memory( session.id, MemoryAddress.physical("0x000A0000"), 64, ) ``` 注意：`regs.general["esp"]` 是固定宽度十六进制字符串，适合直接传给 `MemoryAddress.segmented()`。如果地址未映射、selector 无效、超过 segment limit 或 page 不存在，client 会抛出 `AddressNotMappedError`；错误对象的 `data` 中包含失败地址和底层原因。 ### 6.5 单步进入与越过 ```python stopped, after_step = agent.step(session.id, mode="into") print(stopped.stop_reason.kind, after_step.instruction_pointer) stopped, after_over = agent.step(session.id, mode="over") ``` `into` 进入当前指令的执行路径；`over` 使用 debugger 的现有 step-over 语义越过 call、interrupt 或重复指令的目标。每次返回都包含新的 register snapshot，不需要再次调用 `get_registers`，但可以再读内存验证副作用。 ### 6.6 观察内存变化 优先使用已知代码位置的 execution breakpoint 和前后内存快照。heavy-debug build 还可以使用 memory-change breakpoint： ```python if memory_breakpoint_enabled: watch = agent.create_breakpoint( session.id, "memory_change", MemoryAddress.segmented(regs.segments["ds"], "0x00000200"), ) ``` 不支持 memory-change breakpoint 时，必须改变分析策略，不能自动降级为 execution breakpoint。 ### 6.7 修改 guest memory 写入前先读取并保留 SHA-256，然后用 `expected_sha256` 做乐观并发保护： ```python address = MemoryAddress.segmented(regs.segments["ds"], "0x00000200") before = agent.read_memory(session.id, address, 4) written = agent.write_memory( session.id, address, b"\x90\x90\x90\x90", expected_sha256=before.sha256, ) print(written.before_sha256, written.after_sha256) ``` 如果目标在读取和写入之间发生变化，server 返回 `MemoryPreconditionFailedError`，本次写入不会发生。写入超时后不要自动重试；如果要由 Agent 明确重试，复用同一个 `request_id` 并确认请求 payload 完全相同。 ## 7. Trace 和 debugger 输出 ### 7.1 CPU trace CPU trace 需要 `C_HEAVY_DEBUG=1`。trace 是 session-local 的结构化数据，不依赖 `LOGCPU.TXT`： ```python if capabilities["trace"]["cpu"]: agent.start_trace(session.id, detail="normal", instruction_count=200) operation = agent.continue_(session.id) stopped = agent.wait(session.id, operation.id, timeout_ms=10000) while stopped.running: stopped = agent.wait(session.id, operation.id, timeout_ms=10000) cursor = None while True: page = agent.read_trace(session.id, cursor, limit=50) for event in page.events: print(event.sequence, event.address, event.instruction, event.register_changes) if page.next_cursor is None: break cursor = page.next_cursor agent.stop_trace(session.id) ``` `trace.read` 的 `cursor` 初始为 `None`，下一页使用上一次返回的 `next_cursor`。不要在 cursor 过期后从头拼接；应重新建立 trace 分析窗口。`instruction_count` 受配置的 `max_trace_events` 限制。 ### 7.2 Debugger 输出 正式 API 不需要解析 debugger 文本。需要诊断时只能执行当前 adapter allowlist 中的只读命令：`HELP`、`CPU`、`PIC`。 ```python diagnostic = agent.execute_command(session.id, "CPU") if diagnostic.accepted: print(diagnostic.raw_output) ``` 输出记录来自 session-local ring buffer，可用 cursor 分页： ```python cursor = None while True: page = agent.read_output(session.id, cursor, limit=100) for record in page.records: print(record.sequence, record.source, record.message) if page.next_cursor is None: break cursor = page.next_cursor ``` `RUN`、`SM`、`SR`、`BP*`、`LOG*` 等会改变调试状态或 collector 状态的命令不属于稳定诊断 API，server 会拒绝它们。请使用对应的结构化 RPC 方法。 ## 8. 会话状态和等待规则 | 状态 | Agent 可以做什么 | | --- | --- | | `starting` | 查询 `session.status`、读取输出；等待 `session.start` 完成。 | | `stopped` | 读取/写入 memory、读取 registers、断点管理、continue、pause、step、trace 操作。 | | `running` | `session.status`、`execution.pause`、`execution.wait`、`debug.output.read`。其他状态观察和修改请求返回 `TARGET_RUNNING`。 | | `exited` | 查询最终状态、读取最后 stop reason、重复 stop；不能读取 guest registers/memory。 | | `failed` | 查询状态、读取输出、执行 stop 清理；不要继续调用 target 操作。 | 每次成功响应都带 `state_revision`。Agent 应保存最后一次 revision，在断点命中、单步、memory.write 或断点集合变化后更新自己的观察缓存。旧 revision 的内存快照不能直接当成当前状态。 `execution.wait` 的超时不是失败： ```python result = agent.wait(session.id, operation.id, timeout_ms=100) if result.running: # 目标仍在运行；继续使用同一个 operation_id 轮询或先 pause。 ... ``` 不要把 `running=true` 当成程序退出，也不要为“确认停止”创建新的 continue 请求。 ## 9. 错误处理和重试 Python client 会把 RPC business error 映射为 typed exception： ```python from dosbox_agent import ( AddressNotMappedError, AgentConnectionError, CapabilityUnavailableError, MemoryPreconditionFailedError, TargetRunningError, ) try: registers = agent.get_registers(session.id) except TargetRunningError: # 先 pause 或 wait，不要读取不一致状态。 ... except AddressNotMappedError as error: print(error.data) except MemoryPreconditionFailedError: # 重新 read，确认目标状态后再决定是否写入。 ... except CapabilityUnavailableError: # 改用 capability 已声明支持的分析方法。 ... except AgentConnectionError: # 进程或 named pipe 已不可用；不要重发有副作用请求。 ... ``` 重要错误： | reason | Agent 行为 | | --- | --- | | `SESSION_BUSY` | 当前进程已有 active session；先 stop 并 wait，或创建新的 DOSBox-X 进程。 | | `SESSION_NOT_FOUND` | 丢弃本地 session 引用，不能继续使用旧 operation。 | | `TARGET_RUNNING` | 先 pause 或等待停止。 | | `TARGET_NOT_STOPPED` / `TARGET_EXITED` | 修正状态机，不要重试相同请求。 | | `CAPABILITY_UNAVAILABLE` | 读取 capabilities，选择其他策略。 | | `ADDRESS_NOT_MAPPED` | 检查 address space、segment、offset 和 CPU mode。 | | `MEMORY_PRECONDITION_FAILED` | 重新读取并重新判断，不能盲写。 | | `REQUEST_TOO_LARGE` | 减小 memory/trace/page 请求；server 不截断。 | | `OPERATION_TIMEOUT` | 对 start 或 adapter 操作记录失败/不确定状态；不要自动重复有副作用调用。 | | `CURSOR_EXPIRED` | 丢弃旧 cursor，重新开始一个明确的新读取窗口。 | | `REQUEST_ID_CONFLICT` | 同一 request id 被用于不同 payload；生成新的逻辑请求 id。 | server 会缓存 session 生命周期内最近完成的请求结果。只有 Agent 明确确认“同一个逻辑请求、同一个 payload”时，才可以用相同 `request_id` 重试。Python client 不会自动重试写内存、断点、单步、原始命令或其他有副作用调用。 ## 10. 非 Python Agent 的最小 JSON-RPC 调用 传输是 UTF-8 JSON Lines，一行一个 request，一行一个 response。下面是入口启动请求： ```json {"jsonrpc":"2.0","id":"start-1","method":"session.start","params":{"target":{"command":"GAME.COM","arguments":[]},"mounts":[{"drive":"C","host_path":"D:/workspace/reverse/game-01"}],"break_at":"entry"}} ``` 入口成功响应的核心字段： ```json { "jsonrpc": "2.0", "id": "start-1", "result": { "session_id": "ses-1", "state": "stopped", "state_revision": 1, "stop_reason": { "kind": "startup", "address": { "space": "segmented", "segment": "0x0812", "offset": "0x00000100" } } } } ``` 读取寄存器： ```json {"jsonrpc":"2.0","id":"regs-1","method":"state.get_registers","params":{"session_id":"ses-1"}} ``` 读取内存： ```json {"jsonrpc":"2.0","id":"mem-1","method":"memory.read","params":{"session_id":"ses-1","address":{"space":"segmented","segment":"0x0812","offset":"0x00000100"},"length":16}} ``` ## 11. 结束分析 正常结束必须停止 session 并等待终态。当前 server 的终止 deadline 使用配置中的 `request_timeout_ms`；`graceful_timeout_ms` 会被 client 发送，但目前不改变 server 的终止 deadline，因此不要依赖它延长等待： ```python stop_operation = agent.stop(session.id, graceful_timeout_ms=5000) result = agent.wait(session.id, stop_operation.id, timeout_ms=5000) while result.running: result = agent.wait(session.id, stop_operation.id, timeout_ms=5000) assert result.session.state == "exited" ``` 无论分析成功还是异常，都应在 `finally` 中： 1. 对仍存在的 session 执行 stop/wait。 2. 关闭 `AgentClient`。 3. 回收 DOSBox-X 进程。 4. 为下一次分析使用新的 endpoint 或确认上一次进程已经完全退出。 不要复用上一次运行产生的 `LOGCPU.TXT`、`MEMDUMP.*`、trace 文件或未清理的 runtime 目录来推断本次分析结果。 ## 12. 推荐的 Agent 分析策略 一个稳定的逆向 Agent 可以按以下顺序工作： 1. 创建干净 runtime，复制目标和输入数据。 2. 启动 DOSBox-X，调用 `agent.capabilities`。 3. `session.start` 停在入口，保存寄存器、入口 code bytes 和关键内存 SHA-256。 4. 根据静态分析或前一轮动态结果设置少量 execution breakpoint。 5. `continue_` + `wait` 到断点，记录 stop reason、register snapshot 和 `state_revision`。 6. 围绕当前 `CS:IP`、栈和数据段读取小范围内存，不要一次读取整个 guest memory。 7. 对可疑指令使用 `step("into")` 或 `step("over")`，每一步只做必要的观察。 8. 需要批量指令历史时，在 heavy-debug build 中使用 session-local CPU trace。 9. 需要验证 patch 时使用 `read -> expected_sha256 -> write -> read` 闭环。 10. 每轮实验结束后 stop/wait，保存结构化结果，再清理进程和 runtime。 这样可以把“静态假设”和“动态证据”绑定到明确的地址、寄存器、内存 hash、断点 id、trace sequence 和 state revision，而不是依赖窗口截图或文本日志。 ## 13. 相关文件和验收命令 - 协议契约：[rpc.md](rpc.md) - 开发和验收规范：[rpc-development.md](rpc-development.md) - Python client：`client/python/dosbox_agent/` - Python E2E：`client/python/tests/test_e2e.py` - 测试配置：`tests/agent/agent-test.env` - 测试夹具：`tests/agent/fixtures/` 开发者可用以下命令验证 client 和完整闭环： ```powershell py -m unittest discover -s client\python\tests -t client\python -v py client\python\tests\test_e2e.py --config tests\agent\agent-test.env ``` 测试配置使用 `profile=test` 和仓库 runtime；生产 Agent 不应直接复用它，而应创建绝对路径的 `profile=production` 配置。 --- ### Rpc Development (`docs/rpc-development.md`) # DOSBox-X Debugger Agent RPC 开发与验收规范 状态：计划基线 版本：v1.0-draft 日期：2026-09-02 需求来源：[rpc.md](rpc.md) 本文是 `docs/rpc.md` 的可实施、可验收版本。开发任务、代码评审和发布验收必须以本文的检查项为准。检查项只有在命令执行成功、结果满足预期且证据已记录后才能勾选；不得用“接口存在”“命令未报错”替代行为验证。 ## 1. 目标与边界 目标是在不重写 CPU、内存和断点机制的前提下，将 DOSBox-X 现有 debugger 能力封装为稳定的结构化 Agent API。Agent 只消费 JSON 数据，不依赖 ncurses 窗口或 debugger 的人类可读文本。 v1 的部署、构建和验收范围仅限 Windows。 第一期必须支持： - 启动、停止并查询 DOS 调试会话。 - 继续、暂停、单步进入、单步越过和等待停止事件。 - 获取通用寄存器、段寄存器、IP/EIP 和 FLAGS/EFLAGS。 - 在 segmented、linear、physical 三种地址空间读取和写入内存。 - 创建、列出和删除执行断点；仅在底层实际支持时提供内存变化断点。 - 读取结构化 debugger 输出和执行 trace。 - 执行原始 debugger 命令，作为诊断接口而不是主 API。 - 提供 Python client 和端到端验收夹具。 第一期禁止： - 修改 `src/cpu/` 的执行语义、指令实现或调度语义。 - 修改 `src/hardware/memory.cpp` 的内存映射、分页和读写语义。 - 自行实现第二套断点、单步或内存访问机制。 - 让 RPC 直接返回 `AX=...`、`CS:IP=...` 等 debugger 文本并要求客户端解析。 - 因地址无法解析、capability 不存在或 target 正在运行而悄悄降级、填充默认值或自动重试写操作。 ## 2. 当前代码基线 实现必须复用以下现有能力，并在 adapter 内提供结构化包装： | 能力 | 现有入口 | 说明 | | --- | --- | --- | | Debugger 生命周期 | `include/debug.h`、`src/debug/debug.cpp` | `DEBUG_EnableDebugger()`、`DEBUG_ExitLoop()` 和 `DEBUG_Enable_Handler()` 已管理 debugger 进入/退出。 | | 执行与单步 | `src/debug/debug.cpp` | 现有 `RUN`、`RUNWATCH`、F10/F11 和 `StepOver()` 提供底层控制。 | | 断点 | `src/debug/debug.cpp` | `CBreakpoint` 已实现执行、interrupt、memory/protected/linear 等类型。 | | 寄存器与地址解析 | `src/debug/debug.cpp`、`include/cpu.h` | debugger 已读取 `reg_*`、段寄存器及 `GetAddress()`。 | | 内存访问 | `src/debug/debug.cpp`、`src/hardware/memory.cpp` | debugger 已通过 checked memory API 访问 guest memory。 | | Debugger 命令 | `README.debugger`、`src/debug/debug.cpp` | 现有 `RUN`、`SR`、`SM`、`BP`、`BPLIST`、`BPDEL`、`MEMDUMP*`、`LOG*` 等命令可作兼容和诊断参考。 | | 单元测试入口 | `tests/tests.h`、`tests/readme.txt` | Debug build 可通过 `dosbox-x -tests` 运行 gTest。 | 构建前置条件： - `C_DEBUG=1` 是 v1 的硬性条件；未启用时 server 必须拒绝启动并返回 `DEBUGGER_UNAVAILABLE`。 - `C_HEAVY_DEBUG=1` 是 CPU trace 的硬性条件；未启用时 `agent.capabilities` 必须声明 `trace.cpu=false`，`trace.start` 返回 `CAPABILITY_UNAVAILABLE`。 - Windows 当前 `.vcxproj` 配置没有显式定义 `C_DEBUG`。实现必须新增或调整一个专用调试配置，例如 `Agent Debug SDL2`，显式定义 `C_DEBUG=1;C_HEAVY_DEBUG=1;C_DOSBOX_AGENT=1`。不得使用现有 Release 输出作为 RPC 验收对象。 ## 3. 总体架构 ```text Python client / AI Agent | | JSON-RPC 2.0 v dosbox-agent server - transport - session manager - request validation | | authenticated local IPC v DOSBox-X in-process agent bridge - emulation-thread command queue - DebuggerAdapter - output/trace collector | v DOSBox-X debugger and emulator ``` 职责划分： - `dosbox-agent server` 管理 RPC transport、DOSBox-X 子进程、会话状态和客户端连接。它不直接读取 `reg_*` 或 guest memory。 - in-process bridge 仅在 DOSBox-X 的 emulation thread 上调用 debugger、CPU 和 memory 已有能力。任何 RPC worker thread 都不得直接访问 debugger globals。 - `DebuggerAdapter` 以 C++ 类型返回 registers、memory、breakpoints、stop reason 和 trace events；transport 层只负责编解码 JSON。 - `DebuggerOutputParser` 只服务于 `debugger.execute_command` 的非结构化诊断输出。核心方法不得以 parser 作为数据来源。 现有 debugger 使用大量全局状态，例如 `debugging`、`debug_running`、`runnormal` 和 `CBreakpoint::BPoints`。因此 v1 每个 DOSBox-X 进程只允许一个活动调试会话。第二个 `session.start` 必须返回 `SESSION_BUSY`，不得共享或覆盖已有 session。 ## 4. 目录与构建交付物 实现完成后的目标目录如下。可以按仓库既有布局微调文件名，但职责不能合并或缺失。 ```text src/agent/ protocol/ # JSON model、错误码、schema version server/ # host-side process/session manager transport/ # Windows named pipe / test transport bridge/ # in-process queue and emulation-thread dispatch debugger/ # DebuggerAdapter and structured collectors parser/ # raw debugger output parser only trace/ # trace sink and paged store include/agent/ debugger_adapter.h agent_bridge.h agent_protocol.h client/python/dosbox_agent/ __init__.py client.py models.py errors.py tests/agent/ agent_adapter_tests.cpp agent_parser_tests.cpp agent_protocol_tests.cpp fixtures/agent_fixture.asm fixtures/agent_fixture.com agent-test.env client/python/tests/ test_client.py test_e2e.py ``` 必须更新 Windows 的 `vs/dosbox-x.vcxproj` 和对应 solution configuration。新增源文件未进入 Windows 构建系统即视为未完成。 ## 5. 配置、传输与安全 ### 5.1 配置文件 不得读取系统环境变量决定 endpoint、目标程序、工作目录、超时或认证信息。server 仅从显式传入的 `--agent-config ` 读取 dotenv 格式的配置文件，例如 `tests/agent/agent-test.env`： ```env transport=named_pipe endpoint=\\.\pipe\dosbox-agent-test dosbox_executable=bin\x64\Agent Debug SDL2\dosbox-x.exe dosbox_workdir=tests\agent\runtime profile=production request_timeout_ms=5000 max_message_bytes=1048576 max_memory_read_bytes=65536 max_trace_events=10000 ``` 生产配置必须使用绝对路径；测试配置可使用相对 `--agent-config` 文件所在目录的路径。启动时必须将解析后的配置写入结构化启动日志，但不得记录认证 token。 ### 5.2 传输 `IRpcTransport` 是唯一 transport 抽象，协议层不得依赖具体 named pipe 实现或 stdio。 - Windows v1 默认使用 named pipe，并设置仅当前用户可访问的 ACL。 - v1 不开放 TCP listener，不支持远程无认证连接。 - 所有 transport 必须按 JSON-RPC 2.0 传输完整 request/response；流式 transport 的 framing 采用 UTF-8 JSON Lines，一行一个 JSON object，最大消息长度受 `max_message_bytes` 限制。 - server 与 bridge 使用独立的本地 IPC endpoint，并使用每次启动生成的一次性 token 做握手；token 只经受限配置文件或命令行参数传递，不写入日志。 ### 5.3 并发、超时和幂等 - 每个 session 的调试操作严格串行；请求按接收顺序进入 emulation-thread queue。 - `execution.continue`、`execution.pause` 和 `session.stop` 返回 `operation_id`。客户端通过 `execution.wait` 获取停止结果。 - 写内存、创建/删除断点、单步和原始 debugger 命令是有副作用操作。超时后 server 不得自动重试。 - 客户端重试必须携带相同 `request_id`。server 在 session 生命周期内缓存最近 1024 个已完成 request 的结果；同一 `request_id` 的 payload 不同返回 `REQUEST_ID_CONFLICT`。 - `execution.wait` 超时只表示仍在运行，返回 `running=true`；不得把超时伪造为断点、暂停或程序退出。 ## 6. 会话状态机 ```text created -> starting -> stopped -> running -> stopped | | | v v v failed exited stopping -> exited ``` 状态定义： - `starting`：DOSBox-X 已启动，DOS environment 或 target 正在初始化。 - `stopped`：emulation 已停在 entry、breakpoint、step、明确 pause 或 fault；可读取/写入 state、memory 和 breakpoints。 - `running`：target 正在执行；除 `session.status`、`execution.pause`、`execution.wait`、`debug.output.read` 外，所有观察/修改状态方法必须返回 `TARGET_RUNNING`。 - `exited`：target 已退出；会话状态与最后 stop reason 可读，内存和寄存器不可读。 - `failed`：启动或 bridge 出错；只允许 `session.status`、`debug.output.read` 和 `session.stop`。 所有成功响应必须包含 `session_id` 和递增的 `state_revision`。`state_revision` 在每次停止、单步、写内存和断点集合变化后增加，用于客户端发现过期观察结果。 ## 7. 数据约定 ### 7.1 数值和地址 JSON number 不能无损表达所有未来 guest address，v1 所有 guest 寄存器、offset、selector、flags 和 breakpoint 地址均使用固定宽度的大写十六进制字符串。 ```json { "ax": "0x1234", "eip": "0x00000100", "flags": "0x00000202" } ``` 地址是 tagged object，不接受模糊的 `"segment:offset"` 字符串： ```json { "space": "segmented", "segment": "0x1234", "offset": "0x00000020" } ``` 支持的 `space` 值： - `segmented`：按当前 CPU mode 解析 selector:offset；real/v86 mode 中按段地址计算，protected mode 中遵从 descriptor 和 paging。 - `linear`：CPU linear/virtual address；需要正常的 paging 检查。 - `physical`：物理内存地址；不通过 CPU paging。 地址未映射、selector 无效、offset 越界或 page 不存在时必须返回 `ADDRESS_NOT_MAPPED`，并在 `error.data` 返回 `space`、`address` 和底层原因（`invalid_selector`、`segment_limit`、`page_not_present` 等）。不得将访问悄悄转换为 physical 地址。 ### 7.2 二进制和分页 - 内存 bytes 一律使用 RFC 4648 base64；返回 `byte_count`、`data_base64` 和 SHA-256。 - 单个 `memory.read` 最大值为配置的 `max_memory_read_bytes`；超限返回 `REQUEST_TOO_LARGE`，不截断。 - `debug.output.read` 和 `trace.read` 使用显式 `cursor`、`limit` 和 `next_cursor` 分页。cursor 过期返回 `CURSOR_EXPIRED`，不从头补发。 ### 7.3 统一响应和错误 成功响应： ```json { "jsonrpc": "2.0", "id": "req-0001", "result": { "session_id": "ses-...", "state_revision": 7 } } ``` 错误响应： ```json { "jsonrpc": "2.0", "id": "req-0001", "error": { "code": -32012, "message": "Target must be stopped before reading memory", "data": { "reason": "TARGET_RUNNING", "session_id": "ses-...", "state": "running" } } } ``` 保留 JSON-RPC 标准错误 `-32600`、`-32601`、`-32602` 和 `-32603`。业务错误使用 `-32000` 到 `-32099`，至少包含： | reason | 使用场景 | | --- | --- | | `DEBUGGER_UNAVAILABLE` | 构建未启用 `C_DEBUG`。 | | `SESSION_BUSY` / `SESSION_NOT_FOUND` | 会话生命周期错误。 | | `TARGET_RUNNING` / `TARGET_NOT_STOPPED` / `TARGET_EXITED` | 状态机违反。 | | `CAPABILITY_UNAVAILABLE` | 例如未启用 `C_HEAVY_DEBUG` 的 trace，或底层不支持的 memory breakpoint。 | | `ADDRESS_NOT_MAPPED` / `ADDRESS_OUT_OF_RANGE` | 地址解析或访问失败。 | | `REQUEST_TOO_LARGE` / `INVALID_BINARY_LENGTH` | 二进制请求不满足限额或长度。 | | `REQUEST_ID_CONFLICT` | 相同 request id 的 payload 不一致。 | | `COMMAND_REJECTED` / `COMMAND_PARSE_FAILED` | 原始 debugger 命令失败。 | | `OPERATION_TIMEOUT` / `CURSOR_EXPIRED` | 等待或分页语义错误。 | ## 8. RPC v1 方法 所有方法名和 request/response field 是兼容性契约。新字段只能可选新增；删除或改变现有字段语义必须升级 major version。 ### 8.1 Capability 和会话 | 方法 | 参数 | 成功结果 | 约束 | | --- | --- | --- | --- | | `agent.capabilities` | 无 | `protocol_version`、`debugger`、`trace.cpu`、`breakpoints.memory_change`、limits、address spaces | 无 session 时也可调用。 | | `session.start` | `target.command`、`target.arguments`、`mounts`、`break_at` | `session_id`、`state=stopped`、`stop_reason=startup` | `break_at` v1 固定为 `entry`；目标必须在受控 workdir 内启动。 | | `session.status` | `session_id` | state、target、last_stop、state_revision | 任意 session 状态均可调用。 | | `session.stop` | `session_id`、`graceful_timeout_ms` | `operation_id` | 只请求停止；结果由 `execution.wait` 获取。 | `session.start` 示例： ```json { "jsonrpc": "2.0", "id": "req-start", "method": "session.start", "params": { "target": { "command": "AGENTFIX.COM", "arguments": [] }, "mounts": [ {"drive": "C", "host_path": "tests/agent/runtime"} ], "break_at": "entry" } } ``` ### 8.2 执行控制 | 方法 | 参数 | 成功结果 | 底层对应 | | --- | --- | --- | --- | | `execution.continue` | `session_id` | `operation_id`、`state=running` | `RUN` 及现有 normal loop。 | | `execution.pause` | `session_id` | `operation_id` | 调度 debugger break；不能从 worker thread 直接改 debugger state。 | | `execution.step` | `session_id`、`mode=into|over` | `stop_reason=step`、register snapshot | 复用 debugger 单步/F10/F11 行为。 | | `execution.wait` | `session_id`、`operation_id`、`timeout_ms` | stopped/exited 状态，或 `running=true` | 等待 continue、pause 或 stop 的终态。 | 停止事件必须是结构化对象： ```json { "kind": "breakpoint", "breakpoint_id": "bp-3", "address": { "space": "segmented", "segment": "0x1234", "offset": "0x00000106" } } ``` 合法 `kind`：`startup`、`step`、`breakpoint`、`pause`、`program_exit`、`fault`。`execution.wait` 返回 `running=true` 时不得同时给出 `stop_reason`。 ### 8.3 CPU state `state.get_registers` 仅允许在 `stopped` 状态调用，返回： ```json { "general": { "eax": "0x00001234", "ebx": "0x00000000", "ecx": "0x00000000", "edx": "0x00000000", "esi": "0x00000000", "edi": "0x00000000", "ebp": "0x00000000", "esp": "0x0000FFFE" }, "segments": { "cs": "0x1234", "ds": "0x1234", "es": "0x1234", "fs": "0x0000", "gs": "0x0000", "ss": "0x1234" }, "instruction_pointer": "0x00000103", "flags": "0x00000202", "cpu_mode": "real", "state_revision": 8 } ``` v1 不提供 `state.set_registers`。现有 `SR` 仅可通过 `debugger.execute_command` 用于人工诊断，不能成为 Python client 的稳定写寄存器 API。 ### 8.4 Memory `memory.read` 参数：`session_id`、`address`、`length`。成功结果包含原始地址、`byte_count`、`data_base64`、`sha256`。 `memory.write` 参数：`session_id`、`address`、`data_base64`、可选 `expected_sha256`。实现要求： - base64 decode 后长度必须大于 0 且不超过 `max_memory_read_bytes`。 - `expected_sha256` 存在时，先读取同一范围并精确匹配；不匹配返回 `MEMORY_PRECONDITION_FAILED`，不得写入任何字节。 - 写入前后读取同一范围，返回 `before_sha256`、`after_sha256` 和 `byte_count`。 - 任意一个字节地址不可访问时，整次写入失败并返回失败地址；不得只写可访问前缀。 ### 8.5 Breakpoint | 方法 | 参数 | 结果 | 规则 | | --- | --- | --- | --- | | `breakpoints.create` | `session_id`、`kind`、`address`、可选 `once` | 稳定 `breakpoint_id`、规范化地址 | `kind=execution` 为 v1 必选。 | | `breakpoints.list` | `session_id` | 所有 breakpoint 的 id、kind、enabled、address、once | 不能解析 `BPLIST` 文本作为正式实现。 | | `breakpoints.delete` | `session_id`、`breakpoint_id` | 删除确认和 revision | 删除不存在 id 返回 `BREAKPOINT_NOT_FOUND`。 | `kind=memory_change` 仅在 `agent.capabilities.breakpoints.memory_change=true` 时可创建。实现必须明确 address space 和 CPU mode 的支持范围；不支持时返回 `CAPABILITY_UNAVAILABLE`，不得创建普通执行断点替代。 RPC id 不得复用 `CBreakpoint` 当前显示列表 index。adapter 必须生成 session 内稳定的 `bp-*` id，并维护其与底层 breakpoint object 的映射，避免 `BPDEL` 后索引变化导致误删。 ### 8.6 Debugger 输出、原始命令和 trace | 方法 | 作用 | 结果约束 | | --- | --- | --- | | `debug.output.read` | 读取 debugger output ring buffer | 每条记录有 `sequence`、`timestamp`、`level`、`source`、`message`。 | | `debugger.execute_command` | 执行原始 debugger command | 返回 `accepted`、`raw_output`、可选 parser result；不替代任何结构化方法。 | | `trace.start` | 开始指定条数的 CPU trace | 仅 `C_HEAVY_DEBUG`；参数为 `detail=short|normal|long|csip`、`instruction_count`。 | | `trace.read` | 分页读取 trace events | event 必含 address、instruction、register_changes 和 sequence。 | | `trace.stop` | 停止尚未完成的 trace | 返回已收集 event 数，不能伪称已完成。 | trace 的数据源必须是 in-process `TraceSink`，在既有 heavy debug 指令日志点采集结构化事件。`LOGCPU.TXT` 可以作为人工对照产物，但不得作为正式 API 的唯一来源，也不得在多个 session 间共享文件名。 `DebuggerOutputParser` 的规则： - parser 输入、输出和失败原因必须可单元测试；测试样本存放在 `tests/agent/fixtures/`。 - 解析失败返回 `parse_status=failed` 及 `unparsed_output`，不得返回部分 register object 伪装为完整结果。 - raw command 不能改变 session 状态机以外的状态；检测到 `RUN`、`LOG*`、`BP*`、`SM`、`SR` 等影响正式 API 状态的命令时，server 必须更新 `state_revision` 和相关 collector，或拒绝该命令并返回 `COMMAND_REJECTED`。第一期推荐采用 allowlist，只允许只读诊断命令：`HELP`、`CPU`、`PIC`、`GDT`、`IDT`、`PAGING`、`EMU MEM`、`EMU MACHINE`。 ## 9. Python Client 契约 Python client 必须只调用 JSON-RPC，不得通过 stdout、窗口自动化或直接读取 DOSBox-X 输出文件工作。 ```python from dosbox_agent import AgentClient with AgentClient.from_config("tests/agent/agent-test.env") as agent: session = agent.start("AGENTFIX.COM", break_at="entry") agent.create_execution_breakpoint(session.id, segment="0x1234", offset="0x00000106") operation = agent.continue_(session.id) stop = agent.wait(session.id, operation.id, timeout_ms=5000) registers = agent.get_registers(session.id) memory = agent.read_memory(session.id, space="segmented", segment="0x1234", offset="0x00000200", length=4) ``` 要求： - 公开方法使用 typed model，并把 RPC error 转为 `AgentError` 子类；不得让调用方检查错误字符串。 - `continue_` 使用尾部下划线避免与 Python keyword 冲突。 - bytes 参数和结果使用 `bytes`；client 负责 base64 编解码。 - client 对有副作用调用不自动重试；可由调用方以同一 request id 明确重试。 - 每个 RPC method 至少有一个 client unit test 和一个反序列化/错误映射测试。 ## 10. 测试夹具 新增确定性的 16-bit COM 测试程序 `tests/agent/fixtures/agent_fixture.asm`，编译产物 `agent_fixture.com` 必须提交或由确定性构建步骤生成。程序固定从 `CS:0100` 开始，执行以下语义： ```asm org 100h start: mov ax, 0x1234 mov bx, 0x5678 mov si, 0x0200 mov byte [si], 0x41 mov word [si + 1], 0x4243 nop int 0x20 ``` 夹具的明确断言： - entry 的 `IP` 为 `0x00000100`。 - 单步进入一次后，`AX` 为 `0x00001234`，`IP` 为 `0x00000103`。 - `CS:0106` 是第二条指令的起点；在此创建执行断点后 continue 必须停在该地址。 - 五次单步后，`DS:0200` 的四字节是 `41 43 42 00`（最后一个字节为 fixture 未写入的原始值，不作为固定断言）；前三字节必须精确为 `41 43 42`。 - memory.write 向可读写的 fixture data 区写入四字节后，随后 memory.read 的 bytes 和 SHA-256 必须一致。 夹具必须在 runtime 目录的干净副本中运行。验收不得复用上次运行的 `LOGCPU.TXT`、`MEMDUMP.*`、DOSBox-X 配置或 trace 文件。 ## 11. 分阶段实施与检查表 每一项格式为：`[ ] ID - 交付物；检查；通过标准；证据`，完成项使用 `[x]`。证据可填写测试日志路径、CI job URL 或 commit hash。 以下所有命令均在 Windows PowerShell 中执行，路径使用 Windows 分隔符。 当前状态（2026-09-02）：阶段 A、B、C、D、E 已完成并已在 Windows 验证。 ### 阶段 A：构建和架构（已完成） - [x] `RPC-A01` - 新增 `Agent Debug SDL2` Windows 配置，定义 `C_DEBUG=1;C_HEAVY_DEBUG=1;C_DOSBOX_AGENT=1`；运行 `msbuild .\vs\dosbox-x.sln /m /t:dosbox-x /p:Configuration="Agent Debug SDL2" /p:Platform=x64`；退出码为 0，生成 `bin\x64\Agent Debug SDL2\dosbox-x.exe`；证据：2026-09-02 Windows PowerShell，`msbuild` 退出码 0，0 个警告、0 个错误。 - [x] `RPC-A02` - Windows 项目编入全部 agent 源文件；检查 `vs\dosbox-x.vcxproj` 的 `ClCompile` 项包含 `src\agent\`，并执行 `msbuild .\vs\dosbox-x.sln /m /t:dosbox-x /p:Configuration="Agent Debug SDL2" /p:Platform=x64`；退出码为 0；证据：2026-09-02，项目已编入 bridge、debugger、protocol、server 和 transport 的 5 个 agent 源文件；同次 `msbuild` 退出码 0。 - [x] `RPC-A03` - agent bridge、server、transport 和 adapter 有独立职责与头文件边界；审查 `src\agent\`，确认 transport 层没有 `reg_`、`mem_read`、`CBreakpoint` 引用；证据：2026-09-02，`rg -n 'reg_|mem_read|CBreakpoint' src\agent\transport` 无匹配。 - [x] `RPC-A04` - 所有 DOSBox-X 状态访问都在 emulation-thread queue 执行；执行 `& '.\bin\x64\Agent Debug SDL2\dosbox-x.exe' --agent-config tests\agent\agent-test.env --agent-self-test`，连续并发提交 1000 个只读请求；退出码为 0，日志确认 FIFO 响应顺序与提交顺序一致，且无死锁或失败请求；证据：2026-09-02，命令退出码 0，输出 `Agent self-test completed: success`。 - [x] `RPC-A05` - 配置只从 `--agent-config` 加载；执行 `Remove-Item Env:DOSBOX_AGENT_ENDPOINT, Env:DOSBOX_AGENT_WORKDIR, Env:DOSBOX_AGENT_REQUEST_TIMEOUT_MS -ErrorAction SilentlyContinue` 后，以 `--agent-config tests\agent\agent-test.env` 启动；endpoint、workdir 和 limit 与 env 文件一致；证据：2026-09-02，清空当前 PowerShell 进程变量后自检成功；日志确认 endpoint=`\\.\pipe\dosbox-agent-test`、workdir=`D:\workspace\ai\yl\dosbox-x\tests\agent\runtime`、request_timeout_ms=5000、max_message_bytes=1048576、max_memory_read_bytes=65536、max_trace_events=10000。 ### 阶段 B：协议和会话（已完成） - [x] `RPC-B01` - JSON-RPC 2.0 request validation；对缺失 `jsonrpc`、未知 method、错误 field type 和超过 `max_message_bytes` 的请求运行 protocol unit tests；分别返回标准 JSON-RPC 错误或 `REQUEST_TOO_LARGE`；证据：2026-09-02，`--agent-self-test` 成功，覆盖缺失 `jsonrpc`、未知 method、params 类型错误和超限 request。 - [x] `RPC-B02` - `agent.capabilities`；在 `C_DEBUG` + `C_HEAVY_DEBUG` build 中调用；声明 `debugger=true`、`trace.cpu=true`、三种 address space 和数值 limits；证据：2026-09-02，Windows named pipe 返回 `debugger=true`、`trace.cpu=true`、`segmented`、`linear`、`physical` 及配置 limits。 - [x] `RPC-B03` - non-heavy build capability 退化；在仅 `C_DEBUG` build 中调用 `agent.capabilities` 和 `trace.start`；前者 `trace.cpu=false`，后者返回 `CAPABILITY_UNAVAILABLE`；证据：2026-09-02，`Agent Debug No Heavy SDL2|x64` named-pipe 验收返回上述结果。 - [x] `RPC-B04` - `session.start` 在 fixture entry 停止；调用 start 后 `session.status.state=stopped`、`last_stop.kind=startup`、`IP=0x00000100`；证据：2026-09-02，`AGENTFIX.COM` 返回 `0x0812:0x00000100` 和 `kind=startup`。 - [x] `RPC-B05` - 单 session 限制；会话处于 stopped 或 running 时再次调用 `session.start`；返回 `SESSION_BUSY`，原会话的 state_revision 和 breakpoint 集合不变；证据：2026-09-02，重复 start 返回 `SESSION_BUSY`，原 session revision 保持 `1`。 - [x] `RPC-B06` - 状态机错误；会话 running 时调用 `state.get_registers`、`memory.read`、`memory.write`；全部返回 `TARGET_RUNNING`，不改变 target；证据：2026-09-02，`AGENTRUN.COM` continue 后三种请求均返回 `TARGET_RUNNING`。 - [x] `RPC-B07` - operation timeout 语义；continue 后用极短 timeout 调用 wait；返回 `running=true` 且没有 stop_reason，随后 pause/wait 可以得到 `kind=pause`；证据：2026-09-02，`execution.wait(timeout_ms=1)` 返回 `running=true`，pause 后 wait 返回 stopped、revision `2` 和 `kind=pause`。 - [x] `RPC-B08` - request id 幂等；相同 payload 使用同一 request id 重发 memory.write，验证只执行一次；同 id 不同 payload 返回 `REQUEST_ID_CONFLICT`；证据：2026-09-02，干净 runtime 中对 `AGENTFIX.COM` 的 `0x0812:0x00000200` 写入 `AGENT` 后，两次 `id=write-once` 响应均为 revision `2`、SHA-256=`b62db7d0b79d11100c7103df97dd87bc90ff6ffbd88de6019cf1e933069c8525`；相同 id 的不同 payload 返回 `REQUEST_ID_CONFLICT`。 ### 阶段 C：Debugger Adapter - [x] `RPC-C01` - `state.get_registers` 结构化完整性；fixture entry 和单步后分别读取；general、segments、instruction_pointer、flags、cpu_mode 均存在且使用固定宽度十六进制字符串；证据：2026-09-02，Windows named-pipe 对 `AGENTFIX.COM` entry 返回 `cpu_mode=real`、`IP=0x00000100`、8 个 32-bit general registers、6 个 16-bit segments 和 32-bit flags；单步结果亦返回完整 snapshot。 - [x] `RPC-C02` - `execution.step(mode=into)`；从 fixture entry 单步一次；AX=`0x00001234`、IP=`0x00000103`、stop_reason.kind=`step`；证据：2026-09-02，Windows named-pipe 对 `AGENTFIX.COM` 返回 `kind=step`、`EAX=0x00001234`、`instruction_pointer=0x00000103`。 - [x] `RPC-C03` - `execution.step(mode=over)`；在 call、int、rep 夹具扩展用例中验证越过语义；执行后不进入被越过目标，且没有泄漏临时 breakpoint；证据：2026-09-02，Windows named-pipe 使用独立进程运行 `AGCALL.COM`、`AGENTINT.COM`、`AGENTREP.COM`；分别停在 `0x00000103`、`0x00000102`、`0x0000010B`，REP destination 为 `41 42`，三次 `breakpoints.list` 均无泄漏的用户可见 breakpoint，over 响应均含 registers snapshot。 - [x] `RPC-C04` - `memory.read`；读取 fixture code 开头 6 bytes；base64 decode 后严格等于 `B8 34 12 BB 78 56`，byte_count=6，SHA-256 匹配；证据：2026-09-02，Windows named-pipe 分别使用 segmented、linear、physical 读取，均返回 base64 `uDQSu3hW`、`byte_count=6`、SHA-256 `99b28729074f19ab5d3edfab4f425f3d88e4601acd37ae9e6fd43e46b85a5c9f`。 - [x] `RPC-C05` - 三地址空间和失败路径；分别测试 segmented、linear、physical 成功访问，以及 invalid selector、segment limit、page not present；失败必须返回 `ADDRESS_NOT_MAPPED` 和精确 reason；证据：2026-09-02，`tests\agent\verify_memory_mapping.ps1` 在 Windows named-pipe 运行 `AGPMODE.COM`，保护模式分页停止后，三地址空间各读取 1 byte 成功；失败请求均返回 `ADDRESS_NOT_MAPPED`，cause 分别为 `invalid_selector`、`segment_limit`、`page_not_present`。 - [x] `RPC-C06` - `memory.write` 原子性；expected SHA 不匹配时验证写前后 bytes 相同；匹配时验证写后 bytes、after SHA 和 byte_count；证据：2026-09-02，Windows named-pipe 对 fixture `DS:0200` 写入 `DE AD BE EF`；错误 SHA 返回 `MEMORY_PRECONDITION_FAILED` 且读回 bytes 未变，正确 SHA 返回 `byte_count=4`、after SHA `5f78c33274e43fa9de5659265c1d917e25c03722dcb0b8d27db8d5feaa813953`，随后读回 `DE AD BE EF`。 - [x] `RPC-C07` - 执行断点；在 `CS:0106` 创建后依次调用 continue 和 wait；stop_reason.kind=`breakpoint` 且 address 与创建地址一致；证据：2026-09-02，Windows named-pipe 创建 `bp-1` 于 `CS:0x00000106`，continue/wait 返回 `kind=breakpoint`、`breakpoint_id=bp-1` 和同一 segmented address。 - [x] `RPC-C08` - breakpoint stable id；创建多个 breakpoint、删除中间一个并列出；其他 id 不改变，按原 id 删除目标不会误删；证据：2026-09-02，Windows named-pipe 创建 `bp-1`、`bp-2`、`bp-3`，删除 `bp-2` 后 list 仍为 `bp-1`、`bp-3`，再按 `bp-3` 删除成功。 - [x] `RPC-C09` - memory-change breakpoint capability；capability true 的 build 使用实际底层支持的 CPU mode 验证一次触发；capability false 的 build 返回 `CAPABILITY_UNAVAILABLE`，不得转换为 execution breakpoint；证据：2026-09-02，heavy build 在 real mode 对 `DS:0200` 创建 `bp-1` 并由 fixture 写入实际触发；`Agent Debug No Heavy SDL2|x64` capability 返回 `memory_change=false`，同一 create 返回 `CAPABILITY_UNAVAILABLE`。 - [x] `RPC-C10` - `session.stop`；running 和 stopped 状态各执行一次 stop/wait；最终 state=exited，重复 stop 为幂等且不崩溃；证据：2026-09-02，Windows named-pipe 分别在 stopped `AGENTFIX.COM` 和 running `AGENTRUN.COM` 调用 stop/wait，均返回 `state=exited`、`kind=program_exit`；exited 后重复 stop 返回 `op-stop-complete`。 ### 阶段 D：输出、trace 和兼容命令 - [x] `RPC-D01` - debugger output ring buffer；执行一个只读诊断命令后 `debug.output.read` 返回递增 sequence、非空 source 和完整 message；分页连接无重复无遗漏；证据：2026-09-02，`tests\agent\verify_output_trace.ps1` 以 `limit=1` 完整读取 `HELP`、`CPU`、`PIC` 三条输出记录，sequence 连续且 source/message 非空。 - [x] `RPC-D02` - 原始命令 allowlist；`HELP`、`CPU`、`PIC` 可执行并返回 raw_output；`RUN`、`SM`、`SR`、`BP*` 在 v1 被拒绝或由 adapter 显式同步 state，测试覆盖每种结果；证据：2026-09-02，`tests\agent\verify_output_trace.ps1` 验证三个允许命令均有非空 `raw_output`，并验证 `RUN`、`SM`、`SR`、`BP 1234:0100` 均返回 `COMMAND_REJECTED`。 - [x] `RPC-D03` - parser 失败显式化；向 parser fixture 输入缺少字段和异常空格的文本；返回 `parse_status=failed` 和未解析文本，不产生伪完整对象；证据：2026-09-02，新增 `tests\agent\agent_parser_tests.cpp` 和两个 fixture；`& '.\bin\x64\Agent Debug SDL2\dosbox-x.exe' -tests` 退出码为 0。 - [x] `RPC-D04` - CPU trace；在 heavy build 对 fixture 启动 `trace.start(instruction_count=2)` 并运行；`trace.read` 恰好得到 2 条递增 sequence，每条包含 address、instruction、register_changes；证据：2026-09-02，`tests\agent\verify_output_trace.ps1` 返回通过，两个 trace event 的 sequence 为 1、2，均包含所需字段。 - [x] `RPC-D05` - trace 隔离；连续两个独立 session 的 trace 不读取前一 session 的事件或文件；删除 runtime 后重跑结果一致；证据：2026-09-02，`tests\agent\verify_output_trace.ps1` 在同一 DOSBox-X 进程的第二个 session 启动独立 1 条 trace，并确认 sequence 从 1 开始且没有前一 session 的事件；另以原本不存在的 `tests\agent\runtime-d05-clean` 目录、显式 `agent-d05-clean.env` 和新生成 fixture 重跑同一脚本通过，未复用原 runtime 文件。 ### 阶段 E：Client 和端到端 - [x] `RPC-E01` - Python client unit tests；执行 `py -m unittest discover -s client\python\tests -t client\python -v`；退出码为 0，覆盖 model、base64、error mapping、request id retry；证据：2026-09-02，3 个 unittest 全部通过；覆盖显式 dotenv 相对路径解析、全部 v1 client method 的 typed response、base64 写入、`TARGET_RUNNING` error mapping 和相同 `request_id` 的显式重试。 - [x] `RPC-E02` - 端到端 fixture；执行 `py client\python\tests\test_e2e.py --config tests\agent\agent-test.env`；依次完成 start、breakpoint、continue、wait、register read、memory read、memory write、step、stop；退出码为 0；证据：2026-09-02，Windows named-pipe E2E 通过，同时验证 `CPU` 诊断输出和 `trace.start(instruction_count=2)` 的 session-local sequence。 - [x] `RPC-E03` - DOSBox-X 既有单元测试未回归；执行 `& '.\bin\x64\Agent Debug SDL2\dosbox-x.exe' -tests`；退出码为 0 且输出 `Unit test completed: success`；证据：2026-09-02，`-tests` 退出码为 0；当前 Windows GUI build 不向调用 PowerShell 转发测试日志，`shell.cpp` 的 `RUN_ALL_TESTS()` 返回值为进程退出状态。 - [x] `RPC-E04` - 干净运行可重复；执行 `if (Test-Path tests\agent\runtime) { Remove-Item -Recurse -Force tests\agent\runtime }; New-Item -ItemType Directory -Path tests\agent\runtime` 后，连续运行 E02 三次；三次均通过，trace 和输出无跨运行数据；证据：2026-09-02，`tests\agent\verify_client_clean_runs.ps1` 创建三个此前不存在的独立 runtime/config 目录并串行运行 E02 三次，三次均通过；每次 E2E 断言 output sequence 从 1 开始、trace sequence 恰为 1、2。 ## 12. 发布门禁 以下条件全部满足才能将 RPC 标为可用： - 阶段 A 到 E 所有必选检查项已勾选并有可访问证据。 - `docs/rpc.md` 中列出的第一期能力均映射到一个 v1 RPC method 或在 `agent.capabilities` 中有明确、可测试的 unavailable 说明。 - 所有结构化接口都有正向、非法参数、running 状态和底层 capability 缺失测试。 - Python client 的 public API 与本文第 9 节一致，且示例可通过 E02。 - 默认 transport 只暴露本机当前用户；没有未认证 TCP listener。 - 不存在依赖 parser、ncurses 截图、stdout scraping 或共享 `LOGCPU.TXT` 的核心 API 数据路径。 ## 13. 非目标与后续版本 以下能力不属于 v1，不能为了“先跑通”混入稳定 API：CPU instrumentation、全量 memory access trace、interrupt trace、file I/O trace、screenshot analysis、IDA/Ghidra 集成、多目标并行 session、远程 TCP transport 和写寄存器 API。 后续能力必须先补充本文：数据模型、capability 标志、线程模型、测试夹具和至少一个端到端检查项，然后才能实现。 --- ### Rpc (`docs/rpc.md`) 开发一个 DOSBox-X Debugger Agent Interface，将 DOSBox-X 现有 Debugger 能力封装成 Agent 可调用接口。 面向 AI Agent 的实际使用流程请先阅读：[DOSBox-X Debugger Agent 使用手册](rpc-agent-usage.md)。 协议契约与验收规范分别见：[rpc-development.md](rpc-development.md)。 目标： 让 AI Agent 能够通过结构化 API 控制 DOSBox-X 调试环境，实现： \- 启动/停止 DOS 程序 \- 控制程序运行 \- 单步执行 \- 获取 CPU 状态 \- 读取/修改内存 \- 设置/管理断点 \- 获取调试输出 \- 获取执行日志 用于： \- DOS 程序逆向分析 \- 老游戏行为分析 \- 二进制重构辅助 \- 动态验证静态分析结果 不要重新实现： \- CPU 调试逻辑 \- 内存访问逻辑 \- 断点机制 \- 单步机制 利用 DOSBox-X 已有能力。 目标： ``` Agent | RPC Interface | Debugger Adapter | DOSBox-X Debugger | DOSBox-X Emulator | DOS Program ``` 第一阶段： 不要修改： \- CPU execution core \- memory subsystem \- DOS kernel simulation 除非现有 debugger 能力不足。 不要简单暴露： ``` 输入命令 返回文本 ``` 需要提供： 结构化接口： 例如： ```json { "registers":{ "ax":123, "cs":"1000", "ip":"0020" } } ``` 而不是： ```ini AX=0000 BX=0001... ``` ```markdown AI Agent | | Agent API / RPC Layer | | DOSBox Debugger Adapter | | DOSBox-X Debugger | | DOSBox-X Runtime | | DOS Executable ``` 负责： \- 启动 DOSBox-X \- 管理调试会话 \- 维护连接状态 \- 发送 debugger command \- 获取 debugger response 抽象接口： ```scss start() stop() execute(command) status() ``` 要求： \- 不绑定具体通信方式 提供 Agent 调用入口。 要求： 支持： \- JSON RPC \- 或等价结构化接口 API 设计保持稳定。 提供： 功能： \- 加载 DOS 环境 \- 启动目标程序 继续运行程序。 对应 debugger： ``` G ``` 单步执行。 对应： ``` T ``` 暂停执行。 提供： \- 通用寄存器 \- 段寄存器 \- IP \- FLAGS 例如： 请求： ``` get\_registers ``` 返回： ```json { "ax": "...", "bx": "...", "cs": "...", "ip": "...", "flags": "..." } ``` 提供： 能力： \- 指定地址读取 \- 返回二进制数据 例如： ```json { "address":"segment:offset", "length":128 } ``` 能力： 修改目标内存。 提供： 支持： \- 地址断点 \- 内存访问断点（如果 debugger 支持） 保留底层能力。 提供： ``` execute\_debugger\_command ``` 例如： Agent 可以执行： ``` CPU MEMDUMP HELP ``` 用途： \- 调试 \- 新功能验证 \- 后续扩展 利用 DOSBox-X 已有日志能力。 提供： 例如： ``` trace.start ``` 返回结构化数据： ```json \[ { "address":"", "instruction":"", "register\_change":{} } ] ``` 如果当前 debugger 输出无法完全解析： 设计独立 parser 模块。 不要让 Agent 直接依赖 DOSBox-X 文本输出。 增加： ``` Debugger Output Parser ``` 负责： 文本： ```yaml AX=xxxx CS:IP=xxxx ``` 转换： 结构： ```json { "ax":"", "cs":"", "ip":"" } ``` ``` dosbox-agent-interface ├── server │ ├── debugger │ ├── session │ ├── command │ └── parser │ ├── api │ ├── client │ ├── tests │ └── docs ``` 目标： 成功控制 DOSBox-X debugger。 实现： \- session管理 \- command发送 \- 输出读取 \- 基础解析 实现： \- run \- step \- continue \- registers \- memory \- breakpoint 实现： \- debugger log采集 \- trace解析 \- 结构化输出 提供： Python client： 让 Agent 可以： ```javascript debugger.run() debugger.get\_registers() debugger.step() debugger.read\_memory() ``` 保持接口可扩展： 未来可以加入： \- CPU instrumentation \- memory access tracing \- interrupt tracing \- file IO tracing \- screenshot analysis \- IDA/Ghidra integration 但不要在当前项目阶段实现。 完成后，一个 Agent 可以： 1\. 启动 DOS 程序 2\. 连接 debugger 3\. 设置断点 4\. 运行到断点 5\. 获取寄存器状态 6\. 读取内存 7\. 单步执行 8\. 收集执行日志 9\. 根据结构化数据进行分析 最终目标： 把 DOSBox-X Debugger 转换为一个 AI Agent 可以调用的动态分析后端，而不是重新开发一个调试器。 --- ### PLANS General TODO (`docs/PLANS/General TODO.txt`) Some plans for future DOSBox-X developments =========================================== * Clock domains - Current DOSBox-X emulation is CPU-centric, which is OK, I guess, but not ideal for much of the emulation. - Many delays and timing are based on floating point values in milliseconds. If clock domains were implemented, devices could count very accurately in clocks at a rate of their choosing and better maintain timing that way. - Clock domains could be coded to cascade as well. The obvious example is original PC hardware where the ISA 14.31818MHz clock is divided by 12 to produce the 1.193182MHz clock used to drive the 8253 PIT and other system components. - One part of the emulation that would definitely benefit from clock domain based timing would be the VGA emulation. Pixel and scanline counting would become very accurate if counted with integers according to pixel clocks instead of using the floating math it does now. - Another part of the emulation that would benefit would be the I/O delay emulation. Instead of bumping a fixed number of CPU cycles, the code could more accurately delay CPU timing according to, say, an ISA bus clock domain set to 8.3333MHz and knowledge of a typical ISA bus cycle. Or, if the I/O cycle is directed at the PCI bus (clock domain set to 33.3333Mhz) delays emulated according to PCI bus cycle or PCI burst I/O cycle counting. * Runtime hooks for patching - If told to do so, DOSBox-X can look for an IPS file (or DOSBox-X equivalent) and when a particular EXE is loaded into memory, patch the executable image before executing it. The idea being that DOS gamers can use this mechanism to fix bugs in a game or demo without modifying the original EXE. It can also be a fun way to cheat :) - We could assist this mechanism with the ability for the user to force an EXE file to be loaded at a specific segment, in case the patch is sensitive to segment values. Loading must fail of course if we cannot put the EXE at that location. - We could also add the ability for DOSBox-X to patch data files when the game reads them from disk, to hack textures, sounds, etc. Again the data file remains unmodified on disk. - On a related note, it would be cool if we could implement something like Bochs volatile mode where data is read from the file, but writes to the file are directed to a temporary file which is thrown away on exit, keeping your files unmodified. - If executable unpacking is involved, some additional tools could be provided where the patch does not modify the loaded EXE, but can be instructed to patch an area of memory when the instruction pointer hits a specific location in the EXE (usually when the EXE has finished unpacking and checking it's integrity). * Support for loading BIOS images and mapping them into adapter ROM - We could test various VGA BIOSes in DOSBox-X that way - We could also run old motherboard BIOSes in DOSBox-X that way * Convert I/O and memory handler callback system to on-demand bus mapping and tracing design - Instead of statically assigning I/O and memory handlers, all handlers are initialized to a lookup function. - The DOSBox motherboard emulator at this point would also maintain its own setup of a system bus hierarchy (system resources and RAM vs forwarding to the ISA bus, or a PCI-based system that forwards to PCI which can forward via PCI-ISA bridge to ISA, etc.). - When the lookup function is called, the function goes through the "slow" path where it asks the motherboard what function to call, and the motherboard returns a function pointer according to its designation of routing, which is then written into the I/O handler. In other words, once an I/O port has been accessed, further access is a direct call to the I/O callback. - If resources or mappings change on the bus, then all I/O ports affected have the I/O handler changed back to the slow path lookup function, so that changes take effect when the lookup function returns the new result according to changed routing. - If the memory access involves resources that don't overlap but are too small for DOSBox's 4KB granularity, then the "slow path" function will always be invoked for that memory I/O because it is impossible to write both callbacks to that memory page callback. - Once all I/O and memory resources are routed this way, it becomes easy for DOSBox to emulate devices that can change I/O ports such as ISA PnP type devices that allow you to change the base I/O port of a resource. Sound Blaster ISA PnP emulation will no longer emit a warning when Windows or any DOS program attempts to reprogram the base address. It would also allow DOS or Windows to change, relocate, enable/disable PCI resources. The only performance penalty is that every time a mapping changes, the emulator has to invalidate all I/O and memory callbacks to allow the "slow" path to do its work again. - The reason we don't do bus lookup every time is one of performance. DOSBox-X already has an array of IO handlers and memory page handlers, so why not make use of it as a cache for callbacks once the slow path has been taken? Doing it this way should have almost no impact on emulator performance. * Motherboard & BIOS oriented system construction - Rather than the one way the DOSBox kernel does it, the user could instead specify that DOSBox-X emulate a particular motherboard with a particular BIOS. The code could then construct I/O and memory management to emulate the known chipsets on that board. Example: If instructed to, it would emulate the system board on an old laptop of mine with a Chips & Tech 386 AT ASIC, or it could emulate an old AST motherboard and instruct PIC emulation to act like a Unicorn Microelectronics UM8259A-2. The default of course would carefully mimic for backwards compatibility the DOSBox mainline "motherboard" and "bios" configuration. - Such construction may include (if on original hardware), VGA emulation tuned to mimic the onboard VGA of the original hardware if user says it should be enabled. * CD audio - Refactor the code to ensure we feed out CD audio in a way the guest can track its time accurately. - IDE/ATAPI emulation needs to support the mode select or whatever commands used by DOS programs to change CD audio volume. * Full Intellimouse scrollwheel emulation - Apparently it's possible to respond to scrollwheel events via SDL, so add code to receive them. - Add code to store scroll wheel deltas in mouse event queue. - Add code to take overall scrollwheel delta and transmit as 4th byte in Intellimouse mouse on AUX. - And toughest of them all: Figure out how the hell Windows 98 is able to use scrollwheel data when it is still reliant on the INT 15h device callback that's documented only to carry the X, Y, and button state data. If VirtualBox can do it, so can I! The ONLY documentation I have on this are snippets of code from CuteMouse that is apparently able to turn on PS/2 BIOS scroll wheel capture for DOS. In any case perhaps the best course of action is to write a program in DOSLIB that hooks the device callback and records the values on the stack during the call, with and without the Intellimouse knock sequence in effect. * Multiple DOS kernel emulation - Instead of only emulating a general mishmash of MS-DOS 5.0 to 6.x syscalls, allow dosbox-x.conf setting (and command line at runtime) to choose that a particular brand and version is emulated. For example, if you say that you want DOSBox-X to emulate MS-DOS 3.3, then it will return values and act like MS-DOS 3.3 (including the shorter form of the disk parameter table prior to MS-DOS 4.0). With the current code, if you say that you want DOSBox-X to emulate MS-DOS 7.1, then it tries to return values and act like MS-DOS 7.1 (including support for FAT32 drives and long filenames). * Misc - If PCI emulation is enabled, then byte-sized writes to port 0xCF9 should follow Intel PIIX chipset documentation: if bit 1 changes from 0 to 1 the system undergoes a "soft reset", and if bit 2 changes from 0 to 1 the system undergoes a "hard reset". Reset method confirmed working on an old Pentium system of mine, and it complements the keyboard and port 0x92 methods well. - So what exotic method are older demos using the clear the screen anyway? Can you figure it out? Their failure to clear the screen properly is causing graphic anomalies during certain parts. - Memory and I/O resource management code. Rather than just pick regions of memory and I/O space, the emulation code should have a way to mark that such and such region has been taken for e.g. "VGA BIOS" and "ROM BIOS". It would make it possible for adapter emulation to know if the region it is trying to take is already occupied, and provide a way for said adapter (unless instructed by user) to automatically pick a location that it can claim. It would also make for good debugging information. - Peripherals should assign themselves to a bus if possible. Sound Blaster emulation for example would register itself to the I/O ports on the ISA bus. Voodoo 3dfx emulation would register itself to the PCI bus. - Support not just for declaring an ISA bus and its frequency, but also for ISA devices that you would declare "attached" to the bus (same with PCI). Instead of DOSBox's design where peripherals are static and only one at a time is possible, we would allow you to declare peripherals and declare them attached to the ISA bus slots. If you want to emulate a DOS machine with three Sound Blaster cards, you could declare three of them (with different resources of course) and have them attached to the ISA bus. - If I wanted DOSBox-X to emulate a VESA local bus peripheral, would I have to do anything extra to do it? - CMOS emulation. The current code sucks. Why exactly are all values hard-coded? That's not how the RTC/CMOS works! The real deal has clock registers in the first 14 bytes and the rest is non-volatile RAM! Fix! - RTC/CMOS emulation: support non-realtime mode (where the clock does NOT follow host time but follows time accurately according to emulation). But of course the option to follow realtime would also be available. - IDE/ATA emulation: What exactly is triggering the debugger when you boot MS-DOS 6.22 using the OAK CD-ROM driver? It needs to be removed, it makes using MS-DOS 6.22 with debugger builds annoying. - VGA Linewise rendering is currently hardcoded for 16bpp RGB. Why? Add option to allow the user to choose 32bpp RGBA linewise rendering. - VGA output: add optional scalar or post-processing that can auto-detect fake high-color VGA modes in demos, and process the scanlines such that the RGB spreads vertically to mask the raster scanline effect (so that the picture is presentable even if rescaled later) - VGA output: Add support for VESA 24bpp modes (DOSBox's scalers are currently hard-coded for 8/16/32bpp). There are some demos (a 1997 Coma demo comes to mind) where if a 24/32bpp mode is chosen the demo assumes 24bpp RGB arrangement. - VGA debugging function: Add code that, if triggered, allows DOSBox to pause emulation and allow you to examine/play with the VGA palette, DAC, registers, screen bytes, etc. The reason I want to do this is that there are some demos (listed elsewhere in the BUGS section) where for whatever reason there are VGA palette errors, some entries remain black despite palette animation. The "debugger" would overlay the VGA screen, of course. - VGA "fixed output" mode. A scalar would be invoked if enabled that upscales all VGA raster output to a specific resolution and frame rate (the frame rate would be 250Hz, 300Hz, or 1000Hz so that refresh rate changes are better represented accurately). This would be one way to capture demos that change VGA modes once per section in one AVI file. Scalar would include common VGA resolutions like 640x400, 640x480, 800x600, and 1024x768. The scalar could also distort the picture if the CRT timing is known to cause older VGA monitors to squeeze the picture (such as some older DOS demos that speed up vertical retrace knowing it would cause older VGA monitors to focus the raster beam in the center for a "widescreen" effect). - VGA overscan border rendering. Update/rewrite the raster linewise renderers to include overscan border in output if asked to. - ODML AVI capture. The current code works great until you hit 2GB, then falls apart. I would be willing to throw in code I've written for other professional projects to write AVI files that are ODML compliant to enable longer captures >= 2GB (contain both the AVIOLDINDEX and ODML AVI index). --- ### PLANS VMX Core (Using Linux KVM Virtualization Driver) (`docs/PLANS/VMX core (using Linux KVM virtualization driver).txt`) For more demanding emulation I thought it might be a neat idea if DOSBox could use the KVM virtualization driver in Linux (and eventually, equivalent interfaces in other OSes) which could then be selectable by setting core=vmx. To explain: Almost all modern CPUs have virtualization extensions built in (Intel VMX and AMD AMD-V). In cases where a DOS or Windows game is very demanding, why not code DOSBox to allow using those virtualization extensions? If done right, the game could run at almost native speed of the processor. It would also open the possibility of running more modern OSes under DOSBox (ones that would normally run too slow in the full and dynamic cores). --- ### Windows 3.1 INSTALL (`docs/Windows 3.1/INSTALL.MD`) # Installing Windows 3.X in DOSBox-X (instructions tested against `dosbox-x-windows-20180901-124401-windows.zip`) ## Remarks ### Mouse For the mouse to work after Windows installation, either of the following scenarios has to be met: Scenario 1 - Mouse integration driver is not installed - Mouse is captured with Ctrl+F10 - DOSBox-X has been reset at least once with F11+R Scenario 2 - Mouse integration driver is installed and defined in dosbox-x.conf - DOSBox-X has been reset at least once with F11+R ### Video This guide assumes you are using the default DOSBox-X machine `svga_s3` for which DOSBox-X sets video memory to `512Kb` by default. To use higher resolutions and color depths, adjust `vmemsize` in `[dosbox]` accordingly to your needs (see `dosbox-x.reference.conf`). ## Installation ### Windows 3.1 - download and extract [DOSBox-X](https://github.com/joncampbell123/dosbox-x/releases) - next to `DOSBox-X.exe`, create a `VirtualHD` directory - inside `VirtualHD` create a `WINSETUP` directory - extract Windows 3.1 floppies into it ([7-Zip](https://www.7-zip.org/) can extract IMG files) - next to `DOSBox-x.exe` create a `dosbox-x.conf` with the following content: ``` [cpu] cycles=25000 integration device=true [sblaster] irq=5 [autoexec] @ECHO OFF PATH=%PATH%;C:\WINDOWS MOUNT C .\VirtualHD C: ``` - start DOSBox-X - launch Windows installation `C:\WINSETUP\SETUP.EXE` - install Windows using `Express Setup (Recommended)` option - follow the installation instructions - when asked, reboot, start Windows by typing `WIN` ### Mouse integration driver - launch `Windows Setup` program - open menu item `Options\Change System Settings` - in `Mouse`, pick `Other mouse (Requires disk from OEM)...` - type location to where you've extracted the driver and press `OK` - `DOSBox-X Mouse Pointer Integration` is selected by default - press `OK` - press `OK` again - press `Restart Windows` ### S3 video driver - launch `Windows Setup` program - open menu item `Options\Change System Settings` - in `Display`, pick `Other display (Requires disk from OEM)...` - type location to where you've extracted the driver and press `OK` - `S3 Trio64V 1.70.04 640x480 256` is selected by default - press `OK` - press `OK` again - when asked for the driver location again, type `C:\WINDOWS\SYSTEM` - press `OK` - press `Restart Windows` ### Sound Blaster 16 sound driver - installation must be done from command-prompt, exit Windows if not already - `cd` to where you've extract the driver and run `INSTALL.EXE` - choose `Full Installation (Recommended)` - follow installation instructions - when asked to overwrite some files, choose `Backup` - restart DOSBox-X - start Windows - the installer finishes with setting up the group of audio software for Windows ## Links Here are some extra references to further tweak your installation: [Windows 3.1x DOSBox Guide](https://www.vogons.org/viewtopic.php?t=9405) [Windows 3x Downloads](http://www.sierrahelp.com/Utilities/Emulators/DOSBox/3x_Downloads.html) [VOGONS Vintage Driver Library](http://vogonsdrivers.com/) Tip : backup your working installation before attempting to install other drivers. --- ### PLANS Sound Blaster AWE32 Emulation AWE32P10 (`docs/PLANS/Sound Blaster AWE32 emulation/AWE32P10.TXT`) ========================================================= || The Un-official Sound Blaster AWE32 Programming Guide || || || || by Vince Vu a.k.a. Judge Dredd || || || || V.1.0 September 28, 1995 || ========================================================= Table of Contents ----------------- 1. Last minute notes I. Introduction II. Understanding the registers III. Detection IV. Initialization V. Sample uploading/downloading VI. Playing a sample VII. The registers VIII. Effects Engine data. IX. Other documents you should refer to. 1. Last minute notes ----------------------- Yes, I know most of this is poorly written. (Thoughts not carefully organized). But if you have any questions, of misunderstandings, or difficulties, *mail me*. I like getting mail. It helps feed my ego ;) BTW Don't believe any of the b.s. Creative Labs says about the effects engine on the EMU8000 not being programmable, because it is. Please pester them to release info on the effects engine. Greets: Morbius My homie. Thor Cool Hungarian dude that wrote AWEPlay BLR BlazeRuner, long time no see. Darron Myrick Here's the whole enchilada. Mathias Hjelt Now, you can tweak your AWE32 to perfection. Maybe... And... All the other guys, gals, neuters, etc... not mentioned. I. Introduction ------------------ Mail all questions, comments, etc... to jdredd@netcom.com Well, a month overdue, but its finally here. This guide will attempt to describe in detail how to program the EMU8000 chip on the Sound Blaster AWE32, directly. This past summer, I spent nearly a month, figuring out how to program the EMU8000 on a low-level. And I succeeded. This is a compilation of all that I figured out, and please, if you use this information in any production, program, etc..., acknowledge me. Why am I writing down and distributing my hard-earned knowledge? Well, after reading an AWE32 in demos thread on comp.sys.ibm.pc.demos, I realized that the best way to improve the AWE32's status would be to release a programming guide. This document will not cover Sound Blaster basics, only the EMU8000 sub-system on the SBAWE32 card. If you need info on programming the OPL3, the DAC/ADC, or the mixer then you need to refer to a SB16 programming guide. Umm... please refer to the register list in section VII, when reading this document, in case you have trouble with recognizing what register 0x5400 or something else is. The EMU8000 Specs: 32 Oscillators of 16-Bit, 44100Hz sound. 32 Megabytes of address space. The first 4 or reserved for ROM, and the last 28 are for RAM. Internally the EMU8000 only has 24-Bit address space. So only 16-bit samples can be used. 3 Point sample interpolation. 1 Onboard Digital Effects Processor. 2 Low Frequency Oscillators' per oscillator. 2 DAHDSR Envelope Generators' per oscillator. 1 Resonant Filter per oscillator. 32 Independant reverb send levels for each oscillator. 32 Independant chorus send levels for each oscillator. 1 SP/DIF Digital Output. Glossary: EG1 Envelope Generator 1 (Filter/Pitch) EG2 Envelope Generator 2 (Volume) LFO1 Low Frequency Oscillator 1 (Pitch/Volume/Filter) LFO2 Low Frequency Oscillator 2 (Pitch) FilterQ Filter Resonance Explaination of EG's: Delay: The time before the attack, during which the envelope's level stays at 0. Attack: The time during which the envelope's level fades from 0 to peak. Hold: The time before the decay, during which the envelope's level stays at peak. Decay: The time during which the envelope's level fades from peak to the sustain level. Sustain:The level at which the envelope stays before release. Release:The time during which the envelope's level fades from sustain to 0. _ / \ __ ____/ \ ^ ^ ^^ ^ ^ | | || | | | | || | - Release Time | | || ---- Sustain Level | | | ------- Decay Time | | ------- Hold Time | ----------- Attack Time ---------------- Delay Time II. Understanding the registers --------------------------------- Here are the ports that are mapped to the EMU8000 (they are all word ports): 0x620 Data Read/Write 0x622 Data Read/Write 0xA20 Data Read/Write 0xA22 Data Read/Write 0xE20 Data Read/Write 0xE22 Index Simply replace the middle digit of the port with the proper base. e.g. 0xE30 if the SBAWE32's base port is 0x230. The rest of this document will refer to the ports as 0x620, 0xE20, etc... But remember to replace the middle digit. In order to write or read from a register, one would first have to select the proper index (if not already selected) by writing the index to the index port, 0xE22. Example: outpw(0xE22, 0); That's it. Now that you have selected the proper index you can now write/read the data from the anyone of the registers that uses that index, by reading/writing the correct port. Example: outpw(0xE20, 57344); //Port 0xE20, Index 0, is the pitch register //for oscillator 0. Some registers, however, maybe 32-bits (double word), writing to them is almost the same as writing to a 16-bit register as above. Example: outpw(0xA20, 0x0000); //Port 0xA20, Index 0 is the //FilterQ/DramControl/PlayPosition register outpw(0xA22, 0x0002); //for oscillator 0. What we just did is write //the low word to //0xA20, and then write the high word to //0xA22. Reading from a register is exactly the same as writing, except you read (in) from that port instead of write. IMPORTANT!!! MUST READ!!! For the rest of this document I will refer to registers the following way: 0x141d ^^^^ || | || - The oscillator number in hex. Here it is 0x1d or 29 in || decimal. || | --- The port. 0=0x620, 2=0x622, 4=0xA20, 6=0xA22, 8=0xE20. Here it | is 4, 0xA20. | ---- The base index. Multiply by 32 and add the oscillator number to get the real index, to write to 0xE22. Here it is 1, so the base index is 32. III. Detection --------------- There are several methods for detecting the presence of the EMU8000. But I will only show one way. Here's a simple detection routine in Ansi C: int awe32Detect(unsigned baseport) { int scratch; outpw(0xc02+baseport, 224); //Select index 224 by writing to 0xe22 inpw(0xc00+baseport); //Read from port 0xe20 scratch = inpw(0xc00+baseport); //Read from port 0xe20 again if((scratch&0xf) != 0xc) return 1; //Not detected outpw(0xc02+baseport, 51); //Select index 51 (base index 32, //oscillator 29) scratch = inpw(0x800+baseport); //Read from port 0xa20 if((scratch&0x7e) != 0x58) return 1; //Not detected outpw(0xc02+baseport, 52); //Select index 52 (base index 32, //oscillator 30) scratch = inpw(0x800+baseport); //Read from port 0xa20 if((scratch&0x03) != 0x03) return 1; //Not detected return 0; //Woo-hoo! Detected! } IV. Initialization -------------------- The most complex part of programming the EMU8000 is probably initialization. There are several things you can initialize, but you do not have to. You should initialize the effects engine. You should initialize OPL3/FM pass- through, which also enables DRAM refresh. And you should detect the ammount of DRAM. oh, and you should also stop all oscillators from playing, just in case they are. You should do those things in this order: Stop the oscillators from playing, just in case. Initialize the Effects Engine. Initialize the FM Passthrough (You don't need to if you use only ROM sounds) Detect the DRAM Initializing the effects engine: Registers 0x2400 through 0x361f are entirely devoted to effects engine data. In order to initialize the effects engine, here's C code to intialize the engine to Reverb&Chorus: ------------------------------------------------------------------------------- unsigned Reset_FX[128] = {...} //The real data is in section VIII. of //this document. unsigned Reverb&Chorus_FX[128] = {...} //The real data is in section VIII. of //this document. unsigned clock_wait; outpw(0xE22, 32+20); //Select base index 32+oscillator 20 outpw(0xA20, 0); outpw(0xE22, 32+21); //Select base index 32+oscillator 21 outpw(0xA20, 0); outpw(0xE22, 32+22); //Select base index 32+oscillator 22 outpw(0xA20, 0); outpw(0xE22, 32+23); //Select base index 32+oscillator 23 outpw(0xA20, 0); outpw(0xE22, 32+21); //Select base index 32+oscillator 21 outpw(0xA20, 0); for(i=0; i<32; i++) { outpw(0xE22, 96+i); //Select base index 96+oscillator //number outpw(0xA22, Reset_FX[i]); } for(i=0; i<32; i++) { outpw(0xE22, 96+i); //Select base index 96+oscillator //number outpw(0xA20, Reset_FX[32+i]); } for(i=0; i<32; i++) { outpw(0xE22, 64+(i/32)); //Select base index 64+oscillator //number outpw(0xA22, Reset_FX[64+i]); } for(i=0; i<32; i++) { outpw(0xE22, 64+(i/32)); //Select base index 64+oscillator //number outpw(0xA20, Reset_FX[96+i]); } outpw(0xE22, 32+27); //Select base index 32+oscillator 27 clock = inpw(0xA22); //This is the 44100Hz clock counter. clock += 0x400; //Add 1024 to the counter. if((clock-0x400)clock); //i.e. clock went past 65535 while(inpw(0xA22)>16); outpw(0xE22, 32+26); //Select register 0x141A outpw(0xA20, 0x1234); //Write 0x1234 to memory outpw(0xA20, 0x1234); //Write it to the next offset. outpw(0xA20, 0x1234); //Write it to the next offset. outpw(0xE22, 32+20); //Register 0x1414 outpw(0xA20, 0x0000); //Set the read offset to begginning outpw(0xA20, 0x0020); //of DRAM /* Check for wrap around */ outpw(0xE22, 32+26); //Set index for register 0x141A inpw(0xA20); //Ignore the first read. if(inpw(0xA20) != 0xFFFF) return dramsize; //Write wrapped around /* Check to make sure it was written */ outpw(0xE22, 32+20); //Register 0x1414 (read offset) outpw(0xA20, (dramsize+0x200000)&0xFFFF); outpw(0xA22, (dramsize+0x200000)>>16); outpw(0xE22, 32+26); //Set index for register 0x141A inpw(0xA20); //Ignore the first read. if(inpw(0xA20) != 0x1234) return dramsize; //Did not write! if(inpw(0xA20) != 0x1234) return dramsize; //Did not write! if(inpw(0xA20) != 0x1234) return dramsize; //Did not write! } /* Reset the oscillators */ for(i=0; i<30; i++) { outpw(0xE22, 0+i); outpw(0xA20, 0x0000); outpw(0xA22, 0x0000); outpw(0xE22, 160+i); outpw(0xA20, 0x807F); } return dramsize; -------------------------------------------------------------------------------- The dramsize is returned in words, so you need to multiply by 2 to get the number of bytes. V. Sample Uploading/Downloading ---------------------------------- When I say uploading, I mean transferring samples from your computer to the EMU8000's memory. And downloading is transferring samples from the EMU8000's memory (ROM included) to your computer. All transfers are done through I/O ports, and not DMA. This is the simplest part of programming the EMU8000. Here's C code to setup the oscillators for transfering: ------------------------------------------------------------------------------- /* i is the oscillator number (0-31) */ outpw(0xE22, 160+i); //Select Index (i won't comment these //anymore) outpw(0xA20, 0x80); //Disable EG2, register 0x5400|i outpw(0xE22, 96+i); outpw(0x620, 0x0000); //Register 0x3000+i outpw(0x622, 0x0000); //Set RawFilter/Raw Volume to 0 outpw(0xE22, 64+i); outpw(0x620, 0x0000); //Register 0x2000+i outpw(0x622, 0x0000); //I don't know what this does outpw(0xE22, 192+i); outpw(0x620, 0x0000); //Register 0x6000+i outpw(0x622, 0x0000); //Set pan, and loopstart to 0 outpw(0xE22, 224+i); outpw(0x620, 0x0000); //Register 0x7000+i outpw(0x622, 0x0000); //Set chorus send, and loopend to 0 outpw(0xE22, 32+i); outpw(0x620, 0x0000); //Register 0x1000+i outpw(0x622, 0x4000); //Set frequency to ??? outpw(0xE22, 0+i); outpw(0x620, 0x0000); //Register 0x0000+i outpw(0x622, 0x4000); //I don't know what this does outpw(0xE22, 0+i); outpw(0xA20, 0x0000); //Register 0x0400+i outpw(0xA22, 0x0600); //Write mode //replace 0x0600 with 0x0400 ------------------------------------------------------------------------------- You can setup as many oscillators as you want for transferring. You can setup as little as 1 and as many as 32. But if you setup 32, remember to reinitialize fm passthrough. The more oscillators you use the faster the transfers. You do not have to setup a block of oscillators for transfers. You can setup say, for example oscillators: 0, 3, 7, and 12. That is perfectly fine, and you can transfer samples, while still playing. Just as long as the oscillator you setup for transfer is not in use (playing, or transferring). Here's pseudo-c code to upload (assuming the oscillators are setup already): outpw(0xE22, 32+22); outpw(0xA20, write_offset&0xFFFF); outpw(0xA22, write_offset>>16); outpw(0xE22, 32+26); for(i=0; i>16); outpw(0xE22, 32+26); inpw(0xA20); //Discard the first read //I don't know why, but do it. for(i=0; i 23756.8ms] ------------------- 0x4600 O 128 A22 W EG2 Hold Time/EG2 Attack Time The upper byte of the register contains the time at which EG2 will stay at peak before beginning the decay. RegisterValue = 127-(HoldTime_ms/92) The range for HoldTime is [0ms -> 11684ms] The lower byte of the register contains the attack, the time during which EG2 fades from 0 to peak. if AttackTime_ms >= 360ms: RegisterValue = 11878/AttackTime_ms - 1 if AttackTime_ms < 360ms and AttackTime != 0: RegisterValue = 32 + [16/log(1/2)] * log(360_ms/AttackTime_ms) if AttackTime_ms == 0 RegisterValue = 0x7F The range for attack time is [0ms -> 11878ms] ------------------- 0x4800 O 128 E20 W LFO1 to Volume/LFO1 Frequency The upper byte contains the the ammount the volume of the oscillator will be changed by at the peak of LFO1. This byte is signed. RegisterValue = Volume_dB * 12/128 The range for Volume_dB is [-12dB -> 12dB] The lower byte contains the frequency at which LFO1 oscillates. RegisterValue = Frequency_Hz * 21.44/256 ------------------- 0x5400 O 160 A20 W EG2 Override/EG2 Sustain/EG2 Decay, Release or EG2 Ramper If the MSB (most significant bit or bit 15) of this register is set, the Decay/Release will begin immediately, overriding the Delay, Attack, and Hold. Otherwise the Decay/Release will wait until the Delay, Attack, and Hold are finished. If you set the MSB of this register, you can use it as a volume ramper, as on the GUS. The upper byte (except the MSB), contains the destination volume, and the lower byte contains the ramp time. Sustain Portion: RegisterValue = SustainLevel_dB*4/3 Decay/Release Portion: RegisterValue = 2*log(1/2)*log(23756_ms/DecayTime_ms) range: [0ms, 23756ms], obviously at 0ms, the register value is 127. Notes!!! Setting the sustain of this register above zero, turns on the volume. Basically, setting this register with proper decay, and sustain time (any value other than zero), will start the _audible_ playing of the sample. (remember: the emu is always constantly playing). Setting the sustain to zero, and the decay to appropriate time, will start the release of the sample. Thus turning off the sample, _audibly_. What I did in my modplayer (AWEMod), is each time I turned the note on, I set the upper byte to 0xFF, which sets the upper byte, and sets the sustain level to 96dB, and set the decay time to 0x7F (0 ms). This gave me the sharpest attacks, because EG2, skips the delay, attack, hold, and goes straight to sustain. ------------------- 0x5600 O 160 A22 W LFO1 Delay Time This is exactly the same as EG1 Delay Time, register 0x4400. ------------------- 0x5800 O 160 E20 W LFO2 to Pitch/LFO2 Frequency The format of the upper byte (LFO2 to Pitch) is exactly the same as for EG1 (register 0x2800), and the format of the lower byte is exactly the same as for register 0x4800 (LFO1 Frequency). ------------------- 0x6000 O 192 620 D Pan Level/Loop Start Offset The highest byte contains the pan level. Zero is full right, 128 is dead center, and 255 is full left. Simple. The last 3 bytes are the Loop Start Offset. ------------------- 0x6400 O 192 A20 W EG1 Delay Time Same as EG2 Delay Time, register 0x4400. ------------------- 0x6600 O 192 A22 W EG1 Hold Time/EG1 Attack Time Same as EG2 Hold/EG2 Attack, register 0x4600. ------------------- 0x7000 O 224 620 D Chorus Send/Loop End Offset The highest byte contains the Chorus Send level. Zero is none, and 255 is full. The last 3 bytes contain the Loop End Offset. Notes!!! Setting the Loop Start Offset and the Loop End Offset to the same value, will cause the oscillator to loop the entire memory. ------------------- 0x7400 O 224 A20 W EG1 Override/EG1 Sustain/EG1 Decay, Release or EG1 Ramper This is exactly the same as register 0x5400. Except it controls the filter cutoff, and pitch of the oscillator. ------------------- 0x7600 O 224 A22 W LFO2 Delay Same as LFO1 Delay, register 0x5600. VIII. Effects Engine Data ------------------------- Data to reset/clear the engine, not quite sure what it does (128 Words): 0x3FF,0x30,0x7FF,0x130,0xBFF,0x230,0xFFF,0x330,0x13FF,0x430,0x17FF,0x530, 0x1BFF,0x630,0x1FFF,0x730,0x23FF,0x830,0x27FF,0x930,0x2BFF,0xA30,0x2FFF,0xB30, 0x33FF,0xC30,0x37FF,0xD30,0x3BFF,0xE30,0x3FFF,0xF30,0x43FF,0x030,0x47FF,0x130, 0x4BFF,0x230,0x4FFF,0x330,0x53FF,0x430,0x57FF,0x530,0x5BFF,0x630,0x5FFF,0x730, 0x63FF,0x830,0x67FF,0x930,0x6BFF,0xA30,0x6FFF,0xB30,0x73FF,0xC30,0x77FF,0xD30, 0x7BFF,0xE30,0x7FFF,0xF30,0x83FF,0x030,0x87FF,0x130,0x8BFF,0x230,0x8FFF,0x330, 0x93FF,0x430,0x97FF,0x530,0x9BFF,0x630,0x9FFF,0x730,0xA3FF,0x830,0xA7FF,0x930, 0xABFF,0xA30,0xAFFF,0xB30,0xB3FF,0xC30,0xB7FF,0xD30,0xBBFF,0xE30,0xBFFF,0xF30, 0xC3FF,0x030,0xC7FF,0x130,0xCBFF,0x230,0xCFFF,0x330,0xD3FF,0x430,0xD7FF,0x530, 0xDBFF,0x630,0xDFFF,0x730,0xE3FF,0x830,0xE7FF,0x930,0xEBFF,0xA30,0xEFFF,0xB30, 0xF3FF,0xC30,0xF7FF,0xD30,0xFBFF,0xE30,0xFFFF,0xF30 Reverb&Chorus Algorithm Data [Hall 2, Chorus 3] (128 Words): 0x0C10,0x8470,0x14FE,0xB488,0x167F,0xA470,0x18E7,0x84B5,0x1B6E,0x842A,0x1F1D, 0x852A,0x0DA3,0x0F7C,0x167E,0x7254,0x0000,0x842A,0x0001,0x852A,0x18E6,0x0BAA, 0x1B6D,0x7234,0x229F,0x8429,0x2746,0x8529,0x1F1C,0x6E7,0x229E,0x7224,0x0DA4, 0x8429,0x2C29,0x8529,0x2745,0x7F6,0x2C28,0x7254,0x383B,0x8428,0x320F,0x8528, 0x320E,0xF02,0x1341,0x7264,0x3EB6,0x8428,0x3EB9,0x8528,0x383A,0xFA9,0x3EB5, 0x7294,0x3EB7,0x8474,0x3EBA,0x8575,0x3EB8,0x44C3,0x3EBB,0x45C3,0x0000,0xA404, 0x0001,0xA504,0x141F,0x671,0x14FD,0x287,0x3EBC,0xE610,0x3EC8,0xC7B,0x031A, 0x07E6,0x3EC8,0x86F7,0x3EC0,0x821E,0x3EBE,0xD280,0x3EBD,0x21F,0x3ECA,0x0386, 0x3EC1,0xC03,0x3EC9,0x31E,0x3ECA,0x8C4C,0x3EBF,0xC55,0x3EC9,0xC280,0x3EC4, 0xBC84,0x3EC8,0xEAD,0x3EC8,0xD380,0x3EC2,0x8F7E,0x3ECB,0x219,0x3ECB,0xD2E6, 0x3EC5,0x31F,0x3EC6,0xC380,0x3EC3,0x327F,0x3EC9,0x265,0x3EC9,0x8319,0x1342, 0xD3E6,0x3EC7,0x337F,0x0000,0x8365,0x1420,0x9570 Modifications to effects engine data registers to get other reverb&chorus variations (also stuff about custom effects), and different EQ's: Registers to write the Chorus Parameters to (all are 16-bit, unless noted): 0x3409 0x340C 0x3603 0x1409 (32-Bit) 0x140A (32-Bit) then write 0x8000 to 0x140D (32-Bit) and then 0x0000 to 0x140E (32-Bit) Chorus Parameters: Chorus 1 Chorus 2 Chorus 3 Chorus 4 Feedback Flanger 0xE600 0xE608 0xE610 0xE620 0xE680 0xE6E0 0x03F6 0x031A 0x031A 0x0269 0x04D3 0x044E 0xBC2C 0xBC6E 0xBC84 0xBC6E 0xBCA6 0xBC37 0x0000 0x0000 0x0000 0x0000 0x0000 0x0000 0x006D 0x017C 0x0083 0x017C 0x005B 0x0026 Short Delay Short Delay + Feedback 0xE600 0xE6C0 0x0B06 0x0B06 0xBC00 0xBC00 0xE000 0xE000 0x0083 0x0083 Registers to write the Reverb Parameters to (they are all 16-bit): 0x2403,0x2405,0x361F,0x2407,0x2614,0x2616,0x240F,0x2417, 0x241F,0x2607,0x260F,0x2617,0x261D,0x261F,0x3401,0x3403, 0x2409,0x240B,0x2411,0x2413,0x2419,0x241B,0x2601,0x2603, 0x2609,0x260B,0x2611,0x2613 Reverb Parameters: Room 1: 0xB488,0xA450,0x9550,0x84B5,0x383A,0x3EB5,0x72F4,0x72A4, 0x7254,0x7204,0x7204,0x7204,0x4416,0x4516,0xA490,0xA590, 0x842A,0x852A,0x842A,0x852A,0x8429,0x8529,0x8429,0x8529, 0x8428,0x8528,0x8428,0x8528 Room 2: 0xB488,0xA458,0x9558,0x84B5,0x383A,0x3EB5,0x7284,0x7254, 0x7224,0x7224,0x7254,0x7284,0x4448,0x4548,0xA440,0xA540, 0x842A,0x852A,0x842A,0x852A,0x8429,0x8529,0x8429,0x8529, 0x8428,0x8528,0x8428,0x8528 Room 3: 0xB488,0xA460,0x9560,0x84B5,0x383A,0x3EB5,0x7284,0x7254, 0x7224,0x7224,0x7254,0x7284,0x4416,0x4516,0xA490,0xA590, 0x842C,0x852C,0x842C,0x852C,0x842B,0x852B,0x842B,0x852B, 0x842A,0x852A,0x842A,0x852A Hall 1: 0xB488,0xA470,0x9570,0x84B5,0x383A,0x3EB5,0x7284,0x7254, 0x7224,0x7224,0x7254,0x7284,0x4448,0x4548,0xA440,0xA540, 0x842B,0x852B,0x842B,0x852B,0x842A,0x852A,0x842A,0x852A, 0x8429,0x8529,0x8429,0x8529 Hall 2: 0xB488,0xA470,0x9570,0x84B5,0x383A,0x3EB5,0x7254,0x7234, 0x7224,0x7254,0x7264,0x7294,0x44C3,0x45C3,0xA404,0xA504, 0x842A,0x852A,0x842A,0x852A,0x8429,0x8529,0x8429,0x8529, 0x8428,0x8528,0x8428,0x8528 Plate: 0xB4FF,0xA470,0x9570,0x84B5,0x383A,0x3EB5,0x7234,0x7234, 0x7234,0x7234,0x7234,0x7234,0x4448,0x4548,0xA440,0xA540, 0x842A,0x852A,0x842A,0x852A,0x8429,0x8529,0x8429,0x8529, 0x8428,0x8528,0x8428,0x8528 Delay: 0xB4FF,0xA470,0x9500,0x84B5,0x333A,0x39B5,0x7204,0x7204, 0x7204,0x7204,0x7204,0x72F4,0x4400,0x4500,0xA4FF,0xA5FF, 0x8420,0x8520,0x8420,0x8520,0x8420,0x8520,0x8420,0x8520, 0x8420,0x8520,0x8420,0x8520 Panning Delay: 0xB4FF,0xA490,0x9590,0x8474,0x333A,0x39B5,0x7204,0x7204, 0x7204,0x7204,0x7204,0x72F4,0x4400,0x4500,0xA4FF,0xA5FF, 0x8420,0x8520,0x8420,0x8520,0x8420,0x8520,0x8420,0x8520, 0x8420,0x8520,0x8420,0x8520 Registers to write the EQ Parameters to (16-Bit): Bass: 0x3601 0x3611 Treble: 0x3411 0x3413 0x341B 0x3607 0x360B 0x360D 0x3617 0x3619 Total: write the 0x0263 + 3rd parameter of the Bass EQ + 9th parameter of Treble EQ to 0x3615. write the 0x8363 + 3rd parameter of the Bass EQ + 9th parameter of Treble EQ to 0x3615. Bass Parameters: 0: 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 0xD26A 0xD25B 0xD24C 0xD23D 0xD21F 0xC208 0xC219 0xC22A 0xC24C 0xC26E 0xC248 0xC26A 0xD36A 0xD35B 0xD34C 0xD33D 0xC31F 0xC308 0xC308 0xC32A 0xC34C 0xC36E 0xC384 0xC36A 0x0000 0x0000 0x0000 0x0000 0x0000 0x0001 0x0001 0x0001 0x0001 0x0001 0x0002 0x0002 Treble Parameters: 0: 1: 2: 3: 4: 5: 6: 7: 8: 9: 10: 11: 0x821E 0x821E 0x821E 0x821E 0x821E 0x821E 0x821E 0x821E 0x821E 0x821E 0x821D 0x821C 0xC26A 0xC25B 0xC24C 0xC23D 0xC21F 0xD208 0xD208 0xD208 0xD208 0xD208 0xD219 0xD22A 0x031E 0x031E 0x031E 0x031E 0x031E 0x031E 0x031E 0x031E 0x031E 0x031E 0x031D 0x031C 0xC36A 0xC35B 0xC34C 0xC33D 0xC31F 0xD308 0xD308 0xD308 0xD308 0xD308 0xD319 0xD32A 0x021E 0x021E 0x021E 0x021E 0x021E 0x021E 0x021D 0x021C 0x021A 0x0219 0x0219 0x0219 0xD208 0xD208 0xD208 0xD208 0xD208 0xD208 0xD219 0xD22A 0xD24C 0xD26E 0xD26E 0xD26E 0x831E 0x831E 0x831E 0x831E 0x831E 0x831E 0x831D 0x831C 0x831A 0x8319 0x8319 0x8319 0xD308 0xD308 0xD308 0xD308 0xD308 0xD308 0xD3019 0xD32A 0xD34C 0xD36E 0xD36E 0xD36E 0x0001 0x0001 0x0001 0x0001 0x0001 0x0002 0x0002 0x0002 0x0002 0x0002 0x0002 0x0002 To see what the parameters do, use the program FXTweak, which I wrote and included with the programming guide. IX. Other documents you should refer to ----------------------------------------- ftp.cdrom.com:/pub/demos/music/programs/players/omega06.zip This is the only modplayer for the AWE32 that comes with source code. Although the source, isn't very helpful, but it's working code. ftp.creaf.com:/pub/creative/files/awe/awe32faq.exe Great resource. Refer to NRPN 22, in this faq, for a description of the resonance coefficients. --- ### Vs Freetype Docs FTL (`vs/freetype/docs/FTL.TXT`) The FreeType Project LICENSE ---------------------------- 2006-Jan-27 Copyright 1996-2002, 2006 by David Turner, Robert Wilhelm, and Werner Lemberg Introduction ============ The FreeType Project is distributed in several archive packages; some of them may contain, in addition to the FreeType font engine, various tools and contributions which rely on, or relate to, the FreeType Project. This license applies to all files found in such packages, and which do not fall under their own explicit license. The license affects thus the FreeType font engine, the test programs, documentation and makefiles, at the very least. This license was inspired by the BSD, Artistic, and IJG (Independent JPEG Group) licenses, which all encourage inclusion and use of free software in commercial and freeware products alike. As a consequence, its main points are that: o We don't promise that this software works. However, we will be interested in any kind of bug reports. (`as is' distribution) o You can use this software for whatever you want, in parts or full form, without having to pay us. (`royalty-free' usage) o You may not pretend that you wrote this software. If you use it, or only parts of it, in a program, you must acknowledge somewhere in your documentation that you have used the FreeType code. (`credits') We specifically permit and encourage the inclusion of this software, with or without modifications, in commercial products. We disclaim all warranties covering The FreeType Project and assume no liability related to The FreeType Project. Finally, many people asked us for a preferred form for a credit/disclaimer to use in compliance with this license. We thus encourage you to use the following text: """ Portions of this software are copyright © The FreeType Project (www.freetype.org). All rights reserved. """ Please replace with the value from the FreeType version you actually use. Legal Terms =========== 0. Definitions -------------- Throughout this license, the terms `package', `FreeType Project', and `FreeType archive' refer to the set of files originally distributed by the authors (David Turner, Robert Wilhelm, and Werner Lemberg) as the `FreeType Project', be they named as alpha, beta or final release. `You' refers to the licensee, or person using the project, where `using' is a generic term including compiling the project's source code as well as linking it to form a `program' or `executable'. This program is referred to as `a program using the FreeType engine'. This license applies to all files distributed in the original FreeType Project, including all source code, binaries and documentation, unless otherwise stated in the file in its original, unmodified form as distributed in the original archive. If you are unsure whether or not a particular file is covered by this license, you must contact us to verify this. The FreeType Project is copyright (C) 1996-2000 by David Turner, Robert Wilhelm, and Werner Lemberg. All rights reserved except as specified below. 1. No Warranty -------------- THE FREETYPE PROJECT IS PROVIDED `AS IS' WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. IN NO EVENT WILL ANY OF THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY DAMAGES CAUSED BY THE USE OR THE INABILITY TO USE, OF THE FREETYPE PROJECT. 2. Redistribution ----------------- This license grants a worldwide, royalty-free, perpetual and irrevocable right and license to use, execute, perform, compile, display, copy, create derivative works of, distribute and sublicense the FreeType Project (in both source and object code forms) and derivative works thereof for any purpose; and to authorize others to exercise some or all of the rights granted herein, subject to the following conditions: o Redistribution of source code must retain this license file (`FTL.TXT') unaltered; any additions, deletions or changes to the original files must be clearly indicated in accompanying documentation. The copyright notices of the unaltered, original files must be preserved in all copies of source files. o Redistribution in binary form must provide a disclaimer that states that the software is based in part of the work of the FreeType Team, in the distribution documentation. We also encourage you to put an URL to the FreeType web page in your documentation, though this isn't mandatory. These conditions apply to any software derived from or based on the FreeType Project, not just the unmodified files. If you use our work, you must acknowledge us. However, no fee need be paid to us. 3. Advertising -------------- Neither the FreeType authors and contributors nor you shall use the name of the other for commercial, advertising, or promotional purposes without specific prior written permission. We suggest, but do not require, that you use one or more of the following phrases to refer to this software in your documentation or advertising materials: `FreeType Project', `FreeType Engine', `FreeType library', or `FreeType Distribution'. As you have not signed this license, you are not required to accept it. However, as the FreeType Project is copyrighted material, only this license, or another one contracted with the authors, grants you the right to use, distribute, and modify it. Therefore, by using, distributing, or modifying the FreeType Project, you indicate that you understand and accept all the terms of this license. 4. Contacts ----------- There are two mailing lists related to FreeType: o freetype@nongnu.org Discusses general use and applications of FreeType, as well as future and wanted additions to the library and distribution. If you are looking for support, start in this list if you haven't found anything to help you in the documentation. o freetype-devel@nongnu.org Discusses bugs, as well as engine internals, design issues, specific licenses, porting, etc. Our home page can be found at https://www.freetype.org --- end of FTL.TXT --- --- ### Vs Freetype Docs GPLv2 (`vs/freetype/docs/GPLv2.TXT`) GNU GENERAL PUBLIC LICENSE Version 2, June 1991 Copyright (C) 1989, 1991 Free Software Foundation, Inc. 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed. Preamble The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software--to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation's software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Library General Public License instead.) You can apply it to your programs, too. When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things. To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it. For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights. We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software. Also, for each author's protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors' reputations. Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone's free use or not licensed at all. The precise terms and conditions for copying, distribution and modification follow. GNU GENERAL PUBLIC LICENSE TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION 0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The "Program", below, refers to any such program or work, and a "work based on the Program" means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term "modification".) Each licensee is addressed as "you". Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does. 1. You may copy and distribute verbatim copies of the Program's source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program. You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee. 2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions: a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change. b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License. c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.) These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it. Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program. In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License. 3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following: a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or, c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.) The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable. If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code. 4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance. 5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it. 6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients' exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License. 7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program. If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances. It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice. This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License. 8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License. 9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and "any later version", you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation. 10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally. NO WARRANTY 11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. 12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. END OF TERMS AND CONDITIONS How to Apply These Terms to Your New Programs If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms. To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the "copyright" line and a pointer to where the full notice is found. Copyright (C) This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA Also add information on how to contact you by electronic and paper mail. If the program is interactive, make it output a short notice like this when it starts in an interactive mode: Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. This is free software, and you are welcome to redistribute it under certain conditions; type `show c' for details. The hypothetical commands `show w' and `show c' should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w' and `show c'; they could even be mouse-clicks or menu items--whatever suits your program. You should also get your employer (if you work as a programmer) or your school, if any, to sign a "copyright disclaimer" for the program, if necessary. Here is a sample; alter the names: Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision' (which makes passes at compilers) written by James Hacker. , 1 April 1989 Ty Coon, President of Vice This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Library General Public License instead of this License. --- ### Vs Freetype Docs VERSIONS (`vs/freetype/docs/VERSIONS.TXT`) Due to our use of `libtool' to generate and install the FreeType 2 libraries on Unix systems, as well as other historical events, it is generally very difficult to know precisely which release of the font engine is installed on a given system. This file tries to explain why and to document ways to properly detect FreeType on Unix. 1. Version and Release numbers ------------------------------ For each new public release of FreeType 2, there are generally *three* distinct `version' numbers to consider: * The official FreeType 2 release number, like 2.7.0 or 2.10.2. * The libtool (and Unix) specific version number, like 23.2.17. This is what pkg-config freetype2 --modversion or freetype-config --version returns. * The platform-specific shared object number, used for example when the library is installed as `/usr/lib/libfreetype.so.6.17.2'. The platform-specific number is, unsurprisingly, platform-specific and varies with the operating system you are using (several variants of Linux, FreeBSD, Solaris, etc.). You should thus _never_ use it, even for simple tests. The libtool-specific number does not equal the release number but is tied to it. The release number is available at *compile* time through the following macros defined in `freetype.h': - FREETYPE_MAJOR: major release number - FREETYPE_MINOR: minor release number - FREETYPE_PATCH: patch release number See below for a small autoconf fragment. The release number is also available at *runtime* through the `FT_Library_Version' API. 2. History ---------- The following table gives, for all releases since 2.5.0, the corresponding libtool number, as well as the shared object number found on _most_ systems, but not all of them: release libtool so ------------------------------- 2.13.1 26.0.20 6.20.0 2.13.0 25.0.19 6.19.0 2.12.1 24.3.18 6.18.3 2.12.0 24.2.18 6.18.2 2.11.1 24.1.18 6.18.1 2.11.0 24.0.18 6.18.0 2.10.4 23.4.17 6.17.4 2.10.3 23.3.17 6.17.3 2.10.2 23.2.17 6.17.2 2.10.1 23.1.17 6.17.1 2.10.0 23.0.17 6.17.0 2.9.1 22.1.16 6.16.1 2.9.0 22.0.16 6.16.0 2.8.1 21.0.15 6.15.0 2.8.0 20.0.14 6.14.0 2.7.1 19.0.13 6.13.0 2.7.0 18.6.12 6.12.6 2.6.5 18.5.12 6.12.5 2.6.4 18.4.12 6.12.4 2.6.3 18.3.12 6.12.3 2.6.2 18.2.12 6.12.2 2.6.1 18.1.12 6.12.1 2.6.0 18.0.12 6.12.0 2.5.5 17.4.11 6.11.4 2.5.4 17.3.11 6.11.3 2.5.3 17.2.11 6.11.2 2.5.2 17.1.11 6.11.1 2.5.1 17.0.11 6.11.0 2.5.0 16.2.10 6.10.2 3. Autoconf Code Fragment ------------------------- Lars Clausen contributed the following autoconf fragment to check which version of FreeType is installed on a system (now updated to use `pkg-config' instead of `freetype-config'). This one tests for a version that is at least 2.10.2; you should change it to check against other release numbers. AC_MSG_CHECKING([whether FreeType version is 2.10.2 or higher]) old_CPPFLAGS="$CPPFLAGS" CPPFLAGS=`pkg-config freetype2 --cflags` AC_TRY_CPP([ #include #include #if FREETYPE_MAJOR*10000 + FREETYPE_MINOR*100 + FREETYPE_PATCH < 21002 # error FreeType version too low. #endif ], [AC_MSG_RESULT(yes) FREETYPE_LIBS=`pkg-config freetype2 --libs` AC_SUBST(FREETYPE_LIBS) AC_DEFINE(HAVE_FREETYPE,1,[Define if you have the FreeType2 library]) CPPFLAGS="$old_CPPFLAGS"], [AC_MSG_ERROR([Need FreeType library version 2.10.2 or higher])]) ---------------------------------------------------------------------- Copyright (C) 2002-2023 by David Turner, Robert Wilhelm, and Werner Lemberg. This file is part of the FreeType project, and may only be used, modified, and distributed under the terms of the FreeType project license, LICENSE.TXT. By continuing to use, modify, or distribute this file you indicate that you have read the license and understand and accept it fully. --- end of VERSIONS.TXT --- --- ### Vs Freetype Docs Formats (`vs/freetype/docs/formats.txt`) This file contains a list of various font formats. It gives the reference document and whether it is supported in FreeType 2. Table fields ------------ wrapper format The format used to represent the font data. In the table below it is used only if the font format differs. Possible values are SFNT binary PFB binary PS a text header, followed by binary or text data LZW compressed with either `gzip' or `compress' BZ2 compressed with `bzip2'. font format How the font is to be accessed, possibly after converting the file type and wrapper format into a generic form. Bitmap formats are `BDF', `PCF', and one form of `WINFNT'; all others are vector formats. `PS' indicates third-order, `TT' second-order Bézier curves. font type Sub-formats of the font format. `SBIT' and `MACSBIT' are bitmap formats, `MM' and `VAR' support optical axes. `CFF2' supports optical axes also. glyph access If not specified, the glyph access is `standard' to the font format. Values are `CID' for CID-keyed fonts, `SYNTHETIC' for fonts that are modified versions of other fonts by means of a transformation matrix, and `TYPE_0' for PS fonts which are to be accessed in a tree-like structure. FreeType driver The module in the FreeType library which handles the specific font format. A missing entry means that FreeType doesn't support the font format (yet). Notes ----- The SFNT container format also provides `collections' (usually having the file extension `.ttc' or `.otc'). A collection contains multiple font faces that share some tables to avoid redundancy, thus reducing the file size. In FreeType, elements of a collection can be accessed with a proper face index. Both the GX and the OpenType 1.8 variation fonts provide `named instances'. FreeType maps them to face indices (they can also be accessed with the standard MM interface). Other font formats (not using the SFNT wrapper) also provide multiple faces within one file; they are marked with an asterisk (`*') in the table below. FreeType can be configured to support Mac files (on older Mac OS versions, a `file' is stored as a data and a resource fork, that is, within two separate data chunks). If a file can't be opened as a font, FreeType then checks whether it is a resource fork, trying to extract the contained font data from either a `POST' or `sfnt' resource. Please send additions and/or corrections to wl@gnu.org or to the FreeType developer's list at freetype-devel@nongnu.org (for subscribers only). If you can provide a font example for a format which isn't supported yet please send a mail too. wrapper font font glyph FreeType reference format format type access driver documents ----------------------------------------------------------------------------- --- BDF --- --- bdf 5005.BDF_Spec.pdf, X11 SFNT PS TYPE_1 --- type1 Type 1 GX Font Format [7] (for the Mac; not supported) SFNT PS TYPE_1 CID cid 5180.sfnt.pdf (for the Mac) [3] SFNT PS CFF --- cff OT spec, 5176.CFF.pdf (`OTTO' format) SFNT PS CFF CID cff OT spec, 5176.CFF.pdf SFNT PS CFF SYNTHETIC --- OT spec, 5176.CFF.pdf SFNT PS CFF2 --- cff OT spec 1.8 SFNT TT SBIT --- sfnt XFree86 (bitmaps only; with `head' table) SFNT TT MACSBIT --- sfnt OT spec (for the Mac; bitmaps only; `bhed' table) SFNT TT --- --- truetype OT spec (`normal' TT font) SFNT TT VAR --- truetype GX spec (`?var' tables) SFNT TT VAR --- truetype OT spec 1.8 (`?var' + `?VAR' tables) WOFF --- --- --- cff, Compressed SFNT, ver. 1.0 [6] truetype WOFF2 --- --- --- cff, Compressed SFNT, ver. 2.0 [6] truetype --- PS TYPE_1 --- type1 T1_SPEC.pdf (PFA, Type 1 font resource) PFB PS TYPE_1 --- type1 T1_SPEC.pdf, 5040.Download_Fonts.pdf (`normal' Type 1 font) --- PS TYPE_1 CID cid PLRM.pdf (CID Font Type 0; Type 9 font) --- PS MM --- type1 5015.Type1_Supp.pdf (Multiple Masters) --- PS CFF --- cff 5176.CFF.pdf (`pure' CFF) --- PS* CFF CID cff 5176.CFF.pdf (`pure' CFF) --- PS CFF SYNTHETIC --- 5176.CFF.pdf (`pure' CFF) --- PS CFF/MM --- cff old 5167.CFF.pdf (`pure' CFF) [3] --- PS* CFF/MM CID cff old 5167.CFF.pdf (`pure' CFF) [3] --- PS CFF/MM SYNTHETIC --- old 5167.CFF.pdf (`pure' CFF) [3] PS PS CFF --- --- PLRM.pdf (Type 2) [1] PS PS* CFF CID --- PLRM.pdf (Type 2) [1] PS PS CFF SYNTHETIC --- PLRM.pdf (Type 2) [1] PS PS CFF/MM --- --- PLRM.pdf (Type 2) [1] PS PS* CFF/MM CID --- PLRM.pdf (Type 2) [1] PS PS CFF/MM SYNTHETIC --- PLRM.pdf (Type 2) [1] --- PS --- TYPE_0 --- PLRM.pdf --- PS TYPE_3 --- --- PLRM.pdf (never supported) --- PS TYPE_3 CID --- PLRM.pdf (CID Font Type 1; Type 10 font; never supported) PS PS TYPE_14 --- --- PLRM.pdf (Chameleon font; Type 14 font; never supported?) --- PS TYPE_32 CID --- PLRM.pdf (CID Font Type 4; Type 32 font; never supported?) PS TT --- --- type42 5012.Type42_Spec.pdf (Type 42 font) PS TT --- CID --- PLRM.pdf (CID Font Type 2; Type 11 font) ? ? CEF ? cff ? --- PCF --- --- pcf X11 [4] LZW PCF --- --- pcf X11 [4] BZ2 PCF --- --- pcf X11 [4] --- PFR* PFR0 --- pfr [2] --- PFR PFR1 --- --- (undocumented, proprietary; probably never supported) --- WINFNT* --- --- winfonts Windows developer's notes [5] --- WINFNT VECTOR --- --- Windows developer's notes [5] [1] Support should be rather simple since this is identical to `CFF' but in a PS wrapper. [2] The official PFR specification is no longer available, but archive.org has archived it: https://web.archive.org/web/20091014062300/http://www.bitstream.com/font_rendering/products/truedoc/pfrspec.html https://web.archive.org/web/20081115152605/http://www.bitstream.com/font_rendering/pdfs/pfrspec1.3.pdf The syntax of the auxiliary data is not defined there, but is partially defined in MHP 1.0.3 (also called ETSI TS 101812 V1.3.1) section 7.4. https://www.etsi.org/ https://webapp.etsi.org/workprogram/Report_WorkItem.asp?WKI_ID=18799 [3] Support is rudimentary currently; some tables or data are not loaded yet. [4] See THE X WINDOW SYSTEM SERVER: X VERSION 11, RELEASE 5 Elias Israel, Erik Fortune, Digital Press, 1992 ISBN 1-55558-096-3 for a specification given in Appendix D on pgs. 436-450. However, this information might be out of date; unfortunately, there is no PCF specification available online, and this book is out of print. George Williams deduced the font format from the X11 sources and documented it for his FontForge font editor: https://fontforge.github.io/pcf-format.html [5] This is from MS Windows 3; see Microsoft's Knowledge Base article at https://support.microsoft.com/kb/65123 [6] Supported font formats are TrueType and OpenType fonts as defined in the OpenType specification 1.6 and newer. [7] `The Type 1 GX Font Format' (dated 1995-09-27) was distributed in Apple Developer CD-ROM in those days. The content of `TYP1' table is a PostScript Type 1 font without the eexec encryption. Current versions of FreeType don't not support this format, but FontForge can load it. ------------------------------------------------------------------------ Copyright (C) 2004-2023 by David Turner, Robert Wilhelm, and Werner Lemberg. This file is part of the FreeType project, and may only be used, modified, and distributed under the terms of the FreeType project license, LICENSE.TXT. By continuing to use, modify, or distribute this file you indicate that you have read the license and understand and accept it fully. --- end of formats.txt --- Local Variables: coding: utf-8 End: --- ### Vs Freetype Docs Raster (`vs/freetype/docs/raster.txt`) How FreeType's rasterizer work by David Turner Revised 2007-Feb-01 This file is an attempt to explain the internals of the FreeType rasterizer. The rasterizer is of quite general purpose and could easily be integrated into other programs. I. Introduction II. Rendering Technology 1. Requirements 2. Profiles and Spans a. Sweeping the Shape b. Decomposing Outlines into Profiles c. The Render Pool d. Computing Profiles Extents e. Computing Profiles Coordinates f. Sweeping and Sorting the Spans I. Introduction =============== A rasterizer is a library in charge of converting a vectorial representation of a shape into a bitmap. The FreeType rasterizer has been originally developed to render the glyphs found in TrueType files, made up of segments and second-order Béziers. Meanwhile it has been extended to render third-order Bézier curves also. This document is an explanation of its design and implementation. While these explanations start from the basics, a knowledge of common rasterization techniques is assumed. II. Rendering Technology ======================== 1. Requirements --------------- We assume that all scaling, rotating, hinting, etc., has been already done. The glyph is thus described by a list of points in the device space. - All point coordinates are in the 26.6 fixed float format. The used orientation is: ^ y | reference orientation | *----> x 0 `26.6' means that 26 bits are used for the integer part of a value and 6 bits are used for the fractional part. Consequently, the `distance' between two neighbouring pixels is 64 `units' (1 unit = 1/64 of a pixel). Note that, for the rasterizer, pixel centers are located at integer coordinates. The TrueType bytecode interpreter, however, assumes that the lower left edge of a pixel (which is taken to be a square with a length of 1 unit) has integer coordinates. ^ y ^ y | | | (1,1) | (0.5,0.5) +-----------+ +-----+-----+ | | | | | | | | | | | | | o-----+-----> x | | | (0,0) | | | | | o-----------+-----> x +-----------+ (0,0) (-0.5,-0.5) TrueType bytecode interpreter FreeType rasterizer A pixel line in the target bitmap is called a `scanline'. - A glyph is usually made of several contours, also called `outlines'. A contour is simply a closed curve that delimits an outer or inner region of the glyph. It is described by a series of successive points of the points table. Each point of the glyph has an associated flag that indicates whether it is `on' or `off' the curve. Two successive `on' points indicate a line segment joining the two points. One `off' point amidst two `on' points indicates a second-degree (conic) Bézier parametric arc, defined by these three points (the `off' point being the control point, and the `on' ones the start and end points). Similarly, a third-degree (cubic) Bézier curve is described by four points (two `off' control points between two `on' points). Finally, for second-order curves only, two successive `off' points forces the rasterizer to create, during rendering, an `on' point amidst them, at their exact middle. This greatly facilitates the definition of successive Bézier arcs. The parametric form of a second-order Bézier curve is: P(t) = (1-t)^2*P1 + 2*t*(1-t)*P2 + t^2*P3 (P1 and P3 are the end points, P2 the control point.) The parametric form of a third-order Bézier curve is: P(t) = (1-t)^3*P1 + 3*t*(1-t)^2*P2 + 3*t^2*(1-t)*P3 + t^3*P4 (P1 and P4 are the end points, P2 and P3 the control points.) For both formulae, t is a real number in the range [0..1]. Note that the rasterizer does not use these formulae directly. They exhibit, however, one very useful property of Bézier arcs: Each point of the curve is a weighted average of the control points. As all weights are positive and always sum up to 1, whatever the value of t, each arc point lies within the triangle (polygon) defined by the arc's three (four) control points. In the following, only second-order curves are discussed since rasterization of third-order curves is completely identical. Here some samples for second-order curves. * # on curve * off curve __---__ #-__ _-- -_ --__ _- - --__ # \ --__ # -# Two `on' points Two `on' points and one `off' point between them * # __ Two `on' points with two `off' \ - - points between them. The point \ / \ marked `0' is the middle of the - 0 \ `off' points, and is a `virtual -_ _- # on' point where the curve passes. -- It does not appear in the point * list. 2. Profiles and Spans --------------------- The following is a basic explanation of the _kind_ of computations made by the rasterizer to build a bitmap from a vector representation. Note that the actual implementation is slightly different, due to performance tuning and other factors. However, the following ideas remain in the same category, and are more convenient to understand. a. Sweeping the Shape The best way to fill a shape is to decompose it into a number of simple horizontal segments, then turn them on in the target bitmap. These segments are called `spans'. __---__ _-- -_ _- - - \ / \ / \ | \ __---__ Example: filling a shape _----------_ with spans. _-------------- ----------------\ /-----------------\ This is typically done from the top / \ to the bottom of the shape, in a | | \ movement called a `sweep'. V __---__ _----------_ _-------------- ----------------\ /-----------------\ /-------------------\ |---------------------\ In order to draw a span, the rasterizer must compute its coordinates, which are simply the x coordinates of the shape's contours, taken on the y scanlines. /---/ |---| Note that there are usually /---/ |---| several spans per scanline. | /---/ |---| | /---/_______|---| When rendering this shape to the V /----------------| current scanline y, we must /-----------------| compute the x values of the a /----| |---| points a, b, c, and d. - - - * * - - - - * * - - y - / / b c| |d /---/ |---| /---/ |---| And then turn on the spans a-b /---/ |---| and c-d. /---/_______|---| /----------------| /-----------------| a /----| |---| - - - ####### - - - - ##### - - y - / / b c| |d b. Decomposing Outlines into Profiles For each scanline during the sweep, we need the following information: o The number of spans on the current scanline, given by the number of shape points intersecting the scanline (these are the points a, b, c, and d in the above example). o The x coordinates of these points. x coordinates are computed before the sweep, in a phase called `decomposition' which converts the glyph into *profiles*. Put it simply, a `profile' is a contour's portion that can only be either ascending or descending, i.e., it is monotonic in the vertical direction (we also say y-monotonic). There is no such thing as a horizontal profile, as we shall see. Here are a few examples: this square 1 2 ---->---- is made of two | | | | | | profiles | | ^ v ^ + v | | | | | | | | ----<---- up down this triangle P2 1 2 |\ is made of two | \ ^ | \ \ | \ | | \ \ profiles | \ | | | \ v ^ | \ | | \ | | + \ v | \ | | \ P1 ---___ \ ---___ \ ---_\ ---_ \ <--__ P3 up down A more general contour can be made of more than two profiles: __ ^ / | / ___ / | / | / | / | / | | | / / => | v / / | | | | | | ^ | ^ | |___| | | ^ + | + | + v | | | v | | | | | up | |___________| | down | <-- up down Successive profiles are always joined by horizontal segments that are not part of the profiles themselves. For the rasterizer, a profile is simply an *array* that associates one horizontal *pixel* coordinate to each bitmap *scanline* crossed by the contour's section containing the profile. Note that profiles are *oriented* up or down along the glyph's original flow orientation. In other graphics libraries, profiles are also called `edges' or `edgelists'. c. The Render Pool FreeType has been designed to be able to run well on _very_ light systems, including embedded systems with very few memory. A render pool will be allocated once; the rasterizer uses this pool for all its needs by managing this memory directly in it. The algorithms that are used for profile computation make it possible to use the pool as a simple growing heap. This means that this memory management is actually quite easy and faster than any kind of malloc()/free() combination. Moreover, we'll see later that the rasterizer is able, when dealing with profiles too large and numerous to lie all at once in the render pool, to immediately decompose recursively the rendering process into independent sub-tasks, each taking less memory to be performed (see `sub-banding' below). The render pool doesn't need to be large. A 4KByte pool is enough for nearly all renditions, though nearly 100% slower than a more comfortable 16KByte or 32KByte pool (that was tested with complex glyphs at sizes over 500 pixels). d. Computing Profiles Extents Remember that a profile is an array, associating a _scanline_ to the x pixel coordinate of its intersection with a contour. Though it's not exactly how the FreeType rasterizer works, it is convenient to think that we need a profile's height before allocating it in the pool and computing its coordinates. The profile's height is the number of scanlines crossed by the y-monotonic section of a contour. We thus need to compute these sections from the vectorial description. In order to do that, we are obliged to compute all (local and global) y extrema of the glyph (minima and maxima). P2 For instance, this triangle has only two y-extrema, which are simply |\ | \ P2.y as a vertical maximum | \ P3.y as a vertical minimum | \ | \ P1.y is not a vertical extremum (though | \ it is a horizontal minimum, which we P1 ---___ \ don't need). ---_\ P3 Note that the extrema are expressed in pixel units, not in scanlines. The triangle's height is certainly (P3.y-P2.y+1) pixel units, but its profiles' heights are computed in scanlines. The exact conversion is simple: - min scanline = FLOOR ( min y ) - max scanline = CEILING( max y ) A problem arises with Bézier Arcs. While a segment is always necessarily y-monotonic (i.e., flat, ascending, or descending), which makes extrema computations easy, the ascent of an arc can vary between its control points. P2 * # on curve * off curve __-x--_ _-- -_ P1 _- - A non y-monotonic Bézier arc. # \ - The arc goes from P1 to P3. \ \ P3 # We first need to be able to easily detect non-monotonic arcs, according to their control points. I will state here, without proof, that the monotony condition can be expressed as: P1.y <= P2.y <= P3.y for an ever-ascending arc P1.y >= P2.y >= P3.y for an ever-descending arc with the special case of P1.y = P2.y = P3.y where the arc is said to be `flat'. As you can see, these conditions can be very easily tested. They are, however, extremely important, as any arc that does not satisfy them necessarily contains an extremum. Note also that a monotonic arc can contain an extremum too, which is then one of its `on' points: P1 P2 #---__ * P1P2P3 is ever-descending, but P1 -_ is an y-extremum. - ---_ \ -> \ \ P3 # Let's go back to our previous example: P2 * # on curve * off curve __-x--_ _-- -_ P1 _- - A non-y-monotonic Bézier arc. # \ - Here we have \ P2.y >= P1.y && \ P3 P2.y >= P3.y (!) # We need to compute the vertical maximum of this arc to be able to compute a profile's height (the point marked by an `x'). The arc's equation indicates that a direct computation is possible, but we rely on a different technique, which use will become apparent soon. Bézier arcs have the special property of being very easily decomposed into two sub-arcs, which are themselves Bézier arcs. Moreover, it is easy to prove that there is at most one vertical extremum on each Bézier arc (for second-degree curves; similar conditions can be found for third-order arcs). For instance, the following arc P1P2P3 can be decomposed into two sub-arcs Q1Q2Q3 and R1R2R3: P2 * # on curve * off curve original Bézier arc P1P2P3. __---__ _-- --_ _- -_ - - / \ / \ # # P1 P3 P2 * Q3 Decomposed into two subarcs Q2 R2 Q1Q2Q3 and R1R2R3 * __-#-__ * _-- --_ _- R1 -_ Q1 = P1 R3 = P3 - - Q2 = (P1+P2)/2 R2 = (P2+P3)/2 / \ / \ Q3 = R1 = (Q2+R2)/2 # # Q1 R3 Note that Q2, R2, and Q3=R1 are on a single line which is tangent to the curve. We have then decomposed a non-y-monotonic Bézier curve into two smaller sub-arcs. Note that in the above drawing, both sub-arcs are monotonic, and that the extremum is then Q3=R1. However, in a more general case, only one sub-arc is guaranteed to be monotonic. Getting back to our former example: Q2 * __-x--_ R1 _-- #_ Q1 _- Q3 - R2 # \ * - \ \ R3 # Here, we see that, though Q1Q2Q3 is still non-monotonic, R1R2R3 is ever descending: We thus know that it doesn't contain the extremum. We can then re-subdivide Q1Q2Q3 into two sub-arcs and go on recursively, stopping when we encounter two monotonic subarcs, or when the subarcs become simply too small. We will finally find the vertical extremum. Note that the iterative process of finding an extremum is called `flattening'. e. Computing Profiles Coordinates Once we have the height of each profile, we are able to allocate it in the render pool. The next task is to compute coordinates for each scanline. In the case of segments, the computation is straightforward, using the Euclidean algorithm (also known as Bresenham). However, for Bézier arcs, the job is a little more complicated. We assume that all Béziers that are part of a profile are the result of flattening the curve, which means that they are all y-monotonic (ascending or descending, and never flat). We now have to compute the intersections of arcs with the profile's scanlines. One way is to use a similar scheme to flattening called `stepping'. Consider this arc, going from P1 to --------------------- P3. Suppose that we need to compute its intersections with the drawn scanlines. As already --------------------- mentioned this can be done directly, but the involved * P2 _---# P3 algorithm is far too slow. ------------- _-- -- _- _/ Instead, it is still possible to ---------/----------- use the decomposition property in / the same recursive way, i.e., | subdivide the arc into subarcs ------|-------------- until these get too small to cross | more than one scanline! | -----|--------------- This is very easily done using a | rasterizer-managed stack of | subarcs. # P1 f. Sweeping and Sorting the Spans Once all our profiles have been computed, we begin the sweep to build (and fill) the spans. As both the TrueType and Type 1 specifications use the winding fill rule (but with opposite directions), we place, on each scanline, the present profiles in two separate lists. One list, called the `left' one, only contains ascending profiles, while the other `right' list contains the descending profiles. As each glyph is made of closed curves, a simple geometric property ensures that the two lists contain the same number of elements. Creating spans is thus straightforward: 1. We sort each list in increasing horizontal order. 2. We pair each value of the left list with its corresponding value in the right list. / / | | For example, we have here / / | | four profiles. Two of >/ / | | | them are ascending (1 & 1// / ^ | | | 2 3), while the two others // // 3| | | v are descending (2 & 4). / //4 | | | On the given scanline, a / /< | | the left list is (1,3), - - - *-----* - - - - *---* - - y - and the right one is / / b c| |d (4,2) (sorted). There are then two spans, joining 1 to 4 (i.e. a-b) and 3 to 2 (i.e. c-d)! Sorting doesn't necessarily take much time, as in 99 cases out of 100, the lists' order is kept from one scanline to the next. We can thus implement it with two simple singly-linked lists, sorted by a classic bubble-sort, which takes a minimum amount of time when the lists are already sorted. A previous version of the rasterizer used more elaborate structures, like arrays to perform `faster' sorting. It turned out that this old scheme is not faster than the one described above. Once the spans have been `created', we can simply draw them in the target bitmap. ------------------------------------------------------------------------ Copyright (C) 2003-2023 by David Turner, Robert Wilhelm, and Werner Lemberg. This file is part of the FreeType project, and may only be used, modified, and distributed under the terms of the FreeType project license, LICENSE.TXT. By continuing to use, modify, or distribute this file you indicate that you have read the license and understand and accept it fully. --- end of raster.txt --- Local Variables: coding: utf-8 End: --- ### Vs Libpdcurses Doc Sdl (`vs/libpdcurses/doc/sdl.md`) SDL Considerations ================== There are no special requirements to use PDCurses for SDL -- all PDCurses-compatible code should work fine. (In fact, you can even build against the Windows console pdcurses.dll, and then swap in the SDL pdcurses.dll.) Nothing extra is needed beyond the base SDL library. However, there are some optional special features, described here. The SDL ports operate in one of two ways, depending on whether or not they were built with WIDE=Y: 8-bit mode ---------- The font is a simple BMP, 32 characters wide by 8 characters tall, preferably with a palette. (BMPs without palettes still work, but in that case, no attributes will be available, nor will the cursor work.) The first entry in the palette (usually black) is treated as the background color; the last entry (usually white) is treated as the foreground. These are changed or made transparent as appropriate; any other colors in the palette are passed through unchanged. So -- although a one-bit depth is sufficient for a normal font -- you could redraw some characters as multi-colored tiles. The font must be monospaced. The size of each character is derived by dividing the width of the BMP by 32 and the height by 8. There is no constraint on the dimensions. As provided in the default font and expected by acs_map[], the font is in Code Page 437 form. But you can of course use any layout if you're not relying on correct values for the ACS_* macros. The font can be set via the environment variable PDC_FONT. If it's not set, PDCurses looks for a file named "pdcfont.bmp" in the current directory at the time of initscr(). If neither is found, it uses the built-in default font encoded in deffont.h. 16-bit mode ----------- Instead of a BMP, PDC_FONT points to a TrueType font. Only true monospaced fonts work well. The font can be set at compile time via PDC_FONT_PATH, and/or at runtime via pdc_ttffont. The environment variable PDC_FONT_SIZE is also available to control the font size (also as a compile-time define, and at runtime as pdc_font_size.) The character mapping for chtypes is 16-bit Unicode (the Basic Multilingual Plane). The default font (if not redefined) is based on the OS: - Windows: C:/Windows/Fonts/lucon.ttf - Mac: /Library/Fonts/Courier New.ttf - Other: /usr/share/fonts/truetype/freefont/FreeMono.ttf Backgrounds ----------- PDCurses for SDL supports an optional background image BMP. This is used whenever start_color() has not been called (see the ptest demo for an example), or when use_default_colors() has been called after start_color(), and the background color of a pair has been set to -1 (see ozdemo, worm, and rain for examples). The usage parallels that of ncurses in an appropriate terminal (e.g., Gnome Terminal). The image is tiled to cover the PDCurses window, and can be any size or depth. As with the font, you can point to a location for the background via the environment variable PDC_BACKGROUND; "pdcback.bmp" is the fallback. (There is no default background.) Icons ----- The icon (used with SDL_WM_SetIcon() -- not used for the executable file) can be set via the environment variable PDC_ICON, and falls back to "pdcicon.bmp", and then to the built-in icon from deficon.h. The built-in icon is the PDCurses logo, as seen in ../x11/little_icon.xbm. The SDL docs say that the icon must be 32x32, at least for use with MS Windows. If pdc_screen is preinitialized (see below), PDCurses does not attempt to set the icon. Screen size ----------- The default screen size is 80x25 characters (whatever size they may be), but you can override this via the environment variables PDC_COLS and/or PDC_LINES. (Some other ports use COLS and LINES; this is not done here because those values are, or should be, those of the controlling terminal, and PDCurses for SDL is independent of the terminal.) If pdc_screen is preinitialized (see below), these are ignored. Integration with SDL -------------------- If you want to go further, you can mix PDCurses and SDL functions. (Of course this is extremely non-portable!) To aid you, there are several external variables and functions specific to the SDL ports; you could include pdcsdl.h, or just add the declarations you need in your code: PDCEX SDL_Surface *pdc_screen, *pdc_font, *pdc_icon, *pdc_back; PDCEX int pdc_sheight, pdc_swidth, pdc_yoffset, pdc_xoffset; PDCEX void PDC_update_rects(void); PDCEX void PDC_retile(void); The SDL2 port adds: PDCEX SDL_Window *pdc_window; pdc_screen is the main surface, unless it's preset before initscr(). In SDL1, pdc_screen is created by SDL_SetVideoMode(); in SDL2, pdc_window is the main window, created by SDL_CreateWindow(), and pdc_screen is set by SDL_GetWindowSurface(pdc_window). (See sdltest.c for examples.) You can perform normal SDL operations on this surface, but PDCurses won't respect them when it updates. (For that, see PDC_retile().) As an alternative, you can preinitialize this surface before calling initscr(). In that case, you can use pdc_sheight, pdc_swidth, pdc_yoffset and/or pdc_xoffset (q.v.) to confine PDCurses to only a specific area of the surface, reserving the rest for other SDL operations. If you preinitialize pdc_screen, you'll have to close it yourself; PDCurses will ignore resize events, and won't try to set the icon. Also note that if you preinitialize pdc_screen, it need not be the display surface. pdc_font (in 8-bit mode), pdc_icon, and pdc_back are the SDL_surfaces for the font, icon, and background, respectively. You can set any or all of them before initscr(), and thus override any of the other ways to set them. But note that pdc_icon will be ignored if pdc_screen is preset. pdc_sheight and pdc_swidth are the dimensions of the area of pdc_screen to be used by PDCurses. You can preset them before initscr(); if either is not set, it defaults to the full screen size minus the x or y offset, as appropriate. pdc_xoffset and pdc_yoffset are the x and y offset for the area of pdc_screen to be used by PDCurses. See the sdltest demo for an example. PDC_retile() makes a copy of pdc_screen, then tiles it with the background image, if any. The resulting surface is used as the background for transparent character cells. PDC_retile() is called from initscr() and resize_term(). However, you can also use it at other times, to take advantage of the way it copies pdc_screen: Draw some SDL stuff; call PDC_retile(); do some curses stuff -- it will use whatever was on pdc_screen as the background. Then you can erase the curses screen, do some more SDL stuff, and call PDC_retile() again to make a new background. (If you don't erase the curses screen, it will be incorporated into the background when you call PDC_retile().) But this only works if no background image is set. PDC_update_rects() is how the screen actually gets updated. For performance reasons, when drawing, PDCurses for SDL maintains a table of rectangles that need updating, and only updates (by calling this function) during getch(), napms(), or when the table gets full. Normally, this is sufficient; but if you're pausing in some way other than by using napms(), and you're not doing keyboard checks, you may get an incomplete update. If that happens, you can call PDC_update_rects() manually. Interaction with stdio ---------------------- As with X11, it's a bad idea to mix curses and stdio calls. (In fact, that's true for PDCurses on any platform; but especially these two, which don't run under terminals.) Depending on how SDL is built, stdout and stderr may be redirected to files. --- ### Vs Libpdcurses Doc X11 (`vs/libpdcurses/doc/x11.md`) X11 Considerations ================== PDCurses for X11 uses the System V IPC shared memory facility, along with sockets, to share data between the curses program and the child process created to manage the X stuff. When compiling your application, you need to include the \ or \ that comes with PDCurses. You also need to link your code with libXCurses. You may need to link with the following libraries under X11R5: Xaw Xmu Xt X11 or, under X11R6: Xaw Xmu Xt X11 SM ICE Xext You can run "xcurses-config --libs" to show the link parameters for your system. If using dynamic linking, on some systems, "-lXCurses" suffices. By calling Xinitscr() rather than initscr(), you can pass your program name and resource overrides to PDCurses. The program name is used as the title of the X window, and for defining X resources specific to your program. Interaction with stdio ---------------------- Be aware that curses programs that expect to have a normal tty underneath them will be very disappointed! Output directed to stdout will go to the xterm that invoked the PDCurses application, or to the console if not invoked directly from an xterm. Similarly, stdin will expect its input from the same place as stdout. X Resources ----------- PDCurses for X11 recognizes the following resources: ### lines Specifies the number of lines the "screen" will have. Directly equates to LINES. There is no theoretical maximum. The minimum value must be 2. Default: 24 ### cols Specifies the number of columns the "screen" will have. Directly equates to COLS. There is no theoretical maximum. The minimum value must be 2. Default: 80 ### normalFont The name of a fixed width font. Defaults: - narrow: -misc-fixed-medium-r-normal--13-120-75-75-c-70-iso8859-1 - wide: -misc-fixed-medium-r-normal--20-200-75-75-c-100-iso10646-1 ### italicFont The name of a fixed width font to be used for characters with A_ITALIC attributes. Must have the same cell size as normalFont. Defaults: - narrow: -misc-fixed-medium-o-normal--13-120-75-75-c-70-iso8859-1 - wide: -misc-fixed-medium-o-normal--20-200-75-75-c-100-iso10646-1 ### boldFont The name of a fixed width font to be used for characters with A_BOLD attributes. Must have the same cell size as normalFont. Defaults: - narrow: -misc-fixed-bold-r-normal--13-120-75-75-c-70-iso8859-1 - wide: -misc-fixed-bold-r-normal--20-200-75-75-c-100-iso10646-1 ### pointer The name of a valid pointer cursor. Default: xterm ### pointerForeColor The foreground color of the pointer. Default: black ### pointerBackColor The background color of the pointer. Default: white ### cursorColor ### textCursor The alignment of the text cursor; horizontal or vertical. Default: horizontal ### colorBlack The color of the COLOR_BLACK attribute. Default: Black ### colorRed The color of the COLOR_RED attribute. Default: red3 ### colorGreen The color of the COLOR_GREEN attribute. Default: green3 ### colorYellow The color of the COLOR_YELLOW attribute. Default: yellow3 ### colorBlue The color of the COLOR_BLUE attribute. Default: blue3 ### colorMagenta The color of the COLOR_MAGENTA attribute. Default: magenta3 ### colorCyan The color of the COLOR_CYAN attribute. Default: cyan3 ### colorWhite The color of the COLOR_WHITE attribute. Default: Grey ### colorBoldBlack COLOR_BLACK combined with A_BOLD. Default: grey40 ### colorBoldRed COLOR_RED combined with A_BOLD. Default: red1 ### colorBoldGreen COLOR_GREEN combined with A_BOLD. Default: green1 ### colorBoldYellow COLOR_YELLOW combined with A_BOLD. Default: yellow1 ### colorBoldBlue COLOR_BLUE combined with A_BOLD. Default: blue1 ### colorBoldMagenta COLOR_MAGENTA combined with A_BOLD. Default: magenta1 ### colorBoldCyan COLOR_CYAN combined with A_BOLD. Default: cyan1 ### colorBoldWhite COLOR_WHITE combined with A_BOLD. Default: White ### bitmap The name of a valid bitmap file of depth 1 (black and white) used for the application's icon. The file is an X bitmap. Default: a 32x32 or 64x64 pixmap depending on the window manager ### pixmap The name of a valid pixmap file of any depth supported by the window manager (color) for the application's icon, The file is an X11 pixmap. This resource is only available if the libXpm package has been installed (most systems have this by default). This resource overrides the "bitmap" resource. Default: none, uses default bitmap above ### translations Translations enable the user to customize the action that occurs when a key, combination of keys, or a button is pressed. The translations are similar to those used by xterm. Defaults: : XCursesKeyPress() : XCursesKeyPress() : XCursesButton() : XCursesButton() : XCursesButton() The most useful action for KeyPress translations is string(). The argument to the string() action can be either a string or a hex representation of a character; e.g., string(0x1b) will send the ASCII escape character to the application; string("[11~") will send [ 1 1 ~ , as separate keystrokes. ### shmmin On most systems, there are two Unix kernel parameters that determine the allowable size of a shared memory segment. These parameters are usually something like SHMMIN and SHMMAX. To use shared memory, a program must allocate a segment of shared memory that is between these two values. Usually these values are like 1 for SHMMIN and some large number for SHMMAX. Sometimes the Unix kernel is configured to have a value of SHMMIN that is bigger than the size of one of the shared memory segments that libXCurses uses. On these systems an error message like: Cannot allocate shared memory for SCREEN: Invalid argument will result. To overcome this problem, this resource should be set to the kernel value for SHMMIN. This ensures that a shared memory segment will always be bigger than the kernel value for SHMMIN (and hopefully less than SHMMAX!) Default: 0 ### borderColor The color of the border around the screen. Default: black ### borderWidth The width in pixels of the border around the screen. Default: 0 ### clickPeriod The period (in milliseconds) between a button press and a button release that determines if a click of a button has occurred. Default: 100 ### doubleClickPeriod The period (in milliseconds) between two button press events that determines if a double click of a button has occurred. Default: 200 ### composeKey The name of the X key that defines the "compose key", which is used to enter characters in the Latin-1 character set above 0xA0. (See "Compose Keys for Latin-1" below.) This is used only when PDCurses is built without XIM support. While in compose mode, the text cursor will appear as a hollow rectangle. Default: Multi_key Using Resources --------------- All applications have a top-level class name of "XCurses". If Xinitscr() is used, it sets an application's top-level widget name. (Otherwise the name defaults to "PDCurses".) Examples for app-defaults or .Xdefaults: ! ! resources for XCurses class of programs ! XCurses*lines: 30 XCurses*cols: 80 XCurses*normalFont: 9x13 XCurses*bitmap: /tmp/xcurses.xbm XCurses*pointer: top_left_arrow ! ! resources for testcurs - XCurses ! testcurs.colorRed: orange testcurs.colorBlack: midnightblue testcurs.lines: 25 *testcurs.Translations: #override \n \ F12: string(0x1b) string("[11~") \n ! ! resources for THE - XCurses ! ! resources with the * wildcard can be overridden by a parameter passed ! to initscr() ! the*normalFont: 9x15 the*lines: 40 the*cols: 86 the*pointer: xterm the*pointerForeColor: white the*pointerBackColor: black ! ! resources with the . format can not be overridden by a parameter passed ! to Xinitscr() ! the.bitmap: /home/mark/the/the64.xbm the.pixmap: /home/mark/the/the64.xpm Resources may also be passed as parameters to the Xinitscr() function. Parameters are strings in the form of switches; e.g., to set the color "red" to "indianred", and the number of lines to 30, the string passed to Xinitscr would be: "-colorRed indianred -lines 30" Compose Keys for Latin-1 ------------------------ When built without XIM support, PDCurses for X11 provides its own, limited compose key system for Latin-1 characters. The available combinations are listed here. For a given character, any of the combinations shown in the last column may be used. To generate a character, press the "compose" key followed by one of the pairs of keystrokes. Where no key is evident, the spacebar is used. Thus, to generate the NO-BREAK SPACE, press the "compose" key followed by two hits of the spacebar. With a typical modern X server, you can get many more compose key combinations by using XIM instead. Configure PDCurses with --enable-xim to use XIM support. This document is encoded in UTF-8. Hex| Dec |Chr| Description ISO 10646-1:1993(E) | Compose key combinations ---+-----+---+---------------------------------+------------------------- A0 | 160 | | NO-BREAK SPACE | A1 | 161 | ¡ | INVERTED EXCLAMATION MARK | ! !! A2 | 162 | ¢ | CENT SIGN | c| |c c/ c$ C$ C| A3 | 163 | £ | POUND SIGN | L- L$ L= l- l$ l= |- A4 | 164 | ¤ | CURRENCY SIGN | xo ox XO g$ A5 | 165 | ¥ | YEN SIGN | =y y= =Y Y= Y- y$ y- A6 | 166 | ¦ | BROKEN BAR | | || vb VB |^ A7 | 167 | § | SECTION SIGN | SO SS s! S! so A8 | 168 | ¨ | DIAERESIS | " "" A9 | 169 | © | COPYRIGHT SIGN | CO co OC AA | 170 | ª | FEMININE ORDINAL INDICATOR | sa SA a_ A_ AB | 171 | « | LEFT DOUBLE ANGLE QUOTES | << AC | 172 | ¬ | NOT SIGN | -, no NO AD | 173 | ­ | SOFT HYPHEN | - -- AE | 174 | ® | REGISTERED SIGN | RO ro OR AF | 175 | ¯ | MACRON | -^ _^ __ B0 | 176 | ° | DEGREE SIGN | o 0^ 0* de DE ^0 B1 | 177 | ± | PLUS-MINUS SIGN | -+ +- B2 | 178 | ² | SUPERSCRIPT TWO | 2 2^ s2 ^2 B3 | 179 | ³ | SUPERSCRIPT THREE | 3 3^ s3 ^3 B4 | 180 | ´ | ACUTE ACCENT | ' '' B5 | 181 | µ | MICRO SIGN | u /u /U *m *M B6 | 182 | ¶ | PILCROW SIGN | p! P! pg PG B7 | 183 | · | MIDDLE DOT | . .^ .. B8 | 184 | ¸ | CEDILLA | , ,, B9 | 185 | ¹ | SUPERSCRIPT ONE | 1 1^ s1 ^1 BA | 186 | º | MASCULINE ORDINAL INDICATOR | o_ s0 S0 BB | 187 | » | RIGHT DOUBLE ANGLE QUOTES | >> BC | 188 | ¼ | VULGAR FRACTION ONE QUARTER | 14 BD | 189 | ½ | VULGAR FRACTION ONE HALF | 12 BE | 190 | ¾ | VULGAR FRACTION THREE QUARTERS | 34 BF | 191 | ¿ | INVERTED QUESTION MARK | ? ?? C0 | 192 | À | CAPITAL A WITH GRAVE ACCENT | `A A` C1 | 193 | Á | CAPITAL A WITH ACUTE ACCENT | 'A A' C2 | 194 | Â | CAPITAL A WITH CIRCUMFLEX ACCENT| ^A A^ A> C3 | 195 | Ã | CAPITAL A WITH TILDE | ~A A~ A- C4 | 196 | Ä | CAPITAL A WITH DIAERESIS | "A A" C5 | 197 | Å | CAPITAL A WITH RING ABOVE | oA Ao A* OA *A C6 | 198 | Æ | CAPITAL LIGATURE AE | AE C7 | 199 | Ç | CAPITAL C WITH CEDILLA | ,C C, C8 | 200 | È | CAPITAL E WITH GRAVE ACCENT | `E E` C9 | 201 | É | CAPITAL E WITH ACUTE ACCENT | 'E E' CA | 202 | Ê | CAPITAL E WITH CIRCUMFLEX ACCENT| ^E E^ E> CB | 203 | Ë | CAPITAL E WITH DIAERESIS | "E E" CC | 204 | Ì | CAPITAL I WITH GRAVE ACCENT | `I I` CD | 205 | Í | CAPITAL I WITH ACUTE ACCENT | 'I I' CE | 206 | Î | CAPITAL I WITH CIRCUMFLEX ACCENT| ^I I^ I> CF | 207 | Ï | CAPITAL I WITH DIAERESIS | "I I" D0 | 208 | Ð | CAPITAL ETH | D- D1 | 209 | Ñ | CAPITAL N WITH TILDE | ~N N~ N- D2 | 210 | Ò | CAPITAL O WITH GRAVE ACCENT | `O O` D3 | 211 | Ó | CAPITAL O WITH ACUTE ACCENT | 'O O' D4 | 212 | Ô | CAPITAL O WITH CIRCUMFLEX ACCENT| ^O O^ O> D5 | 213 | Õ | CAPITAL O WITH TILDE | ~O O~ O- D6 | 214 | Ö | CAPITAL O WITH DIAERESIS | "O O" D7 | 215 | × | MULTIPLICATION SIGN | x xx XX mu MU D8 | 216 | Ø | CAPITAL O WITH STROKE | /O O/ D9 | 217 | Ù | CAPITAL U WITH GRAVE ACCENT | `U U` DA | 218 | Ú | CAPITAL U WITH ACUTE ACCENT | 'U U' DB | 219 | Û | CAPITAL U WITH CIRCUMFLEX ACCENT| ^U U^ U> DC | 220 | Ü | CAPITAL U WITH DIAERESIS | "U U" DD | 221 | Ý | CAPITAL Y WITH ACUTE ACCENT | 'Y Y' DE | 222 | Þ | CAPITAL THORN | P TH |P DF | 223 | ß | SMALL SHARP S | ss E0 | 224 | à | SMALL A WITH GRAVE ACCENT | `a a` E1 | 225 | á | SMALL A WITH ACUTE ACCENT | 'a a' E2 | 226 | â | SMALL A WITH CIRCUMFLEX ACCENT | ^a a^ a> E3 | 227 | ã | SMALL A WITH TILDE | ~a a~ a- E4 | 228 | ä | SMALL A WITH DIAERESIS | "a a" E5 | 229 | å | SMALL A WITH RING ABOVE | oa ao Oa a* *a E6 | 230 | æ | SMALL LIGATURE AE | ae E7 | 231 | ç | SMALL C WITH CEDILLA | ,c c, E8 | 232 | è | SMALL E WITH GRAVE ACCENT | `e e` E9 | 233 | é | SMALL E WITH ACUTE ACCENT | 'e e' EA | 234 | ê | SMALL E WITH CIRCUMFLEX ACCENT | ^e e^ e> EB | 235 | ë | SMALL E WITH DIAERESIS | "e e" EC | 236 | ì | SMALL I WITH GRAVE ACCENT | `i i` ED | 237 | í | SMALL I WITH ACUTE ACCENT | 'i i' EE | 238 | î | SMALL I WITH CIRCUMFLEX ACCENT | ^i i^ i> EF | 239 | ï | SMALL I WITH DIAERESIS | "i i" F0 | 240 | ð | SMALL ETH | d- F1 | 241 | ñ | SMALL N WITH TILDE | ~n n~ n- F2 | 242 | ò | SMALL O WITH GRAVE ACCENT | `o o` F3 | 243 | ó | SMALL O WITH ACUTE ACCENT | 'o o' F4 | 244 | ô | SMALL O WITH CIRCUMFLEX ACCENT | ^o o^ o> F5 | 245 | õ | SMALL O WITH TILDE | ~o o~ o- F6 | 246 | ö | SMALL O WITH DIAERESIS | "o o" F7 | 247 | ÷ | DIVISION SIGN | -: :- F8 | 248 | ø | SMALL O WITH OBLIQUE BAR | /o o/ F9 | 249 | ù | SMALL U WITH GRAVE ACCENT | `u u` FA | 250 | ú | SMALL U WITH ACUTE ACCENT | 'u u' FB | 251 | û | SMALL U WITH CIRCUMFLEX ACCENT | ^u u^ u> FC | 252 | ü | SMALL U WITH DIAERESIS | "u u" FD | 253 | ý | SMALL Y WITH ACUTE ACCENT | 'y y' FE | 254 | þ | SMALL THORN | p th |p FF | 255 | ÿ | SMALL Y WITH DIAERESIS | "y y" Deprecated ---------- XCursesProgramName is no longer used. To set the program name, you must use Xinitscr(), or PDC_set_title() to set just the window title. The XCursesExit() function is now called automatically via atexit(). (Multiple calls to it are OK, so you don't need to remove it if you've already added it for previous versions of PDCurses.) XCURSES is no longer defined automatically, but need not be defined, unless you want the X11-specific prototypes. (Normal curses programs won't need it.) --- ### Vs Sdl2 Docs CONTRIBUTING (`vs/sdl2/docs/CONTRIBUTING.md`) # Contributing to SDL We appreciate your interest in contributing to SDL, this document will describe how to report bugs, contribute code or ideas or edit documentation. **Table Of Contents** - [Filing a GitHub issue](#filing-a-github-issue) - [Reporting a bug](#reporting-a-bug) - [Suggesting enhancements](#suggesting-enhancements) - [Contributing code](#contributing-code) - [Forking the project](#forking-the-project) - [Following the style guide](#following-the-style-guide) - [Running the tests](#running-the-tests) - [Opening a pull request](#opening-a-pull-request) - [Contributing to the documentation](#contributing-to-the-documentation) - [Editing a function documentation](#editing-a-function-documentation) - [Editing the wiki](#editing-the-wiki) ## Filing a GitHub issue ### Reporting a bug If you think you have found a bug and would like to report it, here are the steps you should take: - Before opening a new issue, ensure your bug has not already been reported on the [GitHub Issues page](https://github.com/libsdl-org/SDL/issues). - On the issue tracker, click on [New Issue](https://github.com/libsdl-org/SDL/issues/new). - Include details about your environment, such as your Operating System and SDL version. - If possible, provide a small example that reproduces your bug. ### Suggesting enhancements If you want to suggest changes for the project, here are the steps you should take: - Check if the suggestion has already been made on: - the [issue tracker](https://github.com/libsdl-org/SDL/issues); - the [discourse forum](https://discourse.libsdl.org/); - or if a [pull request](https://github.com/libsdl-org/SDL/pulls) already exists. - On the issue tracker, click on [New Issue](https://github.com/libsdl-org/SDL/issues/new). - Describe what change you would like to happen. ## Contributing code This section will cover how the process of forking the project, making a change and opening a pull request. ### Forking the project The first step consists in making a fork of the project, this is only necessary for the first contribution. Head over to https://github.com/libsdl-org/SDL and click on the `Fork` button in the top right corner of your screen, you may leave the fields unchanged and click `Create Fork`. You will be redirected to your fork of the repository, click the green `Code` button and copy the git clone link. If you had already forked the repository, you may update it from the web page using the `Fetch upstream` button. ### Following the style guide Code formatting is done using a custom `.clang-format` file, you can learn more about how to run it [here](https://clang.llvm.org/docs/ClangFormat.html). Some legacy code may not be formatted, as such avoid formatting the whole file at once and only format around your changes. For your commit message to be properly displayed on GitHub, it should contain: - A short description of the commit of 50 characters or less on the first line. - If necessary, add a blank line followed by a long description, each line should be 72 characters or less. For example: ``` Fix crash in SDL_FooBar. This addresses the issue #123456 by making sure Foo was successful before calling Bar. ``` ### Running the tests Tests allow you to verify if your changes did not break any behaviour, here are the steps to follow: - Before pushing, run the `testautomation` suite on your machine, there should be no more failing tests after your change than before. - After pushing to your fork, Continuous Integration (GitHub Actions) will ensure compilation and tests still pass on other systems. ### Opening a pull request - Head over to your fork's GitHub page. - Click on the `Contribute` button and `Open Pull Request`. - Fill out the pull request template. - If any changes are requested, you can add new commits to your fork and they will be automatically added to the pull request. ## Contributing to the documentation ### Editing a function documentation The wiki documentation for API functions is synchronised from the headers' doxygen comments. As such, all modifications to syntax; function parameters; return value; version; related functions should be done in the header directly. ### Editing the wiki Other changes to the wiki should done directly from https://wiki.libsdl.org/ --- ### Vs Sdl2 Docs README Android (`vs/sdl2/docs/README-android.md`) Android ================================================================================ Matt Styles wrote a tutorial on building SDL for Android with Visual Studio: http://trederia.blogspot.de/2017/03/building-sdl2-for-android-with-visual.html The rest of this README covers the Android gradle style build process. If you are using the older ant build process, it is no longer officially supported, but you can use the "android-project-ant" directory as a template. Requirements ================================================================================ Android SDK (version 34 or later) https://developer.android.com/sdk/index.html Android NDK r15c or later https://developer.android.com/tools/sdk/ndk/index.html Minimum API level supported by SDL: 19 (Android 4.4) How the port works ================================================================================ - Android applications are Java-based, optionally with parts written in C - As SDL apps are C-based, we use a small Java shim that uses JNI to talk to the SDL library - This means that your application C code must be placed inside an Android Java project, along with some C support code that communicates with Java - This eventually produces a standard Android .apk package The Android Java code implements an "Activity" and can be found in: android-project/app/src/main/java/org/libsdl/app/SDLActivity.java The Java code loads your game code, the SDL shared library, and dispatches to native functions implemented in the SDL library: src/core/android/SDL_android.c Building an app ================================================================================ For simple projects you can use the script located at build-scripts/androidbuild.sh There's two ways of using it: androidbuild.sh com.yourcompany.yourapp < sources.list androidbuild.sh com.yourcompany.yourapp source1.c source2.c ...sourceN.c sources.list should be a text file with a source file name in each line Filenames should be specified relative to the current directory, for example if you are in the build-scripts directory and want to create the testgles.c test, you'll run: ./androidbuild.sh org.libsdl.testgles ../test/testgles.c One limitation of this script is that all sources provided will be aggregated into a single directory, thus all your source files should have a unique name. Once the project is complete the script will tell you where the debug APK is located. If you want to create a signed release APK, you can use the project created by this utility to generate it. Finally, a word of caution: re running androidbuild.sh wipes any changes you may have done in the build directory for the app! For more complex projects, follow these instructions: 1. Get the source code for SDL and copy the 'android-project' directory located at SDL/android-project to a suitable location. Also make sure to rename it to your project name (In these examples: YOURPROJECT). (The 'android-project' directory can basically be seen as a sort of starting point for the android-port of your project. It contains the glue code between the Android Java 'frontend' and the SDL code 'backend'. It also contains some standard behaviour, like how events should be handled, which you will be able to change.) 2. Move or [symlink](https://en.wikipedia.org/wiki/Symbolic_link) the SDL directory into the "YOURPROJECT/app/jni" directory (This is needed as the source of SDL has to be compiled by the Android compiler) 3. Edit "YOURPROJECT/app/jni/src/Android.mk" to include your source files. (They should be separated by spaces after the "LOCAL_SRC_FILES := " declaration) 4a. If you want to use Android Studio, simply open your 'YOURPROJECT' directory and start building. 4b. If you want to build manually, run './gradlew installDebug' in the project directory. This compiles the .java, creates an .apk with the native code embedded, and installs it on any connected Android device If you already have a project that uses CMake, the instructions change somewhat: 1. Do points 1 and 2 from the instruction above. 2. Edit "YOURPROJECT/app/build.gradle" to comment out or remove sections containing ndk-build and uncomment the cmake sections. Add arguments to the CMake invocation as needed. 3. Edit "YOURPROJECT/app/jni/CMakeLists.txt" to include your project (it defaults to adding the "src" subdirectory). Note that you'll have SDL2, SDL2main and SDL2-static as targets in your project, so you should have "target_link_libraries(yourgame SDL2 SDL2main)" in your CMakeLists.txt file. Also be aware that you should use add_library() instead of add_executable() for the target containing your "main" function. If you wish to use Android Studio, you can skip the last step. 4. Run './gradlew installDebug' or './gradlew installRelease' in the project directory. It will build and install your .apk on any connected Android device Here's an explanation of the files in the Android project, so you can customize them: android-project/app build.gradle - build info including the application version and SDK src/main/AndroidManifest.xml - package manifest. Among others, it contains the class name of the main Activity and the package name of the application. jni/ - directory holding native code jni/Application.mk - Application JNI settings, including target platform and STL library jni/Android.mk - Android makefile that can call recursively the Android.mk files in all subdirectories jni/CMakeLists.txt - Top-level CMake project that adds SDL as a subproject jni/SDL/ - (symlink to) directory holding the SDL library files jni/SDL/Android.mk - Android makefile for creating the SDL shared library jni/src/ - directory holding your C/C++ source jni/src/Android.mk - Android makefile that you should customize to include your source code and any library references jni/src/CMakeLists.txt - CMake file that you may customize to include your source code and any library references src/main/assets/ - directory holding asset files for your application src/main/res/ - directory holding resources for your application src/main/res/mipmap-* - directories holding icons for different phone hardware src/main/res/values/strings.xml - strings used in your application, including the application name src/main/java/org/libsdl/app/SDLActivity.java - the Java class handling the initialization and binding to SDL. Be very careful changing this, as the SDL library relies on this implementation. You should instead subclass this for your application. Customizing your application name ================================================================================ To customize your application name, edit AndroidManifest.xml and replace "org.libsdl.app" with an identifier for your product package. Then create a Java class extending SDLActivity and place it in a directory under src matching your package, e.g. src/com/gamemaker/game/MyGame.java Here's an example of a minimal class file: --- MyGame.java -------------------------- package com.gamemaker.game; import org.libsdl.app.SDLActivity; /** * A sample wrapper class that just calls SDLActivity */ public class MyGame extends SDLActivity { } ------------------------------------------ Then replace "SDLActivity" in AndroidManifest.xml with the name of your class, .e.g. "MyGame" Customizing your application icon ================================================================================ Conceptually changing your icon is just replacing the "ic_launcher.png" files in the drawable directories under the res directory. There are several directories for different screen sizes. Loading assets ================================================================================ Any files you put in the "app/src/main/assets" directory of your project directory will get bundled into the application package and you can load them using the standard functions in SDL_rwops.h. There are also a few Android specific functions that allow you to get other useful paths for saving and loading data: * SDL_AndroidGetInternalStoragePath() * SDL_AndroidGetExternalStorageState() * SDL_AndroidGetExternalStoragePath() See SDL_system.h for more details on these functions. The asset packaging system will, by default, compress certain file extensions. SDL includes two asset file access mechanisms, the preferred one is the so called "File Descriptor" method, which is faster and doesn't involve the Dalvik GC, but given this method does not work on compressed assets, there is also the "Input Stream" method, which is automatically used as a fall back by SDL. You may want to keep this fact in mind when building your APK, specially when large files are involved. For more information on which extensions get compressed by default and how to disable this behaviour, see for example: http://ponystyle.com/blog/2010/03/26/dealing-with-asset-compression-in-android-apps/ Pause / Resume behaviour ================================================================================ If SDL_HINT_ANDROID_BLOCK_ON_PAUSE hint is set (the default), the event loop will block itself when the app is paused (ie, when the user returns to the main Android dashboard). Blocking is better in terms of battery use, and it allows your app to spring back to life instantaneously after resume (versus polling for a resume message). Upon resume, SDL will attempt to restore the GL context automatically. In modern devices (Android 3.0 and up) this will most likely succeed and your app can continue to operate as it was. However, there's a chance (on older hardware, or on systems under heavy load), where the GL context can not be restored. In that case you have to listen for a specific message (SDL_RENDER_DEVICE_RESET) and restore your textures manually or quit the app. You should not use the SDL renderer API while the app going in background: - SDL_APP_WILLENTERBACKGROUND: after you read this message, GL context gets backed-up and you should not use the SDL renderer API. When this event is received, you have to set the render target to NULL, if you're using it. (eg call SDL_SetRenderTarget(renderer, NULL)) - SDL_APP_DIDENTERFOREGROUND: GL context is restored, and the SDL renderer API is available (unless you receive SDL_RENDER_DEVICE_RESET). Mouse / Touch events ================================================================================ In some case, SDL generates synthetic mouse (resp. touch) events for touch (resp. mouse) devices. To enable/disable this behavior, see SDL_hints.h: - SDL_HINT_TOUCH_MOUSE_EVENTS - SDL_HINT_MOUSE_TOUCH_EVENTS Misc ================================================================================ For some device, it appears to works better setting explicitly GL attributes before creating a window: SDL_GL_SetAttribute(SDL_GL_RED_SIZE, 5); SDL_GL_SetAttribute(SDL_GL_GREEN_SIZE, 6); SDL_GL_SetAttribute(SDL_GL_BLUE_SIZE, 5); Threads and the Java VM ================================================================================ For a quick tour on how Linux native threads interoperate with the Java VM, take a look here: https://developer.android.com/guide/practices/jni.html If you want to use threads in your SDL app, it's strongly recommended that you do so by creating them using SDL functions. This way, the required attach/detach handling is managed by SDL automagically. If you have threads created by other means and they make calls to SDL functions, make sure that you call Android_JNI_SetupThread() before doing anything else otherwise SDL will attach your thread automatically anyway (when you make an SDL call), but it'll never detach it. If you ever want to use JNI in a native thread (created by "SDL_CreateThread()"), it won't be able to find your java class and method because of the java class loader which is different for native threads, than for java threads (eg your "main()"). the work-around is to find class/method, in you "main()" thread, and to use them in your native thread. see: https://developer.android.com/training/articles/perf-jni#faq:-why-didnt-findclass-find-my-class Using STL ================================================================================ You can use STL in your project by creating an Application.mk file in the jni folder and adding the following line: APP_STL := c++_shared For more information go here: https://developer.android.com/ndk/guides/cpp-support Using the emulator ================================================================================ There are some good tips and tricks for getting the most out of the emulator here: https://developer.android.com/tools/devices/emulator.html Especially useful is the info on setting up OpenGL ES 2.0 emulation. Notice that this software emulator is incredibly slow and needs a lot of disk space. Using a real device works better. Troubleshooting ================================================================================ You can see if adb can see any devices with the following command: adb devices You can see the output of log messages on the default device with: adb logcat You can push files to the device with: adb push local_file remote_path_and_file You can push files to the SD Card at /sdcard, for example: adb push moose.dat /sdcard/moose.dat You can see the files on the SD card with a shell command: adb shell ls /sdcard/ You can start a command shell on the default device with: adb shell You can remove the library files of your project (and not the SDL lib files) with: ndk-build clean You can do a build with the following command: ndk-build You can see the complete command line that ndk-build is using by passing V=1 on the command line: ndk-build V=1 If your application crashes in native code, you can use ndk-stack to get a symbolic stack trace: https://developer.android.com/ndk/guides/ndk-stack If you want to go through the process manually, you can use addr2line to convert the addresses in the stack trace to lines in your code. For example, if your crash looks like this: I/DEBUG ( 31): signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 400085d0 I/DEBUG ( 31): r0 00000000 r1 00001000 r2 00000003 r3 400085d4 I/DEBUG ( 31): r4 400085d0 r5 40008000 r6 afd41504 r7 436c6a7c I/DEBUG ( 31): r8 436c6b30 r9 435c6fb0 10 435c6f9c fp 4168d82c I/DEBUG ( 31): ip 8346aff0 sp 436c6a60 lr afd1c8ff pc afd1c902 cpsr 60000030 I/DEBUG ( 31): #00 pc 0001c902 /system/lib/libc.so I/DEBUG ( 31): #01 pc 0001ccf6 /system/lib/libc.so I/DEBUG ( 31): #02 pc 000014bc /data/data/org.libsdl.app/lib/libmain.so I/DEBUG ( 31): #03 pc 00001506 /data/data/org.libsdl.app/lib/libmain.so You can see that there's a crash in the C library being called from the main code. I run addr2line with the debug version of my code: arm-eabi-addr2line -C -f -e obj/local/armeabi/libmain.so and then paste in the number after "pc" in the call stack, from the line that I care about: 000014bc I get output from addr2line showing that it's in the quit function, in testspriteminimal.c, on line 23. You can add logging to your code to help show what's happening: #include __android_log_print(ANDROID_LOG_INFO, "foo", "Something happened! x = %d", x); If you need to build without optimization turned on, you can create a file called "Application.mk" in the jni directory, with the following line in it: APP_OPTIM := debug Memory debugging ================================================================================ The best (and slowest) way to debug memory issues on Android is valgrind. Valgrind has support for Android out of the box, just grab code using: svn co svn://svn.valgrind.org/valgrind/trunk valgrind ... and follow the instructions in the file README.android to build it. One thing I needed to do on Mac OS X was change the path to the toolchain, and add ranlib to the environment variables: export RANLIB=$NDKROOT/toolchains/arm-linux-androideabi-4.4.3/prebuilt/darwin-x86/bin/arm-linux-androideabi-ranlib Once valgrind is built, you can create a wrapper script to launch your application with it, changing org.libsdl.app to your package identifier: --- start_valgrind_app ------------------- #!/system/bin/sh export TMPDIR=/data/data/org.libsdl.app exec /data/local/Inst/bin/valgrind --log-file=/sdcard/valgrind.log --error-limit=no $* ------------------------------------------ Then push it to the device: adb push start_valgrind_app /data/local and make it executable: adb shell chmod 755 /data/local/start_valgrind_app and tell Android to use the script to launch your application: adb shell setprop wrap.org.libsdl.app "logwrapper /data/local/start_valgrind_app" If the setprop command says "could not set property", it's likely that your package name is too long and you should make it shorter by changing AndroidManifest.xml and the path to your class file in android-project/src You can then launch your application normally and waaaaaaaiiittt for it. You can monitor the startup process with the logcat command above, and when it's done (or even while it's running) you can grab the valgrind output file: adb pull /sdcard/valgrind.log When you're done instrumenting with valgrind, you can disable the wrapper: adb shell setprop wrap.org.libsdl.app "" Graphics debugging ================================================================================ If you are developing on a compatible Tegra-based tablet, NVidia provides Tegra Graphics Debugger at their website. Because SDL2 dynamically loads EGL and GLES libraries, you must follow their instructions for installing the interposer library on a rooted device. The non-rooted instructions are not compatible with applications that use SDL2 for video. The Tegra Graphics Debugger is available from NVidia here: https://developer.nvidia.com/tegra-graphics-debugger Why is API level 19 the minimum required? ================================================================================ The latest NDK toolchain doesn't support targeting earlier than API level 19. As of this writing, according to https://www.composables.com/tools/distribution-chart about 99.7% of the Android devices accessing Google Play support API level 19 or higher (August 2023). A note regarding the use of the "dirty rectangles" rendering technique ================================================================================ If your app uses a variation of the "dirty rectangles" rendering technique, where you only update a portion of the screen on each frame, you may notice a variety of visual glitches on Android, that are not present on other platforms. This is caused by SDL's use of EGL as the support system to handle OpenGL ES/ES2 contexts, in particular the use of the eglSwapBuffers function. As stated in the documentation for the function "The contents of ancillary buffers are always undefined after calling eglSwapBuffers". Setting the EGL_SWAP_BEHAVIOR attribute of the surface to EGL_BUFFER_PRESERVED is not possible for SDL as it requires EGL 1.4, available only on the API level 17+, so the only workaround available on this platform is to redraw the entire screen each frame. Reference: http://www.khronos.org/registry/egl/specs/EGLTechNote0001.html Ending your application ================================================================================ Two legitimate ways: - return from your main() function. Java side will automatically terminate the Activity by calling Activity.finish(). - Android OS can decide to terminate your application by calling onDestroy() (see Activity life cycle). Your application will receive a SDL_QUIT event you can handle to save things and quit. Don't call exit() as it stops the activity badly. NB: "Back button" can be handled as a SDL_KEYDOWN/UP events, with Keycode SDLK_AC_BACK, for any purpose. Known issues ================================================================================ - The number of buttons reported for each joystick is hardcoded to be 36, which is the current maximum number of buttons Android can report. --- ### Vs Sdl2 Docs README Cmake (`vs/sdl2/docs/README-cmake.md`) # CMake (www.cmake.org) SDL's build system was traditionally based on autotools. Over time, this approach has suffered from several issues across the different supported platforms. To solve these problems, a new build system based on CMake was introduced. It is developed in parallel to the legacy autotools build system, so users can experiment with it without complication. The CMake build system is supported on the following platforms: * FreeBSD * Linux * Microsoft Visual C * MinGW and Msys * macOS, iOS, and tvOS, with support for XCode * Android * Emscripten * RiscOS * Playstation Vita ## Building SDL Assuming the source for SDL is located at `~/sdl` ```sh cd ~ mkdir build cd build cmake ~/sdl cmake --build . ``` This will build the static and dynamic versions of SDL in the `~/build` directory. Installation can be done using: ```sh cmake --install . # '--install' requires CMake 3.15, or newer ``` ## Including SDL in your project SDL can be included in your project in 2 major ways: - using a system SDL library, provided by your (*nix) distribution or a package manager - using a vendored SDL library: this is SDL copied or symlinked in a subfolder. The following CMake script supports both, depending on the value of `MYGAME_VENDORED`. ```cmake cmake_minimum_required(VERSION 3.5) project(mygame) # Create an option to switch between a system sdl library and a vendored sdl library option(MYGAME_VENDORED "Use vendored libraries" OFF) if(MYGAME_VENDORED) add_subdirectory(vendored/sdl EXCLUDE_FROM_ALL) else() # 1. Look for a SDL2 package, 2. look for the SDL2 component and 3. fail if none can be found find_package(SDL2 REQUIRED CONFIG REQUIRED COMPONENTS SDL2) # 1. Look for a SDL2 package, 2. Look for the SDL2maincomponent and 3. DO NOT fail when SDL2main is not available find_package(SDL2 REQUIRED CONFIG COMPONENTS SDL2main) endif() # Create your game executable target as usual add_executable(mygame WIN32 mygame.c) # SDL2::SDL2main may or may not be available. It is e.g. required by Windows GUI applications if(TARGET SDL2::SDL2main) # It has an implicit dependency on SDL2 functions, so it MUST be added before SDL2::SDL2 (or SDL2::SDL2-static) target_link_libraries(mygame PRIVATE SDL2::SDL2main) endif() # Link to the actual SDL2 library. SDL2::SDL2 is the shared SDL library, SDL2::SDL2-static is the static SDL libarary. target_link_libraries(mygame PRIVATE SDL2::SDL2) ``` ### A system SDL library For CMake to find SDL, it must be installed in [a default location CMake is looking for](https://cmake.org/cmake/help/latest/command/find_package.html#config-mode-search-procedure). The following components are available, to be used as an argument of `find_package`. | Component name | Description | |----------------|--------------------------------------------------------------------------------------------| | SDL2 | The SDL2 shared library, available through the `SDL2::SDL2` target [^SDL_TARGET_EXCEPTION] | | SDL2-static | The SDL2 static library, available through the `SDL2::SDL2-static` target | | SDL2main | The SDL2main static library, available through the `SDL2::SDL2main` target | | SDL2test | The SDL2test static library, available through the `SDL2::SDL2test` target | ### Using a vendored SDL This only requires a copy of SDL in a subdirectory. ## CMake configuration options for platforms ### iOS/tvOS CMake 3.14+ natively includes support for iOS and tvOS. SDL binaries may be built using Xcode or Make, possibly among other build-systems. When using a recent version of CMake (3.14+), it should be possible to: - build SDL for iOS, both static and dynamic - build SDL test apps (as iOS/tvOS .app bundles) - generate a working SDL_config.h for iOS (using SDL_config.h.cmake as a basis) To use, set the following CMake variables when running CMake's configuration stage: - `CMAKE_SYSTEM_NAME=` (either `iOS` or `tvOS`) - `CMAKE_OSX_SYSROOT=` (examples: `iphoneos`, `iphonesimulator`, `iphoneos12.4`, `/full/path/to/iPhoneOS.sdk`, `appletvos`, `appletvsimulator`, `appletvos12.4`, `/full/path/to/AppleTVOS.sdk`, etc.) - `CMAKE_OSX_ARCHITECTURES=` (example: "arm64;armv7s;x86_64") #### Examples - for iOS-Simulator, using the latest, installed SDK: ```bash cmake ~/sdl -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_SYSROOT=iphonesimulator -DCMAKE_OSX_ARCHITECTURES=x86_64 ``` - for iOS-Device, using the latest, installed SDK, 64-bit only ```bash cmake ~/sdl -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_SYSROOT=iphoneos -DCMAKE_OSX_ARCHITECTURES=arm64 ``` - for iOS-Device, using the latest, installed SDK, mixed 32/64 bit ```cmake cmake ~/sdl -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_SYSROOT=iphoneos -DCMAKE_OSX_ARCHITECTURES="arm64;armv7s" ``` - for iOS-Device, using a specific SDK revision (iOS 12.4, in this example): ```cmake cmake ~/sdl -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_SYSROOT=iphoneos12.4 -DCMAKE_OSX_ARCHITECTURES=arm64 ``` - for iOS-Simulator, using the latest, installed SDK, and building SDL test apps (as .app bundles): ```cmake cmake ~/sdl -DSDL_TESTS=1 -DCMAKE_SYSTEM_NAME=iOS -DCMAKE_OSX_SYSROOT=iphonesimulator -DCMAKE_OSX_ARCHITECTURES=x86_64 ``` - for tvOS-Simulator, using the latest, installed SDK: ```cmake cmake ~/sdl -DCMAKE_SYSTEM_NAME=tvOS -DCMAKE_OSX_SYSROOT=appletvsimulator -DCMAKE_OSX_ARCHITECTURES=x86_64 ``` - for tvOS-Device, using the latest, installed SDK: ```cmake cmake ~/sdl -DCMAKE_SYSTEM_NAME=tvOS -DCMAKE_OSX_SYSROOT=appletvos -DCMAKE_OSX_ARCHITECTURES=arm64` ``` [^SDL_TARGET_EXCEPTION]: `SDL2::SDL2` can be an ALIAS to a static `SDL2::SDL2-static` target for multiple reasons. --- ### Vs Sdl2 Docs README Directfb (`vs/sdl2/docs/README-directfb.md`) DirectFB ======== Supports: - Hardware YUV overlays - OpenGL - software only - 2D/3D accelerations (depends on directfb driver) - multiple displays - windows What you need: * DirectFB 1.0.1, 1.2.x, 1.3.0 * Kernel-Framebuffer support: required: vesafb, radeonfb .... * Mesa 7.0.x - optional for OpenGL The `/etc/directfbrc` file should contain the following lines to make your joystick work and avoid crashes: ``` disable-module=joystick disable-module=cle266 disable-module=cyber5k no-linux-input-grab ``` To disable to use x11 backend when DISPLAY variable is found use ``` export SDL_DIRECTFB_X11_CHECK=0 ``` To disable the use of linux input devices, i.e. multimice/multikeyboard support, use ``` export SDL_DIRECTFB_LINUX_INPUT=0 ``` To use hardware accelerated YUV-overlays for YUV-textures, use: ``` export SDL_DIRECTFB_YUV_DIRECT=1 ``` This is disabled by default. It will only support one YUV texture, namely the first. Every other YUV texture will be rendered in software. In addition, you may use (directfb-1.2.x) ``` export SDL_DIRECTFB_YUV_UNDERLAY=1 ``` to make the YUV texture an underlay. This will make the cursor to be shown. Simple Window Manager ===================== The driver has support for a very, very basic window manager you may want to use when running with `wm=default`. Use ``` export SDL_DIRECTFB_WM=1 ``` to enable basic window borders. In order to have the window title rendered, you need to have the following font installed: ``` /usr/share/fonts/truetype/freefont/FreeSans.ttf ``` OpenGL Support ============== The following instructions will give you *software* OpenGL. However this works at least on all directfb supported platforms. As of this writing 20100802 you need to pull Mesa from git and do the following: ``` git clone git://anongit.freedesktop.org/git/mesa/mesa cd mesa git checkout 2c9fdaf7292423c157fc79b5ce43f0f199dd753a ``` Edit `configs/linux-directfb` so that the Directories-section looks like this: ``` # Directories SRC_DIRS = mesa glu GLU_DIRS = sgi DRIVER_DIRS = directfb PROGRAM_DIRS = ``` Then do the following: ``` make linux-directfb make echo Installing - please enter sudo pw. sudo make install INSTALL_DIR=/usr/local/dfb_GL cd src/mesa/drivers/directfb make sudo make install INSTALL_DIR=/usr/local/dfb_GL ``` To run the SDL - testprograms: ``` export SDL_VIDEODRIVER=directfb export LD_LIBRARY_PATH=/usr/local/dfb_GL/lib export LD_PRELOAD=/usr/local/dfb_GL/libGL.so.7 ./testgl ``` --- ### Vs Sdl2 Docs README Dynapi (`vs/sdl2/docs/README-dynapi.md`) # Dynamic API Originally posted on Ryan's Google+ account. Background: - The Steam Runtime has (at least in theory) a really kick-ass build of SDL2, but developers are shipping their own SDL2 with individual Steam games. These games might stop getting updates, but a newer SDL2 might be needed later. Certainly we'll always be fixing bugs in SDL, even if a new video target isn't ever needed, and these fixes won't make it to a game shipping its own SDL. - Even if we replace the SDL2 in those games with a compatible one, that is to say, edit a developer's Steam depot (yuck!), there are developers that are statically linking SDL2 that we can't do this for. We can't even force the dynamic loader to ignore their SDL2 in this case, of course. - If you don't ship an SDL2 with the game in some form, people that disabled the Steam Runtime, or just tried to run the game from the command line instead of Steam might find themselves unable to run the game, due to a missing dependency. - If you want to ship on non-Steam platforms like GOG or Humble Bundle, or target generic Linux boxes that may or may not have SDL2 installed, you have to ship the library or risk a total failure to launch. So now, you might have to have a non-Steam build plus a Steam build (that is, one with and one without SDL2 included), which is inconvenient if you could have had one universal build that works everywhere. - We like the zlib license, but the biggest complaint from the open source community about the license change is the static linking. The LGPL forced this as a legal, not technical issue, but zlib doesn't care. Even those that aren't concerned about the GNU freedoms found themselves solving the same problems: swapping in a newer SDL to an older game often times can save the day. Static linking stops this dead. So here's what we did: SDL now has, internally, a table of function pointers. So, this is what SDL_Init now looks like: ```c Uint32 SDL_Init(Uint32 flags) { return jump_table.SDL_Init(flags); } ``` Except that is all done with a bunch of macro magic so we don't have to maintain every one of these. What is jump_table.SDL_init()? Eventually, that's a function pointer of the real SDL_Init() that you've been calling all this time. But at startup, it looks more like this: ```c Uint32 SDL_Init_DEFAULT(Uint32 flags) { SDL_InitDynamicAPI(); return jump_table.SDL_Init(flags); } ``` SDL_InitDynamicAPI() fills in jump_table with all the actual SDL function pointers, which means that this `_DEFAULT` function never gets called again. First call to any SDL function sets the whole thing up. So you might be asking, what was the value in that? Isn't this what the operating system's dynamic loader was supposed to do for us? Yes, but now we've got this level of indirection, we can do things like this: ```bash export SDL_DYNAMIC_API=/my/actual/libSDL-2.0.so.0 ./MyGameThatIsStaticallyLinkedToSDL2 ``` And now, this game that is statically linked to SDL, can still be overridden with a newer, or better, SDL. The statically linked one will only be used as far as calling into the jump table in this case. But in cases where no override is desired, the statically linked version will provide its own jump table, and everyone is happy. So now: - Developers can statically link SDL, and users can still replace it. (We'd still rather you ship a shared library, though!) - Developers can ship an SDL with their game, Valve can override it for, say, new features on SteamOS, or distros can override it for their own needs, but it'll also just work in the default case. - Developers can ship the same package to everyone (Humble Bundle, GOG, etc), and it'll do the right thing. - End users (and Valve) can update a game's SDL in almost any case, to keep abandoned games running on newer platforms. - Everyone develops with SDL exactly as they have been doing all along. Same headers, same ABI. Just get the latest version to enable this magic. A little more about SDL_InitDynamicAPI(): Internally, InitAPI does some locking to make sure everything waits until a single thread initializes everything (although even SDL_CreateThread() goes through here before spinning a thread, too), and then decides if it should use an external SDL library. If not, it sets up the jump table using the current SDL's function pointers (which might be statically linked into a program, or in a shared library of its own). If so, it loads that library and looks for and calls a single function: ```c Sint32 SDL_DYNAPI_entry(Uint32 version, void *table, Uint32 tablesize); ``` That function takes a version number (more on that in a moment), the address of the jump table, and the size, in bytes, of the table. Now, we've got policy here: this table's layout never changes; new stuff gets added to the end. Therefore SDL_DYNAPI_entry() knows that it can provide all the needed functions if tablesize <= sizeof its own jump table. If tablesize is bigger (say, SDL 2.0.4 is trying to load SDL 2.0.3), then we know to abort, but if it's smaller, we know we can provide the entire API that the caller needs. The version variable is a failsafe switch. Right now it's always 1. This number changes when there are major API changes (so we know if the tablesize might be smaller, or entries in it have changed). Right now SDL_DYNAPI_entry gives up if the version doesn't match, but it's not inconceivable to have a small dispatch library that only supplies this one function and loads different, otherwise-incompatible SDL libraries and has the right one initialize the jump table based on the version. For something that must generically catch lots of different versions of SDL over time, like the Steam Client, this isn't a bad option. Finally, I'm sure some people are reading this and thinking, "I don't want that overhead in my project!" To which I would point out that the extra function call through the jump table probably wouldn't even show up in a profile, but lucky you: this can all be disabled. You can build SDL without this if you absolutely must, but we would encourage you not to do that. However, on heavily locked down platforms like iOS, or maybe when debugging, it makes sense to disable it. The way this is designed in SDL, you just have to change one #define, and the entire system vaporizes out, and SDL functions exactly like it always did. Most of it is macro magic, so the system is contained to one C file and a few headers. However, this is on by default and you have to edit a header file to turn it off. Our hopes is that if we make it easy to disable, but not too easy, everyone will ultimately be able to get what they want, but we've gently nudged everyone towards what we think is the best solution. --- ### Vs Sdl2 Docs README Emscripten (`vs/sdl2/docs/README-emscripten.md`) # Emscripten ## The state of things (As of September 2023, but things move quickly and we don't update this document often.) In modern times, all the browsers you probably care about (Chrome, Firefox, Edge, and Safari, on Windows, macOS, Linux, iOS and Android), support some reasonable base configurations: - WebAssembly (don't bother with asm.js any more) - WebGL (which will look like OpenGL ES 2 or 3 to your app). - Threads (see caveats, though!) - Game controllers - Autoupdating (so you can assume they have a recent version of the browser) All this to say we're at the point where you don't have to make a lot of concessions to get even a fairly complex SDL-based game up and running. ## RTFM This document is a quick rundown of some high-level details. The documentation at [emscripten.org](https://emscripten.org/) is vast and extremely detailed for a wide variety of topics, and you should at least skim through it at some point. ## Porting your app to Emscripten Many many things just need some simple adjustments and they'll compile like any other C/C++ code, as long as SDL was handling the platform-specific work for your program. First, you probably need this in at least one of your source files: ```c #ifdef __EMSCRIPTEN__ #include #endif ``` Second: assembly language code has to go. Replace it with C. You can even use [x86 SIMD intrinsic functions in Emscripten](https://emscripten.org/docs/porting/simd.html)! Third: Middleware has to go. If you have a third-party library you link against, you either need an Emscripten port of it, or the source code to it to compile yourself, or you need to remove it. Fourth: You still start in a function called main(), but you need to get out of it and into a function that gets called repeatedly, and returns quickly, called a mainloop. Somewhere in your program, you probably have something that looks like a more complicated version of this: ```c void main(void) { initialize_the_game(); while (game_is_still_running) { check_for_new_input(); think_about_stuff(); draw_the_next_frame(); } deinitialize_the_game(); } ``` This will not work on Emscripten, because the main thread needs to be free to do stuff and can't sit in this loop forever. So Emscripten lets you set up a [mainloop](https://emscripten.org/docs/porting/emscripten-runtime-environment.html#browser-main-loop). ```c static void mainloop(void) /* this will run often, possibly at the monitor's refresh rate */ { if (!game_is_still_running) { deinitialize_the_game(); #ifdef __EMSCRIPTEN__ emscripten_cancel_main_loop(); /* this should "kill" the app. */ #else exit(0); #endif } check_for_new_input(); think_about_stuff(); draw_the_next_frame(); } void main(void) { initialize_the_game(); #ifdef __EMSCRIPTEN__ emscripten_set_main_loop(mainloop, 0, 1); #else while (1) { mainloop(); } #endif } ``` Basically, `emscripten_set_main_loop(mainloop, 0, 1);` says "run `mainloop` over and over until I end the program." The function will run, and return, freeing the main thread for other tasks, and then run again when it's time. The `1` parameter does some magic to make your main() function end immediately; this is useful because you don't want any shutdown code that might be sitting below this code to actually run if main() were to continue on, since we're just getting started. There's a lot of little details that are beyond the scope of this document, but that's the biggest intial set of hurdles to porting your app to the web. ## Do you need threads? If you plan to use threads, they work on all major browsers now. HOWEVER, they bring with them a lot of careful considerations. Rendering _must_ be done on the main thread. This is a general guideline for many platforms, but a hard requirement on the web. Many other things also must happen on the main thread; often times SDL and Emscripten make efforts to "proxy" work to the main thread that must be there, but you have to be careful (and read more detailed documentation than this for the finer points). Even when using threads, your main thread needs to set an Emscripten mainloop that runs quickly and returns, or things will fail to work correctly. You should definitely read [Emscripten's pthreads docs](https://emscripten.org/docs/porting/pthreads.html) for all the finer points. Mostly SDL's thread API will work as expected, but is built on pthreads, so it shares the same little incompatibilities that are documented there, such as where you can use a mutex, and when a thread will start running, etc. IMPORTANT: You have to decide to either build something that uses threads or something that doesn't; you can't have one build that works everywhere. This is an Emscripten (or maybe WebAssembly? Or just web browsers in general?) limitation. If you aren't using threads, it's easier to not enable them at all, at build time. If you use threads, you _have to_ run from a web server that has [COOP/COEP headers set correctly](https://web.dev/why-coop-coep/) or your program will fail to start at all. If building with threads, `__EMSCRIPTEN_PTHREADS__` will be defined for checking with the C preprocessor, so you can build something different depending on what sort of build you're compiling. ## Audio Audio works as expected at the API level, but not exactly like other platforms. You'll only see a single default audio device. Audio capture also works; if the browser pops up a prompt to ask for permission to access the microphone, the SDL_OpenAudioDevice call will succeed and start producing silence at a regular interval. Once the user approves the request, real audio data will flow. If the user denies it, the app is not informed and will just continue to receive silence. Modern web browsers will not permit web pages to produce sound before the user has interacted with them (clicked or tapped on them, usually); this is for several reasons, not the least of which being that no one likes when a random browser tab suddenly starts making noise and the user has to scramble to figure out which and silence it. SDL will allow you to open the audio device for playback in this circumstance, and your audio callback will fire, but SDL will throw the audio data away until the user interacts with the page. This helps apps that depend on the audio callback to make progress, and also keeps audio playback in sync once the app is finally allowed to make noise. There are two reasonable ways to deal with the silence at the app level: if you are writing some sort of media player thing, where the user expects there to be a volume control when you mouseover the canvas, just default that control to a muted state; if the user clicks on the control to unmute it, on this first click, open the audio device. This allows the media to play at start, and the user can reasonably opt-in to listening. Many games do not have this sort of UI, and are more rigid about starting audio along with everything else at the start of the process. For these, your best bet is to write a little Javascript that puts up a "Click here to play!" UI, and upon the user clicking, remove that UI and then call the Emscripten app's main() function. As far as the application knows, the audio device was available to be opened as soon as the program started, and since this magic happens in a little Javascript, you don't have to change your C/C++ code at all to make it happen. Please see the discussion at https://github.com/libsdl-org/SDL/issues/6385 for some Javascript code to steal for this approach. ## Rendering If you use SDL's 2D render API, it will use GLES2 internally, which Emscripten will turn into WebGL calls. You can also use OpenGL ES 2 directly by creating a GL context and drawing into it. Calling SDL_RenderPresent (or SDL_GL_SwapWindow) will not actually present anything on the screen until your return from your mainloop function. ## Building SDL/emscripten First: do you _really_ need to build SDL from source? If you aren't developing SDL itself, have a desire to mess with its source code, or need something on the bleeding edge, don't build SDL. Just use Emscripten's packaged version! Compile and link your app with `-sUSE_SDL=2` and it'll use a build of SDL packaged with Emscripten. This comes from the same source code and fixes the Emscripten project makes to SDL are generally merged into SDL's revision control, so often this is much easier for app developers. `-sUSE_SDL=1` will select Emscripten's JavaScript reimplementation of SDL 1.2 instead; if you need SDL 1.2, this might be fine, but we generally recommend you don't use SDL 1.2 in modern times. If you want to build SDL, though... SDL currently requires at least Emscripten 3.1.35 to build. Newer versions are likely to work, as well. Build: This works on Linux/Unix and macOS. Please send comments about Windows. Make sure you've [installed emsdk](https://emscripten.org/docs/getting_started/downloads.html) first, and run `source emsdk_env.sh` at the command line so it finds the tools. (These configure options might be overkill, but this has worked for me.) ```bash cd SDL mkdir build cd build emconfigure ../configure --host=wasm32-unknown-emscripten --disable-pthreads --disable-assembly --disable-cpuinfo CFLAGS="-sUSE_SDL=0 -O3" emmake make -j4 ``` If you want to build with thread support, something like this works: ```bash emconfigure ../configure --host=wasm32-unknown-emscripten --enable-pthreads --disable-assembly --disable-cpuinfo CFLAGS="-sUSE_SDL=0 -O3 -pthread" LDFLAGS="-pthread" ``` Or with cmake: ```bash mkdir build cd build emcmake cmake .. emmake make -j4 ``` To build one of the tests: ```bash cd test/ emcc -O2 --js-opts 0 -g4 testdraw2.c -I../include ../build/.libs/libSDL2.a ../build/libSDL2_test.a -o a.html ``` ## Building your app You need to compile with `emcc` instead of `gcc` or `clang` or whatever, but mostly it uses the same command line arguments as Clang. Link against the SDL/build/.libs/libSDL2.a file you generated by building SDL, link with `-sUSE_SDL=2` to use Emscripten's prepackaged SDL2 build. Usually you would produce a binary like this: ```bash gcc -o mygame mygame.c # or whatever ``` But for Emscripten, you want to output something else: ```bash emcc -o index.html mygame.c ``` This will produce several files...support Javascript and WebAssembly (.wasm) files. The `-o index.html` will produce a simple HTML page that loads and runs your app. You will (probably) eventually want to replace or customize that file and do `-o index.js` instead to just build the code pieces. If you're working on a program of any serious size, you'll likely need to link with `-sALLOW_MEMORY_GROWTH=1 -sMAXIMUM_MEMORY=1gb` to get access to more memory. If using pthreads, you'll need the `-sMAXIMUM_MEMORY=1gb` or the app will fail to start on iOS browsers, but this might be a bug that goes away in the future. ## Data files Your game probably has data files. Here's how to access them. Filesystem access works like a Unix filesystem; you have a single directory tree, possibly interpolated from several mounted locations, no drive letters, '/' for a path separator. You can access them with standard file APIs like open() or fopen() or SDL_RWops. You can read or write from the filesystem. By default, you probably have a "MEMFS" filesystem (all files are stored in memory, but access to them is immediate and doesn't need to block). There are other options, like "IDBFS" (files are stored in a local database, so they don't need to be in RAM all the time and they can persist between runs of the program, but access is not synchronous). You can mix and match these file systems, mounting a MEMFS filesystem at one place and idbfs elsewhere, etc, but that's beyond the scope of this document. Please refer to Emscripten's [page on the topic](https://emscripten.org/docs/porting/files/file_systems_overview.html) for more info. The _easiest_ (but not the best) way to get at your data files is to embed them in the app itself. Emscripten's linker has support for automating this. ```bash emcc -o index.html loopwave.c --embed-file=../test/sample.wav@/sounds/sample.wav ``` This will pack ../test/sample.wav in your app, and make it available at "/sounds/sample.wav" at runtime. Emscripten makes sure this data is available before your main() function runs, and since it's in MEMFS, you can just read it like you do on other platforms. `--embed-file` can also accept a directory to pack an entire tree, and you can specify the argument multiple times to pack unrelated things into the final installation. Note that this is absolutely the best approach if you have a few small files to include and shouldn't worry about the issue further. However, if you have hundreds of megabytes and/or thousands of files, this is not so great, since the user will download it all every time they load your page, and it all has to live in memory at runtime. [Emscripten's documentation on the matter](https://emscripten.org/docs/porting/files/packaging_files.html) gives other options and details, and is worth a read. ## Debugging Debugging web apps is a mixed bag. You should compile and link with `-gsource-map`, which embeds a ton of source-level debugging information into the build, and make sure _the app source code is available on the web server_, which is often a scary proposition for various reasons. When you debug from the browser's tools and hit a breakpoint, you can step through the actual C/C++ source code, though, which can be nice. If you try debugging in Firefox and it doesn't work well for no apparent reason, try Chrome, and vice-versa. These tools are still relatively new, and improving all the time. SDL_Log() (or even plain old printf) will write to the Javascript console, and honestly I find printf-style debugging to be easier than setting up a build for proper debugging, so use whatever tools work best for you. ## Questions? Please give us feedback on this document at [the SDL bug tracker](https://github.com/libsdl-org/SDL/issues). If something is wrong or unclear, we want to know! --- ### Vs Sdl2 Docs README Gdk (`vs/sdl2/docs/README-gdk.md`) GDK ===== This port allows SDL applications to run via Microsoft's Game Development Kit (GDK). Windows (GDK) and Xbox One/Xbox Series (GDKX) are both supported and all the required code is included in this public SDL release. However, only licensed Xbox developers have access to the GDKX libraries which will allow you to build the Xbox targets. Requirements ------------ * Microsoft Visual Studio 2022 (in theory, it should also work in 2017 or 2019, but this has not been tested) * Microsoft GDK June 2022 or newer (public release [here](https://github.com/microsoft/GDK/releases/tag/June_2022)) * For Xbox, you will need the corresponding GDKX version (licensed developers only) * To publish a package or successfully authenticate a user, you will need to create an app id/configure services in Partner Center. However, for local testing purposes (without authenticating on Xbox Live), the identifiers used by the GDK test programs in the included solution will work. Windows GDK Status ------ The Windows GDK port supports the full set of Win32 APIs, renderers, controllers, input devices, etc., as the normal Windows x64 build of SDL. * Additionally, the GDK port adds the following: * Compile-time platform detection for SDL programs. The `__GDK__` is `#define`d on every GDK platform, and the `__WINGDK__` is `#define`d on Windows GDK, specifically. (This distinction exists because other GDK platforms support a smaller subset of functionality. This allows you to mark code for "any" GDK separate from Windows GDK.) * GDK-specific setup: * Initializing/uninitializing the game runtime, and initializing Xbox Live services * Creating a global task queue and setting it as the default for the process. When running any async operations, passing in `NULL` as the task queue will make the task get added to the global task queue. * An implementation on `WinMain` that performs the above GDK setup (you should link against SDL2main.lib, as in Windows x64). If you are unable to do this, you can instead manually call `SDL_GDKRunApp` from your entry point, passing in your `SDL_main` function and `NULL` as the parameters. * Global task queue callbacks are dispatched during `SDL_PumpEvents` (which is also called internally if using `SDL_PollEvent`). * You can get the handle of the global task queue through `SDL_GDKGetTaskQueue`, if needed. When done with the queue, be sure to use `XTaskQueueCloseHandle` to decrement the reference count (otherwise it will cause a resource leak). * Single-player games have some additional features available: * Call `SDL_GDKGetDefaultUser` to get the default XUserHandle pointer. * `SDL_GetPrefPath` still works, but only for single-player titles. These functions mostly wrap around async APIs, and thus should be treated as synchronous alternatives. Also note that the single-player functions return on any OS errors, so be sure to validate the return values! * What doesn't work: * Compilation with anything other than through the included Visual C++ solution file ## VisualC-GDK Solution The included `VisualC-GDK/SDL.sln` solution includes the following targets for the Gaming.Desktop.x64 configuration: * SDL2 (DLL) - This is the typical SDL2.dll, but for Gaming.Desktop.x64. * SDL2main (lib) - This contains a drop-in implementation of `WinMain` that is used as the entry point for GDK programs. * tests/testgamecontroller - Standard SDL test program demonstrating controller functionality. * tests/testgdk - GDK-specific test program that demonstrates using the global task queue to login a user into Xbox Live. *NOTE*: As of the June 2022 GDK, you cannot test user logins without a valid Title ID and MSAAppId. You will need to manually change the identifiers in the `MicrosoftGame.config` to your valid IDs from Partner Center if you wish to test this. * tests/testsprite2 - Standard SDL test program demonstrating sprite drawing functionality. If you set one of the test programs as a startup project, you can run it directly from Visual Studio. Windows GDK Setup, Detailed Steps --------------------- These steps assume you already have a game using SDL that runs on Windows x64 along with a corresponding Visual Studio solution file for the x64 version. If you don't have this, it's easiest to use one of the test program vcxproj files in the `VisualC-GDK` directory as a starting point, though you will still need to do most of the steps below. ### 1. Add a Gaming.Desktop.x64 Configuration ### In your game's existing Visual Studio Solution, go to Build > Configuration Manager. From the "Active solution platform" drop-down select "New...". From the drop-down list, select Gaming.Desktop.x64 and copy the settings from the x64 configuration. ### 2. Build SDL2 and SDL2main for GDK ### Open `VisualC-GDK/SDL.sln` in Visual Studio, you need to build the SDL2 and SDL2main targets for the Gaming.Desktop.x64 platform (Release is recommended). You will need to copy/keep track of the `SDL2.dll`, `XCurl.dll` (which is output by Gaming.Desktop.x64), `SDL2.lib`, and `SDL2main.lib` output files for your game project. *Alternatively*, you could setup your solution file to instead reference the SDL2/SDL2main project file targets from the SDL source, and add those projects as a dependency. This would mean that SDL2 and SDL2main would both be built when your game is built. ### 3. Configuring Project Settings ### While the Gaming.Desktop.x64 configuration sets most of the required settings, there are some additional items to configure for your game project under the Gaming.Desktop.x64 Configuration: * Under C/C++ > General > Additional Include Directories, make sure the `SDL/include` path is referenced * Under Linker > General > Additional Library Directories, make sure to reference the path where the newly-built SDL2.lib and SDL2main.lib are * Under Linker > Input > Additional Dependencies, you need the following: * `SDL2.lib` * `SDL2main.lib` (unless not using) * `xgameruntime.lib` * `../Microsoft.Xbox.Services.141.GDK.C.Thunks.lib` * Note that in general, the GDK libraries depend on the MSVC C/C++ runtime, so there is no way to remove this dependency from a GDK program that links against GDK. ### 4. Setting up SDL_main ### Rather than using your own implementation of `WinMain`, it's recommended that you instead `#include "SDL_main.h"` and declare a standard main function. If you are unable to do this, you can instead manually call `SDL_GDKRunApp` from your entry point, passing in your `SDL_main` function and `NULL` as the parameters. ### 5. Required DLLs ### The game will not launch in the debugger unless required DLLs are included in the directory that contains the game's .exe file. You need to make sure that the following files are copied into the directory: * Your SDL2.dll * "$(Console_GRDKExtLibRoot)Xbox.Services.API.C\DesignTime\CommonConfiguration\Neutral\Lib\Release\Microsoft.Xbox.Services.141.GDK.C.Thunks.dll" * XCurl.dll You can either copy these in a post-build step, or you can add the dlls into the project and set its Configuration Properties > General > Item type to "Copy file," which will also copy them into the output directory. ### 6. Setting up MicrosoftGame.config ### You can copy `VisualC-GDK/tests/testgdk/MicrosoftGame.config` and use that as a starting point in your project. Minimally, you will want to change the Executable Name attribute, the DefaultDisplayName, and the Description. This file must be copied into the same directory as the game's .exe file. As with the DLLs, you can either use a post-build step or the "Copy file" item type. For basic testing, you do not need to change anything else in `MicrosoftGame.config`. However, if you want to test any Xbox Live services (such as logging in users) _or_ publish a package, you will need to setup a Game app on Partner Center. Then, you need to set the following values to the values from Partner Center: * Identity tag - Name and Publisher attributes * TitleId * MSAAppId ### 7. Adding Required Logos Several logo PNG files are required to be able to launch the game, even from the debugger. You can use the sample logos provided in `VisualC-GDK/logos`. As with the other files, they must be copied into the same directory as the game's .exe file. ### 8. Copying any Data Files ### When debugging GDK games, there is no way to specify a working directory. Therefore, any required game data must also be copied into the output directory, likely in a post-build step. ### 9. Build and Run from Visual Studio ### At this point, you should be able to build and run your game from the Visual Studio Debugger. If you get any linker errors, make sure you double-check that you referenced all the required libs. If you are testing Xbox Live functionality, it's likely you will need to change to the Sandbox for your title. To do this: 1. Run "Desktop VS 2022 Gaming Command Prompt" from the Start Menu 2. Switch the sandbox name with: `XblPCSandbox SANDBOX.#` 3. (To switch back to the retail sandbox): `XblPCSandbox RETAIL` ### 10. Packaging and Installing Locally You can use one of the test program's `PackageLayout.xml` as a starting point. Minimally, you will need to change the exe to the correct name and also reference any required game data. As with the other data files, it's easiest if you have this copy to the output directory, although it's not a requirement as you can specify relative paths to files. To create the package: 1. Run "Desktop VS 2022 Gaming Command Prompt" from the Start Menu 2. `cd` to the directory containing the `PackageLayout.xml` with the correct paths (if you use the local path as in the sample package layout, this would be from your .exe output directory) 3. `mkdir Package` to create an output directory 4. To package the file into the `Package` directory, use: `makepkg pack /f PackageLayout.xml /lt /d . /nogameos /pc /pd Package` 5. To install the package, use: `wdapp install PACKAGENAME.msixvc` 6. Once the package is installed, you can run it from the start menu. 7. As with when running from Visual Studio, if you need to test any Xbox Live functionality you must switch to the correct sandbox. Xbox GDKX Setup --------------------- In general, the same process in the Windows GDK instructions work. There are just a few additional notes: * For Xbox One consoles, use the Gaming.Xbox.XboxOne.x64 target * For Xbox Series consoles, use the Gaming.Xbox.Scarlett.x64 target * The Xbox One target sets the `__XBOXONE__` define and the Xbox Series target sets the `__XBOXSERIES__` define * You don't need to link against the Xbox.Services Thunks lib nor include that dll in your package (it doesn't exist for Xbox) * The shader blobs for Xbox are created in a pre-build step for the Xbox targets, rather than included in the source (due to NDA and version compatability reasons) * To create a package, use: `makepkg pack /f PackageLayout.xml /lt /d . /pd Package` * To install the package, use: `xbapp install [PACKAGE].xvc` * For some reason, if you make changes that require SDL2.dll to build, and you are running through the debugger (instead of a package), you have to rebuild your .exe target for the debugger to recognize the dll has changed and needs to be transferred to the console again * While there are successful releases of Xbox titles using this port, it is not as extensively tested as other targets Troubleshooting --------------- #### Xbox Live Login does not work As of June 2022 GDK, you must have a valid Title Id and MSAAppId in order to test Xbox Live functionality such as user login. Make sure these are set correctly in the `MicrosoftGame.config`. This means that even testgdk will not let you login without setting these properties to valid values. Furthermore, confirm that your PC is set to the correct sandbox. #### "The current user has already installed an unpackaged version of this app. A packaged version cannot replace this." error when installing Prior to June 2022 GDK, running from the Visual Studio debugger would still locally register the app (and it would appear on the start menu). To fix this, you have to uninstall it (it's simplest to right click on it from the start menu to uninstall it). --- ### Vs Sdl2 Docs README Gesture (`vs/sdl2/docs/README-gesture.md`) Dollar Gestures =========================================================================== SDL provides an implementation of the $1 gesture recognition system. This allows for recording, saving, loading, and performing single stroke gestures. Gestures can be performed with any number of fingers (the centroid of the fingers must follow the path of the gesture), but the number of fingers must be constant (a finger cannot go down in the middle of a gesture). The path of a gesture is considered the path from the time when the final finger went down, to the first time any finger comes up. Dollar gestures are assigned an Id based on a hash function. This is guaranteed to remain constant for a given gesture. There is a (small) chance that two different gestures will be assigned the same ID. In this case, simply re-recording one of the gestures should result in a different ID. Recording: ---------- To begin recording on a touch device call: SDL_RecordGesture(SDL_TouchID touchId), where touchId is the id of the touch device you wish to record on, or -1 to record on all connected devices. Recording terminates as soon as a finger comes up. Recording is acknowledged by an SDL_DOLLARRECORD event. A SDL_DOLLARRECORD event is a dgesture with the following fields: * event.dgesture.touchId - the Id of the touch used to record the gesture. * event.dgesture.gestureId - the unique id of the recorded gesture. Performing: ----------- As long as there is a dollar gesture assigned to a touch, every finger-up event will also cause an SDL_DOLLARGESTURE event with the following fields: * event.dgesture.touchId - the Id of the touch which performed the gesture. * event.dgesture.gestureId - the unique id of the closest gesture to the performed stroke. * event.dgesture.error - the difference between the gesture template and the actual performed gesture. Lower error is a better match. * event.dgesture.numFingers - the number of fingers used to draw the stroke. Most programs will want to define an appropriate error threshold and check to be sure that the error of a gesture is not abnormally high (an indicator that no gesture was performed). Saving: ------- To save a template, call SDL_SaveDollarTemplate(gestureId, dst) where gestureId is the id of the gesture you want to save, and dst is an SDL_RWops pointer to the file where the gesture will be stored. To save all currently loaded templates, call SDL_SaveAllDollarTemplates(dst) where dst is an SDL_RWops pointer to the file where the gesture will be stored. Both functions return the number of gestures successfully saved. Loading: -------- To load templates from a file, call SDL_LoadDollarTemplates(touchId,src) where touchId is the id of the touch to load to (or -1 to load to all touch devices), and src is an SDL_RWops pointer to a gesture save file. SDL_LoadDollarTemplates returns the number of templates successfully loaded. =========================================================================== Multi Gestures =========================================================================== SDL provides simple support for pinch/rotate/swipe gestures. Every time a finger is moved an SDL_MULTIGESTURE event is sent with the following fields: * event.mgesture.touchId - the Id of the touch on which the gesture was performed. * event.mgesture.x - the normalized x coordinate of the gesture. (0..1) * event.mgesture.y - the normalized y coordinate of the gesture. (0..1) * event.mgesture.dTheta - the amount that the fingers rotated during this motion. * event.mgesture.dDist - the amount that the fingers pinched during this motion. * event.mgesture.numFingers - the number of fingers used in the gesture. =========================================================================== Notes =========================================================================== For a complete example see test/testgesture.c Please direct questions/comments to: jim.tla+sdl_touch@gmail.com --- ### Vs Sdl2 Docs README Git (`vs/sdl2/docs/README-git.md`) git ========= The latest development version of SDL is available via git. Git allows you to get up-to-the-minute fixes and enhancements; as a developer works on a source tree, you can use "git" to mirror that source tree instead of waiting for an official release. Please look at the Git website ( https://git-scm.com/ ) for more information on using git, where you can also download software for macOS, Windows, and Unix systems. git clone https://github.com/libsdl-org/SDL If you are building SDL via configure, you will need to run autogen.sh before running configure. There is a web interface to the Git repository at: http://github.com/libsdl-org/SDL/ --- ### Vs Sdl2 Docs README Hg (`vs/sdl2/docs/README-hg.md`) We are no longer hosted in Mercurial. Please see README-git.md for details. Thanks! --- ### Vs Sdl2 Docs README Ios (`vs/sdl2/docs/README-ios.md`) iOS ====== Building the Simple DirectMedia Layer for iOS 9.0+ ============================================================================== Requirements: Mac OS X 10.9 or later and the iOS 9.0 or newer SDK. Instructions: 1. Open SDL.xcodeproj (located in Xcode/SDL) in Xcode. 2. Select your desired target, and hit build. Using the Simple DirectMedia Layer for iOS ============================================================================== 1. Run Xcode and create a new project using the iOS Game template, selecting the Objective C language and Metal game technology. 2. In the main view, delete all files except for Assets and LaunchScreen 3. Right click the project in the main view, select "Add Files...", and add the SDL project, Xcode/SDL/SDL.xcodeproj 4. Select the project in the main view, go to the "Info" tab and under "Custom iOS Target Properties" remove the line "Main storyboard file base name" 5. Select the project in the main view, go to the "Build Settings" tab, select "All", and edit "Header Search Path" and drag over the SDL "Public Headers" folder from the left 6. Select the project in the main view, go to the "Build Phases" tab, select "Link Binary With Libraries", and add SDL2.framework from "Framework-iOS" 7. Select the project in the main view, go to the "General" tab, scroll down to "Frameworks, Libraries, and Embedded Content", and select "Embed & Sign" for the SDL library. 8. In the main view, expand SDL -> Library Source -> main -> uikit and drag SDL_uikit_main.c into your game files 9. Add the source files that you would normally have for an SDL program, making sure to have #include "SDL.h" at the top of the file containing your main() function. 10. Add any assets that your application needs. 11. Enjoy! TODO: Add information regarding App Store requirements such as icons, etc. Notes -- Retina / High-DPI and window sizes ============================================================================== Window and display mode sizes in SDL are in "screen coordinates" (or "points", in Apple's terminology) rather than in pixels. On iOS this means that a window created on an iPhone 6 will have a size in screen coordinates of 375 x 667, rather than a size in pixels of 750 x 1334. All iOS apps are expected to size their content based on screen coordinates / points rather than pixels, as this allows different iOS devices to have different pixel densities (Retina versus non-Retina screens, etc.) without apps caring too much. By default SDL will not use the full pixel density of the screen on Retina/high-dpi capable devices. Use the SDL_WINDOW_ALLOW_HIGHDPI flag when creating your window to enable high-dpi support. When high-dpi support is enabled, SDL_GetWindowSize() and display mode sizes will still be in "screen coordinates" rather than pixels, but the window will have a much greater pixel density when the device supports it, and the SDL_GL_GetDrawableSize() or SDL_GetRendererOutputSize() functions (depending on whether raw OpenGL or the SDL_Render API is used) can be queried to determine the size in pixels of the drawable screen framebuffer. Some OpenGL ES functions such as glViewport expect sizes in pixels rather than sizes in screen coordinates. When doing 2D rendering with OpenGL ES, an orthographic projection matrix using the size in screen coordinates (SDL_GetWindowSize()) can be used in order to display content at the same scale no matter whether a Retina device is used or not. Notes -- Application events ============================================================================== On iOS the application goes through a fixed life cycle and you will get notifications of state changes via application events. When these events are delivered you must handle them in an event callback because the OS may not give you any processing time after the events are delivered. e.g. int HandleAppEvents(void *userdata, SDL_Event *event) { switch (event->type) { case SDL_APP_TERMINATING: /* Terminate the app. Shut everything down before returning from this function. */ return 0; case SDL_APP_LOWMEMORY: /* You will get this when your app is paused and iOS wants more memory. Release as much memory as possible. */ return 0; case SDL_APP_WILLENTERBACKGROUND: /* Prepare your app to go into the background. Stop loops, etc. This gets called when the user hits the home button, or gets a call. */ return 0; case SDL_APP_DIDENTERBACKGROUND: /* This will get called if the user accepted whatever sent your app to the background. If the user got a phone call and canceled it, you'll instead get an SDL_APP_DIDENTERFOREGROUND event and restart your loops. When you get this, you have 5 seconds to save all your state or the app will be terminated. Your app is NOT active at this point. */ return 0; case SDL_APP_WILLENTERFOREGROUND: /* This call happens when your app is coming back to the foreground. Restore all your state here. */ return 0; case SDL_APP_DIDENTERFOREGROUND: /* Restart your loops here. Your app is interactive and getting CPU again. */ return 0; default: /* No special processing, add it to the event queue */ return 1; } } int main(int argc, char *argv[]) { SDL_SetEventFilter(HandleAppEvents, NULL); ... run your main loop return 0; } Notes -- Accelerometer as Joystick ============================================================================== SDL for iPhone supports polling the built in accelerometer as a joystick device. For an example on how to do this, see the accelerometer.c in the demos directory. The main thing to note when using the accelerometer with SDL is that while the iPhone natively reports accelerometer as floating point values in units of g-force, SDL_JoystickGetAxis() reports joystick values as signed integers. Hence, in order to convert between the two, some clamping and scaling is necessary on the part of the iPhone SDL joystick driver. To convert SDL_JoystickGetAxis() reported values BACK to units of g-force, simply multiply the values by SDL_IPHONE_MAX_GFORCE / 0x7FFF. Notes -- OpenGL ES ============================================================================== Your SDL application for iOS uses OpenGL ES for video by default. OpenGL ES for iOS supports several display pixel formats, such as RGBA8 and RGB565, which provide a 32 bit and 16 bit color buffer respectively. By default, the implementation uses RGB565, but you may use RGBA8 by setting each color component to 8 bits in SDL_GL_SetAttribute(). If your application doesn't use OpenGL's depth buffer, you may find significant performance improvement by setting SDL_GL_DEPTH_SIZE to 0. Finally, if your application completely redraws the screen each frame, you may find significant performance improvement by setting the attribute SDL_GL_RETAINED_BACKING to 0. OpenGL ES on iOS doesn't use the traditional system-framebuffer setup provided in other operating systems. Special care must be taken because of this: - The drawable Renderbuffer must be bound to the GL_RENDERBUFFER binding point when SDL_GL_SwapWindow() is called. - The drawable Framebuffer Object must be bound while rendering to the screen and when SDL_GL_SwapWindow() is called. - If multisample antialiasing (MSAA) is used and glReadPixels is used on the screen, the drawable framebuffer must be resolved to the MSAA resolve framebuffer (via glBlitFramebuffer or glResolveMultisampleFramebufferAPPLE), and the MSAA resolve framebuffer must be bound to the GL_READ_FRAMEBUFFER binding point, before glReadPixels is called. The above objects can be obtained via SDL_GetWindowWMInfo() (in SDL_syswm.h). Notes -- Keyboard ============================================================================== The SDL keyboard API has been extended to support on-screen keyboards: void SDL_StartTextInput() -- enables text events and reveals the onscreen keyboard. void SDL_StopTextInput() -- disables text events and hides the onscreen keyboard. SDL_bool SDL_IsTextInputActive() -- returns whether or not text events are enabled (and the onscreen keyboard is visible) Notes -- Mouse ============================================================================== iOS now supports Bluetooth mice on iPad, but by default will provide the mouse input as touch. In order for SDL to see the real mouse events, you should set the key UIApplicationSupportsIndirectInputEvents to true in your Info.plist Notes -- Reading and Writing files ============================================================================== Each application installed on iPhone resides in a sandbox which includes its own Application Home directory. Your application may not access files outside this directory. Once your application is installed its directory tree looks like: MySDLApp Home/ MySDLApp.app Documents/ Library/ Preferences/ tmp/ When your SDL based iPhone application starts up, it sets the working directory to the main bundle (MySDLApp Home/MySDLApp.app), where your application resources are stored. You cannot write to this directory. Instead, I advise you to write document files to "../Documents/" and preferences to "../Library/Preferences". More information on this subject is available here: http://developer.apple.com/library/ios/#documentation/iPhone/Conceptual/iPhoneOSProgrammingGuide/Introduction/Introduction.html Notes -- xcFramework ============================================================================== The SDL.xcodeproj file now includes a target to build SDL2.xcframework. An xcframework is a new (Xcode 11) uber-framework which can handle any combination of processor type and target OS platform. In the past, iOS devices were always an ARM variant processor, and the simulator was always i386 or x86_64, and thus libraries could be combined into a single framework for both simulator and device. With the introduction of the Apple Silicon ARM-based machines, regular frameworks would collide as CPU type was no longer sufficient to differentiate the platform. So Apple created the new xcframework library package. The xcframework target builds into a Products directory alongside the SDL.xcodeproj file, as SDL2.xcframework. This can be brought in to any iOS project and will function properly for both simulator and device, no matter their CPUs. Note that Intel Macs cannot cross-compile for Apple Silicon Macs. If you need AS compatibility, perform this build on an Apple Silicon Mac. This target requires Xcode 11 or later. The target will simply fail to build if attempted on older Xcodes. In addition, on Apple platforms, main() cannot be in a dynamically loaded library. This means that iOS apps which used the statically-linked libSDL2.lib and now link with the xcframwork will need to define their own main() to call SDL_UIKitRunApp(), like this: #ifndef SDL_MAIN_HANDLED #ifdef main #undef main #endif int main(int argc, char *argv[]) { return SDL_UIKitRunApp(argc, argv, SDL_main); } #endif /* !SDL_MAIN_HANDLED */ Using an xcFramework is similar to using a regular framework. However, issues have been seen with the build system not seeing the headers in the xcFramework. To remedy this, add the path to the xcFramework in your app's target ==> Build Settings ==> Framework Search Paths and mark it recursive (this is critical). Also critical is to remove "*.framework" from Build Settings ==> Sub-Directories to Exclude in Recursive Searches. Clean the build folder, and on your next build the build system should be able to see any of these in your code, as expected: #include "SDL_main.h" #include #include Notes -- iPhone SDL limitations ============================================================================== Windows: Full-size, single window applications only. You cannot create multi-window SDL applications for iPhone OS. The application window will fill the display, though you have the option of turning on or off the menu-bar (pass SDL_CreateWindow() the flag SDL_WINDOW_BORDERLESS). Textures: The optimal texture formats on iOS are SDL_PIXELFORMAT_ABGR8888, SDL_PIXELFORMAT_ABGR8888, SDL_PIXELFORMAT_BGR888, and SDL_PIXELFORMAT_RGB24 pixel formats. Loading Shared Objects: This is disabled by default since it seems to break the terms of the iOS SDK agreement for iOS versions prior to iOS 8. It can be re-enabled in SDL_config_iphoneos.h. Notes -- CoreBluetooth.framework ============================================================================== SDL_JOYSTICK_HIDAPI is disabled by default. It can give you access to a lot more game controller devices, but it requires permission from the user before your app will be able to talk to the Bluetooth hardware. "Made For iOS" branded controllers do not need this as we don't have to speak to them directly with raw bluetooth, so many apps can live without this. You'll need to link with CoreBluetooth.framework and add something like this to your Info.plist: NSBluetoothPeripheralUsageDescription MyApp would like to remain connected to nearby bluetooth Game Controllers and Game Pads even when you're not using the app. Game Center ============================================================================== Game Center integration might require that you break up your main loop in order to yield control back to the system. In other words, instead of running an endless main loop, you run each frame in a callback function, using: int SDL_iPhoneSetAnimationCallback(SDL_Window * window, int interval, SDL_iOSAnimationCallback callback, void *callbackParam); This will set up the given function to be called back on the animation callback, and then you have to return from main() to let the Cocoa event loop run. e.g. extern "C" void ShowFrame(void*) { ... do event handling, frame logic and rendering ... } int main(int argc, char *argv[]) { ... initialize game ... #ifdef __IPHONEOS__ // Initialize the Game Center for scoring and matchmaking InitGameCenter(); // Set up the game to run in the window animation callback on iOS // so that Game Center and so forth works correctly. SDL_iPhoneSetAnimationCallback(window, 1, ShowFrame, NULL); #else while ( running ) { ShowFrame(0); DelayFrame(); } #endif return 0; } Deploying to older versions of iOS ============================================================================== SDL supports deploying to older versions of iOS than are supported by the latest version of Xcode, all the way back to iOS 8.0 In order to do that you need to download an older version of Xcode: https://developer.apple.com/download/more/?name=Xcode Open the package contents of the older Xcode and your newer version of Xcode and copy over the folders in Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/DeviceSupport Then open the file Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/SDKSettings.plist and add the versions of iOS you want to deploy to the key Root/DefaultProperties/DEPLOYMENT_TARGET_SUGGESTED_VALUES Open your project and set your deployment target to the desired version of iOS Finally, remove GameController from the list of frameworks linked by your application and edit the build settings for "Other Linker Flags" and add -weak_framework GameController --- ### Vs Sdl2 Docs README Kmsbsd (`vs/sdl2/docs/README-kmsbsd.md`) KMSDRM on *BSD ================================================== KMSDRM is supported on FreeBSD and OpenBSD. DragonFlyBSD works but requires being a root user. NetBSD isn't supported yet because the application will crash when creating the KMSDRM screen. WSCONS support has been brought back, but only as an input backend. It will not be brought back as a video backend to ease maintenance. OpenBSD note: Note that the video backend assumes that the user has read/write permissions to the /dev/drm* devices. SDL2 WSCONS input backend features =================================================== 1. It is keymap-aware; it will work properly with different keymaps. 2. It has mouse support. 3. Accent input is supported. 4. Compose keys are supported. 5. AltGr and Meta Shift keys work as intended. Partially working or no input on OpenBSD/NetBSD. ================================================== The WSCONS input backend needs read/write access to the /dev/wskbd* devices, without which it will not work properly. /dev/wsmouse must also be read/write accessible, otherwise mouse input will not work. Partially working or no input on FreeBSD. ================================================== The evdev devices are only accessible to the root user by default. Edit devfs rules to allow access to such devices. The /dev/kbd* devices are also only accessible to the root user by default. Edit devfs rules to allow access to such devices. --- ### Vs Sdl2 Docs README Linux (`vs/sdl2/docs/README-linux.md`) Linux ================================================================================ By default SDL will only link against glibc, the rest of the features will be enabled dynamically at runtime depending on the available features on the target system. So, for example if you built SDL with XRandR support and the target system does not have the XRandR libraries installed, it will be disabled at runtime, and you won't get a missing library error, at least with the default configuration parameters. Build Dependencies -------------------------------------------------------------------------------- Ubuntu 18.04, all available features enabled: sudo apt-get install build-essential git make autoconf automake libtool \ pkg-config cmake ninja-build gnome-desktop-testing libasound2-dev libpulse-dev \ libaudio-dev libjack-dev libsndio-dev libsamplerate0-dev libx11-dev libxext-dev \ libxrandr-dev libxcursor-dev libxfixes-dev libxi-dev libxss-dev libwayland-dev \ libxkbcommon-dev libdrm-dev libgbm-dev libgl1-mesa-dev libgles2-mesa-dev \ libegl1-mesa-dev libdbus-1-dev libibus-1.0-dev libudev-dev fcitx-libs-dev Ubuntu 22.04+ can also add `libpipewire-0.3-dev libdecor-0-dev` to that command line. Fedora 35, all available features enabled: sudo yum install gcc git-core make cmake autoconf automake libtool \ alsa-lib-devel pulseaudio-libs-devel nas-devel pipewire-devel \ libX11-devel libXext-devel libXrandr-devel libXcursor-devel libXfixes-devel \ libXi-devel libXScrnSaver-devel dbus-devel ibus-devel fcitx-devel \ systemd-devel mesa-libGL-devel libxkbcommon-devel mesa-libGLES-devel \ mesa-libEGL-devel vulkan-devel wayland-devel wayland-protocols-devel \ libdrm-devel mesa-libgbm-devel libusb-devel libdecor-devel \ libsamplerate-devel pipewire-jack-audio-connection-kit-devel \ NOTES: - This includes all the audio targets except arts and esd, because Ubuntu (and/or Debian) pulled their packages, but in theory SDL still supports them. The sndio audio target is also unavailable on Fedora. - libsamplerate0-dev lets SDL optionally link to libresamplerate at runtime for higher-quality audio resampling. SDL will work without it if the library is missing, so it's safe to build in support even if the end user doesn't have this library installed. - DirectFB isn't included because the configure script (currently) fails to find it at all. You can do "sudo apt-get install libdirectfb-dev" and fix the configure script to include DirectFB support. Send patches. :) Joystick does not work -------------------------------------------------------------------------------- If you compiled or are using a version of SDL with udev support (and you should!) there's a few issues that may cause SDL to fail to detect your joystick. To debug this, start by installing the evtest utility. On Ubuntu/Debian: sudo apt-get install evtest Then run: sudo evtest You'll hopefully see your joystick listed along with a name like "/dev/input/eventXX" Now run: cat /dev/input/event/XX If you get a permission error, you need to set a udev rule to change the mode of your device (see below) Also, try: sudo udevadm info --query=all --name=input/eventXX If you see a line stating ID_INPUT_JOYSTICK=1, great, if you don't see it, you need to set up an udev rule to force this variable. A combined rule for the Saitek Pro Flight Rudder Pedals to fix both issues looks like: SUBSYSTEM=="input", ATTRS{idProduct}=="0763", ATTRS{idVendor}=="06a3", MODE="0666", ENV{ID_INPUT_JOYSTICK}="1" SUBSYSTEM=="input", ATTRS{idProduct}=="0764", ATTRS{idVendor}=="06a3", MODE="0666", ENV{ID_INPUT_JOYSTICK}="1" You can set up similar rules for your device by changing the values listed in idProduct and idVendor. To obtain these values, try: sudo udevadm info -a --name=input/eventXX | grep idVendor sudo udevadm info -a --name=input/eventXX | grep idProduct If multiple values come up for each of these, the one you want is the first one of each. On other systems which ship with an older udev (such as CentOS), you may need to set up a rule such as: SUBSYSTEM=="input", ENV{ID_CLASS}=="joystick", ENV{ID_INPUT_JOYSTICK}="1" --- ### Vs Sdl2 Docs README Macos (`vs/sdl2/docs/README-macos.md`) # Mac OS X (aka macOS). These instructions are for people using Apple's Mac OS X (pronounced "ten"), which in newer versions is just referred to as "macOS". From the developer's point of view, macOS is a sort of hybrid Mac and Unix system, and you have the option of using either traditional command line tools or Apple's IDE Xcode. # Command Line Build To build SDL using the command line, use the standard configure and make process: ```bash mkdir build cd build ../configure make sudo make install ``` CMake is also known to work, although it continues to be a work in progress: ```bash mkdir build cd build cmake -DCMAKE_BUILD_TYPE=Release .. make sudo make install ``` You can also build SDL as a Universal library (a single binary for both 64-bit Intel and ARM architectures), by using the build-scripts/clang-fat.sh script. ```bash mkdir build cd build CC=$PWD/../build-scripts/clang-fat.sh ../configure make sudo make install ``` This script builds SDL with 10.9 ABI compatibility on 64-bit Intel and 11.0 ABI compatibility on ARM64 architectures. For best compatibility you should compile your application the same way. Please note that building SDL requires at least Xcode 6 and the 10.9 SDK. PowerPC support for macOS has been officially dropped as of SDL 2.0.2. 32-bit Intel and macOS 10.8 runtime support has been officially dropped as of SDL 2.24.0. To use the library once it's built, you essential have two possibilities: use the traditional autoconf/automake/make method, or use Xcode. # Caveats for using SDL with Mac OS X If you register your own NSApplicationDelegate (using [NSApp setDelegate:]), SDL will not register its own. This means that SDL will not terminate using SDL_Quit if it receives a termination request, it will terminate like a normal app, and it will not send a SDL_DROPFILE when you request to open a file with the app. To solve these issues, put the following code in your NSApplicationDelegate implementation: ```objc - (NSApplicationTerminateReply)applicationShouldTerminate:(NSApplication *)sender { if (SDL_GetEventState(SDL_QUIT) == SDL_ENABLE) { SDL_Event event; event.type = SDL_QUIT; SDL_PushEvent(&event); } return NSTerminateCancel; } - (BOOL)application:(NSApplication *)theApplication openFile:(NSString *)filename { if (SDL_GetEventState(SDL_DROPFILE) == SDL_ENABLE) { SDL_Event event; event.type = SDL_DROPFILE; event.drop.file = SDL_strdup([filename UTF8String]); return (SDL_PushEvent(&event) > 0); } return NO; } ``` # Using the Simple DirectMedia Layer with a traditional Makefile An existing autoconf/automake build system for your SDL app has good chances to work almost unchanged on macOS. However, to produce a "real" Mac binary that you can distribute to users, you need to put the generated binary into a so called "bundle", which is basically a fancy folder with a name like "MyCoolGame.app". To get this build automatically, add something like the following rule to your Makefile.am: ```make bundle_contents = APP_NAME.app/Contents APP_NAME_bundle: EXE_NAME mkdir -p $(bundle_contents)/MacOS mkdir -p $(bundle_contents)/Resources echo "APPL????" > $(bundle_contents)/PkgInfo $(INSTALL_PROGRAM) $< $(bundle_contents)/MacOS/ ``` You should replace `EXE_NAME` with the name of the executable. `APP_NAME` is what will be visible to the user in the Finder. Usually it will be the same as `EXE_NAME` but capitalized. E.g. if `EXE_NAME` is "testgame" then `APP_NAME` usually is "TestGame". You might also want to use `@PACKAGE@` to use the package name as specified in your configure.ac file. If your project builds more than one application, you will have to do a bit more. For each of your target applications, you need a separate rule. If you want the created bundles to be installed, you may want to add this rule to your Makefile.am: ```make install-exec-hook: APP_NAME_bundle rm -rf $(DESTDIR)$(prefix)/Applications/APP_NAME.app mkdir -p $(DESTDIR)$(prefix)/Applications/ cp -r $< /$(DESTDIR)$(prefix)Applications/ ``` This rule takes the Bundle created by the rule from step 3 and installs them into "$(DESTDIR)$(prefix)/Applications/". Again, if you want to install multiple applications, you will have to augment the make rule accordingly. But beware! That is only part of the story! With the above, you end up with a barebones .app bundle, which is double-clickable from the Finder. But there are some more things you should do before shipping your product... 1. The bundle right now probably is dynamically linked against SDL. That means that when you copy it to another computer, *it will not run*, unless you also install SDL on that other computer. A good solution for this dilemma is to static link against SDL. On OS X, you can achieve that by linking against the libraries listed by ```bash sdl-config --static-libs ``` instead of those listed by ```bash sdl-config --libs ``` Depending on how exactly SDL is integrated into your build systems, the way to achieve that varies, so I won't describe it here in detail 2. Add an 'Info.plist' to your application. That is a special XML file which contains some meta-information about your application (like some copyright information, the version of your app, the name of an optional icon file, and other things). Part of that information is displayed by the Finder when you click on the .app, or if you look at the "Get Info" window. More information about Info.plist files can be found on Apple's homepage. As a final remark, let me add that I use some of the techniques (and some variations of them) in [Exult](https://github.com/exult/exult) and [ScummVM](https://github.com/scummvm/scummvm); both are available in source on the net, so feel free to take a peek at them for inspiration! # Using the Simple DirectMedia Layer with Xcode These instructions are for using Apple's Xcode IDE to build SDL applications. ## First steps The first thing to do is to unpack the Xcode.tar.gz archive in the top level SDL directory (where the Xcode.tar.gz archive resides). Because Stuffit Expander will unpack the archive into a subdirectory, you should unpack the archive manually from the command line: ```bash cd [path_to_SDL_source] tar zxf Xcode.tar.gz ``` This will create a new folder called Xcode, which you can browse normally from the Finder. ## Building the Framework The SDL Library is packaged as a framework bundle, an organized relocatable folder hierarchy of executable code, interface headers, and additional resources. For practical purposes, you can think of a framework as a more user and system-friendly shared library, whose library file behaves more or less like a standard UNIX shared library. To build the framework, simply open the framework project and build it. By default, the framework bundle "SDL.framework" is installed in /Library/Frameworks. Therefore, the testers and project stationary expect it to be located there. However, it will function the same in any of the following locations: * ~/Library/Frameworks * /Local/Library/Frameworks * /System/Library/Frameworks ## Build Options There are two "Build Styles" (See the "Targets" tab) for SDL. "Deployment" should be used if you aren't tweaking the SDL library. "Development" should be used to debug SDL apps or the library itself. ## Building the Testers Open the SDLTest project and build away! ## Using the Project Stationary Copy the stationary to the indicated folders to access it from the "New Project" and "Add target" menus. What could be easier? ## Setting up a new project by hand Some of you won't want to use the Stationary so I'll give some tips: (this is accurate as of Xcode 12.5.) * Click "File" -> "New" -> "Project... * Choose "macOS" and then "App" from the "Application" section. * Fill out the options in the next window. User interface is "XIB" and Language is "Objective-C". * Remove "main.m" from your project * Remove "MainMenu.xib" from your project * Remove "AppDelegates.*" from your project * Add "\$(HOME)/Library/Frameworks/SDL.framework/Headers" to include path * Add "\$(HOME)/Library/Frameworks" to the frameworks search path * Add "-framework SDL -framework Foundation -framework AppKit" to "OTHER_LDFLAGS" * Add your files * Clean and build ## Building from command line Use `xcode-build` in the same directory as your .pbxproj file ## Running your app You can send command line args to your app by either invoking it from the command line (in *.app/Contents/MacOS) or by entering them in the Executables" panel of the target settings. # Implementation Notes Some things that may be of interest about how it all works... ## Working directory In SDL 1.2, the working directory of your SDL app is by default set to its parent, but this is no longer the case in SDL 2.0 and later. SDL2 does not change the working directory, which means it'll be whatever the command line prompt that launched the program was using, or if launched by double-clicking in the Finder, it will be "/", the _root of the filesystem_. Plan accordingly! You can use SDL_GetBasePath() to find where the program is running from and chdir() there directly. ## You have a Cocoa App! Your SDL app is essentially a Cocoa application. When your app starts up and the libraries finish loading, a Cocoa procedure is called, which sets up the working directory and calls your main() method. You are free to modify your Cocoa app with generally no consequence to SDL. You cannot, however, easily change the SDL window itself. Functionality may be added in the future to help this. # Bug reports Bugs are tracked at [the GitHub issue tracker](https://github.com/libsdl-org/SDL/issues/). Please feel free to report bugs there! --- ### Vs Sdl2 Docs README N3ds (`vs/sdl2/docs/README-n3ds.md`) # Nintendo 3DS SDL port for the Nintendo 3DS [Homebrew toolchain](https://devkitpro.org/) contributed by: - [Pierre Wendling](https://github.com/FtZPetruska) Credits to: - The awesome people who ported SDL to other homebrew platforms. - The Devkitpro team for making all the tools necessary to achieve this. ## Building To build for the Nintendo 3DS, make sure you have devkitARM and cmake installed and run: ```bash cmake -S. -Bbuild -DCMAKE_TOOLCHAIN_FILE="$DEVKITPRO/cmake/3DS.cmake" -DCMAKE_BUILD_TYPE=Release cmake --build build cmake --install build ``` ## Notes - Currently only software rendering is supported. - SDL2main should be used to ensure ROMFS is enabled. - By default, the extra L2 cache and higher clock speeds of the New 2/3DS lineup are enabled. If you wish to turn it off, use `osSetSpeedupEnable(false)` in your main function. - `SDL_GetBasePath` returns the romfs root instead of the executable's directory. - The Nintendo 3DS uses a cooperative threading model on a single core, meaning a thread will never yield unless done manually through the `SDL_Delay` functions, or blocking waits (`SDL_LockMutex`, `SDL_SemWait`, `SDL_CondWait`, `SDL_WaitThread`). To avoid starving other threads, `SDL_SemTryWait` and `SDL_SemWaitTimeout` will yield if they fail to acquire the semaphore, see https://github.com/libsdl-org/SDL/pull/6776 for more information. --- ### Vs Sdl2 Docs README Nacl (`vs/sdl2/docs/README-nacl.md`) Native Client ================================================================================ Requirements: * Native Client SDK (https://developer.chrome.com/native-client), (tested with Pepper version 33 or higher). The SDL backend for Chrome's Native Client has been tested only with the PNaCl toolchain, which generates binaries designed to run on ARM and x86_32/64 platforms. This does not mean it won't work with the other toolchains! ================================================================================ Building SDL for NaCl ================================================================================ Set up the right environment variables (see naclbuild.sh), then configure SDL with: configure --host=pnacl --prefix some/install/destination Then "make". As an example of how to create a deployable app a Makefile project is provided in test/nacl/Makefile, which includes some monkey patching of the common.mk file provided by NaCl, without which linking properly to SDL won't work (the search path can't be modified externally, so the linker won't find SDL's binaries unless you dump them into the SDK path, which is inconvenient). Also provided in test/nacl is the required support file, such as index.html, manifest.json, etc. SDL apps for NaCl run on a worker thread using the ppapi_simple infrastructure. This allows for blocking calls on all the relevant systems (OpenGL ES, filesystem), hiding the asynchronous nature of the browser behind the scenes...which is not the same as making it disappear! ================================================================================ Running tests ================================================================================ Due to the nature of NaCl programs, building and running SDL tests is not as straightforward as one would hope. The script naclbuild.sh in build-scripts automates the process and should serve as a guide for users of SDL trying to build their own applications. Basic usage: ./naclbuild.sh path/to/pepper/toolchain (i.e. ~/naclsdk/pepper_35) This will build testgles2.c by default. If you want to build a different test, for example testrendercopyex.c: SOURCES=~/sdl/SDL/test/testrendercopyex.c ./naclbuild.sh ~/naclsdk/pepper_35 Once the build finishes, you have to serve the contents with a web server (the script will give you instructions on how to do that with Python). ================================================================================ RWops and nacl_io ================================================================================ SDL_RWops work transparently with nacl_io. Two functions control the mount points: int mount(const char* source, const char* target, const char* filesystemtype, unsigned long mountflags, const void *data); int umount(const char *target); For convenience, SDL will by default mount an httpfs tree at / before calling the app's main function. Such setting can be overridden by calling: umount("/"); And then mounting a different filesystem at / It's important to consider that the asynchronous nature of file operations on a browser is hidden from the application, effectively providing the developer with a set of blocking file operations just like you get in a regular desktop environment, which eases the job of porting to Native Client, but also introduces a set of challenges of its own, in particular when big file sizes and slow connections are involved. For more information on how nacl_io and mount points work, see: https://developer.chrome.com/native-client/devguide/coding/nacl_io https://src.chromium.org/chrome/trunk/src/native_client_sdk/src/libraries/nacl_io/nacl_io.h To be able to save into the directory "/save/" (like backup of game) : mount("", "/save", "html5fs", 0, "type=PERSISTENT"); And add to manifest.json : "permissions": [ "unlimitedStorage" ] ================================================================================ TODO - Known Issues ================================================================================ * Testing of all systems with a real application (something other than SDL's tests) * Key events don't seem to work properly --- ### Vs Sdl2 Docs README Ngage (`vs/sdl2/docs/README-ngage.md`) Nokia N-Gage ============ SDL2 port for Symbian S60v1 and v2 with a main focus on the Nokia N-Gage (Classic and QD) by [Michael Fitzmayer](https://github.com/mupfdev). Compiling --------- SDL is part of the [N-Gage SDK.](https://github.com/ngagesdk) project. The library is included in the [toolchain](https://github.com/ngagesdk/ngage-toolchain) as a sub-module. A complete example project based on SDL2 can be found in the GitHub account of the SDK: [Wordle](https://github.com/ngagesdk/wordle). Current level of implementation ------------------------------- The video driver currently provides full screen video support with keyboard input. At the moment only the software renderer works. Audio is not yet implemented. Acknowledgements ---------------- Thanks to Hannu Viitala, Kimmo Kinnunen and Markus Mertama for the valuable insight into Symbian programming. Without the SDL 1.2 port which was specially developed for CDoom (Doom for the Nokia 9210), this adaptation would not have been possible. I would like to thank my friends [Razvan](https://twitter.com/bewarerazvan) and [Dan Whelan](https://danwhelan.ie/), for their continuous support. Without you and the [N-Gage community](https://discord.gg/dbUzqJ26vs), I would have lost my patience long ago. Last but not least, I would like to thank the development team of [EKA2L1](https://12z1.com/) (an experimental Symbian OS emulator). Your patience and support in troubleshooting helped me a lot. --- ### Vs Sdl2 Docs README Os2 (`vs/sdl2/docs/README-os2.md`) Simple DirectMedia Layer 2 for OS/2 & eComStation ================================================================================ SDL port for OS/2, authored by Andrey Vasilkin , 2016 OpenGL not supported by this port. Additional optional environment variables: SDL_AUDIO_SHARE Values: 0 or 1, default is 0 Initializes the device as shareable or exclusively acquired. SDL_VIDEODRIVER Values: DIVE or VMAN, default is DIVE Use video subsystem: Direct interface video extensions (DIVE) or Video Manager (VMAN). You may significantly increase video output speed with OS4 kernel and patched files vman.dll and dive.dll or with latest versions of ACPI support and video driver Panorama. Latest versions of OS/4 kernel: http://gus.biysk.ru/os4/ (Info: https://www.os2world.com/wiki/index.php/Phoenix_OS/4) Patched files vman.dll and dive.dll: http://gus.biysk.ru/os4/test/pached_dll/PATCHED_DLL.RAR Compiling: ---------- Open Watcom 1.9 or newer is tested. For the new Open Watcom V2 fork, see: https://github.com/open-watcom/ and https://open-watcom.github.io WATCOM environment variable must to be set to the Open Watcom install directory. To compile, run: wmake -f Makefile.os2 Installing: ----------- - eComStation: If you have previously installed SDL2, make a Backup copy of SDL2.dll located in D:\ecs\dll (where D: is disk on which installed eComStation). Stop all programs running with SDL2. Copy SDL2.dll to D:\ecs\dll - OS/2: Copy SDL2.dll to any directory on your LIBPATH. If you have a previous version installed, close all SDL2 applications before replacing the old copy. Also make sure that any other older versions of DLLs are removed from your system. Joysticks in SDL2: ------------------ The joystick code in SDL2 is a direct forward-port from the SDL-1.2 version. Here is the original documentation from SDL-1.2: The Joystick detection only works for standard joysticks (2 buttons, 2 axes and the like). Therefore, if you use a non-standard joystick, you should specify its features in the SDL_OS2_JOYSTICK environment variable in a batch file or CONFIG.SYS, so SDL applications can provide full capability to your device. The syntax is: SET SDL_OS2_JOYSTICK=[JOYSTICK_NAME] [AXES] [BUTTONS] [HATS] [BALLS] So, it you have a Gravis GamePad with 4 axes, 2 buttons, 2 hats and 0 balls, the line should be: SET SDL_OS2_JOYSTICK=Gravis_GamePad 4 2 2 0 If you want to add spaces in your joystick name, just surround it with quotes or double-quotes: SET SDL_OS2_JOYSTICK='Gravis GamePad' 4 2 2 0 or SET SDL_OS2_JOYSTICK="Gravis GamePad" 4 2 2 0 Note however that Balls and Hats are not supported under OS/2, and the value will be ignored... but it is wise to define these correctly because in the future those can be supported. Also the number of buttons is limited to 2 when using two joysticks, 4 when using one joystick with 4 axes, 6 when using a joystick with 3 axes and 8 when using a joystick with 2 axes. Notice however these are limitations of the Joystick Port hardware, not OS/2. --- ### Vs Sdl2 Docs README Pandora (`vs/sdl2/docs/README-pandora.md`) Pandora ===================================================================== ( http://openpandora.org/ ) - A pandora specific video driver was written to allow SDL 2.0 with OpenGL ES support to work on the pandora under the framebuffer. This driver do not have input support for now, so if you use it you will have to add your own control code. The video driver name is "pandora" so if you have problem running it from the framebuffer, try to set the following variable before starting your application : "export SDL_VIDEODRIVER=pandora" - OpenGL ES support was added to the x11 driver, so it's working like the normal x11 driver one with OpenGLX support, with SDL input event's etc.. David Carré (Cpasjuste) cpasjuste@gmail.com --- ### Vs Sdl2 Docs README Platforms (`vs/sdl2/docs/README-platforms.md`) Platforms ========= We maintain the list of supported platforms on our wiki now, and how to build and install SDL for those platforms: https://wiki.libsdl.org/Installation --- ### Vs Sdl2 Docs README Porting (`vs/sdl2/docs/README-porting.md`) Porting ======= * Porting To A New Platform The first thing you have to do when porting to a new platform, is look at include/SDL_platform.h and create an entry there for your operating system. The standard format is "__PLATFORM__", where PLATFORM is the name of the OS. Ideally SDL_platform.h will be able to auto-detect the system it's building on based on C preprocessor symbols. There are two basic ways of building SDL at the moment: 1. The "UNIX" way: ./configure; make; make install If you have a GNUish system, then you might try this. Edit configure.ac, take a look at the large section labelled: "Set up the configuration based on the host platform!" Add a section for your platform, and then re-run autogen.sh and build! 2. Using an IDE: If you're using an IDE or other non-configure build system, you'll probably want to create a custom SDL_config.h for your platform. Edit SDL_config.h, add a section for your platform, and create a custom SDL_config_{platform}.h, based on SDL_config_minimal.h and SDL_config.h.in Add the top level include directory to the header search path, and then add the following sources to the project: src/*.c src/atomic/*.c src/audio/*.c src/cpuinfo/*.c src/events/*.c src/file/*.c src/haptic/*.c src/joystick/*.c src/power/*.c src/render/*.c src/render/software/*.c src/stdlib/*.c src/thread/*.c src/timer/*.c src/video/*.c src/audio/disk/*.c src/audio/dummy/*.c src/filesystem/dummy/*.c src/video/dummy/*.c src/haptic/dummy/*.c src/joystick/dummy/*.c src/main/dummy/*.c src/thread/generic/*.c src/timer/dummy/*.c src/loadso/dummy/*.c Once you have a working library without any drivers, you can go back to each of the major subsystems and start implementing drivers for your platform. If you have any questions, don't hesitate to ask on the SDL mailing list: http://www.libsdl.org/mailing-list.php Enjoy! Sam Lantinga (slouken@libsdl.org) --- ### Vs Sdl2 Docs README Ps2 (`vs/sdl2/docs/README-ps2.md`) PS2 ====== SDL2 port for the Sony Playstation 2 contributed by: - Francisco Javier Trujillo Mata Credit to - The guys that ported SDL to PSP & Vita because I'm taking them as reference. - David G. F. for helping me with several issues and tests. ## Building To build SDL2 library for the PS2, make sure you have the latest PS2Dev status and run: ```bash cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=$PS2DEV/ps2sdk/ps2dev.cmake cmake --build build cmake --install build ``` ## Hints The PS2 port has a special Hint for having a dynamic VSYNC. The Hint is `SDL_HINT_PS2_DYNAMIC_VSYNC`. If you enabled the dynamic vsync having as well `SDL_RENDERER_PRESENTVSYNC` enabled, then if the app is not able to run at 60 FPS, automatically the `vsync` will be disabled having a better performance, instead of droping FPS to 30. ## Notes If you trying to debug a SDL app through [ps2client](https://github.com/ps2dev/ps2client) you need to avoid the IOP reset, otherwise you will lose the conection with your computer. So to avoid the reset of the IOP CPU, you need to call to the macro `SDL_PS2_SKIP_IOP_RESET();`. It could be something similar as: ```c ..... SDL_PS2_SKIP_IOP_RESET(); int main(int argc, char *argv[]) { ..... ``` For a release binary is recommendable to reset the IOP always. Remember to do a clean compilation everytime you enable or disable the `SDL_PS2_SKIP_IOP_RESET` otherwise the change won't be reflected. ## Getting PS2 Dev [Installing PS2 Dev](https://github.com/ps2dev/ps2dev) ## Running on PCSX2 Emulator [PCSX2](https://github.com/PCSX2/pcsx2) [More PCSX2 information](https://pcsx2.net/) ## To Do - PS2 Screen Keyboard - Dialogs - Others --- ### Vs Sdl2 Docs README Psp (`vs/sdl2/docs/README-psp.md`) PSP ====== SDL2 port for the Sony PSP contributed by: - Captian Lex - Francisco Javier Trujillo Mata - Wouter Wijsman Credit to Marcus R.Brown,Jim Paris,Matthew H for the original SDL 1.2 for PSP Geecko for his PSP GU lib "Glib2d" ## Building To build SDL2 library for the PSP, make sure you have the latest PSPDev status and run: ```bash cmake -S. -Bbuild -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE=$PSPDEV/psp/share/pspdev.cmake cmake --build build cmake --install build ``` ## Getting PSP Dev [Installing PSP Dev](https://github.com/pspdev/pspdev) ## Running on PPSSPP Emulator [PPSSPP](https://github.com/hrydgard/ppsspp) [Build Instructions](https://github.com/hrydgard/ppsspp/wiki/Build-instructions) ## Compiling a HelloWorld [PSP Hello World](https://psp-dev.org/doku.php?id=tutorial:hello_world) ## To Do - PSP Screen Keyboard - Dialogs --- ### Vs Sdl2 Docs README Raspberrypi (`vs/sdl2/docs/README-raspberrypi.md`) Raspberry Pi ============ Requirements: Raspbian (other Linux distros may work as well). Features -------- * Works without X11 * Hardware accelerated OpenGL ES 2.x * Sound via ALSA * Input (mouse/keyboard/joystick) via EVDEV * Hotplugging of input devices via UDEV Raspbian Build Dependencies --------------------------- sudo apt-get install libudev-dev libasound2-dev libdbus-1-dev You also need the VideoCore binary stuff that ships in /opt/vc for EGL and OpenGL ES 2.x, it usually comes pre-installed, but in any case: sudo apt-get install libraspberrypi0 libraspberrypi-bin libraspberrypi-dev NEON ---- If your Pi has NEON support, make sure you add -mfpu=neon to your CFLAGS so that SDL will select some otherwise-disabled highly-optimized code. The original Pi units don't have NEON, the Pi2 probably does, and the Pi3 definitely does. Cross compiling from x86 Linux ------------------------------ To cross compile SDL for Raspbian from your desktop machine, you'll need a Raspbian system root and the cross compilation tools. We'll assume these tools will be placed in /opt/rpi-tools sudo git clone --depth 1 https://github.com/raspberrypi/tools /opt/rpi-tools You'll also need a Raspbian binary image. Get it from: http://downloads.raspberrypi.org/raspbian_latest After unzipping, you'll get file with a name like: "-wheezy-raspbian.img" Let's assume the sysroot will be built in /opt/rpi-sysroot. export SYSROOT=/opt/rpi-sysroot sudo kpartx -a -v .img sudo mount -o loop /dev/mapper/loop0p2 /mnt sudo cp -r /mnt $SYSROOT sudo apt-get install qemu binfmt-support qemu-user-static sudo cp /usr/bin/qemu-arm-static $SYSROOT/usr/bin sudo mount --bind /dev $SYSROOT/dev sudo mount --bind /proc $SYSROOT/proc sudo mount --bind /sys $SYSROOT/sys Now, before chrooting into the ARM sysroot, you'll need to apply a workaround, edit $SYSROOT/etc/ld.so.preload and comment out all lines in it. sudo chroot $SYSROOT apt-get install libudev-dev libasound2-dev libdbus-1-dev libraspberrypi0 libraspberrypi-bin libraspberrypi-dev libx11-dev libxext-dev libxrandr-dev libxcursor-dev libxi-dev libxss-dev exit sudo umount $SYSROOT/dev sudo umount $SYSROOT/proc sudo umount $SYSROOT/sys sudo umount /mnt There's one more fix required, as the libdl.so symlink uses an absolute path which doesn't quite work in our setup. sudo rm -rf $SYSROOT/usr/lib/arm-linux-gnueabihf/libdl.so sudo ln -s ../../../lib/arm-linux-gnueabihf/libdl.so.2 $SYSROOT/usr/lib/arm-linux-gnueabihf/libdl.so The final step is compiling SDL itself. export CC="/opt/rpi-tools/arm-bcm2708/gcc-linaro-arm-linux-gnueabihf-raspbian/bin/arm-linux-gnueabihf-gcc --sysroot=$SYSROOT -I$SYSROOT/opt/vc/include -I$SYSROOT/usr/include -I$SYSROOT/opt/vc/include/interface/vcos/pthreads -I$SYSROOT/opt/vc/include/interface/vmcs_host/linux" cd mkdir -p build;cd build LDFLAGS="-L$SYSROOT/opt/vc/lib" ../configure --with-sysroot=$SYSROOT --host=arm-raspberry-linux-gnueabihf --prefix=$PWD/rpi-sdl2-installed --disable-pulseaudio --disable-esd make make install To be able to deploy this to /usr/local in the Raspbian system you need to fix up a few paths: perl -w -pi -e "s#$PWD/rpi-sdl2-installed#/usr/local#g;" ./rpi-sdl2-installed/lib/libSDL2.la ./rpi-sdl2-installed/lib/pkgconfig/sdl2.pc ./rpi-sdl2-installed/bin/sdl2-config Apps don't work or poor video/audio performance ----------------------------------------------- If you get sound problems, buffer underruns, etc, run "sudo rpi-update" to update the RPi's firmware. Note that doing so will fix these problems, but it will also render the CMA - Dynamic Memory Split functionality useless. Also, by default the Raspbian distro configures the GPU RAM at 64MB, this is too low in general, specially if a 1080p TV is hooked up. See here how to configure this setting: http://elinux.org/RPiconfig Using a fixed gpu_mem=128 is the best option (specially if you updated the firmware, using CMA probably won't work, at least it's the current case). No input -------- Make sure you belong to the "input" group. sudo usermod -aG input `whoami` No HDMI Audio ------------- If you notice that ALSA works but there's no audio over HDMI, try adding: hdmi_drive=2 to your config.txt file and reboot. Reference: http://www.raspberrypi.org/phpBB3/viewtopic.php?t=5062 Text Input API support ---------------------- The Text Input API is supported, with translation of scan codes done via the kernel symbol tables. For this to work, SDL needs access to a valid console. If you notice there's no SDL_TEXTINPUT message being emitted, double check that your app has read access to one of the following: * /proc/self/fd/0 * /dev/tty * /dev/tty[0...6] * /dev/vc/0 * /dev/console This is usually not a problem if you run from the physical terminal (as opposed to running from a pseudo terminal, such as via SSH). If running from a PTS, a quick workaround is to run your app as root or add yourself to the tty group, then re-login to the system. sudo usermod -aG tty `whoami` The keyboard layout used by SDL is the same as the one the kernel uses. To configure the layout on Raspbian: sudo dpkg-reconfigure keyboard-configuration To configure the locale, which controls which keys are interpreted as letters, this determining the CAPS LOCK behavior: sudo dpkg-reconfigure locales OpenGL problems --------------- If you have desktop OpenGL headers installed at build time in your RPi or cross compilation environment, support for it will be built in. However, the chipset does not actually have support for it, which causes issues in certain SDL apps since the presence of OpenGL support supersedes the ES/ES2 variants. The workaround is to disable OpenGL at configuration time: ./configure --disable-video-opengl Or if the application uses the Render functions, you can use the SDL_RENDER_DRIVER environment variable: export SDL_RENDER_DRIVER=opengles2 Notes ----- * When launching apps remotely (via SSH), SDL can prevent local keystrokes from leaking into the console only if it has root privileges. Launching apps locally does not suffer from this issue. --- ### Vs Sdl2 Docs README Riscos (`vs/sdl2/docs/README-riscos.md`) RISC OS ======= Requirements: * RISC OS 3.5 or later. * [SharedUnixLibrary](http://www.riscos.info/packages/LibraryDetails.html#SharedUnixLibraryarm). * [DigitalRenderer](http://www.riscos.info/packages/LibraryDetails.html#DRendererarm), for audio support. * [Iconv](http://www.netsurf-browser.org/projects/iconv/), for `SDL_iconv` and related functions. Compiling: ---------- Currently, SDL2 for RISC OS only supports compiling with GCCSDK under Linux. Both the autoconf and CMake build systems are supported. The following commands can be used to build SDL2 for RISC OS using autoconf: ./configure --host=arm-unknown-riscos --prefix=$GCCSDK_INSTALL_ENV --disable-gcc-atomics make make install The following commands can be used to build SDL2 for RISC OS using CMake: cmake -Bbuild-riscos -DCMAKE_TOOLCHAIN_FILE=$GCCSDK_INSTALL_ENV/toolchain-riscos.cmake -DRISCOS=ON -DCMAKE_INSTALL_PREFIX=$GCCSDK_INSTALL_ENV -DCMAKE_BUILD_TYPE=Release -DSDL_GCC_ATOMICS=OFF cmake --build build-riscos cmake --build build-riscos --target install Current level of implementation ------------------------------- The video driver currently provides full screen video support with keyboard and mouse input. Windowed mode is not yet supported, but is planned in the future. Only software rendering is supported. The filesystem APIs return either Unix-style paths or RISC OS-style paths based on the value of the `__riscosify_control` symbol, as is standard for UnixLib functions. The audio, loadso, thread and timer APIs are currently provided by UnixLib. GCC atomics are currently broken on some platforms, meaning it's currently necessary to compile with `--disable-gcc-atomics` using autotools or `-DSDL_GCC_ATOMICS=OFF` using CMake. The joystick, locale and power APIs are not yet implemented. --- ### Vs Sdl2 Docs README Touch (`vs/sdl2/docs/README-touch.md`) Touch =========================================================================== System Specific Notes =========================================================================== Linux: The linux touch system is currently based off event streams, and proc/bus/devices. The active user must be given permissions to read /dev/input/TOUCHDEVICE, where TOUCHDEVICE is the event stream for your device. Currently only Wacom tablets are supported. If you have an unsupported tablet contact me at jim.tla+sdl_touch@gmail.com and I will help you get support for it. Mac: The Mac and iPhone APIs are pretty. If your touch device supports them then you'll be fine. If it doesn't, then there isn't much we can do. iPhone: Works out of box. Windows: Unfortunately there is no windows support as of yet. Support for Windows 7 is planned, but we currently have no way to test. If you have a Windows 7 WM_TOUCH supported device, and are willing to help test please contact me at jim.tla+sdl_touch@gmail.com =========================================================================== Events =========================================================================== SDL_FINGERDOWN: Sent when a finger (or stylus) is placed on a touch device. Fields: * event.tfinger.touchId - the Id of the touch device. * event.tfinger.fingerId - the Id of the finger which just went down. * event.tfinger.x - the x coordinate of the touch (0..1) * event.tfinger.y - the y coordinate of the touch (0..1) * event.tfinger.pressure - the pressure of the touch (0..1) SDL_FINGERMOTION: Sent when a finger (or stylus) is moved on the touch device. Fields: Same as SDL_FINGERDOWN but with additional: * event.tfinger.dx - change in x coordinate during this motion event. * event.tfinger.dy - change in y coordinate during this motion event. SDL_FINGERUP: Sent when a finger (or stylus) is lifted from the touch device. Fields: Same as SDL_FINGERDOWN. =========================================================================== Functions =========================================================================== SDL provides the ability to access the underlying SDL_Finger structures. These structures should _never_ be modified. The following functions are included from SDL_touch.h To get a SDL_TouchID call SDL_GetTouchDevice(int index). This returns a SDL_TouchID. IMPORTANT: If the touch has been removed, or there is no touch with the given index, SDL_GetTouchDevice() will return 0. Be sure to check for this! The number of touch devices can be queried with SDL_GetNumTouchDevices(). A SDL_TouchID may be used to get pointers to SDL_Finger. SDL_GetNumTouchFingers(touchID) may be used to get the number of fingers currently down on the device. The most common reason to access SDL_Finger is to query the fingers outside the event. In most cases accessing the fingers is using the event. This would be accomplished by code like the following: float x = event.tfinger.x; float y = event.tfinger.y; To get a SDL_Finger, call SDL_GetTouchFinger(SDL_TouchID touchID, int index), where touchID is a SDL_TouchID, and index is the requested finger. This returns a SDL_Finger *, or NULL if the finger does not exist, or has been removed. A SDL_Finger is guaranteed to be persistent for the duration of a touch, but it will be de-allocated as soon as the finger is removed. This occurs when the SDL_FINGERUP event is _added_ to the event queue, and thus _before_ the SDL_FINGERUP event is polled. As a result, be very careful to check for NULL return values. A SDL_Finger has the following fields: * x, y: The current coordinates of the touch. * pressure: The pressure of the touch. =========================================================================== Notes =========================================================================== For a complete example see test/testgesture.c Please direct questions/comments to: jim.tla+sdl_touch@gmail.com (original author, API was changed since) --- ### Vs Sdl2 Docs README Versions (`vs/sdl2/docs/README-versions.md`) # Versioning ## Since 2.23.0 SDL follows an "odd/even" versioning policy, similar to GLib, GTK, Flatpak and older versions of the Linux kernel: * The major version (first part) increases when backwards compatibility is broken, which will happen infrequently. * If the minor version (second part) is divisible by 2 (for example 2.24.x, 2.26.x), this indicates a version of SDL that is believed to be stable and suitable for production use. * In stable releases, the patchlevel or micro version (third part) indicates bugfix releases. Bugfix releases should not add or remove ABI, so the ".0" release (for example 2.24.0) should be forwards-compatible with all the bugfix releases from the same cycle (for example 2.24.1). * The minor version increases when new API or ABI is added, or when other significant changes are made. Newer minor versions are backwards-compatible, but not fully forwards-compatible. For example, programs built against SDL 2.24.x should work fine with SDL 2.26.x, but programs built against SDL 2.26.x will not necessarily work with 2.24.x. * If the minor version (second part) is not divisible by 2 (for example 2.23.x, 2.25.x), this indicates a development prerelease of SDL that is not suitable for stable software distributions. Use with caution. * The patchlevel or micro version (third part) increases with each prerelease. * Each prerelease might add new API and/or ABI. * Prereleases are backwards-compatible with older stable branches. For example, 2.25.x will be backwards-compatible with 2.24.x. * Prereleases are not guaranteed to be backwards-compatible with each other. For example, new API or ABI added in 2.25.1 might be removed or changed in 2.25.2. If this would be a problem for you, please do not use prereleases. * Only upgrade to a prerelease if you can guarantee that you will promptly upgrade to the stable release that follows it. For example, do not upgrade to 2.23.x unless you will be able to upgrade to 2.24.0 when it becomes available. * Software distributions that have a freeze policy (in particular Linux distributions with a release cycle, such as Debian and Fedora) should usually only package stable releases, and not prereleases. ## Before 2.23.0 Older versions of SDL followed a similar policy, but instead of the odd/even rule applying to the minor version, it applied to the patchlevel (micro version, third part). For example, 2.0.22 was a stable release and 2.0.21 was a prerelease. --- ### Vs Sdl2 Docs README Visualc (`vs/sdl2/docs/README-visualc.md`) Using SDL with Microsoft Visual C++ =================================== ### by Lion Kimbro with additions by James Turk You can either use the precompiled libraries from the [SDL](https://www.libsdl.org/download.php) web site, or you can build SDL yourself. ### Building SDL 0. To build SDL, your machine must, at a minimum, have the DirectX9.0c SDK installed. It may or may not be retrievable from the [Microsoft](https://www.microsoft.com) website, so you might need to locate it [online](https://duckduckgo.com/?q=directx9.0c+sdk+download&t=h_&ia=web). _Editor's note: I've been able to successfully build SDL using Visual Studio 2019 **without** the DX9.0c SDK_ 1. Open the Visual Studio solution file at `./VisualC/SDL.sln`. 2. Your IDE will likely prompt you to upgrade this solution file to whatever later version of the IDE you're using. In the `Retarget Projects` dialog, all of the affected project files should be checked allowing you to use the latest `Windows SDK Version` you have installed, along with the `Platform Toolset`. If you choose *NOT* to upgrade to use the latest `Windows SDK Version` or `Platform Toolset`, then you'll need the `Visual Studio 2010 Platform Toolset`. 3. Build the `.dll` and `.lib` files by right clicking on each project in turn (Projects are listed in the _Workspace_ panel in the _FileView_ tab), and selecting `Build`. You may get a few warnings, but you should not get any errors. Later, we will refer to the following `.lib` and `.dll` files that have just been generated: - `./VisualC/Win32/Debug/SDL2.dll` or `./VisualC/Win32/Release/SDL2.dll` - `./VisualC/Win32/Debug/SDL2.lib` or `./VisualC/Win32/Release/SDL2.lib` - `./VisualC/Win32/Debug/SDL2main.lib` or `./VisualC/Win32/Release/SDL2main.lib` _Note for the `x64` versions, just replace `Win32` in the path with `x64`_ ### Creating a Project with SDL - Create a project as a `Win32 Application`. - Create a C++ file for your project. - Set the C runtime to `Multi-threaded DLL` in the menu: `Project|Settings|C/C++ tab|Code Generation|Runtime Library `. - Add the SDL `include` directory to your list of includes in the menu: `Project|Settings|C/C++ tab|Preprocessor|Additional include directories ` *VC7 Specific: Instead of doing this, I find it easier to add the include and library directories to the list that VC7 keeps. Do this by selecting Tools|Options|Projects|VC++ Directories and under the "Show Directories For:" dropbox select "Include Files", and click the "New Directory Icon" and add the [SDLROOT]\\include directory (e.g. If you installed to c:\\SDL\\ add c:\\SDL\\include). Proceed to change the dropbox selection to "Library Files" and add [SDLROOT]\\lib.* The "include directory" I am referring to is the `./include` folder. Now we're going to use the files that we had created earlier in the *Build SDL* step. Copy the following file into your Project directory: - `SDL2.dll` Add the following files to your project (It is not necessary to copy them to your project directory): - `SDL2.lib` - `SDL2main.lib` To add them to your project, right click on your project, and select `Add files to project`. **Instead of adding the files to your project, it is more desirable to add them to the linker options: Project|Properties|Linker|Command Line and type the names of the libraries to link with in the "Additional Options:" box. Note: This must be done for each build configuration (e.g. Release,Debug).** ### Hello SDL2 Here's a sample SDL snippet to verify everything is setup in your IDE: ``` #include "SDL.h" int main( int argc, char* argv[] ) { const int WIDTH = 640; const int HEIGHT = 480; SDL_Window* window = NULL; SDL_Renderer* renderer = NULL; SDL_Init(SDL_INIT_VIDEO); window = SDL_CreateWindow("SDL2 Test", SDL_WINDOWPOS_UNDEFINED, SDL_WINDOWPOS_UNDEFINED, WIDTH, HEIGHT, SDL_WINDOW_SHOWN); renderer = SDL_CreateRenderer(window, -1, SDL_RENDERER_ACCELERATED | SDL_RENDERER_PRESENTVSYNC); SDL_DestroyRenderer(renderer); SDL_DestroyWindow(window); SDL_Quit(); return 0; } ``` ### That's it! I hope that this document has helped you get through the most difficult part of using the SDL: installing it. Suggestions for improvements should be posted to the [Github Issues](https://github.com/libsdl-org/SDL/issues). ### Credits Thanks to [Paulus Esterhazy](mailto:pesterhazy@gmx.net), for the work on VC++ port. This document was originally called "VisualC.txt", and was written by [Sam Lantinga](mailto:slouken@libsdl.org). Later, it was converted to HTML and expanded into the document that you see today by [Lion Kimbro](mailto:snowlion@sprynet.com). Minor Fixes and Visual C++ 7 Information (In Green) was added by [James Turk](mailto:james@conceptofzero.net) --- ### Vs Sdl2 Docs README Vita (`vs/sdl2/docs/README-vita.md`) PS Vita ======= SDL port for the Sony Playstation Vita and Sony Playstation TV Credit to * xerpi, cpasjuste and rsn8887 for initial (vita2d) port * vitasdk/dolcesdk devs * CBPS discord (Namely Graphene and SonicMastr) Building -------- To build for the PSVita, make sure you have vitasdk and cmake installed and run: ``` cmake -S. -Bbuild -DCMAKE_TOOLCHAIN_FILE=${VITASDK}/share/vita.toolchain.cmake -DCMAKE_BUILD_TYPE=Release cmake --build build cmake --install build ``` Notes ----- * gles1/gles2 support and renderers are disabled by default and can be enabled by configuring with `-DVIDEO_VITA_PVR=ON` These renderers support 720p and 1080i resolutions. These can be specified with: `SDL_setenv("VITA_RESOLUTION", "720", 1);` and `SDL_setenv("VITA_RESOLUTION", "1080", 1);` * Desktop GL 1.X and 2.X support and renderers are also disabled by default and also can be enabled with `-DVIDEO_VITA_PVR=ON` as long as gl4es4vita is present in your SDK. They support the same resolutions as the gles1/gles2 backends and require specifying `SDL_setenv("VITA_PVR_OGL", "1", 1);` anytime before video subsystem initialization. * gles2 support via PIB is disabled by default and can be enabled by configuring with `-DVIDEO_VITA_PIB=ON` * By default SDL emits mouse events for touch events on every touchscreen. Vita has two touchscreens, so it's recommended to use `SDL_SetHint(SDL_HINT_TOUCH_MOUSE_EVENTS, "0");` and handle touch events instead. Individual touchscreens can be disabled with: `SDL_setenv("VITA_DISABLE_TOUCH_FRONT", "1", 1);` and `SDL_setenv("VITA_DISABLE_TOUCH_BACK", "1", 1);` * Support for L2/R2/R3/R3 buttons, haptic feedback and gamepad led only available on PSTV, or when using external ds4 gamepad on vita. --- ### Vs Sdl2 Docs README Wince (`vs/sdl2/docs/README-wince.md`) WinCE ===== Windows CE is no longer supported by SDL. We have left the CE support in SDL 1.2 for those that must have it, and we have support for Windows Phone 8 and WinRT in SDL2, as of SDL 2.0.3. --ryan. --- ### Vs Sdl2 Docs README Windows (`vs/sdl2/docs/README-windows.md`) # Windows ## LLVM and Intel C++ compiler support SDL will build with the Visual Studio project files with LLVM-based compilers, such as the Intel oneAPI C++ compiler, but you'll have to manually add the "-msse3" command line option to at least the SDL_audiocvt.c source file, and possibly others. This may not be necessary if you build SDL with CMake instead of the included Visual Studio solution. Details are here: https://github.com/libsdl-org/SDL/issues/5186 ## OpenGL ES 2.x support SDL has support for OpenGL ES 2.x under Windows via two alternative implementations. The most straightforward method consists in running your app in a system with a graphic card paired with a relatively recent (as of November of 2013) driver which supports the WGL_EXT_create_context_es2_profile extension. Vendors known to ship said extension on Windows currently include nVidia and Intel. The other method involves using the [ANGLE library](https://code.google.com/p/angleproject/). If an OpenGL ES 2.x context is requested and no WGL_EXT_create_context_es2_profile extension is found, SDL will try to load the libEGL.dll library provided by ANGLE. To obtain the ANGLE binaries, you can either compile from source from https://chromium.googlesource.com/angle/angle or copy the relevant binaries from a recent Chrome/Chromium install for Windows. The files you need are: - libEGL.dll - libGLESv2.dll - d3dcompiler_46.dll (supports Windows Vista or later, better shader compiler) *or* d3dcompiler_43.dll (supports Windows XP or later) If you compile ANGLE from source, you can configure it so it does not need the d3dcompiler_* DLL at all (for details on this, see their documentation). However, by default SDL will try to preload the d3dcompiler_46.dll to comply with ANGLE's requirements. If you wish SDL to preload d3dcompiler_43.dll (to support Windows XP) or to skip this step at all, you can use the SDL_HINT_VIDEO_WIN_D3DCOMPILER hint (see SDL_hints.h for more details). Known Bugs: - SDL_GL_SetSwapInterval is currently a no op when using ANGLE. It appears that there's a bug in the library which prevents the window contents from refreshing if this is set to anything other than the default value. ## Vulkan Surface Support Support for creating Vulkan surfaces is configured on by default. To disable it change the value of `SDL_VIDEO_VULKAN` to 0 in `SDL_config_windows.h`. You must install the [Vulkan SDK](https://www.lunarg.com/vulkan-sdk/) in order to use Vulkan graphics in your application. --- ### Vs Sdl2 Docs README Winrt (`vs/sdl2/docs/README-winrt.md`) WinRT ===== This port allows SDL applications to run on Microsoft's platforms that require use of "Windows Runtime", aka. "WinRT", APIs. Microsoft may, in some cases, refer to them as either "Windows Store", or for Windows 10, "UWP" apps. In the past, SDL has supported Windows RT 8.x, Windows Phone, etc, but in modern times this port is focused on UWP apps, which run on Windows 10, and modern Xbox consoles. Requirements ------------ * Microsoft Visual C++ (aka Visual Studio) 2019. - Free, "Community" or "Express" editions may be used, so long as they include support for either "Windows Store" or "Windows Phone" apps. "Express" versions marked as supporting "Windows Desktop" development typically do not include support for creating WinRT apps, to note. (The "Community" editions of Visual C++ do, however, support both desktop/Win32 and WinRT development). * A valid Microsoft account - This requirement is not imposed by SDL, but rather by Microsoft's Visual C++ toolchain. This is required to launch or debug apps. Status ------ Here is a rough list of what works, and what doesn't: * What works: * compilation via Visual C++ 2019. * compile-time platform detection for SDL programs. The C/C++ #define, `__WINRT__`, will be set to 1 (by SDL) when compiling for WinRT. * GPU-accelerated 2D rendering, via SDL_Renderer. * OpenGL ES 2, via the ANGLE library (included separately from SDL) * software rendering, via either SDL_Surface (optionally in conjunction with SDL_GetWindowSurface() and SDL_UpdateWindowSurface()) or via the SDL_Renderer APIs * threads * timers (via SDL_GetTicks(), SDL_AddTimer(), SDL_GetPerformanceCounter(), SDL_GetPerformanceFrequency(), etc.) * file I/O via SDL_RWops * mouse input (unsupported on Windows Phone) * audio, via SDL's WASAPI backend (if you want to record, your app must have "Microphone" capabilities enabled in its manifest, and the user must not have blocked access. Otherwise, capture devices will fail to work, presenting as a device disconnect shortly after opening it.) * .DLL file loading. Libraries *MUST* be packaged inside applications. Loading anything outside of the app is not supported. * system path retrieval via SDL's filesystem APIs * game controllers. Support is provided via the SDL_Joystick and SDL_GameController APIs, and is backed by Microsoft's XInput API. Please note, however, that Windows limits game-controller support in UWP apps to, "Xbox compatible controllers" (many controllers that work in Win32 apps, do not work in UWP, due to restrictions in UWP itself.) * multi-touch input * app events. SDL_APP_WILLENTER* and SDL_APP_DIDENTER* events get sent out as appropriate. * window events * using Direct3D 11.x APIs outside of SDL. Non-XAML / Direct3D-only apps can choose to render content directly via Direct3D, using SDL to manage the internal WinRT window, as well as input and audio. (Use SDL_GetWindowWMInfo() to get the WinRT 'CoreWindow', and pass it into IDXGIFactory2::CreateSwapChainForCoreWindow() as appropriate.) * What partially works: * keyboard input. Most of WinRT's documented virtual keys are supported, as well as many keys with documented hardware scancodes. Converting SDL_Scancodes to or from SDL_Keycodes may not work, due to missing APIs (MapVirtualKey()) in Microsoft's Windows Store / UWP APIs. * SDLmain. WinRT uses a different signature for each app's main() function. SDL-based apps that use this port must compile in SDL_winrt_main_NonXAML.cpp (in `SDL\src\main\winrt\`) directly in order for their C-style main() functions to be called. * What doesn't work: * compilation with anything other than Visual C++ * programmatically-created custom cursors. These don't appear to be supported by WinRT. Different OS-provided cursors can, however, be created via SDL_CreateSystemCursor() (unsupported on Windows Phone) * SDL_WarpMouseInWindow() or SDL_WarpMouseGlobal(). This are not currently supported by WinRT itself. * joysticks and game controllers that either are not supported by Microsoft's XInput API, or are not supported within UWP apps (many controllers that work in Win32, do not work in UWP, due to restrictions in UWP itself). * turning off VSync when rendering on Windows Phone. Attempts to turn VSync off on Windows Phone result either in Direct3D not drawing anything, or it forcing VSync back on. As such, SDL_RENDERER_PRESENTVSYNC will always get turned-on on Windows Phone. This limitation is not present in non-Phone WinRT (such as Windows 8.x), where turning off VSync appears to work. * probably anything else that's not listed as supported Upgrade Notes ------------- #### SDL_GetPrefPath() usage when upgrading WinRT apps from SDL 2.0.3 SDL 2.0.4 fixes two bugs found in the WinRT version of SDL_GetPrefPath(). The fixes may affect older, SDL 2.0.3-based apps' save data. Please note that these changes only apply to SDL-based WinRT apps, and not to apps for any other platform. 1. SDL_GetPrefPath() would return an invalid path, one in which the path's directory had not been created. Attempts to create files there (via fopen(), for example), would fail, unless that directory was explicitly created beforehand. 2. SDL_GetPrefPath(), for non-WinPhone-based apps, would return a path inside a WinRT 'Roaming' folder, the contents of which get automatically synchronized across multiple devices. This process can occur while an application runs, and can cause existing save-data to be overwritten at unexpected times, with data from other devices. (Windows Phone apps written with SDL 2.0.3 did not utilize a Roaming folder, due to API restrictions in Windows Phone 8.0). SDL_GetPrefPath(), starting with SDL 2.0.4, addresses these by: 1. making sure that SDL_GetPrefPath() returns a directory in which data can be written to immediately, without first needing to create directories. 2. basing SDL_GetPrefPath() off of a different, non-Roaming folder, the contents of which do not automatically get synchronized across devices (and which require less work to use safely, in terms of data integrity). Apps that wish to get their Roaming folder's path can do so either by using SDL_WinRTGetFSPathUTF8(), SDL_WinRTGetFSPathUNICODE() (which returns a UCS-2/wide-char string), or directly through the WinRT class, Windows.Storage.ApplicationData. Setup, High-Level Steps ----------------------- The steps for setting up a project for an SDL/WinRT app looks like the following, at a high-level: 1. create a new Visual C++ project using Microsoft's template for a, "Direct3D App". 2. remove most of the files from the project. 3. make your app's project directly reference SDL/WinRT's own Visual C++ project file, via use of Visual C++'s "References" dialog. This will setup the linker, and will copy SDL's .dll files to your app's final output. 4. adjust your app's build settings, at minimum, telling it where to find SDL's header files. 5. add files that contains a WinRT-appropriate main function, along with some data to make sure mouse-cursor-hiding (via SDL_ShowCursor(SDL_DISABLE) calls) work properly. 6. add SDL-specific app code. 7. build and run your app. Setup, Detailed Steps --------------------- ### 1. Create a new project ### Create a new project using one of Visual C++'s templates for a plain, non-XAML, "Direct3D App" (XAML support for SDL/WinRT is not yet ready for use). If you don't see one of these templates, in Visual C++'s 'New Project' dialog, try using the textbox titled, 'Search Installed Templates' to look for one. ### 2. Remove unneeded files from the project ### In the new project, delete any file that has one of the following extensions: - .cpp - .h - .hlsl When you are done, you should be left with a few files, each of which will be a necessary part of your app's project. These files will consist of: - an .appxmanifest file, which contains metadata on your WinRT app. This is similar to an Info.plist file on iOS, or an AndroidManifest.xml on Android. - a few .png files, one of which is a splash screen (displayed when your app launches), others are app icons. - a .pfx file, used for code signing purposes. ### 3. Add references to SDL's project files ### SDL/WinRT can be built in multiple variations, spanning across three different CPU architectures (x86, x64, and ARM) and two different configurations (Debug and Release). WinRT and Visual C++ do not currently provide a means for combining multiple variations of one library into a single file. Furthermore, it does not provide an easy means for copying pre-built .dll files into your app's final output (via Post-Build steps, for example). It does, however, provide a system whereby an app can reference the MSVC projects of libraries such that, when the app is built: 1. each library gets built for the appropriate CPU architecture(s) and WinRT platform(s). 2. each library's output, such as .dll files, get copied to the app's build output. To set this up for SDL/WinRT, you'll need to run through the following steps: 1. open up the Solution Explorer inside Visual C++ (under the "View" menu, then "Solution Explorer") 2. right click on your app's solution. 3. navigate to "Add", then to "Existing Project..." 4. find SDL/WinRT's Visual C++ project file and open it, in the `VisualC-WinRT` directory. 5. once the project has been added, right-click on your app's project and select, "References..." 6. click on the button titled, "Add New Reference..." 7. check the box next to SDL 8. click OK to close the dialog 9. SDL will now show up in the list of references. Click OK to close that dialog. Your project is now linked to SDL's project, insofar that when the app is built, SDL will be built as well, with its build output getting included with your app. ### 4. Adjust Your App's Build Settings ### Some build settings need to be changed in your app's project. This guide will outline the following: - making sure that the compiler knows where to find SDL's header files - **Optional for C++, but NECESSARY for compiling C code:** telling the compiler not to use Microsoft's C++ extensions for WinRT development. - **Optional:** telling the compiler not generate errors due to missing precompiled header files. To change these settings: 1. right-click on the project 2. choose "Properties" 3. in the drop-down box next to "Configuration", choose, "All Configurations" 4. in the drop-down box next to "Platform", choose, "All Platforms" 5. in the left-hand list, expand the "C/C++" section 6. select "General" 7. edit the "Additional Include Directories" setting, and add a path to SDL's "include" directory 8. **Optional: to enable compilation of C code:** change the setting for "Consume Windows Runtime Extension" from "Yes (/ZW)" to "No". If you're working with a completely C++ based project, this step can usually be omitted. 9. **Optional: to disable precompiled headers (which can produce 'stdafx.h'-related build errors, if setup incorrectly:** in the left-hand list, select "Precompiled Headers", then change the setting for "Precompiled Header" from "Use (/Yu)" to "Not Using Precompiled Headers". 10. close the dialog, saving settings, by clicking the "OK" button ### 5. Add a WinRT-appropriate main function, and a blank-cursor image, to the app. ### A few files should be included directly in your app's MSVC project, specifically: 1. a WinRT-appropriate main function (which is different than main() functions on other platforms) 2. a Win32-style cursor resource, used by SDL_ShowCursor() to hide the mouse cursor (if and when the app needs to do so). *If this cursor resource is not included, mouse-position reporting may fail if and when the cursor is hidden, due to possible bugs/design-oddities in Windows itself.* To include these files for C/C++ projects: 1. right-click on your project (again, in Visual C++'s Solution Explorer), navigate to "Add", then choose "Existing Item...". 2. navigate to the directory containing SDL's source code, then into its subdirectory, 'src/main/winrt/'. Select, then add, the following files: - `SDL_winrt_main_NonXAML.cpp` - `SDL2-WinRTResources.rc` - `SDL2-WinRTResource_BlankCursor.cur` 3. right-click on the file `SDL_winrt_main_NonXAML.cpp` (as listed in your project), then click on "Properties...". 4. in the drop-down box next to "Configuration", choose, "All Configurations" 5. in the drop-down box next to "Platform", choose, "All Platforms" 6. in the left-hand list, click on "C/C++" 7. change the setting for "Consume Windows Runtime Extension" to "Yes (/ZW)". 8. click the OK button. This will close the dialog. **NOTE: C++/CX compilation is currently required in at least one file of your app's project. This is to make sure that Visual C++'s linker builds a 'Windows Metadata' file (.winmd) for your app. Not doing so can lead to build errors.** For non-C++ projects, you will need to call SDL_WinRTRunApp from your language's main function, and generate SDL2-WinRTResources.res manually by using `rc` via the Developer Command Prompt and including it as a within the first block in your Visual Studio project file. ### 6. Add app code and assets ### At this point, you can add in SDL-specific source code. Be sure to include a C-style main function (ie: `int main(int argc, char *argv[])`). From there you should be able to create a single `SDL_Window` (WinRT apps can only have one window, at present), as well as an `SDL_Renderer`. Direct3D will be used to draw content. Events are received via SDL's usual event functions (`SDL_PollEvent`, etc.) If you have a set of existing source files and assets, you can start adding them to the project now. If not, or if you would like to make sure that you're setup correctly, some short and simple sample code is provided below. #### 6.A. ... when creating a new app #### If you are creating a new app (rather than porting an existing SDL-based app), or if you would just like a simple app to test SDL/WinRT with before trying to get existing code working, some working SDL/WinRT code is provided below. To set this up: 1. right click on your app's project 2. select Add, then New Item. An "Add New Item" dialog will show up. 3. from the left-hand list, choose "Visual C++" 4. from the middle/main list, choose "C++ File (.cpp)" 5. near the bottom of the dialog, next to "Name:", type in a name for your source file, such as, "main.cpp". 6. click on the Add button. This will close the dialog, add the new file to your project, and open the file in Visual C++'s text editor. 7. Copy and paste the following code into the new file, then save it. ```c #include int main(int argc, char **argv) { SDL_DisplayMode mode; SDL_Window * window = NULL; SDL_Renderer * renderer = NULL; SDL_Event evt; SDL_bool keep_going = SDL_TRUE; if (SDL_Init(SDL_INIT_VIDEO) != 0) { return 1; } else if (SDL_GetCurrentDisplayMode(0, &mode) != 0) { return 1; } else if (SDL_CreateWindowAndRenderer(mode.w, mode.h, SDL_WINDOW_FULLSCREEN, &window, &renderer) != 0) { return 1; } while (keep_going) { while (SDL_PollEvent(&evt)) { if ((evt.type == SDL_KEYDOWN) && (evt.key.keysym.sym == SDLK_ESCAPE)) { keep_going = SDL_FALSE; } } SDL_SetRenderDrawColor(renderer, 0, 255, 0, 255); SDL_RenderClear(renderer); SDL_RenderPresent(renderer); } SDL_Quit(); return 0; } ``` #### 6.B. Adding code and assets #### If you have existing code and assets that you'd like to add, you should be able to add them now. The process for adding a set of files is as such. 1. right click on the app's project 2. select Add, then click on "New Item..." 3. open any source, header, or asset files as appropriate. Support for C and C++ is available. Do note that WinRT only supports a subset of the APIs that are available to Win32-based apps. Many portions of the Win32 API and the C runtime are not available. A list of unsupported C APIs can be found at General information on using the C runtime in WinRT can be found at A list of supported Win32 APIs for WinRT apps can be found at . To note, the list of supported Win32 APIs for Windows Phone 8.0 is different. That list can be found at ### 7. Build and run your app ### Your app project should now be setup, and you should be ready to build your app. To run it on the local machine, open the Debug menu and choose "Start Debugging". This will build your app, then run your app full-screen. To switch out of your app, press the Windows key. Alternatively, you can choose to run your app in a window. To do this, before building and running your app, find the drop-down menu in Visual C++'s toolbar that says, "Local Machine". Expand this by clicking on the arrow on the right side of the list, then click on Simulator. Once you do that, any time you build and run the app, the app will launch in window, rather than full-screen. #### 7.A. Running apps on older, ARM-based, "Windows RT" devices #### **These instructions do not include Windows Phone, despite Windows Phone typically running on ARM processors.** They are specifically for devices that use the "Windows RT" operating system, which was a modified version of Windows 8.x that ran primarily on ARM-based tablet computers. To build and run the app on ARM-based, "Windows RT" devices, you'll need to: - install Microsoft's "Remote Debugger" on the device. Visual C++ installs and debugs ARM-based apps via IP networks. - change a few options on the development machine, both to make sure it builds for ARM (rather than x86 or x64), and to make sure it knows how to find the Windows RT device (on the network). Microsoft's Remote Debugger can be found at . Please note that separate versions of this debugger exist for different versions of Visual C++, one each for MSVC 2015, 2013, and 2012. To setup Visual C++ to launch your app on an ARM device: 1. make sure the Remote Debugger is running on your ARM device, and that it's on the same IP network as your development machine. 2. from Visual C++'s toolbar, find a drop-down menu that says, "Win32". Click it, then change the value to "ARM". 3. make sure Visual C++ knows the hostname or IP address of the ARM device. To do this: 1. open the app project's properties 2. select "Debugging" 3. next to "Machine Name", enter the hostname or IP address of the ARM device 4. if, and only if, you've turned off authentication in the Remote Debugger, then change the setting for "Require Authentication" to No 5. click "OK" 4. build and run the app (from Visual C++). The first time you do this, a prompt will show up on the ARM device, asking for a Microsoft Account. You do, unfortunately, need to log in here, and will need to follow the subsequent registration steps in order to launch the app. After you do so, if the app didn't already launch, try relaunching it again from within Visual C++. Troubleshooting --------------- #### Build fails with message, "error LNK2038: mismatch detected for 'vccorlib_lib_should_be_specified_before_msvcrt_lib_to_linker'" Try adding the following to your linker flags. In MSVC, this can be done by right-clicking on the app project, navigating to Configuration Properties -> Linker -> Command Line, then adding them to the Additional Options section. * For Release builds / MSVC-Configurations, add: /nodefaultlib:vccorlib /nodefaultlib:msvcrt vccorlib.lib msvcrt.lib * For Debug builds / MSVC-Configurations, add: /nodefaultlib:vccorlibd /nodefaultlib:msvcrtd vccorlibd.lib msvcrtd.lib #### Mouse-motion events fail to get sent, or SDL_GetMouseState() fails to return updated values This may be caused by a bug in Windows itself, whereby hiding the mouse cursor can cause mouse-position reporting to fail. SDL provides a workaround for this, but it requires that an app links to a set of Win32-style cursor image-resource files. A copy of suitable resource files can be found in `src/main/winrt/`. Adding them to an app's Visual C++ project file should be sufficient to get the app to use them. #### SDL's Visual Studio project file fails to open, with message, "The system can't find the file specified." This can be caused for any one of a few reasons, which Visual Studio can report, but won't always do so in an up-front manner. To help determine why this error comes up: 1. open a copy of Visual Studio without opening a project file. This can be accomplished via Windows' Start Menu, among other means. 2. show Visual Studio's Output window. This can be done by going to VS' menu bar, then to View, and then to Output. 3. try opening the SDL project file directly by going to VS' menu bar, then to File, then to Open, then to Project/Solution. When a File-Open dialog appears, open the SDL project (such as the one in SDL's source code, in its directory, VisualC-WinRT/UWP_VS2015/). 4. after attempting to open SDL's Visual Studio project file, additional error information will be output to the Output window. If Visual Studio reports (via its Output window) that the project: "could not be loaded because it's missing install components. To fix this launch Visual Studio setup with the following selections: Microsoft.VisualStudio.ComponentGroup.UWP.VC" ... then you will need to re-launch Visual Studio's installer, and make sure that the workflow for "Universal Windows Platform development" is checked, and that its optional component, "C++ Universal Windows Platform tools" is also checked. While you are there, if you are planning on targeting UWP / Windows 10, also make sure that you check the optional component, "Windows 10 SDK (10.0.10240.0)". After making sure these items are checked as-appropriate, install them. Once you install these components, try re-launching Visual Studio, and re-opening the SDL project file. If you still get the error dialog, try using the Output window, again, seeing what Visual Studio says about it. #### Game controllers / joysticks aren't working! Windows only permits certain game controllers and joysticks to work within WinRT / UWP apps. Even if a game controller or joystick works in a Win32 app, that device is not guaranteed to work inside a WinRT / UWP app. According to Microsoft, "Xbox compatible controllers" should work inside UWP apps, potentially with more working in the future. This includes, but may not be limited to, Microsoft-made Xbox controllers and USB adapters. (Source: https://social.msdn.microsoft.com/Forums/en-US/9064838b-e8c3-4c18-8a83-19bf0dfe150d/xinput-fails-to-detect-game-controllers?forum=wpdevelop) --- ### Vs Sdl2 Docs Release Checklist (`vs/sdl2/docs/release_checklist.md`) # Release checklist When changing the version, run `build-scripts/update-version.sh X Y Z`, where `X Y Z` are the major version, minor version, and patch level. So `2 28 1` means "change the version to 2.28.1". This script does much of the mechanical work. ## New feature release * Update `WhatsNew.txt` * Bump version number to 2.EVEN.0: * `./build-scripts/update-version.sh 2 EVEN 0` * Do the release * Update the website file include/header.inc.php to reflect the new version ## New bugfix release * Check that no new API/ABI was added * If it was, do a new feature release (see above) instead * Bump version number from 2.Y.Z to 2.Y.(Z+1) (Y is even) * `./build-scripts/update-version.sh 2 Y Z+1` * Do the release * Update the website file include/header.inc.php to reflect the new version ## After a feature release * Create a branch like `release-2.24.x` * Bump version number to 2.ODD.0 for next development branch * `./build-scripts/update-version.sh 2 ODD 0` ## New development prerelease * Bump version number from 2.Y.Z to 2.Y.(Z+1) (Y is odd) * `./build-scripts/update-version.sh 2 Y Z+1` * Do the release --- ### Vs Zlib Doc Algorithm (`vs/zlib/doc/algorithm.txt`) 1. Compression algorithm (deflate) The deflation algorithm used by gzip (also zip and zlib) is a variation of LZ77 (Lempel-Ziv 1977, see reference below). It finds duplicated strings in the input data. The second occurrence of a string is replaced by a pointer to the previous string, in the form of a pair (distance, length). Distances are limited to 32K bytes, and lengths are limited to 258 bytes. When a string does not occur anywhere in the previous 32K bytes, it is emitted as a sequence of literal bytes. (In this description, `string' must be taken as an arbitrary sequence of bytes, and is not restricted to printable characters.) Literals or match lengths are compressed with one Huffman tree, and match distances are compressed with another tree. The trees are stored in a compact form at the start of each block. The blocks can have any size (except that the compressed data for one block must fit in available memory). A block is terminated when deflate() determines that it would be useful to start another block with fresh trees. (This is somewhat similar to the behavior of LZW-based _compress_.) Duplicated strings are found using a hash table. All input strings of length 3 are inserted in the hash table. A hash index is computed for the next 3 bytes. If the hash chain for this index is not empty, all strings in the chain are compared with the current input string, and the longest match is selected. The hash chains are searched starting with the most recent strings, to favor small distances and thus take advantage of the Huffman encoding. The hash chains are singly linked. There are no deletions from the hash chains, the algorithm simply discards matches that are too old. To avoid a worst-case situation, very long hash chains are arbitrarily truncated at a certain length, determined by a runtime option (level parameter of deflateInit). So deflate() does not always find the longest possible match but generally finds a match which is long enough. deflate() also defers the selection of matches with a lazy evaluation mechanism. After a match of length N has been found, deflate() searches for a longer match at the next input byte. If a longer match is found, the previous match is truncated to a length of one (thus producing a single literal byte) and the process of lazy evaluation begins again. Otherwise, the original match is kept, and the next match search is attempted only N steps later. The lazy match evaluation is also subject to a runtime parameter. If the current match is long enough, deflate() reduces the search for a longer match, thus speeding up the whole process. If compression ratio is more important than speed, deflate() attempts a complete second search even if the first match is already long enough. The lazy match evaluation is not performed for the fastest compression modes (level parameter 1 to 3). For these fast modes, new strings are inserted in the hash table only when no match was found, or when the match is not too long. This degrades the compression ratio but saves time since there are both fewer insertions and fewer searches. 2. Decompression algorithm (inflate) 2.1 Introduction The key question is how to represent a Huffman code (or any prefix code) so that you can decode fast. The most important characteristic is that shorter codes are much more common than longer codes, so pay attention to decoding the short codes fast, and let the long codes take longer to decode. inflate() sets up a first level table that covers some number of bits of input less than the length of longest code. It gets that many bits from the stream, and looks it up in the table. The table will tell if the next code is that many bits or less and how many, and if it is, it will tell the value, else it will point to the next level table for which inflate() grabs more bits and tries to decode a longer code. How many bits to make the first lookup is a tradeoff between the time it takes to decode and the time it takes to build the table. If building the table took no time (and if you had infinite memory), then there would only be a first level table to cover all the way to the longest code. However, building the table ends up taking a lot longer for more bits since short codes are replicated many times in such a table. What inflate() does is simply to make the number of bits in the first table a variable, and then to set that variable for the maximum speed. For inflate, which has 286 possible codes for the literal/length tree, the size of the first table is nine bits. Also the distance trees have 30 possible values, and the size of the first table is six bits. Note that for each of those cases, the table ended up one bit longer than the ``average'' code length, i.e. the code length of an approximately flat code which would be a little more than eight bits for 286 symbols and a little less than five bits for 30 symbols. 2.2 More details on the inflate table lookup Ok, you want to know what this cleverly obfuscated inflate tree actually looks like. You are correct that it's not a Huffman tree. It is simply a lookup table for the first, let's say, nine bits of a Huffman symbol. The symbol could be as short as one bit or as long as 15 bits. If a particular symbol is shorter than nine bits, then that symbol's translation is duplicated in all those entries that start with that symbol's bits. For example, if the symbol is four bits, then it's duplicated 32 times in a nine-bit table. If a symbol is nine bits long, it appears in the table once. If the symbol is longer than nine bits, then that entry in the table points to another similar table for the remaining bits. Again, there are duplicated entries as needed. The idea is that most of the time the symbol will be short and there will only be one table look up. (That's whole idea behind data compression in the first place.) For the less frequent long symbols, there will be two lookups. If you had a compression method with really long symbols, you could have as many levels of lookups as is efficient. For inflate, two is enough. So a table entry either points to another table (in which case nine bits in the above example are gobbled), or it contains the translation for the symbol and the number of bits to gobble. Then you start again with the next ungobbled bit. You may wonder: why not just have one lookup table for how ever many bits the longest symbol is? The reason is that if you do that, you end up spending more time filling in duplicate symbol entries than you do actually decoding. At least for deflate's output that generates new trees every several 10's of kbytes. You can imagine that filling in a 2^15 entry table for a 15-bit code would take too long if you're only decoding several thousand symbols. At the other extreme, you could make a new table for every bit in the code. In fact, that's essentially a Huffman tree. But then you spend too much time traversing the tree while decoding, even for short symbols. So the number of bits for the first lookup table is a trade of the time to fill out the table vs. the time spent looking at the second level and above of the table. Here is an example, scaled down: The code being decoded, with 10 symbols, from 1 to 6 bits long: A: 0 B: 10 C: 1100 D: 11010 E: 11011 F: 11100 G: 11101 H: 11110 I: 111110 J: 111111 Let's make the first table three bits long (eight entries): 000: A,1 001: A,1 010: A,1 011: A,1 100: B,2 101: B,2 110: -> table X (gobble 3 bits) 111: -> table Y (gobble 3 bits) Each entry is what the bits decode as and how many bits that is, i.e. how many bits to gobble. Or the entry points to another table, with the number of bits to gobble implicit in the size of the table. Table X is two bits long since the longest code starting with 110 is five bits long: 00: C,1 01: C,1 10: D,2 11: E,2 Table Y is three bits long since the longest code starting with 111 is six bits long: 000: F,2 001: F,2 010: G,2 011: G,2 100: H,2 101: H,2 110: I,3 111: J,3 So what we have here are three tables with a total of 20 entries that had to be constructed. That's compared to 64 entries for a single table. Or compared to 16 entries for a Huffman tree (six two entry tables and one four entry table). Assuming that the code ideally represents the probability of the symbols, it takes on the average 1.25 lookups per symbol. That's compared to one lookup for the single table, or 1.66 lookups per symbol for the Huffman tree. There, I think that gives you a picture of what's going on. For inflate, the meaning of a particular symbol is often more than just a letter. It can be a byte (a "literal"), or it can be either a length or a distance which indicates a base value and a number of bits to fetch after the code that is added to the base value. Or it might be the special end-of-block code. The data structures created in inftrees.c try to encode all that information compactly in the tables. Jean-loup Gailly Mark Adler jloup@gzip.org madler@alumni.caltech.edu References: [LZ77] Ziv J., Lempel A., ``A Universal Algorithm for Sequential Data Compression,'' IEEE Transactions on Information Theory, Vol. 23, No. 3, pp. 337-343. ``DEFLATE Compressed Data Format Specification'' available in http://tools.ietf.org/html/rfc1951 --- ### Vs Zlib Doc Rfc1950 (`vs/zlib/doc/rfc1950.txt`) Network Working Group P. Deutsch Request for Comments: 1950 Aladdin Enterprises Category: Informational J-L. Gailly Info-ZIP May 1996 ZLIB Compressed Data Format Specification version 3.3 Status of This Memo This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. IESG Note: The IESG takes no position on the validity of any Intellectual Property Rights statements contained in this document. Notices Copyright (c) 1996 L. Peter Deutsch and Jean-Loup Gailly Permission is granted to copy and distribute this document for any purpose and without charge, including translations into other languages and incorporation into compilations, provided that the copyright notice and this notice are preserved, and that any substantive changes or deletions from the original are clearly marked. A pointer to the latest version of this and related documentation in HTML format can be found at the URL . Abstract This specification defines a lossless compressed data format. The data can be produced or consumed, even for an arbitrarily long sequentially presented input data stream, using only an a priori bounded amount of intermediate storage. The format presently uses the DEFLATE compression method but can be easily extended to use other compression methods. It can be implemented readily in a manner not covered by patents. This specification also defines the ADLER-32 checksum (an extension and improvement of the Fletcher checksum), used for detection of data corruption, and provides an algorithm for computing it. Deutsch & Gailly Informational [Page 1] RFC 1950 ZLIB Compressed Data Format Specification May 1996 Table of Contents 1. Introduction ................................................... 2 1.1. Purpose ................................................... 2 1.2. Intended audience ......................................... 3 1.3. Scope ..................................................... 3 1.4. Compliance ................................................ 3 1.5. Definitions of terms and conventions used ................ 3 1.6. Changes from previous versions ............................ 3 2. Detailed specification ......................................... 3 2.1. Overall conventions ....................................... 3 2.2. Data format ............................................... 4 2.3. Compliance ................................................ 7 3. References ..................................................... 7 4. Source code .................................................... 8 5. Security Considerations ........................................ 8 6. Acknowledgements ............................................... 8 7. Authors' Addresses ............................................. 8 8. Appendix: Rationale ............................................ 9 9. Appendix: Sample code ..........................................10 1. Introduction 1.1. Purpose The purpose of this specification is to define a lossless compressed data format that: * Is independent of CPU type, operating system, file system, and character set, and hence can be used for interchange; * Can be produced or consumed, even for an arbitrarily long sequentially presented input data stream, using only an a priori bounded amount of intermediate storage, and hence can be used in data communications or similar structures such as Unix filters; * Can use a number of different compression methods; * Can be implemented readily in a manner not covered by patents, and hence can be practiced freely. The data format defined by this specification does not attempt to allow random access to compressed data. Deutsch & Gailly Informational [Page 2] RFC 1950 ZLIB Compressed Data Format Specification May 1996 1.2. Intended audience This specification is intended for use by implementors of software to compress data into zlib format and/or decompress data from zlib format. The text of the specification assumes a basic background in programming at the level of bits and other primitive data representations. 1.3. Scope The specification specifies a compressed data format that can be used for in-memory compression of a sequence of arbitrary bytes. 1.4. Compliance Unless otherwise indicated below, a compliant decompressor must be able to accept and decompress any data set that conforms to all the specifications presented here; a compliant compressor must produce data sets that conform to all the specifications presented here. 1.5. Definitions of terms and conventions used byte: 8 bits stored or transmitted as a unit (same as an octet). (For this specification, a byte is exactly 8 bits, even on machines which store a character on a number of bits different from 8.) See below, for the numbering of bits within a byte. 1.6. Changes from previous versions Version 3.1 was the first public release of this specification. In version 3.2, some terminology was changed and the Adler-32 sample code was rewritten for clarity. In version 3.3, the support for a preset dictionary was introduced, and the specification was converted to RFC style. 2. Detailed specification 2.1. Overall conventions In the diagrams below, a box like this: +---+ | | <-- the vertical bars might be missing +---+ Deutsch & Gailly Informational [Page 3] RFC 1950 ZLIB Compressed Data Format Specification May 1996 represents one byte; a box like this: +==============+ | | +==============+ represents a variable number of bytes. Bytes stored within a computer do not have a "bit order", since they are always treated as a unit. However, a byte considered as an integer between 0 and 255 does have a most- and least- significant bit, and since we write numbers with the most- significant digit on the left, we also write bytes with the most- significant bit on the left. In the diagrams below, we number the bits of a byte so that bit 0 is the least-significant bit, i.e., the bits are numbered: +--------+ |76543210| +--------+ Within a computer, a number may occupy multiple bytes. All multi-byte numbers in the format described here are stored with the MOST-significant byte first (at the lower memory address). For example, the decimal number 520 is stored as: 0 1 +--------+--------+ |00000010|00001000| +--------+--------+ ^ ^ | | | + less significant byte = 8 + more significant byte = 2 x 256 2.2. Data format A zlib stream has the following structure: 0 1 +---+---+ |CMF|FLG| (more-->) +---+---+ Deutsch & Gailly Informational [Page 4] RFC 1950 ZLIB Compressed Data Format Specification May 1996 (if FLG.FDICT set) 0 1 2 3 +---+---+---+---+ | DICTID | (more-->) +---+---+---+---+ +=====================+---+---+---+---+ |...compressed data...| ADLER32 | +=====================+---+---+---+---+ Any data which may appear after ADLER32 are not part of the zlib stream. CMF (Compression Method and flags) This byte is divided into a 4-bit compression method and a 4- bit information field depending on the compression method. bits 0 to 3 CM Compression method bits 4 to 7 CINFO Compression info CM (Compression method) This identifies the compression method used in the file. CM = 8 denotes the "deflate" compression method with a window size up to 32K. This is the method used by gzip and PNG (see references [1] and [2] in Chapter 3, below, for the reference documents). CM = 15 is reserved. It might be used in a future version of this specification to indicate the presence of an extra field before the compressed data. CINFO (Compression info) For CM = 8, CINFO is the base-2 logarithm of the LZ77 window size, minus eight (CINFO=7 indicates a 32K window size). Values of CINFO above 7 are not allowed in this version of the specification. CINFO is not defined in this specification for CM not equal to 8. FLG (FLaGs) This flag byte is divided as follows: bits 0 to 4 FCHECK (check bits for CMF and FLG) bit 5 FDICT (preset dictionary) bits 6 to 7 FLEVEL (compression level) The FCHECK value must be such that CMF and FLG, when viewed as a 16-bit unsigned integer stored in MSB order (CMF*256 + FLG), is a multiple of 31. Deutsch & Gailly Informational [Page 5] RFC 1950 ZLIB Compressed Data Format Specification May 1996 FDICT (Preset dictionary) If FDICT is set, a DICT dictionary identifier is present immediately after the FLG byte. The dictionary is a sequence of bytes which are initially fed to the compressor without producing any compressed output. DICT is the Adler-32 checksum of this sequence of bytes (see the definition of ADLER32 below). The decompressor can use this identifier to determine which dictionary has been used by the compressor. FLEVEL (Compression level) These flags are available for use by specific compression methods. The "deflate" method (CM = 8) sets these flags as follows: 0 - compressor used fastest algorithm 1 - compressor used fast algorithm 2 - compressor used default algorithm 3 - compressor used maximum compression, slowest algorithm The information in FLEVEL is not needed for decompression; it is there to indicate if recompression might be worthwhile. compressed data For compression method 8, the compressed data is stored in the deflate compressed data format as described in the document "DEFLATE Compressed Data Format Specification" by L. Peter Deutsch. (See reference [3] in Chapter 3, below) Other compressed data formats are not specified in this version of the zlib specification. ADLER32 (Adler-32 checksum) This contains a checksum value of the uncompressed data (excluding any dictionary data) computed according to Adler-32 algorithm. This algorithm is a 32-bit extension and improvement of the Fletcher algorithm, used in the ITU-T X.224 / ISO 8073 standard. See references [4] and [5] in Chapter 3, below) Adler-32 is composed of two sums accumulated per byte: s1 is the sum of all bytes, s2 is the sum of all s1 values. Both sums are done modulo 65521. s1 is initialized to 1, s2 to zero. The Adler-32 checksum is stored as s2*65536 + s1 in most- significant-byte first (network) order. Deutsch & Gailly Informational [Page 6] RFC 1950 ZLIB Compressed Data Format Specification May 1996 2.3. Compliance A compliant compressor must produce streams with correct CMF, FLG and ADLER32, but need not support preset dictionaries. When the zlib data format is used as part of another standard data format, the compressor may use only preset dictionaries that are specified by this other data format. If this other format does not use the preset dictionary feature, the compressor must not set the FDICT flag. A compliant decompressor must check CMF, FLG, and ADLER32, and provide an error indication if any of these have incorrect values. A compliant decompressor must give an error indication if CM is not one of the values defined in this specification (only the value 8 is permitted in this version), since another value could indicate the presence of new features that would cause subsequent data to be interpreted incorrectly. A compliant decompressor must give an error indication if FDICT is set and DICTID is not the identifier of a known preset dictionary. A decompressor may ignore FLEVEL and still be compliant. When the zlib data format is being used as a part of another standard format, a compliant decompressor must support all the preset dictionaries specified by the other format. When the other format does not use the preset dictionary feature, a compliant decompressor must reject any stream in which the FDICT flag is set. 3. References [1] Deutsch, L.P.,"GZIP Compressed Data Format Specification", available in ftp://ftp.uu.net/pub/archiving/zip/doc/ [2] Thomas Boutell, "PNG (Portable Network Graphics) specification", available in ftp://ftp.uu.net/graphics/png/documents/ [3] Deutsch, L.P.,"DEFLATE Compressed Data Format Specification", available in ftp://ftp.uu.net/pub/archiving/zip/doc/ [4] Fletcher, J. G., "An Arithmetic Checksum for Serial Transmissions," IEEE Transactions on Communications, Vol. COM-30, No. 1, January 1982, pp. 247-252. [5] ITU-T Recommendation X.224, Annex D, "Checksum Algorithms," November, 1993, pp. 144, 145. (Available from gopher://info.itu.ch). ITU-T X.244 is also the same as ISO 8073. Deutsch & Gailly Informational [Page 7] RFC 1950 ZLIB Compressed Data Format Specification May 1996 4. Source code Source code for a C language implementation of a "zlib" compliant library is available at ftp://ftp.uu.net/pub/archiving/zip/zlib/. 5. Security Considerations A decoder that fails to check the ADLER32 checksum value may be subject to undetected data corruption. 6. Acknowledgements Trademarks cited in this document are the property of their respective owners. Jean-Loup Gailly and Mark Adler designed the zlib format and wrote the related software described in this specification. Glenn Randers-Pehrson converted this document to RFC and HTML format. 7. Authors' Addresses L. Peter Deutsch Aladdin Enterprises 203 Santa Margarita Ave. Menlo Park, CA 94025 Phone: (415) 322-0103 (AM only) FAX: (415) 322-1734 EMail: Jean-Loup Gailly EMail: Questions about the technical content of this specification can be sent by email to Jean-Loup Gailly and Mark Adler Editorial comments on this specification can be sent by email to L. Peter Deutsch and Glenn Randers-Pehrson Deutsch & Gailly Informational [Page 8] RFC 1950 ZLIB Compressed Data Format Specification May 1996 8. Appendix: Rationale 8.1. Preset dictionaries A preset dictionary is specially useful to compress short input sequences. The compressor can take advantage of the dictionary context to encode the input in a more compact manner. The decompressor can be initialized with the appropriate context by virtually decompressing a compressed version of the dictionary without producing any output. However for certain compression algorithms such as the deflate algorithm this operation can be achieved without actually performing any decompression. The compressor and the decompressor must use exactly the same dictionary. The dictionary may be fixed or may be chosen among a certain number of predefined dictionaries, according to the kind of input data. The decompressor can determine which dictionary has been chosen by the compressor by checking the dictionary identifier. This document does not specify the contents of predefined dictionaries, since the optimal dictionaries are application specific. Standard data formats using this feature of the zlib specification must precisely define the allowed dictionaries. 8.2. The Adler-32 algorithm The Adler-32 algorithm is much faster than the CRC32 algorithm yet still provides an extremely low probability of undetected errors. The modulo on unsigned long accumulators can be delayed for 5552 bytes, so the modulo operation time is negligible. If the bytes are a, b, c, the second sum is 3a + 2b + c + 3, and so is position and order sensitive, unlike the first sum, which is just a checksum. That 65521 is prime is important to avoid a possible large class of two-byte errors that leave the check unchanged. (The Fletcher checksum uses 255, which is not prime and which also makes the Fletcher check insensitive to single byte changes 0 <-> 255.) The sum s1 is initialized to 1 instead of zero to make the length of the sequence part of s2, so that the length does not have to be checked separately. (Any sequence of zeroes has a Fletcher checksum of zero.) Deutsch & Gailly Informational [Page 9] RFC 1950 ZLIB Compressed Data Format Specification May 1996 9. Appendix: Sample code The following C code computes the Adler-32 checksum of a data buffer. It is written for clarity, not for speed. The sample code is in the ANSI C programming language. Non C users may find it easier to read with these hints: & Bitwise AND operator. >> Bitwise right shift operator. When applied to an unsigned quantity, as here, right shift inserts zero bit(s) at the left. << Bitwise left shift operator. Left shift inserts zero bit(s) at the right. ++ "n++" increments the variable n. % modulo operator: a % b is the remainder of a divided by b. #define BASE 65521 /* largest prime smaller than 65536 */ /* Update a running Adler-32 checksum with the bytes buf[0..len-1] and return the updated checksum. The Adler-32 checksum should be initialized to 1. Usage example: unsigned long adler = 1L; while (read_buffer(buffer, length) != EOF) { adler = update_adler32(adler, buffer, length); } if (adler != original_adler) error(); */ unsigned long update_adler32(unsigned long adler, unsigned char *buf, int len) { unsigned long s1 = adler & 0xffff; unsigned long s2 = (adler >> 16) & 0xffff; int n; for (n = 0; n < len; n++) { s1 = (s1 + buf[n]) % BASE; s2 = (s2 + s1) % BASE; } return (s2 << 16) + s1; } /* Return the adler32 of the bytes buf[0..len-1] */ Deutsch & Gailly Informational [Page 10] RFC 1950 ZLIB Compressed Data Format Specification May 1996 unsigned long adler32(unsigned char *buf, int len) { return update_adler32(1L, buf, len); } Deutsch & Gailly Informational [Page 11] --- ### Vs Zlib Doc Rfc1951 (`vs/zlib/doc/rfc1951.txt`) Network Working Group P. Deutsch Request for Comments: 1951 Aladdin Enterprises Category: Informational May 1996 DEFLATE Compressed Data Format Specification version 1.3 Status of This Memo This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. IESG Note: The IESG takes no position on the validity of any Intellectual Property Rights statements contained in this document. Notices Copyright (c) 1996 L. Peter Deutsch Permission is granted to copy and distribute this document for any purpose and without charge, including translations into other languages and incorporation into compilations, provided that the copyright notice and this notice are preserved, and that any substantive changes or deletions from the original are clearly marked. A pointer to the latest version of this and related documentation in HTML format can be found at the URL . Abstract This specification defines a lossless compressed data format that compresses data using a combination of the LZ77 algorithm and Huffman coding, with efficiency comparable to the best currently available general-purpose compression methods. The data can be produced or consumed, even for an arbitrarily long sequentially presented input data stream, using only an a priori bounded amount of intermediate storage. The format can be implemented readily in a manner not covered by patents. Deutsch Informational [Page 1] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 Table of Contents 1. Introduction ................................................... 2 1.1. Purpose ................................................... 2 1.2. Intended audience ......................................... 3 1.3. Scope ..................................................... 3 1.4. Compliance ................................................ 3 1.5. Definitions of terms and conventions used ................ 3 1.6. Changes from previous versions ............................ 4 2. Compressed representation overview ............................. 4 3. Detailed specification ......................................... 5 3.1. Overall conventions ....................................... 5 3.1.1. Packing into bytes .................................. 5 3.2. Compressed block format ................................... 6 3.2.1. Synopsis of prefix and Huffman coding ............... 6 3.2.2. Use of Huffman coding in the "deflate" format ....... 7 3.2.3. Details of block format ............................. 9 3.2.4. Non-compressed blocks (BTYPE=00) ................... 11 3.2.5. Compressed blocks (length and distance codes) ...... 11 3.2.6. Compression with fixed Huffman codes (BTYPE=01) .... 12 3.2.7. Compression with dynamic Huffman codes (BTYPE=10) .. 13 3.3. Compliance ............................................... 14 4. Compression algorithm details ................................. 14 5. References .................................................... 16 6. Security Considerations ....................................... 16 7. Source code ................................................... 16 8. Acknowledgements .............................................. 16 9. Author's Address .............................................. 17 1. Introduction 1.1. Purpose The purpose of this specification is to define a lossless compressed data format that: * Is independent of CPU type, operating system, file system, and character set, and hence can be used for interchange; * Can be produced or consumed, even for an arbitrarily long sequentially presented input data stream, using only an a priori bounded amount of intermediate storage, and hence can be used in data communications or similar structures such as Unix filters; * Compresses data with efficiency comparable to the best currently available general-purpose compression methods, and in particular considerably better than the "compress" program; * Can be implemented readily in a manner not covered by patents, and hence can be practiced freely; Deutsch Informational [Page 2] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 * Is compatible with the file format produced by the current widely used gzip utility, in that conforming decompressors will be able to read data produced by the existing gzip compressor. The data format defined by this specification does not attempt to: * Allow random access to compressed data; * Compress specialized data (e.g., raster graphics) as well as the best currently available specialized algorithms. A simple counting argument shows that no lossless compression algorithm can compress every possible input data set. For the format defined here, the worst case expansion is 5 bytes per 32K- byte block, i.e., a size increase of 0.015% for large data sets. English text usually compresses by a factor of 2.5 to 3; executable files usually compress somewhat less; graphical data such as raster images may compress much more. 1.2. Intended audience This specification is intended for use by implementors of software to compress data into "deflate" format and/or decompress data from "deflate" format. The text of the specification assumes a basic background in programming at the level of bits and other primitive data representations. Familiarity with the technique of Huffman coding is helpful but not required. 1.3. Scope The specification specifies a method for representing a sequence of bytes as a (usually shorter) sequence of bits, and a method for packing the latter bit sequence into bytes. 1.4. Compliance Unless otherwise indicated below, a compliant decompressor must be able to accept and decompress any data set that conforms to all the specifications presented here; a compliant compressor must produce data sets that conform to all the specifications presented here. 1.5. Definitions of terms and conventions used Byte: 8 bits stored or transmitted as a unit (same as an octet). For this specification, a byte is exactly 8 bits, even on machines Deutsch Informational [Page 3] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 which store a character on a number of bits different from eight. See below, for the numbering of bits within a byte. String: a sequence of arbitrary bytes. 1.6. Changes from previous versions There have been no technical changes to the deflate format since version 1.1 of this specification. In version 1.2, some terminology was changed. Version 1.3 is a conversion of the specification to RFC style. 2. Compressed representation overview A compressed data set consists of a series of blocks, corresponding to successive blocks of input data. The block sizes are arbitrary, except that non-compressible blocks are limited to 65,535 bytes. Each block is compressed using a combination of the LZ77 algorithm and Huffman coding. The Huffman trees for each block are independent of those for previous or subsequent blocks; the LZ77 algorithm may use a reference to a duplicated string occurring in a previous block, up to 32K input bytes before. Each block consists of two parts: a pair of Huffman code trees that describe the representation of the compressed data part, and a compressed data part. (The Huffman trees themselves are compressed using Huffman encoding.) The compressed data consists of a series of elements of two types: literal bytes (of strings that have not been detected as duplicated within the previous 32K input bytes), and pointers to duplicated strings, where a pointer is represented as a pair . The representation used in the "deflate" format limits distances to 32K bytes and lengths to 258 bytes, but does not limit the size of a block, except for uncompressible blocks, which are limited as noted above. Each type of value (literals, distances, and lengths) in the compressed data is represented using a Huffman code, using one code tree for literals and lengths and a separate code tree for distances. The code trees for each block appear in a compact form just before the compressed data for that block. Deutsch Informational [Page 4] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 3. Detailed specification 3.1. Overall conventions In the diagrams below, a box like this: +---+ | | <-- the vertical bars might be missing +---+ represents one byte; a box like this: +==============+ | | +==============+ represents a variable number of bytes. Bytes stored within a computer do not have a "bit order", since they are always treated as a unit. However, a byte considered as an integer between 0 and 255 does have a most- and least- significant bit, and since we write numbers with the most- significant digit on the left, we also write bytes with the most- significant bit on the left. In the diagrams below, we number the bits of a byte so that bit 0 is the least-significant bit, i.e., the bits are numbered: +--------+ |76543210| +--------+ Within a computer, a number may occupy multiple bytes. All multi-byte numbers in the format described here are stored with the least-significant byte first (at the lower memory address). For example, the decimal number 520 is stored as: 0 1 +--------+--------+ |00001000|00000010| +--------+--------+ ^ ^ | | | + more significant byte = 2 x 256 + less significant byte = 8 3.1.1. Packing into bytes This document does not address the issue of the order in which bits of a byte are transmitted on a bit-sequential medium, since the final data format described here is byte- rather than Deutsch Informational [Page 5] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 bit-oriented. However, we describe the compressed block format in below, as a sequence of data elements of various bit lengths, not a sequence of bytes. We must therefore specify how to pack these data elements into bytes to form the final compressed byte sequence: * Data elements are packed into bytes in order of increasing bit number within the byte, i.e., starting with the least-significant bit of the byte. * Data elements other than Huffman codes are packed starting with the least-significant bit of the data element. * Huffman codes are packed starting with the most- significant bit of the code. In other words, if one were to print out the compressed data as a sequence of bytes, starting with the first byte at the *right* margin and proceeding to the *left*, with the most- significant bit of each byte on the left as usual, one would be able to parse the result from right to left, with fixed-width elements in the correct MSB-to-LSB order and Huffman codes in bit-reversed order (i.e., with the first bit of the code in the relative LSB position). 3.2. Compressed block format 3.2.1. Synopsis of prefix and Huffman coding Prefix coding represents symbols from an a priori known alphabet by bit sequences (codes), one code for each symbol, in a manner such that different symbols may be represented by bit sequences of different lengths, but a parser can always parse an encoded string unambiguously symbol-by-symbol. We define a prefix code in terms of a binary tree in which the two edges descending from each non-leaf node are labeled 0 and 1 and in which the leaf nodes correspond one-for-one with (are labeled with) the symbols of the alphabet; then the code for a symbol is the sequence of 0's and 1's on the edges leading from the root to the leaf labeled with that symbol. For example: Deutsch Informational [Page 6] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 /\ Symbol Code 0 1 ------ ---- / \ A 00 /\ B B 1 0 1 C 011 / \ D 010 A /\ 0 1 / \ D C A parser can decode the next symbol from an encoded input stream by walking down the tree from the root, at each step choosing the edge corresponding to the next input bit. Given an alphabet with known symbol frequencies, the Huffman algorithm allows the construction of an optimal prefix code (one which represents strings with those symbol frequencies using the fewest bits of any possible prefix codes for that alphabet). Such a code is called a Huffman code. (See reference [1] in Chapter 5, references for additional information on Huffman codes.) Note that in the "deflate" format, the Huffman codes for the various alphabets must not exceed certain maximum code lengths. This constraint complicates the algorithm for computing code lengths from symbol frequencies. Again, see Chapter 5, references for details. 3.2.2. Use of Huffman coding in the "deflate" format The Huffman codes used for each alphabet in the "deflate" format have two additional rules: * All codes of a given bit length have lexicographically consecutive values, in the same order as the symbols they represent; * Shorter codes lexicographically precede longer codes. Deutsch Informational [Page 7] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 We could recode the example above to follow this rule as follows, assuming that the order of the alphabet is ABCD: Symbol Code ------ ---- A 10 B 0 C 110 D 111 I.e., 0 precedes 10 which precedes 11x, and 110 and 111 are lexicographically consecutive. Given this rule, we can define the Huffman code for an alphabet just by giving the bit lengths of the codes for each symbol of the alphabet in order; this is sufficient to determine the actual codes. In our example, the code is completely defined by the sequence of bit lengths (2, 1, 3, 3). The following algorithm generates the codes as integers, intended to be read from most- to least-significant bit. The code lengths are initially in tree[I].Len; the codes are produced in tree[I].Code. 1) Count the number of codes for each code length. Let bl_count[N] be the number of codes of length N, N >= 1. 2) Find the numerical value of the smallest code for each code length: code = 0; bl_count[0] = 0; for (bits = 1; bits <= MAX_BITS; bits++) { code = (code + bl_count[bits-1]) << 1; next_code[bits] = code; } 3) Assign numerical values to all codes, using consecutive values for all codes of the same length with the base values determined at step 2. Codes that are never used (which have a bit length of zero) must not be assigned a value. for (n = 0; n <= max_code; n++) { len = tree[n].Len; if (len != 0) { tree[n].Code = next_code[len]; next_code[len]++; } Deutsch Informational [Page 8] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 } Example: Consider the alphabet ABCDEFGH, with bit lengths (3, 3, 3, 3, 3, 2, 4, 4). After step 1, we have: N bl_count[N] - ----------- 2 1 3 5 4 2 Step 2 computes the following next_code values: N next_code[N] - ------------ 1 0 2 0 3 2 4 14 Step 3 produces the following code values: Symbol Length Code ------ ------ ---- A 3 010 B 3 011 C 3 100 D 3 101 E 3 110 F 2 00 G 4 1110 H 4 1111 3.2.3. Details of block format Each block of compressed data begins with 3 header bits containing the following data: first bit BFINAL next 2 bits BTYPE Note that the header bits do not necessarily begin on a byte boundary, since a block does not necessarily occupy an integral number of bytes. Deutsch Informational [Page 9] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 BFINAL is set if and only if this is the last block of the data set. BTYPE specifies how the data are compressed, as follows: 00 - no compression 01 - compressed with fixed Huffman codes 10 - compressed with dynamic Huffman codes 11 - reserved (error) The only difference between the two compressed cases is how the Huffman codes for the literal/length and distance alphabets are defined. In all cases, the decoding algorithm for the actual data is as follows: do read block header from input stream. if stored with no compression skip any remaining bits in current partially processed byte read LEN and NLEN (see next section) copy LEN bytes of data to output otherwise if compressed with dynamic Huffman codes read representation of code trees (see subsection below) loop (until end of block code recognized) decode literal/length value from input stream if value < 256 copy value (literal byte) to output stream otherwise if value = end of block (256) break from loop otherwise (value = 257..285) decode distance from input stream move backwards distance bytes in the output stream, and copy length bytes from this position to the output stream. end loop while not last block Note that a duplicated string reference may refer to a string in a previous block; i.e., the backward distance may cross one or more block boundaries. However a distance cannot refer past the beginning of the output stream. (An application using a Deutsch Informational [Page 10] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 preset dictionary might discard part of the output stream; a distance can refer to that part of the output stream anyway) Note also that the referenced string may overlap the current position; for example, if the last 2 bytes decoded have values X and Y, a string reference with adds X,Y,X,Y,X to the output stream. We now specify each compression method in turn. 3.2.4. Non-compressed blocks (BTYPE=00) Any bits of input up to the next byte boundary are ignored. The rest of the block consists of the following information: 0 1 2 3 4... +---+---+---+---+================================+ | LEN | NLEN |... LEN bytes of literal data...| +---+---+---+---+================================+ LEN is the number of data bytes in the block. NLEN is the one's complement of LEN. 3.2.5. Compressed blocks (length and distance codes) As noted above, encoded data blocks in the "deflate" format consist of sequences of symbols drawn from three conceptually distinct alphabets: either literal bytes, from the alphabet of byte values (0..255), or pairs, where the length is drawn from (3..258) and the distance is drawn from (1..32,768). In fact, the literal and length alphabets are merged into a single alphabet (0..285), where values 0..255 represent literal bytes, the value 256 indicates end-of-block, and values 257..285 represent length codes (possibly in conjunction with extra bits following the symbol code) as follows: Deutsch Informational [Page 11] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 Extra Extra Extra Code Bits Length(s) Code Bits Lengths Code Bits Length(s) ---- ---- ------ ---- ---- ------- ---- ---- ------- 257 0 3 267 1 15,16 277 4 67-82 258 0 4 268 1 17,18 278 4 83-98 259 0 5 269 2 19-22 279 4 99-114 260 0 6 270 2 23-26 280 4 115-130 261 0 7 271 2 27-30 281 5 131-162 262 0 8 272 2 31-34 282 5 163-194 263 0 9 273 3 35-42 283 5 195-226 264 0 10 274 3 43-50 284 5 227-257 265 1 11,12 275 3 51-58 285 0 258 266 1 13,14 276 3 59-66 The extra bits should be interpreted as a machine integer stored with the most-significant bit first, e.g., bits 1110 represent the value 14. Extra Extra Extra Code Bits Dist Code Bits Dist Code Bits Distance ---- ---- ---- ---- ---- ------ ---- ---- -------- 0 0 1 10 4 33-48 20 9 1025-1536 1 0 2 11 4 49-64 21 9 1537-2048 2 0 3 12 5 65-96 22 10 2049-3072 3 0 4 13 5 97-128 23 10 3073-4096 4 1 5,6 14 6 129-192 24 11 4097-6144 5 1 7,8 15 6 193-256 25 11 6145-8192 6 2 9-12 16 7 257-384 26 12 8193-12288 7 2 13-16 17 7 385-512 27 12 12289-16384 8 3 17-24 18 8 513-768 28 13 16385-24576 9 3 25-32 19 8 769-1024 29 13 24577-32768 3.2.6. Compression with fixed Huffman codes (BTYPE=01) The Huffman codes for the two alphabets are fixed, and are not represented explicitly in the data. The Huffman code lengths for the literal/length alphabet are: Lit Value Bits Codes --------- ---- ----- 0 - 143 8 00110000 through 10111111 144 - 255 9 110010000 through 111111111 256 - 279 7 0000000 through 0010111 280 - 287 8 11000000 through 11000111 Deutsch Informational [Page 12] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 The code lengths are sufficient to generate the actual codes, as described above; we show the codes in the table for added clarity. Literal/length values 286-287 will never actually occur in the compressed data, but participate in the code construction. Distance codes 0-31 are represented by (fixed-length) 5-bit codes, with possible additional bits as shown in the table shown in Paragraph 3.2.5, above. Note that distance codes 30- 31 will never actually occur in the compressed data. 3.2.7. Compression with dynamic Huffman codes (BTYPE=10) The Huffman codes for the two alphabets appear in the block immediately after the header bits and before the actual compressed data, first the literal/length code and then the distance code. Each code is defined by a sequence of code lengths, as discussed in Paragraph 3.2.2, above. For even greater compactness, the code length sequences themselves are compressed using a Huffman code. The alphabet for code lengths is as follows: 0 - 15: Represent code lengths of 0 - 15 16: Copy the previous code length 3 - 6 times. The next 2 bits indicate repeat length (0 = 3, ... , 3 = 6) Example: Codes 8, 16 (+2 bits 11), 16 (+2 bits 10) will expand to 12 code lengths of 8 (1 + 6 + 5) 17: Repeat a code length of 0 for 3 - 10 times. (3 bits of length) 18: Repeat a code length of 0 for 11 - 138 times (7 bits of length) A code length of 0 indicates that the corresponding symbol in the literal/length or distance alphabet will not occur in the block, and should not participate in the Huffman code construction algorithm given earlier. If only one distance code is used, it is encoded using one bit, not zero bits; in this case there is a single code length of one, with one unused code. One distance code of zero bits means that there are no distance codes used at all (the data is all literals). We can now define the format of the block: 5 Bits: HLIT, # of Literal/Length codes - 257 (257 - 286) 5 Bits: HDIST, # of Distance codes - 1 (1 - 32) 4 Bits: HCLEN, # of Code Length codes - 4 (4 - 19) Deutsch Informational [Page 13] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 (HCLEN + 4) x 3 bits: code lengths for the code length alphabet given just above, in the order: 16, 17, 18, 0, 8, 7, 9, 6, 10, 5, 11, 4, 12, 3, 13, 2, 14, 1, 15 These code lengths are interpreted as 3-bit integers (0-7); as above, a code length of 0 means the corresponding symbol (literal/length or distance code length) is not used. HLIT + 257 code lengths for the literal/length alphabet, encoded using the code length Huffman code HDIST + 1 code lengths for the distance alphabet, encoded using the code length Huffman code The actual compressed data of the block, encoded using the literal/length and distance Huffman codes The literal/length symbol 256 (end of data), encoded using the literal/length Huffman code The code length repeat codes can cross from HLIT + 257 to the HDIST + 1 code lengths. In other words, all code lengths form a single sequence of HLIT + HDIST + 258 values. 3.3. Compliance A compressor may limit further the ranges of values specified in the previous section and still be compliant; for example, it may limit the range of backward pointers to some value smaller than 32K. Similarly, a compressor may limit the size of blocks so that a compressible block fits in memory. A compliant decompressor must accept the full range of possible values defined in the previous section, and must accept blocks of arbitrary size. 4. Compression algorithm details While it is the intent of this document to define the "deflate" compressed data format without reference to any particular compression algorithm, the format is related to the compressed formats produced by LZ77 (Lempel-Ziv 1977, see reference [2] below); since many variations of LZ77 are patented, it is strongly recommended that the implementor of a compressor follow the general algorithm presented here, which is known not to be patented per se. The material in this section is not part of the definition of the Deutsch Informational [Page 14] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 specification per se, and a compressor need not follow it in order to be compliant. The compressor terminates a block when it determines that starting a new block with fresh trees would be useful, or when the block size fills up the compressor's block buffer. The compressor uses a chained hash table to find duplicated strings, using a hash function that operates on 3-byte sequences. At any given point during compression, let XYZ be the next 3 input bytes to be examined (not necessarily all different, of course). First, the compressor examines the hash chain for XYZ. If the chain is empty, the compressor simply writes out X as a literal byte and advances one byte in the input. If the hash chain is not empty, indicating that the sequence XYZ (or, if we are unlucky, some other 3 bytes with the same hash function value) has occurred recently, the compressor compares all strings on the XYZ hash chain with the actual input data sequence starting at the current point, and selects the longest match. The compressor searches the hash chains starting with the most recent strings, to favor small distances and thus take advantage of the Huffman encoding. The hash chains are singly linked. There are no deletions from the hash chains; the algorithm simply discards matches that are too old. To avoid a worst-case situation, very long hash chains are arbitrarily truncated at a certain length, determined by a run-time parameter. To improve overall compression, the compressor optionally defers the selection of matches ("lazy matching"): after a match of length N has been found, the compressor searches for a longer match starting at the next input byte. If it finds a longer match, it truncates the previous match to a length of one (thus producing a single literal byte) and then emits the longer match. Otherwise, it emits the original match, and, as described above, advances N bytes before continuing. Run-time parameters also control this "lazy match" procedure. If compression ratio is most important, the compressor attempts a complete second search regardless of the length of the first match. In the normal case, if the current match is "long enough", the compressor reduces the search for a longer match, thus speeding up the process. If speed is most important, the compressor inserts new strings in the hash table only when no match was found, or when the match is not "too long". This degrades the compression ratio but saves time since there are both fewer insertions and fewer searches. Deutsch Informational [Page 15] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 5. References [1] Huffman, D. A., "A Method for the Construction of Minimum Redundancy Codes", Proceedings of the Institute of Radio Engineers, September 1952, Volume 40, Number 9, pp. 1098-1101. [2] Ziv J., Lempel A., "A Universal Algorithm for Sequential Data Compression", IEEE Transactions on Information Theory, Vol. 23, No. 3, pp. 337-343. [3] Gailly, J.-L., and Adler, M., ZLIB documentation and sources, available in ftp://ftp.uu.net/pub/archiving/zip/doc/ [4] Gailly, J.-L., and Adler, M., GZIP documentation and sources, available as gzip-*.tar in ftp://prep.ai.mit.edu/pub/gnu/ [5] Schwartz, E. S., and Kallick, B. "Generating a canonical prefix encoding." Comm. ACM, 7,3 (Mar. 1964), pp. 166-169. [6] Hirschberg and Lelewer, "Efficient decoding of prefix codes," Comm. ACM, 33,4, April 1990, pp. 449-459. 6. Security Considerations Any data compression method involves the reduction of redundancy in the data. Consequently, any corruption of the data is likely to have severe effects and be difficult to correct. Uncompressed text, on the other hand, will probably still be readable despite the presence of some corrupted bytes. It is recommended that systems using this data format provide some means of validating the integrity of the compressed data. See reference [3], for example. 7. Source code Source code for a C language implementation of a "deflate" compliant compressor and decompressor is available within the zlib package at ftp://ftp.uu.net/pub/archiving/zip/zlib/. 8. Acknowledgements Trademarks cited in this document are the property of their respective owners. Phil Katz designed the deflate format. Jean-Loup Gailly and Mark Adler wrote the related software described in this specification. Glenn Randers-Pehrson converted this document to RFC and HTML format. Deutsch Informational [Page 16] RFC 1951 DEFLATE Compressed Data Format Specification May 1996 9. Author's Address L. Peter Deutsch Aladdin Enterprises 203 Santa Margarita Ave. Menlo Park, CA 94025 Phone: (415) 322-0103 (AM only) FAX: (415) 322-1734 EMail: Questions about the technical content of this specification can be sent by email to: Jean-Loup Gailly and Mark Adler Editorial comments on this specification can be sent by email to: L. Peter Deutsch and Glenn Randers-Pehrson Deutsch Informational [Page 17] --- ### Vs Zlib Doc Rfc1952 (`vs/zlib/doc/rfc1952.txt`) Network Working Group P. Deutsch Request for Comments: 1952 Aladdin Enterprises Category: Informational May 1996 GZIP file format specification version 4.3 Status of This Memo This memo provides information for the Internet community. This memo does not specify an Internet standard of any kind. Distribution of this memo is unlimited. IESG Note: The IESG takes no position on the validity of any Intellectual Property Rights statements contained in this document. Notices Copyright (c) 1996 L. Peter Deutsch Permission is granted to copy and distribute this document for any purpose and without charge, including translations into other languages and incorporation into compilations, provided that the copyright notice and this notice are preserved, and that any substantive changes or deletions from the original are clearly marked. A pointer to the latest version of this and related documentation in HTML format can be found at the URL . Abstract This specification defines a lossless compressed data format that is compatible with the widely used GZIP utility. The format includes a cyclic redundancy check value for detecting data corruption. The format presently uses the DEFLATE method of compression but can be easily extended to use other compression methods. The format can be implemented readily in a manner not covered by patents. Deutsch Informational [Page 1] RFC 1952 GZIP File Format Specification May 1996 Table of Contents 1. Introduction ................................................... 2 1.1. Purpose ................................................... 2 1.2. Intended audience ......................................... 3 1.3. Scope ..................................................... 3 1.4. Compliance ................................................ 3 1.5. Definitions of terms and conventions used ................. 3 1.6. Changes from previous versions ............................ 3 2. Detailed specification ......................................... 4 2.1. Overall conventions ....................................... 4 2.2. File format ............................................... 5 2.3. Member format ............................................. 5 2.3.1. Member header and trailer ........................... 6 2.3.1.1. Extra field ................................... 8 2.3.1.2. Compliance .................................... 9 3. References .................................................. 9 4. Security Considerations .................................... 10 5. Acknowledgements ........................................... 10 6. Author's Address ........................................... 10 7. Appendix: Jean-Loup Gailly's gzip utility .................. 11 8. Appendix: Sample CRC Code .................................. 11 1. Introduction 1.1. Purpose The purpose of this specification is to define a lossless compressed data format that: * Is independent of CPU type, operating system, file system, and character set, and hence can be used for interchange; * Can compress or decompress a data stream (as opposed to a randomly accessible file) to produce another data stream, using only an a priori bounded amount of intermediate storage, and hence can be used in data communications or similar structures such as Unix filters; * Compresses data with efficiency comparable to the best currently available general-purpose compression methods, and in particular considerably better than the "compress" program; * Can be implemented readily in a manner not covered by patents, and hence can be practiced freely; * Is compatible with the file format produced by the current widely used gzip utility, in that conforming decompressors will be able to read data produced by the existing gzip compressor. Deutsch Informational [Page 2] RFC 1952 GZIP File Format Specification May 1996 The data format defined by this specification does not attempt to: * Provide random access to compressed data; * Compress specialized data (e.g., raster graphics) as well as the best currently available specialized algorithms. 1.2. Intended audience This specification is intended for use by implementors of software to compress data into gzip format and/or decompress data from gzip format. The text of the specification assumes a basic background in programming at the level of bits and other primitive data representations. 1.3. Scope The specification specifies a compression method and a file format (the latter assuming only that a file can store a sequence of arbitrary bytes). It does not specify any particular interface to a file system or anything about character sets or encodings (except for file names and comments, which are optional). 1.4. Compliance Unless otherwise indicated below, a compliant decompressor must be able to accept and decompress any file that conforms to all the specifications presented here; a compliant compressor must produce files that conform to all the specifications presented here. The material in the appendices is not part of the specification per se and is not relevant to compliance. 1.5. Definitions of terms and conventions used byte: 8 bits stored or transmitted as a unit (same as an octet). (For this specification, a byte is exactly 8 bits, even on machines which store a character on a number of bits different from 8.) See below for the numbering of bits within a byte. 1.6. Changes from previous versions There have been no technical changes to the gzip format since version 4.1 of this specification. In version 4.2, some terminology was changed, and the sample CRC code was rewritten for clarity and to eliminate the requirement for the caller to do pre- and post-conditioning. Version 4.3 is a conversion of the specification to RFC style. Deutsch Informational [Page 3] RFC 1952 GZIP File Format Specification May 1996 2. Detailed specification 2.1. Overall conventions In the diagrams below, a box like this: +---+ | | <-- the vertical bars might be missing +---+ represents one byte; a box like this: +==============+ | | +==============+ represents a variable number of bytes. Bytes stored within a computer do not have a "bit order", since they are always treated as a unit. However, a byte considered as an integer between 0 and 255 does have a most- and least- significant bit, and since we write numbers with the most- significant digit on the left, we also write bytes with the most- significant bit on the left. In the diagrams below, we number the bits of a byte so that bit 0 is the least-significant bit, i.e., the bits are numbered: +--------+ |76543210| +--------+ This document does not address the issue of the order in which bits of a byte are transmitted on a bit-sequential medium, since the data format described here is byte- rather than bit-oriented. Within a computer, a number may occupy multiple bytes. All multi-byte numbers in the format described here are stored with the least-significant byte first (at the lower memory address). For example, the decimal number 520 is stored as: 0 1 +--------+--------+ |00001000|00000010| +--------+--------+ ^ ^ | | | + more significant byte = 2 x 256 + less significant byte = 8 Deutsch Informational [Page 4] RFC 1952 GZIP File Format Specification May 1996 2.2. File format A gzip file consists of a series of "members" (compressed data sets). The format of each member is specified in the following section. The members simply appear one after another in the file, with no additional information before, between, or after them. 2.3. Member format Each member has the following structure: +---+---+---+---+---+---+---+---+---+---+ |ID1|ID2|CM |FLG| MTIME |XFL|OS | (more-->) +---+---+---+---+---+---+---+---+---+---+ (if FLG.FEXTRA set) +---+---+=================================+ | XLEN |...XLEN bytes of "extra field"...| (more-->) +---+---+=================================+ (if FLG.FNAME set) +=========================================+ |...original file name, zero-terminated...| (more-->) +=========================================+ (if FLG.FCOMMENT set) +===================================+ |...file comment, zero-terminated...| (more-->) +===================================+ (if FLG.FHCRC set) +---+---+ | CRC16 | +---+---+ +=======================+ |...compressed blocks...| (more-->) +=======================+ 0 1 2 3 4 5 6 7 +---+---+---+---+---+---+---+---+ | CRC32 | ISIZE | +---+---+---+---+---+---+---+---+ Deutsch Informational [Page 5] RFC 1952 GZIP File Format Specification May 1996 2.3.1. Member header and trailer ID1 (IDentification 1) ID2 (IDentification 2) These have the fixed values ID1 = 31 (0x1f, \037), ID2 = 139 (0x8b, \213), to identify the file as being in gzip format. CM (Compression Method) This identifies the compression method used in the file. CM = 0-7 are reserved. CM = 8 denotes the "deflate" compression method, which is the one customarily used by gzip and which is documented elsewhere. FLG (FLaGs) This flag byte is divided into individual bits as follows: bit 0 FTEXT bit 1 FHCRC bit 2 FEXTRA bit 3 FNAME bit 4 FCOMMENT bit 5 reserved bit 6 reserved bit 7 reserved If FTEXT is set, the file is probably ASCII text. This is an optional indication, which the compressor may set by checking a small amount of the input data to see whether any non-ASCII characters are present. In case of doubt, FTEXT is cleared, indicating binary data. For systems which have different file formats for ascii text and binary data, the decompressor can use FTEXT to choose the appropriate format. We deliberately do not specify the algorithm used to set this bit, since a compressor always has the option of leaving it cleared and a decompressor always has the option of ignoring it and letting some other program handle issues of data conversion. If FHCRC is set, a CRC16 for the gzip header is present, immediately before the compressed data. The CRC16 consists of the two least significant bytes of the CRC32 for all bytes of the gzip header up to and not including the CRC16. [The FHCRC bit was never set by versions of gzip up to 1.2.4, even though it was documented with a different meaning in gzip 1.2.4.] If FEXTRA is set, optional extra fields are present, as described in a following section. Deutsch Informational [Page 6] RFC 1952 GZIP File Format Specification May 1996 If FNAME is set, an original file name is present, terminated by a zero byte. The name must consist of ISO 8859-1 (LATIN-1) characters; on operating systems using EBCDIC or any other character set for file names, the name must be translated to the ISO LATIN-1 character set. This is the original name of the file being compressed, with any directory components removed, and, if the file being compressed is on a file system with case insensitive names, forced to lower case. There is no original file name if the data was compressed from a source other than a named file; for example, if the source was stdin on a Unix system, there is no file name. If FCOMMENT is set, a zero-terminated file comment is present. This comment is not interpreted; it is only intended for human consumption. The comment must consist of ISO 8859-1 (LATIN-1) characters. Line breaks should be denoted by a single line feed character (10 decimal). Reserved FLG bits must be zero. MTIME (Modification TIME) This gives the most recent modification time of the original file being compressed. The time is in Unix format, i.e., seconds since 00:00:00 GMT, Jan. 1, 1970. (Note that this may cause problems for MS-DOS and other systems that use local rather than Universal time.) If the compressed data did not come from a file, MTIME is set to the time at which compression started. MTIME = 0 means no time stamp is available. XFL (eXtra FLags) These flags are available for use by specific compression methods. The "deflate" method (CM = 8) sets these flags as follows: XFL = 2 - compressor used maximum compression, slowest algorithm XFL = 4 - compressor used fastest algorithm OS (Operating System) This identifies the type of file system on which compression took place. This may be useful in determining end-of-line convention for text files. The currently defined values are as follows: Deutsch Informational [Page 7] RFC 1952 GZIP File Format Specification May 1996 0 - FAT filesystem (MS-DOS, OS/2, NT/Win32) 1 - Amiga 2 - VMS (or OpenVMS) 3 - Unix 4 - VM/CMS 5 - Atari TOS 6 - HPFS filesystem (OS/2, NT) 7 - Macintosh 8 - Z-System 9 - CP/M 10 - TOPS-20 11 - NTFS filesystem (NT) 12 - QDOS 13 - Acorn RISCOS 255 - unknown XLEN (eXtra LENgth) If FLG.FEXTRA is set, this gives the length of the optional extra field. See below for details. CRC32 (CRC-32) This contains a Cyclic Redundancy Check value of the uncompressed data computed according to CRC-32 algorithm used in the ISO 3309 standard and in section 8.1.1.6.2 of ITU-T recommendation V.42. (See http://www.iso.ch for ordering ISO documents. See gopher://info.itu.ch for an online version of ITU-T V.42.) ISIZE (Input SIZE) This contains the size of the original (uncompressed) input data modulo 2^32. 2.3.1.1. Extra field If the FLG.FEXTRA bit is set, an "extra field" is present in the header, with total length XLEN bytes. It consists of a series of subfields, each of the form: +---+---+---+---+==================================+ |SI1|SI2| LEN |... LEN bytes of subfield data ...| +---+---+---+---+==================================+ SI1 and SI2 provide a subfield ID, typically two ASCII letters with some mnemonic value. Jean-Loup Gailly is maintaining a registry of subfield IDs; please send him any subfield ID you wish to use. Subfield IDs with SI2 = 0 are reserved for future use. The following IDs are currently defined: Deutsch Informational [Page 8] RFC 1952 GZIP File Format Specification May 1996 SI1 SI2 Data ---------- ---------- ---- 0x41 ('A') 0x70 ('P') Apollo file type information LEN gives the length of the subfield data, excluding the 4 initial bytes. 2.3.1.2. Compliance A compliant compressor must produce files with correct ID1, ID2, CM, CRC32, and ISIZE, but may set all the other fields in the fixed-length part of the header to default values (255 for OS, 0 for all others). The compressor must set all reserved bits to zero. A compliant decompressor must check ID1, ID2, and CM, and provide an error indication if any of these have incorrect values. It must examine FEXTRA/XLEN, FNAME, FCOMMENT and FHCRC at least so it can skip over the optional fields if they are present. It need not examine any other part of the header or trailer; in particular, a decompressor may ignore FTEXT and OS and always produce binary output, and still be compliant. A compliant decompressor must give an error indication if any reserved bit is non-zero, since such a bit could indicate the presence of a new field that would cause subsequent data to be interpreted incorrectly. 3. References [1] "Information Processing - 8-bit single-byte coded graphic character sets - Part 1: Latin alphabet No.1" (ISO 8859-1:1987). The ISO 8859-1 (Latin-1) character set is a superset of 7-bit ASCII. Files defining this character set are available as iso_8859-1.* in ftp://ftp.uu.net/graphics/png/documents/ [2] ISO 3309 [3] ITU-T recommendation V.42 [4] Deutsch, L.P.,"DEFLATE Compressed Data Format Specification", available in ftp://ftp.uu.net/pub/archiving/zip/doc/ [5] Gailly, J.-L., GZIP documentation, available as gzip-*.tar in ftp://prep.ai.mit.edu/pub/gnu/ [6] Sarwate, D.V., "Computation of Cyclic Redundancy Checks via Table Look-Up", Communications of the ACM, 31(8), pp.1008-1013. Deutsch Informational [Page 9] RFC 1952 GZIP File Format Specification May 1996 [7] Schwaderer, W.D., "CRC Calculation", April 85 PC Tech Journal, pp.118-133. [8] ftp://ftp.adelaide.edu.au/pub/rocksoft/papers/crc_v3.txt, describing the CRC concept. 4. Security Considerations Any data compression method involves the reduction of redundancy in the data. Consequently, any corruption of the data is likely to have severe effects and be difficult to correct. Uncompressed text, on the other hand, will probably still be readable despite the presence of some corrupted bytes. It is recommended that systems using this data format provide some means of validating the integrity of the compressed data, such as by setting and checking the CRC-32 check value. 5. Acknowledgements Trademarks cited in this document are the property of their respective owners. Jean-Loup Gailly designed the gzip format and wrote, with Mark Adler, the related software described in this specification. Glenn Randers-Pehrson converted this document to RFC and HTML format. 6. Author's Address L. Peter Deutsch Aladdin Enterprises 203 Santa Margarita Ave. Menlo Park, CA 94025 Phone: (415) 322-0103 (AM only) FAX: (415) 322-1734 EMail: Questions about the technical content of this specification can be sent by email to: Jean-Loup Gailly and Mark Adler Editorial comments on this specification can be sent by email to: L. Peter Deutsch and Glenn Randers-Pehrson Deutsch Informational [Page 10] RFC 1952 GZIP File Format Specification May 1996 7. Appendix: Jean-Loup Gailly's gzip utility The most widely used implementation of gzip compression, and the original documentation on which this specification is based, were created by Jean-Loup Gailly . Since this implementation is a de facto standard, we mention some more of its features here. Again, the material in this section is not part of the specification per se, and implementations need not follow it to be compliant. When compressing or decompressing a file, gzip preserves the protection, ownership, and modification time attributes on the local file system, since there is no provision for representing protection attributes in the gzip file format itself. Since the file format includes a modification time, the gzip decompressor provides a command line switch that assigns the modification time from the file, rather than the local modification time of the compressed input, to the decompressed output. 8. Appendix: Sample CRC Code The following sample code represents a practical implementation of the CRC (Cyclic Redundancy Check). (See also ISO 3309 and ITU-T V.42 for a formal specification.) The sample code is in the ANSI C programming language. Non C users may find it easier to read with these hints: & Bitwise AND operator. ^ Bitwise exclusive-OR operator. >> Bitwise right shift operator. When applied to an unsigned quantity, as here, right shift inserts zero bit(s) at the left. ! Logical NOT operator. ++ "n++" increments the variable n. 0xNNN 0x introduces a hexadecimal (base 16) constant. Suffix L indicates a long value (at least 32 bits). /* Table of CRCs of all 8-bit messages. */ unsigned long crc_table[256]; /* Flag: has the table been computed? Initially false. */ int crc_table_computed = 0; /* Make the table for a fast CRC. */ void make_crc_table(void) { unsigned long c; Deutsch Informational [Page 11] RFC 1952 GZIP File Format Specification May 1996 int n, k; for (n = 0; n < 256; n++) { c = (unsigned long) n; for (k = 0; k < 8; k++) { if (c & 1) { c = 0xedb88320L ^ (c >> 1); } else { c = c >> 1; } } crc_table[n] = c; } crc_table_computed = 1; } /* Update a running crc with the bytes buf[0..len-1] and return the updated crc. The crc should be initialized to zero. Pre- and post-conditioning (one's complement) is performed within this function so it shouldn't be done by the caller. Usage example: unsigned long crc = 0L; while (read_buffer(buffer, length) != EOF) { crc = update_crc(crc, buffer, length); } if (crc != original_crc) error(); */ unsigned long update_crc(unsigned long crc, unsigned char *buf, int len) { unsigned long c = crc ^ 0xffffffffL; int n; if (!crc_table_computed) make_crc_table(); for (n = 0; n < len; n++) { c = crc_table[(c ^ buf[n]) & 0xff] ^ (c >> 8); } return c ^ 0xffffffffL; } /* Return the CRC of the bytes buf[0..len-1]. */ unsigned long crc(unsigned char *buf, int len) { return update_crc(0L, buf, len); } Deutsch Informational [Page 12] --- ### Vs Zlib Doc Txtvsbin (`vs/zlib/doc/txtvsbin.txt`) A Fast Method for Identifying Plain Text Files ============================================== Introduction ------------ Given a file coming from an unknown source, it is sometimes desirable to find out whether the format of that file is plain text. Although this may appear like a simple task, a fully accurate detection of the file type requires heavy-duty semantic analysis on the file contents. It is, however, possible to obtain satisfactory results by employing various heuristics. Previous versions of PKZip and other zip-compatible compression tools were using a crude detection scheme: if more than 80% (4/5) of the bytes found in a certain buffer are within the range [7..127], the file is labeled as plain text, otherwise it is labeled as binary. A prominent limitation of this scheme is the restriction to Latin-based alphabets. Other alphabets, like Greek, Cyrillic or Asian, make extensive use of the bytes within the range [128..255], and texts using these alphabets are most often misidentified by this scheme; in other words, the rate of false negatives is sometimes too high, which means that the recall is low. Another weakness of this scheme is a reduced precision, due to the false positives that may occur when binary files containing large amounts of textual characters are misidentified as plain text. In this article we propose a new, simple detection scheme that features a much increased precision and a near-100% recall. This scheme is designed to work on ASCII, Unicode and other ASCII-derived alphabets, and it handles single-byte encodings (ISO-8859, MacRoman, KOI8, etc.) and variable-sized encodings (ISO-2022, UTF-8, etc.). Wider encodings (UCS-2/UTF-16 and UCS-4/UTF-32) are not handled, however. The Algorithm ------------- The algorithm works by dividing the set of bytecodes [0..255] into three categories: - The allow list of textual bytecodes: 9 (TAB), 10 (LF), 13 (CR), 32 (SPACE) to 255. - The gray list of tolerated bytecodes: 7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB), 27 (ESC). - The block list of undesired, non-textual bytecodes: 0 (NUL) to 6, 14 to 31. If a file contains at least one byte that belongs to the allow list and no byte that belongs to the block list, then the file is categorized as plain text; otherwise, it is categorized as binary. (The boundary case, when the file is empty, automatically falls into the latter category.) Rationale --------- The idea behind this algorithm relies on two observations. The first observation is that, although the full range of 7-bit codes [0..127] is properly specified by the ASCII standard, most control characters in the range [0..31] are not used in practice. The only widely-used, almost universally-portable control codes are 9 (TAB), 10 (LF) and 13 (CR). There are a few more control codes that are recognized on a reduced range of platforms and text viewers/editors: 7 (BEL), 8 (BS), 11 (VT), 12 (FF), 26 (SUB) and 27 (ESC); but these codes are rarely (if ever) used alone, without being accompanied by some printable text. Even the newer, portable text formats such as XML avoid using control characters outside the list mentioned here. The second observation is that most of the binary files tend to contain control characters, especially 0 (NUL). Even though the older text detection schemes observe the presence of non-ASCII codes from the range [128..255], the precision rarely has to suffer if this upper range is labeled as textual, because the files that are genuinely binary tend to contain both control characters and codes from the upper range. On the other hand, the upper range needs to be labeled as textual, because it is used by virtually all ASCII extensions. In particular, this range is used for encoding non-Latin scripts. Since there is no counting involved, other than simply observing the presence or the absence of some byte values, the algorithm produces consistent results, regardless what alphabet encoding is being used. (If counting were involved, it could be possible to obtain different results on a text encoded, say, using ISO-8859-16 versus UTF-8.) There is an extra category of plain text files that are "polluted" with one or more block-listed codes, either by mistake or by peculiar design considerations. In such cases, a scheme that tolerates a small fraction of block-listed codes would provide an increased recall (i.e. more true positives). This, however, incurs a reduced precision overall, since false positives are more likely to appear in binary files that contain large chunks of textual data. Furthermore, "polluted" plain text should be regarded as binary by general-purpose text detection schemes, because general-purpose text processing algorithms might not be applicable. Under this premise, it is safe to say that our detection method provides a near-100% recall. Experiments have been run on many files coming from various platforms and applications. We tried plain text files, system logs, source code, formatted office documents, compiled object code, etc. The results confirm the optimistic assumptions about the capabilities of this algorithm. -- Cosmin Truta Last updated: 2006-May-28 --- --- METRICS --- - Files Extracted: 62 - Estimated Token Budget: ~132219 tokens - Canonical Reference: https://codewiki.google/github.com/joncampbell123/dosbox-x