## 1. Project Overview & Quickstart (chaiNNer-org/data-representation) # Technical Documentation: chaiNNer-org/data-representation > ℹī¸ **Provenance:** Multi-Tier Ingestion ¡ [CodeWiki Reference](https://codewiki.google/github.com/chaiNNer-org/data-representation) # data-representation ### Agent Skills & Rules - Discovered 21 agent rule(s) and prompt configurations. --- METRICS --- - Files Extracted: 1 - Estimated Token Budget: ~27 tokens - Canonical Reference: https://codewiki.google/github.com/chaiNNer-org/data-representation ## 2. Official Technical Reference & Guides (chaiNNer-org/chaiNNer-org.github.io) ## File: src/content/posts/alpha-v0-1-0.md --- title: Alpha v0.1.0 date: 2022-01-17 tag: v0.1.0 summary: The initial alpha release, and the first day of chaiNNer being public. source: discord --- chaiNNer is now public! You can find the release on GitHub. A flowchart/node-based image processing GUI aimed at making chaining image processing tasks (especially those done by neural networks) easy, intuitive, and customizable. ## Update 1 Thanks for the interest in chaiNNer! Just a reminder that this is still pretty early in development so expect there to be issues. Specifically if it seems to be stuck loading on the splash screen (a common problem) let me know ASAP. ## Update 2 Thanks everyone for a good launch day! As I anticipated, a few issues popped up early on but luckily the major ones are fixed now and I have plans to fix the others. I'm thankful for all the kind words I've received so far and I can't wait to continue adding more to chaiNNer 🙂 --- ## File: src/content/posts/alpha-v0-3-0.md --- title: Alpha v0.3.0 date: 2022-03-07 tag: v0.3.0 summary: NCNN support brings upscaling to AMD and Intel GPUs, and every node gets renamed and recategorized around what it does rather than what implements it. source: discord --- There's two really big features in this build: NCNN support and a node redesign. It might not seem like it, but this is my biggest update yet with a few pretty big features and changes. ## NCNN support The biggest by far is NCNN support. If you don't know what that means, it just means that far more devices (AMD and Intel GPUs that support Vulkan) will be able to do upscaling. This is huge and puts chaiNNer closer to feature parity with other GUIs. If you're wondering why this took so long, I basically had to figure out myself (with the help of styler00dollar) how to get Vulkan working with the official NCNN Python api, and I ended up having to publish an entirely separate package to PyPi to get it working. ## A node redesign The other big change in this release is a complete redesign of how nodes are named and categorized. I decided that the existing way was too technical and that I should make everything a bit more understandable for the end-user, rather than over-complicating things. So for example, nodes aren't categorized by "OpenCV" or "NumPy" anymore, but rather "Image" and "Image (Utility)", and node names are no longer "Image::Read" and "Image::Show" but rather "Load Image" and "Preview Image". You also might notice each node also now has an icon associated with it instead of each category all being the same icon. Under the hood, many of these are exactly the same, there's just a new coat of paint on top. And don't worry, old chain files (.chn) will be automatically converted to this new format. More is still to come. --- ## File: src/content/posts/alpha-v0-5-0.md --- title: Alpha v0.5.0 date: 2022-04-05 tag: v0.5.0 summary: Batch upscaling arrives, through a new kind of node called an iterator, along with DDS and TGA support via Pillow. source: discord --- Batch Upscaling is finally here! This is chaiNNer's biggest update yet with this long overdue, highly requested feature being the center of attention. ## Iterators Batch upscaling is performed using an "Image File Iterator" node. Iterators are a new type of node in chaiNNer that allow you to run the nodes inside the iterator on multiple items. Right now, the only iterator is the "Image File Iterator" which takes a folder input and runs the sub-nodes on each image file in the folder (and its sub-folders). This means you can finally perform batch upscaling in chaiNNer! They can be a bit tricky to get used to, so if you have any questions about how to use them, don't hesitate to ask in the Discord server. ## More file types Along with Iterators, DDS, TGA, and many more filetypes are now supported when you install the Pillow dependency in the dependency manager. You still cannot save images as these filetypes at the moment, but at least this should help when upscaling game textures. This was another highly requested feature that I am happy to finally have in chaiNNer. For Iterators especially, please let me know as soon as possible if you run into any issues with them, and I will try to get those issues fixed as soon as I can. --- ## File: src/content/posts/alpha-v0-10-0.md --- title: Alpha v0.10.0 date: 2022-07-14 tag: v0.10.0 summary: The Navi type system, colored handles that show you what connects to what, a node search context menu, and the ability to disable nodes. source: discord --- Huge improvements! This might be the biggest update yet. This is a pretty exciting update for chaiNNer! The first thing you might notice is all the new colors, but it's a lot more than just a new look. As a result, I will be going over this changelog in a bit more detail than usual. ## New Type System This update features a major overhaul of types. If you aren't a programmer or just aren't familiar with types, they're the reason chaiNNer doesn't let you connect an image output to a model input. Model and images are different types, so they aren't compatible. Previously, types were extemely simple. Without getting too technical, types were simply the name of the type (e.g. model or image) and types were compatible if they had the same name. Now, chaiNNer has a brand new type system called "Navi" with set-theorey-based types. These types are much more powerful and enable lots of things that would have been impossible before. E.g. inputs can now accept multiple types, and image inputs can now require the connected image to have a certain number of channels (e.g. RGB or RGBA). Per-node validation has also been improved, and now we are able to show you the output size of almost every image output without even having to run anything. So you might be wondering to yourself: "I didn't even know types were a thing before, I just kept trying to connect things and guessing what went where. How exactly does this make anything better for me?" Well, wonder no more. Each handle now has a dedicated color for each main type. This means that with a quick glance, you can now see what can be connected to what. E.g. images are yellow, numbers are blue, PyTorch models are orange. It should be pretty easy now to be able to see what can be connected where. Furthermore, as soon as you start dragging a connection out, any incompatible connections turn gray and slightly transparent. This way, you will always 100% know what is connectable and what isn't. And speaking of connecting things easily... ## Node Selector Context Menu A highly requested feature, chaiNNer now has a context menu that will show up both when right-clicking in the blank space of the editor, as well as when dragging a connection from a node to the blank space. This context menu allows you to quickly find a node to place at that position, and automatically connect it (in the case of dragging a connection to the canvas). It also filters the node list automatically based on the type of the input or output you dragged from! It makes setting up chains far easier than before, to the point where I've been told by someone it makes the regular node selection panel obsolete for them. I highly recommend you play around with this, as it really can speed up making a chain. ## Disabling Nodes Another big new feature is the ability to disable nodes. You may notice now that every node has a little toggle in the bottom left corner (as well as an entry in the right-click context menu) for disabling/enabling a node. Disabling a node will prevent it from running in the chain, so now you can have floating nodes not connected to anything without worrying about getting errors. Disabling a node will also disable any nodes connected downstream from it, so it should be pretty easy now to temporarily turn off parts of a large chain. --- ## File: src/content/posts/alpha-v0-10-1.md --- title: Alpha v0.10.1 date: 2022-07-19 tag: v0.10.1 summary: Nodes already on the canvas can be dragged into an iterator, and released back out again. source: discord --- Iterator QOL improvements & more. This update brings a much needed quality of life feature that's been missing from chaiNNer for quite a while now: the ability to drag nodes that are already placed down into iterators. I have seen lots of confusion among people new to chaiNNer that did not realize you needed to drag and drop from the selection panel into the iterator in order to use it properly. Now, that is no longer a concern. Now, you can drag any node into the iterator area and it will automatically be placed inside the iterator. You can even select and drag multiple nodes in at once, meaning you can now convert an existing chain into an iterable chain fairly easily. To get a node out of an iterator, a new option has been added to the right-click context menu to "release" the node from the iterator. It will then be placed outside the iterator for you to use there. I really hope this makes using iterators a lot less painful. --- ## File: src/content/posts/alpha-v0-11-0.md --- title: Alpha v0.11.0 date: 2022-08-14 tag: v0.11.0 summary: Image previews inside the editor, via a new View Image node and a rework of how previews and starting nodes work. source: discord --- In-editor image previews & more. This update took a lot longer than expected, but I think it will be worth the wait. ## Previewing images without leaving chaiNNer You can finally preview images inside chaiNNer without having to open an external program! It's not a fully-fledged built-in image viewer (yet), but rather a node that will show an image preview similar to how Load Image has always done it. Part of this feature included reworking how previews work in general, so you'll see that the previews for both Load Image and PyTorch's Load Model now appear on the outputs of the node rather than the input. This is due to the fact that we now automatically run valid "starting" nodes (nodes with no ability to create input connections) and use their outputs for setting types and showing previews (if applicable). This means we now allow more data to be automatically typed and previewed through the type system as well, meaning things like file name and directory outputs now are previewed via the type previews. ## Previews inside iterators You'll also notice this happens in iterators. The new View Image node allows you to preview the input of iterators, something which was not possible before. And, due to the other outputs types being set, you can see in real-time other information being updated (if you use it in the chain). --- ## File: src/content/posts/alpha-v0-12-0.md --- title: Alpha v0.12.0 date: 2022-09-04 tag: v0.12.0 summary: Direct PyTorch to NCNN conversion, SwinIR support, faster PyTorch auto-tiling, and lower RAM overhead on large chains. source: discord --- PyTorch to NCNN conversion is here! And so is SwinIR support, and much much more! ## PyTorch to NCNN conversion AMD users rejoice: a long-awaited, highly requested feature is finally here: PyTorch to NCNN conversion! theflyingzamboNNi did a wonderful job converting the original onnx2ncnn C++ code into python code in order to make this happen. Under the hood, we're converting from PyTorch to ONNX to NCNN (which you can still do if you want to), but we also have direct PyTorch to NCNN conversion for convenience. This means no more needing to convert to ONNX and use convertmodel.com! It can all be done in chaiNNer. Note: Make sure you have up-to-date NCNN and ONNX packages from the dependency manager in order for this to work properly. ## SwinIR support The other big thing this release is SwinIR support (for PyTorch). I've wanted to add this architecture for a while and I finally got around to it. It is a really good Super Resolution architecture with lots of great pretrained models to use. Ideally, we support every variation of SwinIR model (with all the params auto-detected), but if any of the SwinIR models give you an error please let me know and I'll try to resolve it. One weird thing about SwinIR is that it actually doesn't support fp16, so that will be automatically disabled when using this arch (so that means no fp16 speed-up). It also doesn't support conversion to NCNN, as NCNN does not support all the operations required for the architecture. Next-up architecture additions: GFPGAN and HAT (no ETA on these yet though). ## Faster auto-tiling, and less RAM I've also reworked how the PyTorch auto-tiling works. After some experimenting, I was able to use the VRAM estimations I added a while back in the logging (with a bit of added safety) instead of the old try/catch splitting method. This yielded a significant performance boost for PyTorch upscaling in general, though only when upscaling with GPU. CPU upscaling is unaffected by this change. I've also done a bit to reduce some of the RAM overhead we previously had. Before, we weren't clearing out our internal output cache until after the entire chain was finished processing. This caused RAM usage to build up significantly for large chains, especially with large images. Now, we clear out things we no longer need so that RAM usage stays as low as possible at all times. --- ## File: src/content/posts/alpha-v0-13-0.md --- title: Alpha v0.13.0 date: 2022-09-29 tag: v0.13.0 summary: GFPGAN face restoration arrives as the Face Upscale node, and the repository moves to the chaiNNer-org organization. source: discord --- GFPGAN support is finally here! Before we begin, you might notice chaiNNer has a new home! This repository now lives in an organization (chaiNNer-org) rather than my personal GitHub account. This means that the URL to the repository is now https://github.com/chaiNNer-org/chaiNNer. The old one should still redirect here though, but you should update any links you have (in descriptions, tutorials, etc) just in case. ## GFPGAN support This update brings a long-awaited addition: GFPGAN support! Luckily, we were able to add this without much hassle. However, it does not support the first v1 GFPGAN model (but it does support v1.2+). This is because GFPGAN v1 requires compiled CUDA extensions, which is not simple to support at this time. The good news is, the later models are much better anyway, so you are better off using those to begin with. GFPGAN support does require installing a new dependency package from the dependency manager, facexlib (as part of the PyTorch package collection), so make sure you do that! To use GFPGAN with chaiNNer, there is now a new node: Face Upscale. You pass the loaded model into this node, and can optionally pass in an upscaled version of the background as well. This allows you to fully customize the background upscale, unlike the official GFPGAN code which only allows you to upscale with RealESRGAN at a fixed scale. ## About that scale option Speaking of scale, I did add an additional scale option for the GFPGAN output. While GFPGAN internally always does an 8x upscale, the official code as well as existing GUIs have included an output scale option (which just downscales the result). To ease confusion from people expecting this, I just decided to implement this as well. Unlike these other implementations though, the scale can be any number between 1 and 8, instead of just powers of 2. The important thing to keep in mind here is that adjusting the scale DOES NOT make it actually process with lower VRAM or anything like that. Another important note: The first time you use GFPGAN, facexlib will automatically download some necessary models to `/chaiNNer/python/gfpgan`. This only happens for the first use, and depending on your internet speed might take a while. Just let it download and once finished everything should work fine. --- ## File: src/content/posts/alpha-v0-14-0.md --- title: Alpha v0.14.0 date: 2022-10-27 tag: v0.14.0 summary: Video support rebuilt on a full FFmpeg build, and auto-tiling reworked to use real tile sizes instead of a split factor. source: discord --- Revamped video support, auto-tiling changes, and more! After a long wait, 0.14.0 is finally here. This update brings a couple of pretty important changes. ## Revamped video support One of the biggest changes is improved video support. Many of you have found that the current video support isn't great, as it has limited encoding options which tend to not work depending on the machine. This is due to how OpenCV's video support works (or I guess, doesn't work), which I won't go into detail about here. As of this update, the video support has been revamped to use a fully-featured FFMPEG build. This means that now we are able to have many more encoding options compared to before, so it gives you a lot more flexibility. It also means that it's pretty much guaranteed to work on any system now. This is a huge improvement that is long overdue. ## Auto-tiling changes Another long overdue change is a revamp of how auto-tiling works. For a long time, chaiNNer hid tiling options from users, and just automatically handled it behind the scenes. If you don't know, tiling is something that many upscaling applications implement to get around VRAM limitations when an upscale would use more VRAM than your GPU physically has. We eventually decided to expose manual tiling options to the users, but this still used our internal "split factor" system, leading to the option being called "number of tiles". This was temporary but stuck around longer than we thought it would, and it rightfully confused users. Thanks to Michael's tiling changes, we now have a similar but much better implementation that uses actual tile sizes. This is much more in line with other upscaling applications and is a bit easier to comprehend and plan around. We also have more options in the dropdown, as well as new VRAM estimation for NCNN. You'll see this as "Auto (estimate)" and "Maximum". Picking "Maximum" will function similarly to how it did previously, while "Auto" will function by pre-determining a tile size to use based on available VRAM. PyTorch has been using this method for a while, but now NCNN has it as well. And of course, now we have better options for picking actual tile sizes, meaning you can use the same tile size on multiple images and not have to worry about changing it all the time (if you need to set it manually, that is). We still generally recommend you use the Auto or Maximum modes though. --- ## File: src/content/posts/alpha-v0-16-0.md --- title: Alpha v0.16.0 date: 2022-12-08 tag: v0.16.0 summary: A much more compact node design, inserting a node onto an existing connection, explanations for invalid connections, sharper previews, and DDS support on Windows. source: discord --- ## Major UI improvements The first thing you might notice in this version is a bit of a redesign. Nodes have been majorly overhauled to be much more compact. While it might be a bit of a shock how different it looks at first, I guarantee you'll quickly get used to it and prefer this new look. The compactness of the nodes gives you much more space to fill out your chains with, and there is just a lot less wasted space overall. Check out this comparison of the space used by the old vs the new nodes: ```compare { "before": "/images/blog/v0-16-0-nodes-before.jpg", "after": "/images/blog/v0-16-0-nodes-after.jpg", "beforeLabel": "Before", "afterLabel": "After", "width": 1161, "height": 816, "alt": "The same chain before and after the redesign." } ``` With this compactness change also came a change to sliders. In my opinion, they are now a lot easier to see and interact with, and are closer to something like you'd see in blender. There have also been changes to pop-up boxes. Settings had a bit of an overhaul to fix some scrolling issues, and all pop-ups now have a darker color. Alerts also got icons representing the type of alert. ## Node-on-Connection insertion Something we all have always wanted in chaiNNer was the ability to insert a compatible node in-between two connected nodes, without having to delete the connection and make two new ones. Well, now this is a reality. All you have to do is hold alt while dragging a node in the editor, and both the node and the connection line will visually show you that the node is able to be connected this way. It's really convenient to be able to insert nodes into existing chains this way, and can save a lot of time. ## Informing you about invalid connections The UI now will inform you when a connection you are trying to make is invalid. Before, we would tell you why a node was invalid after it was already connected and then made invalid. However, we had never explained why some connections were not possible to begin with. This had confused many users as they would try to do things like connect a GFPGAN model to Upscale Image and think there was something wrong rather than it being something intentional. While the UI would not let these users make that connection, it would not tell them why. This update adds an explanation tooltip that explains why a connection can't be made as you attempt to drop the connection on a handle. This should hopefully improve the user experience and lead to less confusion when this happens. ## Increased View Image preview resolution View Image just became a lot more useful. Now, instead of seeing only a 512x512 preview of your image, you will see a maximum of 2K resolution when zoomed in to the max. This still isn't ideal for extremely large images (larger than 2k) where you need to zoom in even more than chaiNNer allows, but a feature for that will be coming in the future. For now, this is a huge improvement and makes in-line previewing much much better. We also improved the performance of the node, so you may notice previews loading a bit faster than they used to. ## Better DDS Support (Reading & writing, Windows only) With this update, previously unsupported DDS textures files can now be read and written on Windows. We make use of Texconv, which is a small texture utility that we now bundle with chaiNNer. Unfortunately, Textconv is Windows-only, meaning Linux & MacOS users will not be able to take advantage of this. But we figured it was still better to have it than not. --- ## File: src/content/posts/alpha-v0-19-0.md --- title: Alpha v0.19.0 date: 2023-07-31 tag: v0.19.0 source: discord summary: Searchable documentation for every node, a rewritten update screen, backend setup changes, and the first Rust code in chaiNNer. --- Hey everyone. It's been a while since the last update, and in my opinion, it's been worth the wait. We have a lot of interesting improvements and new features to share with all of you. ## Documentation for every node We now have documentation for all available nodes. This can be accessed both by a button in the top right button bar, as well as by right-clicking on a node in the sidebar and clicking "Open Documentation". The documentation includes a lot of information about nodes and their inputs & outputs. Not to mention, the docs are also searchable. This will hopefully be a huge help to both new and existing users. ## A new update screen The update screen has finally received a makeover. This has been a long-overdue change. Starting with 0.19.0, you will get a popup when there is a new update, which will have an option for ignoring the update or viewing the update's changelog. If you view the changelog, a modal will open that will show you a full changelog of the update — or multiple updates if you happen to be a few versions behind. From there you can view the release on GitHub or open chaiNNer.app to download it. If you do choose to ignore the update, don't worry, because you will see a green download indicator next to chaiNNer's version number which you can click on to view the update again. ## Backend setup changes We've made some improvements to how the backend sets up chaiNNer. This is mostly to prep for future features, but it also conveniently makes it slightly easier to set up chaiNNer on a remote machine, as all the necessary dependencies will be installed via the backend, instead of the frontend. This shouldn't affect existing users much, except you may notice that on starting up chaiNNer this time you see a slightly different progress indication on the splash screen as it installs the few new necessary dependencies. We also now bundle a couple of small, cross-platform dependencies that should make first-time setup a bit faster. If anyone is having trouble with the new first-time setup process, definitely let us know ASAP. ## Some of chaiNNer is now Rust chaiNNer is now taking advantage of some first-party Rust code. This means we were able to speed up certain nodes by a significant amount. Expect future updates to improve the speed of other nodes as we move some intensive operations over to Rust. --- ## File: src/content/posts/alpha-v0-24-0.md --- title: Alpha v0.24.0 date: 2024-06-01 tag: v0.24.0 source: discord summary: Conditional branching, a skipped state for nodes, selectable themes, and PyTorch upscaling roughly 1.25x faster than before. --- Conditional branching, node passthrough/skipping, new themes, PyTorch optimizations, and many more fixes and quality of life improvements all around. ## Branching, mostly The conditional and compare nodes bring a long-awaited feature: branching support. Well, mostly. These nodes allow you to switch between inputs based on a condition created by comparing two values in various ways. This means users can finally set up chains that do one thing under a certain condition (e.g. if an image is over a certain size) and another thing under another. The thing to keep in mind is how you have to set this up might be opposite of what's intuitive. Since you're switching between values, it's what comes before the condition that is decided on, not what comes after. ## Enabled, disabled, skipped The disable toggle has also been replaced with a button that allows you to swap between three states: "enabled", "disabled", and "skipped". In skipped mode, the node will not be run, but it also will not disable downstream nodes like disabling it would. This is another heavily-requested feature that we are happy is finally in chaiNNer. ## Themes New themes can be selected in settings. This is semi-experimental as we plan on adding support for fully custom themes in the future, as well as improving the existing default selection. The current selection is mainly just a proof of concept to show that we can support themes of some kind, and also to give users some options for customization. We'd love to hear feedback on the current selection and if you have any suggestions on themes to add. ## PyTorch upscaling is faster PyTorch upscaling has been heavily optimized. I was able to use a visual profiling tool (called VizTracer, which I highly recommend for anyone wanting to optimize a python project) to see what was slowing down upscaling. Upscaling has been optimized to a point where it's roughly 1.25x faster than it was before (though YMMV depending on what model you use, as well as other factors). While not perfect, it's still a large improvement and should help your PyTorch upscaling chains run faster.