## 1. Project Overview & Quickstart (linebender/LICENSE) # LICENSE Open-source repository linebender/LICENSE ### Repository Details - **Repository:** [linebender/LICENSE](https://github.com/linebender/LICENSE) - **Primary Language:** Code *Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.* ## 2. Official Technical Reference & Guides (linebender/linebender.github.io) ## File: README.md This repository contains the website for [linebender.org](https://linebender.org). More details are available on the website. # Development **Note: The site is currently built with Zola version 0.20.0.** The steps you need to get building the website locally are: 1. [Install Zola](https://www.getzola.org/documentation/getting-started/installation/) 2. Run `zola serve` in a terminal 3. Go to the address logged to the terminal window (usually [localhost:1111](http://localhost:1111)). # Deployment This website is automatically deployed every time a PR is merged using GitHub Actions. --- ## File: content/blog/2024-01-10-xilem-2024/index.md +++ title = "Xilem 2024 plans" authors = ["Raph Levien"] date = "2024-01-10 07:34:42" +++ 2024 promises to be an exciting year for Xilem. The big news is that Google Fonts is funding the work of four open source contributors to the project: Aaron Muir Hamilton, Daniel McNab, Matt Campbell, and Olivier Faure. Aaron will be working on text layout, building on [parley](https://github.com/dfrg/parley), [text input](https://lord.io/text-editing-hates-you-too/) including IME, and Android platform integration. Daniel will be working on GPU infrastructure and performance, with much of the work upstreamed to [wgpu]. Matt will be working on accessibility, and Olivier will be building the core widget tree, with a focus on developer experience. Xilem is an ambitious project to explore the limits of UI performance, as described in my [High Performance Rust UI](https://youtu.be/zVUTZlNCb8U) talk. The goals include gathering knowledge about the viability of Rust for UI, and what patterns work well. While there is a research focus, we plan to gather this information by building a real toolkit, solving many of the hard problems. The current state of the project is a fairly rough, though promising, prototype, so there is much work to be done. It’s exciting to have the bandwidth and experience of these people working toward that goal, but perhaps the most exciting prospect is bringing together more of the community, people passionate about building high quality UI. Our goal as an open source community is to facilitate that, and to coordinate the work efficiently. What we have now is not a production-ready product, but we wish to move in that direction, including quality documentation - some of the consistent feedback we’re getting about the current state. Another goal is to bake in accessibility from the start. Too often, it’s an afterthought, tacked on at the end. We’re excited to have Matt, primary author of [AccessKit](https://github.com/AccessKit/accesskit), on board. Xilem is designed in layers, with a reactivity layer on top, a widget tree adapted from [Druid](https://github.com/linebender/druid), and with [Vello](https://github.com/linebender/vello) as the 2D rendering engine. These layers are designed to work well together, but the individual pieces may be useful in other contexts. The funded work will focus on Android as a target, but is intended to work well on desktop platforms as well; this is one of the core strengths of the Rust ecosystem. We don’t have iOS as an immediate priority, but attention from the open source community could change that, and the foundations should support it. The status of the web backend for the Xilem widget set is more complex. Simply put, it is not a priority for us right now. We have done some explorations into generating DOM nodes from Xilem, but doing it well would increase scope and complexity beyond what we’re comfortable with. For applications deployed *primarily* on the web, xilem_web (which generates DOM nodes) should be a viable choice, with the possibility of Tauri or Electron for desktop deployment. To enable online demos of native applications, it is most likely that we will implement a version which draws to a canvas element. This has limitations in accessibility and native look and feel - for example, text input and scrolling may behave contrary to user expectations. Until more work is done refining the web backend, we would not recommend this approach for production applications using Xilem. ## 2023 retrospective Looking back at 2023, there was a great deal of foundational work. Much of the effort was on Vello, the 2D rendering engine that serves as the graphical foundation of the project. A particular strength is high performance path rendering on GPU, which enables fully dynamic text effects including animation of variable font parameters. Though we had a reasonably good implementation of path rendering in place, much of the second half of the year was taken up by “stroke rework” which computes all stroke styles on the GPU, and also improves the performance and accuracy of antialiasing for all paths. To this end, we invented new computational geometry algorithms for computing parallel curves, the newest based on [Euler spirals](https://raphlinus.github.io/curves/2021/02/19/parallel-curves.html), with some elegant math tuned for GPU evaluation. A writeup explaining the details is in the pipeline. There was slow and steady work on the higher levels of the stack as well, including factoring out the reactive layer into a generic xilem_core module, and validating that with generation of DOM nodes for browser deployment. The widget tree saw some development as well, including integration with the [Taffy](https://github.com/DioxusLabs/taffy) layout engine. ## Governance and community Though we now are funded to work on the project, the open source nature of the projects continues unchanged. We’ve been building up infrastructure for open source community, including a [Zulip instance](https://xi.zulipchat.com/), weekly office hours, [wiki](https://linebender.org/wiki/), and, as of this post, a shared [blog](https://linebender.org/blog/). Our [contributor guidelines](https://linebender.org/contributor-guidelines/) explain a bit about our processes. Almost all decision making is by consensus. Only very rarely do I need to step in and tie-break. As we scale up, we’re going to be more systematic, starting to use [RFC documents](https://github.com/linebender/rfcs) for major changes. We’re intentionally designing our community to foster learning and collaboration. For example, discussions are on Zulip rather than Discord because it’s searchable, meaning that the same questions are less likely to come up over and over. Its grouping into topics lets participants quickly skim activity as opposed to getting drowned in chatter. It’s working well for us. We also invite others in the Rust UI ecosystem to use our Zulip and wiki. As the Kurzgesagt video [The Internet is Worse Than Ever – Now What?](https://www.youtube.com/watch?v=fuFlMtZmvY0) observes, large social spaces are degrading, and the video recommends seeking out smaller, more focused communities. I hope ours is one such. In any case, we are happy to share bits of infrastructure with other Rust UI efforts, and are always eager to learn from them. I see many of us working toward common goals rather than competing, even if there are different approaches and priorities. ## Thoughts on the Rust UI ecosystem I’ve been thinking about doing UI in Rust for about 8 years now, and have explored many, many approaches. I’m not alone in this, there have been well over two dozen projects started. As mentioned in a [previous blog](https://raphlinus.github.io/rust/gui/2022/07/15/next-dozen-guis.html), any given application will have a set of requirements. Is Web one of the main deployment targets? Will there be embedded video? Is there a need to integrate with some other subsystems, such as a game engine? Each of these has profound implications. Some problem spaces (compilers are an example) are “smooth,” in that continual refinement will lead to fairly similar outcomes no matter the starting point, but over time I’ve come to the conclusion that UI is especially lumpy. I believe this contributes to the continuing pattern of new UI toolkits coming out every couple months or so; the author surveys what’s available, finds none that match the specific set of requirements, and creates a new one. In some ways, Xilem is in the same pattern, but there are *some* things we’re doing differently. For one, it’s explicitly a research project, with knowledge about how to build UI in Rust an outcome possibly as important as the codebase. We’re trying to be systematic about finding the best ways to do things, which ideally will make some solutions more general. And, increasingly, we’re designing things as modular layers that can be swapped out. With luck, even if people don’t end up adopting the entire stack, there will be some useful components for the ecosystem. A good overview of the opportunities and challenges in Rust UI, particularly for making mobile apps, is [parallel futures in mobile application development](https://wingolog.org/archives/2023/06/15/parallel-futures-in-mobile-application-development) by Andy Wingo. The linked blog posts in that series also go into considerable detail about JavaScript and TypeScript based approaches, which have important lessons as well. ### Declarative UI patterns The trend in UI programming has been overwhelmingly away from a soup of mutable objects with interlinked references and toward various declarative or reactive patterns. That’s a good thing for Rust UI, because the former is very un-ergonomic in Rust. In the JavaScript world, there is a Cambrian explosion of reactive techniques. React is by far the most popular, but there is a strong sentiment it’s possible to be better, particularly on performance, but also robustness, as the logic for deciding [when to re-render](https://www.joshwcomeau.com/react/why-react-re-renders/) is fickle, and can [miss re-renders](https://arxiv.org/abs/2310.04368). Elm deserves special mention, as its avoidance of global shared mutability makes it particularly easy to adapt to Rust, so it serves as the reactive core of the [iced], [relm], and [vizia] UI toolkits, as well as [yew] on the WASM side. But there is also [Ember Octane] (based on Adapton), a number of fine-grained reactivity approaches such as [SolidJS], and many, many more (not to mention the incredible diversity of solutions for state management). We’re seeing much of that diversity mirrored into the Rust world. On the WASM side, we see [Dioxus] as a fairly faithful adaptation of React, [leptos] (also [sycamore] and [tachys]) adhering fairly closely to SolidJS, yew as an adaption of elm, and [more](https://github.com/flosse/rust-web-framework-comparison). A reactive layer such as Dioxus or leptos can be compiled to WASM and run in a browser to generate DOM nodes. More recently, we’re seeing that it can be decoupled from that environment, and a reactive core can drive a widget tree. Examples include [Freya](https://github.com/marc2332/freya) which uses Dioxus, and [floem](https://github.com/lapce/floem) which implements fine-grained reactivity inspired by leptos. Xilem doesn’t directly correspond to any JavaScript framework; it relies on static typing and is actually closer to SwiftUI than anything web based. One of the central questions in Rust UI going forward is: which approach to declarative UI is best? It’s possible a clear winner will emerge, or perhaps there will be a different answer depending on the use case, or it might just come down to a matter of personal style, with a number of viable contenders (arguably that’s where JavaScript is). We’re taking a two pronged approach. First, we’re moving forward with Xilem. There’s a specific reason - Xilem is designed to support multithreaded rendering easily, while many of the other frameworks rely on thread local storage for access to a store of observables, an approach with high impedance mismatch to multithreading. Since performance is a goal, we didn’t want to give that up by adopting another framework. I also think Xilem is pretty good, but that remains to be seen. Another way to gather more evidence on the Xilem declarative layer is to use it to generate DOM nodes in a browser, in other words decouple it from the widget set. As described in the [Xilem Vector Graphics](https://www.youtube.com/watch?v=XjbVnwBtVEk) talk, we have been experimenting with that a bit, and a major development in the last year was to factor out the core declarative logic into xilem_core, and also develop a xilem_web layer for creation of DOM nodes. Early indications suggest that this may be a viable alternative to frameworks such as Dioxus and leptos, and we're eager to learn more from those explorations. Xilem is untested, though promising, and it may be that some other approach is better. If so, we’re prepared to pivot. We’ll be designing the widget set so that it doesn’t have a hard-coded dependency on the Xilem declarative layer, and could, at least in theory, be driven from other frameworks. Olivier is author of the [masonry](https://github.com/PoignardAzur/masonry-rs) crate, where being agnostic to the declarative layer was an explicit goal. He’ll be adapting many of the ideas from that. Making a widget set agnostic has other potential benefits. While writing app logic in Rust is very promising, there are still compelling reasons to use other languages. A widget tree implemented in Rust with fast graphics and smooth interactivity, scripted in some other language, is compelling. I made a [prototype](https://github.com/linebender/druid/pull/2185) in Python, which, though rough, shows promise. ## Infrastructure A UI toolkit depends on a great deal of infrastructure. We’re making careful choices around that, and also looking for opportunities to collaborate with the rest of the ecosystem. One choice I’m particularly excited about is investing in [wgpu]. I think WebGPU will move modern GPU usage forward considerably, especially compute shaders, which are heavily used in Vello. This technology will make truly portable applications requiring high compute throughput possible for the first time, and we’re shooting for smooth UI integration with the GPU usage in these applications. Whether to use [winit] or build our own platform integration has long been a subject of discussion. We’re reopening that discussion with the winit maintainers as of this writing, but it’s still too early to say whether we’ll join forces, or decide that the goals of the project support having a separate codebase. A huge part of any UI effort is text, and that’s also a long-standing interest of mine. I’m impressed with [cosmic-text](https://github.com/pop-os/cosmic-text) but ultimately we decided to move forward with [parley](https://github.com/dfrg/parley), in part so we can test out ideas with variable fonts. The lower levels of the text stack will be rebased on the [fontations](https://github.com/googlefonts/fontations) work of the Google Fonts team, which is rapidly becoming a production-quality implementation. Building common infrastructure for accessibility also benefits the entire ecosystem, and we're thrilled to see the increasing adoption of AccessKit. Indeed, on all these topics, we’re happy to share what we’re doing with other projects, as most of the infrastructure described above is general in application, rather than specific to our projects. ## Conclusion The potential for Rust UI is huge. The Rust language enables top-notch performance and reliability and is also expressive for high-level application logic. We plan to prove that out, based on encouraging existing research and explorations, as well as new research to discover the patterns for Rust UI that work best at scale. This involves work at all levels of the stack, including infrastructure we are happy to share with other Rust UI efforts. Xilem is still an early prototype. For those who desire a “batteries included” toolkit for building UI, we ask your patience. For those who are interested, you are invited to come build this future with us. [wgpu]: https://wgpu.rs/ [winit]: https://github.com/rust-windowing/winit [Dioxus]: https://dioxuslabs.com/ [vizia]: https://github.com/vizia/vizia [leptos]: https://github.com/leptos-rs/leptos [tachys]: https://github.com/gbj/tachys [Sycamore]: https://sycamore-rs.netlify.app/ [yew]: https://yew.rs/ [iced]: https://github.com/iced-rs/iced [relm]: https://github.com/antoyo/relm [Ember Octane]: https://emberjs.com/editions/octane/ [SolidJS]: https://www.solidjs.com/ --- ## File: content/blog/2024-01-11-xilem-backend-roadmap/index.md +++ title = "Roadmap for the Xilem backend in 2024" authors = ["Olivier Faure"] date = "2024-01-19 13:00:00" +++ As you may have heard by now, Google Fonts is funding me this year to work on Xilem. I'm not alone in that: Aaron Muir Hamilton, Daniel McNab and Matt Campbell were funded as well to work on various parts of the ecosystem. I believe this is Matt's third year getting funding from Google Fonts. Now, *what* I was hired to do might be unclear to you. The scope we agreed on was fairly broad and was stated as "contributing to Xilem's view tree and developer experience", but that could mean a lot of things. The first thing I want to do is communicate a clearer picture. I have a a lot of plans for Xilem, and they involve major changes from the current architectures, changes that might be worrying to the community at large or even to Raph Levien. I certainly don't want to give the impression that I'm wresting the project away from the Linebender community for Google's interest, and that means being transparent early and often about the things I want to rework. ## What I want for Xilem Xilem is a research-and-development project, the latest in a series (Xi-Core, Druid, Crochet, Lasagna, Idiopath), that aims to explore how UI development can be made compatible with Rust. This might feel a little banal now, because Rust GUI frameworks are popping up everywhere and starting to settle on similar patterns, but those patterns weren't obvious in 2019! Xilem is very different from Druid, and getting to its current architecture has been a process. Xilem has been perma-experimental since May 2022. Once it got started, the main Druid maintainers stopped contributing almost entirely with the expectation that Xilem would replace Druid once it was ready. Unfortunately, it still hasn't made enough progress to be on parity with Druid, yet Druid still lies abandoned, which isn't great when people had started relying on it. It's debatable how much this could have been avoided. As I've pointed out before, the Rust GUI ecosystem is subject to massive [yak-shaving](https://seths.blog/2005/03/dont_shave_that/): many of us came here because we wanted to build a text editor, and now we're all learning about [text rendering](https://faultlore.com/blah/text-hates-you/), [text editing](https://lord.io/text-editing-hates-you-too/), [compositing](https://raphlinus.github.io/ui/graphics/2020/09/13/compositor-is-evil.html)[^1], [accessibility trees](https://accesskit.dev/accesskit-integration-makes-bevy-the-first-general-purpose-game-engine-with-built-in-accessibility-support/), [using monoids to implement stuff on the GPU](https://raphlinus.github.io/gpu/2021/05/13/stack-monoid-revisited.html), [ECS](https://www.leafwing-studios.com/blog/ecs-gui-framework/), and some concepts that I'm absolutely certain Raph made up like [Bézier paths](https://raphlinus.github.io/curves/2023/04/18/bezpath-simplify.html) and [C++](https://raphlinus.github.io/rust/2023/04/01/rust-to-cpp.html). And to a certain extent, yak-shaving is good! "I'm going to do better than X without learning how X works" is a child's attitude. It's the attitude we all start with, and it fills us with courage, but that courage is naive. True improvement comes from keeping that courage after spending years learning how immense the problem space is, and keeping faith that it's possible to do better. The Rust community is one that values not just doing things from scratch, but also learning from the past to get them right. This is good, but the "get them right" part takes a lot of time, and in the meantime, I'd argue that we've neglected end users a bit. We've abandoned Druid before Xilem was ready as an alternative, and then we worked on foundational and experimental projects that would make Xilem stronger and sounder in the long term, but left the project in an awkward state in the meantime (except for the web backend, which made a lot of progress). In the [ECS-backed GUI framework](https://www.leafwing-studios.com/blog/ecs-gui-framework/) article I linked, Bevy maintainers have this to say: > Not to be too harsh, but a lot of the existing Rust GUI solutions... just aren't very good. There's a lot of passable options, but they all have non-trivial drawbacks. No one has really risen to the top as a clear winner. [...] Deep down, we all know that we can do better, and we should. My ambition is to makes this paragraph obsolete before the end of the year. I want Xilem to get back to being suggested to newbies in the same breath as Iced and SlintUI. In the next few years, I want the entire ecosystem to get to a point where people talk about Rust GUI like they talk about ripgrep or rustls. To give one specific example: my personal goal is to implement seamless hot reloading for Xilem before the end of 2024. I believe it's feasible, and I believe doing it will radically change the way people think about Rust GUI, along with [other improvements I hope to unlock over time](https://poignardazur.github.io/2023/02/02/masonry-01-and-my-vision-for-rust-ui/). In the shorter term, my plan is to focus on basic features, widgets and styling options so people can experiment with Xilem, building on Raph's work on Vello over the past year. [John Skottis's Xilem UX Notes](https://gist.github.com/giannissc/172c6c591997ee14f6120134a7990697) give a good picture of the kinds of things I'll be working on. The other big short-term priority is going to be documentation. We've seen a lot of people complain that the Linebender project were poorly documented and hard to get into; it was the most popular reaction to our [2024 announcement](https://linebender.org/blog/xilem-2024/), but it echoed sentiments we'd seen before. We're aware of the problem, and we intend to work on it. In the medium term, I also plan to work on the performance aspect of Xilem. People from Google Fonts have expressed interest in seeing how much performance we can get from Rust UI. While my mission isn't *limited* to performance work, it will be an ongoing focus, especially since it's an area where the Rust language brings unique opportunities. I'll go into more details on Rust performance and power efficiency in a later article. ## Major changes I want to implement soon These are the big changes I want to work on soon. Realistically, "soon" is going to be relative, because some of these changes are going to be controversial, and part of the work is going to be convincing people of their value. ### Switching to Masonry as a backend Okay, this one makes me blush a little. I'm the only maintainer of Masonry, a GUI crate with barely over 300 stars on GitHub and little outside interest. Can I really justify taking the money Google pays me to improve Xilem and spending my time on Masonry instead? That choice isn't random, mind you: Masonry was forked from Druid, Xilem's ancestor, and in fact Xilem and Masonry share a lot of code right now. In fact, I'd argue that Masonry's codebase is of better quality than Xilem's current native backend. This isn't meant to be a put-down of the work of Xilem maintainers; rather, it's the outcome of a difference in priorities. Nobody was very interested in Xilem's native backend (though Raph was aware of its importance, hence me getting paid to write this), and lots of people were very interested in the frontend and the web backends, therefore the frontend and web backends saw the most work. From our [RFC#0001](https://github.com/linebender/rfcs/pull/1): > As a result, Xilem's native backend is in a poor state: > > - There is [code commented out](https://github.com/linebender/xilem/blob/ea45b9f8c14e3708f0fcbe0a0e1c760f59146323/src/widget/widget.rs#L113-L120). > - There are [*entire modules* commented out](https://github.com/linebender/xilem/blob/ea45b9f8c14e3708f0fcbe0a0e1c760f59146323/src/widget/mod.rs#L19-L20). > - There is [documentation referring to items from Druid that no longer exist](https://github.com/linebender/xilem/blob/ea45b9f8c14e3708f0fcbe0a0e1c760f59146323/src/widget/widget.rs#L51-L71). > - There are [TODOs without an associated issue](https://github.com/linebender/xilem/blob/ea45b9f8c14e3708f0fcbe0a0e1c760f59146323/src/widget/core.rs#L569C5-L569C66). Masonry's backend codebase is a healthier starting point. Masonry also comes with some built-in perks, like powerful unit tests and a structured widget graph. It's not clear whether we want to import the Masonry codebase, the crate, or the entire repository, and it's not clear what the ultimate branding should be. The RFC lays out the different possibilities, and discussion is still ongoing, so the community can reach a consensus before doing more work (including possibly not using Masonry at all). If we do move forward with this, then Masonry will need some porting work: - Using Vello and Parley instead of Piet. - Using Glazier instead of druid-shell. - Integrating AccessKit. - Removing the dependency on GTK. AccessKit support in particular is table stakes, by now. The Rust ecosystem is converging on the idea that accessibility (or at least, screen reader support) should be integrated early on in the design of GUI frameworks, not tacked on at the end. There are other things Xilem added over Druid (notably virtual lists and async support), but I believe Masonry doesn't need to implement them right away for parity. Both will be very important for eventual performance work. ### Avoiding custom Widgets I'm told by Raph this is a "molotov cocktail" which, uh, fair. We should write Xilem's backend under the assumption that end users of the library (including the Xilem frontend) will very rarely create their own widgets. Instead, they will usually compose the primitives given to them the same way they compose DOM elements in the browser. There will be escape hatches, from a fixed-size canvas surface where end-users can use arbitrary paint primitives to full-on custom widget code, but we should design the framework under the assumption people will almost never *need* those hatches. Clearly, this is a radical move. **There will be a prototyping phase before we choose whether we commit to it**, and RFCs to lay out the details. I'm confident it will pay off. Developers of hobby GUI frameworks tend to provide a severely anemic set of primitives, under the assumption that if anyone wants other features in their widgets, they can implement them themselves. Both Druid and Xilem suffer from this. On the other end of the spectrum, the web has shown that you can implement complex custom UIs given a rigid but rich set of primitives. More importantly, the web is moving towards a powerful paradigm: declarative UI. As time goes on, people are realizing you can get better performance and composition by describing what things should look like and composing them than by imperatively making them do things. Part of this is that native code is more performant than JS, but part of it just that declarative code is easier to work around than imperative code: its bounds are better defined, it lets you [commit to limitations](https://developer.mozilla.org/en-US/docs/Web/CSS/contain) that make the engine's job easier, and it's easier to analyze. A set of proposals called [CSS Houdini](https://developer.mozilla.org/en-US/docs/Web/API/Houdini) came out that went in the exact opposite direction: pushing the layout engine towards more customization and JavaScript code. The proposal hasn't seen much movement since 2021, and I don't think it's a coincidence that the part that would have given the most power to imperative code, the Layout API, hasn't been stabilized in any browser. The bottom line is that, while implementing custom widgets *will always be possible*, I want to encourage developers to go as far as they can composing declarative elements instead, and see how far it's possible to go using those building blocks. #### More opinionated layout One major customization point I want to restrict in Xilem is layout. Today all Rust GUI frameworks ship their own layout algorithm, but none of them are *competing* on their layout algorithm. Often the layout algorithm is just a means to an end; developers that do want to work on a state-of-the-art layout engine will provide it in a modular crate like [Morphorm](https://github.com/vizia/morphorm) and [Taffy](https://github.com/DioxusLabs/taffy). And here's the thing: realistically, most layout engines aren't that creative. Every Rust framework and every major GUI framework I'm aware of uses layout that boils down to "FlexBox with small variations". The number of meaningfully different ways to do GUI layout can be counted on, generously, two hands. I'd argue we only need CSS Flow, FlexBox, CSS Grid, and maybe Morphorm to cover most people's needs. Conveniently, this is what Taffy aims to provide. There's already an experimental Taffy integration in Xilem, and I want to make it deeper. That being said, there won't be a hard-coded set of layout algorithms. Rather, layout will rely on a web-inspired protocol between widgets (in other words, trait methods), and that protocol will be strongly opinionated to make implementation of these layout algorithms easier. More on this in a latter blog post. #### No boxing Having a semi-fixed set of widgets means you can avoid using `Box`, which has some interesting performance implications. It means you can add a lot of methods to the Widget trait without worrying about bloating the vtable. And you can write methods under the assumption that they'll be inlined, which opens up interesting DX improvements. ### Using ECS, maybe, sort of In current Xilem code, if you want your FlexWidget to have children, it must have a `Vec>` member. Each Pod directly owns a Widget. I think this is an interesting quirk of Rust GUI, tying directly to Rust's ownership semantics. Non-Rust frameworks often use pointers to child widgets, true, but they're often shared with the framework, with an understanding that the framework is charged with keeping track of how many widgets there are and which is a child of which. In Qt, for instance, it's preferred (but not mandatory) for parents to call `delete_later()` on their children instead of `delete`, because it plays better with event handling. Xilem's approach of "you own your children" is a little bespoke. It means the framework has a hard time iterating over the entire widget tree (say, as part of a DOM inspector) unless each Widget specifically implements the iterating method. And it means if you want to target an event at a specific widget (say, the one with keyboard focus), you need to go through the entire ownership chain, using some heuristics to avoid visiting too many widgets. I believe Widgets should be owned by the library. If your container has children, then the only thing the container will actually own is keys into a structure (probably a slotmap) where the widget is stored. This makes a lot of things easier, like serialization and debugging, but it has an impact on the entire backend. It's an infrastructure investment. Lately, I've seen more and more discussion of implementing GUI through an ECS. A lot of that discussion comes from Bevy, which is natural, since the bevy community ~~is made up of ruthless cultists striving to feed ever more sacrifices to the ECS god until it consumes the Earth~~ is intimately familiar with the ECS pattern and has reached a phase where UI work is getting a lot of attention[^2]. But I've seen discussions about it in the Linebender community too. Whether we actually want to use ECS is something we still need to research. The way I understand it, the big draw of ECS is the "S" part, the Systems: it's about handling lots of similar data at once in arbitrary order. Which is why eg archetypes in bevy are useful: you're grouping together entities with the same components, so that you can say "execute this code for all entities with component X and component Y" and have this code be executed on arrays of values with minimal indirection or branching. And the "arbitrary order" part makes them work really well with slotmaps: since you can just read from the beginning of the slotmap, you don't pay the double-indirection penalty. In other words, the ideal use case for ECS is code of the type `for (entity in entities) entity.position += entity.speed`. GUI is pretty far from that ideal use-case: updates are sparse and should only run on a small number of changed widgets. Order often matters. And I'm not sure archetypes would work in a framework with a large variety of widgets and different attributes. I think there are two things you really want from a Rust ECS library for GUI: slotmaps, and efficient ways to add and remove components from an entity. Implementing those is going to be a major undertaking, which we'll have to divide into small experiments, but one I expect to pay many times over. ## Community involvement and more to come I want to really put emphasis on something: none of the above is set in stone. Part of what Raph wants to achieve this year is to make Xilem more of a community project, and that means running more decisions by the community. I've created [an RFC repository](https://github.com/linebender/rfcs/) for this, and each of the sections above will get its own RFC, where I'll expand on my rationale in more detail. This isn't a rubber-stamping process either: I want to implement those ideas, but if the community[^3] decides that they are bad, then we'll go another direction. There's other ideas I haven't discussed yet that I want to come back to: - **Declarative styling:** Giving Widgets attributes similar to CSS properties, with modifiers for animations, pseudo-classes, media queries and container queries. While inspired by CSS, it wouldn't involve any CSS parsing, selector resolution, specificity resolution, etc. - **Compositing:** Xilem is heading towards a "do everything in the GPU" approach. This is good for performance, but Firefox engineers who have gone down this road before us have warned us of the many perils they faced, especially regarding battery life. In general, after layout and painting, you really want a `composite()` phase to deal with scrolling and some simple animations without touching the GPU at all if possible. [This is a complex subject](https://raphlinus.github.io/ui/graphics/2020/09/13/compositor-is-evil.html), and it would be nice if we could hide most of that complexity from app developers while still giving them good performance. - **Resource loading:** This is another area where we might want to crib from game engines. At the very least, we want to be able to ask "Load me this texture/shader/font/etc" without having to write disk I/O in user code. Then we can start thinking about progressive loading, fallbacks, sharing, etc. These ideas are likely to be the subject of further articles and/or RFCs in the near future. ### Relations with other projects Presenting the current state of the Rust GUI ecosystem would probably take its own article. Things are moving quickly, and anything I write now might not be up to date in two weeks. The projects we're most interested in right now are Winit and Bevy, and there's some potential for cooperation with Servo. The COSMIC ecosystem looks pretty cool, but so far it looks like we have less to share with them. People from parallel projects are communicating and exchanging much, *much* more than they were in 2022. Between Vello, Parley, wgpu, Taffy, COSMIC-Text and others, projects are converging and maintainers are making more effort to reach out and integrate other people's code. Stay tuned for more! ## Hey, this is really cool! Something that came up in discussions I've had with Raph is that it's easy to miss the forest for the trees. In the middle of all the paperwork, blog-writing, RFC-writing, we can end up focusing on the process so much that we forget how excited we are. So, I'm taking the moment to remind myself: this is really freaking exciting! For all my criticisms and caveats and realistic assessments of the problems we need to overcome, I'm still feeling absolutely great about this! I'm super thankful to Raph for getting us this far, to Google Fonts for sponsoring us, and to everyone else who contributed to the Linebender ecosystem. I think we can do something amazing and I'm honestly a little startled to be in this position at all. To reiterate what I said: in the next few years, I want the entire ecosystem to get to a point where people talk about Rust GUI like they talk about ripgrep or rustls. I legitimately think Rust has the potential to get best-in-class GUI frameworks, and now the time has come to prove it. [^1]: By the way, I love this style of writing where we start humanizing a software process and complaining about how much it wants to kill us. The articles I cited are very high-quality too. [^2]: I'm told the Bevy community found that joke funny, but I was too afraid for my sanity to check on their Discord server. [^3]: "The community" is a nebulous concept here. For obvious reason, we'll give more weight to the opinions of people who have already put work into the Linebender ecosystem, but the general hope is to get a consensus from everyone involved. And you're welcome to weigh in even if you haven't contributed before. --- ## File: content/blog/2024-02-06-tmix-01/index.md +++ title = "This Month in Xilem, January 2024" authors = ["Raph Levien"] date = "2024-02-06 07:28:42" +++ Welcome to the first in the “this month in Xilem” series, bringing updates on the [Xilem] UI toolkit. There were two blog posts on the new [Linebender blog](https://linebender.org/blog). The first, [Xilem 2024 plans](https://linebender.org/blog/xilem-2024/), sets out the goals for the project for the year. The second, [Roadmap for the Xilem backend in 2024](https://linebender.org/blog/xilem-backend-roadmap/), by Olivier Faure, presents a roadmap for the Xilem native widget set. A major feature of the roadmap is a clean boundary between the Xilem reactive framework and the lower level widget set. We hope that enables experimentation with other reactive frameworks and possibly integration with other projects, rather than requiring everybody to buy into the Xilem stack. On the planning front, a major decision was whether to continue work on our own window creation library, [Glazier](https://github.com/linebender/glazier), or whether to join forces with [winit](https://github.com/rust-windowing/winit). Among other things, since Android is a primary target for our work, and winit already has a working Android back-end, it offers an appealing path to getting that up quickly. We have decided to try joining forces with winit, and look forward to future collaboration. Over the next few weeks, that will entail porting Xilem over to winit ([xilem#163](https://github.com/linebender/xilem/issues/163)) and also trying to land working Android IME (soft keyboard) support in winit ([winit#2993](https://github.com/rust-windowing/winit/pull/2993)). In Vello, a major advance was porting the low level font handling code from the experimental swash crate to the production-quality [skrifa](https://github.com/googlefonts/fontations) ([vello#423](https://github.com/linebender/vello/pull/423)). Skrifa is being developed by Google Fonts as a memory safe library for loading font files, and is integrated into Chromium builds. Over the coming weeks, a goal is to release Vello (and supporting crates) on crates.io. [Xilem]: https://github.com/linebender/xilem --- ## File: content/blog/2024-03-04-tmix-02/index.md +++ title = "This Month in Xilem, February 2024" authors = ["Raph Levien"] date = "2024-03-04 07:34:42" +++ Early this year, we made a decision to use the winit crate for window creation, as opposed to rolling our own. That work has now landed in main ([xilem#176] was the main PR, with a number of followups). A major motivation was to enable more platforms, with Android being a major goal. That work is in progress; a major subtask is enumerating the system fonts. As expected, this move has uncovered rough edges in winit and some regressions in behavior. We plan to address those, with the improvements hopefully benefiting the entire Rust UI ecosystem. The [Vello crate] has been published as version 0.1, with significant documentation improvements and a friendly example. We consider this an alpha-quality release, as there are known issues. Even so, we want to get a release out to facilitate integrations and shake out problems. Another major area of work on Vello is investigation of startup time, which can take multiple seconds when the platform's GPU drivers are slow at compiling the compute shaders. A number of techniques mitigate that. One is to compile shaders in parallel, using multiple cores, and that has landed ([vello#455]). An even bigger improvement will be caching compiled shaders. Daniel McNab has made an encouraging prototype ([vello#459]), and the ultimate goal is to land that in wgpu so projects across the ecosystem can benefit from it ([wgpu#5292]). The work to compute strokes on the GPU using Euler spirals continues, but has not yet landed. The current state is a draft PR that implements the logic in Rust code, which now needs to be translated to WGSL for GPU evaluation ([vello#496], draft PR). There has been progress at higher levels of Xilem. Among other things, Nico Burns landed a scroll view implementation ([xilem#155]), and Philipp Mildenberger has been steadily refining xilem_core and xilem_web, including fine grained tree structure tracking and minimal/efficient DOM-tree-updates ([xilem#160]). [Xilem]: https://github.com/linebender/xilem [xilem#176]: https://github.com/linebender/xilem/pull/176 [vello#455]: https://github.com/linebender/vello/pull/455 [vello#459]: https://github.com/linebender/vello/pull/459 [wgpu#5292]: https://github.com/gfx-rs/wgpu/pull/5292 [xilem#155]: https://github.com/linebender/xilem/pull/155 [xilem#160]: https://github.com/linebender/xilem/pull/160 [vello#496]: https://github.com/linebender/vello/pull/496 [Vello crate]: https://crates.io/crates/vello --- ## File: content/blog/2024-04-04-tmix-03/index.md +++ title = "This Month in Xilem, March 2024" authors = ["Daniel McNab"] +++ Since the last update, there have been three newly released projects from Linebender: - [](https://crates.io/crates/velato) [Velato][]: A parser and driver for Lottie animation files in applications using Vello - [](https://crates.io/crates/vello_svg) [Vello SVG][]: Allows displaying (some) SVG documents in Vello applications - [](https://crates.io/crates/tracing_android_trace) [Android Trace][]: Resilient bindings to [NDK Tracing][] and an integration with [tracing][] If you have a WebGPU capable browser you can check out the [web demo of Velato][]. Implementation work for [rfcs#1][] has started. Masonry is being ported to Vello in [masonry#56][], with the next step being a move to Winit. This work can be followed in the [#masonry][] stream on Zulip. We have landed some important improvements to Vello. Euler spiral based stroke expansion ([vello#496][]) enables GPU stroking by default, significantly improving performance in scenes which make heavy use of strokes. Additionally, we now support scenes with more than 64k draw objects ([vello#526]). We have also been making progress on [vello#366][], by evaluating different strategies for memory allocation. The first part of this has landed in [vello#537][], which avoids undefined behaviour in more out-of-memory cases. Additionally, we have been able to continue taking advantage of improvements in the [foundational font crates][fontations] from Google Fonts. This has enabled support for font fallback in [xilem#196][]. Hinting has been unblocked by [vello#544][], although the implementation has not yet landed in Xilem. [rfcs#5][] has been approved, which will improve the robustness of our continuous integration. Kaur Kuut, the author of the RFC, has also landed these changes in [vello#523][]. A focus for the next month is preparing for the [RustNL conference][] at the beginning of May. We are hoping to present a compelling demo there. [masonry#56]: https://github.com/PoignardAzur/masonry-rs/pull/56 [Android Trace]: https://github.com/linebender/android_trace [Vello SVG]: https://github.com/linebender/vello_svg [Velato]: https://github.com/linebender/velato [web demo of Velato]: https://linebender.org/velato/ [rfcs#1]: https://github.com/linebender/rfcs/blob/main/rfcs/0001-masonry-backend.md [NDK Tracing]: https://developer.android.com/ndk/reference/group/tracing [tracing]: https://crates.io/crates/tracing [xilem#196]: https://github.com/linebender/xilem/pull/196 [RustNL conference]: https://2024.rustnl.org/ [#masonry]: https://xi.zulipchat.com/#narrow/stream/317477-masonry [vello#366]: https://github.com/linebender/vello/issues/366 [vello#496]: https://github.com/linebender/vello/pull/496 [vello#537]: https://github.com/linebender/vello/pull/537 [vello#544]: https://github.com/linebender/vello/pull/544 [fontations]: https://github.com/googlefonts/fontations [rfcs#5]: https://github.com/linebender/rfcs/blob/main/rfcs/0005-version-matrix.md [vello#523]: https://github.com/linebender/vello/pull/523 [vello#526]: https://github.com/linebender/vello/pull/526 --- ## File: content/blog/2024-05-01-tmix-04/index.md +++ title = "This Month in Xilem, April 2024" authors = ["Raph Levien"] +++ Again, last month saw a substantial amount of infrastructure work in the development of Xilem. The biggest change is landing Masonry as a crate within the Xilem repo ([xilem#203]) and adding a new xilem_masonry front-end. The existing xilem widget tree remains in the tree during a transitional period, but the focus of development is now on Masonry. Masonry is a principled abstraction layer within the stack. While it is designed to work well with the Xilem reactive architecture, it is not dependent on it. We'd love to see experimentation with alternative reactive frameworks, and also language bindings. A lot more detail about the motivation and design of Masonry is in Olivier's [roadmap blog]. Two major infrastructural crates have been published. [Parley] is our text layout crate, supporting variable fonts and other advanced features. Supporting Parley is [fontique], a crate for enumerating the system fonts and making them available for itemization (choosing the right font to support a particular script). In GPU rendering land, we have written a paper on GPU stroke expansion ([arXiv][stroke expansion arXiv], [repo][stroke expansion repo]) and submitted it to a conference. This paper explains quite a bit of the math behind Vello's path rendering, and explains how we adapted it for high performance on the GPU. It's also a good time to note the [wgpu 0.20] release, which contains some work to improve on suboptimal zero initialization that will dramatically improve startup time on many platforms, including Android ([wgpu#5508]). It also has support for subgroups, which will unblock further performance improvements in Vello. We're very much looking forward to participating in RustNL 2024, where Raph will present a talk on Xilem. It looks like it will be a great opportunity to meet with other Rust UI ecosystem projects to find common ground and possibilities to collaborate. [xilem#203]: https://github.com/linebender/xilem/pull/203 [roadmap blog]: https://linebender.org/blog/xilem-backend-roadmap/ [parley]: https://crates.io/crates/parley [fontique]: https://crates.io/crates/fontique [stroke expansion arXiv]: https://arxiv.org/abs/2405.00127 [stroke expansion repo]: https://github.com/linebender/gpu-stroke-expansion-paper [wgpu 0.20]: https://docs.rs/crate/wgpu/0.20.0 [wgpu#5508]: https://github.com/gfx-rs/wgpu/pull/5508 [RustNL 2024]: https://2024.rustnl.org/ --- ## File: content/blog/2024-06-12-tmix-05/index.md +++ title = "This Month in Xilem, May 2024" authors = ["Daniel McNab"] +++ Several members of Linebender attended [RustNL][] at the start of May. Raph gave a talk titled *Xilem: Let's Build High Performance Rust UI* ([YouTube](https://youtu.be/OvfNipIcRiQ)). The conference also provided a great opportunity to meet with developers from other Rust user interface projects; Olivier has produced a report on the unconference, which will be published here shortly. In addition, those in attendance met after the conference and worked on a roadmap for Linebender, which we are also planning to share here. We have taken several important steps on Xilem. At the start of the month, we released [Xilem 0.1.0][] and [Masonry 0.2.0][]. This is the first published release of Xilem, and includes text input ([xilem#241][]), AccessKit integration ([xilem#244][]) and a TodoMVC inspired example ([xilem#257][]). Work through the rest of the month included explorations of a generic `View` trait ([xilem#310][]), and an Android port ([xilem#309][], [xilem#313]). [rfcs#6][] is designed to reduce boilerplate when using Masonry, which should help those creating custom Masonry widgets. Other Linebender projects have seen steady improvements. [vello#575][] significantly reduced shader compilation time at startup for projects using Vello, and [vello#516][] enabled clearer error handling. Nico Burns has started an exciting project for inline box layout ([parley#67][]), which will enable Parley to be used in projects which must implement Web layout. [rustnl]: https://2024.rustnl.org/ [Xilem 0.1.0]: https://github.com/linebender/xilem/releases/tag/v0.1.0 [Masonry 0.2.0]: https://github.com/linebender/xilem/releases/tag/masonry%2Fv0.2.0 [xilem#241]: https://github.com/linebender/xilem/pull/241 [xilem#244]: https://github.com/linebender/xilem/pull/244 [xilem#257]: https://github.com/linebender/xilem/pull/257 [xilem#310]: https://github.com/linebender/xilem/pull/310 [xilem#309]: https://github.com/linebender/xilem/pull/309 [xilem#313]: https://github.com/linebender/xilem/pull/313 [rfcs#6]: https://github.com/linebender/rfcs/pull/6 [vello#575]: https://github.com/linebender/vello/pull/575 [vello#516]: https://github.com/linebender/vello/pull/516 [parley#67]: https://github.com/linebender/parley/pull/67 --- ## File: content/blog/2024-06-15-rustnl-2024-unconference/index.md +++ title = "Report on the RustNL 2024 Unconference" authors = ["Olivier Faure"] +++ On May 7 and 8, 2024, folks from the Linebender team went to [the RustNL conference](https://2024.rustnl.org/) in Delft. We listened to [some Rust-related talks](https://www.youtube.com/playlist?list=PL8Q1w7Ff68DBZZbJt3ie5MUoJV5v2HeA7), Raph [gave one on Xilem](https://www.youtube.com/watch?v=OvfNipIcRiQ&list=PL8Q1w7Ff68DBZZbJt3ie5MUoJV5v2HeA7&index=15), Rik Arends did the "hot-reload my entire editor live" thing that makes every other Rust GUI developer jealous, it was an all-around great time. After RustNL, though, another less-documented event called [The Unconference](https://2024.rustnl.org/unconf/). It was from roughly the same organizers and took place in the same city, but the format was fairly different. The Unconference wasn't livestreamed, and I haven't found any public discussion of it, so I thought I'd take the time to describe a bit of what was going on. ## The format The "Unconference" was two days of loosely structured chats between members of various Rust project teams where people talked about ecosystem collaboration, complained to lang team members that their favorite feature wasn't implemented yet, [went to pester everyone else about variadic generics](https://poignardazur.github.io/2024/05/25/report-on-rustnl-variadics/), and other good fun. Attendees were split into three teams: - The Rust project. - GUI and Applications. - Embedded development. Discussions inside these teams were mostly self-organized, with cross-team discussions mostly taking place during lunch and recess. ### On the tyranny of structurelessness *Note: This section is very much my personal opinion and doesn't represent the Linebender project as a whole.* I'm not sure the self-organized format worked well. From what I saw, the Embedded team took well to it, and the Rust team was kept productive thanks to Alice Cecile's efforts in marshaling everyone. In the case of the GUI team, people were spread in a very large room, which should have been conducive to small side discussions and people splitting up to talk about the problems that interested them. Instead, there was an unspoken accord to progress through agenda items one by one, with a few people dominating the discussions on these items. Because those people we spread around a large room, they had to talk loudly to address each other, which left little room for side-discussions. Because the people talking were naturally the most confident and extroverted, more introverted people ended up taking a passive role in the discussion. To me, this feels like a strategic mistake. It was a setup that encouraged bikeshedding and long back-and-forths and discouraged plurality of opinions. It's no coincidence that the most interesting conversations of the Unconference happened at lunch: lunch was the point of the Unconference where people were most mixed, had the most spontaneous conversations, and were least constrained by having to follow what someone else was saying. Our discussions were still productive, I just feel like the format could have been improved. ## The GUI team As a member of the Linebender project, I was mostly exposed to the GUI team's proceedings. The main topics of interest were: - Build tooling (especially for Android and other non-desktop platforms). - Text layout and editing. - Winit adoption. - Accessibility. - Potential for further collaboration. - Requests for new Rust features. Note that I'm skipping a lot of topics from the agenda and other side discussions. The topics above were the most prominent during those two days. There was also a fair amount of informal chatter around LogLog Games' strongly critical [Leaving Rust gamedev after 3 years](https://loglog.games/blog/leaving-rust-gamedev/) article. I think those discussions mostly reflected the Bevy community's consensus, that the article had some flaws but overall pointed at very clear weaknesses in the Rust GUI / game engine ecosystem. ### Build tooling Everyone agreed that a better solution was needed to ship non-TUI Rust apps. There was discussion about various platforms and build systems, and the difficulty of having to e.g. write Java code for an Android port, and whether anyone wanted to take responsibility for maintaining that glue layer for the rest of the ecosystem. No actionable decisions were made, but the consensus seems to be that although integrated build tools are desirable, external build tooling for complex platforms (primarily Apple and Android) is likely to remain necessary. Nobody seems to have stepped up to work on multi-target builds in Cargo, and since external build tools are required as it is, most people are okay with building everything several times when they release a multi-target package. Most users are comfortable invoking platform-specific tools for bundling, signing, and manifest generation, so there is not much urgency for integrated tooling for these. ### Text layout and editing There was some widespread interest in the Linebender ecosystem's text-handling crates. From what I remember, Rik Arends and others had concerns about whether e.g. Parley and Swash depend on our Vello renderer (they don't). Some people also wanted Parley to be decoupled from Swash. People were especially interested in the idea of having access to a common text-editing widget, or at least infrastructure for creating one. That widget would handle IME, text selection, accessibility actions, etc, in a way that would feel native to different platforms with their own text-editing quirks (mobile, macOS, Linux with vim mode, etc). ### Winit By now all of the Rust ecosystem has firmly converged on [`winit`](https://github.com/rust-windowing/winit) as the windowing platform-abstraction solution of choice. (Well, not all! [One small project](https://github.com/makepad/makepad) with indomitable maintainers still holds out against the invaders.) The consensus during this discussion seemed to be that Winit was there to stay. [TAO](https://github.com/tauri-apps/tao) users present were generally interested in being able to use upstream Winit in the future, if its API adopts or supersedes the improvements made in TAO. Most were happy to see Winit becoming more trait-oriented, and to see the `dpi` crate spun off from Winit. Glazier is still shelved and we're planning to port its features to Winit until it reaches feature parity. A discussion that came up was the idea of splitting more "vocabulary types" from Winit into standalone crates. Vocabulary crates, e.g. crates that only define types and very little code, have a very low cost in memory and build times, while allowing projects to interop with e.g. Winit without actually adding a dependency on Winit. This was connected with the discussions about text editing: having a vocabulary crate that would list Winit's events and also Parley's cursor movements would allow for a very low-dependency text editing widget. ### Accessibility *Note: This section is also my personal opinion and doesn't represent the Linebender project either.* The subject of Accessibility came up informally, but I'll claim it was underexplored. While the subject was on the agenda, by the time we reached it the Unconference had been running long enough that everyone was tired and ready to leave or get to the catering. This is an unfortunate pattern that I think is worth pointing out: if projects don't make a deliberate effort to prioritize discussions of accessibility, the subject will be left out. Matt Campbell, the maintainer of AccessKit, could not attend the Unconference. I'm concerned that, in his absence, other maintainers might have felt a dilution of responsibility: since the accessibility guy wasn't there, nobody felt empowered to bring up accessibility. (This might sound like it contradicts my section about structurelessness above. It doesn't. I'm not saying organizers should have pushed harder for accessibility - they did try - I'm saying that we should strive for a culture where framework developers think about accessibility without waiting for someone to tell them to.) One point did come up during the discussion which I'd like to hammer in: the Rust ecosystem should adopt a holistic view of accessibility. Accessibility isn't just about screen readers, and making a framework accessible doesn't stop at adding AccessKit as a dependency, though doing so is a great first step. Rik asked if there was a minimal set of ARIA attributes that people could start using in their UI, as the set of available attributes is gigantic. The AccessKit framework is rich, and it's not obvious at first glance how much is needed versus nice-to-have. The current reference in the Rust ecosystem is [the AccessKit implementation in egui](https://github.com/emilk/egui/pull/2294). ### Requests for Rust features Jon Kelley, the creator of Dioxus, had [a laundry list of features](https://dioxus.notion.site/Dioxus-Labs-High-level-Rust-5fe1f1c9c8334815ad488410d948f05e) he wanted from the Rust language. That list was later [filed in the Project Goals repository](https://github.com/rust-lang/rust-project-goals/pull/10). The requests were well-documented, with motivating examples in existing Dioxus code that would clearly be improved by the features. Most of them felt both self-contained and like they would tremendously both beginners and veteran Rust developers; though Jon made the point that the papercuts he wanted to remove were especially punishing for new developers trying to become familiar with Rust. Overall people were very enthusiastic about them, and the lang team members in the room seemed pretty optimistic that those features could be added. My favorite proposals were: - `Capture` trait for auto-cloning reference-counted types in closures. - Automatic partial borrows. - Optional struct attributes. Jon also wanted support for a remote cache of pre-built crates, so that users wouldn't suffer from having to rebuild dozens of crates every time you download your crate's dependencies. The consensus from Rust maintainers was that such a cache would be essentially impossible with Rust's current architecture. Niko Matsakis recently released [The borrow checker within](https://smallcultfollowing.com/babysteps/blog/2024/06/02/the-borrow-checker-within/), which feels like a follow-up to some of these discussions. ## Collaboration in the Rust GUI ecosystem Over the last two years I've joked a few times that everybody in the Rust GUI ecosystem wants the ecosystem to standardize on common crates, as long as it's *their* crates. After RustNL, I'm pleased to report this joke isn't accurate anymore. A vibe that permeated all discussions was people wanting work to be pooled. It felt like people didn't strongly care whose crate we adopted, as long as we all agreed the crate made sense: we're slowly crawling out of the [xkcd 927](https://xkcd.com/927/) dilemma. The main ecosystem crates that people are enthusiastic about sharing are: - [winit](https://github.com/rust-windowing/winit) for creating windows. - [AccessKit](https://github.com/AccessKit/accesskit/) for plugging into accessibility APIs. - [wgpu](https://github.com/gfx-rs/wgpu/) for using modern GPU features. Jon Kelley also said he would be interested in using Masonry as the native backend for Dioxus, though we haven't hashed out details yet. ### Corporate funding For open-source projects, "Who funds this" is a difficult question: in any healthy project, there is a large scale of contributions, with individual non-corporate contributors at one end, and companies paying cash to the project's treasury at the other. In-between are self-employed people like me contracted to work on an open-source project, and corporate employees who contribute to the project as part of their 9-to-5 job. Some monetary contributions can also come from individual non-corporate donors: Servo has about fifty of them, for instance. With that in mind, some notable sponsors for projects represented at the Unconference were: - **Google Fonts:** Linebender projects. - **Futurewei:** Dioxus, Makepad, Robius and Servo. - **Embark:** Bevy, winit and rust-gpu. - **Foresight Spatial Labs:** Bevy. - **Rerun.io:** egui. Not present at RustNL but relevant to the ecosystem are **System76** (funding COSMIC-Text and contributing to iced), **Kraken** (funding iced), and **Slint** who are self-funding as a startup targeting embedded UIs and couldn't attend due to time constraints. Overall the number of different backers feels like a symptom of a healthy ecosystem: while some large corporate sponsors bring much more resources than others (Google and Futurewei especially), the ecosystem isn't in a state where any specific backer pulling out would completely collapse progress. ## Conclusion Before anything else, I'd like to express a big thanks to the volunteers who helped organize RustNL. Most of the attendees were absolutely exhausted after two days of talks and two more days of Unconference, but these folks had to *run* the darn thing. I think everyone who attended was impressed with them. Overall RustNL and the Unconference felt like getting a year of coordination done in a week. It was a thoroughly productive time, and I couldn't be happier I attended. Everyone I asked about it shared the same opinion. And the discussions we had during the event showed there was a great appetite to push the ecosystem towards more cooperation, both between GUI projects and with Rust maintainers! I'm feeling more hopeful about the future of Rust GUI than ever. Now it's time to get back to work. --- ## File: content/blog/2024-06-17-roadmap-may-2024/index.md +++ title = "May-July roadmap for Linebender" authors = ["Olivier Faure"] +++ On May 7 and 8, 2024, folks from the Linebender team went to [the RustNL conference](https://2024.rustnl.org/) for two days, followed by two days of Unconference. By that point, we all felt fairly exhausted, but ~~because Raph is a cruel taskmaster~~ since we wanted to use the opportunity we had, we came together in our hotel's conference room to discuss the Linebender project, and what we wanted from it. We'd initially thought that this day would be an informal day of unstructured chatting about the ecosystem, but it fairly quickly turned into a semi-formal planning exercise, where we tried to figure out a roadmap for the next few months. What follows is a comprehensive summary of what exactly we're planning to do. We've tried to focus on blockers, where a missing feature in one project is stalling progress on other projects, because those are the hardest barriers to coordination, and we wanted to take advantage of being in the same room together to knock them down. This is a snapshot of the project status at the time we're writing this. It includes some work we already finished since RustNL, but will not be updated when further work finishes. ## Vello ### Robust dynamic memory - **Priority:** Highest. - **Assigned:** Daniel McNab. - **Description:** Allocating dynamic memory in a robust way is essential to display large scenes without graphical artifacts. We have several possible solutions, we need to implement one. - **Issue:** [vello#366](https://github.com/linebender/vello/issues/366) ### Reliable snapshot tests - **Priority:** High. - **Assigned:** Daniel McNab. - **Description:** Snapshot tests are essential to test regressions in graphic software. Masonry's test suite heavily depends on them. But it's flaky right now because even when the fonts and inputs are exactly the same, renderers can have subtle differences that make the snapshots slightly different. We either need to create a fully deterministic renderer or add testing infrastructure that can tolerate small differences. [`nv-flip`](https://docs.rs/nv-flip/latest/nv_flip/) might have what we want. - **Issue:** [vello#608](https://github.com/linebender/vello/issues/608) ### Sparse strips - **Priority:** High. - **Assigned:** Raph Levien. - **Description:** Sparse strips are a new research direction that might significantly improve our rendering pipeline and our memory footprint. We want to write a prototype implementation so we can confirm the performance benefits. - **Zulip thread:** [Sparse strip path rendering](https://xi.zulipchat.com/#narrow/stream/197075-gpu/topic/Sparse.20strip.20path.20rendering) ### Add tests - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** Adding tests is often a "we'll do it later" item, but it has short-term benefits as well. Adding tests would help Vello contributors check that their modifications don't add regressions to the crate, and help us catch hidden bugs. As a first step, we'd want to at least add some extremely basic tests, such as checking that Vello doesn't crash when we run a sample image through the CPU shaders. - **PR:** [vello#439](https://github.com/linebender/vello/pull/439) ## Masonry ### Complete text input - **Priority:** Highest. - **Assigned:** Olivier Faure, Aaron Muir Hamilton. - **Description:** We'd like to bring up Masonry's textbox widget to be best-in-class among Rust frameworks. We want to support native-feeling interactions out of the box, such as cursor movement (go to next word, go to beginning of line), selection (select to end of text) and editing (IME, deleting an entire word, etc). This is something other framework developers have expressed interest in, so we want to implement it in a composable way. - **Issue:** [xilem#388](https://github.com/linebender/xilem/issues/388) ### Better tracing for debugging - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** Masonry currently uses the `tracing` crate to produce logs usable for debugging. These logs could be improved, so that a user finding a bug can quickly narrow in on information relevant to the bug, without having to restart the app with changed log levels. The bulk of the improvement would be to implement a new subscriber with multiple layers. - **Issue:** [xilem#250](https://github.com/linebender/xilem/issues/250) ### Fix glaring documentation problems - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** Masonry's documentation hasn't been updated since the port to Winit. While a lot of the code is still in flux, and thus hard to document, we should at least do a quick rewrite pass to remove references to Piet, Glazier, etc. - **PR:** [xilem#329](https://github.com/linebender/xilem/pull/329) ### Finish repository port - **Priority:** High. - **Assigned:** Daniel McNab. - **Description:** We've ported the `masonry-rs` repository to the `linebender` organization, but that repository isn't where the crate's code lives anymore. We should update the README to reflect that information, port the issues to `linebender/xilem`, and archive `linebender/masonry`. - **Zulip thread:** [Finishing the Masonry repository transition](https://xi.zulipchat.com/#narrow/stream/317477-masonry/topic/.E2.9C.94.20Finishing.20the.20Masonry.20repository.20transition) ### Write "Widgets in Arena" RFC - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** This was something alluded to in the January blog posts. We want to move Masonry from storing widgets in WidgetPod to storing them in a slotmap-like arena. This is a large enough change to write an RFC about it. - **PR:** [rfcs#6](https://github.com/linebender/rfcs/pull/6) ### Write "Pass order" RFC - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** Right now widgets can use context methods in their different passes to request that other passes be run. What passes can run what other passes isn't clearly documented, especially around edge cases. We should write an RFC to nail down the passes, how they can communicate, and the rationale around them. - **PR:** TBD. ### Rewrite documentation - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** Masonry's documentation hasn't been updated since the port to Winit. Once other major changes are implemented, the documentation should be brought up to date. We should avoid procrastinating on documentation too much just because the architecture is still in progress. - **Issue:** [xilem#389](https://github.com/linebender/xilem/issues/389) ### Improve focus handling - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** We should improve tab focus in Masonry. Right now tab focus has some bugs, doesn't cover buttons, isn't updated by clicks, and uses architecture inherited from Druid that could be strongly simplified once other architectural changes are completed. - **Issue:** [xilem#390](https://github.com/linebender/xilem/issues/390) ### Improve pointer status handling - **Priority:** Medium. - **Assigned:** Olivier Faure. - **Description:** We should centralize and improve how Masonry handles pointers. In particular, we should add an explicit "pointer capture" API that makes other widgets not get events from that pointer until the pointer is released or lost. - **Issue:** [xilem#312](https://github.com/linebender/xilem/issues/312) ## Parley ### Create abstractions for text selection and editing - **Priority:** Highest. - **Assigned:** Chad Brokaw, Aaron Muir Hamilton. - **Description:** We want Parley to implement types and methods that Masonry and other editors will be able to use to create a text-editing widget, or to handle selection in non-editable text. The initial implementation may be located in Masonry. - **Issue:** [parley#52](https://github.com/linebender/parley/issues/52) ### Inline block layout - **Priority:** High. - **Assigned:** Chad Brokaw. - **Description:** Parley should support laying out inline content the way web browsers do, where blocks are spliced into your text and will be laid out according to available inline space the same way words are. - **Issue:** [parley#25](https://github.com/linebender/parley/issues/25) - **PRs:** [parley#67](https://github.com/linebender/parley/pull/67), [parley#76](https://github.com/linebender/parley/pull/76) ### Write adequate documentation - **Priority:** High. - **Assigned:** Nico Burns. - **Description:** Parley currently has virtually no documentation. If we want it to become a foundational crate, we need to tell people how to use it. - **PR:** [parley#26](https://github.com/linebender/parley/pull/26) ### Software renderer - **Priority:** Low. - **Assigned:** Chad Brokaw. - **Description:** Create a software renderer for the glyphs created by Parley, so that users can easily import Parley in their library without depending on a graphics API like Vello. The software renderer should be black-and-white for simplicity. We'll call for volunteers to implement COLRv1 and other advanced features. - **PRs:** [parley#54](https://github.com/linebender/parley/pull/54), [parley#55](https://github.com/linebender/parley/pull/55) ## Xilem ### Android bringup - **Priority:** Highest. - **Assigned:** Daniel McNab. - **Description:** We want Xilem to reliably work on Android. Running your Xilem app should require a small number of steps. Ideally, it should be as simple as running `cargo some-command` in your project. - **PR:** [xilem#309](https://github.com/linebender/xilem/pull/309) ### Update repository layout. - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** We've agreed on a layout for Linebender repositories. We should move projects in the Xilem repo to match that layout. - **PRs:** [xilem#302](https://github.com/linebender/xilem/pull/302), [vello#590](https://github.com/linebender/vello/pull/590) ### Rework xilem_core to be compatible with Masonry backend - **Priority:** High. - **Assigned:** Daniel McNab. - **Description:** The version of Xilem that was ported to Masonry used a different `View` trait that is incompatible with the current `xilem_core` crate. We need to rework `xilem_core` to get traits that can be used by both Masonry and `xilem_web`. Doing so while keeping readable code is non-trivial. - **PR:** [xilem#235](https://github.com/linebender/xilem/pull/235), [xilem#310](https://github.com/linebender/xilem/pull/310) ### Write Documentation - **Priority:** High. - **Assigned:** Olivier Faure. - **Description:** Xilem's lack of documentation is one of the problems new users complain about the most often. We need to avoid staying in that state for too long. - **Issue:** [xilem#392](https://github.com/linebender/xilem/issues/392) ### Re-implement async integration - **Priority:** Medium. - **Assigned:** Daniel McNab. - **Description:** One of Xilem's selling points is the ability to handle virtual lists and async sources of data. That ability was temporarily removed for the Masonry port, but we'd like to add it back. We don't want Masonry to be aware of async stuff, but the AppDriver trait should at least provide a method which lets Xilem access Masonry internals on its own. - **Issue:** [xilem#340](https://github.com/linebender/xilem/issues/340) ### Add unit tests - **Priority:** Low. - **Assigned:** Olivier Faure. - **Description:** We need to figure out an architecture for doing unit tests in Xilem. Tests can then be added by volunteer contributors. At some point we should hopefully switch to a "new test for every new feature" workflow. - **Issue:** [xilem#393](https://github.com/linebender/xilem/issues/393) ### Add basic benchmarks - **Priority:** Low. - **Assigned:** Olivier Faure. - **Description:** Right now we're mostly working off vibes to know whether or not any of our work improved performance or introduced regressions. We should write at least *one* benchmark that would test a basic Xilem use-case, so we get a general idea of how our performance is progressing. - **Issue:** [xilem#362](https://github.com/linebender/xilem/issues/362) ## AccessKit Matt Campbell's contract with Google hasn't started yet. When it does, the top priority will likely be the Android backend. The plan is to study existing Android accessibility implementations for non-Java toolkits, including Flutter, Chromium, and Gio, to get a better understanding of what's required to implement Android accessibility from scratch. Beyond Android support, we've only had cursory discussions of what work Matt will do for the rest of the year. Possible themes include scrollable containers, virtualized lists, richer text support (e.g. exposing font and style attributes) across platforms, and better integration into Xilem/Masonry. ## Android-specific work Getting the Android port right involves a lot of small changes and making things feel non-broken. Some notable sub-goals: ### Subclassing View/Application/Activity - **Priority:** Highest. - **Assigned:** Aaron Muir Hamilton, Raph Levien. - **Description:** The Java classes of an Android application are where the rubber meets the road. We're investigating what we want our boilerplate to look like. ### Handling IME - **Priority:** Highest. - **Assigned:** Aaron Muir Hamilton, Raph Levien. - **Description:** We need to extend Winit to be compatible with Android IME interfaces. An Android application without support for the virtual keyboard and other input methods is a non-starter. Our design choices here will have some influence on accessibility as well. ### Gesture recognition - **Priority:** Low. - **Assigned:** Aaron Muir Hamilton. - **Description:** When the user swipes on the screen, we want to send some kind of scrolling event instead of a normal pointer event. Ideally there should be first-class support for that event to handle momentum scrolling, but a dumber solution would just emulate mouse wheel ticks. ## Conclusion Whew! We have our work cut out for us there. Overall, I'm feeling very optimistic about the coming months. In the weeks since we first drafted this roadmap, we've already made a lot of progress on our respective tasks (for instance Daniel is now running on what async experts call a "work-stealing" algorithm), so it does look like we'll meet our deadlines. What comes after that is a bit hazier. Our three areas of focus are likely to be Android support, accessibility, and developer experience. As we fill out gaps in our feature set, Xilem is moving towards its faraway goal: becoming a stable, mature framework for GUI applications. We're not GUI yet, but we certainly hope to be one day. In the meantime, stay tuned!