## 1. Project Overview & Quickstart (microsoft/WindowsAppSDK) ## File: README.md

Windows App SDK

[About](https://learn.microsoft.com/windows/apps/windows-app-sdk) ยท [Documentation](https://learn.microsoft.com/windows/apps/desktop) ยท [Release notes](https://learn.microsoft.com/windows/apps/windows-app-sdk/release-channels) ยท [Samples](https://github.com/microsoft/WindowsAppSDK-Samples)

[Windows App SDK](https://aka.ms/winappsdk) (formerly Project Reunion) is a set of libraries, frameworks, components, and tools that you can use in your apps to access powerful Windows platform functionality from all kinds of apps on many versions of Windows. The Windows App SDK combines the powers of Win32 native applications alongside modern API usage techniques, so your apps light up everywhere your users are. - **WinUI 3 support:** [WinUI](https://learn.microsoft.com/windows/apps/winui/winui3/) embodies Fluent Design to enable intuitive, accessible, and powerful experiences and the latest user interface patterns. - **Access modern features:** [Modern resource tooling](https://github.com/microsoft/WindowsAppSDK/issues/11), [modern lifecycle helpers](https://github.com/microsoft/WindowsAppSDK/issues/9), [startup tasks](https://github.com/microsoft/WindowsAppSDK/issues/10), and more without having to rewrite your app. - **Backwards compatibility:** Down to Windows 10 1809 (build 17763). There may be some API that are dependent on new OS features (like new Action Center functionality), however we will do our best to ensure this is the exception and not the norm, and provide reasonable fallbacks when possible. - **Wide platform support:** Win32, WPF, WinForms, and more. - **Use your current installer:** no requirement to use MSIX, but there are [reliability/security benefits to using MSIX](https://docs.microsoft.com/windows/msix/overview#key-features). ## ๐Ÿ“‹ Getting started with Windows App SDK * [Build your first app with WinUI](https://aka.ms/winui-getstarted) * [Developer documentation](https://aka.ms/windowsappsdkdocs) * [Samples & resources](https://learn.microsoft.com/windows/apps/get-started/samples) * [Windows App SDK samples](https://github.com/microsoft/WindowsAppSDK-Samples) * [WinUI 3 Gallery](https://github.com/microsoft/WinUI-Gallery) * [AI Dev Gallery (Preview)](https://github.com/microsoft/ai-dev-gallery) * [Windows Community Toolkit](https://github.com/CommunityToolkit/Windows) * [FAQ](docs/faq.md) * [Contribution guide](docs/contributor-guide.md) * [Roadmap](./docs/roadmap.md) * Windows App SDK family repositories * [WinUI](https://github.com/microsoft/microsoft-ui-xaml) * [C++/WinRT](https://github.com/microsoft/CppWinRT) and [C#/WinRT](https://github.com/microsoft/CsWinRT) * [MSIX packaging](https://github.com/microsoft/msix-packaging) ## ๐Ÿ‘จโ€๐Ÿ’ป Join in and stay connected Interested in WinUI and Windows App SDK? Come join us on the quarterly [WinUI Community Calls](https://www.youtube.com/playlist?list=PLI_J2v67C23ZqsolUDaHoFkF1GKvGrttB). You can find and tag us on X using [#WindowsAppSDK](https://twitter.com/search?q=%23WindowsAppSDK). ## ๐Ÿ–ผ๏ธ WinUI 3 Gallery Make sure to also check out the [WinUI 3 Gallery](https://aka.ms/winui-gallery), our interactive sample experience showing everything you can do with WinUI and Windows App SDK. ## ๐Ÿ“ข Contributing - **[File a new issue](https://github.com/microsoft/WindowsAppSDK/issues/new/choose):** Tell us what problem you're trying to solve, how you've tried to solve it so far, and what would be the ideal solution for your app. Bonus points if there's a gist or existing repo we can look at with you. - **[Ask a question](https://github.com/microsoft/WindowsAppSDK/discussions/categories/q-a):** Also, have a look at our [FAQ on WinAppSDK](docs/faq.md) page, which we will update periodically. - **[Start a discussion](https://github.com/microsoft/WindowsAppSDK/discussions):** Let's start a Discussion issue to see if others are interested in a cool new idea you've been thinking about that isn't quite ready to be filed as a more formal Feature Proposal. - **[Code contributions](docs/contributor-guide.md):** See our [contributing guidelines](docs/contributor-guide.md). ## Data collection This project collects usage data and sends it to Microsoft to help improve our products and services. Note, however, that no data collection is performed when using your private builds. Privacy information can be found at https://privacy.microsoft.com ## Code of Conduct This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. ## Licensing This project is licensed under the terms in [LICENSE](LICENSE). The [`.github/skills/`](.github/skills/) directory contains GitHub Copilot agent skills that may have their own licensing terms. Check each skill's directory for its license file. --- ## File: docs/README.md # What Windows App SDK Brings Windows App SDK functionality comes in three main categories - new APIs, converged APIs, and API subsets. Your apps can pick up each different level of functionality **at your own pace**. ## New APIs Whenever possible, new Windows functionality will be delivered as part of the Windows App SDK Family. APIs and functionality delivered as "New" are identity-, packaging-, and isolation- transparent to your application. New functionality will share a common interface for you to use whether your app is MSIX, MSI, setup.exe, Desktop Bridge, or AppContainer. Some of the APIs and functionality listed will come over time; they are listed as examples of the category rather than fully supported on day 1. These APIs are self-consistent and solve specific application problems or help you build additional functionality into your application. Potential categories of APIs here might be: * Application data storage and management to keep your data in one place and the user's system tidy * Cloud app support for identity and connectivity management to use the power of Azure * Packaging and system integration so Windows understands what your app is doing * App-to-app inter-process communication helpers so your apps can work with other app easily * Privacy-aware resource access APIs for camera, microphone, location, and more You'll start using these APIs by adding a reference to the Windows App SDK NuGet package and then adding uses of the APIs in your app code. Use as much or as little of Windows App SDK as helps your application. New Windows App SDK APIs will be available on all supported versions of Windows and for all apps. Where certain APIs behave differently (such as due to identity, isolation, or lifecycle differences) your app can detect it with clear "TheType.IsSupported" checks. ## Converged APIs Windows App SDK provides API surfaces that bridge the gap between Win32 and UWP/AppContainer functionality already within the platform. Authors of shared code (like frameworks) can use a single method and avoid their own "if (AppContainer) { x } else { y }" checks. Windows App SDK APIs provide "polyfill" implementations of APIs when necessary so your app can run on more versions of Windows without additional "is this Windows 10 Creator's Release?" checks. Converged API documentation also includes ways to find uses of the "un converged" code in your project, along with guidance on how to move from common code patterns to the new Windows App SDK converged APIs. Examples of converged APIs include: * Window Content Frame abstraction over Win32 and UWP windowing * Application lifecycle behaviors like start, logoff, restart manager integration, restart-for-update * Startup and Background Task support to reduce your app's power and performance footprint * Application resource management for strings, images, display resolutions, and locales * Advanced clipboard access Once your app adds a reference to the Windows App SDK codebase your app can start migrating blocks of functionality to using these fully-supported converged APIs. You can move as much of your application to Windows App SDK functionality as necessary, keeping your existing application style and behaviors. Whenever possible these APIs will be similar to their existing Windows SDK namespaced APIs, easing the transition to Windows App SDK for your existing code. For existing "flat C" Win32 APIs, an equivalent converged API will be added and then projected to your language and runtime. If you prefer a "flat C" style - or the APIs you want already have a "flat C" expression - we're working on building out a projection of those APIs for you. As your app migrates to Windows App SDK functionality it'll be easier to move between deployment and isolation technologies. Windows App SDK handles the brokering of resource access from a low integrity-level (low-IL) or AppContainer process so your app can reference user resources while respecting user choice & privacy controls. ## Subset API Family Windows App SDK helps you stay within the set of APIs that Windows supports on all editions and all endpoints. Like the Windows Kit API partitions, Windows App SDK defines a subset of supported Windows Platform APIs that work across all versions of Windows. If your code targets this subset - and uses Windows App SDK New + Windows App SDK Converged functionality whenever possible - it'll work everywhere Windows works, without needing additional support work. Windows App SDK Subset APIs for your apps also let you start using Windows App SDK's functionality with minimal changes to your existing code by using an updated and streamlined set of headers and implementations. Feature areas supported by the Windows App SDK Subset include subsets of: * Windowing, Input, Messaging, GDI, and GUI subsystem functionality * Filesystem and storage access * Networking * Printing * Process, threading, memory management, basic application services * DirectX, D3D, DirectML Migrate your app to the Windows App SDK Subset API set by first removing all references to Windows Kit headers (windows.h, kernel32.lib, user32.lib, winsock.h, etc) and then adding references to the core Windows App SDK Subset header and import libraries. While many existing Windows Kit APIs are part of the Windows App SDK Subset, you may have some minimal changes to make to your application's code or behavior. Recompile & relink your application then ship it with our support packages. Is this "subset" model interesting for you and your apps? This feature of Windows App SDK is still in planning for a future release. Let us know by [filing an issue](https://github.com/microsoft/WindowsAppSDK/issues/new/choose) with the "**subset**" tag on it and list the APIs and functionality you'd like to see. # Coming soon While we think these are interesting for applications, you tell us! Part of working in the open is that we learn together about the needs of application developers. Below are a set of functionality that seems interesting to make available through a Converged or New model, but you tell us! Some are linked to an issue thread proposing the functionality. +1 the ones you like, comment on the detailed design & direction, and help us shape where we're going. * **Edge/Chromium-based WebView2** brings the ease of authoring your app in HTML+JS once and reusing it on all platforms. WebView2's use of Windows App SDK technologies like WinUI3 as a hosting frame lets you run modern web-based UX on all editions of Windows. * **Update Scan Integration** help keep your app up to date automatically while running at the same time as other system maintenance tasks are happening. * **Access to user resources** even from AppContainer and isolated applications. Your AppContainer apps can access powerful Win32 technologies like the clipboard, inter-process communication, and the Windows Shell Namespace with user consent through brokering. # Future direction We aim to grow Windows App SDK to provide a modern, simple, and accessible version of much of the power of the Windows Application Platform. Working in the open with our development community ensures we're doing the right things first to solve problems your apps are facing on current and future versions of Windows. Some functionality will be delivered first as Windows App SDK components. As we think about evolving the platform it's important to make sure new functionality & features are available to our developer community just as soon as they're ready to go. Where there's newer in-platform functionality to use Windows App SDK will help you adopt that as soon as it's widely available, without you having to retarget or rewrite - the same Windows App SDK API surface will continue to work. As your app uses more Windows App SDK functionality it'll be ready for additional targets such as packaging, identity, AppContainer, and other future Windows Editions. Depending on Windows App SDK APIs means we take on the work to keep these APIs working as Windows evolves forward. Staying within the Windows App SDK API surface ensures your app can address the widest range of Windows editions and releases while using up-to-date functionality. # How we are doing this ## Extending APIs Some of what Windows App SDK provides is access to functionality that is inside Windows, but for which there is no public API. We're working on the details, but our plan is to deliver a secondary package of code that provides a public API over those previously unexposed components. While those components will be implemented as closed-source they'll come with a metadata-based publicly callable & supported API surface and be included in the Windows App SDK framework packages for use by apps. # What Windows App SDK isn't Windows App SDK **isn't a new application model** or platform from Windows. There won't be a "new Windows App SDK App" template for Visual Studio, VSCode, or other development environments. You'll still have full access to the Windows SDK and associated kits. Over time the features that Windows App SDK provides will grow beyond just merging the existing Win32 and UWP models and provide additional functionality for all apps to use. Windows App SDK **isn't a new packaging or isolation model** for applications. There won't be required changes to your application in order to access Windows App SDK functionality other than using the new functionality itself. If the APIs you use need identity or packaging these requirements will be indicated. Windows App SDK **isn't a new security or privacy model** for applications. Your app's existing security, privacy, reliability, and identity stories continue to work with Windows App SDK. There may be places that Windows App SDK helps your app be more aware of customer security or privacy concerns. Windows App SDK helps you use modern security & privacy functionality that evolves alongside your application's needs. Windows App SDK **isn't a way to run your app in the cloud**. Using Windows App SDK technology will help get your app on modern API families that are cloud ready. Key components like modern lifecycle and state isolation help get your app ready to run wherever your customers are. ## 2. Official Technical Reference & Guides (microsoft/ai-dev-gallery) ## File: README.md

AI Dev Gallery (Preview)

> [!IMPORTANT] > The AI Dev Gallery is currently in public preview, and weโ€™d love your feedback! Share your thoughts by creating an [issue](https://github.com/microsoft/ai-dev-gallery/issues?q=sort%3Aupdated-desc+is%3Aissue+is%3Aopen). Designed for Windows developers, the AI Dev Gallery helps integrate AI capabilities into apps and projects. It includes: - **Explore over 25 interactive samples powered by local AI models** - **Easily browse, download, and run models from Hugging Face and GitHub** - **View the C# source code and export standalone Visual Studio projects with a single click** ## ๐Ÿš€ Getting started Download [AI Dev Gallery from the Microsoft Store](http://aka.ms/ai-dev-gallery-store) or follow these steps to install it manually: ### 1. Set up the environment >**โš ๏ธ Note**: AI Dev Gallery requires [Visual Studio 2022](https://visualstudio.microsoft.com/vs/) or later for building and Windows 10 or newer to run. If you're new to building apps with WinUI and the Windows App SDK, follow the [installation instructions](https://learn.microsoft.com/windows/apps/get-started/start-here). **Required [Visual Studio components](https://learn.microsoft.com/windows/apps/get-started/start-here?tabs=vs-2022-17-10#required-workloads-and-components):** - Windows application development ### 2. Clone the repository ```shell git clone https://github.com/microsoft/AI-Dev-Gallery.git ``` ### 3. Open AIDevGallery.sln with Visual Studio! Ensure that the `AIDevGallery` project is set as the startup project in Visual Studio. Press F5 to run AI Dev Gallery! >**โš ๏ธ Note**: On ARM64-based Copilot+ PCs, make sure to build and run the solution as `ARM64` (and not as `x64`). This is required especially when running the samples that communicate with models such as Phi Silica. >**โš ๏ธ Note**: Having issues installing the app on your machine? Let us know by [opening an issue ](https://github.com/microsoft/AI-Dev-Gallery/issues) and our team will do our best to help! ## ๐Ÿ’ป Device requirements - **Minimum OS version**: Windows 10, version 1809 (10.0; Build 17763) - **Architecture**: x64, ARM64 - **Memory**: At least 16 GB is recommended - **Disk space**: At least 20GB free space is recommended - **GPU**: 8GB of VRAM is recommended for running samples on the GPU ## ๐Ÿ‘ Contribute to AI Dev Gallery Any samples or docs improvements you'd like to see? We're always looking for a helping hand. Feel free to [open an issue ](https://github.com/microsoft/AI-Dev-Gallery/issues) to start the discussion, or even better, create a PR with the change you'd like to see! ## โ“ FAQs - **Q: Is a Microsoft account necessary to use the app?** - **A:** No, the app does NOT require a Microsoft account for use. - **Q: Can I use the app without an internet connection?** - **A:** Yes, the app works offline since the AI models are downloaded locally. However, you will need to be online to download additional AI models from Hugging Face or GitHub. - **Q: What AI models are available in the app?** - **A:** The app features popular open source models and APIs from the [Microsoft Foundry on Windows](https://learn.microsoft.com/windows/ai/overview). When executing a sample, you can select which model you want to use. - **Q: Is the app's source code accessible? Can I contribute new samples?** - **A:** Yes, the app is completely open-source, and its code is accessible on [GitHub](https://github.com/microsoft/AI-Dev-Gallery). Feel free to contribute by [filing an issue](https://github.com/microsoft/AI-Dev-Gallery/issues), or submitting a PR and one of our moderators will review it. - **Q: Where can I provide feedback?** - **A:** Feel free to give us feedback or [open an issue](https://github.com/microsoft/AI-Dev-Gallery/issues/new) on our GitHub repository. - **Q: Do I need to run the app just to try a single sample?** - **A:** Yes, the app is required to run any sample. However, once you've downloaded a model for a sample via the app, you can export the sample as a Visual Studio project and run it independently from there. ## โœจ Contributing This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com. When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA. This project has adopted the [Microsoft Open Source Code of Conduct](https://opensource.microsoft.com/codeofconduct/). For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments. ## ๐Ÿ“š Trademarks This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow [Microsoft's Trademark & Brand Guidelines](https://www.microsoft.com/en-us/legal/intellectualproperty/trademarks/usage/general). Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies. ## ๐Ÿ”” Code of Conduct This project has adopted the [ Microsoft Open Source Code of Conduct](https://github.com/microsoft/AI-Dev-Gallery/blob/main/CODE_OF_CONDUCT.md). ## ๐Ÿ“ข Privacy Statement The application logs basic telemetry. Please read the [ Microsoft privacy statement](http://go.microsoft.com/fwlink/?LinkId=521839) for more information. --- ## File: docs/AddingSamples.md Read this doc on how to add a sample to the app. **Note**: The app is very much in early stages of development and the sample architecture changes frequently - I will try to keep this file updated, but it's likely that it could be out of date. ## Samples All samples go in the `Samples` folder. The samples are loaded dynamically, so all you need to do is to annotate your sample class with the [GallerySample] attribute to get them to show up in the app. You also need to know the structure of the Samples folder. A sample is defined by a `.xaml` and a `.xaml.cs` file, where the class in your `.xaml.cs` needs to be annotated with the [GallerySample] attribute. The [GallerySample] attribute contains metadata about the sample, and the `.xaml.cs`/`.xaml` file is the entry point to the sample. You can add your files anywhere within the Samples folder. Try to group them properly. If your sample uses a Model that hasn't been used before in the gallery app, you might also need to include a new model definition file inside the `Samples\Definitions\Models\` directory. Two formats are supported: - `.model.json` - A simpler format for defining a single model family - `.modelgroup.json` - Used to define a group of related model families with additional grouping metadata If your sample uses a specific API, add it to an `apis.json` file anywhere under the `Samples\Definitions\` directory. The `.model.json`, `.modelgroup.json`, and `apis.json` files are used to group samples together in the app and they show up as a single page where all samples are rendered at the same time. A sample folder that is not inside a model or api folder renders as a single sample in the app. These are good for samples that are higher level concepts (such as guides), or samples that combine multiple models or APIs. ### `Sample.xaml` and `Sample.xaml.cs` This is the entry point to the sample - whatever you put here will show up when the sample is loaded. If you are creating a sample for a model, your sample should inherit from `BaseSamplePage` and override the `LoadModelAsync` method. The model downloading is handled for you. To load the model, override the LoadModelAsync method in your Sample.xaml.cs file: ```csharp protected override async Task LoadModelAsync(SampleNavigationParameters sampleParams) { try { // For language models, get an IChatClient instance chatClient = await sampleParams.GetIChatClientAsync(); // sampleParams.NotifyCompletion() should be called when the model is ready // This will hide the loading spinner in the UI } catch (Exception ex) { ShowException(ex); } sampleParams.NotifyCompletion(); } ``` If your sample requires two models (for example, a language model plus an embeddings model), override the multi-model overload instead: ```csharp protected override async Task LoadModelAsync(MultiModelSampleNavigationParameters sampleParams) { try { // Example: initialize second model using sampleParams.ModelPaths[1] // ... your model-2 init code ... // For language models, still use the chat client helper chatClient = await sampleParams.GetIChatClientAsync(); } catch (Exception ex) { ShowException(ex); } sampleParams.NotifyCompletion(); } ``` It is also important to ensure that the model and any resources are disposed when the page is unloaded. To do this, register a handler to the `Unloaded` event in the constructor of the Sample.xaml.cs file: ```csharp public Sample() { this.Unloaded += (s, e) => { // clean up code goes here // eg: model.Dispose(); }; this.InitializeComponent(); } ``` ### `[GallerySample]` This is an attribute that contains the sample icon, links and more that show up across the app - the properties changes as new metadata is introduced, so look at how other samples are done for what to put here. ```csharp [GallerySample( Name = "Paraphrase", Model1Types = [ModelType.LanguageModels, ModelType.PhiSilica], Scenario = ScenarioType.TextParaphraseText, NugetPackageReferences = [ "Microsoft.Extensions.AI" ], Id = "9e006e82-8e3f-4401-8a83-d4c4c59cc20c", Icon = "\uE8D4")] ``` Key properties: - `Name`: Display name of the sample - `Model1Types`: Array of model types this sample supports (required) - `Model2Types`: Optional second model type array if the sample uses multiple models - `Scenario`: The scenario type defined in `scenarios.json` - `SharedCode`: Optional array of shared code items for sample export; values come from `SharedCodeEnum` (auto-generated from files under `Samples\SharedCode` and `AIDevGallery\Utils\DeviceUtils`). Most basic language model samples can omit this. - `NugetPackageReferences`: Array of NuGet package names needed - `AssetFilenames`: Array of asset files needed by this sample - `Id`: Unique identifier (GUID) - `Icon`: Icon glyph for display The `Scenario` is used to group samples together by scenario. If you have multiple samples that are demonstrating the same scenario (for example: summarizing text), you can give them the same `Scenario` and they will show up together in the app. The scenario must be first defined in `Samples\scenarios.json`. This file is embedded at build time by a source generator โ€” edit it and rebuild to take effect (it is not loaded dynamically at runtime). Prompt templates for language models are defined in `Samples\promptTemplates.json` and are also embedded by a source generator. ### `.model.json` and `.modelgroup.json` Both `.model.json` and `.modelgroup.json` formats are supported for defining models. The main difference is: - `.model.json` - Used for a single model family - `.modelgroup.json` - Used for grouping multiple related model families with additional metadata like display order #### `.model.json` format This format defines a single model family with its variations: ```json { "Phi3Mini": { "Id": "phi3mini", "Name": "Phi 3 Mini", "Description": "Phi-3 Mini is a 3.8B parameter, lightweight, state-of-the-art open language model", "DocsUrl": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx", "Models": { "Phi3MiniDirectML": { "Id": "202d6eaa-7a65-4d40-b2c3-53f1ef12a4eb", "Name": "Phi 3 Mini DirectML", "Url": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/tree/main/directml/directml-int4-awq-block-128", "Description": "Phi 3 Mini DirectML will run on your GPU", "HardwareAccelerator": "GPU", "Size": 2135763374, "Icon": "Microsoft.svg", "ParameterSize": "3.8B", "PromptTemplate": "Phi3", "License": "mit" }, "Phi3MiniCPU": { ... } }, "ReadmeUrl": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/blob/main/README.md" } } ``` #### `.modelgroup.json` format This format groups multiple related model families together with additional metadata like display order: ```json { "LanguageModels": { "Id": "615e2f1c-ea95-4c34-9d22-b1e8574fe476", "Name": "Language", "Icon": "\uE8BD", "Order": 1, "Models": { "Phi3Mini": { "Id": "phi3mini", "Name": "Phi 3 Mini", "Description": "Phi-3 Mini is a 3.8B parameter, lightweight, state-of-the-art open language model", "DocsUrl": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx", "Models": { "Phi3MiniDirectML": { "Id": "202d6eaa-7a65-4d40-b2c3-53f1ef12a4eb", "Name": "Phi 3 Mini DirectML", "Url": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/tree/main/directml/directml-int4-awq-block-128", "Description": "Phi 3 Mini DirectML will run on your GPU", "HardwareAccelerator": "GPU", "Size": 2135763374, "Icon": "Microsoft.svg", "ParameterSize": "3.8B", "PromptTemplate": "Phi3", "License": "mit", "AIToolkitActions": ["Playground", "PromptBuilder"], "AIToolkitId": "Phi-3-mini-4k-directml-int4-awq-block-128-onnx" }, "Phi3MiniCPU": { ... } }, "ReadmeUrl": "https://huggingface.co/microsoft/Phi-3-mini-4k-instruct-onnx/blob/main/README.md" } } } } ``` #### Key fields for both formats: **Model Family fields** (applies to both `.model.json` and models within `.modelgroup.json`): - `Id`: Unique identifier for the model family - `Name`: Display name - `Description`: Description of the model family - `DocsUrl`: External link to the model family's documentation homepage/repository overview, used for external link and source display. - `ReadmeUrl`: Direct link to the model family's README markdown that the app fetches and renders in-app - `Models`: Dictionary of model variations with different hardware requirements - `Url`: HuggingFace URL - can point to repo root, a subfolder, or a single file - `HardwareAccelerator`: One or more of: "CPU", "GPU", "DML", "QNN", "NPU", "WCRAPI", "OLLAMA", "OPENAI", "VitisAI", "OpenVINO" (or an array like ["CPU", "GPU"]) - `Size`: Model size in bytes - `License`: License type (e.g., "mit", "apache-2.0") - `PromptTemplate`: Prompt template to use (for language models) - `SupportedOnQualcomm`: Whether the model is supported on Qualcomm devices (optional) - `Icon`: Icon filename shown for the model (optional) - `FileFilter`: A string or array of strings to filter selectable files (optional) - `AIToolkitActions`: Actions available in AI Toolkit (optional) - `AIToolkitId`: AI Toolkit identifier (optional) - `AIToolkitFinetuningId`: AI Toolkit finetuning identifier (optional) - `InputDimensions` / `OutputDimensions`: Model input/output dimensions (optional) **Additional fields for `.modelgroup.json` only:** - `Order`: Display order in the app (optional) - this field is only available in `.modelgroup.json` - `Models`: Dictionary of model families within this group (in `.modelgroup.json`, this wraps multiple model families) ## Adding a sample To add a sample, follow these steps: 1. **Identify the model or API**: Determine which model or Windows AI API your sample will use. 2. **Create or update model definitions** (if needed): If your sample uses a Model that hasn't been used before in the gallery app, you might also need to include a new model file (`.model.json`)inside the `Samples\Definitions\` directory. - The `.model.json` file is used to define a collection of samples that use a specific model family - You can also add a `.modelgroup.json` file to organize the model family within a model group - The `apis.json` file is used to define a collection of samples that use a specific API 3. **Create the sample files**: Add a new WinUI Blank Page in the appropriate folder: `Samples\[CategoryName]\[SampleName].xaml` 4. **Implement the sample**: - Make your sample class inherit from `BaseSamplePage` - Add the `[GallerySample]` attribute with appropriate metadata (use `Model1Types` array, `Scenario`, etc.) - Override `LoadModelAsync(SampleNavigationParameters sampleParams)` for single-model samples, or `LoadModelAsync(MultiModelSampleNavigationParameters sampleParams)` for dual-model samples - Use `await sampleParams.GetIChatClientAsync()` for language models or appropriate methods for other model types (See [Common entry points cheat sheet](#common-entry-points)) - For WinML-based models, use `sampleParams.WinMlSampleOptions` for EP policy, device type, and compile options - Call `sampleParams.NotifyCompletion()` when initialization is complete - Register cleanup code in the `Unloaded` event handler 5. **Test**: Run the app - the sample should show up as part of the model or API collection, or as a standalone page if it's not part of a collection. ### Common entry points - Language models - Use from navigation params: `await sampleParams.GetIChatClientAsync()` - Factory (direct use): `await OnnxRuntimeGenAIChatClientFactory.CreateAsync(modelDir, LlmPromptTemplate?)` - Phi Silica: `await PhiSilicaClient.CreateAsync()` - Embeddings - Create: `var embeddings = await EmbeddingGenerator.CreateAsync(modelPath, sampleParams.WinMlSampleOptions)` - Use: `await embeddings.GenerateAsync(values)` or `await foreach (var v in embeddings.GenerateStreamingAsync(values)) { ... }` - Speech (Whisper) - Create: `var whisper = await WhisperWrapper.CreateAsync(modelPath, sampleParams.WinMlSampleOptions)` - Use: `await whisper.TranscribeAsync(pcmBytes, language, WhisperWrapper.TaskType.Transcribe)` - Image generation (Stable Diffusion) - Init: `var sd = new StableDiffusion(modelFolder); await sd.InitializeAsync(sampleParams.WinMlSampleOptions);` - Inference: `var image = sd.Inference(prompt, cancellationToken);` --- ## File: docs/PublishingNewVersion.md # Publishing a New Version of the AI Dev Gallery App This document describes the steps to publish a new version of the AI Dev Gallery app to the Microsoft Store. You need to be a member of the AI Dev Gallery App in order to publish a new version of the app. > Note: The current version of the app is stored in the `version.json` file in the root of the repository. The version number is in the format `X.Y.Z`, where `X` is the major version, `Y` is the minor version, and `Z` is the patch version. If you want to publish a new version of the app with a different version number (for example a new major or minor version), you need to update the version number in the `version.json` file before following the steps below. 1. Make sure you don't have any local uncommited changes, and make sure your local clone is in sync with the remote repository by running the following commands: ```console git checkout main git pull ``` 2. Ensure that you have the latest version of nbgv installed on your machine. If not, you can install it by running the following command: ```console dotnet tool install --global nbgv ``` 3. Run the following command to prepare the release and update the version number in the version.json file: ```console nbgv prepare-release ``` This will update the version number in the `version.json` file on the current branch (main), and create a commit with an updated version number. By default it will increment the patch version number, but you can find more information about nbgv's cli [here](https://github.com/dotnet/Nerdbank.GitVersioning/blob/main/docfx/docs/nbgv-cli.md). It will also create a new branch with the name `rel/vX.Y.Z` where `X.Y.Z` is the `current` version number. 4. Now you can push the changes to the remote repository by running the following command: ```console git checkout rel/vX.Y.Z git push -u origin rel/vX.Y.Z ``` 5. To make sure eveybody is now working on the new release, make a pull request from "alias/publish_X.Y.Z" branch to "main" branch. ```console git checkout main git checkout -b alias/publish_X.Y.Z git push -u origin alias/publish_X.Y.Z ``` > Note: This will automatically spin up a new release build in ADO, which automatically uploads the new version to the Microsoft Store.