File: README.md
Brendan Bycroft's Home Page & Projects
This repository contains my (Brendan's) homepage, as well as a number of non-trivial projects.
They are kept in a single repository for ease of deployment, as well as sharing a bunch of js utils
which are otherwise a pain to share around.
License: MIT (see LICENSE), except the personal homepage (src/homepage/, public/me.jpeg)
and third-party assets (fonts, RISC-V tests, etc.), which retain their original rights/licenses.
Projects
The main projects are:
- LLM Visualization: 3D interactive model of a GPT-style LLM network running inference.
- [WIP] CPU Simulation: A 2D digital schematic editor with full a execution model, showcasing a simple
RISC-V based CPU
LLM Visualization
This project displays a 3D model of a working implementation of a GPT-style network. That
is, the network topology that's used in OpenAI's GPT-2, GPT-3, (and maybe GPT-4).
The first network displayed with working weights is a tiny such network, which sorts a small list
of the letters A, B, and C. This is the demo example model from Andrej Karpathy's
minGPT implementation.
The renderer also supports visualizing arbitrary sized networks, and works with the smaller gpt2
size, although the weights aren't downloaded (it's 100's of MBs).
CPU Simulation (WIP; not exposed yet!)
This project runs 2D schematic digital circuits, with a fully fledged editor. The intent is to
add a number of walkthroughs, showing things such as:
- how a simple RISC-V CPU is constructed
- the constituent parts down to gate level: instruction decode, ALU, add, etc
- higher level CPU ideas, like various levels of pipelining, caching, etc
Running Locally
- Install dependencies:
yarn - Start the dev server:
yarn dev
File: ols.json
{
"collections": [
{
"name": "core",
"path": "C:\Software\odin\core"
}
],
"checker_args": "-target:js_wasm32",
"enable_document_symbols": true,
"enable_semantic_tokens": false,
"enable_hover": true,
"enable_snippets": true
}
File: package.json
{
"name": "llm-viz",
"private": true,
"license": "MIT",
"scripts": {
"dev": "next dev -p 3002",
"build": "next build",
"lint": "eslint .",
"start": "next start",
"typecheck": "tsc --noEmit"
},
"dependencies": {
"@fortawesome/fontawesome-svg-core": "6.4.0",
"@fortawesome/free-brands-svg-icons": "6.4.0",
"@fortawesome/free-regular-svg-icons": "6.4.0",
"@fortawesome/free-solid-svg-icons": "6.4.0",
"@fortawesome/react-fontawesome": "^3.5.0",
"clsx": "2.0.0",
"d3-color": "3.1.0",
"next": "16.3.0",
"react": "19.2.8",
"react-dom": "19.2.8"
},
"devDependencies": {
"@next/bundle-analyzer": "16.3.0",
"@types/d3-color": "3.1.0",
"@types/node": "20.1.0",
"@types/react": "19.2.18",
"@types/react-dom": "19.2.4",
"@webgpu/types": "0.1.32",
"add": "2.0.6",
"autoprefixer": "10.4.15",
"eslint": "10.8.1",
"eslint-config-next": "16.3.0",
"eslint-plugin-react": "7.33.1",
"eslint-plugin-react-hooks": "4.6.0",
"eslint-plugin-react-refresh": "0.4.5",
"msdf-bmfont-xml": "2.7.0",
"postcss": "8.4.29",
"postcss-import": "15.1.0",
"postcss-nesting": "12.0.1",
"postcss-scss": "4.0.8",
"sass": "1.66.1",
"svg2ttf": "6.0.3",
"svgicons2svgfont": "12.0.0",
"tailwindcss": "3.3.3",
"typescript": "5.2.2",
"yarn": "1.22.21"
},
"resolutions": {
"jimp": "0.22.10",
"update-notifier": "6.0.2",
"xml2js": "0.5.0",
"@types/react": "19.2.18",
"@types/react-dom": "19.2.4"
},
"eslintConfig": {
"extends": [
"next",
"plugin:react/recommended"
],
"plugins": [
"react-refresh"
],
"rules": {
"react/prop-types": "off",
"react-hooks/exhaustive-deps": "error",
"react/no-unescaped-entities": "off",
"@next/next/no-img-element": "off",
"@next/next/no-page-custom-font": "off",
"react-refresh/only-export-components": "off"
}
},
"packageManager": "[email protected]"
}
File: tsconfig.json
{
"compilerOptions": {
"target": "ES2020",
"lib": [
"dom",
"dom.iterable",
"esnext"
],
"allowJs": true,
"skipLibCheck": true,
"strict": true,
"forceConsistentCasingInFileNames": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "react-jsx",
"incremental": true,
"paths": {
"@/src/": [
"./src/"
],
"@/styles/": [
"./styles/"
]
},
"plugins": [
{
"name": "next"
}
]
},
"include": [
"next-env.d.ts",
"/*.ts",
"/.tsx",
".next/types/**/.ts",
".next/dev/types/**/*.ts"
],
"exclude": [
"node_modules"
]
}
File: .yarnrc.yml
nodeLinker: node-modules
yarnPath: .yarn/releases/yarn-4.1.0.cjs
File: fonts/cmmi12.json
{
"pages": [
"font-atlas.png"
],
"chars": [
{
"id": 88,
"index": 133,
"char": "X",
"width": 38,
"height": 33,
"xoffset": -1,
"yoffset": 3,
"xadvance": 34,
"chnl": 15,
"x": 303,
"y": 189,
"page": 0
},
{
"id": 98,
"index": 75,
"char": "b",
"width": 19,
"height": 34,
"xoffset": 0,
"yoffset": 3,
"xadvance": 17,
"chnl": 15,
"x": 318,
"y": 111,
"page": 0
},
{
"id": 89,
"index": 86,
"char": "Y",
"width": 34,
"height": 33,
"xoffset": -1,
"yoffset": 3,
"xadvance": 24,
"chnl": 15,
"x": 306,
"y": 223,
"page": 0
},
{
"id": 67,
"index": 103,
"char": "C",
"width": 33,
"height": 34,
"xoffset": 0,
"yoffset": 2,
"xadvance": 29,
"chnl": 15,
"x": 319,
"y": 146,
"page": 0
},
{
"id": 90,
"index": 46,
"char": "Z",
"width": 32,
"height": 33,
"xoffset": 0,
"yoffset": 3,
"xadvance": 28,
"chnl": 15,
"x": 341,
"y": 223,
"page": 0
},
{
"id": 84,
"index": 120,
"char": "T",
"width": 32,
"height": 33,
"xoffset": -1,
"yoffset": 3,
"xadvance": 24,
"chnl": 15,
"x": 342,
"y": 181,
"page": 0
},
{
"id": 66,
"index": 52,
"char": "B",
"width": 33,
"height": 33,
"xoffset": 0,
"yoffset": 3,
"xadvance": 31,
"chnl": 15,
"x": 374,
"y": 215,
"page": 0
},
{
"id": 121,
"index": 114,
"char": "y",
"width": 23,
"height": 31,
"xoffset": -1,
"yoffset": 13,
"xadvance": 20,
"chnl": 15,
"x": 305,
"y": 27,
"page": 0
},
{
"id": 116,
"index": 48,
"char": "t",
"width": 17,
"height": 31,
"xoffset": -1,
"yoffset": 6,
"xadvance": 15,
"chnl": 15,
"x": 319,
"y": 59,
"page": 0
},
{
"id": 120,
"index": 63,
"char": "x",
"width": 24,
"height": 23,
"xoffset": -1,
"yoffset": 13,
"xadvance": 23,
"chnl": 15,
"x": 317,
"y": 0,
"page": 0
},
{
"id": 122,
"index": 70,
"char": "z",
"width": 22,
"height": 23,
"xoffset": 0,
"yoffset": 13,
"xadvance": 19,
"chnl": 15,
"x": 329,
"y": 24,
"page": 0
},
{
"id": 99,
"index": 127,
"char": "c",
"width": 20,
"height": 23,
"xoffset": 0,
"yoffset": 13,
"xadvance": 18,
"chnl": 15,
"x": 342,
"y": 0,
"page": 0
}
],
"info": {
"face": "cmmi12",
"size": 42,
"bold": 0,
"italic": 0,
"charset": [
"x",
"y",
"z",
"t",
"c",
"b",
"X",
"Y",
"Z",
"T",
"C",
"B"
],
"unicode": 1,
"stretchH": 100,
"smooth": 1,
"aa": 1,
"padding": [
2,
2,
2,
2
],
"spacing": [
0,
0
],
"outline": 0
},
"common": {
"lineHeight": 38,
"base": 32,
"scaleW": 512,
"scaleH": 256,
"pages": 1,
"packed": 0,
"alphaChnl": 0,
"redChnl": 0,
"greenChnl": 0,
"blueChnl": 0
},
"distanceField": {
"fieldType": "msdf",
"distanceRange": 4
},
"kernings": []
}
File: fonts/cmr12.json
{
"pages": [
"font-atlas.png"
],
"chars": [
{
"id": 63,
"index": 50,
"char": "?",
"width": 19,
"height": 34,
"xoffset": 0,
"yoffset": 2,
"xadvance": 19,
"chnl": 15,
"x": 395,
"y": 123,
"page": 0
},
{
"id": 61,
"index": 41,
"char": "=",
"width": 31,
"height": 13,
"xoffset": 0,
"yoffset": 17,
"xadvance": 32,
"chnl": 15,
"x": 400,
"y": 189,
"page": 0
},
{
"id": 59,
"index": 31,
"char": ";",
"width": 8,
"height": 30,
"xoffset": 2,
"yoffset": 14,
"xadvance": 11,
"chnl": 15,
"x": 408,
"y": 158,
"page": 0
},
{
"id": 58,
"index": 85,
"char": ":",
"width": 8,
"height": 22,
"xoffset": 2,
"yoffset": 14,
"xadvance": 11,
"chnl": 15,
"x": 352,
"y": 24,
"page": 0
},
{
"id": 44,
"index": 21,
"char": ",",
"width": 9,
"height": 16,
"xoffset": 2,
"yoffset": 28,
"xadvance": 11,
"chnl": 15,
"x": 319,
"y": 91,
"page": 0
},
{
"id": 45,
"index": 84,
"char": "-",
"width": 15,
"height": 6,
"xoffset": -2,
"yoffset": 22,
"xadvance": 14,
"chnl": 15,
"x": 131,
"y": 250,
"page": 0
},
{
"id": 46,
"index": 30,
"char": ".",
"width": 8,
"height": 8,
"xoffset": 2,
"yoffset": 28,
"xadvance": 11,
"chnl": 15,
"x": 0,
"y": 248,
"page": 0
}
],
"info": {
"face": "cmr12",
"size": 42,
"bold": 0,
"italic": 0,
"charset": [
"=",
"?",
";",
":",
",",
"-",
"."
],
"unicode": 1,
"stretchH": 100,
"smooth": 1,
"aa": 1,
"padding": [
2,
2,
2,
2
],
"spacing": [
0,
0
],
"outline": 0
},
"common": {
"lineHeight": 38,
"base": 32,
"scaleW": 512,
"scaleH": 256,
"pages": 1,
"packed": 0,
"alphaChnl": 0,
"redChnl": 0,
"greenChnl": 0,
"blueChnl": 0
},
"distanceField": {
"fieldType": "msdf",
"distanceRange": 4
},
"kernings": []
}
File: fonts/cmsy10.json
{
"pages": [
"font-atlas.png"
],
"chars": [
{
"id": 112,
"index": 36,
"char": "p",
"width": 37,
"height": 46,
"xoffset": 1,
"yoffset": 30,
"xadvance": 35,
"chnl": 15,
"x": 337,
"y": 48,
"page": 0
},
{
"id": 106,
"index": 47,
"char": "j",
"width": 6,
"height": 46,
"xoffset": 3,
"yoffset": 1,
"xadvance": 12,
"chnl": 15,
"x": 363,
"y": 0,
"page": 0
},
{
"id": 45,
"index": 65,
"char": "-",
"width": 42,
"height": 42,
"xoffset": -1,
"yoffset": 2,
"xadvance": 42,
"chnl": 15,
"x": 370,
"y": 0,
"page": 0
},
{
"id": 43,
"index": 68,
"char": "+",
"width": 27,
"height": 41,
"xoffset": -1,
"yoffset": 3,
"xadvance": 26,
"chnl": 15,
"x": 338,
"y": 95,
"page": 0
},
{
"id": 40,
"index": 104,
"char": "(",
"width": 41,
"height": 27,
"xoffset": 0,
"yoffset": 10,
"xadvance": 42,
"chnl": 15,
"x": 353,
"y": 137,
"page": 0
},
{
"id": 41,
"index": 112,
"char": ")",
"width": 41,
"height": 27,
"xoffset": 0,
"yoffset": 10,
"xadvance": 42,
"chnl": 15,
"x": 366,
"y": 95,
"page": 0
},
{
"id": 47,
"index": 15,
"char": "/",
"width": 32,
"height": 23,
"xoffset": 0,
"yoffset": 13,
"xadvance": 33,
"chnl": 15,
"x": 375,
"y": 165,
"page": 0
},
{
"id": 161,
"index": 17,
"char": "ยก",
"width": 30,
"height": 6,
"xoffset": 1,
"yoffset": 21,
"xadvance": 33,
"chnl": 15,
"x": 342,
"y": 215,
"page": 0
},
{
"id": 163,
"index": 13,
"char": "ยฃ",
"width": 24,
"height": 24,
"xoffset": 4,
"yoffset": 11,
"xadvance": 33,
"chnl": 15,
"x": 375,
"y": 189,
"page": 0
}
],
"info": {
"face": "cmsy10",
"size": 42,
"bold": 0,
"italic": 0,
"charset": [
"-",
"+",
"/",
"(",
")",
"ยฃ",
"p",
"ยก",
"j"
],
"unicode": 1,
"stretchH": 100,
"smooth": 1,
"aa": 1,
"padding": [
2,
2,
2,
2
],
"spacing": [
0,
0
],
"outline": 0
},
"common": {
"lineHeight": 70,
"base": 32,
"scaleW": 512,
"scaleH": 256,
"pages": 1,
"packed": 0,
"alphaChnl": 0,
"redChnl": 0,
"greenChnl": 0,
"blueChnl": 0
},
"distanceField": {
"fieldType": "msdf",
"distanceRange": 4
},
"kernings": []
}
File: public/fonts/font-def.json
{"faces":[{"name":"Roboto-Regular","common":{"lineHeight":44,"base":34,"scaleW":512,"scaleH":256,"pages":1,"packed":0,"alphaChnl":0,"redChnl":0,"greenChnl":0,"blueChnl":0},"info":{"face":"Roboto-Regular","size":42,"bold":0,"italic":0,"charset":"! "#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~โง\โฮฃฮณฮฒฯฮผฮต","unicode":1,"stretchH":100,"smooth":1,"aa":1,"padding":[2,2,2,2],"spacing":[0,0],"outline":0},"chars":"KAAMACgAAAAAAA8ALwABAAAADgAAAA8AKQANACkAAAAwAA8ALwD//wAADwAAAA8AWwA/AFsAAABgAAwALQABAAAACwAAAA8AXQBBAF0AAACOAAwALQD+/wAACwAAAA8AewBfAHsAAAC8ABAALAD//wEADgAAAA8AfQBhAH0ADQBgABAALAD+/wEADgAAAA8AJAAIACQADQCNABcAKwAAAP//GAAAAA8AQAAkAEAAEQC5ACUAKwAAAAUAJgAAAA8AagBOAGoAEAAAAAwAKwD9/wQACgAAAA8AsgO8ALIDEAAsABcAKgABAAQAGQAAAA8AUQA1AFEAHQAAABwAJwAAAAQAHQAAAA8AVwA7AFcAHgBXACcAIgD//wQAJQAAAA8AfABgAHwAKAAoAAcAJwACAAQACgAAAA8ALwATAC8AMAAoABQAJAD+/wQAEQAAAA8AYgBGAGIAOgAAABcAJAABAAMAGAAAAA8AZABIAGQARQAlABcAJAAAAAMAGAAAAA8AZgBKAGYAUgAAABEAJAD//wIADwAAAA8AaABMAGgAJQB6ABUAJAABAAMAFwAAAA8AawBPAGsANwCfABYAJAABAAMAFQAAAA8AbABQAGwAOwB6AAgAJAABAAMACgAAAA8AXABAAFwARAB6ABQAJAD//wQAEQAAAA8AJQAJACUARgBKAB8AIwAAAAQAHwAAAA8AJgAKACYAXQAlABwAIwAAAAQAGgAAAA8AMAAUADAAZAAAABcAIwAAAAQAGAAAAA8AMwAXADMANwDEABcAIwAAAAQAGAAAAA8AOAAcADgATgCfABcAIwAAAAQAGAAAAA8AQwAnAEMAWQBuABsAIwAAAAQAGwAAAA8ARwArAEcAZgBJABsAIwABAAQAHQAAAA8ATwAzAE8AegAkABwAIwAAAAQAHQAAAA8AUwA3AFMAfAAAABoAIwAAAAQAGQAAAA8AZwBLAGcATwDDABcAIwAAAAsAGAAAAA8AbQBRAG0AEQDlACMAGwABAAsAJQAAAA8AcABUAHAAZgCSABcAIwABAAsAGAAAAA8AcQBVAHEAdQBtABcAIwAAAAsAGAAAAA8AeQBdAHkAggBIABcAIwD+/wwAFAAAAA8AswO9ALMDlwAAABcAIwD//wwAFQAAAA8AvAOlA7wDlwAkABUAIwABAAwAGAAAAA8AIQAFACEAZwC2AAgAIgABAAQACwAAAA8AIwAHACMAZwDZABsAIgAAAAQAGgAAAA8AMQAVADEAcAC2AA8AIgABAAQAGAAAAA8AMgAWADIAfgCRABgAIgAAAAQAGAAAAA8ANAAYADQAjQBsABoAIgD//wQAGAAAAA8ANQAZADUAmgBIABcAIgABAAQAGAAAAA8ANgAaADYArQAkABcAIgABAAQAGAAAAA8ANwAbADcArwAAABgAIgAAAAQAGAAAAA8AOQAdADkAgAC0ABcAIgAAAAQAGAAAAA8APwAjAD8AlwCPABUAIgAAAAQAFAAAAA8AQQAlAEEAqABrAB4AIgD//wQAGwAAAA8AQgAmAEIAsgBHABgAIgABAAQAGgAAAA8ARAAoAEQAxQAjABoAIgABAAQAHAAAAA8ARQApAEUAyAAAABcAIgABAAQAGAAAAA8ARgAqAEYAgwDXABYAIgABAAQAFwAAAA8ASAAsAEgAmACyABsAIgABAAQAHgAAAA8ASQAtAEkArQCOAAgAIgACAAQACwAAAA8ASgAuAEoAmgDVABcAIgD//wQAFwAAAA8ASwAvAEsAsgDVABsAIgABAAQAGgAAAA8ATAAwAEwAtACxABYAIgABAAQAFwAAAA8ATQAxAE0AtgCOACIAIgABAAQAJQAAAA8ATgAyAE4AywCxABsAIgABAAQAHgAAAA8AUAA0AFAAxwBqABkAIgABAAQAGgAAAA8AUgA2AFIAywBGABoAIgABAAQAGgAAAA8AVAA4AFQA2QCNABsAIgD//wQAGQAAAA8AVQA5AFUA4QBpABoAIgABAAQAGwAAAA8AVgA6AFYAzgDUAB4AIgD//wQAGwAAAA8AWAA8AFgA5wCwABwAIgD//wQAGgAAAA8AWQA9AFkA9QCMAB0AIgD+/wQAGQAAAA8AWgA+AFoA7QDTABoAIgAAAAQAGQAAAA8AaQBNAGkABAGvAAgAIgABAAQACgAAAA8AdwBbAHcACAHSACIAGgD//wwAIAAAAA8AowO3AKMDDQGvABkAIgD//wQAGAAAAA8AOwAfADsA4AAAAAoAIAD//wwACQAAAA8AdABYAHQA4AAhABAAIAD+/wYADgAAAA8AFCCCARQgmgD4AB8ABwABABIAIQAAAA8AOgAeADoA6wAAAAkAGwABAAwACgAAAA8AYQBFAGEA5gBCABYAGwAAAAsAFwAAAA8AYwBHAGMA8QAcABcAGwAAAAsAFgAAAA8AZQBJAGUA9QAAABcAGwAAAAsAFgAAAA8AbgBSAG4A/ABeABUAGwABAAsAFwAAAA8AbwBTAG8A/QA4ABgAGwAAAAsAGAAAAA8AcgBWAHIACQEcAA8AGwABAAsADgAAAA8AcwBXAHMADQEAABYAGwAAAAsAFgAAAA8AdQBZAHUAEgFUABUAGwABAAwAFwAAAA8AfgBiAH4A/AB6ABsADAABABIAHQAAAA8AwwPJAMMDFgE4ABoAGwAAAAwAGAAAAA8AtQO/ALUDGQEcABcAGwAAAAsAFwAAAA8AKwAPACsAJAEAABgAGgAAAAkAGAAAAA8AdgBaAHYAEwGHABcAGgD//wwAFAAAAA8AeABcAHgAJwGiABcAGgD//wwAFQAAAA8AegBeAHoAKAFUABYAGgAAAAwAFQAAAA8APAAgADwAKAFvABUAFwD//wsAFQAAAA8APgAiAD4ATwDnABYAFwABAAsAFgAAAA8AXwBDAF8AugD4ABcABwD+/yIAEwAAAA8AKgAOACoANQDoABUAFQD//wQAEgAAAA8APQAhAD0ACAHtABUAEAABAA4AFwAAAA8AXgBCAF4AHgHtABMAEwD//wQAEgAAAA8AIgAGACIAGAFwAAwADgABAAMADQAAAA8AJwALACcAJwG9AAcADgAAAAMABwAAAA8ALAAQACwAAADpAAoADgD//x4ACAAAAA8ALQARAC0AGAF/AA4ABwD//xQADAAAAA8AYABEAGAA5gBeAA0ACgD//wMADQAAAA8ALgASAC4AMABNAAkACQABAB4ACwAAAA8AJyCQAScgEABXAAgACAAAABMACAAAAA8AIAAEACAARQBKAAAAAAD+/yIACgAAAA8A","kernings":"IABUAP//IgAiAP7/IgAnAP7/IgBBAP7/IgBhAP//IgBjAP//IgBkAP//IgBlAP//IgBnAP//IgBvAP//IgBxAP//IgBzAP7/IgDDA///JwAiAP7/JwAnAP7/JwBBAP7/JwBhAP//JwBjAP//JwBkAP//JwBlAP//JwBnAP//JwBvAP//JwBxAP//JwBzAP7/JwDDA///LAAiAP3/LAAnAP3/LgAiAP3/LgAnAP3/LwAvAPv/QQAiAP7/QQAnAP7/QQA/AP//QQBUAP3/QQBWAP7/QQBXAP//QQBZAP7/QQB2AP//QQB3AP//QQB5AP//QQCzA///QgBUAP//QgBZAP//QwApAP//QwBUAP//RAAsAP7/RAAuAP7/RABUAP//RABZAP//RQB2AP//RQB5AP//RQCzA///RgAsAPv/RgAuAPv/RgBBAP3/RgBKAPv/RgBhAP//RgByAP//SABUAP//SABZAP//SQBUAP//SQBZAP//SwAtAP//SwBDAP//SwBHAP//SwBPAP//SwBRAP//SwBjAP//SwBkAP//SwBlAP//SwBnAP//SwBvAP//SwBxAP//SwB2AP//SwB3AP//SwB5AP//SwAUIP//SwCzA///SwDDA///TAAiAPn/TAAnAPn/TABDAP//TABHAP//TABPAP//TABRAP//TABUAPr/TABVAP//TABWAPz/TABXAP3/TABZAPv/TAB1AP//TAB2AP3/TAB3AP7/TAB5AP3/TACzA/3/TQBUAP//TQBZAP//TgBUAP//TgBZAP//TwAsAP7/TwAuAP7/TwBUAP//TwBZAP//UAAsAPn/UAAuAPn/UABBAP3/UABKAPz/UABYAP//UABaAP//UQBUAP//UQBWAP//UQBZAP//UgBUAP7/UgBZAP//VAAgAP//VAAsAPz/VAAtAPv/VAAuAPz/VABBAP7/VABDAP//VABHAP//VABKAPv/VABPAP//VABRAP//VABhAP7/VABjAP7/VABkAP7/VABlAP7/VABnAP7/VABtAP7/VABuAP7/VABvAP7/VABwAP7/VABxAP7/VAByAP7/VABzAP7/VAB1AP7/VAB2AP//VAB3AP//VAB4AP7/VAB5AP//VAB6AP//VAAUIPv/VACzA///VADDA/7/VAC1A/3/VgAsAPv/VgAtAP//VgAuAPv/VgBBAP7/VgBhAP//VgBjAP//VgBkAP//VgBlAP//VgBnAP//VgBvAP//VgBxAP//VgByAP//VgB1AP//VgAUIP//VgDDA///VwAsAP3/VwAtAP//VwAuAP3/VwBBAP//VwBhAP//VwBjAP//VwBkAP//VwBlAP//VwBnAP//VwBvAP//VwBxAP//VwAUIP//VwDDA///WAAtAP//WABDAP//WABHAP//WABPAP//WABRAP//WABjAP//WABkAP//WABlAP//WABnAP//WABxAP//WAB2AP//WAB5AP//WAAUIP//WACzA///WADDA///WQAmAP//WQAqAP//WQAsAPz/WQAtAP//WQAuAPz/WQBBAP7/WQBDAP//WQBHAP//WQBKAP7/WQBPAP//WQBRAP//WQBVAP7/WQBhAP//WQBjAP//WQBkAP//WQBlAP//WQBnAP//WQBtAP//WQBuAP//WQBvAP//WQBwAP//WQBxAP//WQByAP//WQBzAP//WQB1AP//WQB6AP//WQAUIP//WQDDA///WQC1A///WgBDAP//WgBHAP//WgBPAP//WgBRAP//WgB2AP//WgB3AP//WgB5AP//WgCzA///YQAiAP//YQAnAP//YgAiAP//YgAnAP//aAAiAP7/aAAnAP7/bQAiAP7/bQAnAP7/bgAiAP7/bgAnAP7/bwAiAP3/bwAnAP3/cAAiAP//cAAnAP//cgAsAP3/cgAuAP3/cgBhAP//cgB0AAEAdgAsAP7/dgAuAP7/dwAsAP3/dwAuAP3/eQAsAP7/eQAuAP7/owNDAP//owNHAP//owNPAP//owNRAP//swMsAP7/swMuAP7/"},{"name":"cmmi12","common":{"lineHeight":38,"base":32,"scaleW":512,"scaleH":256,"pages":1,"packed":0,"alphaChnl":0,"redChnl":0,"greenChnl":0,"blueChnl":0},"info":{"face":"cmmi12","size":42,"bold":0,"italic":0,"charset":"xyztcbXYZTCB","unicode":1,"stretchH":100,"smooth":1,"aa":1,"padding":[2,2,2,2],"spacing":[0,0],"outline":0},"chars":"WACFAFgALwG9ACYAIQD//wMAIgAAAA8AYgBLAGIAPgFvABMAIgAAAAMAEQAAAA8AWQBWAFkAMgHfACIAIQD//wMAGAAAAA8AQwBnAEMAPwGSACEAIgAAAAIAHQAAAA8AWgAuAFoAVQHfACAAIQAAAAMAHAAAAA8AVAB4AFQAVgG1ACAAIQD//wMAGAAAAA8AQgA0AEIAdgHXACEAIQAAAAMAHwAAAA8AeQByAHkAMQEbABcAHwD//w0AFAAAAA8AdAAwAHQAPwE7ABEAHwD//wYADwAAAA8AeAA/AHgAPQEAABgAFwD//w0AFwAAAA8AegBGAHoASQEYABYAFwAAAA0AEwAAAA8AYwB/AGMAVgEAABQAFwAAAA0AEgAAAA8A","kernings":""},{"name":"cmsy10","common":{"lineHeight":70,"base":32,"scaleW":512,"scaleH":256,"pages":1,"packed":0,"alphaChnl":0,"redChnl":0,"greenChnl":0,"blueChnl":0},"info":{"face":"cmsy10","size":42,"bold":0,"italic":0,"charset":"-+/()ยฃpยกj","unicode":1,"stretchH":100,"smooth":1,"aa":1,"padding":[2,2,2,2],"spacing":[0,0],"outline":0},"chars":"cAAkAHAAUQEwACUALgABAB4AIwAAAA8AagAvAGoAawEAAAYALgADAAEADAAAAA8ALQBBAC0AcgEAACoAKgD//wIAKgAAAA8AKwBEACsAUgFfABsAKQD//wMAGgAAAA8AKABoACgAYQGJACkAGwAAAAoAKgAAAA8AKQBwACkAbgFfACkAGwAAAAoAKgAAAA8ALwAPAC8AdwGlACAAFwAAAA0AIQAAAA8AoQARAKEAVgHXAB4ABgABABUAIQAAAA8AowANAKMAdwG9ABgAGAAEAAsAIQAAAA8A","kernings":""},{"name":"cmr12","common":{"lineHeight":38,"base":32,"scaleW":512,"scaleH":256,"pages":1,"packed":0,"alphaChnl":0,"redChnl":0,"greenChnl":0,"blueChnl":0},"info":{"face":"cmr12","size":42,"bold":0,"italic":0,"charset":"=?;:,-.","unicode":1,"stretchH":100,"smooth":1,"aa":1,"padding":[2,2,2,2],"spacing":[0,0],"outline":0},"chars":"PwAyAD8AiwF7ABMAIgAAAAIAEwAAAA8APQApAD0AkAG9AB8ADQAAABEAIAAAAA8AOwAfADsAmAGeAAgAHgACAA4ACwAAAA8AOgBVADoAYAEYAAgAFgACAA4ACwAAAA8ALAAVACwAPwFbAAkAEAACABwACwAAAA8ALQBUAC0AgwD6AA8ABgD+/xYADgAAAA8ALgAeAC4AAAD4AAgACAACABwACwAAAA8A","kernings":""}],"pages":["font-atlas.png"]}
File: public/fonts/Roboto-Regular.json
{"pages":["font-atlas.png"],"chars":"KAAMACgAAAAAAA8ALwABAAAADgAAAA8AKQANACkAAAAwAA8ALwD//wAADwAAAA8AWwA/AFsAAABgAAwALQABAAAACwAAAA8AXQBBAF0AAACOAAwALQD+/wAACwAAAA8AewBfAHsAAAC8ABAALAD//wEADgAAAA8AfQBhAH0ADQBgABAALAD+/wEADgAAAA8AJAAIACQADQCNABcAKwAAAP//GAAAAA8AQAAkAEAAEQC5ACUAKwAAAAUAJgAAAA8AagBOAGoAEAAAAAwAKwD9/wQACgAAAA8AUQA1AFEAEAAsABwAJwAAAAQAHQAAAA8AVwA7AFcAHQAAACcAIgD//wQAJQAAAA8AfABgAHwAHgBUAAcAJwACAAQACgAAAA8AXABAAFwAJQB8ABQAJAD//wQAEQAAAA8ALwATAC8AJgBUABQAJAD+/wQAEQAAAA8AYgBGAGIALQAjABcAJAABAAMAGAAAAA8AZABIAGQANwChABcAJAAAAAMAGAAAAA8AZgBKAGYAOgB5ABEAJAD//wIADwAAAA8AaABMAGgAOwBIABUAJAABAAMAFwAAAA8AawBPAGsATABtABYAJAABAAMAFQAAAA8AbABQAGwANwDGAAgAJAABAAMACgAAAA8AJQAJACUAQADGAB8AIwAAAAQAHwAAAA8AJgAKACYATwCSABwAIwAAAAQAGgAAAA8AMAAUADAARQAAABcAIwAAAAQAGAAAAA8AMwAXADMARQAkABcAIwAAAAQAGAAAAA8AOAAcADgAUQBIABcAIwAAAAQAGAAAAA8AQwAnAEMAYwBsABsAIwAAAAQAGwAAAA8ARwArAEcAXQAAABsAIwABAAQAHQAAAA8ATwAzAE8AXQAkABwAIwAAAAQAHQAAAA8AUwA3AFMAaQBIABoAIwAAAAQAGQAAAA8AZwBLAGcAeQAAABcAIwAAAAsAGAAAAA8AbQBRAG0AEQDlACMAGwABAAsAJQAAAA8AcABUAHAAegAkABcAIwABAAsAGAAAAA8AcQBVAHEAkQAAABcAIwAAAAsAGAAAAA8AeQBdAHkAYAC2ABcAIwD+/wwAFAAAAA8AIQAFACEAbACQAAgAIgABAAQACwAAAA8AIwAHACMAdQCQABsAIgAAAAQAGgAAAA8AMQAVADEAfwBsAA8AIgABAAQAGAAAAA8AMgAWADIAhABIABgAIgAAAAQAGAAAAA8ANAAYADQAkgAkABoAIgD//wQAGAAAAA8ANQAZADUAqQAAABcAIgABAAQAGAAAAA8ANgAaADYAjwBrABcAIgABAAQAGAAAAA8ANwAbADcAnQBHABgAIgAAAAQAGAAAAA8AOQAdADkArQAjABcAIgAAAAQAGAAAAA8APwAjAD8AwQAAABUAIgAAAAQAFAAAAA8AQQAlAEEAYADaAB4AIgD//wQAGwAAAA8AQgAmAEIAeACzABgAIgABAAQAGgAAAA8ARAAoAEQAfwDWABoAIgABAAQAHAAAAA8ARQApAEUAkQCOABcAIgABAAQAGAAAAA8ARgAqAEYApwBqABYAIgABAAQAFwAAAA8ASAAsAEgAtgBGABsAIgABAAQAHgAAAA8ASQAtAEkAxQAjAAgAIgACAAQACwAAAA8ASgAuAEoAzgAjABcAIgD//wQAFwAAAA8ASwAvAEsA1wAAABsAIgABAAQAGgAAAA8ATAAwAEwAkQCxABYAIgABAAQAFwAAAA8ATQAxAE0AmgDUACIAIgABAAQAJQAAAA8ATgAyAE4AqACxABsAIgABAAQAHgAAAA8AUAA0AFAAqQCNABkAIgABAAQAGgAAAA8AUgA2AFIAvgBpABoAIgABAAQAGgAAAA8AVAA4AFQA0gBGABsAIgD//wQAGQAAAA8AVQA5AFUA5gAjABoAIgABAAQAGwAAAA8AVgA6AFYA8wAAAB4AIgD//wQAGwAAAA8AWAA8AFgAwwCMABwAIgD//wQAGgAAAA8AWQA9AFkA2QBpAB0AIgD+/wQAGQAAAA8AWgA+AFoA7gBGABoAIgAAAAQAGQAAAA8AaQBNAGkAAQEjAAgAIgABAAQACgAAAA8AdwBbAHcAvQDUACIAGgD//wwAIAAAAA8AOwAfADsAxACvAAoAIAD//wwACQAAAA8AdABYAHQAzwCvABAAIAD+/wYADgAAAA8AOgAeADoA4ACMAAkAGwABAAwACgAAAA8AYQBFAGEA4ACoABYAGwAAAAsAFwAAAA8AYwBHAGMA6gCMABcAGwAAAAsAFgAAAA8AZQBJAGUA9wBpABcAGwAAAAsAFgAAAA8AbgBSAG4ACQFGABUAGwABAAsAFwAAAA8AbwBTAG8ACgEjABgAGwAAAAsAGAAAAA8AcgBWAHIAEgEAAA8AGwABAAsADgAAAA8AcwBXAHMAIgEAABYAGwAAAAsAFgAAAA8AdQBZAHUA4ADEABUAGwABAAwAFwAAAA8AfgBiAH4AQADqABsADAABABIAHQAAAA8AKwAPACsA4ADgABgAGgAAAAkAGAAAAA8AdgBaAHYA9gDEABcAGgD//wwAFAAAAA8AeABcAHgA9wCoABcAGgD//wwAFQAAAA8AegBeAHoA+QDfABYAGgAAAAwAFQAAAA8APAAgADwADgHDABUAFwD//wsAFQAAAA8APgAiAD4AAgGFABYAFwABAAsAFgAAAA8AXwBDAF8AfwD5ABcABwD+/yIAEwAAAA8AKgAOACoADwFiABUAFQD//wQAEgAAAA8APQAhAD0AvQDvABUAEAABAA4AFwAAAA8AXgBCAF4AHwE/ABMAEwD//wQAEgAAAA8AIgAGACIA0wDvAAwADgABAAMADQAAAA8AJwALACcAHwFTAAcADgAAAAMABwAAAA8ALAAQACwANQDrAAoADgD//x4ACAAAAA8ALQARAC0AlwD5AA4ABwD//xQADAAAAA8AYABEAGAAEABUAA0ACgD//wMADQAAAA8ALgASAC4AQAD3AAkACQABAB4ACwAAAA8AIAAEACAAUQBsAAAAAAD+/yIACgAAAA8A","info":{"face":"Roboto-Regular","size":42,"bold":0,"italic":0,"charset":"! "#$%&'\()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[]^_`abcdefghijklmnopqrstuvwxyz{|}~","unicode":1,"stretchH":100,"smooth":1,"aa":1,"padding":[2,2,2,2],"spacing":[0,0],"outline":0},"common":{"lineHeight":44,"base":34,"scaleW":512,"scaleH":256,"pages":1,"packed":0,"alphaChnl":0,"redChnl":0,"greenChnl":0,"blueChnl":0},"distanceField":{"fieldType":"msdf","distanceRange":4},"kernings":"IABUAP//IgAiAP7/IgAnAP7/IgBBAP7/IgBhAP//IgBjAP//IgBkAP//IgBlAP//IgBnAP//IgBvAP//IgBxAP//IgBzAP7/JwAiAP7/JwAnAP7/JwBBAP7/JwBhAP//JwBjAP//JwBkAP//JwBlAP//JwBnAP//JwBvAP//JwBxAP//JwBzAP7/LAAiAP3/LAAnAP3/LgAiAP3/LgAnAP3/LwAvAPv/QQAiAP7/QQAnAP7/QQA/AP//QQBUAP3/QQBWAP7/QQBXAP//QQBZAP7/QQB2AP//QQB3AP//QQB5AP//QgBUAP//QgBZAP//QwApAP//QwBUAP//RAAsAP7/RAAuAP7/RABUAP//RABZAP//RQB2AP//RQB5AP//RgAsAPv/RgAuAPv/RgBBAP3/RgBKAPv/RgBhAP//RgByAP//SABUAP//SABZAP//SQBUAP//SQBZAP//SwAtAP//SwBDAP//SwBHAP//SwBPAP//SwBRAP//SwBjAP//SwBkAP//SwBlAP//SwBnAP//SwBvAP//SwBxAP//SwB2AP//SwB3AP//SwB5AP//TAAiAPn/TAAnAPn/TABDAP//TABHAP//TABPAP//TABRAP//TABUAPr/TABVAP//TABWAPz/TABXAP3/TABZAPv/TAB1AP//TAB2AP3/TAB3AP7/TAB5AP3/TQBUAP//TQBZAP//TgBUAP//TgBZAP//TwAsAP7/TwAuAP7/TwBUAP//TwBZAP//UAAsAPn/UAAuAPn/UABBAP3/UABKAPz/UABYAP//UABaAP//UQBUAP//UQBWAP//UQBZAP//UgBUAP7/UgBZAP//VAAgAP//VAAsAPz/VAAtAPv/VAAuAPz/VABBAP7/VABDAP//VABHAP//VABKAPv/VABPAP//VABRAP//VABhAP7/VABjAP7/VABkAP7/VABlAP7/VABnAP7/VABtAP7/VABuAP7/VABvAP7/VABwAP7/VABxAP7/VAByAP7/VABzAP7/VAB1AP7/VAB2AP//VAB3AP//VAB4AP7/VAB5AP//VAB6AP//VgAsAPv/VgAtAP//VgAuAPv/VgBBAP7/VgBhAP//VgBjAP//VgBkAP//VgBlAP//VgBnAP//VgBvAP//VgBxAP//VgByAP//VgB1AP//VwAsAP3/VwAtAP//VwAuAP3/VwBBAP//VwBhAP//VwBjAP//VwBkAP//VwBlAP//VwBnAP//VwBvAP//VwBxAP//WAAtAP//WABDAP//WABHAP//WABPAP//WABRAP//WABjAP//WABkAP//WABlAP//WABnAP//WABxAP//WAB2AP//WAB5AP//WQAmAP//WQAqAP//WQAsAPz/WQAtAP//WQAuAPz/WQBBAP7/WQBDAP//WQBHAP//WQBKAP7/WQBPAP//WQBRAP//WQBVAP7/WQBhAP//WQBjAP//WQBkAP//WQBlAP//WQBnAP//WQBtAP//WQBuAP//WQBvAP//WQBwAP//WQBxAP//WQByAP//WQBzAP//WQB1AP//WQB6AP//WgBDAP//WgBHAP//WgBPAP//WgBRAP//WgB2AP//WgB3AP//WgB5AP//YQAiAP//YQAnAP//YgAiAP//YgAnAP//aAAiAP7/aAAnAP7/bQAiAP7/bQAnAP7/bgAiAP7/bgAnAP7/bwAiAP3/bwAnAP3/cAAiAP//cAAnAP//cgAsAP3/cgAuAP3/cgBhAP//cgB0AAEAdgAsAP7/dgAuAP7/dwAsAP3/dwAuAP3/eQAsAP7/eQAuAP7/"}
File: src/codec/file_layout.txt
! infgen 3.2 output
!
gzip
!
last ! 1
dynamic ! 10
count 269 15 16 ! 1100 01110 01100
code 17 3 ! 011 000
code 18 4 ! 100
code 0 3 ! 011
code 7 3 ! 011 000
code 6 3 ! 011 000
code 5 2 ! 010 000
code 4 3 ! 011 000
code 3 5 ! 101 000
code 2 5 ! 101 000
zeros 10 ! 111 011
lens 5 ! 00
zeros 21 ! 0001010 0111
lens 3 ! 11111
zeros 6 ! 011 011
lens 7 ! 101
zeros 4 ! 001 011
lens 6 ! 001
lens 0 ! 010
lens 5 ! 00
zeros 16 ! 0000101 0111
lens 6 ! 001
zeros 4 ! 001 011
lens 7 ! 101
zeros 3 ! 000 011
lens 6 ! 001
lens 6 ! 001
zeros 9 ! 110 011
lens 7 ! 101
zeros 3 ! 000 011
lens 7 ! 101
zeros 9 ! 110 011
lens 5 ! 00
lens 7 ! 101
lens 5 ! 00
lens 5 ! 00
lens 4 ! 110
lens 7 ! 101
lens 6 ! 001
lens 6 ! 001
lens 5 ! 00
lens 0 ! 010
lens 7 ! 101
lens 5 ! 00
lens 5 ! 00
lens 5 ! 00
lens 4 ! 110
lens 7 ! 101
lens 0 ! 010
lens 5 ! 00
lens 5 ! 00
lens 4 ! 110
lens 7 ! 101
lens 0 ! 010
lens 5 ! 00
lens 7 ! 101
lens 6 ! 001
zeros 134 ! 1111011 0111
lens 7 ! 101
lens 4 ! 110
lens 6 ! 001
lens 6 ! 001
lens 0 ! 010
lens 0 ! 010
lens 6 ! 001
zeros 5 ! 010 011
lens 7 ! 101
zeros 7 ! 100 011
lens 2 ! 01111
lens 0 ! 010
lens 4 ! 110
lens 2 ! 01111
lens 0 ! 010
lens 2 ! 01111
lens 4 ! 110
lens 3 ! 11111
! litlen 10 5
! litlen 32 3
! litlen 39 7
! litlen 44 6
! litlen 46 5
! litlen 63 6
! litlen 68 7
! litlen 72 6
! litlen 73 6
! litlen 83 7
! litlen 87 7
! litlen 97 5
! litlen 98 7
! litlen 99 5
! litlen 100 5
! litlen 101 4
! litlen 102 7
! litlen 103 6
! litlen 104 6
! litlen 105 5
! litlen 107 7
! litlen 108 5
! litlen 109 5
! litlen 110 5
! litlen 111 4
! litlen 112 7
! litlen 114 5
! litlen 115 5
! litlen 116 4
! litlen 117 7
! litlen 119 5
! litlen 120 7
! litlen 121 6
! litlen 256 7
! litlen 257 4
! litlen 258 6
! litlen 259 6
! litlen 262 6
! litlen 268 7
! dist 7 2
! dist 9 4
! dist 10 2
! dist 12 2
! dist 13 4
! dist 14 3
literal 'H ! 010011
literal 'e ! 0100
literal 'l ! 01001
literal 'l ! 01001
literal 'o ! 1100
literal ' ! 000
literal 'W ! 1110111
literal 'o ! 1100
literal 'r ! 10101
literal 'l ! 01001
literal 'd ! 00001
literal ', ! 000011
literal ' ! 000
literal 'I ! 110011
literal '' ! 0010111
literal 'm ! 11001
literal ' ! 000
literal 'g ! 001011
literal 'o ! 1100
literal 'i ! 10001
literal 'n ! 00101
literal 'g ! 001011
literal ' ! 000
literal 't ! 0010
literal 'o ! 1100
literal ' ! 000
literal 'b ! 0001111
literal 'e ! 0100
literal ' ! 000
literal 'c ! 11110
literal 'o ! 1100
literal 'm ! 11001
literal 'p ! 1101111
literal 'r ! 10101
literal 'e ! 0100
literal 's ! 01101
literal 's ! 01101
literal 'e ! 0100
literal 'd ! 00001
literal ' ! 000
literal 'H ! 010011
match 4 40 ! 0111 10 111011
literal '. ! 10110
literal 10 ! 00110
literal 10 ! 00110
literal 'S ! 0110111
literal 'o ! 1100
literal 'm ! 11001
literal 'e ! 0100
literal ' ! 000
literal 'm ! 11001
literal 'o ! 1100
literal 'r ! 10101
literal 'e ! 0100
literal ' ! 000
literal 't ! 0010
literal 'e ! 0100
literal 'x ! 1011111
literal 't ! 0010
match 17 40 ! 0111 10 0 1111111
literal '. ! 10110
literal ' ! 000
literal 'H ! 010011
literal 'o ! 1100
literal 'w ! 11101
literal ' ! 000
literal 'd ! 00001
literal 'o ! 1100
literal 'e ! 0100
literal 's ! 01101
literal ' ! 000
literal 't ! 0010
literal 'h ! 101011
literal 'i ! 10001
literal 's ! 01101
literal ' ! 000
literal 'w ! 11101
literal 'o ! 1100
literal 'r ! 10101
literal 'k ! 0101111
literal '? ! 100011
literal ' ! 000
literal 'D ! 1010111
match 4 16 ! 11 00 111011
literal 'i ! 10001
literal 't ! 0010
match 5 14 ! 01 00 000111
match 3 45 ! 1100 10 1010
literal 'r ! 10101
literal 'r ! 10101
literal 'e ! 0100
literal 'c ! 11110
literal 't ! 0010
literal 'l ! 01001
literal 'y ! 011011
literal '? ! 100011
literal ' ! 000
literal 'I ! 110011
literal ' ! 000
literal 'w ! 11101
literal 'a ! 01110
literal 'n ! 00101
literal 't ! 0010
match 4 26 ! 001 0111 111011
match 3 72 ! 00111 01 1010
literal 'a ! 01110
literal 'c ! 11110
literal 't ! 0010
literal 'u ! 0011111
literal 'a ! 01110
literal 'l ! 01001
literal 'l ! 01001
literal 'y ! 011011
literal 10 ! 00110
match 8 78 ! 01101 01 100111
match 3 66 ! 00001 01 1010
literal 'e ! 0100
literal ' ! 000
literal 'd ! 00001
literal 'a ! 01110
literal 't ! 0010
literal 'a ! 01110
literal ', ! 000011
literal ' ! 000
literal 'a ! 01110
literal 'n ! 00101
literal 'd ! 00001
literal ' ! 000
literal 'u ! 0011111
literal 's ! 01101
match 3 14 ! 01 00 1010
literal 'y ! 011011
literal 'n ! 00101
literal 'a ! 01110
literal 'm ! 11001
literal 'i ! 10001
literal 'c ! 11110
match 3 129 ! 000000 011 1010
literal 'd ! 00001
literal 'e ! 0100
literal ' ! 000
literal 'f ! 1001111
literal 'o ! 1100
literal 'r ! 10101
match 5 35 ! 0010 10 000111
match 8 48 ! 1111 10 100111
literal 'i ! 10001
literal 'o ! 1100
literal 'n ! 00101
literal '. ! 10110
match 8 83 ! 10010 01 100111
match 3 80 ! 01111 01 1010
literal 's ! 01101
literal 'e ! 0100
literal 'e ! 0100
literal ' ! 000
literal 'h ! 101011
match 3 141 ! 001100 011 1010
literal 'w ! 11101
match 3 186 ! 111001 011 1010
match 8 125 ! 11100 1111 100111
literal 's ! 01101
literal '. ! 10110
literal 10 ! 00110
end ! 0111111
! 0000
!
crc
length
File: src/codec/original_file.txt
Hello World, I'm going to be compressed Hello.
Some more text to be compressed. How does this work? Does it work correctly? I want it to actually
compress the data, and use dynamic mode for the compression. I want to see how well it works.
File: src/cpu/README.md
CPU
An investigation of how a CPU functions, from microcontrollers to desktop computers. (<-- a bit ambitious)
Roadmap/Next Steps/TODO list
Interaction
- selection of wires, ports, comps
- mode for making comps transparent (selection of wires)
- join colinear wires, fix subtraction bug
UI
- pinning of info to wires & ports
- add ALU info
- switch to divs for register info
- highlight wire-to-wire nodes & comps; dim others
- movement-along-wire animation
- figure out segments (known), and offsets (need to add)
- some renderSeg(a, b, offset, t)
- then add t value for animation
- probably do a fade out, unless hover
- dim particular wires, like ctrl wires
- think about how to do wire colors given state. not sure if it's worth showing 0's for addresses/data, say.
- add custom rendering for wires, e.g. multi-bit ctrl lines/ports
System Features
- sub-modules
- more test cases
- probably just build this out myself
- but figure out some way to validate that my tests work as expected!
- test-running + test-evaluation
- need a way to define this such that we work both on some working emulator as well as on my system
- i.e. our tests should pass on both
- probably do some magic with macros
- bne a0 a1 -> fail
File: src/cpu/schematics/SchematicManifest.json
{
"comment0": "This file is used as a registry to build the file that pulls in all the schematics.",
"comment1": "Managed by /cpu/api/save-schematic-to-file",
"schematics": [
{
"id": "pcCounter"
},
{
"id": "regFileDemo"
},
{
"id": "riscvBasic"
},
{
"id": "romUsage"
},
{
"id": "insDecodeInternal"
},
{
"id": "riscvBasicAddIns"
},
{
"id": "riscvBasicInsDecodeOnly"
},
{
"id": "aluInternalSimple"
},
{
"id": "adder_RippleAdder"
},
{
"id": "singleBitAdder"
},
{
"id": "mux_2Internal"
},
{
"id": "riscvLoadStoreInternal"
},
{
"id": "shiftLeftInternal"
},
{
"id": "shiftRightInternal"
},
{
"id": "comparitor_4Way"
},
{
"id": "comparitorInternal"
},
{
"id": "addrMapInternal"
},
{
"id": "riscvBasicAlu"
}
]
}
File: src/llm/twitter_thread.txt
Project #2: LLM Visualization
So I created a web-page to visualize a small LLM, of the sort that's behind ChatGPT. Rendered in 3D, it shows all the steps to run a single token inference. (link in bio)
It also contains a walkthrough/guide of the steps, as well as a few interactive elements to play with. Why, you ask? For what purpose did I put all the time & effort into this project?
There's a real advantage to unpacking a set of abstractions, flattening them out. Abstractions can be useful for terseness and management, but they can be a real blocker to seeing the big picture.
With this, you can see the whole thing at once. You can see where the computation takes place, its complexity, and relative sizes of the tensors & weights.
The model with all the animations is tiiny, to make it tractable. For comparison, I threw in a few of the larger models (GPT-2, GPT-3), render-only. And when you see what it takes to just produce a single value in a mat-mul, the sheer scale of these things becomes apparent.
(Here's what goes into calculating a single output value of a matrix-multiply)
What about understanding what each layer does? Uhh, sorry, won't be much help. The project just came out of "Let's build a 3D viz!", so the scope is a bit limited. It's more: here's a way to learn & digest the algorithm, and perhaps think about how to optimize the process.
As for what I got out of creating this: before I made it, I mostly knew how image convolution nets worked, but language-based models seemed kinda magical in comparison. Well, now I know them in a fair amount of detail!
I also learnt a good amount of GL (dF/dx, fwidth, ubos, instancing), and animation approaches. So, uhh, even if no-one sees this, the project definitely has some value to me.
Oh yeah, the link is here: https://bbycroft.net/llm/ Works best on desktop (sorry mobile). Left-click drag, right-click rotate, scroll to zoom. And hover over the tensor cells. Blue cells are weights/parameters, green cells are intermediate values. Each cell is a single number!
Well, I hope you find it interesting. Let me know your thoughts! And if someone makes it through the walkthrough and finds it a little ~incomplete torwards the end I might even getting around to fix it (my attention has largely turned to other projects oops)
Because no-one clicks links, I'll create some threads of the walkthrough. First up is the idea of tokens & embedding.
That's certainly been quite the response! 200k site visits, 1M X views, #1 on HN for a day or so, and plenty of very positive feedback. Here's a quick doc I put together with my sister a couple months ago, "setting expectations" I guess.
A few acknowledgements: @karparthy's mingpt repo was vital to ensuring my GPT impl. was working correctly (plus his YT vids are great), @telmudic for that first QT that got me off the ground, and my sister for getting me to actually have a plan & set a release deadline.
Maybe I'll get around to animating those last few pages, or fixing mobile, but no promises. I'll write a thread or two on how I actually wrote the app, because that's interesting to me.
Oh yeah, I threw in a Stripe tip-jar on my website (to be clear, this is a hobby project & I have a full-time job). P.S. @stripe, @patio11, your onboarding process is smooth as silk.
Here's a technical guide on how I wrote & structured some of the LLM Visualization. A few people have been asking about it, so I thought I'd write it up. Lots of code screenshots, so not for everyone.
[QT the original thread]
Some quick notes: * took me about 200 hours * written in Typescript with next.js, react for anything DOM related * all the 3D stuff is written directly against WEBGL2 * GPT algo itself run in WASM, written in Odin (thx @gingerbill!) * here are my client-side deps:
Most of the logic is executed within an requestAnimationFrame (rAF) loop, within the top-level runProgram() function. The IProgramState has all the state hanging off of it, some cross-frame, and some generated per-frame. This JS logic all takes ~5ms.
Of note, the block generation for the small LLM is run for every frame, and the code looks like this. Very terse, but there are ~50 unique blocks. There's info for how to look-up on-GPU texture-maps for the cell data, and the block's dep structure (for hover & anim).
Here's some walkthrough code samples. First we have the commentary strings (template-strings handy!), interleaved with these t_
We have a per-chapter value t, and when we create each t_
Block rendering: each block is single 3D cube. Drawing the individual cell value circles is done in-shader, querying float32 texture-maps. So the circles & the white grids are all done per-pixel in the fragment shader.
But when we have the blocks hovered, and a row/column/cell is highlighted (+ other animation effects), we split the blocks into sub-blocks. That way they can take unique color/opacity as needed. With a bit of careful maths, the texture-map lookups remain consistent.
The actual eval & population of the green-block values is done in WASM, written in Odin. It runs at init in a few ms (not optimized, eg uses naive matmul). We then pass that data to the GPU in texture-maps. The intro animation of the entire process is all done after-the-fact.
Writing a 3D engine with text, lines, buffer/shader management etc was quite a lot of work, but the animated blocks certainly demanded it. Often navigating a framework is as much effort as Doing It Yourself.
I did a bunch of mini-projects in this, for the fun. * The text layout code for the hover tooltips (2D tho, could have been DOM?) * Constructing & rendering the ribbons with beziers. * A mimalloc-inspired memory allocator. * The ToC diagram highlighting.
So this was a really fun project, engineering-wise. Lots of experimenting with new techniques & approaches. Naturally this was 10x harder than I'd planned. I've got a new, unrelated one on the go, which is probably more ambitious, oops.
File: src/llm/walkthrough/walkthrough_notes.txt
-- Introduction --
Welcome to the walkthrough of the structure & operation of the GPT large language model (LLM) model.
This architecture is of interest as it's what powers OpenAI's GPT series of models: GPT-2 and GPT-3, including ChatGPT.
Here where focusing on inference. I.e. what happens when ChatGPT responds with some text. The other side of ML,
training is not covered here.
The model we'll explore here is aptly named nano-gpt, several orders of magnitude smaller than serious LLMs, but
a lot easier to digest.
It's goal, being so small, is a simple one: take a sequence of six letters, using 'A', 'B', or 'C' such as "CABACA", and
sort them in ascending order, i.e. "AAABCC". There are easier ways to be sure, but it's interesting enough.
We call each of these letters tokens, and is this model's vocabulary. I.e. it only has 3. For larger models, these tokens
could be all the letters of the alphabet, or all words in the English language.
Each token is assigned a number, and our 'A', 'B', and 'C' naturally map to 0, 1, & 2. Now they're ready to be passed into
the model.
Now that we have a set of numbers in a sequence, (2, 1, 0, 1, 1, 2), we can run the model.
Each of the token numbers are transformed into a vector containing 48 numbers, and are then ready to pass through the
steps of the model.
We go through a series of transformers, before reaching the bottom where we end up with ... more numbers.
So what's the output? A prediction of the next element in the sequence! So at the 6th entry, we get probabilities that
the answer is going to be 'A', 'B', or 'C'. Clearly it's 'A' since that's the smallest letter in our original sequence.
We then feed that answer into the 7th slot, rince & repeat.
-- Input --
"Each token, or word/character, is represented by a number." [sequentially highlight the first few tokens & their numbers]
"We could map characters to numbers, or words to numbers, or anything else." [show a char-map, show a gpt-token map; highlight the mappings]
"In our case, we only have 3 'words' in our vocabulary: '0', '1', and '2'." [highlight the 3 distinct words] (footnote about Andrej Karpathy's demo example)
"And these naturally map to the integers 0, 1, and 2." [highlight the 3 distinct integers]
"This tiny model here sorts those numbers into ascending order. Let's see how it works!" [visualize the sort; show the model]
"So we put our sequence into an array, ignoring the rest." [highlight the 6 integers]
"We convert our sequence of 6 tokens/integers to 48 element vectors. one at each 'time' step." [quick op phase; highlight the 48 element vectors]
"These vectors now pass through the stages of the model, going through a series of transformers." [highlight the transformers, quickly step through them]
-- Output --
"And what's the output? A weighted prediction of the next token in the sequence!" [highlight the 6th element]
"And once we've picked the next token, we can place that in the 7th position, and repeat." [slide the output viz to the top; plonk the 7th element at the top; repeat until the end]
"And now we have our prediction. A list of correctly sorted numbers." [highlight the source 6 numbers, and output 6 numbers]
"Clearly this is a very convoluted way to sort 6 numbers! But it's structure and function are identical to that of GPT-2. Well, besides that small matter of scale..."
-- Input detail --
"Now lets look at the model in more detail." [highlight the input]
"The model is composed of Weights, with this structure of inputs, transformers, and outputs, and here we're also showing all the intermediate results." [highlight the weights, transformers, and intermediate results]
"Note that only a small fraction of the intermediate results are actually required at any one time, but we show them all here for clarity." [highlight the intermediate results]
"Let's start at the top. To compute the vectors at each time T we do a couple of steps:" [highlight the input]
"1. Select the i'th column of the token embedding matrix." [highlight the i'th column of the embedding matrix for a couple of them]
"2. Select the t'th column of the position embedding matrix." [highlight the t'th column of the position embedding matrix for a few of them]
"And add them together. Now each vector has some information about the token, and some about it's position, and is ready to enter the transformers." [highlight the addition]
-- Transformer detail --
"Each transformer is made up of a "self-attention" layer, a "feed-forward" layer, connected to what we'll call the "residual backbone"." [highlight the self-attention layer and the feed-forward layer and backbone]
-- Layer normalization --
"Before each of these layers, we apply "layer normalization"." [highlight the layer normalization section]
"Working on each T separately: we compute its mean & variance" [highlight the mean & variance computation]
"We normalize the vector (subtract the mean, divide by the variance)..." [highlight the normalization]
"...and then scale and shift it by these weights." [highlight the scaling and shifting]
"This almost seems like we're undoing the normalization, but now we have learned mean & stddev, which helps with learning on deep networks."
-- Self-attention --
"Now we can apply the self-attention layer." [highlight the self-attention layer]
"Self-attention is the part of the model where parts of the sequence become visible to each other. Up til now, each T could be processed independently."
"Self-attention is made up of several heads. Let's take a look at one." [dim the other heads; expand vertically]
"From each of the input vectors, we compute a query, key, and value vector, each of length A." [highlight the query, key, and value]
"Each vector at each T is a matrix-vector multiply" [highlight the matrix-vector multiply]
"Now comes the core of self-attention. Let's take a look at one of the query vectors." [highlight the query vector]
"It's purpose is to find keys from other T's that it should pay attention to."
"We compute the dot product (multiply pair-wise; sum) of the query vector with each key vector and scale it by 1 / sqrt(A)." [run the dot product]
"The result gives a score for how closely they match, and we store it in the attention matrix." [run the dot product on remaining keys]
"One trick here is that we only consider the keys from the past. This is called 'causal attention'."
"Now we have a row of scores we want to turn it into a row of numbers that sum to 1. We do this by applying a softmax."
"Higher scores will get a larger fraction of that 1, and lower scores will get a smaller fraction."
"To perform this, we take exponential of each score, and then divide by the sum of all those values" [highlight the softmax]
"As a stability trick, we subtract the max value from each score before taking the exponential. It doesn't change the output though."
"Now with this row of scores, we can compute the weighted sum of the value vectors." [highlight the weighted sum]
"This gives us a new vector, which has pulled information from other T's that it's interested in and is now ready to be passed on."
"We run this process for each key, producing a full set of values for each T." [highlight the full set of values]
"And then run the same process for each of the other heads. The different heads will all be looking for different information, but each one can see the entire input." [run the other heads]
"Now that we've done this mixing between T's, we're back to processing each one independently."
"From each of the heads, we concatenate the values into a single vector." [highlight the concatenation]
"GPT models are usually structured so that this vector is the same length as the input vector." [highlight the input vector]
"Before returning to the residual backbone, we apply another matrix-vector multiplication. Typically called a projection." [do the matrix-vector multiply]
"Unlike other matrix-vector multiplies, this one does not have a bias." [note lack of bias]
"Another common feature of modern deep ML networks is the use of the residual connections."
"Instead of passing the output of the transformer directly to the next layer, we first add it to the input." [highlight the residual connection]
"This occurs at each stage of the transformer, and is called the residual backbone." [highlight the residual backbone]
-- Feed forward detail --
"The feed-forward layer is a simple matrix-vector multiply with a bias." [highlight the matrix-vector multiply]
"Again, we act on each T independently, and expand the vector to 4x it's original size." [highlight the expansion]
"Now we apply the activation function. One of the key parts of any neural network, it introduces non-linearity."
"A common choice is the ReLU function, which is just the max of 0 and the input." [highlight the ReLU]
"However, GPT uses a different activation function called GELU, a smooth approximation to the ReLU." [highlight the GELU]
"The activation function is applied to each element of the vector independently." [highlight the activation function]
"Now we apply another matrix-vector multiply, going from 4 * T back to T." [highlight the matrix-vector multiply]
"This result is added to the residual backbone, and now we're done with the transformer!" [highlight the residual backbone]
-- Transformer summary --
"So that's the transformer. Composed of the self-attention and feed-forward layers, connected via the residual backbone." [highlight the self-attention layer, feed-forward layer, and residual backbone]
"A model is made up of a stack of these transformers, with each one feeding into the next." [highlight the stack of transformers]
-- Output --
"Finally we get to the output. First we apply the same layer normalization as before." [highlight the layer normalization]
"Then we apply a matrix-vector multiply on each vector to produce a vector that's the vocab size. A mere 3 in our case!" [highlight the matrix-vector multiply]
"The value of each element gives an indication of how likely it is that the input is that token. Larger ones: more likely." [highlight the value of each element]
"We call these our set of logits."
"Like in self-attention, we apply a softmax to turn these into probabilities that sum to 1." [highlight the softmax]
"So we can look at the hightest probability, and that's our prediction for the next token!" [highlight the highest probability]
"We can take the top few options, or randomly pick one (weighted by the probabilities) to get some variation."
"We can adjust this a bit by adding a temperature parameter (multiply the logits prior to softmax). Higher temperatures make the lower probabilities more likely." [highlight the temperature parameter]
"We've computed values for all T values in the input, but we only care about predicting one token in the future." [highlight the T values in the input]
"So if we input 6 tokens, we take the logit for the 6th (not 7th!) token." [highlight the 6th token]
"Logits after are meaningless, and logits before are just predicting the input." [highlight the logits after and before]
"As in the intro, we're ready to select the token, and feed it back into the top to predict the one after" [highlight the token selection and feeding back into the top]