Clementine Documentation

Tangerine: Clementine Music Player

4,266 stars C++ C++ 8 files Β· ~38,251 tokens #c-plus-plus#linux#mac#music#qt#windows
RAW Doc

Docker & Self-Hosted Deployment Verified OCI

Deploy Clementine on any server, homelab, or cloud container platform.

1-Click Terminal Command
bash (docker run)
docker run -d --name clementine --restart always clementine-player/clementine:latest

Technical Documentation: clementine-player/Clementine

ℹ️ Provenance: Multi-Tier Ingestion Β· CodeWiki Reference

Overview (README.md)

Clementine

Clementine is a modern music player and library organizer for Windows, Linux and macOS.

Opening an issue

Ask for a new feature

Please:

  • Check if the new feature is not already implemented (Changelog)
  • Check if another person didn't already open an issue
  • If there is already an opened issue there is no need to comment "+1", it won't help. Instead, you can subscribe to the issue to be notified of anything new about it

Report a bug

Please:

  • Try the latest build (https://github.com/clementine-player/Clementine/releases) to see if any bug is still present. If it works fine even though you see an open issue, please comment on it and explain that the issue has been fixed.
  • Check if another person has already opened the same issue to avoid duplicates
  • If there already is an open issue you could comment on it to add detail about the problem or confirm it
  • In case there isn't, you can open a new issue with an explicit title and as much information as possible (OS, Clementine version, how to reproduce the problem...)
  • Please use https://pastebin.com/ for logs/debug.

If there are no answers, it doesn't mean we don't care about your feature request/bug. It just means we can't reproduce the bug or haven't had time to implement it :o)

Compiling from source

Get the code (if you haven't already):

text
git clone https://github.com/clementine-player/Clementine.git && cd Clementine

Compile and install:

text
cd bin
cmake ..
make -j$(nproc)
sudo make install

See the Wiki for more instructions and a list of dependencies:
https://github.com/clementine-player/Clementine/wiki#compiling-and-installing-clementine

Formatting your changes

CI checks that C++ code under src/ is formatted with clang-format 23.1.0.
Other versions format some code differently, so install that exact version
with uv (or pipx), which puts it on your PATH:

text
uv tool install clang-format==23.1.0

If uv warns that its tool directory isn't on your PATH, run
uv tool update-shell and open a new terminal.

Then reformat the files you've changed since origin/master:

text
dist/format.py -i

Claude Code does this automatically after each edit, using a hook in
.claude/settings.json.

Release notes

If your change is something users will notice, end its commit message with a
Release-note: trailer: one line, written for users (what's new, not how it
was done):

text
Show the song's lyrics in time with the song

Release-note: Lyrics scroll along with the song, when they're timed.

Git only reads trailers in a message's last paragraph, so keep the note there,
with any other trailers and no blank line between them. A note followed by a
blank line and then, say, Co-Authored-By: is ignored, and the commit never
makes an update. Together, as here, both are read:

text
Release-note: Lyrics scroll along with the song, when they're timed.
Co-Authored-By: Someone <[email protected]>

To check, git log -1 --format='%(trailers:key=Release-note,valueonly)'
should print the note.

Once a week, the macOS update feed gets the newest build if the commits since
its last update have notes, and shows them in the update window. Commits
without one (refactoring, tests, CI, docs) never make an update on their own.
See tools/mac-update/README.md.


Design Remote Streaming (`docs/design/remote-streaming.md`)

Design: streaming to remote devices through the network remote

Status: proposal, with a prototype (see Β§14)
Scope: src/networkremote, src/engines, src/core/player,
ext/libclementine-remote/remotecontrolmessages.proto

1. Summary

Today the network remote lets a device control Clementine: Clementine
decodes and plays the audio on the computer it runs on. A device can also
download files (DOWNLOAD_SONGS), which copies them rather than playing
them.

This design adds a third role: the remote device becomes the speaker.
Clementine keeps everything it does today (the library, playlists, queue,
shuffle and repeat, internet services, scrobbling, now-playing, MPRIS) but
sends the audio to a device on the network instead of its local sound card.

It covers two cases:

  • Streaming: the device plays the original file, which Clementine serves
    over HTTP. This covers most of a local library.
  • Restreaming: Clementine fetches the source through a GStreamer
    pipeline and serves the result, remuxed as-is when the device can play the
    codec and re-encoded when it can't. That
    covers internet services that need Clementine's credentials, CD audio,
    tracks cut from a cue sheet, formats the device can't decode, and audio
    that should carry Clementine's EQ and ReplayGain.

The main decision is that Clementine stays in charge of playback, and the
device is only a renderer
. From the player's point of view, a remote device
is another audio output, much like choosing a different ALSA or PulseAudio
sink. That is why every existing feature keeps working without changes.

2. Goals and non-goals

Goals

  • A user can pick "Play on: <device>" and the current and following tracks
    play there, with play/pause/seek/next/volume working from Clementine, from
    any remote client, and from MPRIS.
  • Everything Clementine can play can be played remotely. Local files go
    directly. Everything else is restreamed.
  • The official remote app can offer "Play on this phone", which registers the
    phone as a renderer.
  • A controller and a renderer can be different devices. A phone can send
    playback to a tablet.
  • Playback can move between outputs mid-track and keep its position.
  • Service credentials (Authorization headers, OAuth tokens) never leave
    Clementine.
  • The protocol stays backward compatible: old clients and old servers keep
    working.

Non-goals (for now)

  • Keeping several renderers in sync (multi-room). The abstraction shouldn't
    rule it out, but it isn't designed here.
  • Streaming outside the LAN. The existing only_non_public_ip policy applies.
  • Replacing the remote's authentication or adding TLS. Section 9 notes where
    this design makes that easier later.

3. How the existing remote works

  • NetworkRemote listens on TCP 5500 (configurable and advertised over
    Zeroconf as _clementine._tcp) and makes one RemoteClient per socket.
  • Messages are a 4-byte length followed by a cpb::remote::Message (proto2,
    version defaults to 21). IncomingDataParser turns requests into Qt
    signals to Player and PlaylistManager. OutgoingDataCreator sends state
    (CURRENT_METAINFO, ENGINE_STATE_CHANGED, UPDATE_TRACK_POSITION, ...)
    to every client.
  • Authentication is an optional numeric auth_code in RequestConnect.
  • Downloads use a separate connection flagged downloader. SongSender sends
    whole files as protobuf SONG_FILE_CHUNKs and can transcode lossless files
    first with Transcoder presets (NetworkRemote::kTranscoderSettingPostfix).
  • Player owns one std::unique_ptr<EngineBase> engine_ (a GstEngine).
    About twenty call sites use app_->player()->engine() directly (analyzer,
    equalizer, seek slider, background streams). Two of them cast it to
    GstEngine* (SettingsDialog and the GStreamer debug console).
  • Player::HandleLoadResult turns playlist URLs into a
    MediaPlaybackRequest: a media URL plus HTTP headers such as Authorization
    from Google Drive or OneDrive. GstEnginePipeline decodes it with
    uridecodebin, sends it through ReplayGain, EQ, balance and volume, then to
    the configured sink.

Two things follow from this:

  1. The engine interface (Engine::Base: Load, Play, Pause, Seek,
    StartPreloading, position_nanosec, state, and the TrackAboutToEnd,
    TrackEnded, StateChanged and Error signals) is already a good fit for
    a remote renderer.
  2. Media bytes should not go through the protobuf socket the way
    SONG_FILE_CHUNK does. Every media stack a renderer might use (ExoPlayer,
    AVPlayer, GStreamer, VLC, browsers, Chromecast, UPnP renderers) plays an
    HTTP URL with Range requests natively. HTTP also gives seeking and
    buffering for free.

4. Architecture

text
/* Detailed source-code truncated for AI context efficiency. */

4.1 `EngineRouter`: output selection without touching `Player`

EngineRouter is a new Engine::Base that takes GstEngine's place as
Player::engine_. It owns the local GstEngine and any number of
RemoteEngines, forwards every call to the active one, and re-emits only
the active engine's signals.

Why a router rather than swapping engine_ at runtime: the engine pointer is
used and connected to in many places (MainWindow, BackgroundStreams,
analyzer, equalizer, NetworkRemote). Swapping it would mean re-connecting
all of them. With a router, the pointer never changes, and those places need
no changes.

Rules:

  • SetOutput(OutputId) is the only way to switch. To hand playback over, it
    reads position_nanosec() and state() from the old engine and stops it.
    It then calls Load(current_req, Engine::Manual, ...) on the new engine,
    followed by Play(position) if the old engine was playing, and emits
    OutputChanged.
  • EQ, stereo balance, ReplayGain and volume settings go to every engine,
    so switching doesn't lose them. Each engine decides what it can apply.
  • Background streams (AddBackgroundStream) always use the local
    GstEngine. They are ambience sounds, not part of the playlist.
  • scope() comes from the active engine. A remote engine returns an empty
    scope unless it is using the Pipeline with Encode (Β§5.2), where it can feed buffers.
  • The two qobject_cast<GstEngine*> call sites change to
    router->local_engine().
  • The selected output is saved in QSettings (NetworkRemote/output), but
    Clementine always starts on local output. A renderer that isn't
    connected can't be restored.

4.2 `RemoteEngine`: one per connected renderer

RemoteEngine : Engine::Base represents a single renderer. It turns engine
calls into protobuf commands and renderer status reports back into engine
signals.

Engine call / signal Renderer protocol
Load(req, change, ...) StreamPlanner::Plan(req, song, caps) β†’ RENDER_LOAD
StartPreloading(req, ...) plan β†’ RENDER_PRELOAD (gapless where the renderer supports it)
Play(offset) / Seek(pos) RENDER_SEEK{offset} (if non-zero) then RENDER_PLAY / RENDER_SEEK{pos}
Pause / Unpause / Stop RENDER_PAUSE / RENDER_PLAY / RENDER_STOP
SetVolumeSW(percent) RENDER_SET_VOLUME
position_nanosec() last RENDERER_STATUS.position_ms, plus wall-clock time since it arrived while playing
StateChanged from RENDERER_STATUS.state
TrackAboutToEnd generated locally from the interpolated position (same logic as EmitAboutToEnd)
TrackEnded RENDERER_TRACK_ENDED (with the item id, so late reports are ignored)
Error / InvalidMediaRequested RENDERER_ERROR, by scope: an item error makes Player skip exactly as it does for local errors, a transient one reloads at the last position, a renderer error falls back to local output
ValidMediaRequested first RENDERER_STATUS in state Playing for that item id

Crossfade and fade-out need two decoders mixed in one place, which a renderer
doesn't have. RemoteEngine turns them off (is_autocrossfade_enabled() and
friends return false) unless it is using live-mix mode (Β§5.4).

Losing the renderer (socket closed, or keep-alive timeout reusing
OutgoingDataCreator's keep-alive) makes the router fall back to local
output, paused at the last reported position. Audio that suddenly starts
on the computer would be a bad surprise. OUTPUTS tells controllers
what happened.

4.3 `StreamPlanner`: choosing stream or restream

StreamPlanner is a pure function, so it is easy to unit-test. It decides how
to deliver each track:

text
Plan StreamPlanner::Plan(const MediaPlaybackRequest& req, const Song& song,
                         const RendererCaps& caps, const StreamSettings& s);

struct Plan {
  enum Mode { Direct, Pipeline };
  Mode mode;
  // Pipeline only: remux the source's own codec, or decode and encode.
  enum Output { Passthrough, Encode };
  Output output;
  TranscoderPreset preset;    // Encode only
  QString mime_type;          // what the renderer will receive
  qint64 length_nanosec;      // known length, used for seek bars and ?t=
};

There are only two modes, and two questions decide them.

1. Can the file be served byte-for-byte (Direct)? Only when all of these
hold:

  • it is a local file, with no cue sheet and no begin/end markers,
  • the renderer lists a format for its codec, and that format allows its
    sample rate (an empty list allows any),
  • DSP isn't being applied remotely (apply_dsp off, or EQ and ReplayGain
    both off),
  • it isn't lossless with s.transcode_lossless on (the setting downloads
    already use), and
  • its bitrate isn't above s.max_bitrate_kbps (if set).

2. Otherwise it goes through the Pipeline. Remux or encode? Passthrough
when the renderer plays the source's codec at its sample rate and it has an entry
in the remux table (Β§5.2) and no DSP, markers or bitrate cap is in force.
Anything else is encoded: CD audio (already PCM), cue tracks, codecs or
sample rates the renderer can't play, codecs with no remux entry, DSP,
lossless-to-lossy and bitrate caps. The encoder's output is limited by the
target format's own entry: its sample rates become a caps filter, from which
audioresample picks the rate nearest the source's, and its channel limit
likewise.

So internet services, Google Drive files and radio usually end up as
Pipeline + Passthrough: the same bytes, fetched by Clementine with
Clementine's credentials.

Handing a renderer the raw upstream URL of a header-less stream stays an
opt-in (s.allow_raw_urls, off by default), because it reveals the URL and
loses stream titles and control of the stream.

"Stream" in the request means Direct. "Restream" means Pipeline, in either
of its outputs.

5. Delivery modes

Both modes are served by MediaHttpServer, a small HTTP/1.1 handler that
shares the remote's existing port (5500 by default) with the protobuf
protocol. That means one port to open in a firewall, one Zeroconf record,
and the same listen addresses and only_non_public_ip check with no extra
code.

5.0 Sharing the port

NetworkRemote::AcceptConnection already runs the only_non_public_ip check.
After that, instead of making a RemoteClient straight away, it waits for the
first byte from the socket and peeks at it without consuming it
(QTcpSocket::peek):

  • The protobuf framing starts with a big-endian quint32 length
    (QDataStream's default byte order). RemoteClient::IncomingData already
    disconnects any client whose length is over 128 MiB (0x08000000), so a
    valid first byte is always 0x00–0x08.
  • Every HTTP/1.x method (GET, HEAD, ...) starts with an upper-case ASCII
    letter, 0x41–0x5A.

The two ranges don't overlap, so this isn't a guess based on low odds: a
connection whose first byte is a letter is one the protobuf protocol would
already reject. One byte decides it, and no buffering is needed. First byte
0x00–0x08 goes to RemoteClient as today. A–Z goes to
MediaHttpServer if streaming is enabled. Anything else, or HTTP while
streaming is off, closes the socket.

The protocols are client-speaks-first: neither the remote nor HTTP sends
anything before the client does, so waiting for the first byte changes
nothing for existing clients. A connection that sends nothing within 10 s is
closed, which also stops idle sockets from piling up before they are
classified.

The same byte range leaves room for TLS later: a TLS ClientHello starts with
0x16, which matches neither protocol, so it can be recognised the same way.

The HTTP side speaks HTTP/1.1 only. Appendix A explains why HTTP/2 isn't
supported and how an HTTP/2 client is handled.

The whole mechanism is a small ProtocolSniffer step in NetworkRemote.
RemoteClient and MediaHttpServer each receive a socket that belongs to
them and never see the other protocol. MediaHttpServer has no
QTcpServer of its own; it takes sockets through HandleConnection(QTcpSocket*).

URLs are opaque: /s/<session-token>/[?t=<ms>]. Media is always on
the renderer's own connection's host and port, so the URL is just a path, and
the renderer resolves it against the host and port it connected to. Through
NAT, Docker's port mapping or a router's port forward, the address
Clementine's socket sees isn't one the renderer can reach; the address the
renderer dialled is. A URL with a scheme and host is still fetched from exactly
there, which leaves room to point a renderer elsewhere, such as at a radio
stream it fetches itself. A renderer only gets a path if it declares
RENDERER_FEATURE_RELATIVE_URLS; the first released remotes don't resolve
paths, so they get http://<local address>:<remote port>/s/..., with the
address the server's end of their connection has.
item-id is a key into a table held by the RemoteEngine. The table maps it
to {Plan, MediaPlaybackRequest, Song}. Entries are made on RENDER_LOAD or
RENDER_PRELOAD, and at most the current and next items are kept. No file
path or upstream URL ever appears in a streaming URL.
Clients can't
enumerate the library through this server, and path traversal isn't possible.

5.1 Direct: streaming local files

  • QFile with Range/206 Partial Content, Content-Length, the
    Content-Type from the file type, and Accept-Ranges: bytes.
  • Seeking and buffering happen on the renderer. RENDER_SEEK just tells it
    where to go.
  • Gapless playback: RENDER_PRELOAD is sent kPreloadGapNanosec before the
    end, as GstEngine does today. The renderer queues it (ExoPlayer
    ConcatenatingMediaSource, AVQueuePlayer, GStreamer about-to-finish).
  • Reads happen in chunks driven by bytesWritten. The whole file is never
    loaded into memory, unlike SongSender::SendSingleSong.

5.2 Pipeline: restreaming everything else

Everything that isn't Direct goes through one kind of pipeline. A new
GstStreamPipeline : GstPipelineBase is built for each request:

text
uridecodebin(uri = req.MediaUrl, caps = raw | <passthrough caps>)
  β”œβ”€ Passthrough: <parser> ! <muxer> ! appsink
  └─ Encode:      audioconvert ! [rgvolume ! rglimiter] ! [equalizer-nbands ! audiopanorama]
                  ! audioresample ! capsfilter(rate/channels <= caps)
                  ! <preset encoder> ! <preset muxer> ! appsink

uridecodebin's caps property sets the format at which it stops decoding.
With the source's compressed caps added to it, uridecodebin hands over
parsed but undecoded audio, so Passthrough costs about as much CPU as a
proxy. There is one pipeline and one response path, with two tails.

Fetching the source. The pipeline sets up its source with the same
SourceSetupCallback logic as GstEnginePipeline (moved into a shared
helper). That code already sets extra-headers (for example the
Authorization header UrlHandler::LoadResult::auth_header_ provides) and
user-agent on the source element. The consequences:

  • Credentials stay in Clementine, with no separate HTTP relay to write.
  • Seeking in a remote file works: the HTTP source sends its own Range
    requests upstream when the pipeline seeks.
  • Radio stream titles (ICY and in-stream tags) come out of the pipeline as
    tags and go into MetaData, just as they do for local playback.
  • A short-lived signed URL (Subsonic's or Plex's UrlHandler) that returns
    401 or 403 mid-stream is resolved again through Player's UrlHandler,
    and the pipeline is rebuilt once at the current position.

Passthrough: the remux table. Passthrough has to produce a container a
renderer can play as a stream:

Source codec Parser β†’ output MIME type
MP3 mpegaudioparse β†’ raw MP3 audio/mpeg
AAC (MP4/M4A or ADTS) aacparse β†’ ADTS audio/aac
FLAC flacparse β†’ raw FLAC audio/flac
Vorbis, Opus vorbisparse/opusparse β†’ oggmux audio/ogg
anything else β€” Encode instead

Remuxing can drop the gapless information some containers carry (MP4's
iTunSMPB, the LAME/Xing header). That is acceptable here, because it only
happens for sources that couldn't be served Direct anyway. It is also why
local files the renderer can play are never remuxed (Β§5.1).

Encode. The encoder and muxer come from the existing
TranscoderPresets
, the ones already offered for downloads (Ogg
Vorbis/Opus, MP3, AAC/M4A, FLAC), so there are no new format lists to
maintain. The default is Opus in Ogg at 128 kbps when the renderer lists it,
otherwise MP3 at 192 kbps. The ReplayGain, EQ and balance elements are made
with the same helper GstEnginePipeline::InitAudioBin uses. That code moves
into a shared function so local and remote playback sound the same. Begin
and end markers (cue tracks) become a segment seek on the decoder, the same
way GstEnginePipeline handles them.

Common to both outputs:

  • The output has no known length, so seeking uses ?t=<ms>. The renderer
    gets RENDER_SEEK{position, url} with a new URL, and the server starts a
    new pipeline that seeks to t before going to PLAYING. The response is
    chunked, with no Content-Length and Accept-Ranges: none. Length comes
    from Song::length_nanosec in RENDER_LOAD, so the renderer's seek bar
    is still correct. Live radio simply has no length.
  • appsink runs with sync=false, so a file-backed pipeline can run faster
    than real time. The renderer can buffer ahead, but not as freely as with
    Direct (Β§5.1): the pipeline pauses when it is more than 60 s of audio ahead
    of what the socket has sent.
  • Resources are bounded: at most one active pipeline per renderer, plus one
    to preload the next track.
  • The Encode tail can optionally tee raw audio into BufferConsumer, so the
    analyzer and moodbar in Clementine's window keep moving while it plays
    remotely.

Why Direct is still a separate mode. A file served byte-for-byte has a
Content-Length and Range support, so the renderer seeks instantly without
asking Clementine and can buffer the whole track quickly. That matters on
phones, where a Wi-Fi drop or the laptop sleeping mid-track shouldn't stop
playback. The original file also keeps its gapless and duration metadata,
is bit-perfect, and is what DLNA renderers (Phase 3) handle best. It is also
the most common case, and it costs a QFile plus Range parsing.

5.4 Live mix (later phase)

This mode keeps crossfades, fade-outs and one continuous stream across
tracks. The engine's audio bin ends in encoder ! muxer ! appsink instead of
audiosink, and every renderer gets one unending HTTP stream (like Icecast).
It uses the same set_output_device path in GstEnginePipeline, plus an
output option for "remote mixed stream".

Costs: 2–5 s of latency between a control action and what the user hears,
position that has to be adjusted for the renderer's buffer, and no native
seeking on the renderer. It is worth having for "party mode", but it
shouldn't be the default. Direct and Pipeline are Phase 1 and 2.

6. Protocol changes

All changes add to the existing messages, so existing clients and servers are
unaffected. Message.version goes from 21 to 22. Clients check for
SERVER_FEATURE_RENDERING in ResponseClementineInfo.features before using
any of it, and the server only offers rendering to clients whose
RequestConnect includes renderer.

6.1 Conventions for new definitions

New definitions follow buf's STANDARD lint rules where they apply to a
proto2 file with no services, plus two house rules:

  • Enums, not bools. A bool can only ever mean two things. Anything that
    might grow a third answer is an enum, and a set of on/off capabilities is a
    repeated enum of features. A new capability is then a new enum value,
    not a new field that every client has to learn about.
  • The first value of every enum is <ENUM_NAME>_UNSPECIFIED = 0
    (ENUM_FIRST_VALUE_ZERO, ENUM_ZERO_VALUE_SUFFIX). In proto2 this
    matters even more than in proto3: an optional enum field that is unset,
    or that holds a value the reader's older copy of the schema doesn't
    know, reads back as the first declared value. With _UNSPECIFIED first,
    "not sent" and "sent something I don't understand" both land on a value
    that means "unknown", never on a real choice made by accident. Readers
    treat _UNSPECIFIED as "use the safe default".
  • Every value carries its enum's name as a prefix (ENUM_VALUE_PREFIX),
    in UPPER_SNAKE_CASE. Enum values share the package's scope in C++, and
    the file already has top-level names like UNKNOWN, Playing and
    Idle, so unprefixed new values would sooner or later collide.
  • No bare scalars in Message. Each new payload is its own message,
    even when it holds a single field today, so it can grow without a new
    Message field.
  • Units in field names (_ms, _hz, _kbps), lower_snake_case
    fields, PascalCase messages and enums, and a leading comment on every new
    message, enum and value (buf's COMMENTS category).
  • Field numbers are never reused. A removed field or value is reserved.

The one exception is MsgType. Its existing values have no prefix and
aren't renamed here, because renaming generated identifiers would break
every client's source even though the wire format wouldn't change. New
MsgType values follow the existing style so the enum stays consistent;
UNKNOWN = 0 already plays the part of its _UNSPECIFIED. The same goes for
the other existing enums (EngineState, RepeatMode, ...): they are left
alone, and new code doesn't reuse them where a new enum fits better (see
RendererState).

6.2 Definitions

text
/* Detailed source-code truncated for AI context efficiency. */

Every _UNSPECIFIED has a stated meaning, so a reader never has to guess:
the comment on the value says what to do (for example, an unspecified
LoadStartState stays paused, which is the safe choice). Where no comment
says otherwise, _UNSPECIFIED means the sender didn't say, and the message is
handled as if the field were absent.

6.3 Notes

  • Limits are per format. Real limits depend on the codec (a renderer may
    play FLAC only up to 96 kHz while playing AAC at any rate), which is also
    how browsers (MediaCapabilities.decodingInfo) and Android
    (MediaCodecInfo) describe them, so each AudioFormat carries its own.
    The limits stop at sample rate and channels because Clementine's Song
    records neither channel count nor bit depth, so bit depth couldn't be
    checked yet; a max_bits_per_sample field can be added later without
    breaking older clients.

  • Sample rates are a list of integers, not an enum or a maximum. A list
    records gaps (a device that plays 48 kHz but not 44.1 kHz) and gives the
    encoder rates to choose from. Rates are an open set of numbers, so an enum
    would need a schema change for each new one and would say nothing the
    number doesn't. One entry per format, rather than one per combination of
    format, rate and channel count, says the same thing without multiplying
    entries.

  • The renderer uses its existing control connection, so authentication,
    keep-alive and disconnect handling work as they do now. A device can be a
    controller and a renderer at once. The "Play on this phone" button just
    sends SET_OUTPUT{own renderer_id}.

  • item_id is on every command and report. It prevents a status arriving
    after a track change from being applied to the wrong track.

  • OUTPUTS is sent to every client when a renderer joins or leaves, so every
    controller's output picker stays up to date.

  • Existing UPDATE_TRACK_POSITION, ENGINE_STATE_CHANGED and
    CURRENT_METAINFO still go to all clients. They are driven by
    EngineRouter, so controllers show the remote renderer's position without
    any changes. RemoteEngine maps RendererState onto Engine::State for
    them (loading and buffering count as playing).

  • To check the new definitions, CI can run buf lint with STANDARD and
    COMMENTS on the proto file, with the existing definitions listed in
    ignore_only so only new ones are held to the rules. The file's location
    also fails PACKAGE_DIRECTORY_MATCH (package cpb.remote in
    ext/libclementine-remote/); moving it is out of scope, so that rule is
    excluded too.

7. Code layout

text
src/engines/enginerouter.{h,cpp}           EngineRouter (Engine::Base)
src/engines/gstpipelinehelpers.{h,cpp}     source setup (headers, user agent) and RG/EQ/balance elements, shared by GstEnginePipeline and GstStreamPipeline
src/networkremote/streaming/
    remoteengine.{h,cpp}                   RemoteEngine (Engine::Base)
    rendererregistry.{h,cpp}               renderer_id -> RemoteEngine; owns OUTPUTS
    streamplanner.{h,cpp}                  pure planning logic
    protocolsniffer.{h,cpp}                first-byte routing of accepted sockets (Β§5.0)
    mediahttpserver.{h,cpp}                HTTP request parsing, token check, dispatch
    directresponder.{h,cpp}                file + Range
    pipelineresponder.{h,cpp}              owns a GstStreamPipeline, appsink -> socket
    gststreampipeline.{h,cpp}              GstPipelineBase subclass
src/ui/outputpicker.{h,cpp}                toolbar button + menu: "This computer", renderers

Changes to existing files:

  • Player: construct an EngineRouter around GstEngine. Nothing else in
    Player changes.
  • NetworkRemote: own RendererRegistry and MediaHttpServer, and route
    each accepted socket by its first byte (Β§5.0) instead of always making a
    RemoteClient.
  • RemoteClient: store RendererCapabilities from CONNECT and issue the
    session token.
  • IncomingDataParser: handle the new message types and pass them to the
    registry.
  • OutgoingDataCreator::SendClementineInfo: add SERVER_FEATURE_RENDERING to
    features.
  • SettingsDialog, MainWindow: qobject_cast<GstEngine*> becomes
    EngineRouter::local_engine().
  • networkremotesettingspage.ui: add a "Streaming" group (Β§8).

8. User interface

  • Output picker: a button next to the volume slider, shown only when the
    remote is enabled, streaming is allowed, and at least one renderer is
    connected. Its menu lists "This computer" and every renderer, with a check
    mark on the active one. The tray menu and MPRIS stay as they are.
  • Settings β†’ Network Remote β†’ Streaming
    • Allow playing on remote devices (off by default. It exposes media the
      same way Allow downloads does, so it follows the same idea.)
    • When the device can't play a file, convert to: [transcoder preset]
      (reuses the preset list downloads already show)
    • Always convert lossless files (shares the setting with downloads)
    • Apply equalizer and ReplayGain on remote devices (forces the Pipeline to encode)
    • Maximum bitrate (none / 320 / 192 / 128 kbps)
  • The status bar shows "Playing on <device>" while a remote output is
    active.

9. Security

  • Off by default. It needs the remote to be enabled and the new
    setting to be on.
  • Same network policy. HTTP arrives on the remote's own sockets, so the
    listen addresses and IpIsPrivate check apply before it is even
    recognised as HTTP. With streaming off, HTTP connections are closed
    unanswered, so the port looks exactly as it does today. Every request's peer address must also
    match the peer address of the control connection that owns the token.
  • Capability URLs. The session token is 128 bits from
    QRandomGenerator::system(), issued only after RemoteClient is
    authenticated. It is revoked when that connection closes. Item ids only
    resolve within their own session, and only for the current and next
    items. Without a live authenticated control connection, the HTTP server
    serves nothing.
  • Credentials stay in Clementine. In Pipeline mode the renderer
    gets Clementine's URL, never the upstream URL or its Authorization
    header. Raw-URL handoff (allow_raw_urls) only applies to URLs without
    headers, and it is off by default.
  • No path exposure. Files are chosen by item_id, never by path. That
    sidesteps the whole class of traversal bugs files_root_folder handling
    has to guard against.
  • Resource limits. A cap on concurrent HTTP responses per renderer (4),
    on pipelines per renderer (2), on responses overall (32) and on requests
    still being sent (32), and on header size (8 KiB); requests over a cap get
    429 or 503. A client that stops reading for 30 s is disconnected.
    Renderer capabilities are bounded too: at most 32 formats of 32 sample
    rates each, with out-of-range values dropped, because the sample rates end
    up in every pipeline's caps.
  • Renderer ids. A renderer id can only be taken over by a connection
    from the same address, so another device can't evict a renderer and
    receive the SET_OUTPUT requests meant for it.
  • Known weakness, unchanged. The remote's auth code is a short number
    sent in plaintext. This design doesn't make that worse. Tokens are only
    handed out on authenticated connections. TLS could be added later on the
    same port, since its first byte is distinguishable too (Β§5.0).

10. Edge cases

Case Behaviour
Renderer disconnects mid-track Fall back to local, paused at the last position. Send OUTPUTS
Renderer reports a track-level error Error + InvalidMediaRequested. Player skips to the next track, as it does locally
The same track fails Direct because the codec was advertised but doesn't actually decode Renderer sends RENDERER_ERROR with RENDERER_ERROR_SCOPE_ITEM. RemoteEngine tries the same item once more through the Pipeline with Encode, before reporting an error
Stream metadata changes (radio) The Pipeline emits the tags into MetaData, so titles update as they do for local playback
Clementine quits while playing remotely RENDER_STOP, then the usual DISCONNECT{Server_Shutdown}
Two controllers choose different outputs Last write wins. Everyone gets OUTPUTS
Hand-off during Pipeline playback New RENDER_LOAD with start_ms. The server makes a fresh pipeline seeked to that point
Stop after current, repeat, shuffle, queue Unchanged. They are Player/PlaylistSequence logic above the engine
Scrobbling / play counts Unchanged. They use Player's position and state signals, which the router drives

11. Phasing

  1. Phase 1: streaming local files. Protocol v22, EngineRouter,
    RemoteEngine, RendererRegistry, MediaHttpServer with Direct only,
    StreamPlanner's Direct check only (tracks that would need the Pipeline
    are skipped with a clear error), output picker, settings. The Android remote gets "Play on this
    phone". This version is already useful for most local libraries.
  2. Phase 2: restreaming. Pipeline mode with both outputs: Passthrough
    (internet services, radio) and Encode (cue, CDDA, unsupported codecs,
    lossless and bitrate caps,
    optional DSP), ?t= seeking, sharing the audio-bin helper.
  3. Phase 3: more renderers and live mix. Because delivery is plain HTTP,
    UPnP/DLNA MediaRenderers (AVTransport.SetAVTransportURI) and Chromecast
    (Default Media Receiver LOAD) are just more RemoteEngine backends
    behind an IRendererTransport interface. They are discovered with SSDP or
    mDNS rather than registering themselves. Live-mix mode (Β§5.4) comes here
    too.

12. Testing

  • streamplanner_test.cpp: a table-driven test of the Direct check and the
    Passthrough/Encode choice in Β§4.3. It is
    pure logic and needs no GStreamer or network.
  • mediahttpserver_test.cpp: Range parsing (bytes=0-, bytes=100-199,
    suffix ranges, invalid ranges β†’ 416), token and peer checks, unknown item
    β†’ 404, a Direct round trip against a temporary file over 127.0.0.1.
  • enginerouter_test.cpp: uses tests/mock_engine.h for both backends and
    checks call forwarding, signal filtering, and hand-off order (stop old β†’
    load new with position β†’ play).
  • remoteengine_test.cpp: a fake renderer over a local socket speaking the
    new messages. Covers position interpolation, stale item_id reports being
    ignored, and disconnect falling back to local.
  • Extend networkremote_test.cpp to check that SERVER_FEATURE_RENDERING is
    only advertised when enabled.
  • protocolsniffer_test.cpp: first byte 0x00–0x08 goes to the remote,
    A–Z to HTTP only when streaming is on, anything else is closed; a
    first byte that arrives in a separate TCP segment from the rest; the
    10 s silence timeout.
  • Pipeline: GStreamer tests through GstStreamPipeline. Encode turns a short
    WAV fixture into Ogg/Opus. Passthrough turns MP3 and FLAC fixtures into
    their raw forms, and the output decodes to the same PCM as the input.
    Both check the caps and approximate duration, including a ?t= start.

13. Open questions

  1. Should Clementine's volume slider set the renderer's device volume, or a
    software gain inside the Pipeline? The proposal is the renderer's volume
    (RENDER_SET_VOLUME), because Direct mode can't apply gain.
  2. Should the choice of output be per playlist ("the kitchen playlist always
    plays in the kitchen")? It's out of scope, but EngineRouter::SetOutput
    should be callable from PlaylistManager later.
  3. Should a renderer be able to pull the next track itself (a playlist
    cursor on the renderer) to survive short losses of the control connection
    on mobile? This design keeps the renderer passive. That is simpler and
    keeps one source of truth, at the cost of a gap if the control connection
    is down exactly when a track ends. RENDER_PRELOAD reduces the risk.

14. Prototype

A prototype of Phases 1 and 2 is on this branch, with a Python command line
client in tools/remote-cli that can act as a renderer or a controller.

The Player always runs its GstEngine behind an EngineRouter, and new
remote connections are sniffed by their first byte. Everything else is off
until Allow playing on remote devices is turned on: nothing is advertised
to clients, renderer and output messages are ignored, and HTTP connections
are closed. A smoke test checks this.

Where it differs from the design above:

  • Threads. NetworkRemote, its RemoteClients and MediaHttpServer
    run on the network remote's own thread, while the Player and its
    engines run on the main thread. RendererRegistry and the
    RemoteEngines live on the main thread and talk to the network thread
    through queued signals carrying serialized messages and client ids.
    StreamItemTable (mutex-guarded) is how the HTTP side sees which items a
    token may fetch.
  • Registration. There are no REGISTER_RENDERER/UNREGISTER_RENDERER
    or OUTPUT_CHANGED messages: a client registers with
    RequestConnect.renderer, unregisters by disconnecting, and OUTPUTS is
    broadcast on every change.
  • HTTP. One request per connection (Connection: close) rather than
    keep-alive. HTTP/2 clients get 400.
  • Encoders. The Pipeline's encoders are a short fixed list in
    StreamPlanner (Opus in Ogg, MP3, Vorbis in Ogg), not the transcoder
    presets. Passthrough is only chosen when the song's file type is known,
    which internet services often don't set.
  • Not done yet. Applying EQ and ReplayGain on remote output, the output
    picker in Clementine's own window, the Pipeline's tee into the
    analyzer. Handing paused playback to the
    local engine plays for a moment before it pauses, because GstEngine can't
    load paused.

Tested end to end against a Clementine with an isolated configuration:
Direct streaming of MP3, FLAC and Ogg with Range seeks; Pipeline encoding
FLAC and Vorbis to MP3, with ?t= seeks; the remux descriptions for MP3,
FLAC, Vorbis, Opus and AAC (checked with gst-launch); pause and resume;
moving playback to the local output and back at the same position; falling
back to local, paused, when the renderer disconnects; gapless preloading;
and the ffplay renderer.

Appendix A: HTTP/2

MediaHttpServer speaks HTTP/1.1 with keep-alive. It doesn't support
HTTP/2, either over TLS (h2) or in cleartext (h2c).

Almost no renderer would use h2c. Most clients only negotiate HTTP/2
during a TLS handshake:

  • Browsers don't implement h2c at all.
  • Apple's AVPlayer/URLSession, Chromecast and GStreamer's souphttpsrc
    (libsoup) only use HTTP/2 over TLS.
  • On Android, ExoPlayer's default HTTP stack is HTTP/1.1. OkHttp can use h2c
    only when told in advance that the server supports it, with no fallback,
    so a client would have to opt in deliberately.
  • DLNA renderers (Phase 3) are HTTP/1.1.

HTTP/2's benefits don't apply to this traffic. Each renderer has at most
two long responses at once (the current and the preloaded item), each
several MB. Multiplexing and header compression gain nothing there, and
HTTP/2 over TCP has the same head-of-line blocking as HTTP/1.1. The one real
gain is cheaper seek cancellation: HTTP/2 can cancel a single request and
keep the connection, where HTTP/1.1 closes it. On a LAN that saves one TCP
handshake per seek, which is negligible.

The costs are real:

  • HTTP/2 has its own per-stream and per-connection flow control. It would
    sit on top of the back-pressure in Β§5.1 and Β§5.2, giving two sets of
    windows to tune against each other.
  • Qt Network has an HTTP/2 client but no server, and nothing in Clementine's
    current dependencies provides one. Supporting it would mean adding nghttp2
    or a new Qt module, plus framing, header compression and stream state, to
    replace a few hundred lines of HTTP/1.1.

How HTTP/2 clients are handled:

  • An h2c client that assumes support opens with the fixed preface
    PRI * HTTP/2.0. P is in the A–Z range, so Β§5.0 routes it to the
    HTTP handler, which closes any connection that starts with PRI. Such a
    client can't fall back to HTTP/1.1 on the same connection, so there is
    nothing useful to reply.
  • An Upgrade: h2c header on an HTTP/1.1 request is ignored, and the
    response is plain HTTP/1.1. RFC 9113 deprecated that upgrade path.

When to revisit. If TLS is added later (Β§5.0), HTTP/2 comes almost for
free through ALPN, as long as the TLS stack already provides it. That is the
point to reconsider, although the gain for this traffic would still be small.

Appendix B: a web client

This appendix isn't part of the plan. It records what a browser-based
remote would involve, so the choices above don't rule it out.

Transport

Browsers can't open raw TCP, so a web client needs a WebSocket. It fits the
shared port (Β§5.0) without a new port:

  • The HTTP handler reads the request line and headers with peek, without
    consuming them, and routes by path: /api/ws to a WebSocket, /s/... to
    media, / to the web app.
  • For /api/ws it hands the untouched socket to
    QWebSocketServer::handleConnection(QTcpSocket*), which performs the
    handshake. This adds a dependency on Qt6::WebSockets.
  • Each WebSocket message carries one existing cpb::remote::Message.
    WebSocket already delimits messages, so the 4-byte length prefix is
    dropped. The browser decodes it with a proto2-capable library such as
    protobuf-es.
  • RemoteClient gets a transport interface with two implementations, the
    existing framed TCP and WebSocket. IncomingDataParser,
    OutgoingDataCreator, authentication, keep-alive and the renderer
    messages don't change.

Serving the app

Clementine has to serve the web app itself, from Qt resources at /:

  • Mixed content. A page served over HTTPS (for example a hosted copy on
    GitHub Pages) isn't allowed to open a plain ws:// connection, so a hosted
    client would need TLS on Clementine first.
  • Same origin. Served from Clementine, the app shares an origin with the
    WebSocket and media URLs. No CORS is needed, and the Origin check below is
    a simple equality test.
  • Versions. The app always matches the server's protocol version.
  • Discovery. Browsers can't do mDNS. The user types host:5500, or scans
    a QR code shown on the remote settings page.

The browser as a renderer

An <audio> element can play both delivery modes: Direct with Range
requests, and the Pipeline's chunked responses. canPlayType(), or
MediaCapabilities.decodingInfo() for sample rates and channels, fills in
RendererCapabilities.formats. This gives "Play on this browser" with
nothing to install, and is the easiest way to test streaming end to end.
Browsers bring their own limits, though:

  • Autoplay. Browsers block audio that starts without a user gesture on
    the page. A RENDER_LOAD with LOAD_START_STATE_PLAYING sent from another
    controller fails until the user has interacted with the browser tab. The
    page must unlock playback on the first click (the "Play on this browser"
    button does that), and report RENDERER_ERROR_SCOPE_TRANSIENT with a clear
    message when a play attempt is refused.
  • No gapless playback. <audio> leaves a gap between tracks, so the
    renderer must not declare RENDERER_FEATURE_GAPLESS. Media Source
    Extensions or Web Audio could close the gap later, at a cost in
    complexity.
  • Background tabs and phones. Browsers throttle timers in background
    tabs (less so while audio plays), which can delay the 1 Hz
    RendererStatus. Mobile browsers may suspend a page entirely when it goes
    to the background, which drops the connection. RemoteEngine must
    tolerate late status reports (the position is interpolated, Β§4.2), and a
    dropped renderer falls back to local output as for any other renderer.
  • Formats vary by browser. Safari's support for Ogg and Opus in
    particular differs from Chrome's and Firefox's. Capabilities come from
    canPlayType() at runtime, never from a fixed list per browser.

Security

Browsers change the threat model. Any page open in a browser on the LAN can
open ws://192.168.1.10:5500: WebSockets aren't bound by the same-origin
policy, and the connection comes from a private address, so
only_non_public_ip lets it through. So:

  • Check Origin. Accept WebSocket upgrades only from Clementine's own
    origin.
  • Check Host. Accept only Host headers that name one of Clementine's own
    addresses or host names, to block DNS rebinding.
  • Rate-limit the auth code. A script could otherwise try every five-digit
    code in minutes. Limit failed attempts per address, with an increasing
    delay.
  • Keep tokens out of URLs. The auth code is exchanged for the session
    token from Β§9. It is sent in the first WebSocket message and kept in
    sessionStorage. Media URLs carry it by design (Β§5), so they must not be
    logged, and the page sets Referrer-Policy: no-referrer.
  • The same checks also protect the media endpoints, because a malicious page
    could otherwise probe them too.

HTTPS

The protocol works over plain HTTP. What's missing without HTTPS and a
valid certificate is a browser that treats the page like an app.

  • As a controller, a page served by Clementine over HTTP works: the
    WebSocket, playlists and controls are all fine. The cost is that pairing
    secrets and session tokens cross the LAN in cleartext.
  • As a renderer, the page is held back:
    • Browsers only let a page install as an app, or run a service worker,
      over HTTPS (or on localhost). Without them, playback depends on a
      browser tab staying open.
    • Other APIs, such as Wake Lock and parts of Web Crypto, are also limited
      to HTTPS. The rules differ by browser, so assume HTTP is missing things.
    • A self-signed certificate doesn't help. Every device gets a full-page
      warning, browsers remember the exception differently, and a page with an
      invalid certificate still can't install. A native app can pin a
      fingerprint from a QR code; a browser can't.
  • A page hosted anywhere else needs TLS on Clementine. An HTTPS page
    can't open a plain ws:// or http:// connection to a LAN address
    (mixed content), and Chrome now also asks the user's permission before a
    public site connects to a private address.

So the choice is between HTTP served by Clementine, with those limits, and a
valid certificate for a name that points at the Clementine machine. There
are three ways to get there.

Option 1: plain HTTP from Clementine

No setup, and the baseline everything else falls back to. The UX work is in
pairing and in being honest about the limits:

  • Pairing instead of the five-digit code. The settings page shows a QR
    code carrying Clementine's URL and a one-time pairing secret. The browser
    trades the secret for a long-lived device token, and the settings page
    lists paired devices with a Revoke button. This beats the auth code under
    either option, though over HTTP the token can be sniffed on the LAN.
  • Setting expectations. The page says plainly: "Keep this tab open. On
    phones, playback may stop when the screen locks." A suspended page drops
    its WebSocket and never hears about the next track, which makes open
    question 3 (a renderer that queues or pulls the next track itself) more
    pressing for web renderers.

Option 2: bring your own certificate

Costs the project nothing to run, and it's what power users will want.

  • Certificate and key settings. Clementine serves HTTPS on the same
    port: a TLS ClientHello starts with 0x16, which the protocol sniffer can
    recognise (Β§5.0). It watches the files and reloads them when they're
    renewed.
  • Tailscale is the lowest-effort way to a real certificate, with no
    domain of your own; see the steps below.
  • A reverse proxy (Caddy, nginx) in front of Clementine also works, with
    the adjustments listed under "What Clementine needs behind a proxy".
  • Your own domain with ACME needs DNS-01 validation through the DNS
    provider's API, because HTTP-01 can't reach a LAN machine. That's fine for
    the few who want it, and too much to build a UI around.

Option 3: a hosted service, like Plex's

Plex gives every server a valid certificate with no user effort:

  1. Plex's DNS answers names that encode an IP address:
    192-168-1-10.<server id>.plex.direct resolves to 192.168.1.10.
  2. Each server gets a wildcard certificate for *.<server id>.plex.direct.
  3. The hosted app (app.plex.tv, on HTTPS) connects to
    https://192-168-1-10.<server id>.plex.direct:32400. The certificate is
    valid and nothing is blocked as mixed content.
  4. Accounts tie it together: the app lists the servers you've signed in on.

A Clementine equivalent (say clementine.direct) would need:

  • A DNS server answering IP-encoded names, on a domain listed on the
    Public Suffix List. Without the listing, Let's Encrypt's limit of about 50
    certificates per registered domain per week caps the whole project.
  • A certificate broker. Clementine makes its key and certificate request
    locally, and the service completes the DNS-01 challenge for that
    install's subdomain. The service never sees the private key.
  • Proof of installation, so the broker only issues a subdomain's
    certificate to the install that owns it. That means registering installs,
    which is close to accounts.
  • Renewal while Clementine runs. Certificate lifetimes are shrinking
    (CA/Browser Forum rules bring the maximum down to 47 days by 2029), so an
    install left closed for a few weeks comes back with an expired
    certificate and must fall back to Option 1 cleanly.
  • Discovery. Without accounts, the pairing QR code can carry the
    clementine.direct URL. That works, but less smoothly than signing in.

The costs:

  • Running it indefinitely. If the service or domain lapses, every
    install's web remote breaks in a way users can't fix.
  • Privacy. The service learns every install's LAN address, public
    address and when it runs, so it needs a published policy.
  • Some networks block it. Routers and DNS filters with DNS-rebinding
    protection refuse public names that resolve to private addresses. Plex
    documents this as a common support problem; those users silently get
    Option 1.
  • Abuse. Automatically issued certificates on a public domain attract
    misuse, which could get the domain flagged.
  • Remote access is separate. Plex's relay for playback away from home
    carries user traffic and is a much bigger service. Tailscale or WireGuard
    cover that instead.

Recommendation

  1. Build Option 1 properly first: pairing tokens, a revocable device
    list, the checks under Security above, and honest messaging about mobile
    background playback.
  2. Add Option 2 next: certificate and key settings, TLS on the shared
    port, and the Tailscale recipe below. That serves the people who most
    want HTTPS at no ongoing cost.
  3. Keep Option 3 possible without committing to it: keep the
    certificate source abstract (files now, a broker later), make the web app
    work whether Clementine serves it or it's hosted with an allowlisted
    Origin, and make pairing carry a URL rather than assume host:port.
    Whether the project wants to run public infrastructure is a decision for
    the maintainers, not a technical one.

Tailscale, step by step

Both the Clementine computer and the phone or laptop running the browser
need to be on the same tailnet. Traffic between them is already encrypted
by WireGuard; the certificate is there so the browser treats the page as
secure. A bonus: it also works away from home.

  1. Install Tailscale on the Clementine computer and on each device that
    will open the web remote, and sign in to the same tailnet on all of
    them.

  2. In the Tailscale admin console, open DNS, make sure MagicDNS is
    on, and under HTTPS Certificates choose Enable HTTPS. The page
    warns that machine names are published in the public Certificate
    Transparency logs. Rename the Clementine machine first if its name
    reveals anything you'd rather not publish.

  3. On the Clementine computer, publish the remote's port over HTTPS:

    sh
    tailscale serve --bg --https=443 localhost:5500
    tailscale serve status    # shows https://<machine>.<tailnet>.ts.net

    On Linux, if it refuses for lack of permission, allow your user to
    manage Tailscale once with sudo tailscale set --operator=$USER.

  4. Open https://<machine>.<tailnet>.ts.net on the phone and pair as
    usual.

To stop: tailscale serve --https=443 localhost:5500 off.

The alternative to step 3 is to let Clementine terminate TLS itself
(Option 2): tailscale cert <machine>.<tailnet>.ts.net writes a Let's
Encrypt certificate and key to files that Clementine's settings point at.
Tailscale's docs note that you then have to renew them yourself before they
expire after 90 days, for example with a timer that re-runs the command,
and Clementine reloads them when they change. tailscale serve renews
automatically, so it's the better default.

What Clementine needs behind a proxy

tailscale serve, Caddy and nginx all put a proxy between the browser and
Clementine, which changes a few assumptions:

  • Every client appears to come from localhost. Clementine sees the
    proxy's address, which on a dual-stack socket is ::ffff:127.0.0.1
    (#7524 makes the private-network check accept it). The network check then
    says nothing about the real client, so access control rests on pairing
    and, for Tailscale, on the tailnet's access rules. The media URL peer
    check (Β§9) is weaker for the same reason, which is acceptable when the
    tokens are unguessable and only sent over TLS.
  • Media URLs must use the name the client used. The prototype builds
    them from the control socket's local address, which behind a proxy is
    127.0.0.1. They should come from the Host of the WebSocket request, or
    from a configured external URL.
  • Host and Origin checks must allow the public name, such as the
    ts.net name, as well as Clementine's own addresses.
  • WebSockets have to pass through the proxy. Caddy and nginx support
    this; confirm it for tailscale serve before relying on it.
  • Only HTTP goes through an HTTP proxy. The native Android remote keeps
    speaking the protobuf protocol on the port directly. Over Tailscale it can
    connect to the machine's tailnet address, which is already encrypted.

Existing messages that don't suit a browser

The web client can reuse most of the protocol as it is, but not all of it:

  • GET_LIBRARY sends the whole library as a SQLite database in chunks.
    A browser could read it only by loading SQLite compiled to WebAssembly,
    and a large library is a big download to a phone. A web client would want
    paged library queries (artists, albums, search) instead.
  • DOWNLOAD_SONGS sends files as SONG_FILE_CHUNK messages, which a
    browser would have to reassemble in memory. Downloads should instead be
    plain HTTP responses from the media endpoint, which the browser saves
    natively.
  • Album art arrives as bytes inside SongMetadata.art. That works, but
    an image URL on the same server would let the browser cache art.

Those are the pieces to design before a web client is practical. The
transport itself is the small part.


3rdparty Libprojectm Docs Building (`3rdparty/libprojectm/docs/building.rst`)

Building projectM from source

Since version 4.0, projectM uses CMake to configure and build the library on all platforms.

This guide contains both the quick-start build instructions and the in-depth CMake reference. For WebAssembly
builds, see :doc:emscripten.

Quick Start (Debian / Ubuntu)

For other operating systems (Windows/macOS), see the OS-specific sections below.

Install the build tools and dependencies

text
Mandatory packages:

.. code:: bash

   sudo apt install build-essential cmake libgl1-mesa-dev mesa-common-dev

**Important:** projectM requires CMake 3.21 or newer. Current stable distributions already ship a new enough
CMake (e.g. Debian 12 "bookworm" with CMake 3.25, Ubuntu 22.04 and newer with CMake 3.22+). Only end-of-life releases
like Debian 11 "bullseye" (CMake 3.18) need a manual install, in which case get the latest CMake
release `from Kitware's download page <https://cmake.org/download/>`__.

Optional packages:

.. code:: bash

   sudo apt install ninja-build # To build projectM with Ninja instead of make
   sudo apt install libsdl2-dev # To build the development test UI

Download the projectM sources
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

For production use, it is highly recommended to use the latest stable version of projectM. Download the latest release
from the `Releases page on GitHub <https://github.com/projectM-visualizer/projectm/releases>`__ and unpack it.

Using a development version
^^^^^^^^^^^^^^^^^^^^^^^^^^^

If you prefer a bleeding-edge version or want to modify the code, clone the Git repository and initialize any
submodules:

.. code:: bash

   sudo apt install git # Probably already installed
   git clone https://github.com/projectM-visualizer/projectm.git /path/to/local/repo
   cd /path/to/local/repo
   git fetch --all --tags
   git submodule init
   git submodule update

Build and install projectM
~~~~~~~~~~~~~~~~~~~~~~~~~~

Replace ``/usr/local`` with your preferred installation prefix.

Configure the project using CMake
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. code:: bash

   sudo apt install cmake
   mkdir build
   cd build
   cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local ..

To generate Ninja scripts instead of Makefiles, add ``-GNinja`` to the above command.

Build and install
^^^^^^^^^^^^^^^^^

These commands will build projectM and install it to /usr/local or the configured installation prefix set in the step
before:

.. code:: bash

   cmake --build . -- -j && sudo cmake --build . --target install

**Note**: You won't need to use ``sudo`` if the install prefix is writeable by your non-privileged user.

Test projectM
^^^^^^^^^^^^^

The SDL-based developer test UI is off by default. To build it, add ``-DENABLE_SDL_UI=ON`` to the configure command
above (requires SDL2, e.g. ``sudo apt install libsdl2-dev``). It is not installed; after building, run it from the build
directory, e.g. ``./src/sdl-test-ui/projectM-Test-UI``.

Dependencies
------------

Depending on the OS/distribution and packaging system, libraries might be split into separate packages with binaries and
development files. To build projectM, both binaries and development files need to be installed.

General build dependencies for all platforms:
  • A working build toolchain (compiler, linker).
  • CMake 3.21 or higher.
  • OpenGL: 3D graphics library. Used to render the visualizations.
  • GLES3: OpenGL libraries for embedded systems, version 3. Required to build projectM on mobile devices, Raspberry
    Pi and Emscripten.
  • glm <https://github.com/g-truc/glm>__: OpenGL Mathematics library. It is optional, projectM will use a bundled
    version if not installed.
  • SDL2 <https://github.com/libsdl-org/SDL>__: Simple Directmedia Layer. Version 2.0.5 or higher is required to build
    the development test application in the build directory.
  • vcpkg <https://github.com/microsoft/vcpkg>__: C++ Dependency Manager. Optional, but recommended to install
    library dependencies and/or using CMake to configure the build. Mainly used on Windows, but also works for Linux and
    macOS.

Building on Linux and macOS

Installing dependencies

text
- Linux distributions will have packages available for most (if not all) required libraries. The package names and
  commands to install them vary widely between distributions (and even versions of the same distribution). Please refer
  to the documentation of your build OS on how to find and install the required libraries.
- On *BSD*, install the appropriate Ports with ``pkg install``.
- On macOS, a working Xcode installation is required. CMake (and libSDL2 for the test UI) can be installed
  using `Homebrew <https://brew.sh/>`__.

Building
~~~~~~~~

The steps documented below are a bare minimum quickstart guide on how to build and install the project. If you want to
configure the build to your needs, require more in-depth information about the build process and available tweaks, or on
how to use libprojectM in your own CMake-based projects, see the `Building with CMake`_ reference section below.

Using CMake is the only supported way of building projectM since version 4.0. CMake is a platform-independent tool that
is able to generate files for multiple build systems and toolsets while using only a single set of build instructions.

Building the project with CMake requires two steps:

- Configure the build and generate project files.
- Build and install the project using the selected build tools.

**Note:** The build directory should always be different from the source directory. Generating the build files directly
inside the source tree is possible, but strongly discouraged. Using a subdirectory, e.g. ``cmake-build`` inside the source
directory is fine though.

This documentation only covers project-specific information. CMake is way too versatile and feature-rich to cover any
possible platform- and toolset-specific configuration details here. If you are not experienced in using CMake, please
first read the `official CMake documentation <https://cmake.org/cmake/help/latest/>`__ (at least
the `User Interaction Guide <https://cmake.org/cmake/help/latest/guide/user-interaction/index.html>`__) for basic usage
instructions.

Configure the build
^^^^^^^^^^^^^^^^^^^

Configuring a non-debug build with default options and install prefix (``/usr/local``) can be done with these commands,
building in a subdirectory inside the source directory:

.. code:: shell

   cd /path/to/source
   mkdir cmake-build
   cd cmake-build
   cmake -DCMAKE_BUILD_TYPE=Release ..

CMake will check all required dependencies and display any errors. If configuration was successful, a summary of the
build configuration is printed and CMake should display a ``Generating done`` line. The project is now ready to build.

Compile and install the project
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Depending on your generator choice, you can use your selected toolset as usual to build and install projectM:

- With ``Unix Makefiles``, run ``make && sudo make install``.
- With ``Ninja``, run ``ninja && sudo ninja install``.
- With ``Xcode``, select the appropriate target and configuration in Xcode and build it, or ``INSTALL`` to install the
  project.

You can also use CMake's build mode to run the selected toolset and build any specified target. CMake knows which
command to call and which parameters to pass, so the syntax works on all platforms with all generators. If you've
already set the top-level build directory as working directory, simply pass ``.`` as ``/path/to/build/dir``:

.. code:: shell

   cmake --build /path/to/build/dir --config Release
   sudo cmake --build /path/to/build/dir --config Release --target install

If you don't need root permissions to install to the given path, running the second command without ``sudo`` is
sufficient.

To control how many files are compiled in parallel, pass the ``--parallel N`` argument to the build command, with ``N``
being the number of concurrent compile commands. If not given explicitly, the build processor (make, ninja, MSBuild,
xcodebuild etc.) will decide the limit, which may range from one file at a time to all files at once.

If you want to provide arguments directly to the toolset command, add ``--`` at the end of the CMake command line followed
by any additional arguments. CMake will pass these *unchanged and unchecked* to the subcommand:

.. code:: shell

   cmake --build /path/to/build/dir --config Release -- <additional build tool args>

Building on Windows
-------------------

To build the projectM library and the SDL-based standalone application, CMake must be used to create the project files
first. Using vcpkg to pull in the build dependencies is highly recommended, as CMake can't use NuGet (NuGet pulls in
dependencies using the project files, while CMake requires the libraries before creating the project files).

Installing the dependencies with vcpkg

As stated above, using vcpkg is the easiest way to get the required dependencies. First,
install vcpkg from GitHub <https://github.com/microsoft/vcpkg>__ by following the official guide.

We've included a vcpkg manifest file in the repository root, designed to automatically install dependencies when you
create your solution.

Creating the Visual Studio solution

text
CMake provides separate generators for different Visual Studio versions. Newer CMake versions will support recent Visual
Studio releases, but may remove generators for older ones. To get a list of available generators from the command line,
use the ``-G`` switch without an argument. The CMake GUI will present you a dropdown list you can easily select from.

To set the build architecture in Visual Studio builds, use the ``-A`` switch and specify either ``Win32`` or ``X64`` as the
argument. If you want to build for both architectures, create separate build directories and configure them accordingly.

To make CMake aware of the installed vcpkg packages, simply use the provided toolchain file when configuring the
projectM build by
pointing `CMAKE_TOOLCHAIN_FILE <https://cmake.org/cmake/help/latest/variable/CMAKE_TOOLCHAIN_FILE.html>`__ to it.

Here is a full command line example to create a Visual Studio 2022 solution for X64:

.. code:: console

   cmake -G "Visual Studio 17 2022" -A "X64" -DCMAKE_TOOLCHAIN_FILE="<path to vcpkg>/scripts/buildsystems/vcpkg.cmake" -S "<path to source dir>" -B "<path to build dir>"

If you use the CMake GUI, check the "Specify toolchain file for cross-compiling" option in the first page of the
configuration assistant, then select the above ``vcpkg.cmake`` file on the second page.

Another option is to open the project folder in a recent Visual Studio version as a CMake project and configure CMake
using Visual Studio's JSON-based settings file.

Building the solution
~~~~~~~~~~~~~~~~~~~~~

To build the project, open the generated solution in Visual Studio and build it like any other solution. Each time the
CMake files are changed, Visual Studio will automatically regenerate the CMake build files and reload the solution
before continuing the build. Be aware that in old Visual Studio versions (2015 and earlier) the reload-and-continue
might not work properly.

You can also build the solution with msbuild via the command line, or use CMake's build wrapper to do that for you:

.. code:: console

   cmake --build "" --config Release

Using Ninja to build
~~~~~~~~~~~~~~~~~~~~

The Ninja build system is shipped with Visual Studio since version 2019 and used by default if loading a CMake project
directly from within the IDE. Ninja can also be `installed separately <https://github.com/ninja-build/ninja/releases>`__.

To configure the build directory for Ninja, pass ``Ninja`` or ``Ninja Multi-Config`` as the argument for the ``-G`` switch.
The difference between both generators is that the former uses ``CMAKE_BUILD_TYPE`` to specify the configuration (
e.g. ``Debug`` or ``Release``) while the latter supports all configurations in a single build directory, specified during
build time.

The architecture is determined from the toolset, so make sure to run the commands in the correct Visual Studio command
prompt, e.g. "Native Tools for X64".

Configure and build for a single-configuration Release build with vcpkg:

.. code:: console

   cmake -G "Ninja" -DCMAKE_BUILD_TYPE=Release -DCMAKE_TOOLCHAIN_FILE="/scripts/buildsystems/vcpkg.cmake" -S "" -B ""
   cmake --build ""

Same, but using the multi-configuration generator:

.. code:: console

   cmake -G "Ninja Multi-Config" -DCMAKE_TOOLCHAIN_FILE="/scripts/buildsystems/vcpkg.cmake" -S "" -B ""
   cmake --build "" --config Release

Notes on other platforms and features
-------------------------------------

Supported platforms
~~~~~~~~~~~~~~~~~~~

libprojectM is designed to be compatible with as many platforms and architectures as possible. Thus, the code base
doesn't use any non-portable code like OS-specific APIs or assembly instructions.

The code compiles for the following target architectures:

- i686 (x32 / x86)
- x86_64 (x64)
- armv7 (arm)
- aarch64 (arm64 / armv8)
- WASM

Other architectures such as PPC, RISC and MIPS should be supported as well. Big-endian support is untested and thus
might be buggy.

libprojectM builds at least on the following platforms:

- Windows
- Linux
- macOS (iOS/tvOS as well, but be aware that Apple's TOS may prevent its use in Store apps!)
- BSD derivatives
- Android
- WebGL/WASM (using emscripten)

As long as a platform provides OpenGL Core 3.3 or OpenGL ES 3.2 libraries and supports at least C++ 14, libprojectM
should build fine.

Raspberry Pi (and other embedded systems)

Make sure to enable GLES support by passing -DENABLE_GLES=TRUE to CMake's configuration command and install the
appropriate OS libraries. Otherwise, follow the Linux build instructions above.

For applications using projectM, it's highly recommended to keep certain performance-related settings low, such as:

  • The per-point mesh resolution should be kept low, e.g. 48x32
  • The rendering resolution should not exceed 720p on the Pi, though most presets will also run at ~60 FPS in 1080p

Build using NDK for Android

text
To build projectM using the Android SDK, please refer to the official NDK docs:

https://developer.android.com/ndk/guides/cmake

It is highly recommended using the latest NDK and CMake >= 3.21 for building.

Using libprojectM with pkgconfig

Some UNIX build systems cannot use CMake config packages, like GNU autotools. To use libprojectM with such build
systems, projectM's build system also creates basic .pc files during the installation process.

Note that the resulting pkgconfig files will not necessarily work in all circumstances, because they are much less
flexible than CMake (or Meson). When using pkgconfig, some required libraries will probably not be linked
automatically (e.g. OpenGL libraries) and have to be added manually depending on the application needs.

Frontends and audio capturing

text
Previous projectM versions (before 4.0) shipped several UIs and audio capture implementations in this repository.
To reduce the size of the code base and separate the release cycles of the core library and the frontends, these were
moved into their own repositories under the `projectM-visualizer organization <https://github.com/projectM-visualizer>`__
(e.g. the SDL-based desktop app and the Qt-based desktop app with PipeWire/PulseAudio/JACK backends).

libprojectM on its own does not have any means of capturing audio, it simply takes PCM data via the API. Applications
using projectM can supply this data from their own audio sources, e.g. music playing in an audio player or capturing
audio data from OS or other external sources.

.. _Building with CMake:

Building with CMake
-------------------

This section contains in-depth information for building with the CMake build system.

To use the library in other projects, it is required to install it. Use ``CMAKE_INSTALL_PREFIX`` to specify the
installation directory. If you use other build systems, you have to specify the include and library paths manually.

Selecting a specific project file generator

Building libprojectM does not require any specific CMake generator. It should work with any available generator, single-
and multi-config.

To specify a CMake generator, use the -G switch, followed by the generator name. Some newer generators take an
additional architecture using the -A switch. To list all available generators available on your current platform,
leave out the generator name:

.. code:: shell

cmake -G

Additional information on the supported generators can be
found in the CMake documentation <https://cmake.org/cmake/help/latest/manual/cmake-generators.7.html>__.

Popular generators
^^^^^^^^^^^^^^^^^^

By default, CMake will use the Unix Makefiles <https://cmake.org/cmake/help/latest/generator/Unix%20Makefiles.html>__
generator on Linux and macOS, which is a good choice and should work. Yet in some circumstances, you might want to
generate project files for a specific build tool or IDE:

.. code:: shell

cmake -G "Unix Makefiles" -S /path/to/source/dir -B /path/to/build/dir

A common alternative is the Ninja <https://cmake.org/cmake/help/latest/generator/Ninja.html>__ generator, which
requires ninja to be installed. It is mostly a make
replacement with less overhead and should work equally well. It is supported on all major platforms, including Windows:

.. code:: shell

cmake -G Ninja -S /path/to/source/dir -B /path/to/build/dir

On macOS, CMake also supports the Xcode <https://cmake.org/cmake/help/latest/generator/Xcode.html>__ generator. It will
create an .xcodeproj bundle which you can open in Xcode. It also adds support for automatic code signing, which might
be required if your application using projectM needs to be notarized for store deployment.

.. code:: shell

cmake -G Xcode -S /path/to/source/dir -B /path/to/build/dir

If you develop on Windows, you will possibly use Visual Studio. While recent visual Studio versions have CMake support
built-in, you can still pre-generate the solution and project files and open the .sln file from the build directory.
CMake provides a separate generator for each Visual Studio release. For Visual Studio 2022 you would use
the Visual Studio 17 2022 <https://cmake.org/cmake/help/latest/generator/Visual%20Studio%2017%202022.html>__ generator
and provide an additional architecture parameter:

.. code:: shell

cmake -G "Visual Studio 17 2022" -A "X64" -S /path/to/source/dir -B /path/to/build/dir

It is not possible to generate multi-arch solutions with CMake though. You need to create separate build directories and
use the respective -A switch for each.

Project-specific configuration options

text
CMake has no built-in way of printing all available configuration options. You can either refer to the
top-level ``CMakeLists.txt`` which contains a block of ``option`` and ``cmake_dependent_option`` commands, or use one of the
available CMake UIs which will display the options after configuring the project once.

Important build switches
^^^^^^^^^^^^^^^^^^^^^^^^

The following table also gives you an overview of important build options and their defaults. All options accept a
boolean
value (``YES``/``NO``, ``TRUE``/``FALSE``, ``ON``/``OFF`` or ``1``/``0``) and can be provided on the configuration-phase command line
using the ``-D`` switch.

.. list-table::
   :header-rows: 1

   * - CMake option
     - Default
     - Required dependencies
     - Description
   * - ``BUILD_TESTING``
     - ``OFF``
     -
     - Builds the unit tests.
   * - ``BUILD_DOCS``
     - ``OFF``
     - ``Doxygen``, ``Sphinx``
     - Builds the API/developer documentation in ``docs/``.
   * - ``BUILD_SHARED_LIBS``
     - ``ON``
     -
     - Build projectM as shared libraries. If ``OFF``, build static libraries.
   * - ``ENABLE_PLAYLIST``
     - ``ON``
     -
     - Builds and installs the playlist library.
   * - ``ENABLE_EMSCRIPTEN``
     - (auto)
     - ``Emscripten``
     - Forced ``ON`` when building with the Emscripten toolchain, otherwise ``OFF``. Not user-settable.
   * - ``ENABLE_GLES``
     - ``OFF``
     - ``GLES``
     - Use OpenGL ES 3 profile for rendering instead of the Core profile.

Note that ``ENABLE_GLES`` will be forcibly set to ``ON`` for Emscripten and Android builds, making it mandatory.

Experimental and application-dependent build switches
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The following table contains a list of build options which are only useful in special circumstances, e.g. when
developing libprojectM, trying experimental features or building the library for a special use-case/environment.

.. list-table::
   :header-rows: 1

   * - CMake option
     - Default
     - Required dependencies
     - Description
   * - ``ENABLE_SDL_UI``
     - ``OFF``
     - ``SDL2``
     - Builds the SDL-based test application. Only used for development testing, will not be installed.
   * - ``ENABLE_INSTALL``
     - ``ON`` (top-level) / ``OFF`` (subproject)
     -
     - Enable projectM install targets. Automatically ``ON`` when built standalone, ``OFF`` when added via ``add_subdirectory()``.
   * - ``ENABLE_DEBUG_POSTFIX``
     - ``ON``
     -
     - Adds ``d`` (by default) to the name of any binary file in debug builds.
   * - ``ENABLE_SYSTEM_GLM``
     - ``OFF``
     -
     - Builds against a system-installed GLM library.
   * - ``ENABLE_SYSTEM_PROJECTM_EVAL``
     - ``ON``
     -
     - Uses a system-installed projectM-eval library if found, otherwise the bundled sources in ``vendor/``.
   * - ``ENABLE_BOOST_FILESYSTEM``
     - ``OFF``
     - ``Boost``
     - Forces the use of ``boost::filesystem`` instead of ``std::filesystem``. Only needed on toolchains without C++17 filesystem support.
   * - ``ENABLE_MACOS_FRAMEWORK``
     - ``OFF``
     - macOS
     - Build as macOS Framework bundles instead of plain shared libraries.
   * - ``ENABLE_CXX_INTERFACE``
     - ``OFF``
     -
     - Exports symbols for the ``ProjectM`` and ``PCM`` C++ classes and installs the additional the headers. Using the C++ interface is not recommended and unsupported.
   * - ``ENABLE_VERBOSE_LOGGING``
     - ``OFF``
     -
     - Enables code for ``TRACE`` and ``DEBUG`` log levels in release builds. By default, these will only be compiled for ``Debug`` builds. Enabling this will negatively affect performance, even if the actual log level is set to ``INFORMATION`` or higher.

Path options
^^^^^^^^^^^^

There are also a few textual parameters that can be used to fine-tune the installation directories. Relative paths in
the following options are appended to the value
of `CMAKE_INSTALL_PREFIX <https://cmake.org/cmake/help/latest/variable/CMAKE_INSTALL_PREFIX.html>`__ (which, on most UNIX
platforms, defaults to ``/usr/local``):

.. list-table::
   :header-rows: 1

   * - CMake option
     - Default
     - Description
   * - ``CMAKE_INSTALL_PREFIX``
     - (OS dependent)
     - Base directory where the projectM libraries, includes and support files will be installed.
   * - ``PROJECTM_BIN_DIR``
     - ``bin``
     - Directory where executables (e.g. the SDL standalone application) are installed.
   * - ``PROJECTM_LIB_DIR``
     - ``lib[64]``
     - Directory where libprojectM is installed.
   * - ``PROJECTM_INCLUDE_DIR``
     - ``include``
     - Directory where the libprojectM include files will be installed under.

Other options
^^^^^^^^^^^^^

Various other options for specific needs.

.. list-table::
   :header-rows: 1

   * - CMake option
     - Default
     - Description
   * - ``CMAKE_DEBUG_POSTFIX``
     - ``d`` (if ``ENABLE_DEBUG_POSTFIX`` is set)
     - Postfix appended to debug libraries.
   * - ``CMAKE_INSTALL_BINDIR``
     - ``bin``
     - Another way to specify the binary installation directory. Used as default value for ``PROJECTM_BIN_DIR``.
   * - ``CMAKE_INSTALL_LIBDIR``
     - ``lib`` or ``lib64``
     - Another way to specify the library installation directory. Used as default value for ``PROJECTM_LIB_DIR``.
   * - ``CMAKE_INSTALL_INCLUDEDIR``
     - ``include``
     - Another way to specify the include file installation directory. Used as default value for ``PROJECTM_INCLUDE_DIR``.

Always perform out-of-tree builds!
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Most classic IDEs and build systems directly make use of the source tree and create project files, temporary build
artifacts (e.g. object files) and the final binaries in the same directory structure as the source files. An advantage
of this approach is that you can find all compiled binaries side-by-side with their sources and generated headers are
already in the same directories as the source files including them. This approach has some drawbacks though:

- Only a single build configuration is supported as files are overwritten in-place.
- A lot of noise is created in the source directory, making it hard to distinguish between generated and original source
  files.
- A very large ``.gitignore`` file is required to cover all unwanted files.
- Mistakes in the build scripts can overwrite source files, causing errors and destroy uncommitted work.

Some of these can be mitigated by providing additional targets (``make clean`` and ``make distclean``) or creating
subdirectories for Debug/Release build configurations.

While CMake also supports in-tree builds, it is "discouraged" in the official documentation, for the above reasons.
Building out-of-tree allows it to create multiple build directories with different configurations which do not influence
each other in any way. If a build directory contains unwanted artifacts, and you want to start fresh, simply delete and
recreate the whole directory - no work is lost.

This project follow this principle by treating the original source tree as read-only and avoiding potential conflicts:

- Everything under `CMAKE_SOURCE_DIR <https://cmake.org/cmake/help/latest/variable/CMAKE_SOURCE_DIR.html>`__ must only be
  read, never changed or written to.
- Everything under `CMAKE_BINARY_DIR <https://cmake.org/cmake/help/latest/variable/CMAKE_BINARY_DIR.html>`__ is temporary
  and related to the current build configuration.
- When generating configuration-dependent files,
  use `CMAKE_CONFIGURATION_TYPES <https://cmake.org/cmake/help/latest/variable/CMAKE_CONFIGURATION_TYPES.html>`__
  and `CMAKE_BUILD_TYPE <https://cmake.org/cmake/help/latest/variable/CMAKE_BUILD_TYPE.html>`__ to create non-conflicting
  files in the build tree.

While this project will not force you to build out-of-tree, there is no mechanism to clean up the generated files after
running cmake in-tree.

CMake build directory layout
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you are new to CMake, the way of how CMake creates the build directory and where it creates the build targets might
be confusing. Here is a summary of what's in the build directory and how it is structured in general.

Using files from the build tree
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

It is generally not good practice to directly take binaries and other files from the build tree for packaging, for
several reasons:

1. The directory structure is generated by CMake and depends on the generator used. The layout might change between
   CMake versions, even for the same generator.
2. On platforms with RPATH support, CMake will store absolute paths in executables and shared libraries which point to
   the absolute paths of any linked dependencies, either from the build tree or external libraries as well. These
   binaries are not relocatable and will most certainly not work if run on any other computer (or even on the same after
   deleting the build directory).
3. For some configurations, even Release build artifacts may contain debug symbols until they are installed.

It is fine to build and run executables from the build directory for development and debugging. For packaging or using
libprojectM in other projects, always use the ``install`` target and copy files from there.

Generated files
^^^^^^^^^^^^^^^

In the top-level build directory, CMake creates a few files that are present on any platform:

- ``CMakeCache.txt``: This file contains all variables and build settings CMake needs to remember from the first
  configuration run. This file can be edited on demand either manually or using a CMake UI to change any values. On the
  next build, CMake will regenerate the project files if this file has been modified.
- ``cmake_install.cmake``: Contains generated install-related settings.
- ``install_manifest.txt``: After installing the project, this file contains a list with absolute filenames of all
  installed files. It can be used for packaging or deleting installed files as CMake doesn't define an ``uninstall``
  target.
- The top-level project file for use with the selected build toolset, e.g. ``Makefile``, ``build.ninja``, ``projectm.sln``
  or ``projectm.xcodeproj``, plus additional toolset-specific files.

The projectM build files generate additional files used in the build and install phases. These are scattered over the
build tree, but installed into the proper directories. Do not try and gather these files yourself from the build tree.

Subdirectory structure
^^^^^^^^^^^^^^^^^^^^^^

The rest of the directory structure generally resembles the source tree. Source directories containing
a ``CMakeLists.txt`` file will also be created in the build tree with the same relative path. Each of these subdirectories
contains a ``CMakeFiles`` directory with CMake-internal data, generated project files for the select toolset, e.g.
makefiles and any temporary compile artifacts.

The directory structure is created by CMake and may change depending on the generator and CMake version used.

Executable and library locations
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Build targets - shared/static libraries and executables - are created in the same subdirectory in the build tree as
the ``CMakeLists.txt`` file that defines the target in the source tree (which, in most cases, resides in the same
directory as the source files). Depending on the generator used, the binaries are created directly in the directory for
single-configuration generators (like ``Unix Makefiles`` or ``Ninja``) and in a subdirectory with the configuration name,
e.g. ``Debug`` or ``Release``, for multi-configuration generators like ``Xcode`` or ``Visual Studio 17 2022``.

You may also find additional files and symbolic links in the same location depending on the platform, e.g. ``.pdb`` files
on Windows.

Using libprojectM in other CMake projects

The projectM library can be used as a static library or shared library in other CMake-based projects to provide embedded
audio visualization. It is highly recommended to build projectM as shared libraries for maximum compatibility and LGPL
compliance.

The build directory is not structured in a way that other projects can make use of it. Use the install target to copy
all required files to the configured installation prefix. You can customize the subdirectories for libraries, includes
and binaries using the PROJECTM_<X>>_DIR variables when configuring the CMake project.

Importing libprojectM targets from the build tree
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This approach is not recommended, but can be useful for projects that either require more in-depth access to the
projectM library files, especially to headers that are not installed as part of the public API. This might cause issues
if the internal headers change, but gives a broader set of features and more control to the developer.

Please refer to the ExternalProject <https://cmake.org/cmake/help/latest/module/ExternalProject.html>__ CMake module
documentation on how to set up the libprojectM build system for use in another project.

Importing libprojectM targets from an installed version
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is the recommended and supported way of importing libprojectM in your project. This project installs a set of CMake
files in <PREFIX>/<LIBDIR>/cmake/projectM4, containing target definitions, version and dependency checks as well as
any additional libraries required for linking. Other projects then use CMake's find_package command to search for
these files in different locations <https://cmake.org/cmake/help/latest/command/find_package.html#search-procedure>__.

In the case projectM libraries and headers are not installed in any system search path, you need to add either the
install prefix path (the top-level install dir) or the directory containing the libraries (the lib dir by default) to
the CMAKE_PREFIX_PATH <https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html>__ list.

If the package was found, you can then link against libprojectM by using the libprojectM::projectM target.

Depending on how the package was built, targets might be available for multiple configurations or only Release. CMake
will automatically select the most appropriate one to link.

Include dirs, additional link dependencies and possible compiler options will be propagated to any target the library is
linked to.

Using the optional playlist library
"""""""""""""""""""""""""""""""""""

If you want to use the optional playlist library, you need to specifically request it as a component:

.. code:: cmake

find_package(projectM4 COMPONENTS Playlist)

You can either use REQUIRED to force a fatal error if the component cannot be found or check if the target exists
using:

.. code:: cmake

if(TARGET libprojectM::playlist)
# ...
endif()

If you link the playlist library, the main libprojectM::projectM target will be linked automatically as a dependency.

Example
"""""""

Searches for projectM and the playlist library and links both to the application:

.. code:: cmake

find_package(projectM4 REQUIRED COMPONENTS Playlist)

add_executable(MyApp main.cpp)

target_link_libraries(MyApp PRIVATE
libprojectM::playlist
)


3rdparty Libprojectm Docs Emscripten (`3rdparty/libprojectm/docs/emscripten.rst`)

Using libprojectM in Emscripten

projectM supports OpenGL ES rendering, and can be compiled into
WebAssembly for use in browsers. WebGL is similar to OpenGL ES, but not
identical, so a few additional considerations apply to get projectM
running with Emscripten.

Additional Build Settings

A few additional build settings will be required when building an
Emscripten wrapper. Pass these flags/parameters to the Emscripten
linker:

  • -sUSE_SDL=2: It is highly recommended to use Emscripten’s
    built-in SDL2 port to set up the rendering context. This flag will
    link the appropriate library.
  • -sMIN_WEBGL_VERSION=2 -sMAX_WEBGL_VERSION=2: Forces the use of
    WebGL 2, which is required for OpenGL ES 3 emulation.
  • -sFULL_ES2=1 -sFULL_ES3=1: Enables full emulation support for
    both OpenGL ES 2.0 and 3.0 variants.
  • -sALLOW_MEMORY_GROWTH=1: Allows allocating additional memory if
    necessary. This may be required to load additional textures etc. in
    projectM.

Initializing Emscripten’s OpenGL Context

In addition to the above linker flags, some additional initialization
steps must be performed to set up the OpenGL rendering context for
projectM. Specifically, the OES_texture_float WebGL extension must
be loaded explicitly to support the required texture format for the
motion vector grid. The following code template can be used to set up a
proper SDL2/WebGL context for projectM:

.. code:: c

#include <emscripten.h>
#include <emscripten/html5_webgl.h>

#include <GL/gl.h>

#include <SDL.h>

int main(void)
{
// Init SDL's video and audio subsystems
SDL_Init(SDL_INIT_VIDEO | SDL_INIT_AUDIO);

text
// Create the SDL window (will be tied to the Emscripten HTML5 canvas)
   SDL_window* window = NULL;
   SDL_renderer* renderer = NULL;
   SDL_CreateWindowAndRenderer(1024, 768, SDL_WINDOW_OPENGL, &window, &renderer);
   if (window == NULL || renderer == NULL)
   {
       fprintf(stderr, "Failed to create SDL renderer: %s\n", SDL_GetError());
       return 1;
   }

   // Enable floating-point texture support for motion vector grid.
   auto webGlContext = emscripten_webgl_get_current_context();
   emscripten_webgl_enable_extension(webGlContext, "OES_texture_float");

   // Initialize projectM and put all other stuff below.
   
   return 0;

}


3rdparty Libprojectm Docs Index (`3rdparty/libprojectm/docs/index.rst`)

.. toctree::
:maxdepth: 3

self
building
emscripten
api/library_root

|Windows Build Status| |Linux Build Status| |macOS Build Status|
|Emscripten Build Status| |Android Build Status|

|Discord Shield| Chat with us on Discord <https://discord.gg/N9DyQfCH4j>__

.. figure:: https://github.com/projectM-visualizer/projectm/raw/master/docs/web/logo.png
:alt: Logo

Logo

projectM - The most advanced open-source music visualizer

Experience psychedelic and mesmerizing visuals by transforming music
into equations that render into a limitless array of user-contributed
visualizations.

projectM is an open-source project that reimplements the esteemed
Winamp Milkdrop <https://en.wikipedia.org/wiki/MilkDrop>__ by Geiss in
a more modern, cross-platform reusable library.

Its purpose in life is to read an audio input and to produce mesmerizing
visuals, detecting tempo, and rendering advanced equations into a
limitless array of user-contributed visualizations.

Important: This repository only contains libprojectM for use in application development!

text
This repository now only contains the projectM shared/static library.
All frontends, plug-ins and other tools were outsourced into separate
repositories. If you’re not a developer and just look for a download to
run projectM visuals on your machine or device, please use one of the
links listed below. The releases section in this repository only
contains source-code and binary releases of the projectM development
libraries and headers, which aren’t useful for end users.

End-User Applications
~~~~~~~~~~~~~~~~~~~~~

**Important**: projectM is currently undergoing heavy development. The
available end-user frontends are development previews, which are not
feature-complete and may have bugs.

Windows
^^^^^^^

-  Standalone (`projectMSDL 2.0
   pre-release <https://github.com/projectM-visualizer/frontend-sdl-cpp/releases>`__)
-  `Steam <https://store.steampowered.com/app/1358800/projectM_Music_Visualizer/>`__
   (Same as standalone development preview)
-  `Windows Store <https://www.microsoft.com/store/apps/9NDCVH0VCWJN>`__
   (Old 3.1.12 release)

macOS
^^^^^

-  Standalone

   -  `C++ app
      pre-release <https://github.com/projectM-visualizer/frontend-sdl-cpp/releases>`__
   -  `Rust app development preview
      (signed) <https://github.com/projectM-visualizer/frontend-sdl-rust/releases/tag/v0.1.0>`__
-  `Steam <https://store.steampowered.com/app/1358800/projectM_Music_Visualizer/>`__
   (Old 3.1.12 release)
-  Music.app Plugin (currently only available as an *unsigned*
   `development
   preview <https://github.com/kblaschke/frontend-music-plug-in/releases/tag/v3.0-pre1>`__)
-  `Brew <https://formulae.brew.sh/formula/projectm>`__ (Old 3.1.12
   release)

Linux
^^^^^

-  Standalone (`projectMSDL 2.0
   pre-release <https://github.com/projectM-visualizer/frontend-sdl-cpp/releases>`__,
   also available as ``.deb`` and ``.tar.gz``)
-  `Steam <https://store.steampowered.com/app/1358800/projectM_Music_Visualizer/>`__
   (Same as standalone development preview)

Or check your distribution’s package manager for a binary release. If it
is outdated, please contact the package maintainer, as the projectM
development team does not maintain any of the distribution-specific
packages.

Android
^^^^^^^

-  `Google
   Play <https://play.google.com/store/apps/details?id=com.psperl.prjM>`__

**Note**: Both the free and paid apps plus the Android TV app are *not*
created or supported by the projectM developers! If you have technical
troubles or other inquiries, please contact the app author via the means
provided in the Play Store. Any bug reports in the projectM issue
tracker regarding the apps will be closed immediately.

Xbox / Windows Phone
^^^^^^^^^^^^^^^^^^^^

-  `Windows Store <https://www.microsoft.com/store/apps/9NDCVH0VCWJN>`__
   (Old 3.1.12 release)

Other
^^^^^

Source code and other resources, mostly aimed at developers.

-  `Library source
   code <https://github.com/projectM-visualizer/projectm/>`__ (this
   repository)
-  `Qt <https://www.qt.io/>`__ based (Qt5/Qt6) desktop apps for Linux
   with `PipeWire <https://pipewire.org/>`__,
   `PulseAudio <https://www.freedesktop.org/wiki/Software/PulseAudio/>`__
   and JACK audio backends (`source
   code <https://github.com/projectM-visualizer/frontend-qt>`__,
   requires libprojectM 4.x).
-  `ALSA, XMMS, Winamp,
   JACK <https://sourceforge.net/projects/projectm/files/>`__ (legacy
   2.x sources for historic purposes, unmaintained since 2012)

Discord chat
~~~~~~~~~~~~

`Chat with us on Discord! <https://discord.gg/N9DyQfCH4j>`__

Click For Demo Videos
~~~~~~~~~~~~~~~~~~~~~

A demo playlist is available on
`YouTube <https://www.youtube.com/watch?v=jJmLQGhYWys&list=PLFLkbObX4o6TK1jGL6pm1wMwvq2FXnpYJ>`__.

Presets
~~~~~~~

The preset files define the visualizations via pixel shaders and
Milkdrop-style equations and parameters.

The projectM library does not ship with any presets. The frontends come
with varying preset packs which can be found in separate repositories in
the projectM repository list:

-  `Base Milkdrop texture
   pack <https://github.com/projectM-visualizer/presets-milkdrop-texture-pack>`__
   - Recommended for use with *any* preset pack!
-  `Cream of the Crop
   Pack <https://github.com/projectM-visualizer/presets-cream-of-the-crop>`__
   - A collection of about 10K presets compiled by Jason Fletcher.
   Currently, projectM’s default preset pack.
-  `Classic projectM
   Presets <https://github.com/projectM-visualizer/presets-projectm-classic>`__
   - A bit over 4K presets shipped with previous versions of projectM.
-  `Milkdrop 2
   Presets <https://github.com/projectM-visualizer/presets-milkdrop-original>`__
   - The original preset collection shipped with Milkdrop and Winamp.
-  `En D
   Presets <https://github.com/projectM-visualizer/presets-en-d>`__ -
   About 50 presets created by β€œEn D”.

Included with projectM are the bltc201, Milkdrop 1 and 2, projectM,
tryptonaut and yin collections. You can grab these presets
`here <http://spiegelmc.com/pub/projectm_presets.zip>`__.

You can also download an enormous 130k+ presets from the MegaPack
`here <https://drive.google.com/file/d/1DlszoqMG-pc5v1Bo9x4NhemGPiwT-0pv/view>`__
(4.08GB zipped, incl. textures).

Also Featured In
~~~~~~~~~~~~~~~~

`|Kodi| Kodi (formerly XBMC) <https://kodi.tv/>`__

`|Helix|
Helix <https://web.archive.org/web/20180628174410/http://ghostfiregames.com/helixhome.html>`__

`|Silverjuke| Silverjuke (FOSS Jukebox) <https://www.silverjuke.net>`__

`|VLC Media Player| VLC Media Player (AKA VideoLAN
Client) <https://www.videolan.org/vlc/>`__

Reminder: These are all third-party integrations of libprojectM and not
developed or supported by the projectM development team. Please report
bugs in those applications to their respective developers.

Screenshots
-----------

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-08-25\ at\ 12.31.20\ AM.png
   :alt: Screenshot

   Screenshot

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-08-25\ at\ 12.33.50\ AM.png
   :alt: Screenshot

   Screenshot

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-07-18\ at\ 2.14.41\ PM.png
   :alt: Screenshot

   Screenshot

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-07-18\ at\ 2.13.53\ PM.png
   :alt: Screenshot

   Screenshot

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-07-18\ at\ 2.15.36\ PM.png
   :alt: Screenshot

   Screenshot

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-08-16\ at\ 4.49.32\ PM.png
   :alt: Screenshot

   Screenshot

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-08-16\ at\ 4.50.37\ PM.png
   :alt: Screenshot

   Screenshot

.. figure:: web/projectM_screenshots/Screen\ Shot\ 2014-08-25\ at\ 12.31.07\ AM.png
   :alt: Screenshot

   Screenshot

Architecture
------------

-  `Article <https://lwn.net/Articles/750152/>`__

Building from source
--------------------

See :doc:`building` and the `developer documentation in the
wiki <https://github.com/projectM-visualizer/projectm/wiki/Building-libprojectM>`__.

Using the library
-----------------

At its core projectM is a library, `libprojectM <src/libprojectM>`__.
This library is responsible for parsing presets, analyzing audio PCM
data with beat detection and FFT, applying the preset to the audio
feature data and rendering the resulting output with OpenGL. It can
render to a dedicated OpenGL context or a texture.

To get started using projectM in your own projects, please go to the
wiki and read the `developer
documentation <https://github.com/projectM-visualizer/projectm/wiki#integrating-projectm-into-applications>`__
available there.

There are some open-source applications that make use of libprojectM
which can be found in the `projectM organization’s
repositories <https://github.com/projectM-visualizer>`__ and elsewhere.

Help
----

Report issues on GitHub in the respective repositories:

-  `Rendering issues, crashes or the libprojectM
   core/API <https://github.com/projectM-visualizer/projectm/issues>`__
-  `Standalone SDL
   app <https://github.com/projectM-visualizer/frontend-sdl2/issues>`__
   (including the Steam release)
-  `Windows Store
   App <https://github.com/projectM-visualizer/frontend-uwp/issues>`__
-  `Apple Music
   plug-in <https://github.com/projectM-visualizer/frontend-music-plug-in/issues>`__
-  Issues regarding the **projectM Android apps in the Play Store**,
   please contact the app author via the Play Store. We cannot help with
   any problems or requests.

If unsure, post your issue in the main `libprojectM issue
tracker <https://github.com/projectM-visualizer/projectm/issues>`__.
Please always check any existing issues if your problem has already been
posted by another user. If so, add your logs and findings to the
existing issue instead of opening a new ticket.

Get in contact with us
----------------------

`Chat with us on Discord. <https://discord.gg/N9DyQfCH4j>`__

Contribute to projectM
----------------------

If you would like to help improve this project, either with
documentation, code, porting, hardware or anything else please let us
know! We gladly accept pull requests and issues.

Before starting to write code, please take your time to read the
`contribution
guidelines <https://github.com/projectM-visualizer/projectm/wiki#contributing-to-projectm>`__
in our wiki.

Package Maintainers
-------------------

If you maintain packages of libprojectM, we are happy to work with you!
Please note well:

-  The main focus of this project is libprojectM. It’s a library that
   only really depends on OpenGL. The other applications are more like
   examples and demos.
-  Many of the frontend applications are likely outdated and of less
   utility than the core library. If you desire to use them or depend on
   them, please file an issue in the respective repository so we can
   help update them.
-  The β€œcanonical” application for actually viewing the visualizations
   is now
   `projectMSDL <https://github.com/projectM-visualizer/frontend-sdl-cpp>`__,
   based on libSDL2 because it supports audio input and is completely
   cross-platform.
-  This is an open source project! If you don’t like something, feel
   free to contribute improvements!
-  Yes, you are looking at the official version. This is not a fork.

Authors
-------

`Authors <https://github.com/projectM-visualizer/projectm/raw/master/AUTHORS.txt>`__

License
-------

The core projectM library is released under the `GNU Lesser General
Public License
2.1 <https://github.com/projectM-visualizer/projectm/raw/master/LICENSE.txt>`__
to keep any changes open-sourced, but also enable the use of libprojectM
in closed-source applications (as a shared library) as long as the
license terms are adhered to. The up- and downstream projects may use
different licenses - please check all parts of the software to be
compatible with your specific project if you plan an integration.

Wiki
----

More information for developers is available from the `projectM
Wiki <https://github.com/projectM-visualizer/projectm/wiki>`__.

.. |Windows Build Status| image:: https://github.com/projectM-visualizer/projectm/actions/workflows/build_windows.yml/badge.svg?branch=master
   :target: https://github.com/projectM-visualizer/projectm/actions/workflows/build_windows.yml
.. |Linux Build Status| image:: https://github.com/projectM-visualizer/projectm/actions/workflows/build_linux.yml/badge.svg?branch=master
   :target: https://github.com/projectM-visualizer/projectm/actions/workflows/build_linux.yml
.. |macOS Build Status| image:: https://github.com/projectM-visualizer/projectm/actions/workflows/build_osx.yml/badge.svg?branch=master
   :target: https://github.com/projectM-visualizer/projectm/actions/workflows/build_osx.yml
.. |Emscripten Build Status| image:: https://github.com/projectM-visualizer/projectm/actions/workflows/build_emscripten.yml/badge.svg?branch=master
   :target: https://github.com/projectM-visualizer/projectm/actions/workflows/build_emscripten.yml
.. |Android Build Status| image:: https://github.com/projectM-visualizer/projectm/actions/workflows/build_android.yml/badge.svg?branch=master
   :target: https://github.com/projectM-visualizer/projectm/actions/workflows/build_android.yml
.. |Discord Shield| image:: https://discordapp.com/api/guilds/737206408482914387/widget.png?style=shield
.. |Kodi| image:: web/kodi.png
.. |Helix| image:: web/helix.jpg
.. |Silverjuke| image:: web/silverjuke.png
.. |VLC Media Player| image:: web/vlc.png

---

### 3rdparty Libprojectm Vendor Projectm Eval Docs Compiler Internals (`3rdparty/libprojectm/vendor/projectm-eval/docs/Compiler-Internals.md`)

Compiler Internals
==================

This document describes how the compiler works and how its functionality can be extended. It is aimed at developers
integrating projectM-EvalLib into their own applications and looking for adding functions and new features on top of the
supported set of Milkdrop-compatible expression syntax.

## Scanner

The scanner (or lexer) is responsible for tokenizing the input data stream and return the type and possible content of
these tokens to the parser. This library uses GNU Flex to generate ode from a rules file, called `Scanner.l`. The
scanner looks for these kinds of symbols, with letters always being case-insensitive:

- Inline comments, starting with `//` until end of the current line
- Block comments, enclosed in `/* ... */`, which can appear anywhere and also span multiple lines.
- HEX constants in the form `$Xnn...`, scanned until any non-hexadecimal character is encountered (`[0-9a-f]`)
- ORD constants in the form `$'c'` where `c` is any single character.
- Math constants `$PI`, `$E` and `$PHI`.
- The `gmem` identifier, as it's used in the non-function form `gmem[index]`, requiring special treatment in the
  grammar.
- Numbers in integer and floating-point notation with an optional exponent. Note that these numbers are always positive,
  negativity is applied later via the `-` unary operator in the parser.
- Identifiers starting with an underscore or alphabetical letter, and also numbers on any following character.
  These can be returned as two different tokens:
    - If the (lower-case) name exists in the function table, the name is returned as a `FUN` token.
    - Anything else is returned as a `VAR` token and identifies a variable.

## Grammar

The grammar is defined in Bison (yacc) syntax and specifies the order in which tokens can appear and how they are
transformed into a tree structure.

In the grammar, different syntactical features are specified:

- Tokens returned by the scanner and their associated C data types in the parser.
- Operator precedence and associativity: Determines which operators are evaluated before others, and in which
  direction (left-to-right or right-to-left).
- The recursively defined grammar, consisting of a list of tokens per rule and an action.

Bison creates a state machine from the grammar, analyze any issues like potential ambiguities and take care of the
proper shift/reduce operations. Bison also provides error handling.

### Compilation Errors

In this grammar, any syntax error encountered will fail the compilation of the whole program. __This is a difference in
comparison to the Milkdrop parser__, which will not fail compilation on certain circumstances, silently ignoring errors
in the code. Reproducing the exact same behaviour would require a completely different approach in the grammar, scanner
and parser.

## Expression Tree

### Node Object

All grammar as described in the previous sections is scanned and compiled into a tree structure by the parser actions.
Each node is a `prjm_eval_exptreenode` struct, which contains:

- A function pointer `func`, which is determined the behaviour of this node.
- A fixed float value `value` to store constant numbers or temporary values.
- A union of either `var`, pointing to a variable storage location, or `memory_buffer` which is a pointer to
  a `projectm_eval_mem_buffer` with (g)megabuf data.
- An array of pointers `args`, pointing to the argument node objects of the function.
- A linked-list pointer `list` which stores an instruction list to be executed by loops or the special `/*list*/`"
  function.

In general, only a few different "objects" will be created as tree nodes:

- Constant: Return simple read-only float-typed numbers. Uses the `value` member to store the constant.
- Variable: Return a read/write variable reference. USes the `var` member to store the variable storage location.
- Memory access function: Operates on the memory buffer specified in `memory_buffer`, either retrieving or setting data.
- Function: Executes objects in `args` and/or the expressions in `list`, then eventually applies an operation on the
  result and sets the return value to either a constant value or a variable reference.

### Node Function

The function in each node object has a `void` return value and takes two parameters:

- The execution context `ctx`, which is a pointer to the `prjm_eval_exptreenode` object the function will be executed on.
  Think of it as the C++ `this` pointers.
- A pointer to a float pointer `ret_val`, which is the return value of the function.

Every node function _must_ return either a value or a value reference. By convention, this is either the result of the
operation the function carried out, or for functions using instruction lists, the result of the _last executed
instruction_.

Depending on the function, the result can either be a simple constant value or a reference to a variable.

When calling a math function like `sin()`, implementing a math operator or returning a constant, the function must
assign the value to the value pointed to by `*ret_val`. Use the `assign_ret_val()` macro to assign the return value:

```c
assign_ret_val(sin(x));
```

Some functions will not carry out an actual math operation, but return the result of an expression, which may be a
variable reference, which may be used as the first argument of the internal `_set` function (or the
AVS-specific `assign` function) to assign it the result of another expression. A reference can be assigned using
the `assign_ret_ref` macro, example taken from the `prjm_eval_func_var` function:

```c
assign_ret_ref(ctx->var);
```

As with all pointers, an assigned pointer must stay valid at least until the end of the program's execution. It is
recommended to only use the following memory references:

- The `var` pointer in variable access functions.
- The address of the `value` member of the current `prjm_eval_exptreenode` object in which context the function runs.

Do not assign pointers to locally defined variables.

Passing `ret_val` as the return value to a function argument and let the function set it to the desired result is also
viable:

```C
prjm_eval_function_decl(myfunc)
{
    assert_valid_ctx();

    invoke_arg(0, ret_val);
}
```

The `if` function evaluates its first argument, and depending on it being a true or false value, returns either the
second (`true`) or the third (`false`) argument. Both arguments may return a variable reference, so the `if` functions
needs to take care of this by simply calling the appropriate function and passing the current `ret_val` to it. This way,
whatever the sub-expression assign to it will be returned by the `if` function.

### Temporary Compiler Objects

#### Node Object

Any tree node object created in a parser action will be wrapped inside a temporary compiler node of
type `prjm_eval_compiler_node`, which stores a few additional flags only required inside the parser's reduce actions:

- The type of the current parser node, either a single expression or an expression list.
- Constness and state-changing flags of the expression (last expression in a list).
- combined constness and state-changing flags of the expression list. Same as the previous one if only a single
  expression is stored in the compiler node.

#### Function Argument List

Inside a function argument list, all expressions are collected in a special `prjm_eval_compiler_arg_list` wrapper holding
one compiler node object for each potential function argument. It also keeps the argument count as a separate variable
for easy access.

When the arguments have been collected and the function is reduced in the parser, the action will then compare the
actual argument count against the count expected by the function. If the numbers don't match, a parse error is thrown.

### Optimizations

The parser does perform two different optimizations during compile time to save execution time:

#### Compile-time Evaluable Functions

The compiler optimizes the generated code by evaluating constant expressions at compile time. These expressions purely
consist of functions which both only have constant-evaluable arguments and have a fixed, deterministic behaviour.
Examples of such functions, given their arguments are also const-evaluable:

- Functions which return compile-time constants.
- Non-assigment Operators.
- Math functions like `sin`.

Non-constant functions are functions which depend on variables or other memory contents which may change during or
between executions or produce a non-deterministic result on each execution. Examples are:

- Functions which have at least one non-const-evaluable argument.
- Variable and memory access functions.
- Random number generators.

During compilation, when a new node is created from a subtree, each argument and the new node are checked for being
const-evaluable. If all are, then the new node is immediately evaluated by calling the function, and a new constant node
is inserted with the calculated value, replacing both the currently processed node and its argument nodes.

#### Expression List Optimization

In function arguments and inside parentheses, expression lists, e.g. multiple expressions separated by semicolons, can
be written. This is a useful feature to execute a list of expressions based on certain conditions (e.g. in `if`
or `loop`). While only the _last_ expression in a list determines the return value of the whole list, other expressions
may perform calculations, set variables etc. and also need to be executed in order to achieve the correct result.

There could also be expressions in the list which do not assign anything, and which are not the last expressions. These
expressions are referred to as "non state-changing", and only waste CPU cycles because the result of any calculation
done there is discarded. The expression compiler will keep track of the state-changing flag hierarchically, similar to
the const flag, and remove any expressions which are nether state-changing nor relevant to the return value.

In the case that a state-changing operator, e.g. an assignment, has a constant value as the left operand, it will be
replaced by the const-evaluation optimization and lose its state-changing flag. Consider the following expression, where
the first two expressions will be optimized out:

```
sin(1 + 2; 3);
tan(5) = x;
5 = rand(100);
x = cos($PI * .54);
5;
```

The third expression cannot be optimized out, because the assignment operator is potentially state-changing if it
assigns to a variable or memory location and the `rand` function is not const-evaluable.

The fourth expression is very similar, but this time, the use of a variable prevents the expression form being
const-evaluable.

The fifth and last expression is a simple constant and determines the return value of the whole expression list.

---

### 3rdparty Libprojectm Vendor Projectm Eval Docs Expression Syntax (`3rdparty/libprojectm/vendor/projectm-eval/docs/Expression-Syntax.md`)

Milkdrop Preset Expression Syntax
=================================

This document contains a full writeup of the expression syntax in Milkdrop presets. It documents the general syntax,
available operators and functions and also any specific, non-obvious behaviours. Any differences between Milkdrop's
original parser and projectM's are also documented where applicable.

This document does _not_ cover the `warp_` and `comp_` shader code blocks. The shader code is written in DirectX HLSL,
which is better explained by Microsoft's documentation in the MSDN.

The document also doesn't cover the different code blocks, predefined variables and scope of the special t/q values.
Please refer to the preset authoring guide for these.

## General Syntax

The expression language only knows a few syntactic features:

- Numbers, with or without decimals. Can also contain a base 10 exponent using the `e[+-]n` notation.
- Variables to hold a single number, both user-defined and predefined. May contain characters a-z, 0-9 and the
  underscore (`_`), while 0-9 must not be the first letter.
- Special constants beginning with `$`.
- A single expression, separated from other expressions in the same list via semicolons (`;`), with the last semicolon
  being optional.
- Calls to built-in functions in the form `func(arg1, arg2, ...)`.
- Mathematical, assignment and other operators.
- Subscript access `[]` for megabuf/gmegabuf access.
- Parentheses (`()`) to group statements and override precedence.
- In-line comments starting with `//` until the end of the current line
- Multi-line comments starting with `/*` and ended with `*/`.

### Return Types

Expressions can return either a numeric value or a reference to a variable or memory location. What is actually returned
is determined by the function or operator that is executed last. If the expression only consists or a single token, then
the following applies:

- If the expression token is a constant number (`1.0;`), the number is returned by value.
- If the expression token is a variable name (`x;`), a reference to the variable is returned.

For most functions, the return type doesn't matter. If it's a reference, the value pointed to by the reference is used.
References only behave differently when used on the left side of assignments or in the forst argument of the  `assign()`
function. In this case, the variable or (g)megabuf location the reference points to is assigned a new value.

### Expression Result

Every expression returns its outermost result, as a value or reference as described above. When writing multiple
expressions (separated by semicolon) in a list, the result of the _rightmost_ (or last) expression is returned. The
results of previous expressions are discarded.

In general, each expression will be in the form `var = expression;`, assigning a value to the given variable (or
megabuf index). Expressions will also always return a value, so when used as function arguments, this value will be
used. For example, all the following statements will call the `sin()` function with the argument `4`:

```
x = sin(val = 2 * 2);
y = sin(2 + 2);
z = sin(4.0);
```

### Variable and Function Names, Limits

Variable and function names are generally case-insensitive, making `variable` equivalent to `vArIaBlE`.

Any text in expressions - starting with either a letter from a to z or an underscore, followed by more of these
characters and also numbers from 0 to 9 - is first checked to be a built-in function. If it is not a function, the
parser will interpret it as a variable.

Functions always need to be followed by parentheses (`sin(expr)`), while variables must not.

Milkdrop imposes a 16-character limit on variable names. projectm-eval does not have a hard limit, but it's advisable to
not use more than 16 characters for Milkdrop compatibility.

Milkdrop also has a limit of 64 user-defined variables, which also isn't applied by projectm-eval.

#### Global "Register" Variables

In addition to gmegabuf, there are an additional 100 variables which can be used to store global values. These variables
are named `reg00`, `reg01` etc. up to `reg99`. Values stored in those variables are available in al execution contexts,
in the same way as gmegabuf.

Note the index must always be written with two digits. `reg3` is _not_ considered a global variable and will only have a
local scope as any other variable. The same is true for more digits like `reg123`.

Same as with gmegabuf, global variables are not necessarily `0` when a preset is initialized, and they can change at any
time when two presets using the same global variables are blended during a transition.

### Constants

All constants are converted into floating-point numbers internally, even if written as integers.

As with all other names, constant names (and hexadecimal values) are also case-insensitive.

#### Numerical Constants

Basic numerical/decimal constants can be defined in the code in the following formats:

- Integer values: `1` (or `1.`)
- Decimal values: `1.234` (values between 0 and 1 can be abbreviated with `.234`, which equals `0.234`)
- Base 10 exponential form: `1.234e5` equals `123400` and `1.234e-5` equals `0.00001234`

#### Predefined Constants

A few named convenience constants can be used:

- `$PI`: Equals `3.141592653589793`
- `$E`: Equals `2.71828183`
- `$PHI`: Equals `1.61803399`

#### Hexadecimal Constants

Hexadecimal constants start with `$X`, immediately followed by the hexadecimal numbers.

Example: `$XFF` converts to `255`.

#### Character (Ordinal) Constants

Constants can also convert a single character into its ordinal (ASCII) value using the form `$'c'`, where `c` is the
character to convert.

Example: `$'a'` converts to `97`.

### Operator Precedence

Operator precedence is defined in a similar way as in C, except the `^` operator, which has a different
meaning in Milkdrop presets (pow() instead of binary XOR).

See the [C precedence table on cppreference.com](https://en.cppreference.com/w/c/language/operator_precedence) for
details. The pow operator is applied just after the unary plus/minus operators, having a slightly lower precedence.

### Subscript Access

The subscript operator or "array access operator" has a slightly different meaning than in other programming languages.
The Milkdrop expression syntax only knows a single data type, individual numbers, so there are no arrays.

If any index value is not an integer, it is _rounded_ to the nearest integer after calculating the final index (e.g.
adding index and offset first, then rounding).

The subscript operator is instead used to address memory locations in megabuf and gmegabuf. There are three possible
syntax variants:

#### Global Memory Access

Accessing the gmegabuf is possible by using the special keyword `gmem`, followed by an index in the subscript brackets.
The following statement sets gmegabuf location 10.000 to the current value of `x`:

```
gmem[10000] = x;
```

Any memory index from 0 to 8.388.607 (= 128 * 65536) can be addressed.

#### Local Memory Access

Accessing the current context memory buffer (megabuf) can be done by writing the index before a set of empty brackets.
The following example is analogous to the above, but it sets index 10.000 of the local memory instead:

```
10000[] = x;
```

#### Local Memory Access with Offset

An optional offset can be provided in the brackets, which is simply added to the index on the outside. The following
example will set memory index 10.123 to the value of `x`:

```
10000[123] = x;
```

Both index and offset values can of course be calculated with expressions. So the following expression is valid:

```
if(x > 5,5000,1000)[(sin(y) + 1 * .5) * 1000] = z;
```

### Using Parentheses

Parentheses can be used to specify a specific order in which operations are executed. Expressions inside parentheses are
always evaluated, and the result of the evaluation is then used to evaluate any outside expression.

Inside parentheses, using expression lists is also valid. The result is, as described above, the value of the last
expression in the list. For example, the following expression will use `5` as the return value:

```
x = a * (b + c; d + e; 2 + 3);
```

The first two operations will do nothing, as they don't set a variable or memory value and the results are discarded.

### Comments

For more complex calculations, it might be a good idea to leave some comments in the expressions to describe what's done
in a specific line or block. Comments can also be useful to quickly disable a line or block of code to try out things
when writing a preset.

Comments can be used as end-of-line or block variants:

```
x = sin(5); // This comment is only valid until the line ends
y = sin($PI * .5); /* This comment
spans
multiple
lines and the following expression is executed: */ z = 5;
```

Comments can appear inside any expression, but _not_ within a variable, function name, number or other constant.
Writing `x = var/*comment*/iable;` is invalid (but `x = sin/*comment*/(variable);` is).

## Operators

The following operators can be used in expressions.

### Mathematical Operators

These operators carry out basic mathematical operations.

- `+`: Adds both operands.
- `-`: Subtracts the right operand from the left.
- `*`: Multiplies both operands.
- `/`: Divides the left operand by the right. If the divisor is zero, the result of the operation is `0` instead
  of `Inf`.
- `%`: Converts the operands to integers and returns the remainder of the left by the right operand.
- `^`: Calculates the left operand to the power of the right operand.

### Equality Operators

These operators check for equality or inequality of the operands and return a boolean result:

- `==`: Equal. Returns `1` if both operands are equal, `0` if not.
- `!=`: Not equal. Returns `1` if both operands are not equal, `0` if they are.
- `<`: Below. Returns `1` if the left operand is below the right operand, `0` otherwise.
- `>`: Above. Returns `1` if the left operand is above the right operand, `0` otherwise.
- `<=`: Below or equal. Returns `1` if the left operand is below or equal to the right operand, `0` otherwise.
- `>=`: Above or equal. Returns `1` if the left operand is above or equal to the right operand, `0` otherwise.

### Binary Arithmetic Operators

The following operators allow to carry out bitwise operations on integer representations of the operands:

- `&`: Converts both arguments to integers and calculates the bitwise AND between the resulting operands.
- `|`: Converts both arguments to integers and calculates the bitwise OR between the resulting operands.

### Boolean Operators

Boolean operators only check an operand to be zero or non-zero, then return the according result:

- `&&`: Returns `1` if both operands are non-zero and `0` if any operand is zero.
- `||`: Returns `1` if one or both operands are non-zero and `0` if both operands are zero.
- `!`: Unary operator. Returns `1` if the operand is zero and `0` if the operand is non-zero.
- `a ? b : c`: The ternary operator. Evaluates the `a` operand and evaluates and returns the `b` operand if the
  condition is non-zero. If the condition evaluates to zero, only the `c` operand is evaluated and returned.

### Assignment Operators

Assignment operators can simply assign the result of the right-hand side to a variable or megabuf index, or in case of
compound operators `a OP= b` also carry out an additional operation equivalent to `a = a OP B`. All assignment operators
return a reference to the left operand.

- `=`: Assigns the value of the right operand to the left operand.
- `+=`: Adds the value of the right operand to the left operand and assigns the result to the left operand.
- `-=`: Subtracts the value of the right operand from the left operand and assigns the result to the left operand.
- `*=`: Multiplies the value of the right operand with the left operand and assigns the result to the left operand.
- `/=`: Divides the left operand by the value of the right operand and assigns the result to the left operand. If the
  divisor is zero, the result of the operation is `0` instead of `Inf`.
- `%=`: Calculates the remainder of the rounded integer values of the left operand divided by the right operand and
  assigns the result to the left operand. If the divisor is `0`, the result will also be `0`.
- `^=`: Calculates the left operand to the power of the right operand and assigns the result to the left operand.
- `&=`: Calculates the binary AND of the rounded integer values of the left and right operands and assigns the result to
  the left operand.
- `|=`: Calculates the binary OR of the rounded integer values of the left and right operands and assigns the result to
  the left operand.

### Subscript Operator

The subscript - or index - operator `[]` is a special notation to access megabuf and gmegabuf. If the inside of the
brackets is empty, it is interpreted as a `0`. For more details, see the "Subscript Access" section above.

## Built-in Functions

The following functions are built into the expression parser and can be used in all Milkdrop 2.x presets.

### above(left, right)

Returns `1` if `left` has a larger value than `right`, otherwise `0`.

Same as the `>` operator.

### abs(val)

Returns the absolute (positive) value of `val`.

### acos(val)

Calculates and returns the arc cosine (inverse cosine) of `val`. The angle is interpreted as radians.

### asin(val)

Calculates and returns the arc sine (inverse sine) of `val`. The angle is interpreted as radians.

### assign(dest, val)

Assign `val` to the reference pointed by `dest` and return `dest`. If `dest` is not a reference, the result is the
_value_ of `val`.

Same as the assignment operator `=`.

### atan(val)

Calculates and returns the arc tangent (inverse tangent) of `val`. The angle is interpreted as radians.

### atan2(x, y)

Calculates and returns the polar coordinate angle associated with the coordinate represented by `x,y`. The angle is
returned as radians.

See the [Wikipedia definition of atan2](https://en.wikipedia.org/wiki/Atan2) for details.

### band(left, right)

Evaluates both arguments and returns `1` if both arguments evaluate to a non-zero value and `0` if at least one argument
is zero.

Note: The boolean AND operator `&&` returns the same result, but only evaluates the second argument if the first one is
not zero!

### below(left, right)

Returns `1` if `left` has a smaller value than `right`, otherwise `0`.

Same as the `<` operator.

### bnot(val)

Boolean NOT operator. Returns `1` if `val` is `0`, otherwise `0`.

Same as the `!` operator.

### bor(left, right)

Evaluates both arguments and returns `1` if at least one argument evaluates to a non-zero value and `0` if both
arguments are zero.

Note: The boolean OR operator `||` returns the same result, but only evaluates the second argument if the first one is
zero!

### ceil(val)

Rounds the value of `val` to the next _larger_ integer number in reference to `0`.

### cos(val)

Calculates and returns the cosine of `val`. The angle is interpreted as radians.

### equal(left, right)

Compares both values and returns `1` of they are equal or `0`if not.

Same as the `==` operator.

### exec2(expr1, expr2)

Executes both expressions and returns the result of the second one.

### exec3(expr1, expr2, expr3)

Executes all three expressions in order and returns the result of the third one.

### exp(val)

Exponential function with base `e` to the power of `val`.

Equivalent to the expression `$E ^ val`.

### floor(val)

Rounds the value of `val` to the next _smaller_ integer number in reference to `0`.

### freembuf(index)

Requests freeing all memory blocks in `megabuf` (NOT `gmegabuf`) beginning with the block `index` is in. This may result
in memory being freed before the given index.

**Note:** This function is currently a no-op in both Milkdrop and projectM. It does _not_ actually free any memory or
reset the memory contents.

Memory blocks are 65536 entries large, and there is a total of 128 blocks.

### gmegabuf(index)

Returns a reference to the storage location of `index` in the global memory buffer.

Any memory index from 0 to 8.388.607 (= 128 * 65536) can be addressed. Any index outside of this range will return the
fixed _value_ `0`, and no memory is changed when assigning another value to it.

### if(cond, trueval, falseval)

Evaluates `cond`, and returns `trueval` if the result is non-zero. Otherwise, `falseval` is returned.

If either `trueval` or `falseval` evaluate to a reference, the reference is passed through. Thus, assigning a value to
the result of `if()` is possible:

`if(x > 5, a, b) = 10;` will assign `10` to `a`  if `x > 5` and to `b` if `x <= 5`.

### int(val)

Truncates any decimals from `val`, effectively converting it into an integer.

### invsqrt(val)

Calculates the [fast inverse square root](https://en.wikipedia.org/wiki/Fast_inverse_square_root) of `val`.

### log(val)

Calculates and returns the ("natural") logarithm to base `e` of `val`.

### log10

Calculates and returns the logarithm to base 10 of `val`.

### loop(count, expr)

Executes `expr` exactly `count` times.

As mentioned in the grammar section, `expr` can also consist of multiple expressions separated by semicolon. this is
also true for `count`, but expressions in this argument are only evaluated once before the first loop iteration starts.

The loop will always abort after 2^20 (~1 million) iterations.

Be careful when nesting loops, as the loop count and associated run time will grow exponentially with each nesting
level, which can hang up the application.

### max(val1, val2)

Returns the larger of the two _values_ of `val1` and `val2`.

### megabuf(index)

Returns a reference to the storage location of `index` in the local memory buffer.

Any memory index from 0 to 8.388.607 (= 128 * 65536) can be addressed. Any index outside of this range will return the
fixed _value_ `0`, and no memory is changed when assigning another value to it.

### memcpy(dest, src, count)

Copies `count` values starting at `src` to `dest`. Returns a reference to `dest`.

Memory areas are allowed to overlap.

Any source or destination memory falls outside the allowed indices of 0 to 8.388.607, the range is trimmed accordingly
and only a data block is copied that fits into both areas.

### memset(dest, value, count)

Fills `count` number of memory slots beginning at `dest` with `value`.

If the range falls outside the allowed indices of 0 to 8.388.607, the range is trimmed accordingly.

This function is _way_ faster than setting individual values to 0 in a `loop()` and should be preferred if a larger
memory block needs to be initialized or reset with a single number.

### min(val1, val2)

Returns the smaller of the two _values_ of `val1` and `val2`.

### pow(base, exp)

Calculates `base` to the power of `exp` and returns the resulting value.

### rand(max)

Calculates a pseudo-random number (using the Mersenne-Twister algorithm) between 0 and `max`.

`max` can be a floating-point value.

### sigmoid(a, b)

Calculates the logistic curve with an argument of `(-a * b)` and returns the result.

This is equivalent to the expression `tmp = 1 + pow($E, (-a + b)); if(tmp > 0.00001, 1 / tmp, 0);`.

### sign(val)

Returns the signage of `val`, which is `-1` for negative values, `1` for positive values and `0` for `0`.

### sin

Calculates and returns the sine of `val`. The angle is interpreted as radians.

### sqr(val)

Calculates and returns the square of `val`.

This is equivalent to the expression `val * val`.

### sqrt(val)

Calculates and returns the square root of `val`.

### tan(val)

Calculates and returns the tangent of `val`. The angle is interpreted as radians.

### while(expr)

Executes `expr` at least once and loops until it returns `0`.

`expr` can be a semicolon-separated list of expressions. Only the return value of the last expression in the list is
used to determine the loop abort condition.

The loop will always abort after 2^20 (~1 million) iterations.

Be careful when nesting loops, as the loop count and associated run time will grow exponentially with each nesting
level, which can hang up the application.

### Internal Functions and Aliases

The following functions are defined internally, some being aliases to the above functions, others being used as actual
implementations of operators. Depending on the operator being unary or binary, each function has one or two parameters.

- _aboeq => Operator `>=`
- _above => Operator `>`
- _add => Operator `+`
- _addop => Operator `+=`
- _and => Operator `&&`
- _andop => Operator `&=`
- _beleq => Operator `<=`
- _below => Operator `<`
- _div => Operator `/`
- _divop => Operator `/=`
- _equal => Operator `==`
- _gmem => Alias for `gmegabuf`
- _if => Operator `?:`, alias for `if`
- _mem => Alias for `megabuf`
- _mod => Operator `%`
- _modop => Operator `%=`
- _mul => Operator `*`
- _mulop => Operator `*=`
- _neg => Negation operator `-`
- _not => Operator `!`
- _noteq => Operator `!=`
- _or => Operator `||`
- _orop => Operator `|=`
- _powop => Operator `^=`
- _set => Operator `=`, alias for `assign`
- _sub => Operator `-`
- _subop => Operator `-=`

### Externally-defined Functions

Be aware that if this parser is used in other projects or non-Milkdrop-compatible presets, additional functions may be
defined which can't be used as variable names. One example are the additional `getosc` and `getspec` functions defined
in AVS.

## Memory Buffers

Each execution context has two associated memory buffers, a local buffer and a global buffer. Each buffer can hold up to
2^23 (8.388.608) individual floating-point numbers which can be addressed via their zero-based index.

The local buffer is only valid within the same execution context. In classic Milkdrop, there are these contexts, each of
which having its own `megabuf`:

- The global per-frame context (`per_frame_init_` and `per_frame_`)
- The per-vertex/per-pixel mesh context (`per_pixel_`)
- The wave per-frame context, one per waveform (`wave_N_init` and `wave_N_per_frame`)
- The wave per-point context, one per waveform (`wave_N_per_point`)
- The shape per-frame context, one per shape (`shape_N_init` and `shape_N_per_frame`)

The global memory buffer, referred to as `gmegabuf`, only exists once and is used for _all_ execution contexts. It is
even shared between different Milkdrop presets. Keep this in mind when using the buffer, as the contents may be changed
by another preset during the transition phase!

Local buffers always start empty and return zero for all indices when being accessed the first time. The global buffer
will also start this way, but is never reset. Code using the global megabuf should initialize the required indices with
known values.

Memory is allocated in blocks of 64K (65.536) values each, with a total of 128 of such blocks per buffer. When using
memory indices, ideally keep the indices close together to allocate only a minimum number of blocks and save the user's
RAM resources.

### Setting Large Blocks of Memory to a Certain Value

If a larger memory region needs to be initialized with a single value, e.g. resetting a part of `gmegabuf`, assigning
single values in a `loop` is time-consuming. The `memset` function can be used to achieve the same result in a very fast
operation. Calling the following function will reset values 10.000 to 19.999 to `5`:

```
memset(10000, 5, 10000);
```

### Copying/Moving a Range of Data

Sometimes it's required to copy or move a continuous block of values, e.g. moving data from previous frames left and
adding a new set of data points at the end. This can be done in a `loop`, but again is very time-consuming. The `memcpy`
function is able to perform this operation in a single call. It also supports moving memory in overlapping areas - the
original contents are first copied into a temporary buffer, then written to the new location. To move data forward in a
1000-value large block by 100 indices, with the block starting at index 5000 looks like this:

```
memcpy(5000, 5100, 900);
```

The last 100 values will appear twice in the block, while the first 100 got overwritten.

## Caveats and Tricks

For most uses, the expression language is quite straightforward to use. Due to the implementation, users can take
advantage of some features while they need to be aware of a few details that may cause problems.

### Using References

As mentioned in the sections above, some functions may or may not return variable or memory references. It is, in any
case, valid to assign a value to any other value, even a constant. So while the following statement doesn't do anything
useful, it is valid:

```
assign(1, 2);
```

The function itself will return the value `2` as expected, but the actual assignment is discarded.

Other constructs are more helpful, for example to choose between two or more variables to assign a value to. The
following code will assign `10` to either `a`, `b` or `c` depending on the value of `x`:

```
assign(if(x < 0, a, if(x > 0), c, b), 10);
```

This works because `if` will pass the result of the appropriate true/false expression, which can be a reference to a
variable. In the above example, it would also be valid to return `0` instead of `b` is no assignment is wanted, but
the `assign()` function should still return the value 10 in any case.

The last example can also be written in the assignment form, with `if()` as the left operand:

```
if(x < 0, a, if(x > 0), c, b) = 10;
```

### Conditions in `if` and `while`

A few functions check a condition to execute a certain instruction. For example, `if` decides whether to use the true or
false argument, and `while` uses the value of the last expression to check if the loop should abort.

Due to how floating-point numbers work in computers, not all calculations may return exactly `0.0`, even when using
simple constants. The reason is that some numbers can't be exactly represented by the binary number used by the CPU and
needs to be slightly modified. Using such a number in math equations may even increase the error, and the result might
never be exactly zero.

To make the expression work as intended, one of the following approaches may be used, depending on what is required:

1. If only the integer part of the value is of interest, use `int(val)` to truncate any decimals.
2. Round the result to the nearest integer with `round(val)` if the error can be larger than `0.5`.
3. Compare the absolute value of the number to be smaller/larger than the allowed error, e.g. `abs(val) > 0.0001`

The boolean comparison operators (`<`, `==` etc.) will always return either `0.0` or `1.0`, which don't need to be
rounded and can be used directly for the condition check. Using those operators should always be preferred over directly
passing a calculated variable value as the conditional.

### Performance

While execution of the compiled programs is generally quite fast, some functions and code constructs can heavily impact
the performance, especially on low-end and embedded devices like the Raspberry Pi. To make presets available to a broad
audience, keep the following considerations in mind:

- Keep code in the `per_pixel_` block small and fast. If possible, perform CPU-intensive work in the `per_frame_` block
  and store the result in the `q` variables (or `gmegabuf` if required).
- The same is true for instanced shape and waveform `per_point` code.
- Do not overuse `loop` and `while`, especially avoid nesting them. If they are required, make sure the abort condition
  on the `while` loop is not affected by rounding errors. Try not to calculate heavy stuff inside a loop.
- Some math functions, specifically trigonometrical ones like `sin` and also `sqrt`, are expensive. Try to cache and
  reuse values instead of recalculating them multiple times.
- Try not to use `megabuf` and `gmegabuf` for everything. Normal variables, including the global `regNN`, have less
  overhead, as they do not require the additional memory allocation checking and address calculation.

---

### 3rdparty Libprojectm Vendor Projectm Eval Docs Memory Handling (`3rdparty/libprojectm/vendor/projectm-eval/docs/Memory-Handling.md`)

Handling Global Memory
======================

Each projectM-Eval context has its own megabuf, which will live inside the context and is deleted when the context is
destroyed. The same is true for any script-defined or externally registered variables. To share global data between
different execution contexts, there are two methods:

- The reg00-reg99 variables.
- The gmegabuf memory buffer.

The reg variables were available for a long time, while megabuf and gmegabuf were introduced in ns-eel2 during Milkdrop
2 development.

## Using the Built-in Global Memory Buffer

If the embedding application doesn't need to use multiple global memory buffers, for example if all execution contexts
will share the same global memory, then the built-in buffer and reg vars can be used by passing NULL for both parameters
in the `projectm_eval_context_create()` function:

```c
struct projectm_eval_context* ctx = projectm_eval_context_create(NULL, NULL);
```

To free and reset the global memory buffer, call `projectm_eval_memory_global_destroy()`. This will not reset the reg
variables though.

## Using Application-defined Global Memory Buffers

Both the gmegabuf and the reg variables can be instantiated by the embedding application to control which execution
contexts will share the data. Both the buffer and the reg variables can be shared individually.

### Custom gmegabuf

To create a custom global memory buffer, call `projectm_eval_memory_buffer_create()` and store the resulting pointer. It
is an opaque data structure which will receive the gmegabuf data as scripts execute. Then pass the pointer to all newly
created contexts. If the buffer is no longer needed, e.g. all contexts using it were destroyed, remember to also free
the buffer.

Example:

```c
projectm_eval_mem_buffer gmbuf = projectm_eval_memory_buffer_create();
struct projectm_eval_context* ctx = projectm_eval_context_create(gmbuf, NULL);

/* Execute stuff */

projectm_eval_context_destroy(ctx);
projectm_eval_memory_buffer_destroy(gmbuf);
```

Note that when creating the buffer, it doesn't allocate any actual buffer memory. Memory is reserved in small blocks
whenever an index is accessed for which no buffer exists yet.

### Custom reg Variables

Similar to the memory buffer, the reg variables need to be allocated and a pointer passed to the context upon creation.
In contrast to the buffer, the reg variables are stored in a simple array of type PRJM_EVAL_F with the size of 100
entries:

```c
PRJM_EVAL_F reg_vars[100] = {0};
struct projectm_eval_context* ctx = projectm_eval_context_create(NULL, &reg_vars);

/* Execute stuff */

projectm_eval_context_destroy(ctx);
```

## Multi-Threading Considerations

In general, executing code in the same context in multiple threads at the same time is generally unsupported as the
variables are shared between contexts. This also holds true for contexts sharing reg variables - while it may work,
multiple scripts changing reg vars in different threads may have negative impact on the other instances.

The above stays true for gmegabuf as well, but allocating memory makes things a bit more complicated. If there is a real
requirement of using the same gmegabuf in multiple threads at once, this can be done by implementing
the `projectm_eval_memory_host_lock_mutex()` and `projectm_eval_memory_host_unlock_mutex()` functions to set and release
proper locks. A non-recursive mutex is sufficient.

Note that using a mutex will prevent race conditions and memory loss (e.g. two thread trying to allocate the same memory
area), but it won't change the unpredictable behaviour of values changing unexpectedly.

As noted in the quick-start guide, an application using projectM-Eval is _required_ to implement the above functions. If
no locking is needed, they can be empty stubs.

---

--- METRICS ---
- Files Extracted: 8
- Estimated Token Budget: ~38163 tokens
- Canonical Reference: https://codewiki.google/github.com/clementine-player/Clementine