# Technical Documentation: Yooooomi/your_spotify > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/Yooooomi/your_spotify) ### Overview (`README.md`) [](https://www.paypal.com/donate/?hosted_button_id=BLAPT49PK9A8G)

# Your Spotify **YourSpotify** is a self-hosted application that tracks what you listen and offers you a dashboard to explore statistics about it! It's composed of a web server which polls the Spotify API every now and then and a web application on which you can explore your statistics. # Table of contents - [Prerequisites](#prerequisites) - [Installation](#installation) - [Using docker](#using-docker-compose) - [Installing locally](#installing-locally-not-recommended) - [Environment](#environment) - [Advanced CORS settings](#advanced-cors-settings) - [Creating the Spotify application](#creating-the-spotify-application) - [Importing past history](#importing-past-history) - [Supported import methods](#supported-import-methods) - [Privacy data](#privacy-data) - [Full privacy data](#full-privacy-data-recommended) - [Troubleshoot](#troubleshoot) - [FAQ](#faq) - [External guides](#external-guides) - [Contributing](#contributing) - [Sponsoring](#sponsoring) # Prerequisites 1. You have to own a Spotify application ID that you can create through their [dashboard](https://developer.spotify.com/dashboard/applications). 2. You need to provide the **Server** environment the **public** AND **secret** key of the application (cf. [Installation](#installation)). 3. You need to provide an **authorized** redirect URI to the `docker-compose` file. > A tutorial is available at the end of this readme. # Installation ## Using `docker-compose` Follow the [docker-compose-example.yml](https://github.com/Yooooomi/your_spotify/blob/master/docker-compose-example.yml) to host your application through docker. ```yml services: server: image: yooooomi/your_spotify_server restart: always ports: - "8080:8080" links: - mongo depends_on: - mongo environment: API_ENDPOINT: http://localhost:8080 # This MUST be included as a valid URL in the spotify dashboard (see below) CLIENT_ENDPOINT: http://localhost:3000 SPOTIFY_PUBLIC: __your_spotify_client_id__ SPOTIFY_SECRET: __your_spotify_secret__ web: image: yooooomi/your_spotify_client restart: always ports: - "3000:3000" environment: API_ENDPOINT: http://localhost:8080 mongo: container_name: mongo image: mongo:8 volumes: - ./your_spotify_db:/data/db ``` ## Installing locally (not recommended) You can follow the instructions [here](https://github.com/Yooooomi/your_spotify/blob/master/LOCAL_INSTALL.md). Note that you will still have to do the steps below. ## Environment | Key | Default value (if any) | Description | | :--- | :--- | :--- | | CLIENT_ENDPOINT | REQUIRED | The endpoint of your web application | | API_ENDPOINT | REQUIRED | The endpoint of your server | | SPOTIFY_PUBLIC | REQUIRED | The public key of your Spotify application (cf [Creating the Spotify Application](#creating-the-spotify-application)) | | SPOTIFY_SECRET | REQUIRED | The secret key of your Spotify application (cf [Creating the Spotify Application](#creating-the-spotify-application)) | | TIMEZONE | Europe/Paris | The timezone of your stats, only affects read requests since data is saved with UTC time | | MONGO_ENDPOINT | mongodb://mongo:27017/your_spotify | The endpoint of the Mongo database, where **mongo** is the name of your service in the compose file | | PROMETHEUS_USERNAME | _not defined_ | Prometheus basic auth username (see [here](https://github.com/Yooooomi/your_spotify/tree/master/apps/server#prometheus)) | | PROMETHEUS_PASSWORD | _not defined_ | Prometheus basic auth password | | LOG_LEVEL | info | The log level, debug is useful if you encouter any bugs | | CORS | _not defined_ | List of comma-separated origin allowed (not required; defaults to CLIENT_ENDPOINT) | | COOKIE_VALIDITY_MS | 1h | Validity time of the authentication cookie, following [this pattern](https://github.com/vercel/ms) | | MAX_IMPORT_CACHE_SIZE | Infinite | The maximum element in the cache when importing data from an outside source, more cache means less requests to Spotify, resulting in faster imports | | MONGO_NO_ADMIN_RIGHTS | false | Do not ask for admin right on the Mongo database | | PORT | 8080 | The port of the server, **do not** modify if you're using docker | | FRAME_ANCESTORS | _not defined_ | Sites allowed to frame the website, comma separated list of URLs (`i-want-a-security-vulnerability-and-want-to-allow-all-frame-ancestors` to allow every website) | ## Advanced CORS settings **Manually specifying CORS configuration is not required for typical deployments.** 99.9% of users do not need to worry about this, it is handled automatically. If your use case requires the backend to be used from multiple frontend origins, you can manually adjust the `CORS` variable. For example, a value of `origin1,origin2` will allow `origin1` and `origin2`. # Creating the Spotify Application For **YourSpotify** to work you need to provide a Spotify application **public** AND **secret** to the server environment. To do so, you need to create a **Spotify application** [here](https://developer.spotify.com/dashboard/applications). 1. Click on **Create app**. 2. Fill out all the information. 3. Set the redirect URI, corresponding to your **server** location on the internet (or your local network) adding the suffix **/oauth/spotify/callback** (**/api/oauth/spotify/callback** if using the [linuxserver](https://github.com/linuxserver/docker-your_spotify) image). - i.e: `http://localhost:8080/oauth/spotify/callback` or `http://home.mydomain.com/your_spotify_backend/oauth/spotify/callback` 4. Check **Web API** 5. Check **I understand and agree** 6. Hit **Settings** at the top right corner 7. Copy the **public** and the **secret** key into your `docker-compose` file under the name of `SPOTIFY_PUBLIC` and `SPOTIFY_SECRET` respectively. 8. Once you have created your application, Spotify wants you to register the users that will be able to access the application. (You don't need to do that for the account that created the application) 1. Click the **User Management** button 2. Enter the required information, a name and the email the user's Spotify account has been created with. 3. (Optional) You can **Request extension** if you do not want to register the users by hand. # Importing past history By default, **YourSpotify** will only retrieve data for the past 24 hours once registered. This is a technical limitation. However, you can import previous data by two ways. The import process uses cache to limit requests to the Spotify API. By default, the cache size is unlimited, but you can limit is with the `MAX_IMPORT_CACHE_SIZE` env variable in the **server**. ## Supported import methods ### Privacy data > Takes a maximum of 5 days. > Only gets you the last year of history. - Request your **privacy data** at Spotify to have access to your history for the past year [here](https://www.spotify.com/us/account/privacy/). - Head to the **Settings** page and choose the **Account data** method. - Input your files starting with `StreamingHistoryX.json`. - Start your import. ### Full privacy data (recommended) > Takes a maximum of 30 days. > Gets you the whole history since the creation of your account. - Request your **Full privacy data** to have access to your history data since the creation of the account [here](https://www.spotify.com/us/account/privacy/). - Head to the **Settings** page and choose the **Extended streaming history** method. - Input your files starting with `Streaming_History_Audio_YYYY-YYYY_X.json`. - Start your import. ## Troubleshoot An import can fail: - If the server reboots. - If a request fails 10 times in a row. A failed import can be retried in the **Settings** page. Be sure to clean your failed imports if you do not want to retry it as it will remove the files used for it. It is safer to import data at account creation. Though **YourSpotify** detects duplicates, some may still be inserted. # FAQ > How can I block new registrations? From an admin account, go to the **Settings** page and hit the **Disable new registrations** button. > Songs don't seem to synchronize anymore. This can happen if you revoked access on your Spotify account. To re-sync the songs, go to settings and hit the **Relog to Spotify** button. > The web application is telling me it cannot retrieve global preferences. This means that your web application can't connect to the backend. Check that your **API_ENDPOINT** env variable is reachable from the device you're using the platform from. > A specific user does not use the application in the same timezone as the server, how can I set a specific timezone for him? Any user can set his proper timezone in the settings, it will be used for any computed statistics. The timezone of the device will be used for everything else, such as song history. # External guides - [BreadNet](https://breadnet.co.uk/your-spotify-2022) installation tutorial # Contributing If you have any issue or any idea that could make the project better, feel free to open an [issue](https://github.com/Yooooomi/your_spotify/issues/new/choose). I'd love to hear about new ideas or bugs you are encountering. # Sponsoring I work on this project on my spare time and try to fix issues as soon as I can. If you feel generous and think this project and my investment are worth a few cents, you can consider sponsoring it with the button on the right, many thanks. --- ### Overview (`apps/server/README.md`) # Your Spotify server ## Table of Contents - [Prometheus](#prometheus) - [API](#api) - [Authentication](#authentication) - [OAuth](#oauth) - [User Management](#user-management) - [Account](#account) - [Admin](#admin) - [Spotify Integration](#spotify-integration) - [Playback](#playback) - [Playlist Management](#playlist-management) - [History & Statistics](#history--statistics) - [Top Items](#top-items) - [Collaborative Features](#collaborative-features) - [Search](#search) - [Artist](#artist) - [Album](#album) - [Track](#track) - [Import](#import) - [Global Preferences](#global-preferences) - [Account Settings](#account-settings) ## Prometheus You can access various prometheus metrics through `/metrics`: - **http_requests_total**: Total number of HTTP requests - **http_request_duration_nanoseconds**: Duration of HTTP requests in nanoseconds - **imports_total**: Total number of imports - **ingested_tracks_total**: Total number of ingested tracks from Spotify API - **ingested_albums_total**: Total number of ingested albums from Spotify API - **ingested_artists_total**: Total number of ingested artists from Spotify API You will need to define to environment variable in the server: - **PROMETHEUS_USERNAME**: the basic auth username expected by the server to authorize the request - **PROMETHEUS_PASSWORD**: the expected password Those values are to be referenced in the configuration of your Prometheus instance: ```yml scrape_configs: - job_name: "your_spotify" basic_auth: username: "myuser" # PROMETHEUS_USERNAME password: "mypassword" # PROMETHEUS_PASSWORD static_configs: - targets: ["example.com:443"] ``` ## API ## Authentication ### OAuth #### `GET /oauth/spotify` Initiates Spotify OAuth flow. **Response:** - Redirects to Spotify login page - For development: Returns 204 if offline mode enabled #### `GET /oauth/spotify/callback` Handles Spotify OAuth callback. **Query Parameters:** - `code`: string - OAuth code from Spotify - `state`: string - State parameter for security **Response:** - Redirects to client application #### `GET /oauth/spotify/me` Gets current user's Spotify profile. **Response:** - `200`: Spotify user profile - `500`: Error with code "SPOTIFY_ERROR" ## User Management ### Account #### `GET /me` Get current user information. **Response:** ```json { "status": boolean, "user": UserObject // Only if status is true } ``` #### `GET /accounts` Get list of all users (requires authentication). **Response:** ```json [{ "id": string, "username": string, "admin": boolean, "firstListenedAt": Date }] ``` #### `PUT /rename` Change username. **Body:** ```json { "newName": string // 2-64 characters } ``` **Response:** - `204`: Success - `500`: Error ### Admin #### `PUT /admin/:id` Set user admin status (requires admin). **Parameters:** - `id`: User ID **Body:** ```json { "status": boolean } ``` **Response:** - `204`: Success - `400`: Error if trying to remove last admin - `500`: Error #### `DELETE /account/:id` Delete user account (requires admin). **Parameters:** - `id`: User ID **Response:** - `204`: Success - `404`: User not found - `500`: Error ## Spotify Integration ### Playback #### `POST /spotify/play` Start playback of a track. **Body:** ```json { "id": string // Spotify track ID } ``` **Response:** - `200`: Success - `400`: Invalid track or playback error - `500`: Server error ### Playlist Management #### `GET /spotify/playlists` Get user's Spotify playlists. **Response:** - `200`: Array of user's Spotify playlists #### `POST /spotify/playlist/create` Create a new Spotify playlist or add tracks to existing playlist. **Body:** ```json { "playlistId": string?, // Existing playlist ID (optional) "name": string?, // New playlist name (optional) "sortKey": string?, // Sort key for tracks (default "count") "type": "top" | "affinity" | "single", // Playlist type // For type "top" "interval": { "start": date, "end": date }, "nb": number, // Number of tracks to include // For type "affinity" "interval": { "start": date, "end": date }, "nb": number, // Number of tracks to include "userIds": string[], // User IDs for collaboration "mode": "intersection" | "union", // Collaboration mode // For type "single" "songId": string // Single track ID to add } ``` **Response:** - `204`: Successfully created or updated playlist - `400`: Invalid request - `500`: Server error ## History & Statistics #### `GET /spotify/gethistory` Get user's listening history. **Query Parameters:** - `number`: number - Maximum number of items to return (max 20) - `offset`: number - Number of items to skip - `start`: date (optional) - Start date - `end`: date (optional) - End date **Response:** - `200`: Array of listening history items #### `GET /spotify/listened_to` Get count of songs listened to in a time period. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) **Response:** - `200`: `{ count: number }` #### `GET /spotify/most_listened` Get most listened songs in a time period. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Array of most listened songs with counts #### `GET /spotify/most_listened_artist` Get most listened artists in a time period. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Array of most listened artists with counts #### `GET /spotify/songs_per` Get song count per time unit. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Array of time periods with song counts #### `GET /spotify/time_per` Get listening time per time unit. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Array of time periods with listening durations #### `GET /spotify/album_date_ratio` Get album release date distribution for listened tracks. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Distribution of tracks by album release date #### `GET /spotify/feat_ratio` Get ratio of songs featuring multiple artists. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Ratio data for featured vs. non-featured tracks #### `GET /spotify/different_artists_per` Get count of unique artists listened to per time unit. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Count of unique artists per time unit #### `GET /spotify/time_per_hour_of_day` Get listening distribution by hour of day. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) **Response:** - `200`: Distribution of listening time by hour #### `GET /spotify/best_artists_per` Get top artists per time unit. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) **Response:** - `200`: Top artists for each time unit ### Top Items #### `GET /spotify/top/songs` Get top songs in a time period. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `nb`: number - Number of items to return (1-30) - `offset`: number - Number of items to skip (default 0) - `sortKey`: string - Sort criteria (default "count") **Response:** - `200`: Array of top song objects #### `GET /spotify/top/artists` Get top artists in a time period. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `nb`: number - Number of items to return (1-30) - `offset`: number - Number of items to skip (default 0) - `sortKey`: string - Sort criteria (default "count") **Response:** - `200`: Array of top artist objects #### `GET /spotify/top/albums` Get top albums in a time period. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `nb`: number - Number of items to return (1-30) - `offset`: number - Number of items to skip (default 0) - `sortKey`: string - Sort criteria (default "count") **Response:** - `200`: Array of top album objects #### `GET /spotify/top/hour-repartition/songs` Get top songs by hour of day. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) **Response:** - `200`: Top songs for each hour of the day #### `GET /spotify/top/hour-repartition/albums` Get top albums by hour of day. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) **Response:** - `200`: Top albums for each hour of the day #### `GET /spotify/top/hour-repartition/artists` Get top artists by hour of day. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) **Response:** - `200`: Top artists for each hour of the day #### `GET /spotify/top/sessions` Get longest listening sessions. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) **Response:** - `200`: Array of listening session data ### Collaborative Features #### `GET /spotify/collaborative/top/songs` Get shared top songs between multiple users. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) - `otherIds`: string[] - Array of other user IDs to compare with - `mode`: string - Collaboration mode (intersection, union) **Response:** - `200`: Array of shared top songs #### `GET /spotify/collaborative/top/albums` Get shared top albums between multiple users. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) - `otherIds`: string[] - Array of other user IDs to compare with - `mode`: string - Collaboration mode (intersection, union) **Response:** - `200`: Array of shared top albums #### `GET /spotify/collaborative/top/artists` Get shared top artists between multiple users. **Query Parameters:** - `start`: date - Start date - `end`: date - End date (defaults to current time) - `timeSplit`: string - Time unit for grouping (day, week, month, year) - `otherIds`: string[] - Array of other user IDs to compare with - `mode`: string - Collaboration mode (intersection, union) **Response:** - `200`: Array of shared top artists ## Search #### `GET /search/:query` Search across tracks, artists and albums. **Parameters:** - `query`: Search term (3-64 characters) **Response:** ```json { "artists": Artist[], "tracks": Track[], "albums": Album[] } ``` ## Artist #### `GET /artist/:ids` Get artist details. **Parameters:** - `ids`: Comma-separated artist IDs #### `GET /artist/:id/stats` Get artist statistics. **Parameters:** - `id`: Artist ID **Response:** ```json { "artist": ArtistObject, "firstLast": FirstLastObject, "mostListened": TrackObject, "albumMostListened": AlbumObject, "bestPeriod": PeriodStats, "total": TotalStats, "dayRepartition": DayStats } ``` #### `GET /artist/search/:query` Search for artists by name. **Parameters:** - `query`: Search term (3-64 characters) **Response:** - `200`: Array of matching artists #### `POST /artist/blacklist/:id` Blacklist an artist. #### `POST /artist/unblacklist/:id` Remove artist from blacklist. #### `GET /artist/:id/rank` Get artist's ranking among all artists. **Parameters:** - `id`: Artist ID **Response:** - Artist ranking information ## Album #### `GET /album/:ids` Get album details. **Parameters:** - `ids`: Comma-separated album IDs #### `GET /album/:id/stats` Get album statistics. **Parameters:** - `id`: Album ID **Response:** ```json { "album": AlbumObject, "artists": Artist[], "firstLast": FirstLastObject, "tracks": Track[] } ``` #### `GET /album/:id/rank` Get album's ranking among all albums. **Parameters:** - `id`: Album ID **Response:** - Album ranking information ## Track #### `GET /track/:ids` Get track details. **Parameters:** - `ids`: Comma-separated track IDs #### `GET /track/:id/stats` Get track statistics. **Parameters:** - `id`: Track ID **Response:** ```json { "track": TrackObject, "artist": ArtistObject, "album": AlbumObject, "bestPeriod": PeriodStats, "firstLast": FirstLastObject, "recentHistory": HistoryObject, "total": TotalStats } ``` #### `GET /track/:id/rank` Get track's ranking among all tracks. **Parameters:** - `id`: Track ID **Response:** - Track ranking information ## Import #### `POST /import/privacy` Import privacy data. **Body:** - Multipart form with files named "imports" **Response:** ```json { "code": "IMPORT_STARTED" | "IMPORT_INIT_FAILED" } ``` #### `POST /import/retry` Retry failed import. **Body:** ```json { "existingStateId": string } ``` #### `DELETE /import/clean/:id` Clean up import. **Parameters:** - `id`: Import ID #### `GET /imports` Get import status. ## Global Preferences #### `GET /global/preferences` Get global application preferences. #### `POST /global/preferences` Update global preferences (admin only). **Body:** ```json { "allowRegistrations": boolean } ``` ## Account Settings #### `POST /settings` Update user settings. **Body:** ```json { "historyLine": boolean?, "preferredStatsPeriod": "day" | "week" | "month" | "year"?, "nbElements": number?, // 5-50 "metricUsed": "number" | "duration"?, "darkMode": "follow" | "dark" | "light"?, "timezone": string?, "dateFormat": string? } ``` #### `POST /generate-public-token` Generate new public access token. #### `POST /delete-public-token` Delete public access token. --- --- METRICS --- - Files Extracted: 2 - Estimated Token Budget: ~6220 tokens - Canonical Reference: https://codewiki.google/github.com/Yooooomi/your_spotify