data-source Documentation

1. Project Overview & Quickstart (gorse-io/data-source) Copy Chapter

data-source

Open-source repository gorse-io/data-source

Repository Details Copy Section

• Repository: gorse-io/data-source
• Primary Language: Code

Note: High-volume repository documentation is actively indexed and synchronized by YakaAI.

2. Official Technical Reference & Guides (gorse-io/docs) Copy Chapter

File: README.md Copy Chapter

Gorse Docs

Official documents for Gorse recommender system.

Setup environment Copy Chapter

This is a VuePress project, so you need to install:

• Node.js
• Yarn

Install dependencies and run Copy Chapter

# Clone the repository
git clone [email protected]:gorse-io/docs.git

# Change directory
cd docs

# Install dependencies
pnpm install

# Run Gorse Docs on http://localhost:8080
pnpm docs:dev

Deployment Copy Chapter

Gorse Docs is deployed as a static website. The main branch is deployed at https://gorse.io.

You can build the static website like this:

pnpm docs:build

The static files will be available in .vuepress/dist.

Acknowledgments Copy Chapter

• The VuePress version of documents is contributed by @andresribeiro.
• The theme of VuePress is vuepress-theme-hope.

File: src/docs/concepts/recommenders/collaborative.md Copy Chapter

icon: collaborate_sider shortTitle: Collaborative Copy Chapter

Collaborative Filtering

Gorse uses matrix factorization[^3] as the collaborative filtering algorithm to provide personalized recommendations. Matrix factorization is a widely used technique in recommender systems that models user-item interactions by representing both users and items in a shared latent space.

Matrix Factorization Copy Chapter

In matrix factorization, items and users are represented by vectors. The probability that a user $u$ likes an item $i$ is predicted by the dot product of two vectors.

$$
\hat y_{ui}=\mathbf{p}_u^T \mathbf{q}_i
$$

where $\mathbf{p}_u$ is the embedding vector of the user $u$, and $\mathbf{q}_i$ is the embedding vector of the item $i$. The model of matrix factorization is simple, but there is more than one training algorithm.

BPR

BPR[^1] (Bayesian Personalized Ranking) is a pairwise training algorithm. The training data for BPR consist of a set of triples:

$$
D_s={(u,i,j)|i\in I_u\wedge j\in I \setminus I_u}
$$

The semantics of $(u, i, j) \in D_S$ is that user $u$ is assumed to prefer $i$ over $j$. The negative cases are regarded implicitly.

The Bayesian formulation of finding the correct personalized ranking for all items is to maximize the following posterior probability where $\Theta$ represents the parameter vectors of the matrix factorization model

$$
p(\Theta|>_u) \propto p(>_u|\Theta)p(\Theta)
$$

where $>_u$ is the desired but latent preference
for user $u$. All users are presumed to act independently of each other. BPR also assumes the ordering of each pair of items $(i, j)$ for a specific user is independent of the ordering of every other pair. Hence, the above user-specific likelihood function $p(>_u|\Theta)$ can first be rewritten as a product of single densities and second be combined for all users $u \in U$.

$$
\prod_{u\in U}p(>u|\Theta)=\prod{(u,i,j)\in D_s}p(i>_u j|\Theta)
$$

where $p(i>u j|\Theta)=\sigma(\hat y{uij})$ and $\hat y_{uij}=\hat y_{ui} - \hat y_{uj}$.

For the parameter vectors, BPR introduces a general prior density $p(\Theta)$ which is a normal distribution with zero mean and variance-covariance matrix $\Sigma_\Theta$.

$$
p(\Theta) \sim N(0,\Sigma_\Theta)
$$

where $Σ_Θ = λ_ΘI$. Then the optimization criterion for personalized ranking is

$$
\begin{split}
\text{BPR-OPT}&=\ln p(\Theta|>u)\
&=\ln p(>u|\Theta)p(\Theta)\
&=\ln\prod{(u,i,j)\in D_s}\sigma(\hat y{uij})p(\Theta)\
&=\sum_{(u,i,j)\in D_s}\ln \sigma(\hat y_{uij})+\ln p(\Theta)\
&=\sum_{(u,i,j)\in D_s}\ln \sigma(\hat y_{uij})-\lambda_\Theta|\Theta|^2
\end{split}
$$

The gradient of BPR-Opt with respect to
the model parameters is:

$$
\begin{split}
\frac{\partial\text{BPR-OPT}}{\partial\Theta}&=\sum_{(u,i,j)\in D_s}\frac{\partial}{\partial\Theta}\ln\sigma(\hat x_{uij})-\lambda_\Theta\frac{\partial}{\partial\Theta}|\Theta|^2\
&\propto\sum_{(u,i,j)\in D_s}\frac{-e^{-\hat y_{uij}}}{1+e^{-\hat y_{uij}}}\cdot \frac{\partial}{\partial\Theta}\hat y_{uij}-\lambda_\Theta\Theta\
\end{split}
$$

A stochastic gradient-descent algorithm
based on a bootstrap sampling of training triples is as follows:

• initialize $\Theta$
  • repeat
    • draw $(u,i,j)$ from $D_s$
    • $\Theta\leftarrow\Theta+\alpha\left(\frac{e^{-\hat y_{uij}}}{1+e^{-\hat y_{uij}}}\cdot \frac{\partial}{\partial\Theta}\hat y_{uij}+\lambda_\Theta\Theta\right)$
  • util convergence
• return $\Theta$

where $\alpha$ . The derivatives from embedding vectors are

$$
\frac{\partial}{\partial\theta}\hat y_{uij}=\begin{cases}
(q_{if}-q_{jf})&\text{if }\theta=p_{uf}\
p_uf&\text{if }\theta=q_if\
-p_uf&\text{if }\theta=q_{jf}\
0&\text{else}
\end{cases}
$$

eALS

eALS[^2] is a point-wise training algorithm. For a pair of a user $u$ and an item $i$, the ground truth for training is

$$
y_{ui}=\begin{cases}
1&i\in I_u\
0&i\notin I_u
\end{cases}
$$

Embedding vectors are optimized by minimizing the following cost function[^8]:

$$
\mathcal{C} = \sum_{u\in U}\sum_{i \in I}w_{ui}(y_{ui}-\hat{y}{ui}^f)^2 + \lambda\left(\sum{u \in U}|\mathcal{p}|^2+\sum_{i \in I}|\mathbf{q}_i|^2\right)
$$

where $\hat{y}{ui}^f=\hat{y}{ui}-p_{uf}q_{if}$ and $w_{ui}$ the weight of feedback

$$
w_{ui}=\begin{cases}
1&i\in I_u\
\alpha&i\notin I_u
\end{cases}
$$

$\alpha$ ($\alpha < 1$) is the weight for negative feedbacks. The derivative of the objective function with respect to $p_{uf}$ is

$$
\frac{\partial J}{\partial p_{uf}}=-2\sum_{i\in I}(y_{ui}-\hat y_{ui}^f)w_{ui}q_{uf} + 2p_{uf}\sum_{i\in I}w_{ui}q_{if}^2 + 2\lambda p_{uf}
$$

By setting this derivative to 0, obtain the solution of $p_{uf}$ (Eq 1):

$$
\begin{split}
p_{uf} &= \frac{\sum_{i \in I}(y_{ui}-\hat y_{ui}^f)w_{ui}q_{if}}{\sum_{i \in I}w_{ui}q^2_{if}+\lambda}\
&=\frac{\sum_{i\in I_u}(y_{ui}-\hat{y}{ui}^f)q{if}-\sum_{i\in I_u}\hat{y}{ui}^f\alpha q{if}}{\sum_{i\in I_u}q^2_{if}+\sum_{i \in I_u}\alpha q_{if}^2+\lambda}\
&=\frac{\sum_{i\in I_u}[y_{ui}-(1-\alpha)\hat{y}{ui}^f]q{if}-\sum_{k\neq f}p_{uk}s^q_{kf}}{\sum_{i\in I_u}(1-\alpha)q^2_{if}+\alpha s^q_{ff}+\lambda}
\end{split}
$$

where $s^q_{kf}$ denotes the $(k, f)^\text{th}$ element of the $\mathbf{S}^q$ cache, defined as $\mathbf{S}^q = \mathbf{Q}^T\mathbf{Q}$. Similarly, get the solver for an item embedding vector (Eq 2):

$$
q_{if} = \frac{\sum_{u \in U_i}(y_{ui}-(1-\alpha)\hat y_{ui})p_{uf}-\alpha\sum_{k\neq f}q_{ik}s^p_{kf}}{\sum_{u \in U_i}(1-\alpha)p^2_{uf} + \alpha s^p_{ff} + \lambda}
$$

where $s^p_{kf}$ denotes the $(k, f)^\text{th}$ element of the $\mathbf{S}^p$ cache, defined as $\mathbf{S}^p = \mathbf{P}^T\mathbf{P}$. The learning algorithm is summarized as

• Randomly initialize $\mathbf{P}$ and $\mathbf{Q}$
• for $(u, i)\in R$ do $\hat{y}_{ui}=\mathbf{p}_u^T\mathbf{q}_i$
• while Stopping criteria is not met do
  • $\mathbf{S}^q = \mathbf{Q}^T\mathbf{Q}$
  • for $u \leftarrow 1$ to $M$ do
    • for $f \leftarrow 1$ to $K$ do
      • for $i \in I_u$ do $\hat{y}{ui}\leftarrow\hat{y}{ui}^f-p_{uf}q_{if}$
      • $p_{uf}\leftarrow$ Eq 1
      • for $i \in I_u$ do $\hat{y}{ui}\leftarrow\hat{y}{ui}^f+p_{uf}q_{if}$
    • end
  • end
  • $\mathbf{S}^p=\mathbf{P}^T\mathbf{P}$
  • for $i \leftarrow 1$ to $N$ do
    • for $f \leftarrow 1$ to $K$ do
      • for $u \in U_i$ do $\hat{y}{ui}\leftarrow\hat{y}{ui}^f-p_{uf}q_{if}$
      • $q_{if}\leftarrow$ Eq 2
      • for $u \in U_i$ do $\hat{y}{ui}\leftarrow\hat{y}{ui}^f+p_{uf}q_{if}$
    • end
  • end
• return $\mathbf{P}$ and $\mathbf{Q}$

Configuration Copy Chapter

The configurations for collaborative filtering are all about model fitting and hyperparameter optimization.

• type is the type of collaborative filtering algorithm. Defaults to none.
  • none: Disable collaborative filtering.
  • mf: Matrix factorization.
• fit_period is the period for model fitting. Defaults to 60m.
• fit_epoch is the number of epochs for model fitting. Defaults to 100.
• optimize_period is the period for hyperparameter optimization. Defaults to 180m.
• optimize_trials is the number of trials for hyperparameter optimization. Defaults to 10.
• early_stopping.patience is the number of epochs with no improvement after which training will be stopped. Defaults to 10.

Example Copy Chapter

In the demo project GitRec, the collaborative filtering is configured as follows:

[recommend.collaborative]
type = "mf"
fit_period = "60m"
fit_epoch = 100
optimize_period = "360m"
optimize_trials = 10
[recommend.collaborative.early_stopping]
patience = 10

[^1]: Rendle, Steffen, et al. "BPR: Bayesian personalized ranking from implicit feedback." Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence. 2009.

[^2]: He, Xiangnan, et al. "Fast matrix factorization for online recommendation with implicit feedback." Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. 2016.

[^3]: Funk, Simon. "Netflix Update: Try This at Home." The Sifter. December 11, 2006. https://sifter.org/~simon/journal/20061211.html.

File: src/docs/concepts/recommenders/external.md Copy Chapter

icon: code shortTitle: External API Copy Chapter

External API Recommenders

Although Gorse already offers several recommenders, the recommendation logic in production is something Gorse cannot fully encompass. For example, YouTube adds new videos from followed creators to recommendation, and Steam adds DLCs for games a user owns to recommendation. The external recommender executes a JavaScript script to access an external HTTP endpoint and retrieve recommendation.

Configuration Copy Chapter

An external recommender has two fields:

• name is used to uniquely identify an external recommender.
• script is used to access the external HTTP endpoint and convert the result into an array of item IDs. Item IDs that do not exist in the recommender system will be ignored.

Gorse integrates QuickJS to execute the JavaScript. The ability to send HTTP requests has been added. Contributions to add more capabilities are welcome.

Fetch API Copy Section

To make a GET request, you can pass a URL string to the fetch function:

const response = fetch('https://example.com/data');
const data = JSON.parse(response.body);

To make a POST request, you can pass an object containing the request options as the second argument to the fetch function:

const response = fetch('https://example.com/api', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    key1: 'value1',
    key2: 'value2'
  })
});
const result = JSON.parse(response.body);

The fetch function accepts the following request options:

• method: The HTTP method, e.g., GET, POST, PUT, DELETE. Defaults to GET.
• headers: An object containing HTTP headers.
• body: The request body, which should be a string.
• url: The request URL. If this option is specified, the first argument can be omitted.

The fetch function returns a response object with the following properties:

• ok: A boolean value indicating whether the request was successful (status code in 200-299).
• status: The HTTP status code.
• statusText: The HTTP status text.
• body: The response body, which is a string.
• headers: An object containing response headers.

Here is a complete example showing how to use the Fetch API to make a POST request and handle the response:

const response = fetch('https://example.com/api', {
    method: 'POST',
    headers: {
        'Content-Type': 'application/json'
    },
    body: JSON.stringify({
        key1: 'value1',
        key2: 'value2'
    })
});
if (!response.ok) {
    throw new Error(`Request failed with status code: ${response.status}`);
}
JSON.parse(response.body)

::: warning

• The Fetch API executes synchronously within the JavaScript script. This means the script will be blocked until the request is completed.
• The Fetch API does not support all features of the Fetch API in web browsers, such as streaming responses.

:::

Example Copy Chapter

The following is an example from GitRec fetching repositories from GitHub Trending:

[[recommend.external]]
name = "trending"
script = """
const body = fetch("https://cdn.jsdelivr.net/gh/isboyjc/github-trending-api/data/daily/all.json").body;
const data = JSON.parse(body);
data["items"].map((item) => {
  return item["title"].toLowerCase().replace("/", ":");
})
"""

File: src/docs/concepts/recommenders/item-to-item.md Copy Chapter

icon: boxes shortTitle: Item-to-Item Copy Chapter

Item-to-Item Recommenders

Recommending similar items based on user preferred items is a simple and effective recommendation strategy. Gorse has implemented item-to-item recommenders that supports embedding similarity, tags similarity and users similarity.

Configuration Copy Chapter

The item-to-item recommenders need to be explicitly configured. The following three fields need to be filled in:

• name is the name of the recommender.
• type is the similarity type, and the following values are supported:
  • embedding is the Euclidean distance between embedding vectors.
  • tags is the similarity based on the number of common tags.
  • users is the similarity based on the number of common users.
• column is the field used by the recommender to calculate the similarity, expressed in the Expr language. Suppose the embedding vectors of the README of GitHub repositories are stored in the embedding field of Labels, and the tags are stored in the topics field:

{
    "ItemId": "gorse-io:gorse",
    "IsHidden": false,
    "Categories": [],
    "Timestamp": "2022-10-23T03:50:24Z",
    "Labels": {
        "embedding": [0.0017246103, -0.009725488, 0.005806058, -0.0187753, -0.015343021, ...],
        "topics": ["machine-learning", "service", "recommender", "go", "recommender-system", "knn", "collaborative-filtering"]
    },
    "Comment": "An open source recommender system service written in Go"
}

If embedding similarity is used, the value of column should be item.Labels.embedding; if tags similarity is used, the value of column should be item.Labels.topics. When type is embedding or tags, column cannot be empty. However, when type is users, column must be empty.

Algorithms Copy Chapter

Embedding Similarity Copy Section

There are $K$-dimensional embedding vectors $p$ and $q$ for two items respectively, the embedding similarity between them is

$$
\sqrt{\sum^K_{k=1}(p_k-q_k)^2}
$$

The dimension of embedding vectors is usually relatively large. For example, the text-embedding-3-small model of OpenAI generates embedding vectors with 1536 dimensions. Using embedding vectors requires more disk space and memory compared to tags. Compared with tags similarity, encoding text and images into embedding vectors can eliminate the cost of manual tags maintenance or automatic tags generation, and the similarity calculated by embedding vectors is more accurate. Embedding vectors can be generated from APIs provided by AI providers such as OpenAI and Anthropic, or self-deployed projects like Ollama and CLIP-as-service.

Tags Similarity Copy Section

Tags similarity holds that the more overlapping tags there are between items, the more similar they are. First, calculate the TF-IDF of each tag

$$
w_l=-\log\left(\frac{|I|}{|I_l|}\right)
$$

where $|I|$ represents the total number of items, and $I_l$ represents the number of items labeled with tag $l$. If a tag is used by more items, it is more general and thus has a lower weight. Then, calculate the tags similarity between items

$$
\frac{\sum_{l\in L_i \cap L_j}w_l}{\sqrt{\sum_{l\in L_i}w_l^2}\sqrt{\sum_{l\in L_j}w_l^2}}
$$

where $L_i$ and $L_j$ represent the tag sets of item $i$ and item $j$ respectively. Although tag similarity is not as accurate as embedding similarity, it can help users discover more content that has similarities but also differences.

Users Similarity Copy Section

Users similarity believes that the more overlapping users there are between items, the more similar they are. Similarly, first, it is necessary to calculate the TF-IDF of each user

$$
w_u = -\log\left(\frac{|I|}{|I_u|}\right)
$$

where $|I|$ represents the total number of items, and $|I_u|$ represents the number of items liked by user $u$. Then, calculate the user similarity between items

$$
\frac{\sum_{u\in U_i \cap U_j}w_u}{\sqrt{\sum_{u\in U_i}w_u^2}\sqrt{\sum_{u\in U_j}w_u^2}}
$$

where $U_i$ and $U_j$ represent the user sets of item $i$ and item $j$ respectively. One drawback of users similarity is that it tends to recommend popular items because popular items always have a high overlap of users with other items.

Recommendation Copy Section

The item-to-item recommender first retrieves the user's positive feedback, then finds similar items for each of the items with positive feedback, and finally sums up the similarity scores of the same item to get the final recommendation score:

$$
\sum_{j \in F_u} s_{ij}
$$

where $F_u$ represents the set of items with positive feedback from user $u$, and $s_{ij}$ represents the similarity between item $i$ and item $j$.

API Copy Chapter

You can access similar items through the following API:

curl http://localhost:8087/api/item-to-item/<name>/

Examples Copy Chapter

Take the dataset of the demo project GitRec as an example. The embedding vectors of the README file of GitHub projects are stored in the embedding field, and the tags are saved in the topics field.

{
    "ItemId": "gorse-io:gorse",
    "IsHidden": false,
    "Categories": [],
    "Timestamp": "2022-10-23T03:50:24Z",
    "Labels": {
        "embedding": [0.0017246103, -0.009725488, 0.005806058, -0.0187753, -0.015343021, ...],
        "topics": ["machine-learning", "service", "recommender", "go", "recommender-system", "knn", "collaborative-filtering"]
    },
    "Comment": "An open source recommender system service written in Go"
}

The configuration entry of an embedding similarity based item-to-item recommender:

[[recommend.item-to-item]]
name = "similar_embedding"
type = "embedding"
column = "item.Labels.embedding"

The configuration entry of a tags similarity based item-to-item recommender:

[[recommend.item-to-item]]
name = "similar_topics"
type = "tags"
column = "item.Labels.topics"

File: src/docs/concepts/recommenders/non-personalized.md Copy Chapter

icon: rankfill shortTitle: Non-personalized Copy Chapter

Non-personalized Recommenders

Gorse recommender system offers a default non-personalized recommenders, the latest recommender. However, it is not flexible enough. To improve the flexibility of non-personalized recommendations, Gorse provides non-personalized recommenders.

Configuration Copy Chapter

Non-personalized recommenders will traverse each item and its feedback, using expressions provided by the user to filtering and scoring. The configuration of a non-personalized recommender consists of three fields:

• name is the name of the recommender.
• score is the scoring function, represented in Expr language, requiring output typed floating point or integer. Items are sorted in descending order.
• filter is the filtering function, represented in Expr language, requiring output typed boolean.

::: tip
If you want to sort items in ascending order, you need to add a negative sign yourself.
:::

In expressions, the item variable represents the current item while feedback variable represents positive feedback to the current item.

The type of item is the Item structure, and the syntax for accessing fields is consistent with Go language. For example, item.Timestamp will return the timestamp of the item.

type Item struct {
    ItemId     string
    IsHidden   bool
    Categories []string
    Timestamp  time.Time
    Labels     any
    Comment    string
}

The type of feedback is []Feedback, a slice of Feedback. The syntax for accessing the array is consistent with the Go language. For example, feedback[0].FeedbackKey.FeedbackType will return the feedback type of the first positive feedback.

type Feedback struct {
    FeedbackType string
    UserId       string
    ItemId       string
    Timestamp   time.Time
    Comment     string
}

Expr language provides a lot of built-in functions that are sufficient to write more complex calculation logic. In addition to built-in functions that handle single variables such as mathematical operators, comparison operators, logical operators, etc., it also provides many array-related built-in functions. To learn more about syntax please visit Expr's official documentation.

API Copy Chapter

You can access non-personalized recommendations through the following API:

curl http://localhost:8087/api/non-personalized/<name>

Examples Copy Chapter

Latest Items Copy Section

You can sort the latest items directly by using timestamps.

[[recommend.non-personalized]]
name = "latest"
score = "item.Timestamp.Unix()"

Since item.Timestamp is a time.Time struct and cannot be used directly for sorting, it needs to call the Unix() method to convert it into int64 type.

Most Liked Items Updated within the Last Week Copy Section

Use filter to exclude items from more than a week ago, and calculate the number of likes as the sorting score.

[[recommend.non-personalized]]
name = "most_liked_weekly"
score = "count(feedback, .FeedbackType == 'like')"
filter = "(now() - item.Timestamp).Hours() < 24 * 7"

The count function will calculate the length of the feedback slice, the second parameter indicates that it only counts when the feedback type is 'like'. The now() function gets the current time, subtracts item time and requires that the number of hours be within a week (24*7 hours).

Hacker News Copy Section

It is said that Hacker News uses the following formula to calculate scores for content ranking:

$$
\frac{p-1}{(T+2)^G}
$$

where $p$ is the number of upvotes for the content, $T$ is the time from posting to score calculation, and $G$ is the gravity value used to control the rate at which content popularity decays over time. If you set $G$ to 0.5, you can add the following configuration item:

[[recommend.non-personalized]]
name = "trending"
score = "(count(feedback, .FeedbackType == 'upvote')-1)/((now() - item.Timestamp).Seconds()+2)^0.5"

File: src/docs/concepts/recommenders/user-to-user.md Copy Chapter

icon: people shortTitle: User-to-User Copy Chapter

User-to-User Recommenders

If two users have similar preferences, they are likely to like the same items. Gorse has implemented user-to-user recommenders that support similarity based on labels or favorite items.

Configuration Copy Chapter

The user-to-user recommenders need to be explicitly configured. The following three fields need to be filled in:

• name is the name of the recommender.
• type is the similarity type, and the following values are supported:
  • embedding is the Euclidean distance between embedding vectors.
  • tags is the similarity based on the number of common labels.
  • items is the similarity based on the number of common favorite items.
• column is the field used by the recommender to calculate the similarity, expressed in the Expr language.

Algorithms Copy Chapter

Embedding Similarity Copy Section

There are $K$-dimensional embedding vectors $p$ and $q$ for two items respectively, the embedding similarity between them is

$$
\sqrt{\sum^K_{k=1}(p_k-q_k)^2}
$$

Tags Similarity Copy Section

First, we use IDF to calculate the weight of each tag. The tag weight is defined as

$$
w_l = -\log\left(\frac{|U_l|}{|U|}\right)
$$

If a tag is used by more users, this tag is more generic and has a lower weight. Then, calculates similarity based on label overlap between users

$$
\frac{\sum_{l\in L_u \cap L_v}w_l}{\sqrt{\sum_{l\in L_u}w_l^2}\sqrt{\sum_{l\in L_v}w_l^2}}
$$

Items Similarity Copy Section

First, we use IDF to calculate the weight of each item.

• Item weight: The item weight is defined as

$$
w_i = -\log\left(\frac{|U_i|}{|U|}\right)
$$

If an item has more users, it means that this item is popular but has a lower weight to characterize users' preferences. Then, calculates similarity based on item overlap between users.

$$
\frac{\sum_{i\in I_u \cap I_v}w_i}{\sqrt{\sum_{i\in I_u}w_i^2}\sqrt{\sum_{i\in I_v}w_i^2}}
$$

Recommendation Copy Chapter

The user-to-user recommender first finds similar users for the target user, then sums up items from positive feedback of similar users, and finally ranks items by similarity scores to get the final recommendation score:

$$
\sum_{v \in S_u} s_{uv} \cdot \mathbf{1}_{i \in I_v}
$$

where $S_u$ represents the set of similar users of user $u$, $s_{uv}$ represents the similarity between user $u$ and user $v$, and $\mathbf{1}_{i \in I_v}$ is an indicator function that equals 1 if item $i$ is in the positive feedback set of user $v$.

API Copy Chapter

You can access similar users through the following API:

curl http://localhost:8087/api/user-to-user/<name>/

Examples Copy Chapter

For example, to recommend items based on similar users calculated from favorite items, the configuration is as follows:

[recommend.user_neighbors]
name = "similar_users"
type = "items"

File: src/docs/concepts/data-source.md Copy Chapter

icon: data Copy Chapter

Data Source

There are three kinds of data collections in Gorse: users, items, and feedback.

Users Copy Chapter

A user consists of a user ID, labels, and comment:

type User struct {
    UserId    string
    Labels    any
    Comment   string
}

• UserId is the unique identifier of the user and cannot contain a slash "/" because of conflicts with the URL definition of the RESTful APIs.
• Labels is the user's label information in JSON format, which is used to describe the user's characteristics to the recommender system. The user labels can be empty, but these labels help to improve recommendations.
• Comment is the user's comment information, which helps to browse users in the dashboard.

Item Copy Chapter

An item consists of 6 fields:

type Item struct {
	ItemId     string
	IsHidden   bool
	Categories []string
	Timestamp  time.Time
	Labels     any
	Comment    string
}

• ItemId is the unique identifier of the item and cannot contain a slash "/" because of conflicts with the URL definition of the RESTful APIs.
• IsHidden indicates whether the item is hidden from recommendations.
• Categories is a list of categories the item belongs to, which is used for filtering items in recommendations.
• Timestamp is the time when the item is added to the recommender system, which is used to determine the freshness of the item.
• Labels is the label information of the item in JSON format, which is used to describe the content of the item to the recommender system. The item labels can be empty, but these labels help to improve recommendations.
• Comment is the comment information of the item, which helps to browse items in the dashboard.

Hide Items Copy Section

In many cases, items in the history are not available for recommendation to other users, e.g.

• If the item is sold out, it can't be recommended to other users.
• If the item is legally risky, it cannot continue to be recommended to other users either.

In Gorse, items can be taken down by setting IsHidden to true for the item via the RESTful API. The recommendation algorithm can use the item during training, but the item will no longer be recommended to other users. Setting IsHidden to true takes effect immediately, but setting it to false resumes item recommendations after the refresh_recommend_period has expired.

Describe Items via Labels Copy Section

If only the item ID is available, the recommender system does not know the item's content, which requires labels to help the recommender system understand the item.

• User-Generated Label: Human-supplied labels are generally the most accurate and can be added by editors or users. For example, for a game, editors can add the publisher and genres as labels, and users can add topics about the game as labels.
• Embedding Vectors: Embedding vectors are dense vector representations of text, images, and other unstructured data. Embedding vectors can be generated using pre-trained models such as EmbeddingGemma for text.

::: warning
Keyword extraction or other automatic label extraction methods have been deprecated because they do not perform as well as embedding vectors.
:::

Feedback Copy Chapter

A feedback consists of user ID, item ID, feedback type, feedback value, and feedback timestamp, where the triad of user ID, item ID, and feedback type is required to be unique in the database.

type Feedback struct {
    FeedbackType string
    UserId       string
    ItemId       string
    Value        float64
    Timestamp    time.Time
}

Feedback represents events that happened between users and items, which can be positive or negative. For example, sharing and liking are the user's positive feedback to an item. If the user does not have further positive feedback after reading, the user's feedback on the item is considered negative. If the user views the item, read feedback will be recorded. Then, if the user gives positive feedback to the item, the read feedback will be overwritten by the positive feedback. Conversely, if the user does not give positive feedback, then the read feedback is considered negative feedback.

The feedback value is used to represent the strength of the feedback. For example, a 5-star rating system can be represented by feedback values from 1 to 5. If the feedback value is not available, it can be set to 0.

Positive and Read Feedback Copy Section

Before inserting feedback into the Gorse recommender system, it is necessary to define which of the user's behaviors are positive feedback and which are read feedback. Read feedback is relatively easy to define, as it can be recorded as read feedback when a user has seen the recommended item. However, the definition of positive feedback depends more on the specific scenario. For TikTok, users can be considered as positive feedback if they “like” or “share” the current video. For YouTube, users can be considered as positive feedback if they watch the video to a certain proportion of completion, “like“ the video, or "share" the video. To summarize, positive feedback and read feedback are defined by the following rules.

• Read Feedback: The user sees the item.
• Positive feedback: The user action that is expected to do by the service provider.

Negative Feedback Copy Section

Negative feedback represents explicit user dislike or disinterest in an item. Unlike read feedback (which indicates the user simply viewed an item), negative feedback explicitly tells the recommender system that the user does not want to see similar items in the future.

Common examples of negative feedback include:

• Dislike button: User clicks a "dislike" or "thumbs down" button
• Not interested: User selects "not interested" or "hide this" option
• Explicit removal: User removes an item from their recommendations

Negative feedback has the highest priority in the recommendation system. When a user gives negative feedback to an item:

• The item will never be recommended to that user again
• Similar items will be deprioritized in recommendations

To configure negative feedback, add the feedback type to negative_feedback_types in the configuration:

::: warning
Negative feedback should be used sparingly. Only configure truly negative actions as negative feedback. Actions like "skip" or "close" should be treated as read feedback, not negative feedback.
:::

Insert Feedback Copy Section

There are two ways to insert feedback into the Gorse recommender system: inserting new feedback and updating existing feedback. Inserting new feedback is done via the PUT /api/feedback API, while updating existing feedback is done via the POST /api/feedback API. Both APIs accept a list of feedback in JSON format.

::: code-tabs

@tab:active POST

curl -X POST "http://localhost:8088/api/feedback" \
    -H "accept: application/json" \
    -H "Content-Type: application/json" \
    -d '[ { "FeedbackType": "read", "ItemId": "10086", "Value": 1, "Timestamp": "2021-10-24T06:42:20.207Z", "UserId": "jack" }]'

@tab PUT

curl -X PUT "http://localhost:8088/api/feedback" \
    -H "accept: application/json" \
    -H "Content-Type: application/json" \
    -d '[ { "FeedbackType": "read", "ItemId": "10086", "Value": 1, "Timestamp": "2021-10-24T06:42:20.207Z", "UserId": "jack" }]'

:::

PUT /api/feedback will overwrite existing feedback with the same user ID, item ID, and feedback type, while POST /api/feedback will only sum the feedback value to the existing feedback value. For example, if there is already a feedback record of { "FeedbackType": "read", "ItemId": "10086", "Value": 1, "Timestamp": "2021-10-24T06:42:20.207Z", "UserId": "jack" } in the database, then inserting the same feedback via PUT /api/feedback will result in the feedback value being 1, while inserting the same feedback via POST /api/feedback will result in the feedback value being 2.

Read Detection Copy Section

Positive feedback can be easily detected when the user takes an action, such as clicking the "like" button. However, read feedback requires the application to detect when the user has "read" the item. There are two ways to detect read feedback: proactive read and automatic read.

Proactive Read

Positive feedback can be inserted into the recommender system when the user takes the action, while read feedback requires the application to detect the user's "read" behavior. The methods for displaying recommendations vary by application but can be generally grouped into two categories.

• Full-screen mode: The most typical application is TikTok, where the user is considered "read" when the full-screen content is shown to them. That is, the application can write a "read" feedback to the recommender system when the recommended content is shown to the user, and the read content will no longer be shown to the user.

::: center

:::

• List mode: The most typical application is YouTube, where the user is not considered "read" after looking at multiple videos in the list. When there are more than one videos, the user's attention is not able to browse the whole list. Moreover, if the read content is quickly discarded in the list mode, the recommended content is consumed too fast. Therefore, the best solution is to write a "read" feedback with a future timestamp to the recommender system when the item is presented to the user in the stream, and the "read" feedback will take effect when the time has reached the timestamp, and the read content will no longer be presented to the user.

Automatic Read

Proactively inserting read feedback into the recommender system requires the application to be able to accurately capture user browsing behavior. This task is easier for mobile applications but more difficult for web applications. To address this problem, Gorse's API for getting recommendation results provides two parameters: write−back−type and write−back−delay.

• In full-screen mode: Get a recommendation and write a "read" feedback, the recommendation will not appear again afterward.

curl -X GET "http://172.18.0.3:8087/api/recommend/zhenghaoz?write-back-type=read&n=1" \
    -H "accept: application/json" \
    -H "X-API-Key: 19260817"

• In list mode: Get 10 recommendations and write "read" feedback with timestamps of 10 minutes later. The 10 recommendations will not be discarded until after 10 minutes.

curl -X GET "http://172.18.0.3:8087/api/recommend/zhenghaoz?write-back-type=read&write-back-delay=10m&n=10" \
    -H "accept: application/json" \
    -H "X-API-Key: 19260817"

The write−back−type and write−back−delay parameters of the recommendation API provide a convenient way to insert read feedback, but of course, if you want the read feedback to be more accurate, it should be written to the recommender system by the application.

Configuration Copy Chapter

There are several configuration options related to data source in Gorse:

• positive_feedback_types: A list of feedback types that are considered positive feedback.
• read_feedback_types: A list of feedback types that are considered read feedback.
• negative_feedback_types: A list of feedback types that are considered negative feedback.
• positive_feedback_ttl: Time-to-live for positive feedback in days. After this period, positive feedback will be ignored in recommendations. Default value: 0 (no expiration).
• item_ttl: Time-to-live for items in days. After this period, items will be automatically hidden from recommendations. Default value: 0 (no expiration).

positive_feedback_types and read_feedback_types can be feedback type or feedback type with a value condition. Condition operators can be >, >=, <, <= , and ==. For example, if the feedback type is "read" and the feedback value is 5, then "read>=3" is considered positive feedback, while "read<3" is considered read feedback.

TTL is used to automatically remove old feedback and items from the recommender system to ensure that recommendations remain fresh and relevant.

Example Copy Chapter

In the demo project GitRec, the following configuration is used to define positive feedback as "star", "like", and "read" with a value greater than or equal to 3. Read feedback is defined as "read". Negative feedback is defined as "dislike". Both positive feedback and items do not expire.

[recommend.data_source]
positive_feedback_types = ["star","like","read>=3"]
negative_feedback_types = ["dislike"]
read_feedback_types = ["read"]
positive_feedback_ttl = 0
item_ttl = 0

File: src/docs/concepts/evaluation.md Copy Chapter

icon: evaluate_fill Copy Chapter

Evaluation

To estimate the performance of a recommender system, evaluation is needed. Gorse provides both online evaluation and offline evaluation.

Online Evaluation Copy Chapter

The target of a recommender system is to maximize the probability every user like recommended items. Thus, the metric for online evaluation is the positive feedback rate

$$
\frac{1}{|U|}\sum_{i\in |U|}\frac{|R^p_u|}{|R^r_u|}
$$

where $R^r_u$ is the set of read feedback from user $u$, and $R^p_U$ is the set of a specific positive feedback. Gorse calculates positive rates every day and plots them as the line chart in the dashboard.

Offline Evaluation Copy Chapter

Offline evaluation is used to estimate the performance of an individual algorithm. It allows users to inspect the status of an individual algorithm. Thus, this section is organized by different algorithms.

Evaluate Factorization Machine Copy Section

The factorization machines model predicts the probability that a user gives positive feedbacks on an item. Items with top probabilities are recommended to the user. The evaluation algorithm estimates the quality of prediction on given pairs of users and items. The dataset will be divided into a training dataset and a testing dataset. The following metrics will be calculated on the testing dataset.

$$
\text{precision}=\frac{tp}{tp+fp}
$$

where $tp=|{i|y_i=1\wedge\hat y_i=1}|$ and $fp=|{i|y_i=0\wedge \hat y_i=1}|$.

$$
\text{recall}=\frac{tp}{tp+fn}
$$

where $fn={i|y_i=1\wedge \hat y_i=0}$.

$$
\text{AUC}=\sum_{i\in P}\sum_{j \in N}\frac{\mathbb{I}(\hat y_i>\hat y_j)}{|P||N|}
$$

where $P={i|y_i=1}$ and $N={i|y_i=0}$. AUC is used as the primary metric to select the best model in Gorse.

Evaluate Collaborative Filtering (Matrix Factorization) Copy Section

The matrix factorization filters out positive feedback from negative feedbacks and unobserved feedbacks. For each user, Gorse randomly leaves one feedback out of other positive feedbacks as the test item. The matrix factorization model is expected to rank the test item before other unobserved items. Since it is too time-consuming to rank all items for every user during evaluation, Gorse followed the common strategy[^1] that randomly samples 100 items that are not interacted with the user, ranking the test item among the 100 items.

The matrix factorization is evaluated in top 10 recommendations. Suppose the matrix factorization recommends 10 items $\hat I^{(10)}_u$ to user $u$ and the test item is $i_u$

$$
\text{NDCG@10}=\sum_{u \in U}\sum_{i=1}^{10}\frac{\mathbb{I}(i=\hat I^{(10)}_{u,i})}{\log_2(i+1)}
$$

$$
\text{Precision@10}=\sum_{u \in U}\sum_{i=1}^{10}\frac{\mathbb{I}(i=\hat I^{(10)}_{u,i})}{10|U|}
$$

$$
\text{Recall@10}=\sum_{u \in U}\sum_{i=1}^{10}\frac{\mathbb{I}(i=\hat I^{(10)}_{u,i})}{|I_u|}
$$

where $\mathbb{I}(i=\hat I^{(10)}_{u,i})$ is the $i$-th item in the top 10 recommendations. NDCG@10 is used as the primary metric to select the best model in Gorse.

[^1]: He, Xiangnan, et al. "Neural collaborative filtering." Proceedings of the 26th international conference on world wide web. 2017.

File: src/docs/concepts/pipeline.md Copy Chapter

icon: pipeline Copy Chapter

Pipeline

Gorse is a multi-way recommender system. Different recommenders are combined to provide high-quality recommendations.

Retrieval and Ranking Copy Chapter

This architecture follows a "Retrieval and Ranking" pattern:

1. The input layer is the datasource. This is where all the raw data is stored.
   • Users: Demographic data or profiles of users using the app.
   • Items: The catalog of content, products, videos and etc. that can be recommended.
   • Feedback: User feedback on items (clicks, likes, purchases, view time). This is crucial for training the algorithms to understand user preferences.
2. The retrieval layer consists of multiple recommenders that generate a large pool of candidate items for each user.
   • Latest Items: Newest content, the predefined recommender in Gorse.
   • Non-personalized: Items are ranked by defined rules, such as most popular items.
   • User to User: "People similar to you liked this."
   • Item to Item: "Because you liked X, here is Y."
   • Collaborative: Matrix factorization finds complex patterns in user-item interactions.
   • External API: Fetches recommendations from third-party services (e.g., an ad network or a partner service).
3. The ranking layer merge all the outputs from different recommenders, removes items that the user has already seen (read items) and scores the remaining items based on the probability that the user will engage with them.
   • Ranker: Scores and sorts the items based on the probability that the user will engage with them.
   • Replacement: Read items are replaced back to recommendations if configured.
   • Fallback: If the ranker cannot provide enough recommendations (e.g., for new users), the fallback module provides alternative recommendations using simpler methods (like latest items).

When a user sees the recommendations and interacts with them (or ignores them), that action is recorded and fed back into the database. This allows the system to learn and improve its accuracy for the next time.

Default Pipeline Copy Section

There is a predefined recommender, latest items. If no pipeline is configured (empty [recommend] section in the configuration file), Gorse will use the default pipeline:

Latest items are recommended to all users. Latest items are easy to implement and can recommend fresh items to users. However, latest items ignore user preferences and cannot provide personalized recommendations. Therefore, it is recommended to configure a complete pipeline to improve recommendation quality.

Caches in Pipeline Copy Section

The following intermediate results are cached and updated periodically:

• Users neighbors for user-to-user recommenders.
• Item neighbors for item-to-item recommenders.
• Results of non-personalized recommenders.
• Output of ranker (with replacement applied) for each user.

Cache will be updated if data changed, configurations changed or expiration time reached. The size and expiration time of caches can be configured in the configuration file.

[recommend]
cache_size = 100
cache_expire = "72h"

Increasing cache size might improve recommendations since the recommender system has more information on data but also consumes more cache storage. The expiration time of the cache should be traded off between freshness and CPU costs.

How Gorse Works Copy Chapter

The pipeline is executed in a distributed manner. There are three types of nodes in Gorse: master, worker, and server.

Master: Dataset, Neighbors, and Models Copy Section

A cluster has only one master node. Workers and servers connect to the master node via GRPC. The master node is responsible the following tasks:

1. Load dataset: Load users, items, and feedback from the database. Non-personalized recommendations are updated during this process.
2. Generate user-to-user recommendation: Calculate user similarities and store user neighbors in the cache database for each user-to-user recommender.
3. Generate item-to-item recommendation: Calculate item similarities and store item neighbors in the cache database for each item-to-item recommender.
4. Train Collaborative Filtering Model: Train matrix factorization models and store the models in the blob storage (local disk on the master node by default).
5. Train Click-Through Rate Prediction Model: Train the factorization machine ranking model and store the model in the blob storage.

If the master node is down, workers and servers will still work but configuration changes and model updates will not be applied.

Worker: Recommendation Copy Section

The retrieval and ranking process is executed by worker nodes. Workers connect to the master node to get configurations and models. Workers periodically generate offline recommendations for all users and store the results in the cache database by the following steps:

1. For each user, get candidate items from all recommenders.
2. Score and sort candidate items by the ranker.
3. Replace read items back to recommendations if configured.
4. Store the final recommendations in the cache database.

More workers can be added to speed up the offline recommendation process. If a worker node is down, other workers will take over its tasks.

Server: Filter and Fallback Copy Section

The server node exposes RESTful APIs for data and recommendations. When a user requests recommendations, the server node performs the following steps:

1. Get offline recommendations from the cache database.
2. Filter out items that have been read by the user.
3. If not enough recommendations, get fallback recommendations from the recommenders.
4. Return recommendations to the user.

Servers are stateless, thus more servers can be added to handle more requests.

Clock Synchronization Copy Section

Gorse supports feedback with future timestamps, thus Gorse relies on a correct clock. However, clocks in different hosts might be different, clock_error should be the maximal difference between clocks.