## File: README.md

<h1 align="center">buku</h1>







### Introduction

`buku` is a powerful bookmark manager and a personal textual mini-web.

For those who prefer the GUI, `bukuserver` exposes a browsable front-end on a local web host server. See [bukuserver page](https://github.com/jarun/buku/tree/master/bukuserver#readme) for config and screenshots.

When I started writing it, I couldn't find a flexible command-line solution with a private, portable, merge-able database along with seamless GUI integration. Hence, `buku`.

`buku` can import bookmarks from browser(s) or fetch the title, tags and description of a URL from the web. Use your favourite editor to add, compose and update bookmarks. Search bookmarks instantly with multiple search options, including regex and a deep scan mode (handy with URLs).

It can look up broken links on the Wayback Machine. There's an Easter Egg to revisit random bookmarks.

There's no tracking, hidden history, obsolete records, usage analytics or homing.

To get started right away, jump to the [Quickstart](#quickstart) section. `buku` has one of the best documentation around. The man page comes with examples. For internal details, please refer to the [operational notes](https://github.com/jarun/buku/wiki/Operational-notes).

`buku` is a library too! There are several related projects, including a browser plug-in.

### Table of Contents

- [Features](#features)
- [Installation](#installation)
  - [Dependencies](#dependencies)
  - [From a package manager](#from-a-package-manager)
  - [Release packages](#release-packages)
  - [From source](#from-source)
  - [Running standalone](#running-standalone)
- [Shell completion](#shell-completion)
- [Usage](#usage)
  - [Command-line options](#command-line-options)
  - [Colors](#colors)
- [Quickstart](#quickstart)
- [Examples](#examples)
- [Automation](#automation)
- [Troubleshooting](#troubleshooting)
  - [Editor integration](#editor-integration)
- [Collaborators](#collaborators)
- [Contributions](#contributions)
- [Related projects](#related-projects)
- [In the Press](#in-the-press)

### Features

- Store bookmarks with auto-fetched title, tags and description
- Auto-import from Firefox, Google Chrome, Chromium, Vivaldi, Vivaldi Beta (Internal), Brave, and MS Edge
- Open bookmarks and search results in browser
- Browse cached page from the Wayback Machine
- Text editor integration
- Lightweight, clean interface, custom colors
- Powerful search options (regex, substring...)
- Continuous search with on the fly mode switch
- Portable, merge-able database to sync between systems
- Import/export bookmarks from/to HTML, XBEL, Markdown, RSS/Atom or Orgfile
- Smart tag management using redirection (>>, >, <<)
- Multi-threaded full DB refresh
- Manual encryption support
- Shell completion scripts, man page with handy examples
- Privacy-aware (no unconfirmed user data collection)
- Can be used as a Python library ([_API documentation_](https://buku.readthedocs.io/en/latest/?badge=latest))
- Has a compation Web-application ([Bukuserver](https://github.com/jarun/buku/wiki/Bukuserver-%28WebUI%29)) with an HTTP-based API (for personal use only)

### Installation

#### Dependencies

| Feature | Dependency |
| --- | --- |
| Lang, SQLite | Python 3.10+ |
| HTTPS | certifi, urllib3 |
| Encryption | cryptography |
| HTML | beautifulsoup4, html5lib |

To copy URL to clipboard `buku` uses `xsel` (or `xclip`) on Linux, `pbcopy` (default installed) on OS X, `clip` (default installed) on Windows, `termux-clipboard` on Termux (terminal emulation for Android), `wl-copy` on Wayland. If X11 is missing, GNU Screen or tmux copy-paste buffers are recognized.

#### From a package manager

To install buku with all its dependencies from PyPI, run:

    # pip3 install buku

You can also install `buku` from your package manager. If the version available is dated try an alternative installation method.

Packaging status (expand)


[](https://repology.org/project/buku/versions)

Unlisted packagers:


● [PyPI](https://pypi.org/project/buku/) (`pip3 install buku`)
● Termux (`pip3 install buku`)



#### Release packages

Auto-generated packages (with only the cli component) for Arch Linux, CentOS, Debian, Fedora, openSUSE Leap and Ubuntu are available with the [latest stable release](https://github.com/jarun/buku/releases/latest).

NOTE: CentOS may not have the python3-beautifulsoup4 package in the repos. Install it using pip3.

#### From source

If you have git installed, clone this repository. Otherwise download the [latest stable release](https://github.com/jarun/buku/releases/latest) or [development version](https://github.com/jarun/buku/archive/master.zip) (*risky*).

Install the dependencies. For example, on Ubuntu:

    $ apt-get install ca-certificates python3-urllib3 python3-cryptography python3-bs4

Install the cli component to default location (`/usr/local`):

    $ sudo make install

To remove, run:

    $ sudo make uninstall

`PREFIX` is supported, in case you want to install to a different location.

#### Running standalone

`buku` is a standalone utility. From the containing directory, run:

    $ chmod +x buku.py
    $ ./buku.py

### Shell completion

Shell completion scripts for Bash, Fish and Zsh can be found in respective subdirectories of [auto-completion/](https://github.com/jarun/buku/blob/master/auto-completion). Please refer to your shell's manual for installation instructions.

### Usage

#### Command-line options

```
/* Detailed source-code truncated for AI context efficiency. */
```

#### Colors

`buku` supports custom colors. Visit the wiki page on how to [customize colors](https://github.com/jarun/buku/wiki/Customize-colors) for more details.

### Quickstart

1. Export `VISUAL` or `EDITOR` to point to your favourite editor. Note that `VISUAL` takes precedence over `EDITOR`.
2. Create a sweeter shortcut with some convenience.

       alias b='buku --suggest'
3. Auto-import bookmarks from your browser(s). Please quit the relevant browsers beforehand to ensure the databases are not locked.

       b --ai
4. Manually add a bookmark (for hands-on).

       b -w
5. List your bookmarks with DB index.

       b -p
6. For GUI and browser integration (or to sync bookmarks with your favourite bookmark management service) refer to the wiki page on [System integration](https://github.com/jarun/buku/wiki/System-integration).
7. Quick (bash/zsh) commands to fuzzy search with fzf and open the selection in Firefox:

       firefox $(buku -p -f 10 | fzf)
       firefox $(buku -p -f 40 | fzf | cut -f1)

   POSIX script to show a preview of the bookmark as well:

   ```sh
   #!/usr/bin/env sh

   url=$(buku -p -f4 | fzf -m --reverse --preview "buku -p {1}" --preview-window=wrap | cut -f2)

   if [ -n "$url" ]; then
       echo "$url" | xargs firefox
   fi
   ```

### Examples

1. **Edit and add** a bookmark from editor:

       $ buku -w
       $ buku -w 'gedit -w'
       $ buku -w 'macvim -f' -a https://ddg.gg search engine, privacy
    The first command picks editor from the environment variable `EDITOR`. The second command opens gedit in blocking mode. The third command opens macvim with option -f and the URL and tags populated in template.
2. **Add** a simple bookmark:

       $ buku --nostdin -a https://github.com/
       2648. GitHub: Let’s build from here · GitHub
       > https://github.com/
       + GitHub is where over 94 million developers shape the future of software, together. Contribute to the open source community, manage your Git repositories, review code like a pro, track bugs
        and features, power your CI/CD and DevOps workflows, and secure code before you commit it.

       $ buku --nostdin -a https://github.com/
       [ERROR] URL [https://github.com/] already exists at index 2648

      `>`: URL, `+`: comment, `#`: tags

      Title, description and tags will be fetched from site. Buku only stores unique URLs and will raise error if the URL already present in the database:
3. **Add** a bookmark with **tags** `search engine` and `privacy`, **comment** `Search engine with perks`, **fetch page title** from the web:

       $ buku -a https://ddg.gg search engine, privacy -c Search engine with perks
       336. DuckDuckGo
       > https://ddg.gg
       + Alternative search engine with perks
       # privacy,search engine
    where, `>`: URL, `+`: comment, `#`: tags
4. **Add** a bookmark with tags `search engine` & `privacy` and **immutable custom title** `DDG`:

       $ buku -a https://ddg.gg search engine, privacy --title 'DDG' --immutable 1
       336. DDG (L)
       > https://ddg.gg
       # privacy,search engine
    Note that URL must precede tags.
5. **Add** a bookmark **without a title** (works for update too):

       $ buku -a https://ddg.gg search engine, privacy --title
6. **Edit and update** a bookmark from editor:

       $ buku -w 15012014
    This will open the existing bookmark's details in the editor for modifications. Environment variable `EDITOR` must be set.
7. **Update** existing bookmark at index 15012014 with new URL, tags and comments, fetch title from the web:

       $ buku -u 15012014 --url http://ddg.gg/ --tag web search, utilities -c Private search engine
8. **Fetch and update only title** for bookmark at 15012014:

       $ buku -u 15012014
9. **Update only comment** for bookmark at 15012014:

       $ buku -u 15012014 -c this is a new comment
    Applies to --url, --title and --tag too.
10. **Export** bookmarks tagged `tag 1` or `tag 2` to HTML, XBEL, Markdown, Orgfile or a new database:

       $ buku -e bookmarks.html --stag tag 1, tag 2
       $ buku -e bookmarks.xbel --stag tag 1, tag 2
       $ buku -e bookmarks.md --stag tag 1, tag 2
       $ buku -e bookmarks.org --stag tag 1, tag 2
       $ buku -e bookmarks.db --stag tag 1, tag 2
    All bookmarks are exported if search is not opted.
11. **Import** bookmarks from HTML, XBEL, Markdown or Orgfile:

        $ buku -i bookmarks.html
        $ buku -i bookmarks.xbel
        $ buku -i bookmarks.md
        $ buku -i bookmarks.org
        $ buku -i bookmarks.db
12. **Delete only comment** for bookmark at 15012014:

        $ buku -u 15012014 -c
    Applies to --title and --tag too. URL cannot be deleted without deleting the bookmark.
13. **Update** or refresh **full DB** with page titles from the web:

        $ buku -u
        $ buku -u --tacit (show only failures and exceptions)
    This operation can update the title or description fields of non-immutable bookmarks by parsing the fetched page. Fields are updated only if the fetched fields are non-empty. Tags remain untouched.
14. **Delete** bookmark at index 15012014:

        $ buku -d 15012014
        Index 15012020 moved to 15012014
    The last index is moved to the deleted index to keep the DB compact. Add `--tacit` to delete without confirmation.
15. **Delete all** bookmarks:

        $ buku -d
16. **Delete** a **range or list** of bookmarks:

        $ buku -d 100-200
        $ buku -d 100 15 200
17. **Search** bookmarks for **ANY** of the keywords `kernel` and `debugging` in URL, title or tags:

        $ buku kernel debugging
        $ buku -s kernel debugging
18. **Search** bookmarks with **ALL** the keywords `kernel` and `debugging` in URL, title or tags:

        $ buku -S kernel debugging
19. **Search** bookmarks **tagged** `general kernel concepts`:

        $ buku --stag general kernel concepts
20. **Search** for bookmarks matching **ANY** of the tags `kernel`, `debugging`, `general kernel concepts`:

        $ buku --stag kernel, debugging, general kernel concepts
21. **Search** for bookmarks matching **ALL** of the tags `kernel`, `debugging`, `general kernel concepts`:

        $ buku --stag kernel + debugging + general kernel concepts
22. **Search** for bookmarks matching any of the keywords `hello` or `world`, excluding the keywords `real` and `life`, matching both the tags `kernel` and `debugging`, but **excluding** the tags `general kernel concepts` and `books`:

        $ buku hello world --exclude real life --stag 'kernel + debugging - general kernel concepts, books'
23. **Search** for bookmarks with different tokens for each field, and print them out sorted by the tags (ascending) and URL (descending)

        $ buku --order +tags,-url --markers --sall 'global substring' '.title substring' ':url substring' :https '> description substring' '#partial,tags:' '#,exact,tags' '*another global substring'
24. List **all unique tags** alphabetically:

        $ buku --stag
25. Run a **search and update** the results:

        $ buku -s kernel debugging -u --tag + linux kernel
26. Run a **search and delete** the results:

        $ buku -s kernel debugging -d
27. **Encrypt or decrypt** DB with **custom number of iterations** (15) to generate key:

        $ buku -l 15
        $ buku -k 15
    The same number of iterations must be specified for one lock & unlock instance. Default is 8, if omitted.
28. **Show details** of bookmarks at index 15012014 and ranges 20-30, 40-50:

        $ buku -p 20-30 15012014 40-50
29. Show details of the **last 10 bookmarks**:

        $ buku -p -10
30. **Show all** bookmarks with real index from database:

        $ buku -p
        $ buku -p | more
31. **Replace tag** 'old tag' with 'new tag':

        $ buku --replace 'old tag' 'new tag'
32. **Delete tag** 'old tag' from DB:

        $ buku --replace 'old tag'
33. **Append (or delete) tags** 'tag 1', 'tag 2' to (or from) existing tags of bookmark at index 15012014:

        $ buku -u 15012014 --tag + tag 1, tag 2
        $ buku -u 15012014 --tag - tag 1, tag 2
34. **Open URL** at index 15012014 in browser:

        $ buku -o 15012014
35. List bookmarks with **no title or tags** for bookkeeping:

        $ buku -S blank
36. List bookmarks with **immutable title**:

        $ buku -S immutable
37. **Append, remove tags at prompt** (taglist index to the left, bookmark index to the right):

        // append tags at taglist indices 4 and 6-9 to existing tags in bookmarks at indices 5 and 2-3
        buku (? for help) g 4 9-6 >> 5 3-2
        // set tags at taglist indices 4 and 6-9 as tags in bookmarks at indices 5 and 2-3
        buku (? for help) g 4 9-6 > 5 3-2
        // remove all tags from bookmarks at indices 5 and 2-3
        buku (? for help) g > 5 3-2
        // remove tags at taglist indices 4 and 6-9 from tags in bookmarks at indices 5 and 2-3
        buku (? for help) g 4 9-6 << 5 3-2
38. List bookmarks with **colored output**:

        $ buku --colors oKlxm -p
39. Add a bookmark after following all permanent redirects, but only if the server doesn't respond with an error (and there's no network failure)

        $ buku --add http://wikipedia.net --url-redirect --del-error
        2. Wikipedia
           > https://www.wikipedia.org/
           + Wikipedia is a free online encyclopedia, created and edited by volunteers around the world and hosted by the Wikimedia Foundation.
40. Add a bookmark with tag `http redirect` if the server responds with a permanent redirect, or tag shaped like `http 404` on an error response:

        $ buku --add http://wikipedia.net/notfound --tag-redirect 'http redirect' --tag-error 'http {}'
        [ERROR] [404] Not Found
        3. Not Found
           > http://wikipedia.net/notfound
           # http 404,http redirect
41. Update all bookmarks matching the search by updating the URL if the server responds with a permanent redirect, deleting the bookmark if the server responds with HTTP error 400, 401, 402, 403, 404 or 500, or adding a tag shaped like `http:{}` in case of any other HTTP error; then export those affected by such changes into an HTML file, marking deleted records as well as old URLs for those replaced by redirect.

        $ buku -S ://wikipedia.net -u --url-redirect --tag-error --del-error 400-404,500 --export-on --export backup.html

42. Print out a single **random** bookmark:

        $ buku --random --print

43. Print out 3 **random** bookmarks **ordered** by netloc (reversed), title and url:

        $ buku --random 3 --order ,-netloc,title,+url --print

44. Print out a single **random** bookmark matching **search** criteria, and **export** into a Markdown file (in DB order):

        $ buku --random -S kernel debugging --export random.md

45. Swap positions of records #4 and #5:

        $ buku --swap 4 5

46. Update indices in all bookmarks to match specified order:

        $ buku --reorder ,-netloc,title,+url

47. More **help**:

        $ buku -h
        $ man buku

### Automation

Interactive workflows can be automated using expect. Issue [#368](https://github.com/jarun/buku/issues/368) has a working example on automating auto-import.

### Troubleshooting

#### Editor integration

You may encounter issues with GUI editors which maintain only one instance by default and return immediately from other instances. Use the appropriate editor option to block the caller when a new document is opened. See issue [#210](https://github.com/jarun/buku/issues/210) for gedit.

### Collaborators

- [Arun Prakash Jana](https://github.com/jarun)
- [Alexey Gulenko](https://github.com/LeXofLeviafan)
- [Rachmadani Haryono](https://github.com/rachmadaniHaryono)
- [Johnathan Jenkins](https://github.com/shaggytwodope)
- [SZ Lin](https://github.com/szlin)

Copyright © 2015-2026 [Arun Prakash Jana](mailto:engineerarun@gmail.com)

[](https://gitter.im/jarun/buku)

### Contributions

Missing a feature? There's a rolling [ToDo List](https://github.com/jarun/buku/issues/484) with identified tasks. Contributions are welcome! Please follow the [PR guidelines](https://github.com/jarun/buku/wiki/PR-guidelines).

See also our documentation here [](http://buku.readthedocs.io/en/stable/?badge=stable)

### Related projects

- [bukubrow](https://github.com/SamHH/bukubrow), WebExtension for browser integration
- [oil](https://github.com/AndreiUlmeyda/oil), search-as-you-type cli front-end
- [buku_run](https://github.com/carnager/buku_run), rofi front-end
- [pinku](https://github.com/mosegontar/pinku), a Pinboard-to-buku import utility
- [buku-dmenu](https://gitlab.com/benoliver999/buku-dmenu), a simple bash dmenu wrapper
- [poku](https://github.com/shanedabes/poku), sync between Pocket and buku
- [Ebuku](https://github.com/flexibeast/ebuku), Emacs interface to buku
- [diigoku](https://github.com/dppdppd/diigoku), buku importer for Diigo
- [BukuBot](https://git.xmpp-it.net/sch/BukuBot), Chat bot for XMPP with an extended visual interface


### Videos

- [Buku: Take Your Bookmarks Everywhere You Go](https://www.youtube.com/embed/9HzEHrUBQXE)
- [Buku is a great open-source bookmark manager](https://www.youtube.com/embed/7VxgKMWm-J8)

### In the Press

- [2daygeek](http://www.2daygeek.com/buku-command-line-bookmark-manager-linux/)
- [Hacker Milk](https://hackermilk.blogspot.com/2020/01/how-to-manage-your-browsers-bookmarks.html)
- [It's F.O.S.S.](https://itsfoss.com/buku-command-line-bookmark-manager-linux/)
- [LinOxide](https://linoxide.com/linux-how-to/buku-browser-bookmarks-linux/)
- [LinuxUser Magazine 01/2017 Issue](http://www.linux-community.de/LU/2017/01/Das-Beste-aus-zwei-Welten)
- [Make Tech Easier](https://www.maketecheasier.com/manage-browser-bookmarks-ubuntu-command-line/)
- [One Thing Well](http://onethingwell.org/post/144952807044/buku)
- [Open Source For You](https://opensourceforu.com/2018/05/buku-a-bookmark-manager-in-the-command-line/)
- [ulno.net](https://ulno.net/blog/2017-07-19/of-bookmarks-tags-and-browsers/)

---

## File: bukuserver/README.md

## Bukuserver

_**Note: see [the runner script](https://github.com/jarun/buku/wiki/Bukuserver-%28WebUI%29#runner-script) for advanced installation/running/DB swapping functionality**_

### Table of Contents

- [Installation](#installation)
  - [Dependencies](#dependencies)
  - [From PyPi](#from-pypi)
  - [From source](#from-source)
  - [Using Docker](#using-docker)
  - [Using Docker Compose](#using-docker-compose)
- [Webserver options](#webserver-options)
- [Configuration](#configuration)
- [API](#api)
- [Screenshots](#screenshots)

### Installation

You need to have some packages before you install `bukuserver` on your server.
So be sure to have `python3`, `python3-pip` , `python3-dev`, `libffi-dev` packages from your distribution.

#### Dependencies

```
$ # venv activation (for development)
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install --upgrade pip
```

#### From PyPi

```sh
$ # regular/venv install
$ pip3 install "buku[server]"
$ # pipx install
$ pipx install "buku[server]"
$ # with locales
$ pipx install "buku[server,locales]"
```

#### From source

```sh
$ git clone https://github.com/jarun/buku
$ cd buku
$ # regular/venv install
$ pip3 install ".[server]"
$ # with locales
$ pip3 install ".[server,locales]"
```

#### Using Docker

To build the image execute the command from the root directory of the project:

```sh
docker build -t bukuserver .
```

To run the generated image.

```sh
docker run -it --rm -v ~/.local/share/buku:/root/.local/share/buku -p 5001:5001 bukuserver
```

All the data generated will be stored in the `~/.local/share/buku` directory.
Feel free to change it to the full path of the location you want to store the
database.

Visit `127.0.0.1:5001` in your browser to access your bookmarks.

#### Using Docker Compose

There is a `docker-compose.yml` file present in the `docker-compose` directory
in the root of this project. You may modify the configurations in this file to
your liking, and then simply execute the below command.

```sh
docker-compose up -d
```

You will have you bukuserver running on port port 80 of the host.

To stop simply run

```sh
docker-compose down
```

In case you want to add basic auth to your hosted instance you may do so by
creating a `.htpasswd` file in the `data/basic_auth` directory. Add a user to
the file using

```sh
htpasswd -c data/basic_auth/.htpasswd your_username
```

And then comment out the basic auth lines from the `data/nginx/nginx.conf` file.

For more information please refer the [nginx docs](https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/).

### Webserver options

To run the server on host 127.0.0.1, port 5001, run following command:

    $ bukuserver run --host 127.0.0.1 --port 5001

Visit `127.0.0.1:5001` in your browser to access your bookmarks.

See more option on `bukuserver run --help` and `bukuserver --help`

### Configuration

The following are [os env config variables](#how-to-specify-environment-variables) available for bukuserver.

_**Important:** all of them have a shared prefix_ **`BUKUSERVER_`**.

| Name (_without prefix_) | Description | Value _²_ |
| --- | --- | --- |
| PER_PAGE | bookmarks per page | positive integer [default: 10] |
| SECRET_KEY | [flask secret key](https://flask.palletsprojects.com/config/#SECRET_KEY) | string [default: random value] |
| URL_RENDER_MODE | url render mode | `full`, `netloc` or `netloc-tag` [default: `full`] |
| DB_FILE | full path to db file*³* | path string [default: standard path for buku] |
| READONLY | read-only mode | boolean*¹* [default: `false`] |
| DISABLE_FAVICON | disable bookmark [favicons](https://wikipedia.org/wiki/Favicon) | boolean*¹* [default: `true`] ([here's why](#why-favicons-are-disabled-by-default))|
| AUTOFETCH | initial Fetch value in Create form | boolean*¹* [default: `true`] |
| OPEN_IN_NEW_TAB | url link open in new tab | boolean*¹* [default: `false`] |
| REVERSE_PROXY_PATH | reverse proxy path*⁵* | string |
| SERVER_NAME | canonical host:port for URL generation*⁶* | string (e.g. `example.com:443`) |
| THEME | [GUI theme](https://bootswatch.com/4) | string [default: `default`] (`slate` is a good pick for dark mode) |
| LOCALE | GUI language*⁴* (partial support) | string [default: `en`] |
| DEBUG | debug mode (verbose logging etc.) | boolean*¹* [default: `false`] |

_**¹**_ valid boolean values are `true`, `false`, `1`, `0` (case-insensitive).

_**²**_ if input is invalid, the default value will be used if defined

_**³**_ `BUKUSERVER_DB_FILE` can be a DB name (plain filename without extension; cannot contain `.`). The specified DB with `.db` extension is located in default DB directory (which you can override with `BUKU_DEFAULT_DBDIR`).

_**⁴**_ `BUKUSERVER_LOCALE` requires buku to be installed with `[locales]`

_**⁵**_ the value for `BUKUSERVER_REVERSE_PROXY_PATH` is recommended to include preceding slash and not have trailing slash (i.e. use `/foo` not `/foo/`)

_**⁶**_ `BUKUSERVER_SERVER_NAME` mitigates [Host header injection](https://owasp.org/www-project-web-security-testing-guide/latest/4-Web_Application_Security_Testing/07-Input_Validation_Testing/17-Testing_for_Host_Header_Injection). When set, absolute URLs (e.g. bookmarklet) use this value instead of the client-supplied Host. **Recommended for production and reverse proxy deployments.**


#### How to specify environment variables

E.g. to set bukuserver to show 100 items per page run the following command:
```
# on linux
$ export BUKUSERVER_PER_PAGE=100

# on windows
$ SET BUKUSERVER_PER_PAGE=100

# in dockerfile
ENV BUKUSERVER_PER_PAGE=100

# in env file
BUKUSERVER_PER_PAGE=100
```

Note: an env file can be supplied either [by providing `--env-file` CLI argument](https://flask.palletsprojects.com/en/stable/cli/#environment-variables-from-dotenv) (requires `python-dotenv` installed),
or as config for [the runner script](https://github.com/jarun/buku/wiki/Bukuserver-%28WebUI%29#runner-script).


#### Why favicons are disabled by default

At Bukuserver, we have [disabled favicon as a default setting](#configuration) in order to prevent any non-user triggered network activity.

Our favicon is generated with the assistance of Google.

It is important to be aware that favicon has the potential to be used for browser fingerprinting,
a technique used to identify and track a person's web browsing habits.

- [Github repo example supercookie](https://github.com/jonasstrehle/supercookie)
- [Paper by Scientists at University of Illinois, Chicago](https://www.cs.uic.edu/~polakis/papers/solomos-ndss21.pdf)
- [Article published in 2021 at Heise Online](https://heise.de/-5027814)
  ([English translation](https://www-heise-de.translate.goog/news/Browser-Fingerprinting-Favicons-als-Super-Cookies-5027814.html?_x_tr_sl=de&_x_tr_tl=en&_x_tr_hl=en))

It is important to note that favicon can potentially be exploited in this way.

### API

Bukuserver implements a RESTful API that provides an HTTP-based interface for the main functionality.

The API root is at `/api`; you can also access a [Swagger](https://swagger.io/tools/swagger-ui)-based interactive doc at the endpoint `/apidocs` (e.g. `http://localhost:5000/apidocs`).

_**Note: unlike regular IDs, indices aren't static; they're likely to change if used with ID**_
* `/api/tags` can be used to `GET` the list of all tags
* `/api/tags/{tag}` can be used to `GET` information on specified tag, as well as `DELETE` or replace it with new tags (`PUT`) in all bookmarks
* `/api/bookmarks` can be used to `GET` or `DELETE` all bookmarks, as well as create (`POST`) a new one
* `/api/bookmarks/{index}` can be used to `GET`, `DELETE` or update (`PUT`) an existing bookmark
* `/api/bookmarks/{start_index}/{end_index}` can be used to `GET`, `DELETE` or update (`PUT`) bookmarks in existing index range
* `/api/bookmarks/search` can be used to `GET` or `DELETE` bookmarks matching the query (you can use it to obtain current index by URL)
* ~~`/api/bookmarks/{index}/tiny` can be used to `GET` a shortened URL~~ ([the service providing this functionality is no longer available](https://web.archive.org/web/20250109212915/https://tny.im/))
* `/api/bookmarks/{index}/refresh` can be used to update (`POST`) data for a bookmark by remotely fetching & parsing the URL
* `/api/bookmarks/refresh` can be used to update (`POST`) data for _**all**_ bookmarks by remotely fetching & parsing the URLs
* `/api/fetch_data` can be used to invoke (`POST`) the fetch+parse functionality for an arbitrary URL
* `/api/network_handle` can be used to invoke (`POST`) the fetch+parse functionality for an arbitrary URL (_outdated interface_)

Also note that certain `POST`/`DELETE` endpoints (bookmarks search & data fetch) expect their parameters in urlencoded format, while others expect JSON.

### Screenshots

_**Note: more screenshots (to show off `default` & `slate` themes) can be found [on the respective project wiki page](https://github.com/jarun/buku/wiki/Bukuserver-%28WebUI%29)**_

---

## File: bukuserver-runner/README.md

# Bukuserver runner

This tool can be used to run and restart Bukuserver, switching databases between runs. It has no third-party dependencies, allowing to run Bukuserver sandboxed in a virtualenv as easily as the system-wide install (which is especifally useful for development).

I suggest installing/symlinking it system-wide (e.g. as an `/usr/local/bin/buku-server` executable). Either of the `*.desktop` files can be edited according to match your setup and installed in your `local/share/applications/` folder for access from system menu.

On Windows, you can create a shortcut file pointing to any Python executable (`python.exe` for windowed mode, `pythonw.exe` for headless) with added CLI arguments: path to `buku-server.py` followed by `--stop-if-running`.

Note that windowed mode may be necessary if you want to see Bukuserver logs, or use noGUI mode (see below). The terminal window can be minimized to tray when not in use, by a program like [KDocker](https://github.com/user-none/KDocker), [RBTray](https://github.com/benbuck/rbtray) or [SmartSystemMenu](https://github.com/AlexanderPro/SmartSystemMenu).

## Usage

When running `buku-server.py` without arguments, it will prompt for database file, then start the Bukuserver. These actions will be repeated once Bukuserver stops running (e.g. after hitting `Ctrl+C`). The script will quit if you cancel the prompt.

In GUI mode, the prompt is implemented as 2 dialogs; a list of databases to choose from, and a text input for creating a new DB. In the shell mode, you can type in DB number from the list, or a new DB name. Note that DB names must be valid filenames in your system (sans the `.db` extension). These files are located in your Buku settings folder (along with the default `bookmarks.db` file).

Running `buku-server.py --stop` will kill the currently running Bukuserver process (thus allowing to restart it in the background, like a daemon). `buku-server.py --stop-if-running` will either start the script or kill Bukuserver if it's running already.

<h3>Screenshots</h3>

  
_DB selection dialog – shown on startup (unless no DB files were found); initially the previous DB is selected_

  
_DB creation dialog – shown if no DB was selected (or none found)_

  
_A confirmation dialog is shown if new DB name is taken already_

  
_DB name must be a valid filename, sans the `.db` extension (invalid chars: `/` on Linux, or any of `<>:"/\|?*` on Windows)_

  
_DB selection prompt in console shell/no-GUI mode (`BUKU_NOGUI=y`)_


## Environment variables

The script behaviour can be configured by setting the following environment variables:
* `BUKUSERVER` specifies path to your Bukuserver executable or Buku source directory.
* `BUKU_DEVMODE` – if not empty, Bukuserver will be run in development mode. Normally used with source directory in `BUKUSERVER`.
* `BUKU_VENV` overrides path to your virtualenv sandbox (default depends on whether `BUKU_DEVMODE` is set):
  - when devmode is off, the virtualenv location defaults to a `venv/` folder in your Buku settings directory;
  - when devmode is on, the virtualenv location defaults to a `venv/` folder in the source directory.
* `BUKU_NOGUI` – if not empty, fallback shell prompt will be used (also happens if Tkinter is not present in your Python installation).
* `BUKU_DEFAULT_DBDIR` – specify directory for DB selection (same as Buku itself).

Default values for all of these (as well as for `BUKUSERVER_` options) can be specified in a `bukuserver.env` file in your Buku settings folder:
```sh
# ~/.local/share/buku/bukuserver.env
BUKUSERVER='~/Sources/buku/'  # when running from sources
BUKUSERVER_THEME=slate
BUKUSERVER_DISABLE_FAVICON=false
BUKUSERVER_OPEN_IN_NEW_TAB=true
```

---

## File: docs/source/buku.rst

buku module
===========

.. automodule:: buku
    :members:
    :undoc-members:
    :show-inheritance:

---

## File: docs/source/bukuserver.rst

bukuserver package
==================

bukuserver.filters module
-------------------------

.. automodule:: bukuserver.filters
   :members:
   :undoc-members:
   :show-inheritance:

bukuserver.forms module
-----------------------

.. automodule:: bukuserver.forms
   :members:
   :undoc-members:
   :show-inheritance:

bukuserver.response module
--------------------------

.. automodule:: bukuserver.response
   :members:
   :undoc-members:
   :show-inheritance:

bukuserver.server module
------------------------

.. automodule:: bukuserver.server
   :members:
   :undoc-members:
   :show-inheritance:

bukuserver.views module
-----------------------

.. automodule:: bukuserver.views
   :members:
   :undoc-members:
   :show-inheritance:

---

## File: docs/source/index.rst

.. buku documentation master file, created by
   sphinx-quickstart on Thu Sep  7 12:54:59 2017.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

buku
====

Bookmark manager like a text-based mini-web.

.. toctree::
   :maxdepth: 2
   :caption: User guide
.. toctree::
   :glob:
   :maxdepth: 2
   :caption: Wiki
   
   wiki/*


.. toctree::
   :maxdepth: 2
   :caption: Buku Documentation

   buku 

.. toctree::
   :maxdepth: 2
   :caption: Bukuserver Documentation

   bukuserver 


Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

---

## File: docs/source/modules.rst

buku
==========

.. toctree::
   :maxdepth: 4

   buku

bukuserver
==========

.. toctree::
   :maxdepth: 4

   bukuserver

---

## File: docs/source/tutorial_for_developer.md

# Tutorial for Developer

## get buku database

```python
>>> import buku
>>> bdb = buku.BukuDb()
```

## simplest way to add url to buku

```python
>>> rec_id = bdb.add_rec('http://example.com')
>>> rec_id
40296
```

## get record id from url

```python
>>> url = 'https://example.com'
>>> rec_id = bdb.add_rec(url)
... if rec_id == -1:
...     rec_id = bdb.get_rec_id(url)
>>> rec_id
40296
```

## get url data from record id

```python
>>> rec = bdb.get_rec_by_id(40296)
>>> rec
(40296, 'http://example.com', 'Example Domain', ',', '', 0)
```

## get tag list

```python
rec[3].split(buku.delim)
```