; } ``` The authors: ```graql-template insert for() do { $author isa author has authorName <author>; } ``` The subjects: ```graql-template insert for() do { $subject isa subject has subjectName <subject>; } ``` Finally, a template to build the relations between the entities: ```graql-template ``` To call the migration script on each template: ```bash ./bin/migration.sh json -t ./json/book-template.gql -i ./json/library-data.json -k grakn ./bin/migration.sh json -t ./json/author-template.gql -i ./json/library-data.json -k grakn ./bin/migration.sh json -t ./json/subject-template.gql -i ./json/library-data.json -k grakn ./bin/migration.sh json -t ./json/publication-template.gql -i ./json/library-data.json -k grakn ``` The resultant Graql for the migration: ```graql ``` ## Visualise the Data ## Add Borrower Data ## Data Export ```bash # Export the ontology <relative-path-to-Grakn>/migration.sh export -ontology > ontology-export.gql # Export the data <relative-path-to-Grakn>/migration.sh export -data > data-export.gql ``` Exporting data or the ontology from Grakn, into Graql, will always redirect to standard out, so above we are sending the output to an appropriately named file. ## Where Next? {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/modern.md --- title: Modern Example in Graql keywords: graql, query last_updated: September 19, 2016 tags: [graql, examples] summary: "A short example to illustrate Graql queries" sidebar: documentation_sidebar permalink: /documentation/examples/modern.html folder: documentation comment_issue_id: 27 --- If you have not yet set up the Grakn environment, please see the [Setup guide](../get-started/setup-guide.html). For a comprehensive guide to all Graql keywords, please see the [Graql documentation](https://grakn.ai/pages/documentation/graql/graql-overview.html). ## Introduction We have a few examples on how to work with Graql using different datasets. For example, see our [blog posts](https://medium.com/p/e1125e02dc85) and the [Quickstart Tutorial](../get-started/quickstart-tutorial.html) that introduces how to make Graql queries. This example takes a very simple example from [TinkerPop3 Documentation](http://tinkerpop.apache.org/docs/3.0.1-incubating/), called TinkerPop Modern. Here it is, shown diagrammatically: The image above is used from the documentation provided for TinkerPop3, and licensed by the [Apache Software Foundation](http://www.apache.org). We have chosen this example as it may already be familiar, and is simple enough to demonstrate some of the fundamentals of Graql. We walk through the entities ("things") and relationships between them and show how to represent them using Graql to define a schema. We then use Graql to add the data to the graph. The main purpose of this example, however, is to use it for practice at making sample queries on the graph. ### Starting Graql If it is not already running, start Grakn, then open a Graql shell: ```bash cd [your Grakn install directory] bin/grakn.sh start bin/graql.sh ``` ## Defining a Schema In the example, we have a number of entities: * people (marko, vadas, josh and peter), each of which has an age. * software (lop and ripple), each of which has an associated programming language. The relationships between the entities are straightforward: * there are two connections between marko and other people who he "knows" (josh and vadas). * there are four connections between people and software, such that the people "created" the software. Lop was created by marko, peter and josh, which ripple was created by josh. ### Entities Here, we add person and software entities, via the Graql shell: ```graql insert person sub entity; insert software sub entity; ``` ### Resources To assign resources to the entities, which you can think of as attributes, we use resources. First, we define what they are (age is a number, programming language is a string that represents the language's name), then we allocate them to the entity in question: ```graql insert age sub resource datatype long; insert name sub resource datatype string; insert person has-resource age, has-resource name; insert lang sub resource datatype string; insert software has-resource lang has-resource name; insert weight sub resource datatype double; ``` ### Relations Let's first define the relationship between people. The diagram shows that marko knows vadas, but we don't have any information about whether the inverse is true (though it seems likely that vadas probably also knows marko). Let's set up a relationship called `knows`, which has two roles - `knower` (for marko) and `known-about` (for vadas): ```graql insert knower sub role; insert known-about sub role; insert person plays-role knower; insert person plays-role known-about; insert knows sub relation, has-role knower, has-role known-about, has-resource weight; ``` Note that the `knows` relation also has an attribute, in the form of a resource called `weight` (though it's not clear from the TinkerPop example what this represents). We can set up a similar relationship between software and the people that created it: ```graql insert programmer sub role; insert programmed sub role; insert person plays-role programmer; insert software plays-role programmed; insert programming sub relation, has-role programmer, has-role programmed, has-resource weight; ``` And that's it. At this point, we have defined the schema of the graph. ## Adding the Data Now we have a schema, we can move on to adding in the data, which is pretty much just a typing exercise: ```graql insert $marko isa person, has name "marko", has age 29; insert $vadas isa person, has name "vadas", has age 27; insert $josh isa person, has name "josh", has age 32; insert $peter isa person, has name "peter", has age 35; match $marko has name "marko"; $josh has name "josh"; insert (knower: $marko, known-about: $josh) isa knows has weight 1.0; match $marko has name "marko"; $vadas has name "vadas"; insert (knower: $marko, known-about: $vadas) isa knows has weight 0.5; ``` ```graql insert $lop isa software, has lang "java", has name "lop"; insert $ripple isa software, has lang "java", has name "ripple"; match $marko has name "marko"; $lop has name "lop"; insert (programmer: $marko, programmed: $lop) isa programming has weight 0.4; match $peter has name "peter"; $lop has name "lop"; insert (programmer: $peter, programmed: $lop) isa programming has weight 0.2; match $josh has name "josh"; $lop has name "lop"; insert (programmer: $josh, programmed: $lop) isa programming has weight 0.4; match $josh has name "josh"; $ripple has name "ripple"; insert (programmer: $josh, programmed: $ripple) isa programming has weight 1.0; ``` ## Querying This example is designed to get you up close and personal with Graql queries. It will run through a few basic examples, then ask you a set of "Test Yourself" questions. OK, so if you've followed the above, you should now have a schema and some data in a graph. How do you go about using the graph to answer your queries? That's where the `match` statement comes in. As with any query language, you use a variable to receive the results of the match query, which you must prefix with a `$`. So, to make the query "List every person in the graph", you would use the following in Graql: ```graql >>> match $x isa person, has name $n; select $n; $n value "vadas" isa name; $n value "marko" isa name; $n value "josh" isa name; $n value "peter" isa name; ``` In Graql, a match is formed of three parts: the `match` statement, an optional `select` statement, and any other optional [modifiers](../graql/match-queries.html#modifiers) that you choose to apply to the listing of results. Only the first part of a match query is needed: the modifier parts are optional. In the `match $x isa person` query we are not using any select or delimiters, so let's add some now. We can add a `select` statement to ask Graql to list out every person and to include their id (which is their name) and age. We use `order by` to modify how the results are listed out - in this case, we order them by ascending age, so the youngest person is shown first. ```graql >>> match $x isa person, has name $n, has age $a; select $n, $a; order by $a asc; $x id "vadas" has age "27"; $x id "marko" has age "29"; $x id "josh" has age "32"; $x id "peter" has age "35"; ``` ## Complete Example Here is the complete example - the code to define the schema and insert the data into a graph. You can load this directly into Graql, if you don't want to type it out for yourself. Cut and paste the Graql below and start Graql: ```bash bin/graql.sh ``` Then type edit, which will open up the systems default text editor where you can paste your chunk of text. Upon exiting the editor, the Graql will execute. ```graql insert age sub resource datatype long; name sub resource datatype string; person sub entity; person has-resource age, has-resource name; $marko isa person; $vadas isa person; $josh isa person; $peter isa person; $marko has age 29, has name "marko"; $josh has age 32, has name "josh"; $vadas has age 27, has name "vadas"; $peter has age 35, has name "peter"; weight sub resource datatype double; knower sub role; known-about sub role; person plays-role knower; person plays-role known-about; knows sub relation has-role knower has-role known-about has-resource weight; (knower: $marko, known-about: $josh) isa knows has weight 1.0; (knower: $marko, known-about: $vadas) isa knows has weight 0.5; lang sub resource datatype string; software sub entity; software has-resource lang, has-resource name; $lop isa software; $ripple isa software; $lop has lang "java", has name "lop"; $ripple has lang "java", has name "ripple"; programmer sub role; programmed sub role; person plays-role programmer; software plays-role programmed; programming sub relation has-role programmer has-role programmed has-resource weight; (programmer: $marko, programmed: $lop) isa programming has weight 0.4; (programmer: $peter, programmed: $lop) isa programming has weight 0.2; (programmer: $josh, programmed: $lop) isa programming has weight 0.4; (programmer: $josh, programmed: $ripple) isa programming has weight 1.0; ``` ## Test Yourself 1. List every person with their name and age in ascending age order 2. List every person who has an age over 30 3. List every person who knows someone else, and every person who is known about 4. List every person that Marko knows 5. List every item of software and the language associated with it 6. List everything that Josh has programmed 7. List everyone who has programmed Lop 8. List everything you know about Marko {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/OWL-migration.md --- title: An Example of Migrating OWL to Grakn keywords: migration last_updated: September 19, 2016 tags: [migration, examples] summary: "A short example to illustrate migration of OWL to Grakn" sidebar: documentation_sidebar permalink: /documentation/examples/OWL-migration.html folder: documentation comment_issue_id: 27 --- For a comprehensive guide to migration, please see both our [Migration Tutorial](../migration/migration-overview.html) and our additional documentation specific to [OWL Migration](../migration/OWL-migration.html). ## Introduction This example addresses the topic of OWL interoperability through a migration of a family tree OWL ontology. The code for this example can be found in our [github repository](https://github.com/graknlabs/sample-projects/tree/master/example-owl-migration). The sole prerequisite of this example is having the Grakn environment installed and the Engine running. If you need help starting Grakn Engine, please see the [setup guide](../get-started/setup-guide.html). ### Running the example You can run this example by running the [`Main`](https://github.com/graknlabs/sample-projects/blob/master/example-owl-migration/src/main/java/Main.java) class. Check out the `OWLResourceMigrator` class for the bulk of the migration code. We run a few queries in the example to prove that the data has been migrated. After running the example, you should be able to answer the following questions (see the bottom of the page for answers - but please don't peek until you've tried it!): + What are the Types in the Family Tree graph in Grakn? + How many people are in the family tree? + How many descendants does Eleanor Pringle have? + Who are the great uncles of Ethel Archer? ## Test Yourself Answers **What are the Types in the Family Tree graph?** Answer: ``` tThing tPerson tMan tWoman ``` **How many people are in the family tree?** Answer: `411` **How many descendants does Eleanor Pringle have?** Answer: `55` **Who are the great uncles of Ethel Archer?** Answer: ``` William Whitfield Harry Whitfield George Whitfield Walter Whitfield James Whitfield ``` ## Where next? After running this OWL migration, check out the [Graql documentation](../graql/graql-overview.html) and the [Graph API documentation](../developing-with-java/graph-api.html) for more instructions on how you can explore the family tree. {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/pokemon.md --- title: Pokemon Example keywords: graql, query last_updated: January 2017 tags: [graql, examples] summary: "A short example to illustrate Graql queries" sidebar: documentation_sidebar permalink: /documentation/examples/pokemon.html folder: documentation comment_issue_id: 27 --- This example uses the pokemon ontology and dataset, which is available from [github](https://github.com/graknlabs/grakn/blob/master/grakn-dist/src/examples/pokemon.gql) and is also in the *examples* folder in the Grakn distribution zip. You can use it as a primer to work out if you're building a Graql query correctly, to revise your knowledge or just walk through it as you learn Graql. First up, you need to load the ontology and dataset (which are all in one file) as follows. Here we will use the default keyspace, so will clean it before we start Grakn engine and open the Graql shell. ```bash <relative-path-to-Grakn>/bin/grakn.sh clean <relative-path-to-Grakn>/bin/grakn.sh start <relative-path-to-Grakn>/bin/graql.sh -f <relative-path-to-Grakn>/examples/pokemon.gql <relative-path-to-Grakn>/bin/graql.sh ``` Here are the queries. Type each one in a separate line and see what happens. Comments are marked with a #, so you can ignore them. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/projects.md --- title: Example Projects keywords: examples, projects last_updated: October 7, 2016 tags: summary: "A list of potential projects that use Grakn." sidebar: documentation_sidebar permalink: /documentation/examples/projects.html folder: documentation --- ## Introduction This page lists potential research projects that could be based upon Grakn. We have estimated the amount of work involved in them as T-shirt sizes (small, medium and large) and indicated the level of expertise we think would be required. If you are unsure of whether an idea would be suitable for you, please do talk to us. We are always happy to discuss your ideas, either based on these suggestions or for other projects. Just see our [Community page](https://grakn.ai/community.html) for ways to get in touch with us! ## Small The following would particularly suit an undergraduate project. ### Rule-based Game Solver Define a simple ontology, which is just a specification of the components of a system and how they relate, and then devise logical rules to construct a simple application. An example may be to produce a set of rules that can validate that a Sudoku puzzle has been filled correctly. ### Links Between Likes Add a new feature to Moogi that allows users to “like” concepts. Then produce a visualisation that displays the link between the things that a user likes. For more information about Moogi, the film discovery engine that is built on Grakn, please see [https://moogi.co/](https://moogi.co/). ### Moogi Data Cleanup Clean Moogi data and port it to Mindmaps. This requires dabbling a bit with Gremlin, scripting, exploring the graph and ETL probably. ### Graph Database Pros and Cons Write a review of Graph Database technologies with pros and cons and example apps. ### JavaScript Driver for Grakn Create a prototype to interface with the Grakn REST API in a simple manner. ### Create a Graph Representation of a Website Write a simple tool that follows links in a website and constructs a graph representation of it in Mindmaps. This could simply be pages and links, but could even pull out structural information such as containers and side panes inside pages. Follow on projects could be: - mapping click stream data onto the graph - and analysing - analysing clusters and shortest paths through the website. ### Graph a Subject's (e.g. Computer Science) Publications Build a graph of academic publications on a particular subject for analysis. ### Sports Results For example, a knowledge graph of the Rio Olympics stats dataset, Premier League results etc. ### Recommendation Engine Shows how to leverage connections in your data to gather insights and start recommending currently unrelated but relevant facts. ## Medium The following would particularly suit an undergraduate project. ### Game Search Engine Develop an application that shows how a graph database can excel over RDBMS for certain domains. For example, a game search engine that has features like search by: genre, ratings, price, keywords, etc. You could pick any area you are interested in (books, music, recipes - the list goes on - the main requirement is something with a lot of highly connected data). Once this project has been tried and tested, it may be interesting to abstract it further and look at building a factory that creates the basic search engine, and can then be customised according to subject area and features. ## Large The following would be most suitable for postgraduate investigation. ### Semantic Machine Learning Using Machine Learning to work with the semantics of the graph. ### Publication Topology Based Ranking Semantic search engine for computer science publications. How do ranking results based on topology compare to user data? Can we use an ensemble measure to get “better” results? ### Life Sciences Knowledge Graph There is potential to use the open data from Elixir and Bioschemas.org to build a life science graph. ### European Broadcast Union The European Broadcast Union are interested interested in semantics and have documented what they are pursuing [here](https://nlpdbpedia2016.files.wordpress.com/2016/09/nlpdbpedia2016_paper_1.pdf). ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. {% include links.html %}

## 1. Project Overview & Quickstart (itzg/docker-minecraft-server) ## File: README.md [](https://hub.docker.com/r/itzg/minecraft-server/) [](https://hub.docker.com/r/itzg/minecraft-server/) [](https://github.com/itzg/docker-minecraft-server/issues) [](https://discord.gg/DXfKpjB) [](https://github.com/itzg/docker-minecraft-server/actions/workflows/build.yml) [](https://www.buymeacoffee.com/itzg) [](https://docker-minecraft-server.readthedocs.io/en/latest/?badge=latest) [](https://docker-minecraft-server.readthedocs.io/) There you will find things like - [Quick start with Docker Compose](https://docker-minecraft-server.readthedocs.io/en/latest/#using-docker-compose) - Running [different versions of Minecraft](https://docker-minecraft-server.readthedocs.io/en/latest/versions/minecraft/) and using [various server types](https://docker-minecraft-server.readthedocs.io/en/latest/types-and-platforms/) for Java Edition - [Setting server properties via container environment variables](https://docker-minecraft-server.readthedocs.io/en/latest/configuration/server-properties/) - [Managing mods and plugins with automated downloads and cleanup](https://docker-minecraft-server.readthedocs.io/en/latest/mods-and-plugins/) - [Using various modpack providers/platforms](https://docker-minecraft-server.readthedocs.io/en/latest/types-and-platforms/) - ...and much more There are also many examples located in [the examples directory](examples) of this repo. This image only supports Java edition natively; however, if looking for a server that is compatible with Bedrock edition, then use [itzg/minecraft-bedrock-server](https://github.com/itzg/docker-minecraft-bedrock-server) or [refer to this section](https://docker-minecraft-server.readthedocs.io/en/latest/misc/examples/#bedrock-compatible-server) to add Bedrock compatibility to a Java edition server. [](#sponsors) [](https://spawnbox.app) [**SpawnBox**](https://spawnbox.app) - Powered by `itzg/minecraft-server`, it's a Windows desktop app for parents, teens, and friend groups who want a Minecraft server on their own PC without learning Docker, WSL2, or networking. [](https://server.pro) [**Server.pro**](https://server.pro) - A game server hosting platform offering one-click Minecraft server deployment powered by `itzg/minecraft-server`, with global locations and an easy-to-use control panel. [](https://seedloaf.com) [**Seedloaf**](https://seedloaf.com) - Built from scratch (by two nerds) on top of `itzg/minecraft-server`. Seedloaf offers free, and premium Minecraft servers where you have full control! [and more...](https://github.com/sponsors/itzg) --- ## File: docs/configuration/auto-rcon-commands.md # Auto-execute RCON commands RCON commands can be configured to execute when the server starts, a client connects, or a client disconnects. !!! note When declaring several commands within a compose file environment variable, it's easiest to use YAML's `|-` [block style indicator](https://yaml-multiline.info/). **On Server Start:** ``` yaml RCON_CMDS_STARTUP: |- gamerule doFireTick false pregen start 200 ``` **On Client Connection:** ``` yaml RCON_CMDS_ON_CONNECT: |- team join New @a[team=] ``` **Note:** * On client connect we only know there was a connection, and not who connected. RCON commands will need to be used for that. **On Client Disconnect:** ``` yaml RCON_CMDS_ON_DISCONNECT: |- gamerule doFireTick true ``` **On First Client Connect** ``` yaml RCON_CMDS_FIRST_CONNECT: |- pregen stop ``` **On Last Client Disconnect** ``` yaml RCON_CMDS_LAST_DISCONNECT: |- kill @e[type=minecraft:boat] pregen start 200 ``` **Example of rules for new players** Uses team NEW and team OLD to track players on the server. So move player with no team to NEW, run a command, move them to team OLD. [Reference Article](https://www.minecraftforum.net/forums/minecraft-java-edition/redstone-discussion-and/2213523-detect-players-first-join) ``` yaml RCON_CMDS_STARTUP: |- /pregen start 200 /gamerule doFireTick false /team add New /team add Old RCON_CMDS_ON_CONNECT: |- /team join New @a[team=] /give @a[team=New] birch_boat /team join Old @a[team=New] RCON_CMDS_FIRST_CONNECT: |- /pregen stop RCON_CMDS_LAST_DISCONNECT: |- /kill @e[type=minecraft:boat] /pregen start 200 ``` --- ## File: docs/configuration/interpolating.md --- title: Modifying config files --- ## Replacing variables inside configs Sometimes you have mods or plugins that require configuration information that is only available at deploy-time. For example if you need to configure a plugin to connect to a database, you don't want to include this information in your Git repository or Docker image. Or maybe you have some runtime information like the server name that needs to be set in your config files after the container starts. For those cases there is the option to replace defined variables inside your configs with environment variables defined at container runtime. When the environment variable `REPLACE_ENV_IN_PLACE` is set to `true` (the default), the startup script will go through all files inside the container's `/data` path and replace variables that match the container's environment variables. Variables can instead (or in addition to) be replaced in files sync'ed from `/plugins`, `/mods`, and `/config` by setting `REPLACE_ENV_DURING_SYNC` to `true` (defaults to `false`). Variables that you want to replace need to be declared inside curly brackets and prefixed with a dollar sign, such as `${CFG_YOUR_VARIABLE}`, which is same as many scripting languages. You can also change `REPLACE_ENV_VARIABLE_PREFIX`, which defaults to "CFG_", to limit which environment variables are allowed to be used. For example, with "CFG_" as the prefix, the variable `${CFG_DB_HOST}` would be substituted, but not `${DB_HOST}`. The prefix can be set to an empty string to allow for matching any variable name. If you want to use a file's content for value, such as when using secrets mounted as files, declare the placeholder named like normal in the file and declare an environment variable named the same but with the suffix `_FILE`. For example, a `my.cnf` file could contain: ```toml [client] password = ${CFG_DB_PASSWORD} ``` ...a secret declared in the compose file with: ```yaml secrets: db_password: external: true ``` ...and finally the environment variable would be named with a `_FILE` suffix and point to the mounted secret: ```yaml environment: CFG_DB_PASSWORD_FILE: /run/secrets/db_password ``` Variables will be replaced in files with the following extensions: `.yml`, `.yaml`, `.txt`, `.cfg`, `.conf`, `.properties`. Specific files can be excluded by listing their name (without path) in the variable `REPLACE_ENV_VARIABLES_EXCLUDES`. Paths can be excluded by listing them in the variable `REPLACE_ENV_VARIABLES_EXCLUDE_PATHS`. Path excludes are recursive. Here is an example: ``` REPLACE_ENV_VARIABLES_EXCLUDE_PATHS="/data/plugins/Essentials/userdata /data/plugins/MyPlugin" ``` Here is a full example where we want to replace values inside a `database.yml`. ```yaml --- database: host: ${CFG_DB_HOST} name: ${CFG_DB_NAME} password: ${CFG_DB_PASSWORD} ``` This is how your `compose.yaml` file could look like: ```yaml title="compose.yaml" # Other docker compose examples in /examples services: minecraft: image: itzg/minecraft-server ports: - "25565:25565" volumes: - "mc:/data" environment: EULA: "TRUE" ENABLE_RCON: "true" RCON_PASSWORD: "testing" RCON_PORT: 28016 # enable env variable replacement REPLACE_ENV_VARIABLES: "TRUE" # define an optional prefix for your env variables you want to replace ENV_VARIABLE_PREFIX: "CFG_" # and here are the actual variables CFG_DB_HOST: "http://localhost:3306" CFG_DB_NAME: "minecraft" CFG_DB_PASSWORD_FILE: "/run/secrets/db_password" volumes: mc: rcon: secrets: db_password: file: ./db_password ``` ## Patching existing files JSON path based patches can be applied to one or more existing files by setting the variable `PATCH_DEFINITIONS` to the path of a directory that contains one or more [patch definition json files](https://github.com/itzg/mc-image-helper#patchdefinition) or a [patch set json file](https://github.com/itzg/mc-image-helper#patchset). The `file` and `value` fields of the patch definitions may contain `${...}` variable placeholders. The allowed environment variables in placeholders can be restricted by setting `REPLACE_ENV_VARIABLE_PREFIX`, which defaults to "CFG_". The following example shows a patch-set file where various fields in the `paper.yaml` configuration file can be modified and added: ```json { "patches": [ { "file": "/data/paper.yml", "ops": [ { "$set": { "path": "$.verbose", "value": true } }, { "$set": { "path": "$.settings['velocity-support'].enabled", "value": "${CFG_VELOCITY_ENABLED}", "value-type": "bool" } }, { "$put": { "path": "$.settings", "key": "my-test-setting", "value": "testing" } } ] } ] } ``` Supports the file formats: - JSON - JSON5 - Yaml - TOML, but processed output is not pretty --- ## File: docs/configuration/jvm-options.md # JVM Options ## Memory Limit By default, the image declares an initial and maximum Java memory-heap limit of 1 GB. There are several ways to adjust the memory settings: - `MEMORY`: "1G" by default, can be used to adjust both initial (`Xms`) and max (`Xmx`) memory heap settings of the JVM - `INIT_MEMORY`: independently sets the initial heap size - `MAX_MEMORY`: independently sets the max heap size The values of all three are passed directly to the JVM and support format/units as `[g|G|m|M|k|K]`. To have control over heap size, without relying on absolute memory sizes percentages are also supported using `%`. !!! info "RAMPercentage parameters" Percentage based heap sizing is enabled using `-XX:InitialRAMPercentage` for `INIT_MEMORY` and `-XX:MaxRAMPercentage` for `MAX_MEMORY`. For details on the function of these parameters look [here](https://www.baeldung.com/java-jvm-parameters-rampercentage). !!! example "Using docker run" ``` -e MEMORY=2G ``` or to use init and max memory: ``` -e INIT_MEMORY=1G -e MAX_MEMORY=4G ``` !!! example "Using compose file" ``` environment: MEMORY: 2G ``` or to use init and max memory: ``` environment: INIT_MEMORY: 1G MAX_MEMORY: 4G ``` To let the JVM calculate the heap size from the container declared memory limit, unset `MEMORY` with an empty value, such as `-e MEMORY=""`. By default, the JVM will use 25% of the container memory limit as the heap limit. !!! important The settings above only set the Java **heap** limits. Memory resource requests and limits on the overall container should also account for non-heap memory usage. An extra 25% is [a general best practice](https://dzone.com/articles/best-practices-java-memory-arguments-for-container). ## Extra JVM Options General JVM options can be passed to the Minecraft Server invocation by passing a `JVM_OPTS` environment variable. If needing to set `-XX` options, then pass those with the `JVM_XX_OPTS` variable instead. Both variables accept space-delimited, raw JVM arguments. ``` docker run ... -e JVM_OPTS="-XsomeJVMOption -DpropName=value" ... ``` **NOTE** When declaring `JVM_OPTS` in a compose file's `environment` section with list syntax, **do not** include the quotes: ```yaml environment: - EULA=true - JVM_OPTS=-XsomeJVMOption -DpropName=value ``` Using object syntax is recommended and more intuitive: ```yaml environment: EULA: "true" JVM_OPTS: "-XsomeJVMOption -DpropName=value" # or without quotes # JVM_OPTS: -XsomeJVMOption -DpropName=value ``` As a shorthand for passing several system properties as `-D` arguments, you can instead pass a comma separated list of `name=value` or `name:value` pairs with `JVM_DD_OPTS`. (The colon syntax is provided for management platforms like Plesk that don't allow `=` inside a value.) For example, instead of passing ```yaml JVM_OPTS: -Dfml.queryResult=confirm -Dname=value ``` you can use ```yaml JVM_DD_OPTS: fml.queryResult=confirm,name=value ``` ## Enable Remote JMX for Profiling To enable remote JMX, such as for profiling with VisualVM or JMC, set the environment variable `ENABLE_JMX` to "true", set `JMX_HOST` to the IP/host running the Docker container, and add a port forwarding of TCP port 7091, such as: !!! example With `docker run` ``` -e ENABLE_JMX=true -e JMX_HOST=$HOSTNAME -p 7091:7091 ``` If needing to map to a different port, then also set the environment variable `JMX_PORT` to the desired host port. !!! example With a compose file: ```yaml environment: ENABLE_JMX: true JMX_HOST: ${HOSTNAME} JMX_PORT: "7092" ports: - "7092:7092" ``` ## Enable Aikar's Flags [Aikar has done some research](https://aikar.co/2018/07/02/tuning-the-jvm-g1gc-garbage-collector-flags-for-minecraft/) into finding the optimal JVM flags for GC tuning, which becomes more important as more users are connected concurrently. [PaperMC also has an explanation](https://docs.papermc.io/paper/aikars-flags) of what the JVM flags are doing. The set of flags documented there can be added using -e USE_AIKAR_FLAGS=true When `MEMORY` is greater than or equal to 12G, then the Aikar flags will be adjusted according to the article. ## Enable MeowIce's Flags [MeowIce has created an updated set of JVM flags](https://github.com/MeowIce/meowice-flags?tab=readme-ov-file#why-would-i-have-to-switch-) based on Aikar's flags but with support for optimizations for Java 17 and above The set of flags documented there can be added by setting the environment variable `USE_MEOWICE_FLAGS` to `true`. There is an optional `USE_MEOWICE_GRAALVM_FLAGS` variable to enable GraalVM specific optimizations, defaults to `FALSE`. --- ## File: docs/configuration/misc-options.md ## Running with a custom server JAR If you would like to run a custom server JAR, set `-e TYPE=CUSTOM` and pass the custom server JAR via `CUSTOM_SERVER`. It can either be a URL or a container path to an existing JAR file. If it is a URL, it will only be downloaded into the `/data` directory if it wasn't already. As such, if you need to upgrade or re-download the JAR, then you will need to stop the container, remove the file from the container's `/data` directory, and start again. ## Force re-download of the server file For VANILLA, FORGE, BUKKIT, SPIGOT, PAPER, CURSEFORGE, SPONGEVANILLA server types, set `$FORCE_REDOWNLOAD` to some value (e.g. 'true) to force a re-download of the server file for the particular server type. by adding a `-e FORCE_REDOWNLOAD=true` to your command-line. For example, with Paper, it would look something like this: ``` docker run -d --pull=always \ -v /path/on/host:/data \ -e TYPE=PAPER -e FORCE_REDOWNLOAD=true \ -p 25565:25565 -e EULA=TRUE --name mc itzg/minecraft-server ``` ## Clean server libraries By default, supported server types remove stale server libraries during installation by setting `CLEAN_SERVER_LIBRARIES` to `true`. Set `CLEAN_SERVER_LIBRARIES` to `false` to disable this cleanup if it causes unexpected behavior. Currently, library cleanup is supported by - `TYPE=PAPER` - `TYPE=PURPUR` ## Running as alternate user/group ID By default, the container will switch to and run the Minecraft server as user ID 1000 and group ID 1000; however, that can be changed by setting the environment variables `UID` and `GID`. The startup will also skip user switching if the `--user`/`-u` argument is passed to `docker run` or `user` is set on the compose service. ## Extra Arguments Arguments that would usually be passed to the jar file (those which are written after the filename) can be passed via the `EXTRA_ARGS` environment variable. See [Custom worlds directory path](../misc/world-data.md#custom-worlds-directory-path) for an example. ## Interactive and Color Console When RCON is enabled, which is the default, and [TTY](https://docs.docker.com/compose/compose-file/05-services/#tty) is enabled on the container, then some server types will output colorized logs and provide a fully interactive console. To access the interactive console, use [`docker attach`](https://docs.docker.com/engine/reference/commandline/container_attach/) (not `exec`). When finished, make sure to use the sequence Control-P, Control-Q to detach without stopping the container. If this behavior interferes with the log content, then disable TTY or remove the setting entirely since the default is disabled. In a compose file, set the service's `tty` parameter to `false`. On the `docker run` command-line remove the `-t` argument. ## Server Shutdown Options To allow time for players to finish what they're doing during a graceful server shutdown, set `STOP_SERVER_ANNOUNCE_DELAY` to a number of seconds to delay after an announcement is posted by the server. To set a custom command to run at the start of this delay period, set `STOP_SERVER_DELAY_COMMAND` to the full command. This will run in place of the announcement. !!! warning "Increase stop grace period" The Docker stop grace period must be increased to a value longer than the announce delay. The value to use that is longer than announce delay will vary based upon the amount of time it takes for final world data saving. If the container exits with exit code 137, then that indicates a longer grace period is needed. The grace period can be increased using [the -t option on `docker compose down`](https://docs.docker.com/compose/reference/down/) or set the [stop_grace_period](https://docs.docker.com/compose/compose-file/05-services/#stop_grace_period) in the compose file. The `STOP_SERVER_ANNOUNCE_DELAY` can be bypassed by sending a `SIGUSR1` signal to the `mc-server-runner` process. `docker`: docker stop --signal SIGUSR1 mc `docker compose`: docker compose kill --signal SIGUSR1 ## Configuration Options for Minecraft Server Health Monitoring The image tags include specific variables to simplify configuration for monitoring the health of a Minecraft server: - `-e SERVER_HOST=localhost` : This variable sets the host address of the Minecraft server to be monitored. By default, it is set to `localhost`, but you can replace it with the actual hostname or IP address of your Minecraft server. - `-e SERVER_PORT=25565` : This variable sets the port number on which the Minecraft server is running. By default, Minecraft servers run on port 25565, but if your server is configured to use a different port, you should replace `25565` with the correct port number. This helps the monitoring system to accurately check the health status of the Minecraft server on the specified port. ## OpenJ9 Specific Options The openj9 image tags include specific variables to simplify configuration: - `-e TUNE_VIRTUALIZED=TRUE` : enables the option to [optimize for virtualized environments](https://www.eclipse.org/openj9/docs/xtunevirtualized/) - `-e TUNE_NURSERY_SIZES=TRUE` : configures nursery sizes where the initial size is 50% of the `MAX_MEMORY` and the max size is 80%. ## Customize Minecraft server logging The image can use a templated log4j2 configuration based on PaperMC's logging setup, which is automatically applied for versions that don't require Log4j security patches. This configuration provides rolling logs and advanced logging features by default. Set the environment variable `GENERATE_LOG4J2_CONFIG` to "true" to enable the following features. !!! note This configures the server logging only. To diagnose container startup, set the `DEBUG` environment variable to "true". ### Customization via environment variables You can customize various aspects of the logging behavior using environment variables: - `LOG_LEVEL` : Root logger level (default: `info`) ``` -e LOG_LEVEL=debug ``` - `ROLLING_LOG_FILE_PATTERN` : Pattern for rolled log file names (default: `logs/%d{yyyy-MM-dd}-%i.log.gz`) ``` -e ROLLING_LOG_FILE_PATTERN="logs/archive/%d{yyyy-MM-dd}-%i.log.gz" ``` - `ROLLING_LOG_MAX_FILES` : Maximum number of archived log files to keep (default: `1000`) ``` -e ROLLING_LOG_MAX_FILES=30 ``` ### Customizing log message formats For full control over how log messages are formatted, you can customize the Log4j2 pattern layouts using these variables. These use [Log4j2 Pattern Layout syntax](https://logging.apache.org/log4j/2.x/manual/layouts.html#PatternLayout): - `LOG_CONSOLE_FORMAT` : Format for console output (what you see in `docker logs`) Default: `[%d{HH:mm:ss}] [%t/%level]: %msg%n` - `LOG_FILE_FORMAT` : Format for file logs (written to `logs/latest.log`) Default: `[%d{HH:mm:ss}] [%t/%level]: %msg%n` - `LOG_TERMINAL_FORMAT` : Format for interactive terminal console (used with `docker attach`) Default: `[%d{HH:mm:ss} %level]: %msg%n` ### Example configurations Simple timestamp customization (most common use case): ```yaml environment: # What you see in docker logs LOG_CONSOLE_FORMAT: "[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%t/%level]: %msg%n" # What gets written to logs/latest.log LOG_FILE_FORMAT: "[%d{yyyy-MM-dd HH:mm:ss.SSS}] [%t/%level]: %msg%n" ``` Advanced customization: ```yaml environment: LOG_LEVEL: debug # Custom ISO8601 format with logger names LOG_CONSOLE_FORMAT: "%d{ISO8601} %-5level [%t] %logger{36} - %msg%n" LOG_FILE_FORMAT: "%d{ISO8601} %-5level [%t] %logger{36} - %msg%n" ROLLING_LOG_MAX_FILES: 50 ``` ### Legacy ENABLE_ROLLING_LOGS option The `ENABLE_ROLLING_LOGS` environment variable is no longer needed for most use cases, as rolling logs are now enabled by default through the templated configuration. This option is maintained for backward compatibility but is only checked for error reporting when rolling logs cannot be used due to Log4j security patches. > **NOTE** The templated log4j2 configuration may interfere with interactive/color consoles [as described in the section above](#interactive-and-color-console) ## Timezone Configuration You can configure the timezone to match yours by setting the `TZ` environment variable: -e TZ=Europe/London such as: docker run -d -it --pull=always -e TZ=Europe/London -p 25565:25565 --name mc itzg/minecraft-server Or mounting `/etc/timezone` as readonly (not supported on Windows): -v /etc/timezone:/etc/timezone:ro such as: docker run -d -it --pull=always -v /etc/timezone:/etc/timezone:ro -p 25565:25565 --name mc itzg/minecraft-server ## HTTP Proxy You may configure the use of an HTTP/HTTPS proxy by passing the proxy's "host:port" via the environment variable `PROXY`. In [the example compose file](https://github.com/itzg/docker-minecraft-server/blob/master/examples/proxied/compose.yml) it references a Squid proxy. The host and port can be separately passed via the environment variables `PROXY_HOST` and `PROXY_PORT`. A `|` delimited list of hosts to exclude from proxying can be passed via `PROXY_NON_PROXY_HOSTS`. ## Prefer IPv6 Addresses and Stack The image can be configured to prefer IPv6 addresses and stack by setting the environment variable `PREFER_IPV6` to "true". This will ensure the Java properties `java.net.preferIPv6Addresses` and `java.net.preferIPv6Stack` are set accordingly for mc-image-helper and the Minecraft server process. ## Using "noconsole" option Some older versions (pre-1.14) of Spigot required `--noconsole` to be passed when detaching stdin, which can be done by setting `-e CONSOLE=FALSE`. ## Explicitly disable GUI Some older servers get confused and think that the GUI interface is enabled. You can explicitly disable that by passing `-e GUI=FALSE`. ## Stop Duration When the container is signaled to stop, the Minecraft process wrapper will attempt to send a "stop" command via RCON or console and waits for the process to gracefully finish. By default, it waits 60 seconds, but that duration can be configured by setting the environment variable `STOP_DURATION` to the number of seconds. Be sure to also increase the shutdown timeout described [here for docker compose down](https://docs.docker.com/reference/cli/docker/compose/down/#options) and [here for docker stop](https://docs.docker.com/reference/cli/docker/container/stop/#options). ## Setup only If you are using a host-attached data directory, then you can have the image setup the Minecraft server files and stop prior to launching the server process by setting `SETUP_ONLY` to `true`. ## Custom server runner By default, the image finishes startup by exec'ing [mc-server-runner](https://github.com/itzg/mc-server-runner) to run the Minecraft server itself. This can be replaced by setting the environment variable `SERVER_RUNNER`. The arguments passed will start with ones specific to mc-server-runner, followed by `--`, and then the actual `java -jar` arguments or equivalent server entry script, such as `run.sh` for Forge. > [!TIP] > The `SERVER_RUNNER` variable is expanded when passed to `exec` so that `mc-server-runner` can be wrapped by another executable that is mounted into the container. ## Enable Flare Flags To enable the JVM flags required to fully support the [Flare profiling suite](https://blog.airplane.gg/flare), set the following variable: -e USE_FLARE_FLAGS=true Flare is built-in to Pufferfish/Purpur, and is available in [plugin form](https://github.com/TECHNOVE/FlarePlugin) for other server types. ## Enable support for optimized SIMD operations To enable support for optimized SIMD operations, the JVM flag can be set with the following variable: -e USE_SIMD_FLAGS=true SIMD optimized operations are supported by Pufferfish and Purpur. For example, to download configuration files for plugins: ```yaml environment: DOWNLOAD_EXTRA_CONFIGS: | plugins/WorldEdit- { "layers": [ { "block": "minecraft:bedrock", "height": 1 }, { "block": "minecraft:stone", "height": 2 }, { "block": "minecraft:sandstone", "height": 15 } ], "biome": "minecraft:desert" } ``` For more details, refer to the Minecraft Wiki sections for [Superflat Multiplayer](https://minecraft.wiki/w/Superflat#Multiplayer) and [generator options tag format](https://minecraft.wiki/w/Java_Edition_level_format#generatorOptions_tag_format). ### Custom Server Resource Pack You can set a link to a custom resource pack and set it's checksum using the `RESOURCE_PACK` and `RESOURCE_PACK_SHA1` options respectively, the default is blank: docker run -d -e 'RESOURCE_PACK=http://link.com/to/pack.zip?=1' -e 'RESOURCE_PACK_SHA1=d5db29cd03a2ed055086cef9c31c252b4587d6d0' You can enforce the resource pack on clients by setting `RESOURCE_PACK_ENFORCE` to `TRUE` (default: `FALSE`). ### Level / World Save Name You can either switch between world saves or run multiple containers with different saves by using the `LEVEL` option, where the default is "world": docker run -d -e LEVEL=bonus ... > **NOTE:** if running multiple containers be sure to either specify a different `-v` host directory for each `LEVEL` in use or don't use `-v` and the container's filesystem will keep things encapsulated. > **INFO** Refer to the [data directory](../data-directory.md) section for a visual description of where the `$LEVEL` directory is situated. ### Server port > **WARNING:** only change this value if you know what you're doing. It only needs to be changed when using host-networking and it is rare that host networking should be used. Use `-p` port mappings instead. If you must, the server port can be set like: !!! example "Using docker run" ``` docker run -d -e SERVER_PORT=25566 ... ``` !!! example "Using compose" ```yaml environment: SERVER_PORT: 25566 ``` **however**, be sure to change your port mapping accordingly and be prepared for some features to break. ### Custom server properties Some mods/plugins utilize custom `server.properties` entries which can be declared via the `CUSTOM_SERVER_PROPERTIES` environment variable. The contents must be newline delimited `name=value` pairs. Within a compose file, newline delimited entries can be declared as shown here: ```yaml CUSTOM_SERVER_PROPERTIES: | custom1=value1 defaultworldgenerator-port=f8c04631-f744-11ec-b260-f02f74b094e0 ``` When using `docker run` from a bash shell, the entries must be quoted with the `$'` syntax, such as ``` -e CUSTOM_SERVER_PROPERTIES=$'k1=v1\nk2=v2' ``` ### Other server property mappings !!! warning "Version compatibility" Not all server properties are supported by all versions of Minecraft. Since this image supports a wide range of versions, please consult the [server properties documentation](https://minecraft.wiki/w/Server.properties) for the version you are using. | Environment Variable | Server Property | |-----------------------------------------|-------------------------------------------------------------------------------------------------------------------------------| | ACCEPTS_TRANSFERS | [accepts-transfers](https://minecraft.wiki/w/Server.properties#accepts-transfers) | | ALLOW_FLIGHT | [allow-flight](https://minecraft.wiki/w/Server.properties#allow-flight) | | ALLOW_NETHER | [allow-nether](https://minecraft.wiki/w/Server.properties#allow-nether) | | ANNOUNCE_PLAYER_ACHIEVEMENTS | [announce-player-achievements](https://minecraft.wiki/w/Server.properties#announce-player-achievements) | | BROADCAST_CONSOLE_TO_OPS | [broadcast-console-to-ops](https://minecraft.wiki/w/Server.properties#broadcast-console-to-ops) | | BROADCAST_RCON_TO_OPS | [broadcast-rcon-to-ops](https://minecraft.wiki/w/Server.properties#broadcast-rcon-to-ops) | | BUG_REPORT_LINK | [bug-report-link](https://minecraft.wiki/w/Server.properties#bug-report-link) | | ENABLE_COMMAND_BLOCK | [enable-command-block](https://minecraft.wiki/w/Server.properties#enable-command-block) | | ENABLE_STATUS | [enable-status](https://minecraft.wiki/w/Server.properties#enable-status) | | ENFORCE_SECURE_PROFILE | [enforce-secure-profile](https://minecraft.wiki/w/Server.properties#enforce-secure-profile) | | ENTITY_BROADCAST_RANGE_PERCENTAGE | [entity-broadcast-range-percentage](https://minecraft.wiki/w/Server.properties#entity-broadcast-range-percentage) | | FORCE_GAMEMODE | [force-gamemode](https://minecraft.wiki/w/Server.properties#force-gamemode) | | FUNCTION_PERMISSION_LEVEL | [function-permission-level](https://minecraft.wiki/w/Server.properties#function-permission-level) | | GENERATE_STRUCTURES | [generate-structures](https://minecraft.wiki/w/Server.properties#generate-structures) | | GENERATOR_SETTINGS | [generator-settings](https://minecraft.wiki/w/Server.properties#generator-settings) | | HARDCORE | [hardcore](https://minecraft.wiki/w/Server.properties#hardcore) | | HIDE_ONLINE_PLAYERS | [hide-online-players](https://minecraft.wiki/w/Server.properties#hide-online-players) | | LOG_IPS | [log-ips](https://minecraft.wiki/w/Server.properties#log-ips) | | MANAGEMENT_SERVER_ALLOWED_ORIGINS | [management-server-allowed-origins](https://minecraft.wiki/w/Server.properties#management-server-allowed-origins) | | MANAGEMENT_SERVER_ENABLED | [management-server-enabled](https://minecraft.wiki/w/Server.properties#management-server-enabled) | | MANAGEMENT_SERVER_HOST | [management-server-host](https://minecraft.wiki/w/Server.properties#management-server-host) | | MANAGEMENT_SERVER_PORT | [management-server-port](https://minecraft.wiki/w/Server.properties#management-server-port) | | MANAGEMENT_SERVER_SECRET | [management-server-secret](https://minecraft.wiki/w/Server.properties#management-server-secret) | | MANAGEMENT_SERVER_TLS_ENABLED | [management-server-tls-enabled](https://minecraft.wiki/w/Server.properties#management-server-tls-enabled) | | MANAGEMENT_SERVER_TLS_KEYSTORE | [management-server-tls-keystore](https://minecraft.wiki/w/Server.properties#management-server-tls-keystore) | | MANAGEMENT_SERVER_TLS_KEYSTORE_PASSWORD | [management-server-tls-keystore-password](https://minecraft.wiki/w/Server.properties#management-server-tls-keystore-password) | | MAX_BUILD_HEIGHT | [max-build-height](https://minecraft.wiki/w/Server.properties#max-build-height) | | MAX_COMMAND_CHAIN_LENGTH | [max-command-chain-length](https://minecraft.wiki/w/Server.properties#max-command-chain-length) | | MAX_ENTITY_CRAMMING | [max-entity-cramming](https://minecraft.wiki/w/Server.properties#max-entity-cramming) | | MAX_ENTITY_COLLISION_RADIUS | [max-entity-collision-radius](https://minecraft.wiki/w/Server.properties#max-entity-collision-radius) | | MAX_FUNCTION_CHAIN_DEPTH | [max-function-chain-depth](https://minecraft.wiki/w/Server.properties#max-function-chain-depth) | | MAX_NEIGHBORS | [max-neighbors](https://minecraft.wiki/w/Server.properties#max-neighbors) | | MAX_CHAINED_NEIGHBOR_UPDATES | [max-chained-neighbor-updates](https://minecraft.wiki/w/Server.properties#max-chained-neighbor-updates) | | MAX_PLAYERS | [max-players](https://minecraft.wiki/w/Server.properties#max-players) | | MAX_TICK_TIME | [max-tick-time](https://minecraft.wiki/w/Server.properties#max-tick-time) | | MAX_WORLD_SIZE | [max-world-size](https://minecraft.wiki/w/Server.properties#max-world-size) | | NETWORK_COMPRESSION_THRESHOLD | [network-compression-threshold](https://minecraft.wiki/w/Server.properties#network-compression-threshold) | | ONLINE_MODE | [online-mode](https://minecraft.wiki/w/Server.properties#online-mode) | | OP_PERMISSION_LEVEL | [op-permission-level](https://minecraft.wiki/w/Server.properties#op-permission-level) | | PAUSE_WHEN_EMPTY_SECONDS | [pause-when-empty-seconds](https://minecraft.wiki/w/Server.properties#pause-when-empty-seconds) | | PLAYER_IDLE_TIMEOUT | [player-idle-timeout](https://minecraft.wiki/w/Server.properties#player-idle-timeout) | | PREVENT_PROXY_CONNECTIONS | [prevent-proxy-connections](https://minecraft.wiki/w/Server.properties#prevent-proxy-connections) | | PVP | [pvp](https://minecraft.wiki/w/Server.properties#pvp) | | RATE_LIMIT | [rate-limit](https://minecraft.wiki/w/Server.properties#rate-limit) | | REGION_FILE_COMPRESSION | [region-file-compression](https://minecraft.wiki/w/Server.properties#region-file-compression) | | RESOURCE_PACK_ID | [resource-pack-id](https://minecraft.wiki/w/Server.properties#resource-pack-id) | | RESOURCE_PACK_PROMPT | [resource-pack-prompt](https://minecraft.wiki/w/Server.properties#resource-pack-prompt) | | SERVER_NAME | [server-name](https://minecraft.wiki/w/Server.properties#server-name) | | SIMULATION_DISTANCE | [simulation-distance](https://minecraft.wiki/w/Server.properties#simulation-distance) | | SNOOPER_ENABLED | [snooper-enabled](https://minecraft.wiki/w/Server.properties#snooper-enabled) | | SPAWN_ANIMALS | [spawn-animals](https://minecraft.wiki/w/Server.properties#spawn-animals) | | SPAWN_MONSTERS | [spawn-monsters](https://minecraft.wiki/w/Server.properties#spawn-monsters) | | SPAWN_NPCS | [spawn-npcs](https://minecraft.wiki/w/Server.properties#spawn-npcs) | | SPAWN_PROTECTION | [spawn-protection](https://minecraft.wiki/w/Server.properties#spawn-protection) | | STATUS_HEARTBEAT_INTERVAL | [status-heartbeat-interval](https://minecraft.wiki/w/Server.properties#status-heartbeat-interval) | | SYNC_CHUNK_WRITES | [sync-chunk-writes](https://minecraft.wiki/w/Server.properties#sync-chunk-writes) | | USE_NATIVE_TRANSPORT | [use-native-transport](https://minecraft.wiki/w/Server.properties#use-native-transport) | | VIEW_DISTANCE | [view-distance](https://minecraft.wiki/w/Server.properties#view-distance) | --- ## File: docs/misc/deployment/index.md # More Deployment Info ## On Kubernetes ### Using Helm - itzg Helm Chart: - [GitHub repo](https://github.com/itzg/minecraft-server-charts) - [Helm Chart repo](https://itzg.github.io/minecraft-server-charts/) - [mcsh/server-deployment](https://github.com/mcserverhosting-net/charts) ### Using Shulker [Shulker](https://github.com/jeremylvln/Shulker) is a Kubernetes operator for managing complex and dynamic Minecraft infrastructures, including game servers and proxies. It uses the docker-minecraft-server and docker-bungeecord images under-the-hood. ## With Ansible [Ansible](https://docs.ansible.com/ansible/latest/getting_started/introduction.html) is an open-source task automation tool built in [Python](https://www.python.org/). Ansible playbooks can be used to automate all kinds of tasks, including deploying remote Minecraft servers. ### Using the MASH playbook [The MASH Playbook](https://github.com/mother-of-all-self-hosting/mash-playbook) is a premade playbook with the option to deploy a [wide variety of open-source services](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/supported-services.md) to your server(s), including [docker-minecraft-server](https://github.com/XHawk87/ansible-role-minecraft), making it a good option if you want all the bells and whistles alongside your Minecraft server. - Check out the [Installation Guide](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/README.md) for the MASH playbook to get started. - You can then enable [Minecraft](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/minecraft.md) in your vars.yml. - Enable any supporting services that you might find useful, such as [user authentication](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/authelia.md), [remote backups](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/backup-borg.md), [email relay](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/exim-relay.md), [cron monitoring](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/healthchecks.md), [audio and video conferencing](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/jitsi.md), databases ([MariaDB](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/mariadb.md), [PostgresSQL](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/postgres.md)), [push notifications](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/ntfy.md), [uptime monitoring](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/uptime-kuma.md), [a website](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/wordpress.md), as well as installing any [extra files, folders, applications, services and running commands](https://github.com/mother-of-all-self-hosting/mash-playbook/blob/main/docs/services/auxiliary.md) automatically on deployment. ## On CloudFormation (AWS) If you're looking for a simple way to deploy this to the Amazon Web Services Cloud, check out the [Minecraft Server Deployment (CloudFormation) repository](https://github.com/vatertime/minecraft-spot-pricing). This repository contains a CloudFormation template that will get you up and running in AWS in a matter of minutes. Optionally it uses Spot Pricing so the server is very cheap, and you can easily turn it off when not in use. ## Supporting Articles Below are supporting articles for server deployment. - "Zero to Minecraft Server with Docker Desktop and Compose" https://dev.to/rela-v/zero-to-minecraft-server-with-docker-desktop-and-compose-500a - This is a reference guide/tutorial on how to set up a vanilla Minecraft server using this project, including step-by-step instructions, and information on topics such as port-forwarding. --- ## File: docs/misc/contributing/development.md ## Adding a server type Adding a new server `TYPE` can vary due to the complexity of obtaining and configuring each type; however, the addition of any server type includes at least the following steps: 1. Copy an existing "start-deploy*" script, such as [start-deployFabric](https://github.com/itzg/docker-minecraft-server/blob/master/scripts/start-deployFabric) and rename it accordingly making sure to retain the "start-deploy" prefix 2. Modify the type-specific behavior between the "start-utils" preamble and the hand-off to `start-setupWorld` at the end of the script 3. Develop and test the changes using the [iterative process described below](#iterative-script-development) 4. Add a case-entry to the `case "${TYPE^^}"` in [start-configuration](https://github.com/itzg/docker-minecraft-server/blob/master/scripts/start-configuration) 5. Add a section to the [docs](https://github.com/itzg/docker-minecraft-server/tree/master/docs). It is recommended to copy-modify an existing section to retain a similar wording and level of detail 6. [Submit a pull request](https://github.com/itzg/docker-minecraft-server/pulls) ## Iterative script development The included `compose-dev.yml` will mount the local `scripts` code into the container and allow for iterative development. Replace `[-e key=value]` with any environment variables you wish to set for testing the modified scripts. ```shell docker compose -f compose-dev.yml run --rm -it [-e key=value] mc-dev ``` !!! tip To speed up the development cycle, it is recommended to set `SETUP_ONLY` to `true` as part of the run command above. ## Building the image with a new release of a tool In this exapmle, let's say that [mc-image-helper](https://github.com/itzg/mc-image-helper) has been [released](https://github.com/itzg/mc-image-helper/releases) at 1.56.0, but the corresponding changes in the image [scripts](https://github.com/itzg/docker-minecraft-server/tree/23205471db9814cff9c6602361dbc6cdd6c4230a/scripts) need to be tested against that version while updating [the Dockerfile](https://github.com/itzg/docker-minecraft-server/blob/23205471db9814cff9c6602361dbc6cdd6c4230a/Dockerfile#L58). ```yaml title="tests/manual/optional-projects/compose.yml" hl_lines="7" services: mc: build: # ...or wherever you cloned the docker-minecraft-server repo context: ../../.. args: MC_HELPER_VERSION: 1.56.0 environment: EULA: true TYPE: "FABRIC" MODRINTH_PROJECTS: | fabric-api pl3xmap?:beta ports: - "25565:25565/tcp" volumes: - ./data:/data ``` ## Using development copy of tools In the cloned repo, such as [`mc-image-helper`](https://github.com/itzg/mc-image-helper), install the distribution locally by running: ```shell ./gradlew installDist ``` The distribution will be installed in the project's `build/install/mc-image-helper`. Obtain the absolute path to that directory use in the next step. Refer to the instructions above to mount any locally modified image scripts or build a local copy of the image using or with alternate `BASE_IMAGE`, as described above: ```shell docker build -t itzg/minecraft-server . ``` Mount the local mc-image-helper distribution directory as a volume in the container at the path `/usr/share/mc-image-helper`, such as ```shell docker run -it --rm \ -v /path/to/mc-image-helper/build/install/mc-image-helper:/usr/share/mc-image-helper \ -e EULA=true \ itzg/minecraft-server ``` In a compose file, include the volume mount in the `volumes` section of the container definition: ```yaml services: mc: # ... usual container definition volumes: - /path/to/mc-image-helper/build/install/mc-image-helper:/usr/share/mc-image-helper:ro ``` For Go base tools, run ```shell goreleaser release --snapshot --clean ``` Clone [itzg/github-releases-proxy](https://github.com/itzg/github-releases-proxy) and run it according to the instructions shown there. In the Docker build, configure the following ```shell --build-arg GITHUB_BASEURL=http://host.docker.internal:8080 \ --build-arg APPS_REV=1 ``` and declare one or more version overrides, such as ``` --build-arg MC_HELPER_VERSION=1.8.1-SNAPSHOT ``` ## Generating release notes The following git command can be used to provide the bulk of release notes content: ```shell script git log --invert-grep --grep "^ci:" --grep "^misc:" --grep "^docs:" --pretty="* %s" 1.1.0..1.2.0 ``` --- ## File: docs/misc/contributing/docs.md # Site documentation The documentation for this image/repository is written in markdown and built using [Zensical](https://zensical.org/) into a documentation website hosted at [Read the Docs](https://readthedocs.org/). [Here is general information about writing Zensical markdown](https://zensical.org/docs/authoring/markdown/). !!! note The README.md rarely needs to be modified and only serves as a brief introduction to the project. The documentation source is maintained in the [docs](https://github.com/itzg/docker-minecraft-server/tree/master/docs) folder and is organized into sections by directory and files. Look through the existing content to determine if an existing file should be updated or a new file/directory added. It will be very helpful to view the rendered documentation as you're editing. To do that run the following from the top-level directory: ```shell docker compose -f compose-docs.yml -p zensical up ``` You can access the live documentation rendering at . ## 2. Official Technical Reference & Guides (itzg/docs) ## File: README.md # GRAKN.AI Documentation [](https://www.codetriage.com/graknlabs/docs) ## Dependencies You need to install the following dependencies to be able to build HTML pages in the documentation repository. **NOTE:** this is *not* necessary for updating the documentation itself. 1. Bundle; you will need to install `bundler` through your package manager of choice. **Arch Linux** ``` $ yaourt -S ruby-bundler ``` **OSX** ``` $ brew install bundler ``` **Ruby Gems (generic)** ``` $ gem install bundler ``` 2. Rake; this is used to automate the rest of the site building process. ``` $ gem install rake ``` With `rake` installed you can now install all other dependencies: ``` $ rake dependencies ``` ## Environment variables There is an environment variable that needs to be set depending on the build you wish to commence: `urlprefix`. If you only wish to preview the generated documentation itself; as server from this repository rather than integrated into the GRAKN.AI website you do not need to do anything. ##### `urlprefix` For local builds (local web server with documentation only), do not set this variable, or make sure it has been cleared with: ``` $ export urlprefix="" ``` For integrating to GRAKN.AI website, or elsewhere set as needed. For example; `www.grakn.ai/docs/` ``` $ export urlprefix=/docs ``` ## Building You can generate the documentation HTML by running the following in the repository top level. ``` $ rake build Generating... done in 1.503 seconds. Auto-regeneration: disabled. Use --watch to enable. $ ``` This will build the documentation site in `_jekyll` and create a symlink `_site` in the repository top level directory which will contain all the generated content. ## Cleaning Clean by running the following command in the repository top level: ``` $ rake clean ``` This will remove all generated files. ## Serving You can also build and server the generated HTML files in one command. A web server will be started listening on `localhost` (127.0.0.1) on port 4005 ``` $ rake serve Server address: http://127.0.0.1:4005/ Server running... press ctrl-c to stop. $ ``` You can now view the documentation by navigating your web browser to `http://127.0.0.1:4005/` --- ## File: pages/documentation/11-examples/analytics.md --- title: Using Grakn for Statistical Analysis keywords: analytics, machine-learning last_updated: March 2017 tags: [analytics, examples] summary: "How to use Grakn for statistical analysis" sidebar: documentation_sidebar permalink: /documentation/examples/analytics.html folder: documentation comment_issue_id: 27 --- ## Introduction This example illustrates the use of `aggregate` and `compute` queries in Graql using a simple dataset to calculate statistics. The code for the example can be found on our [sample-projects](https://github.com/graknlabs/sample-projects/tree/master/example-analytics-mtcars) repository, and is also included in the *examples* folder of the Grakn distribution from v0.12.0. For a detailed overview of calculating statistics using Graql, we recommend that you take a look at the documentation for: * [aggregate queries](../graql/aggregate-queries.html) * [compute queries](../graql/compute-queries.html) ## Data This example takes a dataset that will be familiar to students of R - [mtcars (Motor Trend Car Road Tests) data](https://stat.ethz.ch/R-manual/R-devel/library/datasets/html/mtcars.html). The data was extracted from the 1974 Motor Trend US magazine, and comprises fuel consumption and 10 other aspects of automobile design and performance for 32 automobiles (1973–74 models). We have created a csv file of the data and added two columns to indicate the car maker's name and region that the car was made in (Europe, Japan or North America). The readme file in the repository gives further information for anyone who wishes to migrate the mtcars data directly into a graph, but for the purposes of this example we provide a single data file that you can load to populate a graph. ## Ontology We have provided the following ontology to represent the data, although many other variations are possible: ```graql insert # Entities vehicle sub entity is-abstract; car sub vehicle is-abstract has-resource model has-resource mpg has-resource cyl has-resource disp has-resource hp has-resource wt has-resource gear has-resource carb plays-role made; automatic-car sub car; manual-car sub car; carmaker sub entity is-abstract has-resource maker-name plays-role maker; japanese-maker sub carmaker; american-maker sub carmaker; european-maker sub carmaker; # Resources model sub resource datatype string; maker-name sub resource datatype string; mpg sub resource datatype double; cyl sub resource datatype long; disp sub resource datatype double; hp sub resource datatype long; wt sub resource datatype double; gear sub resource datatype long; carb sub resource datatype long; # Roles and Relations manufactured sub relation has-role maker has-role made; maker sub role; made sub role; ``` To load *ontology.gql* into Grakn, make sure the engine is running and choose a clean keyspace in which to work (here we use the default keyspace, so we are cleaning it before we get started). ```bash /bin/grakn.sh clean /bin/grakn.sh start /bin/graql.sh -f ./ontology.gql ``` ## Data Migration We migrated the CSV data using template Graql files, but for ease of use, we provide a single data file that you can load to populate a graph. ```bash /bin/graql.sh -b ./data.gql ``` Spin up the [Grakn visualiser](../grakn-dashboard/visualiser.html) by pointing your browser to [http://localhost:4567/](http://localhost:4567/). You can submit queries to check the data, or explore it using the Types dropdown menu. Some sample queries: ```graql # Cars where the model name contains "Merc" match $x has model contains "Merc"; # Cars with more than 4 gears match $x has gear > 4; # Japanese-made cars that are manual match $x isa manual-car; $y isa japanese-maker; # European cars that are automatic match $x isa automatic-car; $y isa european-maker; ``` At this point, you are ready to start investigating statistics within the data using Graql [`aggregate`](../graql/aggregate-queries.html) and [`compute`](../graql/compute-queries.html) queries: ## `aggregate` You cannot make [`aggregate`](../graql/aggregate-queries.html) queries from within the **Graph** view in the Grakn visualiser, so you will need to switch views using the left hand navigation pane, from **Graph** to **Console**. This shows a read-write view on Grakn, and you can now submit queries in the usual way, via the form. Alternatively, from your terminal, you can start the Graql shell in its interactive (REPL) mode by typing `graql.sh` at the terminal, from within the *bin* directory of the Grakn installation. Here are some example `aggregate` queries to try: ```graql # Count of all cars match $x isa car; aggregate count; # 32 # Count American car makers match $x isa american-maker; aggregate count; # 6 # Maximum MPG for an automatic car match $x isa automatic-car, has mpg $a; aggregate max $a; # Minimum HP for all cars match $x isa car, has hp $hp; aggregate min $hp; # Mean MPG for manual and automatic cars match $x isa manual-car has mpg $mpg; aggregate mean $mpg; # 24.39 match $x isa automatic-car has mpg $mpg; aggregate mean $mpg; # 17.15 # Median number of cylinders (all Mercedes cars) match $x isa carmaker has maker-name contains "Mercedes"; $y isa car has cyl $c; (maker:$x, made:$y); aggregate median $c; # 6 # Or... match $x has model contains "Merc", has cyl $c; aggregate median $c;# 6 # Maximum number of carburetors (all Chrysler cars) match $x isa carmaker has maker-name contains "Chrysler"; $y isa car has carb $c; (maker:$x, made:$y); aggregate max $c; # 4 # Minimum number of gears (all cars) match $x isa car, has gear $g; aggregate min $g; # 3 ``` ## `compute` Graql also provides [compute queries](../graql/compute-queries.html) that can be used to determine values such as mean, minimum and maximum. These can be submitted using the **Graph** view on the Visualiser. For example, type each of the following into the form and submit: ```graql # Number of automatic and manual cars compute count in automatic-car; # 19 compute count in manual-car; # 13 # Number of Japanese car makers compute count in japanese-maker; # 4 # Median number of cylinders (all cars) compute median of cyl; # 6 # Minimum number of gears (all cars) compute min of gear; # 3 # Maximum number of carburetors (all cars) compute max of carb; # 8 ``` ## When to Use `aggregate` and When to Use `compute`? Aggregate queries are computationally light and run single-threaded on a single machine, and are more flexible than the equivalent compute query (for example, you can use an aggregate query to filter results by resource). ```graql match $x has identifier contains "Elizabeth"; aggregate count; ``` Compute queries are computationally intensive and run in parallel on a cluster, so are good for big data and can be used to calculate results very fast. However, you can't filter the results by resource in the same way as you can for an `aggregate` query. ## Inference ```graql match $x has powerful contains "TRUE" has economical contains "TRUE"; match $x has powerful contains "TRUE" has economical contains "TRUE" has model $y; $x id "106584" isa manual-car; $y value "Ferrari Dino" isa model; $x id "254120" isa automatic-car; $y value "Pontiac Firebird" isa model; ``` ## Where Next? If you haven't already, we recommend that you review the documentation about [Graql analytics](https://grakn.ai/pages/documentation/graql-analytics/analytics-overview.html), since there is more to `compute` than just statistical analysis. Unfortunately, this example is not a good one to illustrate clusters, degrees or shortest path analytics, which is why it isn't described here. This example was based on CSV data migrated into Grakn. Having read it, Yyou may want to further study our documentation about [CSV migration](../migration/CSV-migration.html) and [Graql templating](https://grakn.ai/pages/documentation/graql/graql-templating.html). {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/CSV-migration.md --- title: An Example of Migrating CSV to Grakn keywords: migration last_updated: January 2017 tags: [migration, examples] summary: "An example to illustrate migration of CSV to Grakn" sidebar: documentation_sidebar permalink: /documentation/examples/CSV-migration.html folder: documentation comment_issue_id: 27 --- ## Introduction This example looks at the migration of genealogy data in CSV format to build a graph in GRAKN.AI. The data is used as the basis of a [blog post](https://blog.grakn.ai/family-matters-1bb639396a24#.2e6h72y0m) that illustrates the fundamentals of the Grakn visualiser, reasoner and analytics components. As the blog post explains, the original data was a [document](http://www.lenzenresearch.com/titusnarrlineage.pdf) from [Lenzen Research](http://www.lenzenresearch.com/) that described the family history of Catherine Niesz Titus for three generations of her maternal lineage. In this example, we will walk through how to migrate the CSV data into Grakn, and confirm that we have succeeded using the Grakn visualiser. For a detailed overview of CSV migration, we recommend that you take a look at the Grakn documentation on [CSV Migration](https://grakn.ai/pages/documentation/migration/CSV-migration.html) and [Graql templating](https://grakn.ai/pages/documentation/graql/graql-templating.html). ## Genealogy Data The data for this example can be found as a set of CSV files in the [sample-projects](https://github.com/graknlabs/sample-projects/tree/master/example-csv-migration-genealogy) repository on Github, which is also included in the Grakn distribution zip file. The data was put together by our team from narrative information gleaned from the original Lenzen Research [document](http://www.lenzenresearch.com/titusnarrlineage.pdf), with some minor additions to generate some interesting queries for Grakn's reasoner. Let's take a look at the *raw-data* directory in the example project, which contains the CSV files. These files were put together by hand by our team, mostly by [Michelangelo](https://blog.grakn.ai/@doctormiko). |filename|description| |[*people.csv*](https://github.com/graknlabs/sample-projects/blob/master/example-csv-migration-genealogy/raw-data/people.csv)| This comprehensive CSV contains information about all the individuals discussed in the Lenzen document. For each row, it lists the available information about individuals' names, gender, birth and death dates and age at death. It also assigns each person a person ID ("pid"), which is a string containing the full name of each individual, and is used for identification of individuals in the other CSV files.| |[*births.csv*](https://github.com/graknlabs/sample-projects/blob/master/example-csv-migration-genealogy/raw-data/births.csv)| This CSV lists the person IDs of a child and each of its parents| |[*weddings.csv*](https://github.com/graknlabs/sample-projects/blob/master/example-csv-migration-genealogy/raw-data/weddings.csv)| This CSV comprises a row for each marriage, identifying each by a wedding ID (wid). The rows contain the person IDs of each spouse and the date of their wedding, where it is known.| ### Ontology The ontology is a way to describe the entities and their relationships, so the underlying graph can store them as nodes and edges. You can find out more in our guide to the Grakn Knowledge Model. The ontology allows Grakn to perform: * logical reasoning over the represented knowledge, such as the extraction of implicit information from explicit data (inference) * discovery of inconsistencies in the data (validation). The ontology is shown below. There is a single entity, `person`, which has a number of resources and can play various roles (`parent`, `child`, `spouse1` and `spouse2`) in two possible relations (`parentship` and `marriage`). ```graql insert # Entities person sub entity plays-role parent plays-role child plays-role spouse1 plays-role spouse2 has-resource identifier has-resource firstname has-resource surname has-resource middlename has-resource picture has-resource age has-resource birth-date has-resource death-date has-resource gender; # Roles and Relations marriage sub relation has-role spouse1 has-role spouse2 has-resource picture; spouse1 sub role; spouse2 sub role; parentship sub relation has-role parent has-role child; parent sub role; child sub role; # Resources identifier sub resource datatype string; firstname sub resource datatype string; surname sub resource datatype string; middlename sub resource datatype string; picture sub resource datatype string; age sub resource datatype long; birth-date sub resource datatype string; death-date sub resource datatype string; gender sub resource datatype string; ``` To load *ontology.gql* into Grakn, make sure the engine is running and choose a clean keyspace in which to work (here we use the default keyspace, so we are cleaning it before we get started). ```bash /bin/grakn.sh clean /bin/grakn.sh start /bin/graql.sh -f ./ontology.gql ``` ### Data Migration Having loaded the ontology, the next steps are to populate the graph by migrating data into Grakn from CSV. We will consider three CSV files that contain data to migrate into Grakn. #### people.csv The *people.csv* file contains details of the people that we will use to create seven `person` entities. Note that not all fields are available for each person, but at the very least, each row is expected to have the following: * pid (this is the person identifier, and is a string representing their full name) * first name * gender ```csv name1,name2,surname,gender,born,dead,pid,age,picture Timothy,,Titus, male,,, Timothy Titus,, Mary,,Guthrie,female,,,Mary Guthrie,, John,,Niesz,male,1798-01-02,1872-03-06,John Niesz,74, Mary,,Young,female,1798-04-09,1868-10-28,Mary Young,70, William,Sanford,Titus,male,1818-03-23,01/01/1905,William Sanford Titus,76, Elizabeth,,Niesz,female,1820-08-27,1891-12-08,Elizabeth Niesz,71, Mary,Melissa,Titus,female,1847-08-12,10/05/1946,Mary Melissa Titus,98, ... ``` The migrator is a set of template Graql statements that instruct the Grakn migrator on how the CSV data can be mapped to the ontology. The Grakn migrator applies the template to each row of data in a CSV file, replacing the indicated sections in the template with the value from a specific cell, identified by the column header (the key). This sounds complicated, but isn't really, as we will show. The Graql template code for the people migrator is as follows: ```graql-template insert $p isa person has identifier has firstname , if ( != "") do { has surname , } if ( != "") do { has middlename , } if ( != "") do { has picture , } if ( != "") do { has age @long(), } if ( != "") do { has birth-date , } if ( != "") do { has death-date , } has gender ; ``` For each row in the CSV file, the template inserts a `person` entity with resources that take the value of the cells in that row. Where data is optional, the template checks to see if it is present before adding the resources for middlename, surname, picture, age, birth and death dates. Calling the Grakn migrator on the *people.csv* file using the above template (named `people-migrator.gql`) is performed as follows: ```bash /bin/migration.sh csv -i ./people.csv -t ./migrators/people-migrator.gql -k grakn ``` {% include note.html content="Usage instructions for the Grakn migrator are available as part of our [migration documentation](../migration/migration-overview.html)." %} The data insertion generated by the migrator is as follows: ```graql insert $p0 has death-date "1891-12-08" isa person has gender "female" has identifier "Elizabeth Niesz" has surname "Niesz" has age 71 has firstname "Elizabeth" has birth-date "1820-08-27"; insert $p0 has identifier "William Sanford Titus" has age 76 isa person has firstname "William" has surname "Titus" has gender "male" has birth-date "1818-03-23" has middlename "Sanford" has death-date "1905-01-01"; insert $p0 isa person has surname "Titus" has firstname "Timothy" has identifier "Timothy Titus" has gender "male"; insert $p0 isa person has firstname "Mary" has identifier "Mary Guthrie" has surname "Guthrie" has gender "female"; insert $p0 isa person has firstname "Mary" has death-date "1946-05-10" has surname "Titus" has age 98 has identifier "Mary Melissa Titus" has middlename "Melissa" has gender "female" has birth-date "1847-08-12"; insert $p0 has death-date "1872-03-06" has age 74 has identifier "John Niesz" isa person has birth-date "1798-01-02" has firstname "John" has gender "male" has surname "Niesz"; insert $p0 has identifier "Mary Young" has birth-date "1798-04-09" isa person has firstname "Mary" has death-date "1868-10-28" has surname "Young" has gender "female" has age 70; ... ``` #### births.csv Each row of *births.csv* records a parent and child, with two rows for each of the three children listed: ```csv parent,child Timothy Titus,William Sanford Titus Mary Guthrie, William Sanford Titus John Niesz,Elizabeth Niesz Mary Young,Elizabeth Niesz Elizabeth Niesz,Mary Melissa Titus William Sanford Titus,Mary Melissa Titus ... ``` The Graql template code for the Grakn migrator is as follows: ```graql-template match $c isa person has identifier ; $p isa person has identifier ; insert (child: $c, parent: $p) isa parentship; ``` For each row in the CSV file, the template matches the child and parent cells to their corresponding `person` entities, and then inserts a `parentship` relation, placing the entities it has matched into the `child` and `parent` roles. {% include note.html content="You must ensure that all entities exist in a graph before inserting relations." %} Calling the Grakn migrator on the *births.csv* file using the above template (named `births-migrator.gql`) is performed as follows: ```bash /bin/migration.sh csv -i ./births.csv -t ./migrators/births-migrator.gql -k grakn ``` The data insertion generated by the migrator is as follows: ```graql match $c0 has identifier "William Sanford Titus" isa person; $p0 isa person has identifier "Timothy Titus"; insert (child: $c0, parent: $p0) isa parentship; match $p0 isa person has identifier "Mary Guthrie"; $c0 has identifier "William Sanford Titus" isa person; insert (child: $c0, parent: $p0) isa parentship; match $p0 isa person has identifier "Elizabeth Niesz"; $c0 isa person has identifier "Mary Melissa Titus"; insert (child: $c0, parent: $p0) isa parentship; match $c0 isa person has identifier "Mary Melissa Titus"; $p0 has identifier "William Sanford Titus" isa person; insert (child: $c0, parent: $p0) isa parentship; match $c0 isa person has identifier "Elizabeth Niesz"; $p0 has identifier "John Niesz" isa person; insert (child: $c0, parent: $p0) isa parentship; match $c0 isa person has identifier "Elizabeth Niesz"; $p0 has identifier "Mary Young" isa person; insert (child: $c0, parent: $p0) isa parentship; ... ``` #### weddings.csv The *weddings.csv* file contains two columns that correspond to both spouses in a marriage, and an optional column for a photograph of the happy couple: ```csv spouse1,spouse2,picture Timothy Titus,Mary Guthrie, John Niesz,Mary Young,http://1.bp.blogspot.com/-Ty9Ox8v7LUw/VKoGzIlsMII/AAAAAAAAAZw/UtkUvrujvBQ/s1600/johnandmary.jpg Elizabeth Niesz,William Sanford Titus, ``` The Graql template code for the migrator is as follows: ```graql-template match $x has identifier ; $y has identifier ; insert (spouse1: $x, spouse2: $y) isa marriage if ( != "") do { has picture }; ``` For each row in the CSV file, the template matches the two spouse cells to their corresponding `person` entities, and then inserts a `marriage` relation, placing the entities it has matched into the `spouse1` and `spouse2` roles. If there is data in the picture cell, a `picture` resource is also created for the `marriage` relation. Calling the Grakn migrator on the *weddings.csv* file using the above template (named `weddings-migrator.gql`) is performed as follows: ```bash /bin/migration.sh csv -i ./weddings.csv -t ./migrators/weddings-migrator.gql -k grakn ``` The Graql insertion code is as follows: ```graql match $x0 has identifier "Timothy Titus"; $y0 has identifier "Mary Guthrie"; insert (spouse1: $x0, spouse2: $y0) isa marriage; match $x0 has identifier "John Niesz"; $y0 has identifier "Mary Young"; insert has picture "http:\/\/1.bp.blogspot.com\/-Ty9Ox8v7LUw\/VKoGzIlsMII\/AAAAAAAAAZw\/UtkUvrujvBQ\/s1600\/johnandmary.jpg" (spouse1: $x0, spouse2: $y0) isa marriage; match $y0 has identifier "William Sanford Titus"; $x0 has identifier "Elizabeth Niesz"; insert (spouse1: $x0, spouse2: $y0) isa marriage; ... ``` ## Migration Script For simplicity, the */raw-data/* directory of the example project contains a script called *loader.sh* that calls each migration script in turn, so you can simply call the script from the terminal, passing in the path to the Grakn */bin/* directory. ```bash ./loader.sh /bin ``` The migration will take a minute or two, and the terminal will report which file it is migrating at each step. When it is complete, it will report that it is "Done migrating data". To check that it has been successful, open the [Grakn visualiser](../grakn-dashboard/visualiser.html) and select Types, then Entities, and choose one of those presented to you (the entities should be those described [above](./CSV-migration.html#entities)]. The visualiser will display the entities it has imported. The screenshot below illustrates the result from selecting to see all `person` entities. We have completed the data import, and the graph can now be queried. For example, from the Graql shell: ``` match $x isa person, has identifier $i; aggregate count; ``` There should be 60 people in the dataset. ## Data Export In this example, we have imported a dataset stored in three separate CSV files into Grakn to build a simple graph. We have discussed the ontology and migration templates, and shown how to apply the templates to the CSV data using the shell migrator, using a script file *loader.sh* to automate calling the migrator on each file It is possible to export the data from Grakn, in *.gql* format, so that it can easily be loaded to a graph again without the need to migrate from CSV. To export the data, we use the Grakn *migration.sh* shell script again, as described in the [migration documentation](../migration/migration-overview.html#export-from-grakn): ```bash # Export the ontology /migration.sh export -ontology > ontology-export.gql # Export the data /migration.sh export -data > data-export.gql ``` Exporting data or the ontology from Grakn, into Graql, will always redirect to standard out, so above we are sending the output to an appropriately named file. ## Where Next? This example has illustrated how to migrate CSV data into Grakn. Having read it, you may want to further study our documentation about [CSV migration](../migration/CSV-migration.html) and [Graql templating](https://grakn.ai/pages/documentation/graql/graql-templating.html). {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/examples-overview.md --- title: Examples Overview keywords: examples last_updated: September 23, 2016 tags: [getting-started, examples] summary: "Landing page for Grakn examples." sidebar: documentation_sidebar permalink: /documentation/examples/examples-overview.html folder: documentation comment_issue_id: 27 --- ## Introduction This page lists the examples of Grakn that we suggest you study to learn how to work with our stack. We plan to continue to expand our set, and we also encourage you to let us know if you have example code to share, so we can link to it (for example, to your repo or a blog post). If you would like to request a particular example, please get in touch with us by leaving a comment on this page or posting a question on the discussion boards. Our [Community page](https://grakn.ai/community.html) lists other ways you can talk to us. ### Genealogy Dataset The genealogy dataset is widely used across our documentation about GRAKN.AI, because it allows for a simple, yet powerful, illustration of many key features. As described below, it is used to illustrate CSV migration and the Grakn Reasoner. It is available on the [sample-datasets repo on Github](https://github.com/graknlabs/sample-datasets/tree/master/genealogy-graph), and is also discussed in the ["Family Matters" blog post](https://blog.grakn.ai/family-matters-1bb639396a24#.4gnoaq2hr). ## Use Cases ### CSV Migration There are several examples available: * We use the genealogy dataset to show [how to migrate genealogy data into Grakn from CSV](../examples/CSV-migration.html). * Our sample-projects repository on Github also contains [an example that takes a simple CSV data file of pets](https://github.com/graknlabs/sample-projects/tree/master/example-csv-migration-pets). Please see the [readme file](https://github.com/graknlabs/sample-projects/blob/master/example-csv-migration-pets/README.md) to get started. * CSV migration is also covered in a [blog post](https://blog.grakn.ai/twenty-years-of-games-in-grakn-14faa974b16e#.cuox3cew2). ### JSON Migration There is an example of using the Java Migration API for JSON migration on the [sample-projects repository](https://github.com/graknlabs/sample-projects/tree/master/example-json-migration-giphy) on Github. ### SQL Migration There are several examples available: * A common use-case is to migrate existing SQL data to a graph in Grakn. We walk through a simple example of using the migration script as part of the documentation about [SQL migration](../migration/SQL-migration.html) * There is a an additional example of [SQL migration using the Java API](../examples/SQL-migration.html). * We also cover SQL migration in a [blog post](https://blog.grakn.ai/populating-mindmapsdb-with-the-world-5b2445aee60c#). ### OWL Migration For more information about migrating OWL to Grakn using the Java migration API, please see the [OWL migration example](./OWL-migration.html). ### Reasoning with Graql We use the genealogy dataset to illustrate how to write rules to infer new information from a dataset. You can find the example [here](./graql-reasoning.html). ### Learn Graql * The [Modern example](./modern.html) is a simple one, designed to test your knowledge of Graql. * We have a [simple Pokemon example](./pokemon.html) to illustrate how to form a range of different Graql queries. * The [philosophers.gql](https://github.com/graknlabs/grakn/blob/master/grakn-dist/src/examples/philosophers.gql) file, also distributed in the Grakn release zip, contains a simple ontology and data, for use as an example. ## Languages ### Haskell, R and Python Bindings It is possible to extract data from Grakn and use it as a data science tool for analysis. You can take the results of a Graql query and store the results in a dataframe or similar structure, for use with Haskell, R or Python. * Haskell: This [blog post](https://blog.grakn.ai/grakn-ai-and-haskell-c166c7cc1d23#.9jc7xu79l) is the first in a series of posts about combining GRAKN.AI and Haskell. * R and Python: This [blog post](https://blog.grakn.ai/there-r-pandas-in-my-graph-b8b5f40a2f99#) explains and gives a simple example. * Python: A further [blog post](https://blog.grakn.ai/grakn-pandas-celebrities-5854ad688a4f#.k5zucfp6f) uses the Python driver to examine our example movie dataset. ### Java Examples * Use of the core APIs for creating an ontology, adding data and making basic queries is covered in a [blog post](https://blog.grakn.ai/working-with-grakn-ai-using-java-5f13f24f1269#.giljgrjb3). * JSON migration: There is an example of using the [Java Migration API](../developing-with-java/migration-api.html) for JSON migration on the [sample-projects repository](https://github.com/graknlabs/sample-projects/tree/master/example-json-migration-giphy) on Github. * SQL migration: We have documented an example of [SQL migration using the Java API](../examples/SQL-migration.html). * Pokemon: The [sample-projects](https://github.com/graknlabs/sample-projects/tree/master/example-pokemon) repo on Github contains a Java project that uses the Java Graph API on Pokemon data and an ontology. * Philosophers: The [sample-projects](https://github.com/graknlabs/sample-projects/tree/master/example-philosophers) repo on Github contains a Java project that uses the Java Graph API on a Philosophers dataset and ontology. ### Moogi Movie Database [Moogi](https://moogi.co) is a large database of information about movies. We have provided a subset of its data to try out in our [sample-datasets repository](https://github.com/graknlabs/sample-datasets/tree/master/movies) on Github. ## Where Next? If you are interested in writing an example on Grakn, maybe as a way of trying it out, please take a look at the [Example Projects](./projects.html) page, which lists some ideas that we have for potential examples or research projects. {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/graql-rules.md --- title: Reasoning with Graql keywords: migration last_updated: February 2017 tags: [reasoning, examples] summary: "An example to illustrate inference using genealogy data." sidebar: documentation_sidebar permalink: /documentation/examples/graql-reasoning.html folder: documentation comment_issue_id: 27 --- ## Introduction This is an example of how to use the reasoning facilities of Graql to infer information about family relationships from a simple dataset of genealogy data. The data has been used previously as the basis of a [blog post](https://blog.grakn.ai/family-matters-1bb639396a24#.2e6h72y0m) to illustrate the fundamentals of the Grakn visualiser as well as reasoning and analytics capabilities. As the blog post explained, the original data was a [document](http://www.lenzenresearch.com/titusnarrlineage.pdf) from [Lenzen Research](http://www.lenzenresearch.com/) that described the family history of Catherine Niesz Titus for three generations of her maternal lineage. Our team gathered together a set of CSV files containing basic information about the family, such as names, dates of birth, death and marriage, who was a parent of who, and who married who. The full genealogy-graph project can be found on Grakn's [sample-datasets](https://github.com/graknlabs/sample-datasets/tree/master/genealogy-graph) repository on Github. However, we will use a simpler ontology for this example, which is found in *basic-genealogy.gql*, located within the */examples* folder of the Grakn installation. The file can also be found [on Github](https://github.com/graknlabs/grakn/blob/master/grakn-dist/src/examples/basic-genealogy.gql). In this example, we will explore how to use Grakn to make inferences and find information from the data that is not explicit in the dataset. You can find documentation about writing rules in Graql [here](https://grakn.ai/pages/documentation/graql/graql-rules.html). ## Ontology and Data On GRAKN.AI, the first step when working with a dataset is to define its ontology in Graql. The ontology is a way to describe the entities and their relations, so the underlying graph can store them as nodes and edges. You can find out more in our guide to the Grakn Knowledge Model. The ontology allows Grakn to perform: * logical reasoning over the represented knowledge, such as the extraction of implicit information from explicit data (inference) * discovery of inconsistencies in the data (validation). The complete ontology for the genealogy-graph demo is in [basic-genealogy.gql](https://github.com/graknlabs/grakn/blob/master/grakn-dist/src/examples/basic-genealogy.gql). The ontology contains a `person` entity, and a number of possible family relationships between `person` entities, such as parent/child, siblings, grandparent/grandchild, which we will discuss shortly. The data for this example is discussed more fully in our [example on CSV migration](./CSV-migration.html), which discusses how the original data is laid out in CSV files and migrated into GRAKN.AI. However, for this example, you do not need any familiarity with CSV migration or the origin of the data we are using. ## Loading the Example The *basic-genealogy.gql* file contains the ontology, data and rules needed for this example. To load it into a graph, make sure the engine is running and choose a clean keyspace in which to work (in the example below we use the default keyspace, so we are cleaning it before we get started): ```bash /bin/grakn.sh clean /bin/grakn.sh start /bin/graql.sh -f /examples/basic-genealogy.gql ``` When the terminal prompt returns, the data will have been loaded into the default keyspace, and you can start to look it using the [Grakn visualiser](../grakn-dashboard/visualiser.html), by navigating to [http://localhost:4567](http://localhost:4567) in your browser. Submit a query, such as `match $x isa person` to check that all is well with your graph. ## Inferring Family Relations By default, inference is switched off, and the only information you can query Grakn about is what was directed loaded from the data. For example, if you submit the following query to the visualiser: ```graql match (child: $c, parent: $p) isa parentship; ``` You will receive parentship results, but if you clear the query and then submit a new query for gender-specific results, say mother-daughter relationships: ```graql match (mother: $c, daughter: $p) isa parentship; ``` You will receive no results at all. To find inferred relations between the people in our dataset, you need to activate inference in the Grakn visualiser. Open the Query settings under the cog button, which is on the far right hand side of the horizontal icon menu (at the top of the screen). You will see the "Activate inference" checkbox. Check it, and Grakn is ready to start building some new information about the family. Try the query again: ```graql match (mother: $c, daughter: $p) isa parentship; ``` You should see something similar to the screenshot below in your visualiser window. {% include note.html content="You can alternatively make queries in the graql shell. You will need to use `./graql.sh -n` to enable inference when you start the shell." %} ## Inference Rules Inference is the process of deducing new information from available data. For example, given the following statements: ``` If grass is not an animal. If vegetarians only eat things which are not animals. If sheep only eat grass. ``` It is possible to infer the following: ``` Then sheep are vegetarians. ``` The initial statements can be seen as a set of premises. If all the premises are met we can infer a new fact (that sheep are vegatarians). If we hypothetise that sheep are vegetarians then the whole example can be expressed with a particular two-block structure: IF some premises are met, THEN a given hypothesis is true. This is how reasoning in Graql works. It checks whether the statements in the first block can be verified and, if they can, infers the statement in the second block. The rules are written in Graql, and we call the first set of statements (the IF part or, if you prefer, the antecedent) simply the left hand side (LHS). The second part, not surprisingly, is the right hand side (RHS). Using Graql, both sides of the rule are enclosed in curly braces and preceded by, respectively, the keywords `lhs` and `rhs`. {% include note.html content="The full documentation for writing rules in Graql is available from [here](https://grakn.ai/pages/documentation/graql/graql-rules.html)." %} ### Example 1: Specific relations between parents and children As we saw above, it is possible for Grakn to infer the gender-specific roles (`mother`, `father`, `daughter`, `son`) that a `person` entity plays. It does this by applying the following rules: ```graql $genderizeParentships1 isa inference-rule lhs {(parent: $p, child: $c) isa parentship; $p has gender "male"; $c has gender "male"; } rhs {(father: $p, son: $c) isa parentship;}; $genderizeParentships2 isa inference-rule lhs {(parent: $p, child: $c) isa parentship; $p has gender "male"; $c has gender "female"; } rhs {(father: $p, daughter: $c) isa parentship;}; $genderizeParentships3 isa inference-rule lhs {(parent: $p, child: $c) isa parentship; $p has gender "female"; $c has gender "male"; } rhs {(mother: $p, son: $c) isa parentship;}; $genderizeParentships4 isa inference-rule lhs {(parent: $p, child: $c) isa parentship; $p has gender "female"; $c has gender "female"; } rhs {(mother: $p, daughter: $c) isa parentship;}; ``` The four rules can be broken down as follows: * LHS: In the `parentship` relation between child `$c` and parent `$p`, do they both have a `gender` resource that is `male`? * RHS: The `parentship` relation is between `father` and `son` * LHS: In the `parentship` relation between child `$c` and parent `$p`, does `$c` have a `gender` resource that is `female` and `$p` have a `gender` resource that is `male`? * RHS: The `parentship` relation is between `father` and `daughter` * LHS: In the `parentship` relation between child `$c` and parent `$p`, does `$c` have a `gender` resource that is `male` and `$p` have a `gender` resource that is `female`? * RHS: The `parentship` relation is between `mother` and `son` * LHS: In the `parentship` relation between child `$c` and parent `$p`, do both have a `gender` resource that is `female`? * RHS: The `parentship` relation is between `mother` and `daughter` We can use the rule to easily discover the sons who have the same name as their fathers: ```graql match (father: $p, son: $c) isa parentship; $p has firstname $n; $c has firstname $n; ``` In the genealogy-graph example, there should be two results returned. William and John are names shared between father/son pairs. ### Example 2: A `grandparentship` relation The *basic-genealogy* file contains a number of rules for setting up family relationships, such as siblings, cousins, in-laws and the following, which sets up a relation called `grandparentship`: ```graql $parentsOfParentsAreGrandparents isa inference-rule lhs {(parent:$p, child: $gc) isa parentship; (parent: $gp, child: $p) isa parentship; } rhs {(grandparent: $gp, grandchild: $gc) isa grandparentship;}; ``` Here, the left hand side rules check two statements: * does `$p` play the `parent` role in a `parentship` relation, with (`$gc`) playing the `child` role? * does `$p` also play the `child` role in a `parentship` relation, with (`$gp`) playing the `parent` role? If so, the right hand side of the rules state that: * `$gp` and `$gc` are in a `grandparentship` relation, where `$gp` plays the `grandparent` role and `$gc` plays the `grandchild` role. Some additional rules can add more specifics to the `grandparentship` and assign the entities to the roles `grandson`, `granddaughter`, `grandmother` and `grandfather`: ```graql $grandParents1 isa inference-rule lhs {($p, son: $gc) isa parentship; (father: $gp, $p) isa parentship; } rhs {(grandfather: $gp, grandson: $gc) isa grandparentship;}; $grandParents2 isa inference-rule lhs {($p, daughter: $gc) isa parentship; (father: $gp, $p) isa parentship; } rhs {(grandfather: $gp, granddaughter: $gc) isa grandparentship;}; $grandParents3 isa inference-rule lhs {($p, daughter: $gc) isa parentship; (mother: $gp, $p) isa parentship; } rhs {(grandmother: $gp, granddaughter: $gc) isa grandparentship;}; $grandParents4 isa inference-rule lhs {($p, son: $gc) isa parentship; (mother: $gp, $p) isa parentship; } rhs {(grandmother: $gp, grandson: $gc) isa grandparentship;}; ``` Much as above for the `parentship` relation, the rules can be broken down as follows: * LHS: There is a `parentship` relation between son `$gc` and parent `$p`, and another `parentship` relation where `$p` is now in the child role and `$gp` is a father. * RHS: The `grandparentship` relation is between `grandfather` and `grandson` * LHS: There is a `parentship` relation between daughter `$gc` and parent `$p`, and another `parentship` relation where `$p` is now in the child role and `$gp` is a father. * RHS: The `grandparentship` relation is between `grandfather` and `granddaughter` * LHS: There is a `parentship` relation between daughter `$gc` and parent `$p`, and another `parentship` relation where `$p` is now in the child role and `$gp` is a mother. * RHS: The `grandparentship` relation is between `grandmother` and `granddaughter` * LHS: There is a `parentship` relation between son `$gc` and parent `$p`, and another `parentship` relation where `$p` is now in the child role and `$gp` is a mother. * RHS: The `grandparentship` relation is between `grandmother` and `grandson` These rules allow us to find all `grandparentship` relations, and further, it allows us to query, for example, which grandfather/grandson pairs share the same name: ```graql match (grandfather: $x, grandson: $y) isa grandparentship; $x has firstname $n; $y has firstname $n; ``` In the genealogy-graph example, there should be three results returned. George, Jacob and John are names shared between grandfather/grandson pairs. ### Example 3: A `cousins` relation Another rule can be used to infer `person` entities who are cousins: ``` $peopleWithSiblingsParentsAreCousins isa inference-rule lhs { (parent: $p, child: $c1) isa parentship; ($p, $p2) isa siblings; (parent: $p2, child: $c2) isa parentship; } rhs {(cousin1: $c1, cousin2: $c2) isa cousins;}; ``` * `$p` is a `parent` and has a `child` `$c1` * `$p` is a sibling to `$p2` * `$p2` has a `child` `$c2` * Then `$c1` and `$c2` are in a `cousins` relationship. It would also be possible to infer uncle/aunt/nephew/niece relationships, using a similar approach. We will leave that as an exercise for the reader! ## Other Queries There are some queries that are currently not possible. For example: * Who has no children? * Who has never married? * Who has the most grandchildren? * Who has the most brothers? If you want to infer information from a dataset such as this example, and want to check whether it is possible, please reach out to us via [Slack](https://grakn.ai/slack.html) or our [discussion forum](https://discuss.grakn.ai). ## Where Next? This example has illustrated the basics of the inference rules used by Graql. Having read it, you may want to further study our documentation about [Graql Rules](../graql/graql-rules.html) and further investigate the [example that imports the genealogy data from CSV format into Grakn](./CSV-migration.html). We will be using the genealogy data throughout our documentation. For an overview, please see our [Family Matters](https://blog.grakn.ai/family-matters-1bb639396a24#.uelgekrn2) blog post. ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. {% include links.html %} --- ## File: pages/documentation/11-examples/JSON-migration-NOT-PUBLISHED.md --- title: An Example of Migrating JSON to Grakn keywords: migration last_updated: February 2017 tags: [migration, examples] summary: "An example to illustrate migration of JSON to Grakn" sidebar: documentation_sidebar permalink: /documentation/examples/JSON-migration.html folder: documentation comment_issue_id: 27 --- ## Introduction This example looks at the migration of simple JSON data to build a graph in GRAKN.AI. The data is an invented library catalogue of a very small number of books, and illustrates how to migrate simple data and handle issues such as preventing the creation of duplicate entities. You can find the data, ontology and templates for this example in our [sample projects repo](https://github.com/graknlabs/sample-projects/tree/master/example-json-migration-library) on Github. For a detailed overview of CSV migration, we recommend that you take a look at the Grakn documentation on [JSON Migration](https://grakn.ai/pages/documentation/migration/JSON-migration.html) and [Graql templating](https://grakn.ai/pages/documentation/graql/graql-templating.html). ## Data The JSON data for the book catalogue is as follows: ```json { "books": [ { "title":"The Gruffalo", "id":"1", "author":"Julia Donaldson", "subject":"Children's Fiction" }, { "title":"The Tear Thief", "id":"2", "author":"Carol Ann Duffy", "subject":"Children's Fiction" }, { "title":"Selected Poems", "id":"3", "author":"Carol Ann Duffy", "subject":"Poetry" }, ... ] } ``` The `id` value can be considered to be unique and identify each book. However, other values cannot be assumed to be unique. In the snippet shown above, note that the author `Carol Ann Duffy` and subject `Children's Fiction` are both duplicated. It is possible that a book title may also be duplicated, although this is not shown. ### Ontology The ontology for the book catalogue is as follows: ```graql insert # Entities book sub entity has-resource bookId has-resource title plays-role publication-item; author sub entity has-resource authorName plays-role publication-author; subject sub entity has-resource subjectName plays-role publication-subject; # Resources bookId sub resource datatype string; title sub resource datatype string; authorName sub resource datatype string; subjectName sub resource datatype string; # Relations and Roles publication sub relation has-role publication-item has-role publication-author has-role publication-subject; publication-item sub role; publication-author sub role; publication-subject sub role; ``` Here, there are three entities, to reflect the book, author of the book and possible book subjects. There is one relation, `publication` which between all three entities. To load *ontology.gql* into Grakn, make sure the engine is running and choose a clean keyspace in which to work (here we use the default keyspace, so we are cleaning it before we get started). ```bash /bin/grakn.sh clean /bin/grakn.sh start /bin/graql.sh -f ./ontology.gql ``` ### Data Migration Having loaded the ontology, the next steps are to populate the graph by migrating data into Grakn, using Graql templates. There are separate templates for each entity. First, for the books: ```graql-template insert for() do { $book isa book has bookId has title ; } ``` The authors: ```graql-template insert for() do { $author isa author has authorName <author>; } ``` The subjects: ```graql-template insert for() do { $subject isa subject has subjectName <subject>; } ``` Finally, a template to build the relations between the entities: ```graql-template ``` To call the migration script on each template: ```bash ./bin/migration.sh json -t ./json/book-template.gql -i ./json/library-data.json -k grakn ./bin/migration.sh json -t ./json/author-template.gql -i ./json/library-data.json -k grakn ./bin/migration.sh json -t ./json/subject-template.gql -i ./json/library-data.json -k grakn ./bin/migration.sh json -t ./json/publication-template.gql -i ./json/library-data.json -k grakn ``` The resultant Graql for the migration: ```graql ``` ## Visualise the Data ## Add Borrower Data ## Data Export ```bash # Export the ontology <relative-path-to-Grakn>/migration.sh export -ontology > ontology-export.gql # Export the data <relative-path-to-Grakn>/migration.sh export -data > data-export.gql ``` Exporting data or the ontology from Grakn, into Graql, will always redirect to standard out, so above we are sending the output to an appropriately named file. ## Where Next? {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/modern.md --- title: Modern Example in Graql keywords: graql, query last_updated: September 19, 2016 tags: [graql, examples] summary: "A short example to illustrate Graql queries" sidebar: documentation_sidebar permalink: /documentation/examples/modern.html folder: documentation comment_issue_id: 27 --- If you have not yet set up the Grakn environment, please see the [Setup guide](../get-started/setup-guide.html). For a comprehensive guide to all Graql keywords, please see the [Graql documentation](https://grakn.ai/pages/documentation/graql/graql-overview.html). ## Introduction We have a few examples on how to work with Graql using different datasets. For example, see our [blog posts](https://medium.com/p/e1125e02dc85) and the [Quickstart Tutorial](../get-started/quickstart-tutorial.html) that introduces how to make Graql queries. This example takes a very simple example from [TinkerPop3 Documentation](http://tinkerpop.apache.org/docs/3.0.1-incubating/), called TinkerPop Modern. Here it is, shown diagrammatically: The image above is used from the documentation provided for TinkerPop3, and licensed by the [Apache Software Foundation](http://www.apache.org). We have chosen this example as it may already be familiar, and is simple enough to demonstrate some of the fundamentals of Graql. We walk through the entities ("things") and relationships between them and show how to represent them using Graql to define a schema. We then use Graql to add the data to the graph. The main purpose of this example, however, is to use it for practice at making sample queries on the graph. ### Starting Graql If it is not already running, start Grakn, then open a Graql shell: ```bash cd [your Grakn install directory] bin/grakn.sh start bin/graql.sh ``` ## Defining a Schema In the example, we have a number of entities: * people (marko, vadas, josh and peter), each of which has an age. * software (lop and ripple), each of which has an associated programming language. The relationships between the entities are straightforward: * there are two connections between marko and other people who he "knows" (josh and vadas). * there are four connections between people and software, such that the people "created" the software. Lop was created by marko, peter and josh, which ripple was created by josh. ### Entities Here, we add person and software entities, via the Graql shell: ```graql insert person sub entity; insert software sub entity; ``` ### Resources To assign resources to the entities, which you can think of as attributes, we use resources. First, we define what they are (age is a number, programming language is a string that represents the language's name), then we allocate them to the entity in question: ```graql insert age sub resource datatype long; insert name sub resource datatype string; insert person has-resource age, has-resource name; insert lang sub resource datatype string; insert software has-resource lang has-resource name; insert weight sub resource datatype double; ``` ### Relations Let's first define the relationship between people. The diagram shows that marko knows vadas, but we don't have any information about whether the inverse is true (though it seems likely that vadas probably also knows marko). Let's set up a relationship called `knows`, which has two roles - `knower` (for marko) and `known-about` (for vadas): ```graql insert knower sub role; insert known-about sub role; insert person plays-role knower; insert person plays-role known-about; insert knows sub relation, has-role knower, has-role known-about, has-resource weight; ``` Note that the `knows` relation also has an attribute, in the form of a resource called `weight` (though it's not clear from the TinkerPop example what this represents). We can set up a similar relationship between software and the people that created it: ```graql insert programmer sub role; insert programmed sub role; insert person plays-role programmer; insert software plays-role programmed; insert programming sub relation, has-role programmer, has-role programmed, has-resource weight; ``` And that's it. At this point, we have defined the schema of the graph. ## Adding the Data Now we have a schema, we can move on to adding in the data, which is pretty much just a typing exercise: ```graql insert $marko isa person, has name "marko", has age 29; insert $vadas isa person, has name "vadas", has age 27; insert $josh isa person, has name "josh", has age 32; insert $peter isa person, has name "peter", has age 35; match $marko has name "marko"; $josh has name "josh"; insert (knower: $marko, known-about: $josh) isa knows has weight 1.0; match $marko has name "marko"; $vadas has name "vadas"; insert (knower: $marko, known-about: $vadas) isa knows has weight 0.5; ``` ```graql insert $lop isa software, has lang "java", has name "lop"; insert $ripple isa software, has lang "java", has name "ripple"; match $marko has name "marko"; $lop has name "lop"; insert (programmer: $marko, programmed: $lop) isa programming has weight 0.4; match $peter has name "peter"; $lop has name "lop"; insert (programmer: $peter, programmed: $lop) isa programming has weight 0.2; match $josh has name "josh"; $lop has name "lop"; insert (programmer: $josh, programmed: $lop) isa programming has weight 0.4; match $josh has name "josh"; $ripple has name "ripple"; insert (programmer: $josh, programmed: $ripple) isa programming has weight 1.0; ``` ## Querying This example is designed to get you up close and personal with Graql queries. It will run through a few basic examples, then ask you a set of "Test Yourself" questions. OK, so if you've followed the above, you should now have a schema and some data in a graph. How do you go about using the graph to answer your queries? That's where the `match` statement comes in. As with any query language, you use a variable to receive the results of the match query, which you must prefix with a `$`. So, to make the query "List every person in the graph", you would use the following in Graql: ```graql >>> match $x isa person, has name $n; select $n; $n value "vadas" isa name; $n value "marko" isa name; $n value "josh" isa name; $n value "peter" isa name; ``` In Graql, a match is formed of three parts: the `match` statement, an optional `select` statement, and any other optional [modifiers](../graql/match-queries.html#modifiers) that you choose to apply to the listing of results. Only the first part of a match query is needed: the modifier parts are optional. In the `match $x isa person` query we are not using any select or delimiters, so let's add some now. We can add a `select` statement to ask Graql to list out every person and to include their id (which is their name) and age. We use `order by` to modify how the results are listed out - in this case, we order them by ascending age, so the youngest person is shown first. ```graql >>> match $x isa person, has name $n, has age $a; select $n, $a; order by $a asc; $x id "vadas" has age "27"; $x id "marko" has age "29"; $x id "josh" has age "32"; $x id "peter" has age "35"; ``` ## Complete Example Here is the complete example - the code to define the schema and insert the data into a graph. You can load this directly into Graql, if you don't want to type it out for yourself. Cut and paste the Graql below and start Graql: ```bash bin/graql.sh ``` Then type edit, which will open up the systems default text editor where you can paste your chunk of text. Upon exiting the editor, the Graql will execute. ```graql insert age sub resource datatype long; name sub resource datatype string; person sub entity; person has-resource age, has-resource name; $marko isa person; $vadas isa person; $josh isa person; $peter isa person; $marko has age 29, has name "marko"; $josh has age 32, has name "josh"; $vadas has age 27, has name "vadas"; $peter has age 35, has name "peter"; weight sub resource datatype double; knower sub role; known-about sub role; person plays-role knower; person plays-role known-about; knows sub relation has-role knower has-role known-about has-resource weight; (knower: $marko, known-about: $josh) isa knows has weight 1.0; (knower: $marko, known-about: $vadas) isa knows has weight 0.5; lang sub resource datatype string; software sub entity; software has-resource lang, has-resource name; $lop isa software; $ripple isa software; $lop has lang "java", has name "lop"; $ripple has lang "java", has name "ripple"; programmer sub role; programmed sub role; person plays-role programmer; software plays-role programmed; programming sub relation has-role programmer has-role programmed has-resource weight; (programmer: $marko, programmed: $lop) isa programming has weight 0.4; (programmer: $peter, programmed: $lop) isa programming has weight 0.2; (programmer: $josh, programmed: $lop) isa programming has weight 0.4; (programmer: $josh, programmed: $ripple) isa programming has weight 1.0; ``` ## Test Yourself 1. List every person with their name and age in ascending age order 2. List every person who has an age over 30 3. List every person who knows someone else, and every person who is known about 4. List every person that Marko knows 5. List every item of software and the language associated with it 6. List everything that Josh has programmed 7. List everyone who has programmed Lop 8. List everything you know about Marko {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/OWL-migration.md --- title: An Example of Migrating OWL to Grakn keywords: migration last_updated: September 19, 2016 tags: [migration, examples] summary: "A short example to illustrate migration of OWL to Grakn" sidebar: documentation_sidebar permalink: /documentation/examples/OWL-migration.html folder: documentation comment_issue_id: 27 --- For a comprehensive guide to migration, please see both our [Migration Tutorial](../migration/migration-overview.html) and our additional documentation specific to [OWL Migration](../migration/OWL-migration.html). ## Introduction This example addresses the topic of OWL interoperability through a migration of a family tree OWL ontology. The code for this example can be found in our [github repository](https://github.com/graknlabs/sample-projects/tree/master/example-owl-migration). The sole prerequisite of this example is having the Grakn environment installed and the Engine running. If you need help starting Grakn Engine, please see the [setup guide](../get-started/setup-guide.html). ### Running the example You can run this example by running the [`Main`](https://github.com/graknlabs/sample-projects/blob/master/example-owl-migration/src/main/java/Main.java) class. Check out the `OWLResourceMigrator` class for the bulk of the migration code. We run a few queries in the example to prove that the data has been migrated. After running the example, you should be able to answer the following questions (see the bottom of the page for answers - but please don't peek until you've tried it!): + What are the Types in the Family Tree graph in Grakn? + How many people are in the family tree? + How many descendants does Eleanor Pringle have? + Who are the great uncles of Ethel Archer? ## Test Yourself Answers **What are the Types in the Family Tree graph?** Answer: ``` tThing tPerson tMan tWoman ``` **How many people are in the family tree?** Answer: `411` **How many descendants does Eleanor Pringle have?** Answer: `55` **Who are the great uncles of Ethel Archer?** Answer: ``` William Whitfield Harry Whitfield George Whitfield Walter Whitfield James Whitfield ``` ## Where next? After running this OWL migration, check out the [Graql documentation](../graql/graql-overview.html) and the [Graph API documentation](../developing-with-java/graph-api.html) for more instructions on how you can explore the family tree. {% include links.html %} ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/pokemon.md --- title: Pokemon Example keywords: graql, query last_updated: January 2017 tags: [graql, examples] summary: "A short example to illustrate Graql queries" sidebar: documentation_sidebar permalink: /documentation/examples/pokemon.html folder: documentation comment_issue_id: 27 --- This example uses the pokemon ontology and dataset, which is available from [github](https://github.com/graknlabs/grakn/blob/master/grakn-dist/src/examples/pokemon.gql) and is also in the *examples* folder in the Grakn distribution zip. You can use it as a primer to work out if you're building a Graql query correctly, to revise your knowledge or just walk through it as you learn Graql. First up, you need to load the ontology and dataset (which are all in one file) as follows. Here we will use the default keyspace, so will clean it before we start Grakn engine and open the Graql shell. ```bash <relative-path-to-Grakn>/bin/grakn.sh clean <relative-path-to-Grakn>/bin/grakn.sh start <relative-path-to-Grakn>/bin/graql.sh -f <relative-path-to-Grakn>/examples/pokemon.gql <relative-path-to-Grakn>/bin/graql.sh ``` Here are the queries. Type each one in a separate line and see what happens. Comments are marked with a #, so you can ignore them. ``` /* Detailed source-code truncated for AI context efficiency. */ ``` ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. --- ## File: pages/documentation/11-examples/projects.md --- title: Example Projects keywords: examples, projects last_updated: October 7, 2016 tags: summary: "A list of potential projects that use Grakn." sidebar: documentation_sidebar permalink: /documentation/examples/projects.html folder: documentation --- ## Introduction This page lists potential research projects that could be based upon Grakn. We have estimated the amount of work involved in them as T-shirt sizes (small, medium and large) and indicated the level of expertise we think would be required. If you are unsure of whether an idea would be suitable for you, please do talk to us. We are always happy to discuss your ideas, either based on these suggestions or for other projects. Just see our [Community page](https://grakn.ai/community.html) for ways to get in touch with us! ## Small The following would particularly suit an undergraduate project. ### Rule-based Game Solver Define a simple ontology, which is just a specification of the components of a system and how they relate, and then devise logical rules to construct a simple application. An example may be to produce a set of rules that can validate that a Sudoku puzzle has been filled correctly. ### Links Between Likes Add a new feature to Moogi that allows users to “like” concepts. Then produce a visualisation that displays the link between the things that a user likes. For more information about Moogi, the film discovery engine that is built on Grakn, please see [https://moogi.co/](https://moogi.co/). ### Moogi Data Cleanup Clean Moogi data and port it to Mindmaps. This requires dabbling a bit with Gremlin, scripting, exploring the graph and ETL probably. ### Graph Database Pros and Cons Write a review of Graph Database technologies with pros and cons and example apps. ### JavaScript Driver for Grakn Create a prototype to interface with the Grakn REST API in a simple manner. ### Create a Graph Representation of a Website Write a simple tool that follows links in a website and constructs a graph representation of it in Mindmaps. This could simply be pages and links, but could even pull out structural information such as containers and side panes inside pages. Follow on projects could be: - mapping click stream data onto the graph - and analysing - analysing clusters and shortest paths through the website. ### Graph a Subject's (e.g. Computer Science) Publications Build a graph of academic publications on a particular subject for analysis. ### Sports Results For example, a knowledge graph of the Rio Olympics stats dataset, Premier League results etc. ### Recommendation Engine Shows how to leverage connections in your data to gather insights and start recommending currently unrelated but relevant facts. ## Medium The following would particularly suit an undergraduate project. ### Game Search Engine Develop an application that shows how a graph database can excel over RDBMS for certain domains. For example, a game search engine that has features like search by: genre, ratings, price, keywords, etc. You could pick any area you are interested in (books, music, recipes - the list goes on - the main requirement is something with a lot of highly connected data). Once this project has been tried and tested, it may be interesting to abstract it further and look at building a factory that creates the basic search engine, and can then be customised according to subject area and features. ## Large The following would be most suitable for postgraduate investigation. ### Semantic Machine Learning Using Machine Learning to work with the semantics of the graph. ### Publication Topology Based Ranking Semantic search engine for computer science publications. How do ranking results based on topology compare to user data? Can we use an ensemble measure to get “better” results? ### Life Sciences Knowledge Graph There is potential to use the open data from Elixir and Bioschemas.org to build a life science graph. ### European Broadcast Union The European Broadcast Union are interested interested in semantics and have documented what they are pursuing [here](https://nlpdbpedia2016.files.wordpress.com/2016/09/nlpdbpedia2016_paper_1.pdf). ## Comments Want to leave a comment? Visit [the issues on Github for this page](https://github.com/graknlabs/docs/issues/27) (you'll need a GitHub account). You are also welcome to contribute to our documentation directly via the "Edit me" button at the top of the page. {% include links.html %}