## File: README.md YCSB ==================================== [](https://travis-ci.org/brianfrankcooper/YCSB) Links ----- * To get here, use https://ycsb.site * [Our project docs](https://github.com/brianfrankcooper/YCSB/wiki) * [The original announcement from Yahoo!](https://labs.yahoo.com/news/yahoo-cloud-serving-benchmark/) Getting Started --------------- 1. Download the [latest release of YCSB](https://github.com/brianfrankcooper/YCSB/releases/latest): ```sh curl -O --location https://github.com/brianfrankcooper/YCSB/releases/download/0.17.0/ycsb-0.17.0.tar.gz tar xfvz ycsb-0.17.0.tar.gz cd ycsb-0.17.0 ``` 2. Set up a database to benchmark. There is a README file under each binding directory. 3. Run YCSB command. On Linux: ```sh bin/ycsb.sh load basic -P workloads/workloada bin/ycsb.sh run basic -P workloads/workloada ``` On Windows: ```bat bin/ycsb.bat load basic -P workloads\workloada bin/ycsb.bat run basic -P workloads\workloada ``` Running the `ycsb` command without any argument will print the usage. See https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload for a detailed documentation on how to run a workload. See https://github.com/brianfrankcooper/YCSB/wiki/Core-Properties for the list of available workload properties. Building from source -------------------- YCSB requires the use of Maven 3; if you use Maven 2, you may see [errors such as these](https://github.com/brianfrankcooper/YCSB/issues/406). To build the full distribution, with all database bindings: mvn clean package To build a single database binding: mvn -pl site.ycsb:mongodb-binding -am clean package Running multiple instances and latency percentiles -------------------------------------------------- In general, you shall be interested in 99% percentile (P99) of the latency distribution, and the rest of the tail - 99.9%, 99.99%, 99.999%. The difference between the amount of requests that will be observed by a user that fall into 95% (P95) percentile and 99% percentile may be sufficiently large. For example, see "How Many Nines?" at https://bravenewgeek.com/everything-you-know-about-latency-is-wrong/. The formula to calculate probability of how many clients will observe a specific percentile is: Probability_to_observe = 1 - Percentile ^ Requests That is why almost 30% of the users will observe latency worse than P99 just by loading the default _google.com_ web page: 1 - 0.99 ^ 30 = 0.27 Remember, that - _latencies_ percentiles can't be averaged. Don't fall into this [trap](http://latencytipoftheday.blogspot.com/2014/06/latencytipoftheday-you-cant-average.html). Neither latency averages, nor P99 averages do not make any sense. If you run multiple loaders dump result histograms with: -p hdrhistogram.fileoutput=true -p hdrhistogram.output.path=file.hdr merge them manually and extract required percentiles out of the joined result. Remember that running multiple workloads may distort original workloads distributions they were intended to produce. Merging HDR histogram percentiles --------------------------------- HdrHistogram can serialize its data to HDR files. Use CLI tool to do different operations with your saved histograms https://github.com/nitsanw/HdrLogProcessing. You shall be interested in 3 functions: - Union - to combine result histograms - Summarize - to extract latency percentiles - An ability to print the result into the CSV file and extract tags To extract HDR content into CSV file format use from https://github.com/HdrHistogram/HdrHistogram/: java -cp HdrHistogram-2.1.9.jar org.HdrHistogram.HistogramLogProcessor -i file.hdr -o output_${tag}.csv -csv -tag ${tag} --- ## File: accumulo1.9/README.md ## Quick Start This section describes how to run YCSB on [Accumulo](https://accumulo.apache.org/). ### 1. Start Accumulo See the [Accumulo Documentation](https://accumulo.apache.org/1.9/accumulo_user_manual.html#_installation) for details on installing and running Accumulo. Before running the YCSB test you must create the Accumulo table. Again see the [Accumulo Documentation](https://accumulo.apache.org/1.9/accumulo_user_manual.html#_basic_administration) for details. The default table name is `usertable`. ### 2. Set Up YCSB Download the [latest YCSB](https://github.com/brianfrankcooper/YCSB/releases/latest) file. Follow the instructions. ### 3. Create the Accumulo table By default, YCSB uses a table with the name "usertable". Users must create this table before loading data into Accumulo. For maximum Accumulo performance, the Accumulo table must be pre-split. A simple Ruby script, based on the HBase README, can generate adequate split-point. 10's of Tablets per TabletServer is a good starting point. Unless otherwise specified, the following commands should run on any version of Accumulo. $ echo 'num_splits = 20; puts (1..num_splits).map {|i| "user#{1000+i*(9999-1000)/num_splits}"}' | ruby > /tmp/splits.txt $ accumulo shell -u -p -e "createtable usertable" $ accumulo shell -u -p -e "addsplits -t usertable -sf /tmp/splits.txt" $ accumulo shell -u -p -e "config -t usertable -s table.cache.block.enable=true" Additionally, there are some other configuration properties which can increase performance. These can be set on the Accumulo table via the shell after it is created. Setting the table durability to `flush` relaxes the constraints on data durability during hard power-outages (avoids calls to fsync). Accumulo defaults table compression to `gzip` which is not particularly fast; `snappy` is a faster and similarly-efficient option. The mutation queue property controls how many writes that Accumulo will buffer in memory before performing a flush; this property should be set relative to the amount of JVM heap the TabletServers are given. accumulo> config -s table.durability=flush accumulo> config -s tserver.total.mutation.queue.max=256M accumulo> config -t usertable -s table.file.compress.type=snappy On repeated data loads, the following commands may be helpful to re-set the state of the table quickly. accumulo> createtable tmp --copy-splits usertable --copy-config usertable accumulo> deletetable --force usertable accumulo> renametable tmp usertable accumulo> compact --wait -t accumulo.metadata ### 4. Load Data and Run Tests Load the data: ./bin/ycsb load accumulo1.9 -s -P workloads/workloada \ -p accumulo.zooKeepers=localhost \ -p accumulo.columnFamily=ycsb \ -p accumulo.instanceName=ycsb \ -p accumulo.username=user \ -p accumulo.password=supersecret \ > outputLoad.txt Run the workload test: ./bin/ycsb run accumulo1.9 -s -P workloads/workloada \ -p accumulo.zooKeepers=localhost \ -p accumulo.columnFamily=ycsb \ -p accumulo.instanceName=ycsb \ -p accumulo.username=user \ -p accumulo.password=supersecret \ > outputLoad.txt ## Accumulo Configuration Parameters - `accumulo.zooKeepers` - The Accumulo cluster's [zookeeper servers](https://accumulo.apache.org/1.9/accumulo_user_manual.html#_connecting). - Should contain a comma separated list of of hostname or hostname:port values. - No default value. - `accumulo.columnFamily` - The name of the column family to use to store the data within the table. - No default value. - `accumulo.instanceName` - Name of the Accumulo [instance](https://accumulo.apache.org/1.9/accumulo_user_manual.html#_connecting). - No default value. - `accumulo.username` - The username to use when connecting to Accumulo. - No default value. - `accumulo.password` - The password for the user connecting to Accumulo. - No default value. --- ## File: aerospike/README.md ## Quick Start This section describes how to run YCSB on Aerospike. ### 1. Start Aerospike ### 2. Install Java and Maven ### 3. Set Up YCSB Git clone YCSB and compile: git clone http://github.com/brianfrankcooper/YCSB.git cd YCSB mvn -pl site.ycsb:aerospike-binding -am clean package ### 4. Provide Aerospike Connection Parameters The following connection parameters are available. * `as.host` - The Aerospike cluster to connect to (default: `localhost`) * `as.port` - The port to connect to (default: `3000`) * `as.user` - The user to connect as (no default) * `as.password` - The password for the user (no default) * `as.timeout` - The transaction and connection timeout (in ms, default: `10000`) * `as.namespace` - The namespace to be used for the benchmark (default: `ycsb`) Add them to the workload or set them with the shell command, as in: ./bin/ycsb load aerospike -s -P workloads/workloada -p as.timeout=5000 >outputLoad.txt ### 5. Load Data and Run Tests Load the data: ./bin/ycsb load aerospike -s -P workloads/workloada >outputLoad.txt Run the workload test: ./bin/ycsb run aerospike -s -P workloads/workloada >outputRun.txt --- ## File: arangodb/README.md ## Quick Start This section describes how to run YCSB on ArangoDB. ### 1. Start ArangoDB See https://docs.arangodb.com/Installing/index.html ### 2. Install Java and Maven Go to http://www.oracle.com/technetwork/java/javase/downloads/index.html and get the url to download the rpm into your server. For example: wget http://download.oracle.com/otn-pub/java/jdk/7u40-b43/jdk-7u40-linux-x64.rpm?AuthParam=11232426132 -o jdk-7u40-linux-x64.rpm rpm -Uvh jdk-7u40-linux-x64.rpm Or install via yum/apt-get sudo yum install java-devel Download MVN from http://maven.apache.org/download.cgi wget http://ftp.heanet.ie/mirrors/www.apache.org/dist/maven/maven-3/3.1.1/binaries/apache-maven-3.1.1-bin.tar.gz sudo tar xzf apache-maven-*-bin.tar.gz -C /usr/local cd /usr/local sudo ln -s apache-maven-* maven sudo vi /etc/profile.d/maven.sh Add the following to `maven.sh` export M2_HOME=/usr/local/maven export PATH=${M2_HOME}/bin:${PATH} Reload bash and test mvn bash mvn -version ### 3. Set Up YCSB Clone this YCSB source code: git clone https://github.com/brianfrankcooper/YCSB.git ### 4. Run YCSB Now you are ready to run! First, drop the existing collection: "usertable" under database "ycsb": db._collection("usertable").drop() Then, load the data: ./bin/ycsb load arangodb -s -P workloads/workloada -p arangodb.ip=xxx -p arangodb.port=xxx Then, run the workload: ./bin/ycsb run arangodb -s -P workloads/workloada -p arangodb.ip=xxx -p arangodb.port=xxx See the next section for the list of configuration parameters for ArangoDB. ### 5. Run against ArangoDB 3.0 and previews versions Running YCSB on ArangoDB in version 3.0 or previews versions requires to use HTTP as network protocol. Since VST (VelcoyStream) is the default used protocol one have to set the configuration parameter `arangodb.protocol` to `HTTP_JSON`. For more infos take a look into the official [ArangoDB Java Driver Docs](https://github.com/arangodb/arangodb-java-driver/blob/master/docs/Drivers/Java/Reference/README.md#network-protocol). ## ArangoDB Configuration Parameters - `arangodb.ip` - Default value is `localhost` - `arangodb.port` - Default value is `8529`. - `arangodb.protocol` - Default value is 'VST' - `arangodb.waitForSync` - Default value is `true`. - `arangodb.transactionUpdate` - Default value is `false`. - `arangodb.dropDBBeforeRun` - Default value is `false`. --- ## File: asynchbase/README.md # AsyncHBase Driver for YCSB This driver provides a YCSB workload binding for Apache HBase using an alternative to the included HBase client. AsyncHBase is completely asynchronous for all operations and is particularly useful for write heavy workloads. Note that it supports a subset of the HBase client APIs but supports all public released versions of HBase. ## Quickstart ### 1. Start a HBase Server You need to start a single node or a cluster to point the client at. Please see [Apache HBase Reference Guide](http://hbase.apache.org/book.html) for more details and instructions. ### 2. Set up YCSB Download the [latest YCSB](https://github.com/brianfrankcooper/YCSB/releases/latest) file. Follow the instructions. ### 3. Create a HBase table for testing For best results, use the pre-splitting strategy recommended in [HBASE-4163](https://issues.apache.org/jira/browse/HBASE-4163): ``` hbase(main):001:0> n_splits = 200 # HBase recommends (10 * number of regionservers) hbase(main):002:0> create 'usertable', 'family', {SPLITS => (1..n_splits).map {|i| "user#{1000+i*(9999-1000)/n_splits}"}} ``` *Failing to do so will cause all writes to initially target a single region server*. ### 2. Load a Workload Switch to the root of the YCSB repo and choose the workload you want to run and `load` it first. With the CLI you must provide the column family at a minimum if HBase is running on localhost. Otherwise you must provide connection properties via CLI or the path to a config file. Additional configuration parameters are available below. ``` bin/ycsb load asynchbase -p columnfamily=cf -P workloads/workloada ``` The `load` step only executes inserts into the datastore. After loading data, run the same workload to mix reads with writes. ``` bin/ycsb run asynchbase -p columnfamily=cf -P workloads/workloada ``` ## Configuration Options The following options can be configured using CLI (using the `-p` parameter) or via a JAVA style properties configuration file.. Check the [AsyncHBase Configuration](http://opentsdb.github.io/asynchbase/docs/build/html/configuration.html) project for additional tuning parameters. * `columnfamily`: (Required) The column family to target. * `config`: Optional full path to a configuration file with AsyncHBase options. * `hbase.zookeeper.quorum`: Zookeeper quorum list. * `hbase.zookeeper.znode.parent`: Path used by HBase in Zookeeper. Default is "/hbase". * `debug`: If true, prints debug information to standard out. The default is false. * `clientbuffering`: Whether or not to use client side buffering and batching of write operations. This can significantly improve performance and defaults to true. * `durable`: When set to false, writes and deletes bypass the WAL for quicker responses. Default is true. * `jointimeout`: A timeout value, in milliseconds, for waiting on operations synchronously before an error is thrown. * `prefetchmeta`: Whether or not to read meta for all regions in the table and connect to the proper region servers before starting operations. Defaults to false. --- ## File: azurecosmos/README.md ## Azure Cosmos DB Quick Start This section describes how to run YCSB on Azure Cosmos DB. For more information on Azure Cosmos DB see https://azure.microsoft.com/services/cosmos-db/. ### 1. Setup This benchmark expects you to have pre-created the database "ycsb" and collection "usertable" before running the commands. When prompted for a Partition Key, use "id". For RUs, select a value you want to benchmark. [RUs are the measure of provisioned thoughput](https://docs.microsoft.com/azure/cosmos-db/request-units) that Azure Cosmos DB defines. The higher the RUs, the more throughput you will get. You can override the default database name with the azurecosmos.databaseName configuration value for side-by-side benchmarking. You must set the uri and the primaryKey in the azurecosmos.properties file in the commands below. $YCSB_HOME/bin/ycsb load azurecosmos -P workloads/workloada -P azurecosmos/conf/azurecosmos.properties $YCSB_HOME/bin/ycsb run azurecosmos -P workloads/workloada -P azurecosmos/conf/azurecosmos.properties Optionally you can set the uri and primaryKey as follows: $YCSB_HOME/bin/ycsb load azurecosmos -P workloads/workloada -p azurecosmos.primaryKey= -p azurecosmos.uri= ### 2. Cosmos DB Configuration Parameters #### Required parameters - azurecosmos.uri < uri string > : - Path to your Azure Cosmos DB account and can be obtained from the portal. It will look like the following: https://.documents.azure.com:443/ - azurecosmos.primaryKey < key string > : - Obtained from the portal. The primary key is used to allow both read & write operations. If you are doing read only workloads you can substitute the readonly key from the portal. #### Options parameters - azurecosmos.databaseName < name string > : - Name of the database to use. - Default: ycsb - azurecosmos.useUpsert (true | false): - Set to true to allow inserts to update existing documents. If this is false and a document already exists the insert will fail. - Default: false - azurecosmos.includeExceptionStackInLog (true | false): - Determines if the full stack should be included in the log when an error happens. - The default is false to reduce output size. - Default: false - azurecosmos.userAgent < agent string >: - The value to be appended to the user-agent header. - In most cases, you should leave this as "azurecosmos-ycsb". - Default: "azurecosmos-ycsb" - azurecosmos.useGateway (true | false): - Specify if connection mode should use gateway as opposed to direct. By default, direct mode will be used, as the performance is generally better. - Default: false - azurecosmos.consistencyLevel (STRONG | BOUNDED_STALENESS | SESSION | CONSISTENT_PREFIX | EVENTUAL): - If not specified, session level will be used by default. - Default: SESSION - azurecosmos.maxRetryAttemptsOnThrottledRequests < integer > - Set the maximum number of retries in the case where the request fails due to rate limiting. - Default: uses default value of azurecosmos Java SDK - azurecosmos.maxRetryWaitTimeInSeconds < integer > - Sets the maximum timeout to for retry in seconds. - Default: uses default value of azurecosmos Java SDK - azurecosmos.gatewayMaxConnectionPoolSize < integer > - Set the value of the connection pool size in gateway mode. - azurecosmos.directMaxConnectionsPerEndpoint < integer > - Set the value of the max connections per endpoint in direct mode. - azurecosmos.gatewayIdleConnectionTimeoutInSeconds < integer > - Sets the value of the timeout in seconds for an idle connection in gateway mode. After that time, the connection will be automatically closed. - Default: uses default value of azurecosmos Java SDK - azurecosmos.directIdleConnectionTimeoutInSeconds < integer > - Sets the value of the timeout in seconds for an idle connection in direct mode. After that time, the connection will be automatically closed. - Default: uses default value of azurecosmos Java SDK - azurecosmos.maxDegreeOfParallelism < integer > - Sets the number of concurrent operations run client side during parallel query execution. - Default: -1 - azurecosmos.maxBufferedItemCount < integer > - Sets the maximum number of items that can be buffered client side during parallel query execution. - Default: 0 - azurecosmos.preferredPageSize < integer > - Sets the preferred page size when scanning. - Default: -1 These parameters are also defined in a template configuration file in the following location: $YCSB_HOME/azurecosmos/conf/azurecosmos.properties ### 3. FAQs ### 4. Example command ./bin/ycsb run azurecosmos -P workloads/workloadc -p azurecosmos.primaryKey= -p azurecosmos.uri=https://.documents.azure.com:443/ -p recordcount=100 -p operationcount=100 --- ## File: azuretablestorage/README.md ## Quick Start This section describes how to run YCSB on Azure table storage. ### 1. Create an Azure Storage account. ### https://azure.microsoft.com/en-us/documentation/articles/storage-create-storage-account/#create-a-storage-account ### 2. Install Java and Maven ### 3. Set Up YCSB Git clone YCSB and compile: git clone http://github.com/brianfrankcooper/YCSB.git cd YCSB mvn -pl site.ycsb:azuretablestorage-binding -am clean package ### 4. Provide Azure Storage parameters Set the account name and access key. - `azure.account` - `azure.key` Or, you can set configs with the shell command, EG: ./bin/ycsb load azuretablestorage -s -P workloads/workloada -p azure.account=YourAccountName -p azure.key=YourAccessKey > outputLoad.txt ### 5. Load data and run tests Load the data: ./bin/ycsb load azuretablestorage -s -P workloads/workloada -p azure.account=YourAccountName -p azure.key=YourAccessKey > outputLoad.txt Run the workload test: ./bin/ycsb run azuretablestorage -s -P workloads/workloada -p azure.account=YourAccountName -p azure.key=YourAccessKey > outputRun.txt ### 6. Optional Azure Storage parameters - `azure.batchsize` Could be between 1 ~ 100. Insert records to table in batch if batchsize > 1. - `azure.protocol` https(in default) or http. - `azure.table` The name of the table('usertable' in default). - `azure.partitionkey` The partitionkey('Test' in default). - `azure.endpoint` For Azure stack WOSS. EG: ./bin/ycsb load azuretablestorage -s -P workloads/workloada -p azure.account=YourAccountName -p azure.key=YourAccessKey -p azure.batchsize=100 -p azure.protocol=http --- ## File: cassandra/README.md # Apache Cassandra 2.x CQL binding Binding for [Apache Cassandra](http://cassandra.apache.org), using the CQL API via the [DataStax driver](http://docs.datastax.com/en/developer/java-driver/2.1/java-driver/whatsNew2.html). To run against the (deprecated) Cassandra Thrift API, use the `cassandra-10` binding. ## Creating a table for use with YCSB For keyspace `ycsb`, table `usertable`: cqlsh> create keyspace ycsb WITH REPLICATION = {'class' : 'SimpleStrategy', 'replication_factor': 3 }; cqlsh> USE ycsb; cqlsh> create table usertable ( y_id varchar primary key, field0 varchar, field1 varchar, field2 varchar, field3 varchar, field4 varchar, field5 varchar, field6 varchar, field7 varchar, field8 varchar, field9 varchar); **Note that `replication_factor` and consistency levels (below) will affect performance.** ## Cassandra Configuration Parameters - `hosts` (**required**) - Cassandra nodes to connect to. - No default. * `port` * CQL port for communicating with Cassandra cluster. * Default is `9042`. - `cassandra.keyspace` Keyspace name - must match the keyspace for the table created (see above). See http://docs.datastax.com/en/cql/3.1/cql/cql_reference/create_keyspace_r.html for details. - Default value is `ycsb` - `cassandra.username` - `cassandra.password` - Optional user name and password for authentication. See http://docs.datastax.com/en/cassandra/2.0/cassandra/security/security_config_native_authenticate_t.html for details. * `cassandra.readconsistencylevel` * `cassandra.writeconsistencylevel` * Default value is `QUORUM` - Consistency level for reads and writes, respectively. See the [DataStax documentation](http://docs.datastax.com/en/cassandra/2.0/cassandra/dml/dml_config_consistency_c.html) for details. * `cassandra.maxconnections` * `cassandra.coreconnections` * Defaults for max and core connections can be found here: https://datastax.github.io/java-driver/2.1.8/features/pooling/#pool-size. Cassandra 2.0.X falls under protocol V2, Cassandra 2.1+ falls under protocol V3. * `cassandra.connecttimeoutmillis` * `cassandra.useSSL` * Default value is false. - To connect with SSL set this value to true. * `cassandra.readtimeoutmillis` * Defaults for connect and read timeouts can be found here: https://docs.datastax.com/en/drivers/java/2.0/com/datastax/driver/core/SocketOptions.html. * `cassandra.tracing` * Default is false * https://docs.datastax.com/en/cql/3.3/cql/cql_reference/tracing_r.html --- ## File: cloudspanner/README.md # Cloud Spanner Driver for YCSB This driver provides a YCSB workload binding for Google's Cloud Spanner database, the first relational database service that is both strongly consistent and horizontally scalable. This binding is implemented using the official Java client library for Cloud Spanner which uses GRPC for making calls. For best results, we strongly recommend running the benchmark from a Google Compute Engine (GCE) VM. ## Running a Workload We recommend reading the [general guidelines](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload) in the YCSB documentation, and following the Cloud Spanner specific steps below. ### 1. Set up Cloud Spanner with the Expected Schema Follow the [Quickstart instructions](https://cloud.google.com/spanner/docs/quickstart-console) in the Cloud Spanner documentation to set up a Cloud Spanner instance, and create a database with the following schema: ``` CREATE TABLE usertable ( id STRING(MAX), field0 STRING(MAX), field1 STRING(MAX), field2 STRING(MAX), field3 STRING(MAX), field4 STRING(MAX), field5 STRING(MAX), field6 STRING(MAX), field7 STRING(MAX), field8 STRING(MAX), field9 STRING(MAX), ) PRIMARY KEY(id); ``` Make note of your project ID, instance ID, and database name. ### 2. Set Up Your Environment and Auth Follow the [set up instructions](https://cloud.google.com/spanner/docs/getting-started/set-up) in the Cloud Spanner documentation to set up your environment and authentication. When not running on a GCE VM, make sure you run `gcloud auth application-default login`. ### 3. Edit Properties In your YCSB root directory, edit `cloudspanner/conf/cloudspanner.properties` and specify your project ID, instance ID, and database name. ### 4. Run the YCSB Shell Start the YCBS shell connected to Cloud Spanner using the following command: ``` ./bin/ycsb shell cloudspanner -P cloudspanner/conf/cloudspanner.properties ``` You can use the `insert`, `read`, `update`, `scan`, and `delete` commands in the shell to experiment with your database and make sure the connection works. For example, try the following: ``` insert name field0=adam read name field0 delete name ``` ### 5. Load the Data You can load, say, 10 GB of data into your YCSB database using the following command: ``` ./bin/ycsb load cloudspanner -P cloudspanner/conf/cloudspanner.properties -P workloads/workloada -p recordcount=10000000 -p cloudspanner.batchinserts=1000 -threads 10 -s ``` We recommend batching insertions so as to reach ~1 MB of data per commit request; this is controlled via the `cloudspanner.batchinserts` parameter which we recommend setting to `1000` during data load. If you wish to load a large database, you can run YCSB on multiple client VMs in parallel and use the `insertstart` and `insertcount` parameters to distribute the load as described [here](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload-in-Parallel). In this case, we recommend the following: * Use ordered inserts via specifying the YCSB parameter `insertorder=ordered`; * Use zero-padding so that ordered inserts are actually lexicographically ordered; the option `zeropadding = 12` is set in the default `cloudspanner.properties` file; * Split the key range evenly between client VMs; * Use few threads on each client VM, so that each individual commit request contains keys which are (close to) consecutive, and would thus likely address a single split; this also helps avoid overloading the servers. The idea is that we have a number of 'write heads' which are all writing to different parts of the database (and thus talking to different servers), but each individual head is writing its own data (more or less) in order. See the [best practices page](https://cloud.google.com/spanner/docs/bulk-loading) for further details. ### 6. Run a Workload After data load, you can a run a workload, say, workload B, using the following command: ``` ./bin/ycsb run cloudspanner -P cloudspanner/conf/cloudspanner.properties -P workloads/workloadb -p recordcount=10000000 -p operationcount=1000000 -threads 10 -s ``` Make sure that you use the same `insertorder` (i.e. `ordered` or `hashed`) and `zeropadding` as specified during the data load. Further details about running workloads are given in the [YCSB wiki pages](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload). ## Configuration Options In addition to the standard YCSB parameters, the following Cloud Spanner specific options can be configured using the `-p` parameter or in `cloudspanner/conf/cloudspanner.properties`. * `cloudspanner.database`: (Required) The name of the database created in the instance, e.g. `ycsb-database`. * `cloudspanner.instance`: (Required) The ID of the Cloud Spanner instance, e.g. `ycsb-instance`. * `cloudspanner.project`: The ID of the project containing the Cloud Spanner instance, e.g. `myproject`. This is not strictly required and can often be automatically inferred from the environment. * `cloudspanner.readmode`: Allows choosing between the `read` and `query` interface of Cloud Spanner. The default is `query`. * `cloudspanner.batchinserts`: The number of inserts to batch into a single commit request. The default value is 1 which means no batching is done. Recommended value during data load is 1000. * `cloudspanner.boundedstaleness`: Number of seconds we allow reads to be stale for. Set to 0 for strong reads (default). For performance gains, this should be set to 10 seconds. --- ## File: couchbase/README.md # Couchbase Driver for YCSB This driver is a binding for the YCSB facilities to operate against a Couchbase Server cluster. It uses the official Couchbase Java SDK and provides a rich set of configuration options. ## Quickstart ### 1. Start Couchbase Server You need to start a single node or a cluster to point the client at. Please see [http://couchbase.com](couchbase.com) for more details and instructions. ### 2. Set up YCSB You need to clone the repository and compile everything. ``` git clone git://github.com/brianfrankcooper/YCSB.git cd YCSB mvn clean package ``` ### 3. Run the Workload Before you can actually run the workload, you need to "load" the data first. ``` bin/ycsb load couchbase -s -P workloads/workloada ``` Then, you can run the workload: ``` bin/ycsb run couchbase -s -P workloads/workloada ``` Please see the general instructions in the `doc` folder if you are not sure how it all works. You can apply a property (as seen in the next section) like this: ``` bin/ycsb run couchbase -s -P workloads/workloada -p couchbase.useJson=false ``` ## Scans in the CouchbaseClient The scan operation in the CouchbaseClient requires a Couchbase View to be created manually. To do this: 1. Go to the Couchbase UI, then to Views 2. Create a new development view, specify a ddoc and view name, use these in your YCSB properties. See Configuration Options below. 3. The default map code is sufficient. 4. Save, and publish this View. ## Configuration Options Since no setup is the same and the goal of YCSB is to deliver realistic benchmarks, here are some setups that you can tune. Note that if you need more flexibility (let's say a custom transcoder), you still need to extend this driver and implement the facilities on your own. You can set the following properties (with the default settings applied): - couchbase.url=http://127.0.0.1:8091/pools => The connection URL from one server. - couchbase.bucket=default => The bucket name to use. - couchbase.password= => The password of the bucket. - couchbase.checkFutures=true => If the futures should be inspected (makes ops sync). - couchbase.persistTo=0 => Observe Persistence ("PersistTo" constraint). - couchbase.replicateTo=0 => Observe Replication ("ReplicateTo" constraint). - couchbase.ddoc => The ddoc name used for scanning - couchbase.view => The view name used for scanning - couchbase.stale => How to deal with stale values in View Query for scanning. (OK, FALSE, UPDATE_AFTER) - couchbase.json=true => Use json or java serialization as target format.