# Technical Documentation: ntop/ntopng > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/ntop/ntopng) ### Overview (`README.md`) # ntopng ntopng is a web-based network traffic monitoring application released under GPLv3. It is the new incarnation of the original ntop written in 1998, and now revamped in terms of performance, usability, and features. While you can read more about ntopng on the ntop web site (http://www.ntop.org), we suggest you to start reading the doc/README.ntopng file for learning how to compile and use ntopng. If instead of source code you prefer to use a pre-built package, please go to http://packages.ntop.org We build binary packages for the following platforms: * Ubuntu Linux Server x64 * CentOS/RedHat Linux x64 * Windows x64 * BeagleBoard ARM (based on Ubuntu Linux) * Ubiquity Networks EdgeRouter (MIPS) Enjoy. --- ### Overview (`third-party/zeromq-4.0.5/README.md`) # ZeroMQ [](https://travis-ci.org/zeromq/libzmq) ## Welcome The 0MQ lightweight messaging kernel is a library which extends the standard socket interfaces with features traditionally provided by specialised messaging middleware products. 0MQ sockets provide an abstraction of asynchronous message queues, multiple messaging patterns, message filtering (subscriptions), seamless access to multiple transport protocols and more. ## Stable Fork This repository is used for making stable releases. Please do not send pull requests here unless they are specifically and only for this fork. Rather, send them to libzmq/master. Issues with test cases may be back ported to this repository by its maintainers. ## Building and installation See the INSTALL file included with the distribution. ## Resources Extensive documentation is provided with the distribution. Refer to doc/zmq.html, or "man zmq" after you have installed 0MQ on your system. Website: http://www.zeromq.org/ Development mailing list: zeromq-dev@lists.zeromq.org Announcements mailing list: zeromq-announce@lists.zeromq.org Git repository: http://github.com/zeromq/libzmq 0MQ developers can also be found on the IRC channel #zeromq, on the Freenode network (irc.freenode.net). ## Copying Free use of this software is granted under the terms of the GNU Lesser General Public License (LGPL). For details see the files `COPYING` and `COPYING.LESSER` included with the 0MQ distribution. --- ### Doc README (`doc/README.txt`) Documentation is generated using the doxygen tool that uses graphviz for drawing graphs. In order to generate Lua docs you need to install https://github.com/alecchen/doxygen-lua as follows: cpan App::cpanminus cpan inc::Module::Install git clone https://github.com/alecchen/doxygen-lua.git cd doxygen-lua/ perl Makefile.PL make sudo make install sudo cp bin/lua2dox /opt/local/bin/ --- ### Third Party Rrdtool 1.4.8 Doc Bin Dec Hex (`third-party/rrdtool-1.4.8/doc/bin_dec_hex.txt`) BIN_DEC_HEX(1) rrdtool BIN_DEC_HEX(1) NNAAMMEE bin_dec_hex - How to use binary, decimal, and hexadecimal notation. DDEESSCCRRIIPPTTIIOONN Most people use the decimal numbering system. This system uses ten symbols to represent numbers. When those ten symbols are used up, they start all over again and increment the position to the left. The digit 0 is only shown if it is the only symbol in the sequence, or if it is not the first one. If this sounds cryptic to you, this is what I've just said in numbers: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 and so on. Each time the digit nine is incremented, it is reset to 0 and the position before (to the left) is incremented (from 0 to 1). Then number 9 can be seen as "00009" and when we should increment 9, we reset it to zero and increment the digit just before the 9 so the number becomes "00010". Leading zeros we don't write except if it is the only digit (number 0). And of course, we write zeros if they occur anywhere inside or at the end of a number: "00010" -> " 0010" -> " 010" -> " 10", but not " 1 ". This was pretty basic, you already knew this. Why did I tell it? Well, computers usually do not represent numbers with 10 different digits. They only use two different symbols, namely "0" and "1". Apply the same rules to this set of digits and you get the binary numbering system: 0 1 10 11 100 101 110 111 1000 1001 1010 1011 1100 1101 and so on. If you count the number of rows, you'll see that these are again 14 different numbers. The numbers are the same and mean the same as in the first list, we just used a different representation. This means that you have to know the representation used, or as it is called the numbering system or base. Normally, if we do not explicitly specify the numbering system used, we implicitly use the decimal system. If we want to use any other numbering system, we'll have to make that clear. There are a few widely adopted methods to do so. One common form is to write 1010(2) which means that you wrote down a number in its binary representation. It is the number ten. If you would write 1010 without specifying the base, the number is interpreted as one thousand and ten using base 10. In books, another form is common. It uses subscripts (little characters, more or less in between two rows). You can leave out the parentheses in that case and write down the number in normal characters followed by a little two just behind it. As the numbering system used is also called the base, we talk of the number 1100 base 2, the number 12 base 10. Within the binary system, it is common to write leading zeros. The numbers are written down in series of four, eight or sixteen depending on the context. We can use the binary form when talking to computers (...programming...), but the numbers will have large representations. The number 65'535 (often in the decimal system a ' is used to separate blocks of three digits for readability) would be written down as 1111111111111111(2) which is 16 times the digit 1. This is difficult and prone to errors. Therefore, we usually would use another base, called hexadecimal. It uses 16 different symbols. First the symbols from the decimal system are used, thereafter we continue with alphabetic characters. We get 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E and F. This system is chosen because the hexadecimal form can be converted into the binary system very easily (and back). There is yet another system in use, called the octal system. This was more common in the old days, but is not used very often anymore. As you might find it in use sometimes, you should get used to it and we'll show it below. It's the same story as with the other representations, but with eight different symbols. Binary (2) Octal (8) Decimal (10) Hexadecimal (16) (2) (8) (10) (16) 00000 0 0 0 00001 1 1 1 00010 2 2 2 00011 3 3 3 00100 4 4 4 00101 5 5 5 00110 6 6 6 00111 7 7 7 01000 10 8 8 01001 11 9 9 01010 12 10 A 01011 13 11 B 01100 14 12 C 01101 15 13 D 01110 16 14 E 01111 17 15 F 10000 20 16 10 10001 21 17 11 10010 22 18 12 10011 23 19 13 10100 24 20 14 10101 25 21 15 Most computers used nowadays are using bytes of eight bits. This means that they store eight bits at a time. You can see why the octal system is not the most practical for that: You'd need three digits to represent the eight bits and this means that you'd have to use one complete digit to represent only two bits (2+3+3=8). This is a waste. For hexadecimal digits, you need only two digits which are used completely: (2) (8) (10) (16) 11111111 377 255 FF You can see why binary and hexadecimal can be converted quickly: For each hexadecimal digit there are exactly four binary digits. Take a binary number: take four digits from the right and make a hexadecimal digit from it (see the table above). Repeat this until there are no more digits. And the other way around: Take a hexadecimal number. For each digit, write down its binary equivalent. Computers (or rather the parsers running on them) would have a hard time converting a number like 1234(16). Therefore hexadecimal numbers are specified with a prefix. This prefix depends on the language you're writing in. Some of the prefixes are "0x" for C, "$" for Pascal, "#" for HTML. It is common to assume that if a number starts with a zero, it is octal. It does not matter what is used as long as you know what it is. I will use "0x" for hexadecimal, "%" for binary and "0" for octal. The following numbers are all the same, just their representation (base) is different: 021 0x11 17 %00010001 To do arithmetics and conversions you need to understand one more thing. It is something you already know but perhaps you do not "see" it yet: If you write down 1234, (no prefix, so it is decimal) you are talking about the number one thousand, two hundred and thirty four. In sort of a formula: 1 * 1000 = 1000 2 * 100 = 200 3 * 10 = 30 4 * 1 = 4 This can also be written as: 1 * 10^3 2 * 10^2 3 * 10^1 4 * 10^0 where ^ means "to the power of". We are using the base 10, and the positions 0,1,2 and 3. The right- most position should NOT be multiplied with 10. The second from the right should be multiplied one time with 10. The third from the right is multiplied with 10 two times. This continues for whatever positions are used. It is the same in all other representations: 0x1234 will be 1 * 16^3 2 * 16^2 3 * 16^1 4 * 16^0 01234 would be 1 * 8^3 2 * 8^2 3 * 8^1 4 * 8^0 This example can not be done for binary as that system only uses two symbols. Another example: %1010 would be 1 * 2^3 0 * 2^2 1 * 2^1 0 * 2^0 It would have been easier to convert it to its hexadecimal form and just translate %1010 into 0xA. After a while you get used to it. You will not need to do any calculations anymore, but just know that 0xA means 10. To convert a decimal number into a hexadecimal you could use the next method. It will take some time to be able to do the estimates, but it will be easier when you use the system more frequently. We'll look at yet another way afterwards. First you need to know how many positions will be used in the other system. To do so, you need to know the maximum numbers you'll be using. Well, that's not as hard as it looks. In decimal, the maximum number that you can form with two digits is "99". The maximum for three: "999". The next number would need an extra position. Reverse this idea and you will see that the number can be found by taking 10^3 (10*10*10 is 1000) minus 1 or 10^2 minus one. This can be done for hexadecimal as well: 16^4 = 0x10000 = 65536 16^3 = 0x1000 = 4096 16^2 = 0x100 = 256 16^1 = 0x10 = 16 If a number is smaller than 65'536 it will fit in four positions. If the number is bigger than 4'095, you must use position 4. How many times you can subtract 4'096 from the number without going below zero is the first digit you write down. This will always be a number from 1 to 15 (0x1 to 0xF). Do the same for the other positions. Let's try with 41'029. It is smaller than 16^4 but bigger than 16^3-1. This means that we have to use four positions. We can subtract 16^3 from 41'029 ten times without going below zero. The left-most digit will therefore be "A", so we have 0xA????. The number is reduced to 41'029 - 10*4'096 = 41'029-40'960 = 69. 69 is smaller than 16^3 but not bigger than 16^2-1. The second digit is therefore "0" and we now have 0xA0??. 69 is smaller than 16^2 and bigger than 16^1-1. We can subtract 16^1 (which is just plain 16) four times and write down "4" to get 0xA04?. Subtract 64 from 69 (69 - 4*16) and the last digit is 5 --> 0xA045. The other method builds up the number from the right. Let's try 41'029 again. Divide by 16 and do not use fractions (only whole numbers). 41'029 / 16 is 2'564 with a remainder of 5. Write down 5. 2'564 / 16 is 160 with a remainder of 4. Write the 4 before the 5. 160 / 16 is 10 with no remainder. Prepend 45 with 0. 10 / 16 is below one. End here and prepend 0xA. End up with 0xA045. Which method to use is up to you. Use whatever works for you. I use them both without being able to tell what method I use in each case, it just depends on the number, I think. Fact is, some numbers will occur frequently while programming. If the number is close to one I am familiar with, then I will use the first method (like 32'770 which is into 32'768 + 2 and I just know that it is 0x8000 + 0x2 = 0x8002). For binary the same approach can be used. The base is 2 and not 16, and the number of positions will grow rapidly. Using the second method has the advantage that you can see very easily if you should write down a zero or a one: if you divide by two the remainder will be zero if it is an even number and one if it is an odd number: 41029 / 2 = 20514 remainder 1 20514 / 2 = 10257 remainder 0 10257 / 2 = 5128 remainder 1 5128 / 2 = 2564 remainder 0 2564 / 2 = 1282 remainder 0 1282 / 2 = 641 remainder 0 641 / 2 = 320 remainder 1 320 / 2 = 160 remainder 0 160 / 2 = 80 remainder 0 80 / 2 = 40 remainder 0 40 / 2 = 20 remainder 0 20 / 2 = 10 remainder 0 10 / 2 = 5 remainder 0 5 / 2 = 2 remainder 1 2 / 2 = 1 remainder 0 1 / 2 below 0 remainder 1 Write down the results from right to left: %1010000001000101 Group by four: %1010000001000101 %101000000100 0101 %10100000 0100 0101 %1010 0000 0100 0101 Convert into hexadecimal: 0xA045 Group %1010000001000101 by three and convert into octal: %1010000001000101 %1010000001000 101 %1010000001 000 101 %1010000 001 000 101 %1010 000 001 000 101 %1 010 000 001 000 101 %001 010 000 001 000 101 1 2 0 1 0 5 --> 0120105 So: %1010000001000101 = 0120105 = 0xA045 = 41029 Or: 1010000001000101(2) = 120105(8) = A045(16) = 41029(10) Or: 1010000001000101(2) = 120105(8) = A045(16) = 41029 At first while adding numbers, you'll convert them to their decimal form and then back into their original form after doing the addition. If you use the other numbering system often, you will see that you'll be able to do arithmetics directly in the base that is used. In any representation it is the same, add the numbers on the right, write down the right-most digit from the result, remember the other digits and use them in the next round. Continue with the second digit from the right and so on: %1010 + %0111 --> 10 + 7 --> 17 --> %00010001 will become %1010 %0111 + |||| |||+-- add 0 + 1, result is 1, nothing to remember ||+--- add 1 + 1, result is %10, write down 0 and remember 1 |+---- add 0 + 1 + 1(remembered), result = 0, remember 1 +----- add 1 + 0 + 1(remembered), result = 0, remember 1 nothing to add, 1 remembered, result = 1 -------- %10001 is the result, I like to write it as %00010001 For low values, try to do the calculations yourself, then check them with a calculator. The more you do the calculations yourself, the more you'll find that you didn't make mistakes. In the end, you'll do calculi in other bases as easily as you do them in decimal. When the numbers get bigger, you'll have to realize that a computer is not called a computer just to have a nice name. There are many different calculators available, use them. For Unix you could use "bc" which is short for Binary Calculator. It calculates not only in decimal, but in all bases you'll ever want to use (among them Binary). For people on Windows: Start the calculator (start->programs->accessories->calculator) and if necessary click view->scientific. You now have a scientific calculator and can compute in binary or hexadecimal. AAUUTTHHOORR I hope you enjoyed the examples and their descriptions. If you do, help other people by pointing them to this document when they are asking basic questions. They will not only get their answer, but at the same time learn a whole lot more. Alex van den Bogaerdt 1.4.8 2013-05-23 BIN_DEC_HEX(1) --- ### Third Party Rrdtool 1.4.8 Doc Cdeftutorial (`third-party/rrdtool-1.4.8/doc/cdeftutorial.txt`) CDEFTUTORIAL(1) rrdtool CDEFTUTORIAL(1) NNAAMMEE cdeftutorial - Alex van den Bogaerdt's CDEF tutorial DDEESSCCRRIIPPTTIIOONN Intention of this document: to provide some examples of the commonly used parts of RRDtool's CDEF language. If you think some important feature is not explained properly, and if adding it to this document would benefit most users, please do ask me to add it. I will then try to provide an answer in the next release of this tutorial. No feedback equals no changes! Additions to this document are also welcome. -- Alex van den Bogaerdt WWhhyy tthhiiss ttuuttoorriiaall?? One of the powerful parts of RRDtool is its ability to do all sorts of calculations on the data retrieved from its databases. However, RRDtool's many options and syntax make it difficult for the average user to understand. The manuals are good at explaining what these options do; however they do not (and should not) explain in detail why they are useful. As with my RRDtool tutorial: if you want a simple document in simple language you should read this tutorial. If you are happy with the official documentation, you may find this document too simple or even boring. If you do choose to read this tutorial, I also expect you to have read and fully understand my other tutorial. MMoorree rreeaaddiinngg If you have difficulties with the way I try to explain it please read Steve Rader's rpntutorial. It may help you understand how this all works. WWhhaatt aarree CCDDEEFFss?? When retrieving data from an RRD, you are using a "DEF" to work with that data. Think of it as a variable that changes over time (where time is the x-axis). The value of this variable is what is found in the database at that particular time and you can't do any modifications on it. This is what CDEFs are for: they takes values from DEFs and perform calculations on them. SSyynnttaaxx DEF:var_name_1=some.rrd:ds_name:CF CDEF:var_name_2=RPN_expression You first define "var_name_1" to be data collected from data source "ds_name" found in RRD "some.rrd" with consolidation function "CF". Assume the ifInOctets SNMP counter is saved in mrtg.rrd as the DS "in". Then the following DEF defines a variable for the average of that data source: DEF:inbytes=mrtg.rrd:in:AVERAGE Say you want to display bits per second (instead of bytes per second as stored in the database.) You have to define a calculation (hence "CDEF") on variable "inbytes" and use that variable (inbits) instead of the original: CDEF:inbits=inbytes,8,* This tells RRDtool to multiply inbytes by eight to get inbits. I'll explain later how this works. In the graphing or printing functions, you can now use inbits where you would use inbytes otherwise. Note that the variable name used in the CDEF (inbits) must not be the same as the variable named in the DEF (inbytes)! RRPPNN--eexxpprreessssiioonnss RPN is short-hand for Reverse Polish Notation. It works as follows. You put the variables or numbers on a stack. You also put operations (things-to-do) on the stack and this stack is then processed. The result will be placed on the stack. At the end, there should be exactly one number left: the outcome of the series of operations. If there is not exactly one number left, RRDtool will complain loudly. Above multiplication by eight will look like: 1. Start with an empty stack 2. Put the content of variable inbytes on the stack 3. Put the number eight on the stack 4. Put the operation multiply on the stack 5. Process the stack 6. Retrieve the value from the stack and put it in variable inbits We will now do an example with real numbers. Suppose the variable inbytes would have value 10, the stack would be: 1. || 2. |10| 3. |10|8| 4. |10|8|*| 5. |80| 6. || Processing the stack (step 5) will retrieve one value from the stack (from the right at step 4). This is the operation multiply and this takes two values off the stack as input. The result is put back on the stack (the value 80 in this case). For multiplication the order doesn't matter, but for other operations like subtraction and division it does. Generally speaking you have the following order: y = A - B --> y=minus(A,B) --> CDEF:y=A,B,- This is not very intuitive (at least most people don't think so). For the function f(A,B) you reverse the position of "f", but you do not reverse the order of the variables. CCoonnvveerrttiinngg yyoouurr wwiisshheess ttoo RRPPNN First, get a clear picture of what you want to do. Break down the problem in smaller portions until they cannot be split anymore. Then it is rather simple to convert your ideas into RPN. Suppose you have several RRDs and would like to add up some counters in them. These could be, for instance, the counters for every WAN link you are monitoring. You have: router1.rrd with link1in link2in router2.rrd with link1in link2in router3.rrd with link1in link2in Suppose you would like to add up all these counters, except for link2in inside router2.rrd. You need to do: (in this example, "router1.rrd:link1in" means the DS link1in inside the RRD router1.rrd) router1.rrd:link1in router1.rrd:link2in router2.rrd:link1in router3.rrd:link1in router3.rrd:link2in -------------------- + (outcome of the sum) As a mathematical function, this could be written: "add(router1.rrd:link1in , router1.rrd:link2in , router2.rrd:link1in , router3.rrd:link1in , router3.rrd:link2.in)" With RRDtool and RPN, first, define the inputs: DEF:a=router1.rrd:link1in:AVERAGE DEF:b=router1.rrd:link2in:AVERAGE DEF:c=router2.rrd:link1in:AVERAGE DEF:d=router3.rrd:link1in:AVERAGE DEF:e=router3.rrd:link2in:AVERAGE Now, the mathematical function becomes: "add(a,b,c,d,e)" In RPN, there's no operator that sums more than two values so you need to do several additions. You add a and b, add c to the result, add d to the result and add e to the result. push a: a stack contains the value of a push b and add: b,+ stack contains the result of a+b push c and add: c,+ stack contains the result of a+b+c push d and add: d,+ stack contains the result of a+b+c+d push e and add: e,+ stack contains the result of a+b+c+d+e What was calculated here would be written down as: ( ( ( (a+b) + c) + d) + e) > This is in RPN: "CDEF:result=a,b,+,c,+,d,+,e,+" This is correct but it can be made more clear to humans. It does not matter if you add a to b and then add c to the result or first add b to c and then add a to the result. This makes it possible to rewrite the RPN into "CDEF:result=a,b,c,d,e,+,+,+,+" which is evaluated differently: push value of variable a on the stack: a push value of variable b on the stack: a b push value of variable c on the stack: a b c push value of variable d on the stack: a b c d push value of variable e on the stack: a b c d e push operator + on the stack: a b c d e + and process it: a b c P (where P == d+e) push operator + on the stack: a b c P + and process it: a b Q (where Q == c+P) push operator + on the stack: a b Q + and process it: a R (where R == b+Q) push operator + on the stack: a R + and process it: S (where S == a+R) As you can see the RPN expression "a,b,c,d,e,+,+,+,+,+" will evaluate in "((((d+e)+c)+b)+a)" and it has the same outcome as "a,b,+,c,+,d,+,e,+". This is called the commutative law of addition, but you may forget this right away, as long as you remember what it means. Now look at an expression that contains a multiplication: First in normal math: "let result = a+b*c". In this case you can't choose the order yourself, you have to start with the multiplication and then add a to it. You may alter the position of b and c, you must not alter the position of a and b. You have to take this in consideration when converting this expression into RPN. Read it as: "Add the outcome of b*c to a" and then it is easy to write the RPN expression: "result=a,b,c,*,+" Another expression that would return the same: "result=b,c,*,a,+" In normal math, you may encounter something like "a*(b+c)" and this can also be converted into RPN. The parenthesis just tell you to first add b and c, and then multiply a with the result. Again, now it is easy to write it in RPN: "result=a,b,c,+,*". Note that this is very similar to one of the expressions in the previous paragraph, only the multiplication and the addition changed places. When you have problems with RPN or when RRDtool is complaining, it's usually a good thing to write down the stack on a piece of paper and see what happens. Have the manual ready and pretend to be RRDtool. Just do all the math by hand to see what happens, I'm sure this will solve most, if not all, problems you encounter. SSoommee ssppeecciiaall nnuummbbeerrss TThhee uunnkknnoowwnn vvaalluuee Sometimes collecting your data will fail. This can be very common, especially when querying over busy links. RRDtool can be configured to allow for one (or even more) unknown value(s) and calculate the missing update. You can, for instance, query your device every minute. This is creating one so called PDP or primary data point per minute. If you defined your RRD to contain an RRA that stores 5-minute values, you need five of those PDPs to create one CDP (consolidated data point). These PDPs can become unknown in two cases: 1. The updates are too far apart. This is tuned using the "heartbeat" setting. 2. The update was set to unknown on purpose by inserting no value (using the template option) or by using "U" as the value to insert. When a CDP is calculated, another mechanism determines if this CDP is valid or not. If there are too many PDPs unknown, the CDP is unknown as well. This is determined by the xff factor. Please note that one unknown counter update can result in two unknown PDPs! If you only allow for one unknown PDP per CDP, this makes the CDP go unknown! Suppose the counter increments with one per second and you retrieve it every minute: counter value resulting rate 10'000 10'060 1; (10'060-10'000)/60 == 1 10'120 1; (10'120-10'060)/60 == 1 unknown unknown; you don't know the last value 10'240 unknown; you don't know the previous value 10'300 1; (10'300-10'240)/60 == 1 If the CDP was to be calculated from the last five updates, it would get two unknown PDPs and three known PDPs. If xff would have been set to 0.5 which by the way is a commonly used factor, the CDP would have a known value of 1. If xff would have been set to 0.2 then the resulting CDP would be unknown. You have to decide the proper values for heartbeat, number of PDPs per CDP and the xff factor. As you can see from the previous text they define the behavior of your RRA. WWoorrkkiinngg wwiitthh uunnkknnoowwnn ddaattaa iinn yyoouurr ddaattaabbaassee As you have read in the previous chapter, entries in an RRA can be set to the unknown value. If you do calculations with this type of value, the result has to be unknown too. This means that an expression such as "result=a,b,+" will be unknown if either a or b is unknown. It would be wrong to just ignore the unknown value and return the value of the other parameter. By doing so, you would assume "unknown" means "zero" and this is not true. There has been a case where somebody was collecting data for over a year. A new piece of equipment was installed, a new RRD was created and the scripts were changed to add a counter from the old database and a counter from the new database. The result was disappointing, a large part of the statistics seemed to have vanished mysteriously ... They of course didn't, values from the old database (known values) were added to values from the new database (unknown values) and the result was unknown. In this case, it is fairly reasonable to use a CDEF that alters unknown data into zero. The counters of the device were unknown (after all, it wasn't installed yet!) but you know that the data rate through the device had to be zero (because of the same reason: it was not installed). There are some examples below that make this change. IInnffiinniittyy Infinite data is another form of a special number. It cannot be graphed because by definition you would never reach the infinite value. You can think of positive and negative infinity depending on the position relative to zero. RRDtool is capable of representing (-not- graphing!) infinity by stopping at its current maximum (for positive infinity) or minimum (for negative infinity) without knowing this maximum (minimum). Infinity in RRDtool is mostly used to draw an AREA without knowing its vertical dimensions. You can think of it as drawing an AREA with an infinite height and displaying only the part that is visible in the current graph. This is probably a good way to approximate infinity and it sure allows for some neat tricks. See below for examples. WWoorrkkiinngg wwiitthh uunnkknnoowwnn ddaattaa aanndd iinnffiinniittyy Sometimes you would like to discard unknown data and pretend it is zero (or any other value for that matter) and sometimes you would like to pretend that known data is unknown (to discard known-to-be-wrong data). This is why CDEFs have support for unknown data. There are also examples available that show unknown data by using infinity. SSoommee eexxaammpplleess EExxaammppllee:: uussiinngg aa rreecceennttllyy ccrreeaatteedd RRRRDD You are keeping statistics on your router for over a year now. Recently you installed an extra router and you would like to show the combined throughput for these two devices. If you just add up the counters from router.rrd and router2.rrd, you will add known data (from router.rrd) to unknown data (from router2.rrd) for the bigger part of your stats. You could solve this in a few ways: � While creating the new database, fill it with zeros from the start to now. You have to make the database start at or before the least recent time in the other database. � Alternatively, you could use CDEF and alter unknown data to zero. Both methods have their pros and cons. The first method is troublesome and if you want to do that you have to figure it out yourself. It is not possible to create a database filled with zeros, you have to put them in manually. Implementing the second method is described next: What we want is: "if the value is unknown, replace it with zero". This could be written in pseudo-code as: if (value is unknown) then (zero) else (value). When reading the rrdgraph manual you notice the "UN" function that returns zero or one. You also notice the "IF" function that takes zero or one as input. First look at the "IF" function. It takes three values from the stack, the first value is the decision point, the second value is returned to the stack if the evaluation is "true" and if not, the third value is returned to the stack. We want the "UN" function to decide what happens so we combine those two functions in one CDEF. Lets write down the two possible paths for the "IF" function: if true return a if false return b In RPN: "result=x,a,b,IF" where "x" is either true or false. Now we have to fill in "x", this should be the "(value is unknown)" part and this is in RPN: "result=value,UN" We now combine them: "result=value,UN,a,b,IF" and when we fill in the appropriate things for "a" and "b" we're finished: "CDEF:result=value,UN,0,value,IF" You may want to read Steve Rader's RPN guide if you have difficulties with the way I explained this last example. If you want to check this RPN expression, just mimic RRDtool behavior: For any known value, the expression evaluates as follows: CDEF:result=value,UN,0,value,IF (value,UN) is not true so it becomes 0 CDEF:result=0,0,value,IF "IF" will return the 3rd value CDEF:result=value The known value is returned For the unknown value, this happens: CDEF:result=value,UN,0,value,IF (value,UN) is true so it becomes 1 CDEF:result=1,0,value,IF "IF" sees 1 and returns the 2nd value CDEF:result=0 Zero is returned Of course, if you would like to see another value instead of zero, you can use that other value. Eventually, when all unknown data is removed from the RRD, you may want to remove this rule so that unknown data is properly displayed. EExxaammppllee:: bbeetttteerr hhaannddlliinngg ooff uunnkknnoowwnn ddaattaa,, bbyy uussiinngg ttiimmee The above example has one drawback. If you do log unknown data in your database after installing your new equipment, it will also be translated into zero and therefore you won't see that there was a problem. This is not good and what you really want to do is: � If there is unknown data, look at the time that this sample was taken. � If the unknown value is before time xxx, make it zero. � If it is after time xxx, leave it as unknown data. This is doable: you can compare the time that the sample was taken to some known time. Assuming you started to monitor your device on Friday September 17, 1999, 00:35:57 MET DST. Translate this time in seconds since 1970-01-01 and it becomes 937'521'357. If you process unknown values that were received after this time, you want to leave them unknown and if they were "received" before this time, you want to translate them into zero (so you can effectively ignore them while adding them to your other routers counters). Translating Friday September 17, 1999, 00:35:57 MET DST into 937'521'357 can be done by, for instance, using gnu date: date -d "19990917 00:35:57" +%s You could also dump the database and see where the data starts to be known. There are several other ways of doing this, just pick one. Now we have to create the magic that allows us to process unknown values different depending on the time that the sample was taken. This is a three step process: 1. If the timestamp of the value is after 937'521'357, leave it as is. 2. If the value is a known value, leave it as is. 3. Change the unknown value into zero. Lets look at part one: if (true) return the original value We rewrite this: if (true) return "a" if (false) return "b" We need to calculate true or false from step 1. There is a function available that returns the timestamp for the current sample. It is called, how surprisingly, "TIME". This time has to be compared to a constant number, we need "GT". The output of "GT" is true or false and this is good input to "IF". We want "if (time > 937521357) then (return a) else (return b)". This process was already described thoroughly in the previous chapter so lets do it quick: if (x) then a else b where x represents "time>937521357" where a represents the original value where b represents the outcome of the previous example time>937521357 --> TIME,937521357,GT if (x) then a else b --> x,a,b,IF substitute x --> TIME,937521357,GT,a,b,IF substitute a --> TIME,937521357,GT,value,b,IF substitute b --> TIME,937521357,GT,value,value,UN,0,value,IF,IF We end up with: "CDEF:result=TIME,937521357,GT,value,value,UN,0,value,IF,IF" This looks very complex, however, as you can see, it was not too hard to come up with. EExxaammppllee:: PPrreetteennddiinngg wweeiirrdd ddaattaa iissnn''tt tthheerree Suppose you have a problem that shows up as huge spikes in your graph. You know this happens and why, so you decide to work around the problem. Perhaps you're using your network to do a backup at night and by doing so you get almost 10mb/s while the rest of your network activity does not produce numbers higher than 100kb/s. There are two options: 1. If the number exceeds 100kb/s it is wrong and you want it masked out by changing it into unknown. 2. You don't want the graph to show more than 100kb/s. Pseudo code: if (number > 100) then unknown else number or Pseudo code: if (number > 100) then 100 else number. The second "problem" may also be solved by using the rigid option of RRDtool graph, however this has not the same result. In this example you can end up with a graph that does autoscaling. Also, if you use the numbers to display maxima they will be set to 100kb/s. We use "IF" and "GT" again. "if (x) then (y) else (z)" is written down as "CDEF:result=x,y,z,IF"; now fill in x, y and z. For x you fill in "number greater than 100kb/s" becoming "number,100000,GT" (kilo is 1'000 and b/s is what we measure!). The "z" part is "number" in both cases and the "y" part is either "UNKN" for unknown or "100000" for 100kb/s. The two CDEF expressions would be: CDEF:result=number,100000,GT,UNKN,number,IF CDEF:result=number,100000,GT,100000,number,IF EExxaammppllee:: wwoorrkkiinngg oonn aa cceerrttaaiinn ttiimmee ssppaann If you want a graph that spans a few weeks, but would only want to see some routers' data for one week, you need to "hide" the rest of the time frame. Don't ask me when this would be useful, it's just here for the example :) We need to compare the time stamp to a begin date and an end date. Comparing isn't difficult: TIME,begintime,GE TIME,endtime,LE These two parts of the CDEF produce either 0 for false or 1 for true. We can now check if they are both 0 (or 1) using a few IF statements but, as Wataru Satoh pointed out, we can use the "*" or "+" functions as logical AND and logical OR. For "*", the result will be zero (false) if either one of the two operators is zero. For "+", the result will only be false (0) when two false (0) operators will be added. Warning: *any* number not equal to 0 will be considered "true". This means that, for instance, "-1,1,+" (which should be "true or true") will become FALSE ... In other words, use "+" only if you know for sure that you have positive numbers (or zero) only. Let's compile the complete CDEF: DEF:ds0=router1.rrd:AVERAGE CDEF:ds0modified=TIME,begintime,GT,TIME,endtime,LE,*,ds0,UNKN,IF This will return the value of ds0 if both comparisons return true. You could also do it the other way around: DEF:ds0=router1.rrd:AVERAGE CDEF:ds0modified=TIME,begintime,LT,TIME,endtime,GT,+,UNKN,ds0,IF This will return an UNKNOWN if either comparison returns true. EExxaammppllee:: YYoouu ssuussppeecctt ttoo hhaavvee pprroobblleemmss aanndd wwaanntt ttoo sseeee uunnkknnoowwnn ddaattaa.. Suppose you add up the number of active users on several terminal servers. If one of them doesn't give an answer (or an incorrect one) you get "NaN" in the database ("Not a Number") and NaN is evaluated as Unknown. In this case, you would like to be alerted to it and the sum of the remaining values is of no value to you. It would be something like: DEF:users1=location1.rrd:onlineTS1:LAST DEF:users2=location1.rrd:onlineTS2:LAST DEF:users3=location2.rrd:onlineTS1:LAST DEF:users4=location2.rrd:onlineTS2:LAST CDEF:allusers=users1,users2,users3,users4,+,+,+ If you now plot allusers, unknown data in one of users1..users4 will show up as a gap in your graph. You want to modify this to show a bright red line, not a gap. Define an extra CDEF that is unknown if all is okay and is infinite if there is an unknown value: CDEF:wrongdata=allusers,UN,INF,UNKN,IF "allusers,UN" will evaluate to either true or false, it is the (x) part of the "IF" function and it checks if allusers is unknown. The (y) part of the "IF" function is set to "INF" (which means infinity) and the (z) part of the function returns "UNKN". The logic is: if (allusers == unknown) then return INF else return UNKN. You can now use AREA to display this "wrongdata" in bright red. If it is unknown (because allusers is known) then the red AREA won't show up. If the value is INF (because allusers is unknown) then the red AREA will be filled in on the graph at that particular time. AREA:allusers#0000FF:combined user count AREA:wrongdata#FF0000:unknown data SSaammee eexxaammppllee uusseeffuull wwiitthh SSTTAACCKKeedd ddaattaa:: If you use stack in the previous example (as I would do) then you don't add up the values. Therefore, there is no relationship between the four values and you don't get a single value to test. Suppose users3 would be unknown at one point in time: users1 is plotted, users2 is stacked on top of users1, users3 is unknown and therefore nothing happens, users4 is stacked on top of users2. Add the extra CDEFs anyway and use them to overlay the "normal" graph: DEF:users1=location1.rrd:onlineTS1:LAST DEF:users2=location1.rrd:onlineTS2:LAST DEF:users3=location2.rrd:onlineTS1:LAST DEF:users4=location2.rrd:onlineTS2:LAST CDEF:allusers=users1,users2,users3,users4,+,+,+ CDEF:wrongdata=allusers,UN,INF,UNKN,IF AREA:users1#0000FF:users at ts1 STACK:users2#00FF00:users at ts2 STACK:users3#00FFFF:users at ts3 STACK:users4#FFFF00:users at ts4 AREA:wrongdata#FF0000:unknown data If there is unknown data in one of users1..users4, the "wrongdata" AREA will be drawn and because it starts at the X-axis and has infinite height it will effectively overwrite the STACKed parts. You could combine the two CDEF lines into one (we don't use "allusers") if you like. But there are good reasons for writing two CDEFS: � It improves the readability of the script. � It can be used inside GPRINT to display the total number of users. If you choose to combine them, you can substitute the "allusers" in the second CDEF with the part after the equal sign from the first line: CDEF:wrongdata=users1,users2,users3,users4,+,+,+,UN,INF,UNKN,IF If you do so, you won't be able to use these next GPRINTs: COMMENT:"Total number of users seen" GPRINT:allusers:MAX:"Maximum: %6.0lf" GPRINT:allusers:MIN:"Minimum: %6.0lf" GPRINT:allusers:AVERAGE:"Average: %6.0lf" GPRINT:allusers:LAST:"Current: %6.0lf\n" TThhee eexxaammpplleess ffrroomm tthhee RRRRDD ggrraapphh mmaannuuaall ppaaggee DDeeggrreeeess CCeellssiiuuss vvss.. DDeeggrreeeess FFaahhrreennhheeiitt To convert Celsius into Fahrenheit use the formula F=9/5*C+32 rrdtool graph demo.png --title="Demo Graph" \ DEF:cel=demo.rrd:exhaust:AVERAGE \ CDEF:far=9,5,/,cel,*,32,+ \ LINE2:cel#00a000:"D. Celsius" \ LINE2:far#ff0000:"D. Fahrenheit\c" This example gets the DS called "exhaust" from database "demo.rrd" and puts the values in variable "cel". The CDEF used is evaluated as follows: CDEF:far=9,5,/,cel,*,32,+ 1. push 9, push 5 2. push function "divide" and process it the stack now contains 9/5 3. push variable "cel" 4. push function "multiply" and process it the stack now contains 9/5*cel 5. push 32 6. push function "plus" and process it the stack contains now the temperature in Fahrenheit CChhaannggiinngg uunnkknnoowwnn iinnttoo zzeerroo rrdtool graph demo.png --title="Demo Graph" \ DEF:idat1=interface1.rrd:ds0:AVERAGE \ DEF:idat2=interface2.rrd:ds0:AVERAGE \ DEF:odat1=interface1.rrd:ds1:AVERAGE \ DEF:odat2=interface2.rrd:ds1:AVERAGE \ CDEF:agginput=idat1,UN,0,idat1,IF,idat2,UN,0,idat2,IF,+,8,* \ CDEF:aggoutput=odat1,UN,0,odat1,IF,odat2,UN,0,odat2,IF,+,8,* \ AREA:agginput#00cc00:Input Aggregate \ LINE1:aggoutput#0000FF:Output Aggregate These two CDEFs are built from several functions. It helps to split them when viewing what they do. Starting with the first CDEF we would get: idat1,UN --> a 0 --> b idat1 --> c if (a) then (b) else (c) The result is therefore "0" if it is true that "idat1" equals "UN". If not, the original value of "idat1" is put back on the stack. Lets call this answer "d". The process is repeated for the next five items on the stack, it is done the same and will return answer "h". The resulting stack is therefore "d,h". The expression has been simplified to "d,h,+,8,*" and it will now be easy to see that we add "d" and "h", and multiply the result with eight. The end result is that we have added "idat1" and "idat2" and in the process we effectively ignored unknown values. The result is multiplied by eight, most likely to convert bytes/s to bits/s. IInnffiinniittyy ddeemmoo rrdtool graph example.png --title="INF demo" \ DEF:val1=some.rrd:ds0:AVERAGE \ DEF:val2=some.rrd:ds1:AVERAGE \ DEF:val3=some.rrd:ds2:AVERAGE \ DEF:val4=other.rrd:ds0:AVERAGE \ CDEF:background=val4,POP,TIME,7200,%,3600,LE,INF,UNKN,IF \ CDEF:wipeout=val1,val2,val3,val4,+,+,+,UN,INF,UNKN,IF \ AREA:background#F0F0F0 \ AREA:val1#0000FF:Value1 \ STACK:val2#00C000:Value2 \ STACK:val3#FFFF00:Value3 \ STACK:val4#FFC000:Value4 \ AREA:whipeout#FF0000:Unknown This demo demonstrates two ways to use infinity. It is a bit tricky to see what happens in the "background" CDEF. "val4,POP,TIME,7200,%,3600,LE,INF,UNKN,IF" This RPN takes the value of "val4" as input and then immediately removes it from the stack using "POP". The stack is now empty but as a side effect we now know the time that this sample was taken. This time is put on the stack by the "TIME" function. "TIME,7200,%" takes the modulo of time and 7'200 (which is two hours). The resulting value on the stack will be a number in the range from 0 to 7199. For people who don't know the modulo function: it is the remainder after an integer division. If you divide 16 by 3, the answer would be 5 and the remainder would be 1. So, "16,3,%" returns 1. We have the result of "TIME,7200,%" on the stack, lets call this "a". The start of the RPN has become "a,3600,LE" and this checks if "a" is less or equal than "3600". It is true half of the time. We now have to process the rest of the RPN and this is only a simple "IF" function that returns either "INF" or "UNKN" depending on the time. This is returned to variable "background". The second CDEF has been discussed earlier in this document so we won't do that here. Now you can draw the different layers. Start with the background that is either unknown (nothing to see) or infinite (the whole positive part of the graph gets filled). Next you draw the data on top of this background, it will overlay the background. Suppose one of val1..val4 would be unknown, in that case you end up with only three bars stacked on top of each other. You don't want to see this because the data is only valid when all four variables are valid. This is why you use the second CDEF, it will overlay the data with an AREA so the data cannot be seen anymore. If your data can also have negative values you also need to overwrite the other half of your graph. This can be done in a relatively simple way: what you need is the "wipeout" variable and place a negative sign before it: "CDEF:wipeout2=wipeout,-1,*" FFiilltteerriinngg ddaattaa You may do some complex data filtering: MEDIAN FILTER: filters shot noise DEF:var=database.rrd:traffic:AVERAGE CDEF:prev1=PREV(var) CDEF:prev2=PREV(prev1) CDEF:prev3=PREV(prev2) CDEF:median=prev1,prev2,prev3,+,+,3,/ LINE3:median#000077:filtered LINE1:prev2#007700:'raw data' DERIVATE: DEF:var=database.rrd:traffic:AVERAGE CDEF:prev1=PREV(var) CDEF:time=var,POP,TIME CDEF:prevtime=PREV(time) CDEF:derivate=var,prev1,-,time,prevtime,-,/ LINE3:derivate#000077:derivate LINE1:var#007700:'raw data' OOuutt ooff iiddeeaass ffoorr nnooww This document was created from questions asked by either myself or by other people on the RRDtool mailing list. Please let me know if you find errors in it or if you have trouble understanding it. If you think there should be an addition, mail me: Remember: NNoo ffeeeeddbbaacckk eeqquuaallss nnoo cchhaannggeess!! SSEEEE AALLSSOO The RRDtool manpages AAUUTTHHOORR Alex van den Bogaerdt 1.4.8 2013-05-23 CDEFTUTORIAL(1) --- ### Third Party Rrdtool 1.4.8 Doc Librrd (`third-party/rrdtool-1.4.8/doc/librrd.txt`) librrd(3) rrdtool librrd(3) NNAAMMEE librrd - RRD library functions DDEESSCCRRIIPPTTIIOONN lliibbrrrrdd contains most of the functionality in RRRRDDTTooooll. The command line utilities and language bindings are often just wrappers around the code contained in lliibbrrrrdd. This manual page documents the lliibbrrrrdd API. NNOOTTEE:: This document is a work in progress, and should be considered incomplete as long as this warning persists. For more information about the lliibbrrrrdd functions, always consult the source code. CCOORREE FFUUNNCCTTIIOONNSS rrrrdd__dduummpp__ccbb__rr((cchhaarr **ffiilleennaammee,, iinntt oopptt__hheeaaddeerr,, rrrrdd__oouuttppuutt__ccaallllbbaacckk__tt ccbb,, vvooiidd **uusseerr)) In some situations it is necessary to get the output of "rrd_dump" without writing it to a file or the standard output. In such cases an application can ask rrrrdd__dduummpp__ccbb__rr to call an user-defined function each time there is output to be stored somewhere. This can be used, to e.g. directly feed an XML parser with the dumped output or transfer the resulting string in memory. The arguments for rrrrdd__dduummpp__ccbb__rr are the same as for rrrrdd__dduummpp__oopptt__rr except that the output filename parameter is replaced by the user- defined callback function and an additional parameter for the callback function that is passed untouched, i.e. to store information about the callback state needed for the user-defined callback to function properly. Recent versions of rrrrdd__dduummpp__oopptt__rr internally use this callback mechanism to write their output to the file provided by the user. size_t rrd_dump_opt_cb_fileout( const void *data, size_t len, void *user) { return fwrite(data, 1, len, (FILE *)user); } The associated call for rrrrdd__dduummpp__ccbb__rr looks like res = rrd_dump_cb_r(filename, opt_header, rrd_dump_opt_cb_fileout, (void *)out_file); where the last parameter specifies the file handle rrrrdd__dduummpp__oopptt__ccbb__ffiilleeoouutt should write to. There's no specific condition for the callback to detect when it is called for the first time, nor for the last time. If you require this for initialization and cleanup you should do those tasks before and after calling rrrrdd__dduummpp__ccrr__rr respectively. UUTTIILLIITTYY FFUUNNCCTTIIOONNSS _rr_rr_dd____rr_aa_nn_dd_oo_mm_((_)) Generates random numbers just like _r_a_n_d_o_m_(_). This further ensures that the random number generator is seeded exactly once per process. rrrrdd__aadddd__ppttrr((vvooiidd ******ddeesstt,, ssiizzee__tt **ddeesstt__ssiizzee,, vvooiidd **ssrrcc)) Dynamically resize the array pointed to by "dest". "dest_size" is a pointer to the current size of "dest". Upon successful _r_e_a_l_l_o_c_(_), the "dest_size" is incremented by 1 and the "src" pointer is stored at the end of the new "dest". Returns 1 on success, 0 on failure. type **arr = NULL; type *elem = "whatever"; size_t arr_size = 0; if (!rrd_add_ptr(&arr, &arr_size, elem)) handle_failure(); rrrrdd__aadddd__ssttrrdduupp((cchhaarr ******ddeesstt,, ssiizzee__tt **ddeesstt__ssiizzee,, cchhaarr **ssrrcc)) Like "rrd_add_ptr", except adds a "strdup" of the source string. char **arr = NULL; size_t arr_size = NULL; char *str = "example text"; if (!rrd_add_strdup(&arr, &arr_size, str)) handle_failure(); rrrrdd__ffrreeee__ppttrrss((vvooiidd ******ssrrcc,, ssiizzee__tt **ccnntt)) Free an array of pointers allocated by "rrd_add_ptr" or "rrd_add_strdup". Also frees the array pointer itself. On return, the source pointer will be NULL and the count will be zero. /* created as above */ rrd_free_ptrs(&arr, &arr_size); /* here, arr == NULL && arr_size == 0 */ rrrrdd__mmkkddiirr__pp((ccoonnsstt cchhaarr **ppaatthhnnaammee,, mmooddee__tt mmooddee)) Create the directory named "pathname" including all of its parent directories (similar to "mkdir -p" on the command line - see _m_k_d_i_r(1) for more information). The argument "mode" specifies the permissions to use. It is modified by the process's "umask". See _m_k_d_i_r(2) for more details. The function returns 0 on success, a negative value else. In case of an error, "errno" is set accordingly. Aside from the errors documented in _m_k_d_i_r(2), the function may fail with the following errors: EEIINNVVAALL "pathname" is "NULL" or the empty string. EENNOOMMEEMM Insufficient memory was available. aannyy eerrrroorr rreettuurrnneedd bbyy _ss_tt_aa_tt((22)) In contrast to _m_k_d_i_r(2), the function does nnoott fail if "pathname" already exists and is a directory. AAUUTTHHOORR RRD Contributors 1.4.8 2013-05-23 librrd(3) --- ### Third Party Rrdtool 1.4.8 Doc Rpntutorial (`third-party/rrdtool-1.4.8/doc/rpntutorial.txt`) RPNTUTORIAL(1) rrdtool RPNTUTORIAL(1) NNAAMMEE rpntutorial - Reading RRDtool RPN Expressions by Steve Rader DDEESSCCRRIIPPTTIIOONN This tutorial should help you get to grips with RRDtool RPN expressions as seen in CDEF arguments of RRDtool graph. RReeaaddiinngg CCoommppaarriissoonn OOppeerraattoorrss The LT, LE, GT, GE and EQ RPN logic operators are not as tricky as they appear. These operators act on the two values on the stack preceding them (to the left). Read these two values on the stack from left to right inserting the operator in the middle. If the resulting statement is true, then replace the three values from the stack with "1". If the statement if false, replace the three values with "0". For example, think about "2,1,GT". This RPN expression could be read as "is two greater than one?" The answer to that question is "true". So the three values should be replaced with "1". Thus the RPN expression 2,1,GT evaluates to 1. Now consider "2,1,LE". This RPN expression could be read as "is two less than or equal to one?". The natural response is "no" and thus the RPN expression 2,1,LE evaluates to 0. RReeaaddiinngg tthhee IIFF OOppeerraattoorr The IF RPN logic operator can be straightforward also. The key to reading IF operators is to understand that the condition part of the traditional "if X than Y else Z" notation has *already* been evaluated. So the IF operator acts on only one value on the stack: the third value to the left of the IF value. The second value to the left of the IF corresponds to the true ("Y") branch. And the first value to the left of the IF corresponds to the false ("Z") branch. Read the RPN expression "X,Y,Z,IF" from left to right like so: "if X then Y else Z". For example, consider "1,10,100,IF". It looks bizarre to me. But when I read "if 1 then 10 else 100" it's crystal clear: 1 is true so the answer is 10. Note that only zero is false; all other values are true. "2,20,200,IF" ("if 2 then 20 else 200") evaluates to 20. And "0,1,2,IF" ("if 0 then 1 else 2) evaluates to 2. Notice that none of the above examples really simulate the whole "if X then Y else Z" statement. This is because computer programmers read this statement as "if Some Condition then Y else Z". So it's important to be able to read IF operators along with the LT, LE, GT, GE and EQ operators. SSoommee EExxaammpplleess While compound expressions can look overly complex, they can be considered elegantly simple. To quickly comprehend RPN expressions, you must know the algorithm for evaluating RPN expressions: iterate searches from the left to the right looking for an operator. When it's found, apply that operator by popping the operator and some number of values (and by definition, not operators) off the stack. For example, the stack "1,2,3,+,+" gets "2,3,+" evaluated (as "2+3") during the first iteration and is replaced by 5. This results in the stack "1,5,+". Finally, "1,5,+" is evaluated resulting in the answer 6. For convenience, it's useful to write this set of operations as: 1) 1,2,3,+,+ eval is 2,3,+ = 5 result is 1,5,+ 2) 1,5,+ eval is 1,5,+ = 6 result is 6 3) 6 Let's use that notation to conveniently solve some complex RPN expressions with multiple logic operators: 1) 20,10,GT,10,20,IF eval is 20,10,GT = 1 result is 1,10,20,IF read the eval as pop "20 is greater than 10" so push 1 2) 1,10,20,IF eval is 1,10,20,IF = 10 result is 10 read pop "if 1 then 10 else 20" so push 10. Only 10 is left so 10 is the answer. Let's read a complex RPN expression that also has the traditional multiplication operator: 1) 128,8,*,7000,GT,7000,128,8,*,IF eval 128,8,* result is 1024 2) 1024 ,7000,GT,7000,128,8,*,IF eval 1024,7000,GT result is 0 3) 0, 7000,128,8,*,IF eval 128,8,* result is 1024 4) 0, 7000,1024, IF result is 1024 Now let's go back to the first example of multiple logic operators, but replace the value 20 with the variable "input": 1) input,10,GT,10,input,IF eval is input,10,GT ( lets call this A ) Read eval as "if input > 10 then true" and replace "input,10,GT" with "A": 2) A,10,input,IF eval is A,10,input,IF read "if A then 10 else input". Now replace A with it's verbose description again and--voila!--you have an easily readable description of the expression: if input > 10 then 10 else input Finally, let's go back to the first most complex example and replace the value 128 with "input": 1) input,8,*,7000,GT,7000,input,8,*,IF eval input,8,* result is A where A is "input * 8" 2) A,7000,GT,7000,input,8,*,IF eval is A,7000,GT result is B where B is "if ((input * 8) > 7000) then true" 3) B,7000,input,8,*,IF eval is input,8,* result is C where C is "input * 8" 4) B,7000,C,IF At last we have a readable decoding of the complex RPN expression with a variable: if ((input * 8) > 7000) then 7000 else (input * 8) EExxeerrcciisseess Exercise 1: Compute "3,2,*,1,+ and "3,2,1,+,*" by hand. Rewrite them in traditional notation. Explain why they have different answers. Answer 1: 3*2+1 = 7 and 3*(2+1) = 9. These expressions have different answers because the altering of the plus and times operators alter the order of their evaluation. Exercise 2: One may be tempted to shorten the expression input,8,*,56000,GT,56000,input,*,8,IF by removing the redundant use of "input,8,*" like so: input,56000,GT,56000,input,IF,8,* Use traditional notation to show these expressions are not the same. Write an expression that's equivalent to the first expression, but uses the LE and DIV operators. Answer 2: if (input <= 56000/8 ) { input*8 } else { 56000 } input,56000,8,DIV,LE,input,8,*,56000,IF Exercise 3: Briefly explain why traditional mathematic notation requires the use of parentheses. Explain why RPN notation does not require the use of parentheses. Answer 3: Traditional mathematic expressions are evaluated by doing multiplication and division first, then addition and subtraction. Parentheses are used to force the evaluation of addition before multiplication (etc). RPN does not require parentheses because the ordering of objects on the stack can force the evaluation of addition before multiplication. Exercise 4: Explain why it was desirable for the RRDtool developers to implement RPN notation instead of traditional mathematical notation. Answer 4: The algorithm that implements traditional mathematical notation is more complex then algorithm used for RPN. So implementing RPN allowed Tobias Oetiker to write less code! (The code is also less complex and therefore less likely to have bugs.) AAUUTTHHOORR Steve Rader 1.4.8 2013-05-23 RPNTUTORIAL(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrd Beginners (`third-party/rrdtool-1.4.8/doc/rrd-beginners.txt`) RRD-BEGINNERS(1) rrdtool RRD-BEGINNERS(1) NNAAMMEE rrd-beginners - RRDtool Beginners' Guide SSYYNNOOPPSSIISS Helping new RRDtool users to understand the basics of RRDtool DDEESSCCRRIIPPTTIIOONN This manual is an attempt to assist beginners in understanding the concepts of RRDtool. It sheds a light on differences between RRDtool and other databases. With help of an example, it explains the structure of RRDtool database. This is followed by an overview of the "graph" feature of RRDtool. At the end, it has sample scripts that illustrate the usage/wrapping of RRDtool within Shell or Perl scripts. WWhhaatt mmaakkeess RRRRDDttooooll ssoo ssppeecciiaall?? RRDtool is GNU licensed software developed by Tobias Oetiker, a system manager at the Swiss Federal Institute of Technology. Though it is a database, there are distinct differences between RRDtool databases and other databases as listed below: � RRDtool stores data; that makes it a back-end tool. The RRDtool command set allows the creation of graphs; that makes it a front- end tool as well. Other databases just store data and can not create graphs. � In case of linear databases, new data gets appended at the bottom of the database table. Thus its size keeps on increasing, whereas the size of an RRDtool database is determined at creation time. Imagine an RRDtool database as the perimeter of a circle. Data is added along the perimeter. When new data reaches the starting point, it overwrites existing data. This way, the size of an RRDtool database always remains constant. The name "Round Robin" stems from this behavior. � Other databases store the values as supplied. RRDtool can be configured to calculate the rate of change from the previous to the current value and store this information instead. � Other databases get updated when values are supplied. The RRDtool database is structured in such a way that it needs data at predefined time intervals. If it does not get a new value during the interval, it stores an UNKNOWN value for that interval. So, when using the RRDtool database, it is imperative to use scripts that run at regular intervals to ensure a constant data flow to update the RRDtool database. RRDtool is designed to store time series of data. With every data update, an associated time stamp is stored. Time is always expressed in seconds passed since epoch (01-01-1970). RRDtool can be installed on Unix as well as Windows. It comes with a command set to carry out various operations on RRD databases. This command set can be accessed from the command line, as well as from Shell or Perl scripts. The scripts act as wrappers for accessing data stored in RRDtool databases. UUnnddeerrssttaannddiinngg bbyy aann eexxaammppllee The structure of an RRD database is different than other linear databases. Other databases define tables with columns, and many other parameters. These definitions sometimes are very complex, especially in large databases. RRDtool databases are primarily used for monitoring purposes and hence are very simple in structure. The parameters that need to be defined are variables that hold values and archives of those values. Being time sensitive, a couple of time related parameters are also defined. Because of its structure, the definition of an RRDtool database also includes a provision to specify specific actions to take in the absence of update values. Data Source (DS), heartbeat, Date Source Type (DST), Round Robin Archive (RRA), and Consolidation Function (CF) are some of the terminologies related to RRDtool databases. The structure of a database and the terminology associated with it can be best explained with an example. rrdtool create target.rrd \ --start 1023654125 \ --step 300 \ DS:mem:GAUGE:600:0:671744 \ RRA:AVERAGE:0.5:12:24 \ RRA:AVERAGE:0.5:288:31 This example creates a database named _t_a_r_g_e_t_._r_r_d. Start time (1'023'654'125) is specified in total number of seconds since epoch (time in seconds since 01-01-1970). While updating the database, the update time is also specified. This update time MUST be large (later) then start time and MUST be in seconds since epoch. The step of 300 seconds indicates that database expects new values every 300 seconds. The wrapper script should be scheduled to run every sstteepp seconds so that it updates the database every sstteepp seconds. DS (Data Source) is the actual variable which relates to the parameter on the device that is monitored. Its syntax is DS:variable_name:DST:heartbeat:min:max DDSS is a key word. "variable_name" is a name under which the parameter is saved in the database. There can be as many DSs in a database as needed. After every step interval, a new value of DS is supplied to update the database. This value is also called Primary Data Point ((PPDDPP)). In our example mentioned above, a new PDP is generated every 300 seconds. Note, that if you do NOT supply new data points exactly every 300 seconds, this is not a problem, RRDtool will interpolate the data accordingly. DDSSTT (Data Source Type) defines the type of the DS. It can be COUNTER, DERIVE, ABSOLUTE, GAUGE. A DS declared as COUNTER will save the rate of change of the value over a step period. This assumes that the value is always increasing (the difference between the current and the previous value is greater than 0). Traffic counters on a router are an ideal candidate for using COUNTER as DST. DERIVE is the same as COUNTER, but it allows negative values as well. If you want to see the rate of _c_h_a_n_g_e in free disk space on your server, then you might want to use the DERIVE data type. ABSOLUTE also saves the rate of change, but it assumes that the previous value is set to 0. The difference between the current and the previous value is always equal to the current value. Thus it just stores the current value divided by the step interval (300 seconds in our example). GAUGE does not save the rate of change. It saves the actual value itself. There are no divisions or calculations. Memory consumption in a server is a typical example of gauge. The difference between the different types DSTs can be explained better with the following example: Values = 300, 600, 900, 1200 Step = 300 seconds COUNTER DS = 1, 1, 1, 1 DERIVE DS = 1, 1, 1, 1 ABSOLUTE DS = 1, 2, 3, 4 GAUGE DS = 300, 600, 900, 1200 The next parameter is hheeaarrttbbeeaatt. In our example, heartbeat is 600 seconds. If the database does not get a new PDP within 300 seconds, it will wait for another 300 seconds (total 600 seconds). If it doesn't receive any PDP within 600 seconds, it will save an UNKNOWN value into the database. This UNKNOWN value is a special feature of RRDtool - it is much better than to assume a missing value was 0 (zero) or any other number which might also be a valid data value. For example, the traffic flow counter on a router keeps increasing. Lets say, a value is missed for an interval and 0 is stored instead of UNKNOWN. Now when the next value becomes available, it will calculate the difference between the current value and the previous value (0) which is not correct. So, inserting the value UNKNOWN makes much more sense here. The next two parameters are the minimum and maximum value, respectively. If the variable to be stored has predictable maximum and minimum values, this should be specified here. Any update value falling out of this range will be stored as UNKNOWN. The next line declares a round robin archive (RRA). The syntax for declaring an RRA is RRA:CF:xff:step:rows RRA is the keyword to declare RRAs. The consolidation function (CF) can be AVERAGE, MINIMUM, MAXIMUM, and LAST. The concept of the consolidated data point (CDP) comes into the picture here. A CDP is CFed (averaged, maximum/minimum value or last value) from _s_t_e_p number of PDPs. This RRA will hold _r_o_w_s CDPs. Lets have a look at the example above. For the first RRA, 12 (steps) PDPs (DS variables) are AVERAGEed (CF) to form one CDP. 24 (rows) of theses CDPs are archived. Each PDP occurs at 300 seconds. 12 PDPs represent 12 times 300 seconds which is 1 hour. It means 1 CDP (which is equal to 12 PDPs) represents data worth 1 hour. 24 such CDPs represent 1 day (1 hour times 24 CDPs). This means, this RRA is an archive for one day. After 24 CDPs, CDP number 25 will replace the 1st CDP. The second RRA saves 31 CDPs; each CPD represents an AVERAGE value for a day (288 PDPs, each covering 300 seconds = 24 hours). Therefore this RRA is an archive for one month. A single database can have many RRAs. If there are multiple DSs, each individual RRA will save data for all the DSs in the database. For example, if a database has 3 DSs and daily, weekly, monthly, and yearly RRAs are declared, then each RRA will hold data from all 3 data sources. GGrraapphhiiccaall MMaaggiicc Another important feature of RRDtool is its ability to create graphs. The "graph" command uses the "fetch" command internally to retrieve values from the database. With the retrieved values it draws graphs as defined by the parameters supplied on the command line. A single graph can show different DS (Data Sources) from a database. It is also possible to show the values from more than one database in a single graph. Often, it is necessary to perform some math on the values retrieved from the database before plotting them. For example, in SNMP replies, memory consumption values are usually specified in KBytes and traffic flow on interfaces is specified in Bytes. Graphs for these values will be more meaningful if values are represented in MBytes and mbps. The RRDtool graph command allows to define such conversions. Apart from mathematical calculations, it is also possible to perform logical operations such as greater than, less than, and if/then/else. If a database contains more than one RRA archive, then a question may arise - how does RRDtool decide which RRA archive to use for retrieving the values? RRDtool looks at several things when making its choice. First it makes sure that the RRA covers as much of the graphing time frame as possible. Second it looks at the resolution of the RRA compared to the resolution of the graph. It tries to find one which has the same or higher better resolution. With the "-r" option you can force RRDtool to assume a different resolution than the one calculated from the pixel width of the graph. Values of different variables can be presented in 5 different shapes in a graph - AREA, LINE1, LINE2, LINE3, and STACK. AREA is represented by a solid colored area with values as the boundary of this area. LINE1/2/3 (increasing width) are just plain lines representing the values. STACK is also an area but it is "stack"ed on top AREA or LINE1/2/3. Another important thing to note is that variables are plotted in the order they are defined in the graph command. Therefore care must be taken to define STACK only after defining AREA/LINE. It is also possible to put formatted comments within the graph. Detailed instructions can be found in the graph manual. WWrraappppiinngg RRRRDDttooooll wwiitthhiinn SShheellll//PPeerrll ssccrriipptt After understanding RRDtool it is now a time to actually use RRDtool in scripts. Tasks involved in network management are data collection, data storage, and data retrieval. In the following example, the previously created target.rrd database is used. Data collection and data storage is done using Shell scripts. Data retrieval and report generation is done using Perl scripts. These scripts are shown below: _S_h_e_l_l _s_c_r_i_p_t _(_c_o_l_l_e_c_t_s _d_a_t_a_, _u_p_d_a_t_e_s _d_a_t_a_b_a_s_e_) #!/bin/sh a=0 while [ "$a" == 0 ]; do snmpwalk -c public 192.168.1.250 hrSWRunPerfMem > snmp_reply total_mem=`awk 'BEGIN {tot_mem=0} { if ($NF == "KBytes") {tot_mem=tot_mem+$(NF-1)} } END {print tot_mem}' snmp_reply` # I can use N as a replacement for the current time rrdtool update target.rrd N:$total_mem # sleep until the next 300 seconds are full perl -e 'sleep 300 - time % 300' done # end of while loop _P_e_r_l _s_c_r_i_p_t _(_r_e_t_r_i_e_v_e_s _d_a_t_a _f_r_o_m _d_a_t_a_b_a_s_e _a_n_d _g_e_n_e_r_a_t_e_s _g_r_a_p_h_s _a_n_d _s_t_a_t_i_s_t_i_c_s_) #!/usr/bin/perl -w # This script fetches data from target.rrd, creates a graph of memory # consumption on the target (Dual P3 Processor 1 GHz, 656 MB RAM) # call the RRD perl module use lib qw( /usr/local/rrdtool-1.0.41/lib/perl ../lib/perl ); use RRDs; my $cur_time = time(); # set current time my $end_time = $cur_time - 86400; # set end time to 24 hours ago my $start_time = $end_time - 2592000; # set start 30 days in the past # fetch average values from the RRD database between start and end time my ($start,$step,$ds_names,$data) = RRDs::fetch("target.rrd", "AVERAGE", "-r", "600", "-s", "$start_time", "-e", "$end_time"); # save fetched values in a 2-dimensional array my $rows = 0; my $columns = 0; my $time_variable = $start; foreach $line (@$data) { $vals[$rows][$columns] = $time_variable; $time_variable = $time_variable + $step; foreach $val (@$line) { $vals[$rows][++$columns] = $val;} $rows++; $columns = 0; } my $tot_time = 0; my $count = 0; # save the values from the 2-dimensional into a 1-dimensional array for $i ( 0 .. $#vals ) { $tot_mem[$count] = $vals[$i][1]; $count++; } my $tot_mem_sum = 0; # calculate the total of all values for $i ( 0 .. ($count-1) ) { $tot_mem_sum = $tot_mem_sum + $tot_mem[$i]; } # calculate the average of the array my $tot_mem_ave = $tot_mem_sum/($count); # create the graph RRDs::graph ("/images/mem_$count.png", "--title= Memory Usage", "--vertical-label=Memory Consumption (MB)", "--start=$start_time", "--end=$end_time", "--color=BACK#CCCCCC", "--color=CANVAS#CCFFFF", "--color=SHADEB#9999CC", "--height=125", "--upper-limit=656", "--lower-limit=0", "--rigid", "--base=1024", "DEF:tot_mem=target.rrd:mem:AVERAGE", "CDEF:tot_mem_cor=tot_mem,0,671744,LIMIT,UN,0,tot_mem,IF,1024,/", "CDEF:machine_mem=tot_mem,656,+,tot_mem,-", "COMMENT:Memory Consumption between $start_time", "COMMENT: and $end_time ", "HRULE:656#000000:Maximum Available Memory - 656 MB", "AREA:machine_mem#CCFFFF:Memory Unused", "AREA:tot_mem_cor#6699CC:Total memory consumed in MB"); my $err=RRDs::error; if ($err) {print "problem generating the graph: $err\n";} # print the output print "Average memory consumption is "; printf "%5.2f",$tot_mem_ave/1024; print " MB. Graphical representation can be found at /images/mem_$count.png."; AAUUTTHHOORR Ketan Patel 1.4.8 2013-05-23 RRD-BEGINNERS(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdbuild (`third-party/rrdtool-1.4.8/doc/rrdbuild.txt`) RRDBUILD(1) rrdtool RRDBUILD(1) NNAAMMEE rrdbuild - Instructions for building RRDtool OOVVEERRVVIIEEWW If you downloaded the source of RRDtool you have to compile it. This document will give some information on how this is done. RRDtool relies on services of third part libraries. Some of these libraries may already be installed on your system. You have to compile copies of the other ones before you can build RRDtool. This document will tell you about all the necessary steps to get going. These instructions assume you are using a bbaasshh shell. If you use csh/tcsh, then you can either type _b_a_s_h to switch to bash for the compilation or if you know what you are doing just replace the export bits with setenv. We further assume that your copies of ttaarr and mmaakkee are actually GGNNUU ttaarr and GGNNUU mmaakkee respectively. It could be that they are installed as ggttaarr and ggmmaakkee on your system. OOPPTTIIMMIISSTTIICC BBUUIILLDD Before you start to build RRDtool, you have to decide two things: 1. In which directory you want to build the software. 2. Where you want to install the software. Once you have decided. Save the two locations into environment variables. BUILD_DIR=/tmp/rrdbuild INSTALL_DIR=/opt/rrdtool-1.4.8 If your _/_t_m_p is mounted with the option noexec (RHEL seems todo that) you have to choose a different directory! Now make sure the BUILD_DIR exists and go there: mkdir -p $BUILD_DIR cd $BUILD_DIR Lets first assume you already have all the necessary libraries pre- installed. wget http://oss.oetiker.ch/rrdtool/pub/rrdtool-1.4.8.tar.gz gunzip -c rrdtool-1.4.8.tar.gz | tar xf - cd rrdtool-1.4.8 ./configure --prefix=$INSTALL_DIR && make && make install Ok, this was very optimistic. This try will probably have ended with ccoonnffiigguurree complaining about several missing libraries. IINNSSTTAALLLLIINNGG DDEEPPEENNDDEENNCCIIEESS If your OS lets you install additional packages from a software repository, you may get away with installing the missing packages. When the packages are installed, run configure again and try to compile again. Below you find some hints on getting your OS ready for compiling RRDtool. Additions to this list are welcome. In general RRDtool should work with the latest versions of the libraries. The versions listed here are just what was current when I tested this. OOppeennSSoollaarriiss 22000088..0055 Just add a compiler and the gnome development package: pkg install sunstudioexpress pkg install SUNWgnome-common-devel There is a problem with _c_a_i_r_o_._p_c on OpenSolaris. It suggests that xrender is required for compilation with cairo. This is not true and also bad since OpenSolaris does not include an _x_r_e_n_d_e_r_._p_c file. Use Perl to fix this: perl -i~ -p -e 's/(Requires.*?)\s*xrender.*/$1/' /usr/lib/pkgconfig/cairo.pc Make sure the RRDtool build system finds your new compiler export PATH=/opt/SunStudioExpress/bin DDeebbiiaann // UUbbuunnttuu Use apt-get to make sure you have all that is required. A number of packages will get added through dependencies. apt-get install libpango1.0-dev libxml2-dev GGeennttoooo In Gentoo installing RRDtool is really simple you just need to eemmeerrggee rrrrddttooooll. All dependencies will be handled automatically by the portage system. The only thing you should care about are USE flags, which allow you fine tune features RRDtool will be built with. Currently the following USE flags are available: doc - install .html and .txt documentation into /usr/share/doc/rrdtool-1.x.xx/ perl - build and install perl language bindings python - build and install python language bindings ruby - build and install ruby language bindings tcl - build and install tcl language bindings rrdcgi - build and install rrdcgi After you've decided which USE flags you need, set them either in _m_a_k_e_._c_o_n_f or _/_e_t_c_/_p_o_r_t_a_g_e_/_p_a_c_k_a_g_e_._u_s_e and finally run: # emerge -va rrdtool Take a look at Gentoo handbook for further details on how to manage USE flags: http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=2 BBUUIILLDDIINNGG DDEEPPEENNDDEENNCCIIEESS But again this may have been too optimistic still, and you actually have to compile your own copies of some of the required libraries. Things like libpng and zlib are pretty standard so you will probably have them on your system anyway. Freetype, Fontinst, Cairo, Pango may be installed, but it is possible that they are pretty old and thus don't live up to our expectations, so you may want to compile their latest versions. GGeenneerraall bbuuiilldd ttiippss ffoorr AAIIXX If you are working with AIX, you may find the ----ddiissaabbllee--sshhaarreedd option will cause things to break for you. In that case you may have to install the shared libraries into the RRDtool PREFIX and work with ----ddiissaabbllee--ssttaattiicc instead. Another hint to get RRDtool working on AIX is to use the IBM XL C Compiler: export CC=/usr/vac/bin/cc export PERLCC=$CC (Better instructions for AIX welcome!) BBuuiilldd IInnssttrruuccttiioonnss Some libraries want to know where other libraries are. For this to work, set the following environment variable export PKG_CONFIG_PATH=${INSTALL_DIR}/lib/pkgconfig export PATH=$INSTALL_DIR/bin:$PATH The above relies on the presence of the _p_k_g_c_o_n_f_i_g program. Below you find instructions on how to compile pkgconfig. Since we are compiling libraries dynamically, they must know where to find each other. This is done by setting an appropriate LDFLAGS. Unfortunately, the syntax again differs from system to system: Solaris export LDFLAGS=-R${INSTALL_DIR}/lib if you are using the Sun Studio/Forte compiler, you may also want to set CFLAGS="-xO3 -xcode=pic13" (SPARC) CFLAGS="-xO3 -Kpic" (x86) Linux export LDFLAGS="-Wl,--rpath -Wl,${INSTALL_DIR}/lib" HPUX export LDFLAGS="+b${INSTALL_DIR}/lib" AIX export LDFLAGS="-Wl,-blibpath:${INSTALL_DIR}/lib" If you have GNU make installed and it is not called 'make', then do export MAKE=gmake export GNUMAKE=gmake otherwise just do export MAKE=make _B_u_i_l_d_i_n_g _p_k_g_c_o_n_f_i_g As mentioned above, without pkgconfig the whole build process will be lots of pain and suffering, so make sure you have a copy on your system. If it is not available natively, here is how to compile it. wget http://pkgconfig.freedesktop.org/releases/pkg-config-0.23.tar.gz gunzip -c pkg-config-0.23.tar.gz | tar xf - cd pkg-config-0.23 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" $MAKE $MAKE install After installing pkgconfig in a custom directory, setting up the corresponding environment variable will be helpful. export PKG_CONFIG=$INSTALL_DIR/bin/pkg-config _B_u_i_l_d_i_n_g _z_l_i_b Chances are very high that you already have that on your system ... cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/zlib-1.2.3.tar.gz gunzip -c zlib-1.2.3.tar.gz | tar xf - cd zlib-1.2.3 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" --shared $MAKE $MAKE install _B_u_i_l_d_i_n_g _l_i_b_p_n_g Libpng itself requires zlib to build, so we need to help a bit. If you already have a copy of zlib on your system (which is very likely) you can drop the settings of LDFLAGS and CPPFLAGS. Note that the backslash (\) at the end of line 4 means that line 4 and line 5 are on one line. cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/libpng-1.2.18.tar.gz gunzip -c libpng-1.2.18.tar.gz | tar xf - cd libpng-1.2.10 env CFLAGS="-O3 -fPIC" ./configure --prefix=$INSTALL_DIR $MAKE $MAKE install _B_u_i_l_d_i_n_g _f_r_e_e_t_y_p_e cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/freetype-2.3.5.tar.gz gunzip -c freetype-2.3.5.tar.gz | tar xf - cd freetype-2.3.5 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" $MAKE $MAKE install If you run into problems building freetype on Solaris, you may want to try to add the following at the start the configure line: env EGREP=egrep _B_u_i_l_d_i_n_g _L_i_b_X_M_L_2 cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/libxml2-2.6.32.tar.gz gunzip -c libxml2-2.6.32.tar.gz | tar xf - cd libxml2-2.6.32 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" $MAKE $MAKE install _B_u_i_l_d_i_n_g _f_o_n_t_c_o_n_f_i_g Note that fontconfig has a run time configuration file in INSTALL_DIR/etc you may want to adjust that so that fontconfig finds the fonts on your system. Run the fc-cache program to build the fontconfig cache after changing the config file. cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/fontconfig-2.4.2.tar.gz gunzip -c fontconfig-2.4.2.tar.gz | tar xf - cd fontconfig-2.4.2 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" --with-freetype-config=$INSTALL_DIR/bin/freetype-config $MAKE $MAKE install _B_u_i_l_d_i_n_g _P_i_x_m_a_n cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/pixman-0.10.0.tar.gz gunzip -c pixman-0.10.0.tar.gz | tar xf - cd pixman-0.10.0 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" $MAKE $MAKE install _B_u_i_l_d_i_n_g _C_a_i_r_o cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/cairo-1.6.4.tar.gz gunzip -c cairo-1.6.4.tar.gz | tar xf - cd cairo-1.6.4 ./configure --prefix=$INSTALL_DIR \ --enable-xlib=no \ --enable-xlib-render=no \ --enable-win32=no \ CFLAGS="-O3 -fPIC" $MAKE $MAKE install When building on Solaris you may want todo ./configure --prefix=$INSTALL_DIR \ --enable-xlib=no \ --enable-xlib-render=no \ --enable-win32=no \ CFLAGS="-O3 -fPIC -D_POSIX_PTHREAD_SEMANTICS" _B_u_i_l_d_i_n_g _G_l_i_b cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/glib-2.15.4.tar.gz gunzip -c glib-2.15.4.tar.gz | tar xf - cd glib-2.15.4 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" $MAKE $MAKE install _B_u_i_l_d_i_n_g _P_a_n_g_o cd $BUILD_DIR wget http://oss.oetiker.ch/rrdtool/pub/libs/pango-1.21.1.tar.bz2 bunzip2 -c pango-1.21.1.tar.bz2 | tar xf - cd pango-1.21.1 ./configure --prefix=$INSTALL_DIR CFLAGS="-O3 -fPIC" --without-x $MAKE $MAKE install BBuuiillddiinngg rrrrddttooooll ((sseeccoonndd ttrryy)) Now all the dependent libraries are built and you can try again. This time you tell configure where it should be looking for libraries and include files. This is done via environment variables. Depending on the shell you are running, the syntax for setting environment variables is different. And finally try building again. We disable the python and tcl bindings because it seems that a fair number of people have ill configured python and tcl setups that would prevent RRDtool from building if they are included in their current state. cd $BUILD_DIR/rrdtool-1.4.8 ./configure --prefix=$INSTALL_DIR --disable-tcl --disable-python $MAKE clean $MAKE $MAKE install SOLARIS HINT: if you want to build the Perl module for the native Perl (the one shipping with Solaris) you will need the Sun Forte compiler installed on your box or you have to hand-tune bindings/perl-shared/Makefile while building! Now go to _$_I_N_S_T_A_L_L___D_I_R//sshhaarree//rrrrddttooooll//eexxaammpplleess// and run them to see if your build has been successful. AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDBUILD(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdcached (`third-party/rrdtool-1.4.8/doc/rrdcached.txt`) RRDCACHED(1) rrdtool RRDCACHED(1) NNAAMMEE rrdcached - Data caching daemon for rrdtool SSYYNNOOPPSSIISS rrrrddccaacchheedd [--PP _p_e_r_m_i_s_s_i_o_n_s] [--ll _a_d_d_r_e_s_s] [--ss _g_r_o_u_p] [--ww _t_i_m_e_o_u_t] [--zz _d_e_l_a_y] [--ff _t_i_m_e_o_u_t] [--pp _p_i_d___f_i_l_e] [--tt _w_r_i_t_e___t_h_r_e_a_d_s] [--jj _j_o_u_r_n_a_l___d_i_r] [-F] [-g] [--bb _b_a_s_e___d_i_r [--BB]] DDEESSCCRRIIPPTTIIOONN rrrrddccaacchheedd is a daemon that receives updates to existing RRD files, accumulates them and, if enough have been received or a defined time has passed, writes the updates to the RRD file. A _f_l_u_s_h command may be used to force writing of values to disk, so that graphing facilities and similar can work with up-to-date data. The daemon was written with big setups in mind. Those setups usually run into IO related problems sooner or later for reasons that are beyond the scope of this document. Check the wiki at the RRDtool homepage for details. Also check "SECURITY CONSIDERATIONS" below before using this daemon! A detailed description of how the daemon operates can be found in the "HOW IT WORKS" section below. OOPPTTIIOONNSS --ll _a_d_d_r_e_s_s Tells the daemon to bind to _a_d_d_r_e_s_s and accept incoming connections on that socket. If _a_d_d_r_e_s_s begins with "unix:", everything following that prefix is interpreted as the path to a UNIX domain socket. Otherwise the address or node name are resolved using "getaddrinfo()". For network sockets, a port may be specified by using the form "[[_a_d_d_r_e_s_s]]::_p_o_r_t_". If the address is an IPv4 address or a fully qualified domain name (i. e. the address contains at least one dot (".")), the square brackets can be omitted, resulting in the (simpler) "_a_d_d_r_e_s_s::_p_o_r_t_" pattern. The default port is 4422221177//ttccpp. If you specify a network socket, it is mandatory to read the "SECURITY CONSIDERATIONS" section. The following formats are accepted. Please note that the address of the UNIX domain socket mmuusstt start with a slash in the second case! unix: / []: : If the --ll option is not specified the default address, "unix:/tmp/rrdcached.sock", will be used. --ss _g_r_o_u_p___n_a_m_e|_g_i_d Set the group permissions of a UNIX domain socket. The option accepts either a numeric group id or group name. That group will then have both read and write permissions (the socket will have file permissions 0750) for the socket and, therefore, is able to send commands to the daemon. This may be useful in cases where you cannot easily run all RRD processes with the same user privileges (e.g. graph generating CGI scripts that typically run in the permission context of the web server). This option affects the _f_o_l_l_o_w_i_n_g UNIX socket addresses (the following --ll options) or the default socket (if no --ll options have been specified), i.e., you may specify different settings for different sockets. The default is not to change ownership or permissions of the socket and, thus, use the system default. --mm _m_o_d_e Set the file permissions of a UNIX domain socket. The option accepts an octal number representing the bit pattern for the mode (see _c_h_m_o_d(1) for details). Please note that not all systems honor this setting. On Linux, read/write permissions are required to connect to a UNIX socket. However, many BSD-derived systems ignore permissions for UNIX sockets. See _u_n_i_x(7) for details. This option affects the _f_o_l_l_o_w_i_n_g UNIX socket addresses (the following --ll options) or the default socket (if no --ll options have been specified), i.e., you may specify different settings for different sockets. The default is not to change ownership or permissions of the socket and, thus, use the system default. --PP _c_o_m_m_a_n_d[,_c_o_m_m_a_n_d[,...]] Specifies the commands accepted via a network socket. This allows administrators of _R_R_D_C_a_c_h_e_D to control the actions accepted from various sources. The arguments given to the --PP option is a comma separated list of commands. For example, to allow the "FLUSH" and "PENDING" commands one could specify: rrdcached -P FLUSH,PENDING $MORE_ARGUMENTS The --PP option affects the _f_o_l_l_o_w_i_n_g socket addresses (the following --ll options) or the default socket (if no --ll options have been specified). In the following example, only the IPv4 network socket (address 10.0.0.1) will be restricted to the "FLUSH" and "PENDING" commands: rrdcached -l unix:/some/path -P FLUSH,PENDING -l 10.0.0.1 A complete list of available commands can be found in the section "Valid Commands" below. There are two minor special exceptions: � The "HELP" and "QUIT" commands are always allowed. � If the "BATCH" command is accepted, the .. command will automatically be accepted, too. Please also read "SECURITY CONSIDERATIONS" below. --ww _t_i_m_e_o_u_t Data is written to disk every _t_i_m_e_o_u_t seconds. If this option is not specified the default interval of 300 seconds will be used. --zz _d_e_l_a_y If specified, rrdcached will delay writing of each RRD for a random number of seconds in the range [0,_d_e_l_a_y). This will avoid too many writes being queued simultaneously. This value should be no greater than the value specified in --ww. By default, there is no delay. --ff _t_i_m_e_o_u_t Every _t_i_m_e_o_u_t seconds the entire cache is searched for old values which are written to disk. This only concerns files to which updates have stopped, so setting this to a high value, such as 3600 seconds, is acceptable in most cases. This timeout defaults to 3600 seconds. --pp _f_i_l_e Sets the name and location of the PID-file. If not specified, the default, "_$_l_o_c_a_l_s_t_s_t_e_d_i_r_/_r_u_n_/_r_r_d_c_a_c_h_e_d_._p_i_d_" will be used. --tt _w_r_i_t_e___t_h_r_e_a_d_s Specifies the number of threads used for writing RRD files. The default is 4. Increasing this number will allow rrdcached to have more simultaneous I/O requests into the kernel. This may allow the kernel to re-order disk writes, resulting in better disk throughput. --jj _d_i_r Write updates to a journal in _d_i_r. In the event of a program or system crash, this will allow the daemon to write any updates that were pending at the time of the crash. On startup, the daemon will check for journal files in this directory. If found, all updates therein will be read into memory before the daemon starts accepting new connections. The journal will be rotated with the same frequency as the flush timer given by --ff. When journaling is enabled, the daemon will use a fast shutdown procedure. Rather than flushing all files to disk, it will make sure the journal is properly written and exit immediately. Although the RRD data files are not fully up-to-date, no information is lost; all pending updates will be replayed from the journal next time the daemon starts up. To disable fast shutdown, use the --FF option. --FF ALWAYS flush all updates to the RRD data files when the daemon is shut down, regardless of journal setting. --gg Run in the foreground. The daemon will not _f_o_r_k_(_). --bb _d_i_r The daemon will change into a specific directory at startup. All files passed to the daemon, that are specified by a rreellaattiivvee path, will be interpreted to be relative to this directory. If not given the default, "/tmp", will be used. +------------------------+------------------------+ ! Command line ! File updated ! +------------------------+------------------------+ ! foo.rrd ! /tmp/foo.rrd ! ! foo/bar.rrd ! /tmp/foo/bar.rrd ! ! /var/lib/rrd/foo.rrd ! /var/lib/rrd/foo.rrd ! +------------------------+------------------------+ Paths given on the command line and paths actually updated by the daemon, assuming the base directory "/tmp". WWAARRNNIINNGG:: The paths up to and including the base directory MMUUSSTT NNOOTT BBEE symbolic links. In other words, if the base directory is specified as: -b /base/dir/somewhere ... then NNOONNEE of the following should be symbolic links: /base /base/dir /base/dir/somewhere --BB Only permit writes into the base directory specified in --bb (and any sub-directories). This does NNOOTT detect symbolic links. Paths containing "../" will also be blocked. AAFFFFEECCTTEEDD RRRRDDTTOOOOLL CCOOMMMMAANNDDSS The following commands may be made aware of the rrrrddccaacchheedd using the command line argument ----ddaaeemmoonn or the environment variable RRRRDDCCAACCHHEEDD__AADDDDRREESSSS: � dump � fetch � flush � graph � graphv � info � last � lastupdate � update � xport The uuppddaattee command can send values to the daemon instead of writing them to the disk itself. All other commands can send a FFLLUUSSHH command (see below) to the daemon before accessing the files, so they work with up-to-date data even if the cache timeout is large. EERRRROORR RREEPPOORRTTIINNGG The daemon reports errors in one of two ways: During startup, error messages are printed to "STDERR". One of the steps when starting up is to fork to the background and closing "STDERR" - after this writing directly to the user is no longer possible. Once this has happened, the daemon will send log messages to the system logging daemon using _s_y_s_l_o_g(3). The facility used is "LOG_DAEMON". HHOOWW IITT WWOORRKKSS When receiving an update, rrrrddccaacchheedd does not write to disk but looks for an entry for that file in its internal tree. If not found, an entry is created including the current time (called "First" in the diagram below). This time is nnoott the time specified on the command line but the time the operating system considers to be "now". The value and time of the value (called "Time" in the diagram below) are appended to the tree node. When appending a value to a tree node, it is checked whether it's time to write the values to disk. Values are written to disk if "now() - First >= timeout", where "timeout" is the timeout specified using the --ww option, see "OPTIONS". If the values are "old enough" they will be enqueued in the "update queue", i. e. they will be appended to the linked list shown below. Because the tree nodes and the elements of the linked list are the same data structures in memory, any update to a file that has already been enqueued will be written with the next write to the RRD file, too. A separate "update thread" constantly dequeues the first element in the update queue and writes all its values to the appropriate file. So as long as the update queue is not empty files are written at the highest possible rate. Since the timeout of files is checked only when new values are added to the file, "dead" files, i. e. files that are not updated anymore, would never be written to disk. Therefore, every now and then, controlled by the --ff option, the entire tree is walked and all "old" values are enqueued. Since this only affects "dead" files and walking the tree is relatively expensive, you should set the "flush interval" to a reasonably high value. The default is 3600 seconds (one hour). The downside of caching values is that they won't show up in graphs generated from the RRD files. To get around this, the daemon provides the "flush command" to flush specific files. This means that the file is inserted at the hheeaadd of the update queue or moved there if it is already enqueued. The flush command will return only after the file's pending updates have been written to disk. +------+ +------+ +------+ ! head ! ! root ! ! tail ! +---+--+ +---+--+ +---+--+ ! /\ ! ! / \ ! ! /\ /\ ! ! /\/\ \ `----------------- ... --------, ! V / `-------, ! V +---+----+---+ +------+-----+ +---+----+---+ ! File: foo ! ! File: bar ! ! File: qux ! ! First: 101 ! ! First: 119 ! ! First: 180 ! ! Next:&bar -+--->! Next:&... -+---> ... --->! Next:NULL ! | Prev:NULL !<---+-Prev:&foo !<--- ... ----+-Prev: &... ! +============+ +============+ +============+ ! Time: 100 ! ! Time: 120 ! ! Time: 180 ! ! Value: 10 ! ! Value: 0.1 ! ! Value: 2,2 ! +------------+ +------------+ +------------+ ! Time: 110 ! ! Time: 130 ! ! Time: 190 ! ! Value: 26 ! ! Value: 0.1 ! ! Value: 7,3 ! +------------+ +------------+ +------------+ : : : : : : +------------+ +------------+ +------------+ ! Time: 230 ! ! Time: 250 ! ! Time: 310 ! ! Value: 42 ! ! Value: 0.2 ! ! Value: 1,2 ! +------------+ +------------+ +------------+ The above diagram demonstrates: � Files/values are stored in a (balanced) tree. � Tree nodes and entries in the update queue are the same data structure. � The local time ("First") and the time specified in updates ("Time") may differ. � Timed out values are inserted at the "tail". � Explicitly flushed values are inserted at the "head". � ASCII art rocks. SSEECCUURRIITTYY CCOONNSSIIDDEERRAATTIIOONNSS AAuutthheennttiiccaattiioonn If your rrdtool installation was built without libwrap there is no form of authentication for clients connecting to the rrdcache daemon! If your rrdtool installation was built with libwrap then you can use hosts_access to restrict client access to the rrdcache daemon (rrdcached). For more information on how to use hosts_access to restrict access to the rrdcache daemon you should read the _h_o_s_t_s___a_c_c_e_s_s(5) man pages. It is still highly recommended to install a packet filter or similar mechanism to prevent unauthorized connections. Unless you have a dedicated VLAN or VPN for this, using network sockets is probably a bad idea! AAuutthhoorriizzaattiioonn There is minimal per-socket authorization. Authorization is currently done on a per-socket basis. That means each socket has a list of commands it will accept and it will accept. It will accept only those commands explicitly listed but it will (currently) accept these commands from anyone reaching the socket. If the networking sockets are to be used, it is necessary to restrict the accepted commands to those needed by external clients. If, for example, external clients want to draw graphs of the cached data, they should only be allowed to use the "FLUSH" command. EEnnccrryyppttiioonn There is no encryption. Again, this may be added in the future, but for the time being it is your job to keep your private data private. Install a VPN or an encrypted tunnel if you statistics are confidential! SSaanniittyy cchheecckkiinngg There is no sanity checking. The daemon will blindly write to any file it gets told, so you really should create a separate user just for this daemon. Also it does not do any sanity checks, so if it gets told to write values for a time far in the future, your files will be messed up good! CCoonncclluussiioonn � Security is the job of the administrator. � We recommend to allow write access via UNIX domain sockets only. � You have been warned. PPRROOTTOOCCOOLL The daemon communicates with clients using a line based ASCII protocol which is easy to read and easy to type. This makes it easy for scripts to implement the protocol and possible for users to use telnet to connect to the daemon and test stuff "by hand". The protocol is line based, this means that each record consists of one or more lines. A line is terminated by the line feed character 0x0A, commonly written as "\n". In the examples below, this character will be written as "" ("line feed"). After the connection has been established, the client is expected to send a "command". A command consists of the command keyword, possibly some arguments, and a terminating newline character. For a list of commands, see "Valid Commands" below. Example: FLUSH /tmp/foo.rrd The daemon answers with a line consisting of a status code and a short status message, separated by one or more space characters. A negative status code signals an error, a positive status code or zero signal success. If the status code is greater than zero, it indicates the number of lines that follow the status line. Examples: 0 Success 2 Two lines follow This is the first line And this is the second line VVaalliidd CCoommmmaannddss The following commands are understood by the daemon: FFLLUUSSHH _f_i_l_e_n_a_m_e Causes the daemon to put _f_i_l_e_n_a_m_e to the hheeaadd of the update queue (possibly moving it there if the node is already enqueued). The answer will be sent aafftteerr the node has been dequeued. FFLLUUSSHHAALLLL Causes the daemon to start flushing ALL pending values to disk. This returns immediately, even though the writes may take a long time. PPEENNDDIINNGG _f_i_l_e_n_a_m_e Shows any "pending" updates for a file, in order. The updates shown have not yet been written to the underlying RRD file. FFOORRGGEETT _f_i_l_e_n_a_m_e Removes _f_i_l_e_n_a_m_e from the cache. Any pending updates WWIILLLL BBEE LLOOSSTT. QQUUEEUUEE Shows the files that are on the output queue. Returns zero or more lines in the following format, where is the number of values to be written for the : HHEELLPP [_c_o_m_m_a_n_d] Returns a short usage message. If no command is given, or _c_o_m_m_a_n_d is HHEELLPP, a list of commands supported by the daemon is returned. Otherwise a short description, possibly containing a pointer to a manual page, is returned. Obviously, this is meant for interactive usage and the format in which the commands and usage summaries are returned is not well defined. SSTTAATTSS Returns a list of metrics which can be used to measure the daemons performance and check its status. For a description of the values returned, see "Performance Values" below. The format in which the values are returned is similar to many other line based protocols: Each value is printed on a separate line, each consisting of the name of the value, a colon, one or more spaces and the actual value. Example: 9 Statistics follow QueueLength: 0 UpdatesReceived: 30 FlushesReceived: 2 UpdatesWritten: 13 DataSetsWritten: 390 TreeNodesNumber: 13 TreeDepth: 4 JournalBytes: 190 JournalRotate: 0 UUPPDDAATTEE _f_i_l_e_n_a_m_e _v_a_l_u_e_s [_v_a_l_u_e_s ...] Adds more data to a filename. This is tthhee operation the daemon was designed for, so describing the mechanism again is unnecessary. Read "HOW IT WORKS" above for a detailed explanation. Note that rrdcached only accepts absolute timestamps in the update values. Updates strings like "N:1:2:3" are automatically converted to absolute time by the RRD client library before sending to rrdcached. WWRROOTTEE _f_i_l_e_n_a_m_e This command is written to the journal after a file is successfully written out to disk. It is used during journal replay to determine which updates have already been applied. It is _o_n_l_y valid in the journal; it is not accepted from the other command channels. BBAATTCCHH This command initiates the bulk load of multiple commands. This is designed for installations with extremely high update rates, since it permits more than one command to be issued per _r_e_a_d_(_) and _w_r_i_t_e_(_). All commands are executed just as they would be if given individually, except for output to the user. Messages indicating success are suppressed, and error messages are delayed until the client is finished. Command processing is finished when the client sends a dot (".") on its own line. After the client has finished, the server responds with an error count and the list of error messages (if any). Each error messages indicates the number of the command to which it corresponds, and the error message itself. The first user command after BBAATTCCHH is command number one. client: BATCH server: 0 Go ahead. End with dot '.' on its own line. client: UPDATE x.rrd 1223661439:1:2:3 <--- command #1 client: UPDATE y.rrd 1223661440:3:4:5 <--- command #2 client: and so on... client: . server: 2 Errors server: 1 message for command 1 server: 12 message for command 12 QQUUIITT Disconnect from rrdcached. PPeerrffoorrmmaannccee VVaalluueess The following counters are returned by the SSTTAATTSS command: QQuueeuueeLLeennggtthh _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Number of nodes currently enqueued in the update queue. UUppddaatteessRReecceeiivveedd _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Number of UPDATE commands received. FFlluusshheessRReecceeiivveedd _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Number of FLUSH commands received. UUppddaatteessWWrriitttteenn _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Total number of updates, i. e. calls to "rrd_update_r", since the daemon was started. DDaattaaSSeettssWWrriitttteenn _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Total number of "data sets" written to disk since the daemon was started. A data set is one or more values passed to the UUPPDDAATTEE command. For example: "1223661439:123:456" is one data set with two values. The term "data set" is used to prevent confusion whether individual values or groups of values are counted. TTrreeeeNNooddeessNNuummbbeerr _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Number of nodes in the cache. TTrreeeeDDeepptthh _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Depth of the tree used for fast key lookup. JJoouurrnnaallBByytteess _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Total number of bytes written to the journal since startup. JJoouurrnnaallRRoottaattee _(_u_n_s_i_g_n_e_d _6_4_b_i_t _i_n_t_e_g_e_r_) Number of times the journal has been rotated since startup. SSIIGGNNAALLSS SIGINT and SIGTERM The daemon exits normally on receipt of either of these signals. Pending updates are handled in accordance with the --jj and --FF options. SIGUSR1 The daemon exits AFTER flushing all updates out to disk. This may take a while. SIGUSR2 The daemon exits immediately, without flushing updates out to disk. Pending updates will be replayed from the journal when the daemon starts up again. WWAARRNNIINNGG:: iiff jjoouurrnnaalliinngg ((--jj)) iiss NNOOTT eennaabblleedd,, aannyy ppeennddiinngg uuppddaatteess WWIILLLL BBEE LLOOSSTT. BBUUGGSS No known bugs at the moment. SSEEEE AALLSSOO rrdtool, rrdgraph AAUUTTHHOORR Florian Forster Both rrrrddccaacchheedd and this manual page have been written by Florian. CCOONNTTRRIIBBUUTTOORRSS kevin brintnall 1.4.8 2013-05-23 RRDCACHED(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdcgi (`third-party/rrdtool-1.4.8/doc/rrdcgi.txt`) RRDCGI(1) rrdtool RRDCGI(1) NNAAMMEE rrdcgi - Create web pages containing RRD graphs based on templates SSYYNNOOPPSSIISS "#!/path/to/"rrrrddccggii [----ffiilltteerr] DDEESSCCRRIIPPTTIIOONN rrrrddccggii is a sort of very limited script interpreter. Its purpose is to run as a cgi-program and parse a web page template containing special might give you the name of the remote user given you are using some sort of access control on the directory. RRD::GOODFOR _s_e_c_o_n_d_s Specify the number of seconds this page should remain valid. This will prompt the rrdcgi to output a Last-Modified, an Expire and if the number of seconds is _n_e_g_a_t_i_v_e a Refresh header. RRD::INCLUDE _f_i_l_e_n_a_m_e Include the contents of the specified file into the page returned from the cgi. RRD::SETENV _v_a_r_i_a_b_l_e _v_a_l_u_e If you want to present your graphs in another time zone than your own, you could use to make sure everything is presented in Universal Time. Note that the values permitted to TZ depend on your OS. RRD::SETVAR _v_a_r_i_a_b_l_e _v_a_l_u_e Analog to SETENV but for local variables. RRD::GETVAR _v_a_r_i_a_b_l_e Analog to GETENV but for local variables. RRD::TIME::LAST _r_r_d_-_f_i_l_e _s_t_r_f_t_i_m_e_-_f_o_r_m_a_t This gets replaced by the last modification time of the selected RRD. The time is _s_t_r_f_t_i_m_e-formatted with the string specified in the second argument. RRD::TIME::NOW _s_t_r_f_t_i_m_e_-_f_o_r_m_a_t This gets replaced by the current time of day. The time is _s_t_r_f_t_i_m_e-formatted with the string specified in the argument. Note that if you return : (colons) from your strftime format you may have to escape them using \ if the time is to be used as an argument to a GRAPH command. RRD::TIME::STRFTIME _S_T_A_R_T_|_E_N_D _s_t_a_r_t_-_s_p_e_c _e_n_d_-_s_p_e_c _s_t_r_f_t_i_m_e_-_f_o_r_m_a_t This gets replaced by a strftime-formatted time using the format _s_t_r_f_t_i_m_e_-_f_o_r_m_a_t on either _s_t_a_r_t_-_s_p_e_c or _e_n_d_-_s_p_e_c depending on whether _S_T_A_R_T or _E_N_D is specified. Both _s_t_a_r_t_- _s_p_e_c and _e_n_d_-_s_p_e_c must be supplied as either could be relative to the other. This is intended to allow pretty titles on graphs with times that are easier for non RRDtool folks to figure out than "-2weeks". Note that again, if you return : (colon) from your strftime format, you may have to escape them using \ if the time is to be used as an argument to a GRAPH command. RRD::GRAPH _r_r_d_g_r_a_p_h _a_r_g_u_m_e_n_t_s This tag creates the RRD graph defined by its argument and then is replaced by an appropriate tag referring to the graph. The ----llaazzyy option in RRD graph can be used to make sure that graphs are only regenerated when they are out of date. The arguments to the RRRRDD::::GGRRAAPPHH tag work as described in the rrrrddggrraapphh manual page. Use the ----llaazzyy option in your RRD::GRAPH tags, to reduce the load on your server. This option makes sure that graphs are only regenerated when the old ones are out of date. If you do not specify your own ----iimmggiinnffoo format, the following will be used: Note that %s stands for the filename part of the graph generated, all directories given in the PNG file argument will get dropped. RRD::PRINT _n_u_m_b_e_r If the preceding RRRRDD::::GGRRAAPPHH tag contained and PPRRIINNTT arguments, then you can access their output with this tag. The _n_u_m_b_e_r argument refers to the number of the PPRRIINNTT argument. This first PPRRIINNTT has _n_u_m_b_e_r 0. RRD::INTERNAL This tag gets replaced by an internal var. Currently these vars are known: VERSION, COMPILETIME. These vars represent the compiled-in values. EEXXAAMMPPLLEE 11 The example below creates a web pages with a single RRD graph. #!/usr/local/bin/rrdcgi RRDCGI Demo

RRDCGI Example Page

EEXXAAMMPPLLEE 22 This script is slightly more elaborate, it allows you to run it from a form which sets RRD_NAME. RRD_NAME is then used to select which RRD you want to use as source for your graph. #!/usr/local/bin/rrdcgi RRDCGI Demo

RRDCGI Example Page for

Selection

Room A, Room B.

Graph

.png --lazy --title "Temperatures for " DEF:cel=.rrd:exhaust:AVERAGE LINE2:cel#00a000:"D. Celsius">

EEXXAAMMPPLLEE 33 This example shows how to handle the case where the RRD, graphs and cgi-bins are separate directories #!/.../bin/rrdcgi RRDCGI Demo

RRDCGI test Page

' --lazy --start -1d --end now DEF:http_src=/.../rrds/test.rrd:http_src:AVERAGE AREA:http_src#00ff00:http_src > Note 1: Replace /.../ with the relevant directories Note 2: The SRC=/.../pngs should be paths from the view of the webserver/browser AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDCGI(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdcreate (`third-party/rrdtool-1.4.8/doc/rrdcreate.txt`) RRDCREATE(1) rrdtool RRDCREATE(1) NNAAMMEE rrdcreate - Set up a new Round Robin Database SSYYNNOOPPSSIISS rrrrddttooooll ccrreeaattee _f_i_l_e_n_a_m_e [----ssttaarrtt|--bb _s_t_a_r_t _t_i_m_e] [----sstteepp|--ss _s_t_e_p] [----nnoo--oovveerrwwrriittee] [DDSS::_d_s_-_n_a_m_e::_D_S_T::_d_s_t _a_r_g_u_m_e_n_t_s] [RRRRAA::_C_F::_c_f _a_r_g_u_m_e_n_t_s] DDEESSCCRRIIPPTTIIOONN The create function of RRDtool lets you set up new Round Robin Database (RRRRDD) files. The file is created at its final, full size and filled with _*_U_N_K_N_O_W_N_* data. _f_i_l_e_n_a_m_e The name of the RRRRDD you want to create. RRRRDD files should end with the extension _._r_r_d. However, RRRRDDttooooll will accept any filename. ----ssttaarrtt||--bb _s_t_a_r_t _t_i_m_e ((ddeeffaauulltt:: nnooww -- 1100ss)) Specifies the time in seconds since 1970-01-01 UTC when the first value should be added to the RRRRDD. RRRRDDttooooll will not accept any data timed before or at the time specified. See also AT-STYLE TIME SPECIFICATION section in the _r_r_d_f_e_t_c_h documentation for other ways to specify time. ----sstteepp||--ss _s_t_e_p ((ddeeffaauulltt:: 330000 sseeccoonnddss)) Specifies the base interval in seconds with which data will be fed into the RRRRDD. ----nnoo--oovveerrwwrriittee Do not clobber an existing file of the same name. DDSS::_d_s_-_n_a_m_e::_D_S_T::_d_s_t _a_r_g_u_m_e_n_t_s A single RRRRDD can accept input from several data sources (DDSS), for example incoming and outgoing traffic on a specific communication line. With the DDSS configuration option you must define some basic properties of each data source you want to store in the RRRRDD. _d_s_-_n_a_m_e is the name you will use to reference this particular data source from an RRRRDD. A _d_s_-_n_a_m_e must be 1 to 19 characters long in the characters [a-zA-Z0-9_]. _D_S_T defines the Data Source Type. The remaining arguments of a data source entry depend on the data source type. For GAUGE, COUNTER, DERIVE, and ABSOLUTE the format for a data source entry is: DDSS::_d_s_-_n_a_m_e::_G_A_U_G_E _| _C_O_U_N_T_E_R _| _D_E_R_I_V_E _| _A_B_S_O_L_U_T_E::_h_e_a_r_t_b_e_a_t::_m_i_n::_m_a_x For COMPUTE data sources, the format is: DDSS::_d_s_-_n_a_m_e::_C_O_M_P_U_T_E::_r_p_n_-_e_x_p_r_e_s_s_i_o_n In order to decide which data source type to use, review the definitions that follow. Also consult the section on "HOW TO MEASURE" for further insight. GGAAUUGGEE is for things like temperatures or number of people in a room or the value of a RedHat share. CCOOUUNNTTEERR is for continuous incrementing counters like the ifInOctets counter in a router. The CCOOUUNNTTEERR data source assumes that the counter never decreases, except when a counter overflows. The update function takes the overflow into account. The counter is stored as a per- second rate. When the counter overflows, RRDtool checks if the overflow happened at the 32bit or 64bit border and acts accordingly by adding an appropriate value to the result. DDEERRIIVVEE will store the derivative of the line going from the last to the current value of the data source. This can be useful for gauges, for example, to measure the rate of people entering or leaving a room. Internally, derive works exactly like COUNTER but without overflow checks. So if your counter does not reset at 32 or 64 bit you might want to use DERIVE and combine it with a MIN value of 0. NNOOTTEE oonn CCOOUUNNTTEERR vvss DDEERRIIVVEE by Don Baarda If you cannot tolerate ever mistaking the occasional counter reset for a legitimate counter wrap, and would prefer "Unknowns" for all legitimate counter wraps and resets, always use DERIVE with min=0. Otherwise, using COUNTER with a suitable max will return correct values for all legitimate counter wraps, mark some counter resets as "Unknown", but can mistake some counter resets for a legitimate counter wrap. For a 5 minute step and 32-bit counter, the probability of mistaking a counter reset for a legitimate wrap is arguably about 0.8% per 1Mbps of maximum bandwidth. Note that this equates to 80% for 100Mbps interfaces, so for high bandwidth interfaces and a 32bit counter, DERIVE with min=0 is probably preferable. If you are using a 64bit counter, just about any max setting will eliminate the possibility of mistaking a reset for a counter wrap. AABBSSOOLLUUTTEE is for counters which get reset upon reading. This is used for fast counters which tend to overflow. So instead of reading them normally you reset them after every read to make sure you have a maximum time available before the next overflow. Another usage is for things you count like number of messages since the last update. CCOOMMPPUUTTEE is for storing the result of a formula applied to other data sources in the RRRRDD. This data source is not supplied a value on update, but rather its Primary Data Points (PDPs) are computed from the PDPs of the data sources according to the rpn-expression that defines the formula. Consolidation functions are then applied normally to the PDPs of the COMPUTE data source (that is the rpn- expression is only applied to generate PDPs). In database software, such data sets are referred to as "virtual" or "computed" columns. _h_e_a_r_t_b_e_a_t defines the maximum number of seconds that may pass between two updates of this data source before the value of the data source is assumed to be _*_U_N_K_N_O_W_N_*. _m_i_n and _m_a_x define the expected range values for data supplied by a data source. If _m_i_n and/or _m_a_x are specified any value outside the defined range will be regarded as _*_U_N_K_N_O_W_N_*. If you do not know or care about min and max, set them to U for unknown. Note that min and max always refer to the processed values of the DS. For a traffic-CCOOUUNNTTEERR type DS this would be the maximum and minimum data-rate expected from the device. _I_f _i_n_f_o_r_m_a_t_i_o_n _o_n _m_i_n_i_m_a_l_/_m_a_x_i_m_a_l _e_x_p_e_c_t_e_d _v_a_l_u_e_s _i_s _a_v_a_i_l_a_b_l_e_, _a_l_w_a_y_s _s_e_t _t_h_e _m_i_n _a_n_d_/_o_r _m_a_x _p_r_o_p_e_r_t_i_e_s_. _T_h_i_s _w_i_l_l _h_e_l_p _R_R_D_t_o_o_l _i_n _d_o_i_n_g _a _s_i_m_p_l_e _s_a_n_i_t_y _c_h_e_c_k _o_n _t_h_e _d_a_t_a _s_u_p_p_l_i_e_d _w_h_e_n _r_u_n_n_i_n_g _u_p_d_a_t_e_. _r_p_n_-_e_x_p_r_e_s_s_i_o_n defines the formula used to compute the PDPs of a COMPUTE data source from other data sources in the same . It is similar to defining a CCDDEEFF argument for the graph command. Please refer to that manual page for a list and description of RPN operations supported. For COMPUTE data sources, the following RPN operations are not supported: COUNT, PREV, TIME, and LTIME. In addition, in defining the RPN expression, the COMPUTE data source may only refer to the names of data source listed previously in the create command. This is similar to the restriction that CCDDEEFFs must refer only to DDEEFFs and CCDDEEFFs previously defined in the same graph command. RRRRAA::_C_F::_c_f _a_r_g_u_m_e_n_t_s The purpose of an RRRRDD is to store data in the round robin archives (RRRRAA). An archive consists of a number of data values or statistics for each of the defined data-sources (DDSS) and is defined with an RRRRAA line. When data is entered into an RRRRDD, it is first fit into time slots of the length defined with the --ss option, thus becoming a _p_r_i_m_a_r_y _d_a_t_a _p_o_i_n_t. The data is also processed with the consolidation function (_C_F) of the archive. There are several consolidation functions that consolidate primary data points via an aggregate function: AAVVEERRAAGGEE, MMIINN, MMAAXX, LLAASSTT. AVERAGE the average of the data points is stored. MIN the smallest of the data points is stored. MAX the largest of the data points is stored. LAST the last data points is used. Note that data aggregation inevitably leads to loss of precision and information. The trick is to pick the aggregate function such that the _i_n_t_e_r_e_s_t_i_n_g properties of your data is kept across the aggregation process. The format of RRRRAA line for these consolidation functions is: RRRRAA::_A_V_E_R_A_G_E _| _M_I_N _| _M_A_X _| _L_A_S_T::_x_f_f::_s_t_e_p_s::_r_o_w_s _x_f_f The xfiles factor defines what part of a consolidation interval may be made up from _*_U_N_K_N_O_W_N_* data while the consolidated value is still regarded as known. It is given as the ratio of allowed _*_U_N_K_N_O_W_N_* PDPs to the number of PDPs in the interval. Thus, it ranges from 0 to 1 (exclusive). _s_t_e_p_s defines how many of these _p_r_i_m_a_r_y _d_a_t_a _p_o_i_n_t_s are used to build a _c_o_n_s_o_l_i_d_a_t_e_d _d_a_t_a _p_o_i_n_t which then goes into the archive. _r_o_w_s defines how many generations of data values are kept in an RRRRAA. Obviously, this has to be greater than zero. AAbbeerrrraanntt BBeehhaavviioorr DDeetteeccttiioonn wwiitthh HHoolltt--WWiinntteerrss FFoorreeccaassttiinngg In addition to the aggregate functions, there are a set of specialized functions that enable RRRRDDttooooll to provide data smoothing (via the Holt- Winters forecasting algorithm), confidence bands, and the flagging aberrant behavior in the data source time series: � RRRRAA::_H_W_P_R_E_D_I_C_T::_r_o_w_s::_a_l_p_h_a::_b_e_t_a::_s_e_a_s_o_n_a_l _p_e_r_i_o_d[::_r_r_a_-_n_u_m] � RRRRAA::_M_H_W_P_R_E_D_I_C_T::_r_o_w_s::_a_l_p_h_a::_b_e_t_a::_s_e_a_s_o_n_a_l _p_e_r_i_o_d[::_r_r_a_-_n_u_m] � RRRRAA::_S_E_A_S_O_N_A_L::_s_e_a_s_o_n_a_l _p_e_r_i_o_d::_g_a_m_m_a::_r_r_a_- _n_u_m[::ssmmooootthhiinngg--wwiinnddooww==_f_r_a_c_t_i_o_n] � RRRRAA::_D_E_V_S_E_A_S_O_N_A_L::_s_e_a_s_o_n_a_l _p_e_r_i_o_d::_g_a_m_m_a::_r_r_a_- _n_u_m[::ssmmooootthhiinngg--wwiinnddooww==_f_r_a_c_t_i_o_n] � RRRRAA::_D_E_V_P_R_E_D_I_C_T::_r_o_w_s::_r_r_a_-_n_u_m � RRRRAA::_F_A_I_L_U_R_E_S::_r_o_w_s::_t_h_r_e_s_h_o_l_d::_w_i_n_d_o_w _l_e_n_g_t_h::_r_r_a_-_n_u_m These RRRRAAss differ from the true consolidation functions in several ways. First, each of the RRRRAAs is updated once for every primary data point. Second, these RRRRAAss are interdependent. To generate real-time confidence bounds, a matched set of SEASONAL, DEVSEASONAL, DEVPREDICT, and either HWPREDICT or MHWPREDICT must exist. Generating smoothed values of the primary data points requires a SEASONAL RRRRAA and either an HWPREDICT or MHWPREDICT RRRRAA. Aberrant behavior detection requires FAILURES, DEVSEASONAL, SEASONAL, and either HWPREDICT or MHWPREDICT. The predicted, or smoothed, values are stored in the HWPREDICT or MHWPREDICT RRRRAA. HWPREDICT and MHWPREDICT are actually two variations on the Holt-Winters method. They are interchangeable. Both attempt to decompose data into three components: a baseline, a trend, and a seasonal coefficient. HWPREDICT adds its seasonal coefficient to the baseline to form a prediction, whereas MHWPREDICT multiplies its seasonal coefficient by the baseline to form a prediction. The difference is noticeable when the baseline changes significantly in the course of a season; HWPREDICT will predict the seasonality to stay constant as the baseline changes, but MHWPREDICT will predict the seasonality to grow or shrink in proportion to the baseline. The proper choice of method depends on the thing being modeled. For simplicity, the rest of this discussion will refer to HWPREDICT, but MHWPREDICT may be substituted in its place. The predicted deviations are stored in DEVPREDICT (think a standard deviation which can be scaled to yield a confidence band). The FAILURES RRRRAA stores binary indicators. A 1 marks the indexed observation as failure; that is, the number of confidence bounds violations in the preceding window of observations met or exceeded a specified threshold. An example of using these RRRRAAss to graph confidence bounds and failures appears in rrdgraph. The SEASONAL and DEVSEASONAL RRRRAAss store the seasonal coefficients for the Holt-Winters forecasting algorithm and the seasonal deviations, respectively. There is one entry per observation time point in the seasonal cycle. For example, if primary data points are generated every five minutes and the seasonal cycle is 1 day, both SEASONAL and DEVSEASONAL will have 288 rows. In order to simplify the creation for the novice user, in addition to supporting explicit creation of the HWPREDICT, SEASONAL, DEVPREDICT, DEVSEASONAL, and FAILURES RRRRAAss, the RRRRDDttooooll create command supports implicit creation of the other four when HWPREDICT is specified alone and the final argument _r_r_a_-_n_u_m is omitted. _r_o_w_s specifies the length of the RRRRAA prior to wrap around. Remember that there is a one-to-one correspondence between primary data points and entries in these RRAs. For the HWPREDICT CF, _r_o_w_s should be larger than the _s_e_a_s_o_n_a_l _p_e_r_i_o_d. If the DEVPREDICT RRRRAA is implicitly created, the default number of rows is the same as the HWPREDICT _r_o_w_s argument. If the FAILURES RRRRAA is implicitly created, _r_o_w_s will be set to the _s_e_a_s_o_n_a_l _p_e_r_i_o_d argument of the HWPREDICT RRRRAA. Of course, the RRRRDDttooooll _r_e_s_i_z_e command is available if these defaults are not sufficient and the creator wishes to avoid explicit creations of the other specialized function RRRRAAss. _s_e_a_s_o_n_a_l _p_e_r_i_o_d specifies the number of primary data points in a seasonal cycle. If SEASONAL and DEVSEASONAL are implicitly created, this argument for those RRRRAAss is set automatically to the value specified by HWPREDICT. If they are explicitly created, the creator should verify that all three _s_e_a_s_o_n_a_l _p_e_r_i_o_d arguments agree. _a_l_p_h_a is the adaption parameter of the intercept (or baseline) coefficient in the Holt-Winters forecasting algorithm. See rrdtool for a description of this algorithm. _a_l_p_h_a must lie between 0 and 1. A value closer to 1 means that more recent observations carry greater weight in predicting the baseline component of the forecast. A value closer to 0 means that past history carries greater weight in predicting the baseline component. _b_e_t_a is the adaption parameter of the slope (or linear trend) coefficient in the Holt-Winters forecasting algorithm. _b_e_t_a must lie between 0 and 1 and plays the same role as _a_l_p_h_a with respect to the predicted linear trend. _g_a_m_m_a is the adaption parameter of the seasonal coefficients in the Holt-Winters forecasting algorithm (HWPREDICT) or the adaption parameter in the exponential smoothing update of the seasonal deviations. It must lie between 0 and 1. If the SEASONAL and DEVSEASONAL RRRRAAss are created implicitly, they will both have the same value for _g_a_m_m_a: the value specified for the HWPREDICT _a_l_p_h_a argument. Note that because there is one seasonal coefficient (or deviation) for each time point during the seasonal cycle, the adaptation rate is much slower than the baseline. Each seasonal coefficient is only updated (or adapts) when the observed value occurs at the offset in the seasonal cycle corresponding to that coefficient. If SEASONAL and DEVSEASONAL RRRRAAss are created explicitly, _g_a_m_m_a need not be the same for both. Note that _g_a_m_m_a can also be changed via the RRRRDDttooooll _t_u_n_e command. _s_m_o_o_t_h_i_n_g_-_w_i_n_d_o_w specifies the fraction of a season that should be averaged around each point. By default, the value of _s_m_o_o_t_h_i_n_g_-_w_i_n_d_o_w is 0.05, which means each value in SEASONAL and DEVSEASONAL will be occasionally replaced by averaging it with its (_s_e_a_s_o_n_a_l _p_e_r_i_o_d*0.05) nearest neighbors. Setting _s_m_o_o_t_h_i_n_g_-_w_i_n_d_o_w to zero will disable the running-average smoother altogether. _r_r_a_-_n_u_m provides the links between related RRRRAAss. If HWPREDICT is specified alone and the other RRRRAAss are created implicitly, then there is no need to worry about this argument. If RRRRAAss are created explicitly, then carefully pay attention to this argument. For each RRRRAA which includes this argument, there is a dependency between that RRRRAA and another RRRRAA. The _r_r_a_-_n_u_m argument is the 1-based index in the order of RRRRAA creation (that is, the order they appear in the _c_r_e_a_t_e command). The dependent RRRRAA for each RRRRAA requiring the _r_r_a_-_n_u_m argument is listed here: � HWPREDICT _r_r_a_-_n_u_m is the index of the SEASONAL RRRRAA. � SEASONAL _r_r_a_-_n_u_m is the index of the HWPREDICT RRRRAA. � DEVPREDICT _r_r_a_-_n_u_m is the index of the DEVSEASONAL RRRRAA. � DEVSEASONAL _r_r_a_-_n_u_m is the index of the HWPREDICT RRRRAA. � FAILURES _r_r_a_-_n_u_m is the index of the DEVSEASONAL RRRRAA. _t_h_r_e_s_h_o_l_d is the minimum number of violations (observed values outside the confidence bounds) within a window that constitutes a failure. If the FAILURES RRRRAA is implicitly created, the default value is 7. _w_i_n_d_o_w _l_e_n_g_t_h is the number of time points in the window. Specify an integer greater than or equal to the threshold and less than or equal to 28. The time interval this window represents depends on the interval between primary data points. If the FAILURES RRRRAA is implicitly created, the default value is 9. TThhee HHEEAARRTTBBEEAATT aanndd tthhee SSTTEEPP Here is an explanation by Don Baarda on the inner workings of RRDtool. It may help you to sort out why all this *UNKNOWN* data is popping up in your databases: RRDtool gets fed samples/updates at arbitrary times. From these it builds Primary Data Points (PDPs) on every "step" interval. The PDPs are then accumulated into the RRAs. The "heartbeat" defines the maximum acceptable interval between samples/updates. If the interval between samples is less than "heartbeat", then an average rate is calculated and applied for that interval. If the interval between samples is longer than "heartbeat", then that entire interval is considered "unknown". Note that there are other things that can make a sample interval "unknown", such as the rate exceeding limits, or a sample that was explicitly marked as unknown. The known rates during a PDP's "step" interval are used to calculate an average rate for that PDP. If the total "unknown" time accounts for more than hhaallff the "step", the entire PDP is marked as "unknown". This means that a mixture of known and "unknown" sample times in a single PDP "step" may or may not add up to enough "known" time to warrant a known PDP. The "heartbeat" can be short (unusual) or long (typical) relative to the "step" interval between PDPs. A short "heartbeat" means you require multiple samples per PDP, and if you don't get them mark the PDP unknown. A long heartbeat can span multiple "steps", which means it is acceptable to have multiple PDPs calculated from a single sample. An extreme example of this might be a "step" of 5 minutes and a "heartbeat" of one day, in which case a single sample every day will result in all the PDPs for that entire day period being set to the same average rate. _-_- _D_o_n _B_a_a_r_d_a _<_d_o_n_._b_a_a_r_d_a_@_b_a_e_s_y_s_t_e_m_s_._c_o_m_> time| axis| begin__|00| |01| u|02|----* sample1, restart "hb"-timer u|03| / u|04| / u|05| / u|06|/ "hbt" expired u|07| |08|----* sample2, restart "hb" |09| / |10| / u|11|----* sample3, restart "hb" u|12| / u|13| / step1_u|14| / u|15|/ "swt" expired u|16| |17|----* sample4, restart "hb", create "pdp" for step1 = |18| / = unknown due to 10 "u" labled secs > 0.5 * step |19| / |20| / |21|----* sample5, restart "hb" |22| / |23| / |24|----* sample6, restart "hb" |25| / |26| / |27|----* sample7, restart "hb" step2__|28| / |22| / |23|----* sample8, restart "hb", create "pdp" for step1, create "cdp" |24| / |25| / graphics by _v_l_a_d_i_m_i_r_._l_a_v_r_o_v_@_d_e_s_y_._d_e. HHOOWW TTOO MMEEAASSUURREE Here are a few hints on how to measure: Temperature Usually you have some type of meter you can read to get the temperature. The temperature is not really connected with a time. The only connection is that the temperature reading happened at a certain time. You can use the GGAAUUGGEE data source type for this. RRDtool will then record your reading together with the time. Mail Messages Assume you have a method to count the number of messages transported by your mail server in a certain amount of time, giving you data like '5 messages in the last 65 seconds'. If you look at the count of 5 like an AABBSSOOLLUUTTEE data type you can simply update the RRD with the number 5 and the end time of your monitoring period. RRDtool will then record the number of messages per second. If at some later stage you want to know the number of messages transported in a day, you can get the average messages per second from RRDtool for the day in question and multiply this number with the number of seconds in a day. Because all math is run with Doubles, the precision should be acceptable. It's always a Rate RRDtool stores rates in amount/second for COUNTER, DERIVE and ABSOLUTE data. When you plot the data, you will get on the y axis amount/second which you might be tempted to convert to an absolute amount by multiplying by the delta-time between the points. RRDtool plots continuous data, and as such is not appropriate for plotting absolute amounts as for example "total bytes" sent and received in a router. What you probably want is plot rates that you can scale to bytes/hour, for example, or plot absolute amounts with another tool that draws bar-plots, where the delta-time is clear on the plot for each point (such that when you read the graph you see for example GB on the y axis, days on the x axis and one bar for each day). EEXXAAMMPPLLEE rrdtool create temperature.rrd --step 300 \ DS:temp:GAUGE:600:-273:5000 \ RRA:AVERAGE:0.5:1:1200 \ RRA:MIN:0.5:12:2400 \ RRA:MAX:0.5:12:2400 \ RRA:AVERAGE:0.5:12:2400 This sets up an RRRRDD called _t_e_m_p_e_r_a_t_u_r_e_._r_r_d which accepts one temperature value every 300 seconds. If no new data is supplied for more than 600 seconds, the temperature becomes _*_U_N_K_N_O_W_N_*. The minimum acceptable value is -273 and the maximum is 5'000. A few archive areas are also defined. The first stores the temperatures supplied for 100 hours (1'200 * 300 seconds = 100 hours). The second RRA stores the minimum temperature recorded over every hour (12 * 300 seconds = 1 hour), for 100 days (2'400 hours). The third and the fourth RRA's do the same for the maximum and average temperature, respectively. EEXXAAMMPPLLEE 22 rrdtool create monitor.rrd --step 300 \ DS:ifOutOctets:COUNTER:1800:0:4294967295 \ RRA:AVERAGE:0.5:1:2016 \ RRA:HWPREDICT:1440:0.1:0.0035:288 This example is a monitor of a router interface. The first RRRRAA tracks the traffic flow in octets; the second RRRRAA generates the specialized functions RRRRAAss for aberrant behavior detection. Note that the _r_r_a_-_n_u_m argument of HWPREDICT is missing, so the other RRRRAAss will implicitly be created with default parameter values. In this example, the forecasting algorithm baseline adapts quickly; in fact the most recent one hour of observations (each at 5 minute intervals) accounts for 75% of the baseline prediction. The linear trend forecast adapts much more slowly. Observations made during the last day (at 288 observations per day) account for only 65% of the predicted linear trend. Note: these computations rely on an exponential smoothing formula described in the LISA 2000 paper. The seasonal cycle is one day (288 data points at 300 second intervals), and the seasonal adaption parameter will be set to 0.1. The RRD file will store 5 days (1'440 data points) of forecasts and deviation predictions before wrap around. The file will store 1 day (a seasonal cycle) of 0-1 indicators in the FAILURES RRRRAA. The same RRD file and RRRRAAss are created with the following command, which explicitly creates all specialized function RRRRAAss. rrdtool create monitor.rrd --step 300 \ DS:ifOutOctets:COUNTER:1800:0:4294967295 \ RRA:AVERAGE:0.5:1:2016 \ RRA:HWPREDICT:1440:0.1:0.0035:288:3 \ RRA:SEASONAL:288:0.1:2 \ RRA:DEVPREDICT:1440:5 \ RRA:DEVSEASONAL:288:0.1:2 \ RRA:FAILURES:288:7:9:5 Of course, explicit creation need not replicate implicit create, a number of arguments could be changed. EEXXAAMMPPLLEE 33 rrdtool create proxy.rrd --step 300 \ DS:Total:DERIVE:1800:0:U \ DS:Duration:DERIVE:1800:0:U \ DS:AvgReqDur:COMPUTE:Duration,Requests,0,EQ,1,Requests,IF,/ \ RRA:AVERAGE:0.5:1:2016 This example is monitoring the average request duration during each 300 sec interval for requests processed by a web proxy during the interval. In this case, the proxy exposes two counters, the number of requests processed since boot and the total cumulative duration of all processed requests. Clearly these counters both have some rollover point, but using the DERIVE data source also handles the reset that occurs when the web proxy is stopped and restarted. In the RRRRDD, the first data source stores the requests per second rate during the interval. The second data source stores the total duration of all requests processed during the interval divided by 300. The COMPUTE data source divides each PDP of the AccumDuration by the corresponding PDP of TotalRequests and stores the average request duration. The remainder of the RPN expression handles the divide by zero case. AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDCREATE(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrddump (`third-party/rrdtool-1.4.8/doc/rrddump.txt`) RRDDUMP(1) rrdtool RRDDUMP(1) NNAAMMEE rrddump - dump the contents of an RRD to XML format SSYYNNOOPPSSIISS rrrrddttooooll dduummpp _f_i_l_e_n_a_m_e_._r_r_d [_f_i_l_e_n_a_m_e_._x_m_l] [----hheeaaddeerr|--hh {none,xsd,dtd}] [----nnoo--hheeaaddeerr] [----ddaaeemmoonn _a_d_d_r_e_s_s] > _f_i_l_e_n_a_m_e_._x_m_l DDEESSCCRRIIPPTTIIOONN The dduummpp function writes the contents of an RRRRDD in human readable (?) XML format to a file or to stdout. This format can be read by rrdrestore. Together they allow you to transfer your files from one computer architecture to another as well to manipulate the contents of an RRRRDD file in a somewhat more convenient manner. _f_i_l_e_n_a_m_e_._r_r_d The name of the RRRRDD you want to dump. _f_i_l_e_n_a_m_e_._x_m_l The (optional) filename that you want to write the XML output to. If not specified, the XML will be printed to stdout. ----hheeaaddeerr|--hh {none,xsd,dtd} By default RRDtool will add a dtd header to the xml file. Here you can customize this to and xsd header or no header at all. ----nnoo--hheeaaddeerr A shortcut for --header=none. If you want to restore the dump with RRDtool 1.2 you should use the --no-header option since 1.2 can not deal with xml headers. ----ddaaeemmoonn _a_d_d_r_e_s_s Address of the rrdcached daemon. If specified, a "flush" command is sent to the server before reading the RRD files. This allows rrrrddttooooll to return fresh data even if the daemon is configured to cache values for a long time. For a list of accepted formats, see the --ll option in the rrdcached manual. rrdtool dump --daemon unix:/var/run/rrdcached.sock /var/lib/rrd/foo.rrd EEXXAAMMPPLLEESS To transfer an RRD between architectures, follow these steps: 1. On the same system where the RRD was created, use rrrrddttooooll dduummpp to export the data to XML format. 2. Transfer the XML dump to the target system. 3. Run rrrrddttooooll rreessttoorree to create a new RRD from the XML dump. See rrrrddrreessttoorree for details. EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS The following environment variables may be used to change the behavior of "rrdtool dump": RRRRDDCCAACCHHEEDD__AADDDDRREESSSS If this environment variable is set it will have the same effect as specifying the "--daemon" option on the command line. If both are present, the command line argument takes precedence. AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDDUMP(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdfetch (`third-party/rrdtool-1.4.8/doc/rrdfetch.txt`) RRDFETCH(1) rrdtool RRDFETCH(1) NNAAMMEE rrdfetch - Fetch data from an RRD. SSYYNNOOPPSSIISS rrrrddttooooll ffeettcchh _f_i_l_e_n_a_m_e _C_F [----rreessoolluuttiioonn|--rr _r_e_s_o_l_u_t_i_o_n] [----ssttaarrtt|--ss _s_t_a_r_t] [----eenndd|--ee _e_n_d] [----ddaaeemmoonn _a_d_d_r_e_s_s] DDEESSCCRRIIPPTTIIOONN The ffeettcchh function is normally used internally by the graph function to get data from RRRRDDs. ffeettcchh will analyze the RRRRDD and try to retrieve the data in the resolution requested. The data fetched is printed to stdout. _*_U_N_K_N_O_W_N_* data is often represented by the string "NaN" depending on your OS's printf function. _f_i_l_e_n_a_m_e the name of the RRRRDD you want to fetch the data from. _C_F the consolidation function that is applied to the data you want to fetch (AVERAGE,MIN,MAX,LAST) ----rreessoolluuttiioonn|--rr _r_e_s_o_l_u_t_i_o_n (default is the highest resolution) the interval you want the values to have (seconds per value). rrrrddffeettcchh will try to match your request, but it will return data even if no absolute match is possible. NNBB.. See note below. ----ssttaarrtt|--ss _s_t_a_r_t (default end-1day) start of the time series. A time in seconds since epoch (1970-01-01) is required. Negative numbers are relative to the current time. By default, one day worth of data will be fetched. See also AT-STYLE TIME SPECIFICATION section for a detailed explanation on ways to specify the start time. ----eenndd|--ee _e_n_d (default now) the end of the time series in seconds since epoch. See also AT- STYLE TIME SPECIFICATION section for a detailed explanation of how to specify the end time. ----ddaaeemmoonn _a_d_d_r_e_s_s Address of the rrdcached daemon. If specified, a "flush" command is sent to the server before reading the RRD files. This allows rrrrddttooooll to return fresh data even if the daemon is configured to cache values for a long time. For a list of accepted formats, see the --ll option in the rrdcached manual. rrdtool fetch --daemon unix:/var/run/rrdcached.sock /var/lib/rrd/foo.rrd AVERAGE RREESSOOLLUUTTIIOONN IINNTTEERRVVAALL In order to get RRDtool to fetch anything other than the finest resolution RRA bbootthh the start and end time must be specified on boundaries that are multiples of the desired resolution. Consider the following example: rrdtool create subdata.rrd -s 10 DS:ds0:GAUGE:300:0:U \ RRA:AVERAGE:0.5:30:3600 \ RRA:AVERAGE:0.5:90:1200 \ RRA:AVERAGE:0.5:360:1200 \ RRA:MAX:0.5:360:1200 \ RRA:AVERAGE:0.5:8640:600 \ RRA:MAX:0.5:8640:600 This RRD collects data every 10 seconds and stores its averages over 5 minutes, 15 minutes, 1 hour, and 1 day, as well as the maxima for 1 hour and 1 day. Consider now that you want to fetch the 15 minute average data for the last hour. You might try rrdtool fetch subdata.rrd AVERAGE -r 900 -s -1h However, this will almost always result in a time series that is NNOOTT in the 15 minute RRA. Therefore, the highest resolution RRA, i.e. 5 minute averages, will be chosen which in this case is not what you want. Hence, make sure that 1. both start and end time are a multiple of 900 2. both start and end time are within the desired RRA So, if time now is called "t", do end time == int(t/900)*900, start time == end time - 1hour, resolution == 900. Using the bash shell, this could look be: TIME=$(date +%s) RRDRES=900 rrdtool fetch subdata.rrd AVERAGE -r $RRDRES \ -e $(($TIME/$RRDRES*$RRDRES)) -s e-1h Or in Perl: perl -e '$ctime = time; $rrdres = 900; \ system "rrdtool fetch subdata.rrd AVERAGE \ -r $rrdres -e @{[int($ctime/$rrdres)*$rrdres]} -s e-1h"' AATT--SSTTYYLLEE TTIIMMEE SSPPEECCIIFFIICCAATTIIOONN Apart from the traditional _S_e_c_o_n_d_s _s_i_n_c_e _e_p_o_c_h, RRDtool does also understand at-style time specification. The specification is called "at-style" after the Unix command _a_t(1) that has moderately complex ways to specify time to run your job at a certain date and time. The at-style specification consists of two parts: the TTIIMMEE RREEFFEERREENNCCEE specification and the TTIIMMEE OOFFFFSSEETT specification. TTIIMMEE RREEFFEERREENNCCEE SSPPEECCIIFFIICCAATTIIOONN The time reference specification is used, well, to establish a reference moment in time (to which the time offset is then applied to). When present, it should come first, when omitted, it defaults to nnooww. On its own part, time reference consists of a _t_i_m_e_-_o_f_-_d_a_y reference (which should come first, if present) and a _d_a_y reference. The _t_i_m_e_-_o_f_-_d_a_y can be specified as HHHH::MMMM, HHHH..MMMM, or just HHHH. You can suffix it with aamm or ppmm or use 24-hours clock. Some special times of day are understood as well, including mmiiddnniigghhtt (00:00), nnoooonn (12:00) and British tteeaattiimmee (16:00). The _d_a_y can be specified as _m_o_n_t_h_-_n_a_m_e _d_a_y_-_o_f_-_t_h_e_-_m_o_n_t_h and optional a 2- or 4-digit _y_e_a_r number (e.g. March 8 1999). Alternatively, you can use _d_a_y_-_o_f_-_w_e_e_k_-_n_a_m_e (e.g. Monday), or one of the words: yyeesstteerrddaayy, ttooddaayy, ttoommoorrrrooww. You can also specify the _d_a_y as a full date in several numerical formats, including MMMM//DDDD//[[YYYY]]YYYY, DDDD..MMMM..[[YYYY]]YYYY, or YYYYYYYYMMMMDDDD. _N_O_T_E_1: this is different from the original _a_t(1) behavior, where a single-number date is interpreted as MMDD[YY]YY. _N_O_T_E_2: if you specify the _d_a_y in this way, the _t_i_m_e_-_o_f_-_d_a_y is REQUIRED as well. Finally, you can use the words nnooww, ssttaarrtt, eenndd or eeppoocchh as your time reference. NNooww refers to the current moment (and is also the default time reference). SSttaarrtt (eenndd) can be used to specify a time relative to the start (end) time for those tools that use these categories (rrrrddffeettcchh, rrdgraph) and eeppoocchh indicates the *IX epoch (*IX timestamp 0 = 1970-01-01 00:00:00 UTC). eeppoocchh is useful to disambiguate between a timestamp value and some forms of abbreviated date/time specifications, because it allows to use time offset specifications using units, eg. eeppoocchh+19711205s unambiguously denotes timestamp 19711205 and not 1971-12-05 00:00:00 UTC. Month and day of the week names can be used in their naturally abbreviated form (e.g., Dec for December, Sun for Sunday, etc.). The words nnooww, ssttaarrtt, eenndd can be abbreviated as nn, ss, ee. TTIIMMEE OOFFFFSSEETT SSPPEECCIIFFIICCAATTIIOONN The time offset specification is used to add/subtract certain time intervals to/from the time reference moment. It consists of a _s_i_g_n (++ or --) and an _a_m_o_u_n_t. The following time units can be used to specify the _a_m_o_u_n_t: yyeeaarrss, mmoonntthhss, wweeeekkss, ddaayyss, hhoouurrss, mmiinnuutteess, or sseeccoonnddss. These units can be used in singular or plural form, and abbreviated naturally or to a single letter (e.g. +3days, -1wk, -3y). Several time units can be combined (e.g., -5mon1w2d) or concatenated (e.g., -5h45min = -5h-45min = -6h+15min = -7h+1h30m-15min, etc.) _N_O_T_E_3: If you specify time offset in days, weeks, months, or years, you will end with the time offset that may vary depending on your time reference, because all those time units have no single well defined time interval value (1 year contains either 365 or 366 days, 1 month is 28 to 31 days long, and even 1 day may be not equal to 24 hours twice a year, when DST-related clock adjustments take place). To cope with this, when you use days, weeks, months, or years as your time offset units your time reference date is adjusted accordingly without too much further effort to ensure anything about it (in the hope that _m_k_t_i_m_e(3) will take care of this later). This may lead to some surprising (or even invalid!) results, e.g. 'May 31 -1month' = 'Apr 31' (meaningless) = 'May 1' (after _m_k_t_i_m_e(3) normalization); in the EET timezone '3:30am Mar 29 1999 -1 day' yields '3:30am Mar 28 1999' (Sunday) which is an invalid time/date combination (because of 3am -> 4am DST forward clock adjustment, see the below example). In contrast, hours, minutes, and seconds are well defined time intervals, and these are guaranteed to always produce time offsets exactly as specified (e.g. for EET timezone, '8:00 Mar 27 1999 +2 days' = '8:00 Mar 29 1999', but since there is 1-hour DST forward clock adjustment that occurs around 3:00 Mar 28 1999, the actual time interval between 8:00 Mar 27 1999 and 8:00 Mar 29 1999 equals 47 hours; on the other hand, '8:00 Mar 27 1999 +48 hours' = '9:00 Mar 29 1999', as expected) _N_O_T_E_4: The single-letter abbreviation for both mmoonntthhss and mmiinnuutteess is mm. To disambiguate them, the parser tries to read your mind :) by applying the following two heuristics: 1. If mm is used in context of (i.e. right after the) years, months, weeks, or days it is assumed to mean mmoonntthhss, while in the context of hours, minutes, and seconds it means minutes. (e.g., in -1y6m or +3w1m mm is interpreted as mmoonntthhss, while in -3h20m or +5s2m mm the parser decides for mmiinnuutteess). 2. Out of context (i.e. right after the ++ or -- sign) the meaning of mm is guessed from the number it directly follows. Currently, if the number's absolute value is below 25 it is assumed that mm means mmoonntthhss, otherwise it is treated as mmiinnuutteess. (e.g., -25m == -25 minutes, while +24m == +24 months) _F_i_n_a_l _N_O_T_E_S: Time specification is case-insensitive. Whitespace can be inserted freely or omitted altogether. There are, however, cases when whitespace is required (e.g., 'midnight Thu'). In this case you should either quote the whole phrase to prevent it from being taken apart by your shell or use '_' (underscore) or ',' (comma) which also count as whitespace (e.g., midnight_Thu or midnight,Thu). TTIIMMEE SSPPEECCIIFFIICCAATTIIOONN EEXXAAMMPPLLEESS _O_c_t _1_2 -- October 12 this year _-_1_m_o_n_t_h or _-_1_m -- current time of day, only a month before (may yield surprises, see NOTE3 above). _n_o_o_n _y_e_s_t_e_r_d_a_y _-_3_h_o_u_r_s -- yesterday morning; can also be specified as _9_a_m_-_1_d_a_y. _2_3_:_5_9 _3_1_._1_2_._1_9_9_9 -- 1 minute to the year 2000. _1_2_/_3_1_/_9_9 _1_1_:_5_9_p_m -- 1 minute to the year 2000 for imperialists. _1_2_a_m _0_1_/_0_1_/_0_1 -- start of the new millennium _e_n_d_-_3_w_e_e_k_s or _e_-_3_w -- 3 weeks before end time (may be used as start time specification). _s_t_a_r_t_+_6_h_o_u_r_s or _s_+_6_h -- 6 hours after start time (may be used as end time specification). _9_3_1_2_2_5_5_3_7 -- 18:45 July 5th, 1999 (yes, seconds since 1970 are valid as well). _1_9_9_7_0_7_0_3 _1_2_:_4_5 -- 12:45 July 3th, 1997 (my favorite, and its even got an ISO number (8601)). EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS The following environment variables may be used to change the behavior of "rrdtool fetch": RRRRDDCCAACCHHEEDD__AADDDDRREESSSS If this environment variable is set it will have the same effect as specifying the "--daemon" option on the command line. If both are present, the command line argument takes precedence. AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDFETCH(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdfirst (`third-party/rrdtool-1.4.8/doc/rrdfirst.txt`) RRDFIRST(1) rrdtool RRDFIRST(1) NNAAMMEE rrdfirst - Return the date of the first data sample in an RRA within an RRD SSYYNNOOPPSSIISS rrrrddttooooll ffiirrsstt _f_i_l_e_n_a_m_e [_-_-_r_r_a_i_n_d_e_x _n_u_m_b_e_r] DDEESSCCRRIIPPTTIIOONN The ffiirrsstt function returns the UNIX timestamp of the first data sample entered into the specified RRA of the RRD file. _f_i_l_e_n_a_m_e The name of the RRRRDD that contains the data. _-_-_r_r_a_i_n_d_e_x _n_u_m_b_e_r The index number of the RRRRAA that is to be examined. If not specified, the index defaults to zero. RRRRAA index numbers can be determined through rrrrddttooooll iinnffoo. AAUUTTHHOORR Burton Strauss 1.4.8 2013-05-23 RRDFIRST(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdflushcached (`third-party/rrdtool-1.4.8/doc/rrdflushcached.txt`) RRDFLUSHCACHED(1) rrdtool RRDFLUSHCACHED(1) NNAAMMEE rrdflushcached - Flush the values for a spcific RRD file from memory. SSYYNNOOPPSSIISS rrrrddttooooll fflluusshhccaacchheedd [----ddaaeemmoonn _a_d_d_r_e_s_s] _f_i_l_e_n_a_m_e [_f_i_l_e_n_a_m_e ...] DDEESSCCRRIIPPTTIIOONN The fflluusshhccaacchheedd function connects to rrdcached, the RRD caching daemon, and issues a "flush" command for the given files. The daemon will put the files to the head of the update queue so they are written "soon". The status will be returned only after the files' pending updates have been written to disk. _f_i_l_e_n_a_m_e The name(s) of the RRRRDD file(s) that are to be written to disk. ----ddaaeemmoonn _a_d_d_r_e_s_s Address of the rrdcached daemon. If not specified, the RRDCACHED_ADDRESS environment variable must be set (see below). For a list of accepted formats, see the --ll option in the rrdcached manual. rrdtool flush --daemon unix:/var/run/rrdcached.sock /var/lib/rrd/foo.rrd EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS The following environment variables may be used to change the behavior of "rrdtool flushcached": RRRRDDCCAACCHHEEDD__AADDDDRREESSSS If this environment variable is set it will have the same effect as specifying the "--daemon" option on the command line. If both are present, the command line argument takes precedence. AAUUTTHHOORR Florian Forster 1.4.8 2013-05-23 RRDFLUSHCACHED(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdgraph (`third-party/rrdtool-1.4.8/doc/rrdgraph.txt`) RRDGRAPH(1) rrdtool RRDGRAPH(1) NNAAMMEE rrdgraph - Round Robin Database tool graphing functions SSYYNNOOPPSSIISS rrrrddttooooll ggrraapphh||ggrraapphhvv _f_i_l_e_n_a_m_e [_o_p_t_i_o_n ...] [_d_a_t_a _d_e_f_i_n_i_t_i_o_n ...] [_d_a_t_a _c_a_l_c_u_l_a_t_i_o_n ...] [_v_a_r_i_a_b_l_e _d_e_f_i_n_i_t_i_o_n ...] [_g_r_a_p_h _e_l_e_m_e_n_t ...] [_p_r_i_n_t _e_l_e_m_e_n_t ...] DDEESSCCRRIIPPTTIIOONN The ggrraapphh function of RRRRDDttooooll is used to present the data from an RRRRDD to a human viewer. Its main purpose is to create a nice graphical representation, but it can also generate a numerical report. OOVVEERRVVIIEEWW rrrrddttooooll ggrraapphh needs data to work with, so you must use one or more ddaattaa ddeeffiinniittiioonn statements to collect this data. You are not limited to one database, it's perfectly legal to collect data from two or more databases (one per statement, though). If you want to display averages, maxima, percentiles, etcetera it is best to collect them now using the vvaarriiaabbllee ddeeffiinniittiioonn statement. Currently this makes no difference, but in a future version of RRDtool you may want to collect these values before consolidation. The data fetched from the RRRRAA is then ccoonnssoolliiddaatteedd so that there is exactly one data point per pixel in the graph. If you do not take care yourself, RRRRDDttooooll will expand the range slightly if necessary. Note, in that case the first and/or last pixel may very well become unknown! Sometimes data is not exactly in the format you would like to display it. For instance, you might be collecting bbyytteess per second, but want to display bbiittss per second. This is what the ddaattaa ccaallccuullaattiioonn command is designed for. After ccoonnssoolliiddaattiinngg the data, a copy is made and this copy is modified using a rather powerful RRPPNN command set. When you are done fetching and processing the data, it is time to graph it (or print it). This ends the rrrrddttooooll ggrraapphh sequence. Use ggrraapphhvv instead of ggrraapphh to get detailed information about the graph geometry and data once it is drawn. See the bottom of the document for more information. OOPPTTIIOONNSS _f_i_l_e_n_a_m_e The name and path of the graph to generate. It is recommended to end this in ".png", ".svg" or ".eps", but RRRRDDttooooll does not enforce this. _f_i_l_e_n_a_m_e can be '"-"' to send the image to "stdout". In this case, no other output is generated. TTiimmee rraannggee [--ss|----ssttaarrtt _t_i_m_e] [--ee|----eenndd _t_i_m_e] [--SS|----sstteepp _s_e_c_o_n_d_s] The start and end of the time series you would like to display, and which RRRRAA the data should come from. Defaults are: 1 day ago until now, with the best possible resolution. SSttaarrtt and eenndd can be specified in several formats, see AT-STYLE TIME SPECIFICATION and rrdgraph_examples. By default, rrrrddttooooll ggrraapphh calculates the width of one pixel in the time domain and tries to get data from an RRRRAA with that resolution. With the sstteepp option you can alter this behavior. If you want rrrrddttooooll ggrraapphh to get data at a one-hour resolution from the RRRRDD, set sstteepp to 3'600. Note: a step smaller than one pixel will silently be ignored. LLaabbeellss [--tt|----ttiittllee _s_t_r_i_n_g] [--vv|----vveerrttiiccaall--llaabbeell _s_t_r_i_n_g] A horizontal string at the top of the graph and/or a vertically placed string at the left hand side of the graph. SSiizzee [--ww|----wwiiddtthh _p_i_x_e_l_s] [--hh|----hheeiigghhtt _p_i_x_e_l_s] [--jj|----oonnllyy--ggrraapphh] [--DD|----ffuullll--ssiizzee--mmooddee] By default, the width and height of the ccaannvvaass (the part with the actual data and such). This defaults to 400 pixels by 100 pixels. If you specify the ----ffuullll--ssiizzee--mmooddee option, the width and height specify the final dimensions of the output image and the canvas is automatically resized to fit. If you specify the ----oonnllyy--ggrraapphh option and set the height < 32 pixels you will get a tiny graph image (thumbnail) to use as an icon for use in an overview, for example. All labeling will be stripped off the graph. LLiimmiittss [--uu|----uuppppeerr--lliimmiitt _v_a_l_u_e] [--ll|----lloowweerr--lliimmiitt _v_a_l_u_e] [--rr|----rriiggiidd] By default the graph will be autoscaling so that it will adjust the y-axis to the range of the data. You can change this behavior by explicitly setting the limits. The displayed y-axis will then range at least from lloowweerr--lliimmiitt to uuppppeerr--lliimmiitt. Autoscaling will still permit those boundaries to be stretched unless the rriiggiidd option is set. [--AA|----aalltt--aauuttoossccaallee] Sometimes the default algorithm for selecting the y-axis scale is not satisfactory. Normally the scale is selected from a predefined set of ranges and this fails miserably when you need to graph something like "260 + 0.001 * sin(x)". This option calculates the minimum and maximum y-axis from the actual minimum and maximum data values. Our example would display slightly less than "260-0.001" to slightly more than "260+0.001" (this feature was contributed by Sasha Mikheev). [--JJ|----aalltt--aauuttoossccaallee--mmiinn] Where "--alt-autoscale" will modify both the absolute maximum AND minimum values, this option will only affect the minimum value. The maximum value, if not defined on the command line, will be 0. This option can be useful when graphing router traffic when the WAN line uses compression, and thus the throughput may be higher than the WAN line speed. [--MM|----aalltt--aauuttoossccaallee--mmaaxx] Where "--alt-autoscale" will modify both the absolute maximum AND minimum values, this option will only affect the maximum value. The minimum value, if not defined on the command line, will be 0. This option can be useful when graphing router traffic when the WAN line uses compression, and thus the throughput may be higher than the WAN line speed. [--NN|----nnoo--ggrriiddffiitt] In order to avoid anti-aliasing blurring effects RRDtool snaps points to device resolution pixels, this results in a crisper appearance. If this is not to your liking, you can use this switch to turn this behavior off. Grid-fitting is turned off for PDF, EPS, SVG output by default. XX--AAxxiiss [--xx|----xx--ggrriidd _G_T_M::_G_S_T::_M_T_M::_M_S_T::_L_T_M::_L_S_T::_L_P_R::_L_F_M] [--xx|----xx--ggrriidd nnoonnee] The x-axis label is quite complex to configure. If you don't have very special needs it is probably best to rely on the auto configuration to get this right. You can specify the string "none" to suppress the grid and labels altogether. The grid is defined by specifying a certain amount of time in the _?_T_M positions. You can choose from "SECOND", "MINUTE", "HOUR", "DAY", "WEEK", "MONTH" or "YEAR". Then you define how many of these should pass between each line or label. This pair (_?_T_M_:_?_S_T) needs to be specified for the base grid (_G_?_?), the major grid (_M_?_?) and the labels (_L_?_?). For the labels you also must define a precision in _L_P_R and a _s_t_r_f_t_i_m_e format string in _L_F_M. _L_P_R defines where each label will be placed. If it is zero, the label will be placed right under the corresponding line (useful for hours, dates etcetera). If you specify a number of seconds here the label is centered on this interval (useful for Monday, January etcetera). --x-grid MINUTE:10:HOUR:1:HOUR:4:0:%X This places grid lines every 10 minutes, major grid lines every hour, and labels every 4 hours. The labels are placed under the major grid lines as they specify exactly that time. --x-grid HOUR:8:DAY:1:DAY:1:86400:%A This places grid lines every 8 hours, major grid lines and labels each day. The labels are placed exactly between two major grid lines as they specify the complete day and not just midnight. YY--AAxxiiss [--yy|----yy--ggrriidd _g_r_i_d _s_t_e_p::_l_a_b_e_l _f_a_c_t_o_r] [--yy|----yy--ggrriidd nnoonnee] Y-axis grid lines appear at each _g_r_i_d _s_t_e_p interval. Labels are placed every _l_a_b_e_l _f_a_c_t_o_r lines. You can specify "-y none" to suppress the grid and labels altogether. The default for this option is to automatically select sensible values. If you have set --y-grid to 'none' not only the labels get suppressed, also the space reserved for the labels is removed. You can still add space manually if you use the --units-length command to explicitly reserve space. [--YY|----aalltt--yy--ggrriidd] Place the Y grid dynamically based on the graph's Y range. The algorithm ensures that you always have a grid, that there are enough but not too many grid lines, and that the grid is metric. That is the grid lines are placed every 1, 2, 5 or 10 units. This parameter will also ensure that you get enough decimals displayed even if your graph goes from 69.998 to 70.001. (contributed by Sasha Mikheev). [--oo|----llooggaarriitthhmmiicc] Logarithmic y-axis scaling. [--XX|----uunniittss--eexxppoonneenntt _v_a_l_u_e] This sets the 10**exponent scaling of the y-axis values. Normally, values will be scaled to the appropriate units (k, M, etc.). However, you may wish to display units always in k (Kilo, 10e3) even if the data is in the M (Mega, 10e6) range, for instance. Value should be an integer which is a multiple of 3 between -18 and 18 inclusively. It is the exponent on the units you wish to use. For example, use 3 to display the y-axis values in k (Kilo, 10e3, thousands), use -6 to display the y-axis values in u (Micro, 10e-6, millionths). Use a value of 0 to prevent any scaling of the y-axis values. This option is very effective at confusing the heck out of the default RRDtool autoscaling function and grid painter. If RRDtool detects that it is not successful in labeling the graph under the given circumstances, it will switch to the more robust ----aalltt--yy--ggrriidd mode. [--LL|----uunniittss--lleennggtthh _v_a_l_u_e] How many digits should RRDtool assume the y-axis labels to be? You may have to use this option to make enough space once you start fiddling with the y-axis labeling. [----uunniittss==ssii] With this option y-axis values on logarithmic graphs will be scaled to the appropriate units (k, M, etc.) instead of using exponential notation. Note that for linear graphs, SI notation is used by default. RRiigghhtt YY AAxxiiss [----rriigghhtt--aaxxiiss _s_c_a_l_e::_s_h_i_f_t] [----rriigghhtt--aaxxiiss--llaabbeell _l_a_b_e_l] A second axis will be drawn to the right of the graph. It is tied to the left axis via the scale and shift parameters. You can also define a label for the right axis. [----rriigghhtt--aaxxiiss--ffoorrmmaatt _f_o_r_m_a_t_-_s_t_r_i_n_g] By default the format of the axis labels gets determined automatically. If you want to do this your self, use this option with the same %lf arguments you know from the PRINT and GPRINT commands. LLeeggeenndd [--gg|----nnoo--lleeggeenndd] Suppress generation of the legend; only render the graph. [--FF|----ffoorrccee--rruulleess--lleeggeenndd] Force the generation of HRULE and VRULE legends even if those HRULE or VRULE will not be drawn because out of graph boundaries (mimics behavior of pre 1.0.42 versions). [----lleeggeenndd--ppoossiittiioonn=(north|south|west|east)] Place the legend at the given side of the graph. The default is south. In west or east position it is necessary to add line breaks manually. [----lleeggeenndd--ddiirreeccttiioonn=(topdown|bottomup)] Place the legend items in the given vertical order. The default is topdown. Using bottomup the legend items appear in the same vertical order as a stack of lines or areas. MMiisscceellllaanneeoouuss [--zz|----llaazzyy] Only generate the graph if the current graph is out of date or not existent. Note, that all the calculations will happen regardless so that the output of PRINT and graphv will be complete regardless. Note that the behavior of lazy in this regard has seen several changes over time. The only thing you can really rely on before RRDtool 1.3.7 is that lazy will not generate the graph when it is already there and up to date, and also that it will output the size of the graph. [----ddaaeemmoonn _a_d_d_r_e_s_s] Address of the rrdcached daemon. If specified, a "flush" command is sent to the server before reading the RRD files. This allows the graph to contain fresh data even if the daemon is configured to cache values for a long time. For a list of accepted formats, see the --ll option in the rrdcached manual. rrdtool graph [...] --daemon unix:/var/run/rrdcached.sock [...] [--ff|----iimmggiinnffoo _p_r_i_n_t_f_s_t_r] After the image has been created, the graph function uses printf together with this format string to create output similar to the PRINT function, only that the printf function is supplied with the parameters _f_i_l_e_n_a_m_e, _x_s_i_z_e and _y_s_i_z_e. In order to generate an IIMMGG tag suitable for including the graph into a web page, the command line would look like this: --imginfo 'Demo' [--cc|----ccoolloorr _C_O_L_O_R_T_A_G#_r_r_g_g_b_b[_a_a]] Override the default colors for the standard elements of the graph. The _C_O_L_O_R_T_A_G is one of "BACK" background, "CANVAS" for the background of the actual graph, "SHADEA" for the left and top border, "SHADEB" for the right and bottom border, "GRID", "MGRID" for the major grid, "FONT" for the color of the font, "AXIS" for the axis of the graph, "FRAME" for the line around the color spots, and finally "ARROW" for the arrow head pointing up and forward. Each color is composed out of three hexadecimal numbers specifying its rgb color component (00 is off, FF is maximum) of red, green and blue. Optionally you may add another hexadecimal number specifying the transparency (FF is solid). You may set this option several times to alter multiple defaults. A green arrow is made by: "--color ARROW#00FF00" [----ggrriidd--ddaasshh _o_n::_o_f_f] by default the grid is drawn in a 1 on, 1 off pattern. With this option you can set this yourself --grid-dash 1:3 for a dot grid --grid-dash 1:0 for uninterrupted grid lines [----bboorrddeerr _w_i_d_t_h]] Width in pixels for the 3d border drawn around the image. Default 2, 0 disables the border. See "SHADEA" and "SHADEB" above for setting the border color. [----ddyynnaammiicc--llaabbeellss] Pick the shape of the color marker next to the label according to the element drawn on the graph. [--mm|----zzoooomm _f_a_c_t_o_r] Zoom the graphics by the given amount. The factor must be > 0 [--nn|----ffoonntt _F_O_N_T_T_A_G::_s_i_z_e::[_f_o_n_t]] This lets you customize which font to use for the various text elements on the RRD graphs. "DEFAULT" sets the default value for all elements, "TITLE" for the title, "AXIS" for the axis labels, "UNIT" for the vertical unit label, "LEGEND" for the graph legend, "WATERMARK" for the watermark on the edge of the graph. Use Times for the title: "--font TITLE:13:Times" Note that you need to quote the argument to ----ffoonntt if the font-name contains whitespace: --font "TITLE:13:Some Font" If you do not give a font string you can modify just the size of the default font: "--font TITLE:13:". If you specify the size 0 then you can modify just the font without touching the size. This is especially useful for altering the default font without resetting the default fontsizes: "--font DEFAULT:0:Courier". RRDtool comes with a preset default font. You can set the environment variable "RRD_DEFAULT_FONT" if you want to change this. RRDtool uses Pango for its font handling. This means you can to use the full Pango syntax when selecting your font: The font name has the form "[_F_A_M_I_L_Y_-_L_I_S_T] [_S_T_Y_L_E_-_O_P_T_I_O_N_S] [_S_I_Z_E]", where _F_A_M_I_L_Y_-_L_I_S_T is a comma separated list of families optionally terminated by a comma, _S_T_Y_L_E___O_P_T_I_O_N_S is a whitespace separated list of words where each WORD describes one of style, variant, weight, stretch, or gravity, and _S_I_Z_E is a decimal number (size in points) or optionally followed by the unit modifier "px" for absolute size. Any one of the options may be absent. [--RR|----ffoonntt--rreennddeerr--mmooddee {nnoorrmmaall,lliigghhtt,mmoonnoo}] There are 3 font render modes: nnoorrmmaall: Full Hinting and Anti-aliasing (default) lliigghhtt: Slight Hinting and Anti-aliasing mmoonnoo: Full Hinting and NO Anti-aliasing [--BB|----ffoonntt--ssmmooootthhiinngg--tthhrreesshhoolldd _s_i_z_e] (this gets ignored in 1.3 for now!) This specifies the largest font size which will be rendered bitmapped, that is, without any font smoothing. By default, no text is rendered bitmapped. [--PP|----ppaannggoo--mmaarrkkuupp] All text in RRDtool is rendered using Pango. With the ----ppaannggoo--mmaarrkkuupp option, all text will be processed by pango markup. This allows to embed some simple html like markup tags using text Apart from the verbose syntax, there are also the following short tags available. b Bold big Makes font relatively larger, equivalent to i Italic s Strikethrough sub Subscript sup Superscript small Makes font relatively smaller, equivalent to tt Monospace font u Underline More details on . [--GG|----ggrraapphh--rreennddeerr--mmooddee {nnoorrmmaall,mmoonnoo}] There are 2 render modes: nnoorrmmaall: Graphs are fully Anti-aliased (default) mmoonnoo: No Anti-aliasing [--EE|----ssllooppee--mmooddee] RRDtool graphs are composed of stair case curves by default. This is in line with the way RRDtool calculates its data. Some people favor a more 'organic' look for their graphs even though it is not all that true. [--aa|----iimmggffoorrmmaatt PPNNGG|SSVVGG|EEPPSS|PPDDFF] Image format for the generated graph. For the vector formats you can choose among the standard Postscript fonts Courier-Bold, Courier- BoldOblique, Courier-Oblique, Courier, Helvetica-Bold, Helvetica- BoldOblique, Helvetica-Oblique, Helvetica, Symbol, Times-Bold, Times- BoldItalic, Times-Italic, Times-Roman, and ZapfDingbats. [--ii|----iinntteerrllaacceedd] (this gets ignored in 1.3 for now!) If images are interlaced they become visible on browsers more quickly. [--TT|----ttaabbwwiiddtthh _v_a_l_u_e] By default the tab-width is 40 pixels, use this option to change it. [--bb|----bbaassee _v_a_l_u_e] If you are graphing memory (and NOT network traffic) this switch should be set to 1024 so that one Kb is 1024 byte. For traffic measurement, 1 kb/s is 1000 b/s. [--WW|----wwaatteerrmmaarrkk _s_t_r_i_n_g] Adds the given string as a watermark, horizontally centered, at the bottom of the graph. DDaattaa aanndd vvaarriiaabblleess DDEEFF::_v_n_a_m_e==_r_r_d_f_i_l_e::_d_s_-_n_a_m_e::_C_F[::sstteepp==_s_t_e_p][::ssttaarrtt==_t_i_m_e][::eenndd==_t_i_m_e] CCDDEEFF::_v_n_a_m_e==_R_P_N _e_x_p_r_e_s_s_i_o_n VVDDEEFF::_v_n_a_m_e==_R_P_N _e_x_p_r_e_s_s_i_o_n You need at least one DDEEFF and one LLIINNEE, AARREEAA, GGPPRRIINNTT, PPRRIINNTT statement to generate anything useful. See rrdgraph_data and rrdgraph_rpn for the exact format. NOTE: GGrraapphh aanndd pprriinntt eelleemmeennttss You need at least one graph element to generate an image and/or at least one print statement to generate a report. See rrdgraph_graph for the exact format. ggrraapphhvv Calling RRDtool with the graphv option will return information in the RRDtool info format. On the command line this means that all output will be in key=value format. When used from the Perl and Ruby bindings a hash pointer will be returned from the call. When the filename '-' is given, the contents of the graph itself will also be returned through this interface (hash key 'image'). On the command line the output will look like this: print[0] = "0.020833" print[1] = "0.0440833" graph_left = 51 graph_top = 22 graph_width = 400 graph_height = 100 graph_start = 1232908800 graph_end = 1232914200 image_width = 481 image_height = 154 value_min = 0.0000000000e+00 value_max = 4.0000000000e-02 image = BLOB_SIZE:8196 [... 8196 bytes of image data ...] There is more information returned than in the standard interface. Especially the 'graph_*' keys are new. They help applications that want to know what is where on the graph. EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS The following environment variables may be used to change the behavior of "rrdtool graph": RRRRDDCCAACCHHEEDD__AADDDDRREESSSS If this environment variable is set it will have the same effect as specifying the "--daemon" option on the command line. If both are present, the command line argument takes precedence. SSEEEE AALLSSOO rrdgraph gives an overview of how rrrrddttooooll ggrraapphh works. rrdgraph_data describes DDEEFF,CCDDEEFF and VVDDEEFF in detail. rrdgraph_rpn describes the RRPPNN language used in the ??DDEEFF statements. rrdgraph_graph page describes all of the graph and print functions. Make sure to read rrdgraph_examples for tips&tricks. AAUUTTHHOORR Program by Tobias Oetiker This manual page by Alex van den Bogaerdt with corrections and/or additions by several people 1.4.8 2013-05-23 RRDGRAPH(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdgraph Data (`third-party/rrdtool-1.4.8/doc/rrdgraph_data.txt`) RRDGRAPH_DATA(1) rrdtool RRDGRAPH_DATA(1) NNAAMMEE rrdgraph_data - preparing data for graphing in rrdtool graph SSYYNNOOPPSSIISS DDEEFF::_<_v_n_a_m_e_>=_<_r_r_d_f_i_l_e_>:_<_d_s_-_n_a_m_e_>:_<_C_F_>[:step=_<_s_t_e_p_>][:start=_<_t_i_m_e_>][:end=_<_t_i_m_e_>][:reduce=_<_CC_FF_>] VVDDEEFF:_v_n_a_m_e=_R_P_N _e_x_p_r_e_s_s_i_o_n CCDDEEFF:_v_n_a_m_e=_R_P_N _e_x_p_r_e_s_s_i_o_n DDEESSCCRRIIPPTTIIOONN These three instructions extract data values out of the RRRRDD files, optionally altering them (think, for example, of a bytes to bits conversion). If so desired, you can also define variables containing useful information such as maximum, minimum etcetera. Two of the instructions use a language called RRPPNN which is described in its own manual page. Variable names (_v_n_a_m_e) must be made up strings of the following characters "A-Z, a-z, 0-9, -,_" and a maximum length of 255 characters. When picking variable names, make sure you do not choose a name that is already taken by an RPN operator. A safe bet it to use lowercase or mixed case names for variables since operators will always be in uppercase. DDEEFF DDEEFF::_<_v_n_a_m_e_>=_<_r_r_d_f_i_l_e_>:_<_d_s_-_n_a_m_e_>:_<_C_F_>[:step=_<_s_t_e_p_>][:start=_<_t_i_m_e_>][:end=_<_t_i_m_e_>][:reduce=_<_CC_FF_>] This command fetches data from an RRRRDD file. The virtual name _v_n_a_m_e can then be used throughout the rest of the script. By default, an RRRRAA which contains the correct consolidated data at an appropriate resolution will be chosen. The resolution can be overridden with the --step option. The resolution can again be overridden by specifying the sstteepp ssiizzee. The time span of this data is the same as for the graph by default, you can override this by specifying ssttaarrtt aanndd eenndd. Remember to escape colons in the time specification! If the resolution of the data is higher than the resolution of the graph, the data will be further consolidated. This may result in a graph that spans slightly more time than requested. Ideally each point in the graph should correspond with one CCDDPP from an RRRRAA. For instance, if your RRRRDD has an RRRRAA with a resolution of 1800 seconds per CCDDPP, you should create an image with width 400 and time span 400*1800 seconds (use appropriate start and end times, such as "--start end-8days8hours"). If consolidation needs to be done, the CCFF of the RRRRAA specified in the DDEEFF itself will be used to reduce the data density. This behavior can be changed using ":reduce=__". This optional parameter specifies the CCFF to use during the data reduction phase. Example: DEF:ds0=router.rrd:ds0:AVERAGE DEF:ds0weekly=router.rrd:ds0:AVERAGE:step=7200 DEF:ds0weekly=router.rrd:ds0:AVERAGE:start=end-1h DEF:ds0weekly=router.rrd:ds0:AVERAGE:start=11\:00:end=start+1h VVDDEEFF VVDDEEFF:_v_n_a_m_e=_R_P_N _e_x_p_r_e_s_s_i_o_n This command returns a value and/or a time according to the RRPPNN statements used. The resulting _v_n_a_m_e will, depending on the functions used, have a value and a time component. When you use this _v_n_a_m_e in another RRPPNN expression, you are effectively inserting its value just as if you had put a number at that place. The variable can also be used in the various graph and print elements. Example: "VDEF:avg=mydata,AVERAGE" Note that currently only aggregation functions work in VDEF rpn expressions. Patches to change this are welcome. CCDDEEFF CCDDEEFF:_v_n_a_m_e=_R_P_N _e_x_p_r_e_s_s_i_o_n This command creates a new set of data points (in memory only, not in the RRRRDD file) out of one or more other data series. The RRPPNN instructions are used to evaluate a mathematical function on each data point. The resulting _v_n_a_m_e can then be used further on in the script, just as if it were generated by a DDEEFF instruction. Example: "CDEF:mydatabits=mydata,8,*" AAbboouutt CCDDEEFF vveerrssuuss VVDDEEFF At some point in processing, RRRRDDttooooll has gathered an array of rates ready to display. CCDDEEFF works on such an array. For example, _C_D_E_F_:_n_e_w_=_d_s_0_,_8_,_* would multiply each of the array members by eight (probably transforming bytes into bits). The result is an array containing the new values. VVDDEEFF also works on such an array but in a different way. For example, _V_D_E_F_:_m_a_x_=_d_s_0_,_M_A_X_I_M_U_M would scan each of the array members and store the maximum value. WWhheenn ddoo yyoouu uussee VVDDEEFF vveerrssuuss CCDDEEFF?? Use CCDDEEFF to transform your data prior to graphing. In the above example, we'd use a CCDDEEFF to transform bytes to bits before graphing the bits. You use a VVDDEEFF if you want _m_a_x_(_1_,_5_,_3_,_2_,_4_) to return five which would be displayed in the graph's legend (to answer, what was the maximum value during the graph period). If you want to apply 'complex' operations to the result of a VDEF you have to use a CDEF again since VDEFs only look like RPN expressions, they aren't really. SSEEEE AALLSSOO rrdgraph gives an overview of how rrrrddttooooll ggrraapphh works. rrdgraph_data describes DDEEFF,CCDDEEFF and VVDDEEFF in detail. rrdgraph_rpn describes the RRPPNN language used in the ??DDEEFF statements. rrdgraph_graph page describes all of the graph and print functions. Make sure to read rrdgraph_examples for tips&tricks. AAUUTTHHOORR Program by Tobias Oetiker This manual page by Alex van den Bogaerdt with corrections and/or additions by several people 1.4.8 2013-05-23 RRDGRAPH_DATA(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdgraph Examples (`third-party/rrdtool-1.4.8/doc/rrdgraph_examples.txt`) RRDGRAPH_EXAMPLES(1) rrdtool RRDGRAPH_EXAMPLES(1) NNAAMMEE rrdgraph_examples - Examples for rrdtool graph SSYYNNOOPPSSIISS rrrrddttooooll ggrraapphh //hhoommee//hhttttppdd//hhttmmll//tteesstt..ppnngg ----iimmgg--ffoorrmmaatt PPNNGG followed by any of the examples below DDEESSCCRRIIPPTTIIOONN For your convenience some of the commands are explained here by using detailed examples. They are not always cut-and-paste ready because comments are intermixed with the examples. EEXXAAMMPPLLEESS DDaattaa wwiitthh mmuullttiippllee rreessoolluuttiioonnss --end now --start end-120000s --width 400 DEF:ds0a=/home/rrdtool/data/router1.rrd:ds0:AVERAGE DEF:ds0b=/home/rrdtool/data/router1.rrd:ds0:AVERAGE:step=1800 DEF:ds0c=/home/rrdtool/data/router1.rrd:ds0:AVERAGE:step=7200 LINE1:ds0a#0000FF:"default resolution\l" LINE1:ds0b#00CCFF:"resolution 1800 seconds per interval\l" LINE1:ds0c#FF00FF:"resolution 7200 seconds per interval\l" NNiicceellyy ffoorrmmaatttteedd lleeggeenndd sseeccttiioonn DEF:ds0=/home/rrdtool/data/router1.rrd:ds0:AVERAGE DEF:ds1=/home/rrdtool/data/router1.rrd:ds1:AVERAGE VDEF:ds0max=ds0,MAXIMUM VDEF:ds0avg=ds0,AVERAGE VDEF:ds0min=ds0,MINIMUM VDEF:ds0pct=ds0,95,PERCENT VDEF:ds1max=ds1,MAXIMUM VDEF:ds1avg=ds1,AVERAGE VDEF:ds1min=ds1,MINIMUM VDEF:ds1pct=ds1,95,PERCENT Note: consolidation occurs here. CDEF:ds0bits=ds0,8,* CDEF:ds1bits=ds1,8,* Note: 10 spaces to move text to the right COMMENT:" " Note: the column titles have to be as wide as the columns COMMENT:"Maximum " COMMENT:"Average " COMMENT:"Minimum " COMMENT:"95th percentile\l" AREA:ds0bits#00C000:"Inbound " GPRINT:ds0max:"%6.2lf %Sbps" GPRINT:ds0avg:"%6.2lf %Sbps" GPRINT:ds0min:"%6.2lf %Sbps" GPRINT:ds0pct:"%6.2lf %Sbps\l" LINE1:ds1bits#0000FF:"Outbound" GPRINT:ds1max:"%6.2lf %Sbps" GPRINT:ds1avg:"%6.2lf %Sbps" GPRINT:ds1min:"%6.2lf %Sbps" GPRINT:ds1pct:"%6.2lf %Sbps\l" OOffffsseettttiinngg aa lliinnee oonn tthhee yy--aaxxiiss Depending on your needs you can do this in two ways: � Offset the data, then graph this DEF:mydata=my.rrd:ds:AVERAGE Note: this will also influence any other command that uses "data" CDEF:data=mydata,100,+ LINE1:data#FF0000:"Data with offset" � Graph the original data, with an offset DEF:mydata=my.rrd:ds:AVERAGE Note: no color in the first line so it is not visible LINE1:100 Note: the second line gets stacked on top of the first one LINE1:data#FF0000:"Data with offset":STACK DDrraawwiinngg ddaasshheedd lliinneess Also works for HRULE and VRULE � default style: - - - - - LINE1:data#FF0000:"dashed line":dashes � more fancy style with offset: - - --- - --- - LINE1:data#FF0000:"another dashed line":dashes=15,5,5,10:dash-offset=10 TTiimmee rraannggeess Last four weeks: --start end-4w --end 00:00 January 2001: --start 20010101 --end start+31d January 2001: --start 20010101 --end 20010201 Last hour: --start end-1h Last 24 hours: Yesterday: --end 00:00 VViieewwiinngg tthhee ccuurrrreenntt aanndd pprreevviioouuss wweeeekk ttooggeetthheerr --end now --start end-1w DEF:thisweek=router.rrd:ds0:AVERAGE DEF:lastweek=router.rrd:ds0:AVERAGE:end=now-1w:start=end-1w Shift the data forward by one week (604800 seconds) SHIFT:lastweek:604800 [ more of the usual VDEF and CDEF stuff if you like ] AREA:lastweek#0000FF:Last\ week LINE1:thisweek#FF0000:This\ week AAbbeerrrraanntt BBeehhaavviioouurr DDeetteeccttiioonn If the specialized function RRRRAAss exist for aberrant behavior detection, they can be used to generate the graph of a time series with confidence bands and failures. rrdtool graph example.png \ DEF:obs=monitor.rrd:ifOutOctets:AVERAGE \ DEF:pred=monitor.rrd:ifOutOctets:HWPREDICT \ DEF:dev=monitor.rrd:ifOutOctets:DEVPREDICT \ DEF:fail=monitor.rrd:ifOutOctets:FAILURES \ TICK:fail#ffffa0:1.0:"Failures\: Average bits out" \ CDEF:scaledobs=obs,8,* \ CDEF:upper=pred,dev,2,*,+ \ CDEF:lower=pred,dev,2,*,- \ CDEF:scaledupper=upper,8,* \ CDEF:scaledlower=lower,8,* \ LINE2:scaledobs#0000ff:"Average bits out" \ LINE1:scaledupper#ff0000:"Upper Confidence Bound: Average bits out" \ LINE1:scaledlower#ff0000:"Lower Confidence Bound: Average bits out" This example generates a graph of the data series in blue (LINE2 with the scaledobs virtual data source), confidence bounds in red (scaledupper and scaledlower virtual data sources), and potential failures (i.e. potential aberrant aberrant behavior) marked by vertical yellow lines (the fail data source). The raw data comes from an AVERAGE RRRRAA, the finest resolution of the observed time series (one consolidated data point per primary data point). The predicted (or smoothed) values are stored in the HWPREDICT RRRRAA. The predicted deviations (think standard deviation) values are stored in the DEVPREDICT RRRRAA. Finally, the FAILURES RRRRAA contains indicators, with 1 denoting a potential failure. All of the data is rescaled to bits (instead of Octets) by multiplying by 8. The confidence bounds are computed by an offset of 2 deviations both above and below the predicted values (the CDEFs upper and lower). Vertical lines indicated potential failures are graphed via the TICK graph element, which converts non-zero values in an RRRRAA into tick marks. Here an axis-fraction argument of 1.0 means the tick marks span the entire y-axis, and hence become vertical lines on the graph. The choice of 2 deviations (a scaling factor) matches the default used internally by the FAILURES RRRRAA. If the internal value is changed (see rrdtune), this graphing command should be changed to be consistent. _A _n_o_t_e _o_n _d_a_t_a _r_e_d_u_c_t_i_o_n_: The rrrrddttooooll _g_r_a_p_h command is designed to plot data at a specified temporal resolution, regardless of the actually resolution of the data in the RRD file. This can present a problem for the specialized consolidation functions which maintain a one-to-one mapping between primary data points and consolidated data points. If a graph insists on viewing the contents of these RRRRAAss on a coarser temporal scale, the _g_r_a_p_h command tries to do something intelligent, but the confidence bands and failures no longer have the same meaning and may be misleading. SSEEEE AALLSSOO rrdgraph gives an overview of how rrrrddttooooll ggrraapphh works. rrdgraph_data describes DDEEFF,CCDDEEFF and VVDDEEFF in detail. rrdgraph_rpn describes the RRPPNN language used in the xxDDEEFF statements. rrdgraph_graph page describes all the graph and print functions. AAUUTTHHOORR Program by Tobias Oetiker This manual page by Alex van den Bogaerdt with corrections and/or additions by several people 1.4.8 2013-05-23 RRDGRAPH_EXAMPLES(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdgraph Graph (`third-party/rrdtool-1.4.8/doc/rrdgraph_graph.txt`) RRDGRAPH_GRAPH(1) rrdtool RRDGRAPH_GRAPH(1) NNAAMMEE rrdgraph_graph - rrdtool graph command reference SSYYNNOOPPSSIISS PPRRIINNTT::_v_n_a_m_e::_f_o_r_m_a_t GGPPRRIINNTT::_v_n_a_m_e::_f_o_r_m_a_t CCOOMMMMEENNTT::_t_e_x_t VVRRUULLEE::_t_i_m_e##_c_o_l_o_r[::_l_e_g_e_n_d][::ddaasshheess[==_o_n___s[,_o_f_f___s[,_o_n___s,_o_f_f___s]...]][::ddaasshh--ooffffsseett==_o_f_f_s_e_t]] HHRRUULLEE::_v_a_l_u_e##_c_o_l_o_r[::_l_e_g_e_n_d][::ddaasshheess[==_o_n___s[,_o_f_f___s[,_o_n___s,_o_f_f___s]...]][::ddaasshh--ooffffsseett==_o_f_f_s_e_t]] LLIINNEE[_w_i_d_t_h]::_v_a_l_u_e[##_c_o_l_o_r][::[_l_e_g_e_n_d][::SSTTAACCKK][::sskkiippssccaallee][::ddaasshheess[==_o_n___s[,_o_f_f___s[,_o_n___s,_o_f_f___s]...]][::ddaasshh--ooffffsseett==_o_f_f_s_e_t]] AARREEAA::_v_a_l_u_e[##_c_o_l_o_r][::[_l_e_g_e_n_d][::SSTTAACCKK][::sskkiippssccaallee]] TTIICCKK::_v_n_a_m_e##_r_r_g_g_b_b[_a_a][::_f_r_a_c_t_i_o_n[::_l_e_g_e_n_d]] SSHHIIFFTT::_v_n_a_m_e::_o_f_f_s_e_t TTEEXXTTAALLIIGGNN::{lleefftt|rriigghhtt|jjuussttiiffiieedd|cceenntteerr} PPRRIINNTT::_v_n_a_m_e::_C_F::_f_o_r_m_a_t (deprecated) GGPPRRIINNTT::_v_n_a_m_e::_C_F::_f_o_r_m_a_t (deprecated) SSTTAACCKK::_v_n_a_m_e##_c_o_l_o_r[::_l_e_g_e_n_d] (deprecated) DDEESSCCRRIIPPTTIIOONN These instructions allow you to generate your image or report. If you don't use any graph elements, no graph is generated. Similarly, no report is generated if you don't use print options. PPRRIINNTT _PP_RR_II_NN_TT_::_v_n_a_m_e_::_f_o_r_m_a_t_[_::_ss_tt_rr_ff_tt_ii_mm_ee_] Depending on the context, either the value component or the time component of a VVDDEEFF is printed using _f_o_r_m_a_t. It is an error to specify a _v_n_a_m_e generated by a DDEEFF or CCDDEEFF. Any text in _f_o_r_m_a_t is printed literally with one exception: The percent character introduces a formatter string. This string can be: For printing values: %%%% just prints a literal '%' character %%##..##llee prints numbers like 1.2346e+04. The optional integers # denote field width and decimal precision. %%##..##llff prints numbers like 12345.6789, with optional field width and precision. %%ss place this after %%llee, %%llff or %%llgg. This will be replaced by the appropriate SI magnitude unit and the value will be scaled accordingly (123456 -> 123.456 k). %%SS is similar to %%ss. It does, however, use a previously defined magnitude unit. If there is no such unit yet, it tries to define one (just like %%ss) unless the value is zero, in which case the magnitude unit stays undefined. Thus, formatter strings using %%SS and no %%ss will all use the same magnitude unit except for zero values. If you PRINT a VDEF value, you can also print the time associated with it by appending the string ::ssttrrffttiimmee to the format. Note that RRDtool uses the strftime function of your OSs C library. This means that the conversion specifier may vary. Check the manual page if you are uncertain. The following is a list of conversion specifiers usually supported across the board. %%aa The abbreviated weekday name according to the current locale. %%AA The full weekday name according to the current locale. %%bb The abbreviated month name according to the current locale. %%BB The full month name according to the current locale. %%cc The preferred date and time representation for the current locale. %%dd The day of the month as a decimal number (range 01 to 31). %%HH The hour as a decimal number using a 24-hour clock (range 00 to 23). %%II The hour as a decimal number using a 12-hour clock (range 01 to 12). %%jj The day of the year as a decimal number (range 001 to 366). %%mm The month as a decimal number (range 01 to 12). %%MM The minute as a decimal number (range 00 to 59). %%pp Either `AM' or `PM' according to the given time value, or the corresponding strings for the current locale. Noon is treated as `pm' and midnight as `am'. Note that in many locales and `pm' notation is unsupported and in such cases %p will return an empty string. %%ss The second as a decimal number (range 00 to 61). %%SS The seconds since the epoch (1.1.1970) (libc dependent non standard!) %%UU The week number of the current year as a decimal number, range 00 to 53, starting with the first Sunday as the first day of week 01. See also %V and %W. %%VV The ISO 8601:1988 week number of the current year as a decimal number, range 01 to 53, where week 1 is the first week that has at least 4 days in the current year, and with Monday as the first day of the week. See also %U and %W. %%ww The day of the week as a decimal, range 0 to 6, Sunday being 0. See also %u. %%WW The week number of the current year as a decimal number, range 00 to 53, starting with the first Monday as the first day of week 01. %%xx The preferred date representation for the current locale without the time. %%XX The preferred time representation for the current locale without the date. %%yy The year as a decimal number without a century (range 00 to 99). %%YY The year as a decimal number including the century. %%ZZ The time zone or name or abbreviation. %%%% A literal `%' character. _PP_RR_II_NN_TT_::_v_n_a_m_e_::_C_F_::_f_o_r_m_a_t _D_e_p_r_e_c_a_t_e_d_. _U_s_e _t_h_e _n_e_w _f_o_r_m _o_f _t_h_i_s _c_o_m_m_a_n_d _i_n _n_e_w _s_c_r_i_p_t_s_. The first form of this command is to be used with CCDDEEFF _v_n_a_m_es. GGRRAAPPHH _GG_PP_RR_II_NN_TT_::_v_n_a_m_e_::_f_o_r_m_a_t This is the same as "PRINT", but printed inside the graph. _GG_PP_RR_II_NN_TT_::_v_n_a_m_e_::_C_F_::_f_o_r_m_a_t _D_e_p_r_e_c_a_t_e_d_. _U_s_e _t_h_e _n_e_w _f_o_r_m _o_f _t_h_i_s _c_o_m_m_a_n_d _i_n _n_e_w _s_c_r_i_p_t_s_. This is the same as "PRINT", but printed inside the graph. _CC_OO_MM_MM_EE_NN_TT_::_t_e_x_t Text is printed literally in the legend section of the graph. Note that in RRDtool 1.2 you have to escape colons in COMMENT text in the same way you have to escape them in **PPRRIINNTT commands by writing ''\\::''. _VV_RR_UU_LL_EE_::_t_i_m_e_##_c_o_l_o_r_[_::_l_e_g_e_n_d_]_[_::_dd_aa_ss_hh_ee_ss_[_==_o_n___s_[_,_o_f_f___s_[_,_o_n___s_,_o_f_f___s_]_._._._]_]_[_::_dd_aa_ss_hh_--_oo_ff_ff_ss_ee_tt_==_o_f_f_s_e_t_]_] Draw a vertical line at _t_i_m_e. Its color is composed from three hexadecimal numbers specifying the rgb color components (00 is off, FF is maximum) red, green and blue followed by an optional alpha. Optionally, a legend box and string is printed in the legend section. _t_i_m_e may be a number or a variable from a VVDDEEFF. It is an error to use _v_n_a_m_es from DDEEFF or CCDDEEFF here. Dashed lines can be drawn using the ddaasshheess modifier. See LLIINNEE for more details. _HH_RR_UU_LL_EE_::_v_a_l_u_e_##_c_o_l_o_r_[_::_l_e_g_e_n_d_]_[_::_dd_aa_ss_hh_ee_ss_[_==_o_n___s_[_,_o_f_f___s_[_,_o_n___s_,_o_f_f___s_]_._._._]_]_[_::_dd_aa_ss_hh_--_oo_ff_ff_ss_ee_tt_==_o_f_f_s_e_t_]_] Draw a horizontal line at _v_a_l_u_e. HRULE acts much like LINE except that will have no effect on the scale of the graph. If a HRULE is outside the graphing area it will just not be visible. _LL_II_NN_EE_[_w_i_d_t_h_]_::_v_a_l_u_e_[_##_c_o_l_o_r_]_[_::_[_l_e_g_e_n_d_]_[_::_SS_TT_AA_CC_KK_]_[_::_ss_kk_ii_pp_ss_cc_aa_ll_ee_]_]_[_::_dd_aa_ss_hh_ee_ss_[_==_o_n___s_[_,_o_f_f___s_[_,_o_n___s_,_o_f_f___s_]_._._._]_]_[_::_dd_aa_ss_hh_--_oo_ff_ff_ss_ee_tt_==_o_f_f_s_e_t_]_] Draw a line of the specified width onto the graph. _w_i_d_t_h can be a floating point number. If the color is not specified, the drawing is done 'invisibly'. This is useful when stacking something else on top of this line. Also optional is the legend box and string which will be printed in the legend section if specified. The vvaalluuee can be generated by DDEEFF, VVDDEEFF, and CCDDEEFF. If the optional SSTTAACCKK modifier is used, this line is stacked on top of the previous element which can be a LLIINNEE or an AARREEAA. Normally the graphing function makes sure that the entire LLIINNEE or AARREEAA is visible in the chart. The scaling of the chart will be modified accordingly if necessary. Any LLIINNEE or AARREEAA can be excluded from this process by adding the option sskkiippssccaallee. The ddaasshheess modifier enables dashed line style. Without any further options a symmetric dashed line with a segment length of 5 pixels will be drawn. The dash pattern can be changed if the ddaasshheess== parameter is followed by either one value or an even number (1, 2, 4, 6, ...) of positive values. Each value provides the length of alternate _o_n___s and _o_f_f___s portions of the stroke. The ddaasshh--ooffffsseett parameter specifies an _o_f_f_s_e_t into the pattern at which the stroke begins. When you do not specify a color, you cannot specify a legend. Should you want to use SSTTAACCKK, use the "LINEx:::STACK" form. _AA_RR_EE_AA_::_v_a_l_u_e_[_##_c_o_l_o_r_]_[_::_[_l_e_g_e_n_d_]_[_::_SS_TT_AA_CC_KK_]_[_::_ss_kk_ii_pp_ss_cc_aa_ll_ee_]_] See LLIINNEE, however the area between the x-axis and the line will be filled. _TT_II_CC_KK_::_v_n_a_m_e_##_r_r_g_g_b_b_[_a_a_]_[_::_f_r_a_c_t_i_o_n_[_::_l_e_g_e_n_d_]_] Plot a tick mark (a vertical line) for each value of _v_n_a_m_e that is non- zero and not *UNKNOWN*. The _f_r_a_c_t_i_o_n argument specifies the length of the tick mark as a fraction of the y-axis; the default value is 0.1 (10% of the axis). Note that the color specification is not optional. The TICK marks normally start at the lower edge of the graphing area. If the fraction is negative they start at the upper border of the graphing area. _SS_HH_II_FF_TT_::_v_n_a_m_e_::_o_f_f_s_e_t Using this command RRRRDDttooooll will graph the following elements with the specified offset. For instance, you can specify an offset of ( 7*24*60*60 = ) 604'800 seconds to "look back" one week. Make sure to tell the viewer of your graph you did this ... As with the other graphing elements, you can specify a number or a variable here. _TT_EE_XX_TT_AA_LL_II_GG_NN_::_{_ll_ee_ff_tt_|_rr_ii_gg_hh_tt_|_jj_uu_ss_tt_ii_ff_ii_ee_dd_|_cc_ee_nn_tt_ee_rr_} Labels are placed below the graph. When they overflow to the left, they wrap to the next line. By default, lines are justified left and right. The TTEEXXTTAALLIIGGNN function lets you change this default. This is a command and not an option, so that you can change the default several times in your argument list. _SS_TT_AA_CC_KK_::_v_n_a_m_e_##_c_o_l_o_r_[_::_l_e_g_e_n_d_] _D_e_p_r_e_c_a_t_e_d_. _U_s_e _t_h_e _SS_TT_AA_CC_KK _m_o_d_i_f_i_e_r_s _o_n _t_h_e _o_t_h_e_r _c_o_m_m_a_n_d_s _i_n_s_t_e_a_d_! SSoommee nnootteess oonn ssttaacckkiinngg When stacking, an element is not placed above the X-axis but rather on top of the previous element. There must be something to stack upon. You can use an iinnvviissiibbllee LINE or AREA to stacked upon. An uunnkknnoowwnn value makes the entire stack unknown from that moment on. You don't know where to begin (the unknown value) and therefore do not know where to end. If you want to make sure you will be displaying a certain variable, make sure never to stack upon the unknown value. Use a CDEF instruction with IIFF and UUNN to do so. NNOOTTEESS oonn lleeggeenndd aarrgguummeennttss EEssccaappiinngg tthhee ccoolloonn A colon ':' in a _l_e_g_e_n_d argument will mark the end of the legend. To enter a ':' as part of a legend, the colon must be escaped with a backslash '\:'. Beware that many environments process backslashes themselves, so it may be necessary to write two backslashes in order to one being passed onto rrd_graph. SSttrriinngg FFoorrmmaattttiinngg The text printed below the actual graph can be formatted by appending special escape characters at the end of a text. When ever such a character occurs, all pending text is pushed onto the graph according to the character specified. Valid markers are: \\jj for justified, \\ll for left aligned, \\rr for right aligned, and \\cc for centered. In the next section there is an example showing how to use centered formatting. \\nn is a valid alias for \\ll since incomplete parsing in earlier versions of RRDtool lead to this behavior and a number of people has been using it. Normally there are two space characters inserted between every two items printed into the graph. The space following a string can be suppressed by putting a \\gg at the end of the string. The \\gg also ignores any space inside the string if it is at the very end of the string. This can be used in connection with %%ss to suppress empty unit strings. GPRINT:a:MAX:%lf%s\g A special case is COMMENT:\\ss which inserts some additional vertical space before placing the next row of legends. If you want to have left and right aligned legends on the same line use COMMENT:\\uu to go one line back like this: COMMENT:left\l COMMENT:\u COMMENT:right\r There is also a 'nop' control for situations where you want a string to actually end in a backslash character sequence \\.. COMMENT:OS\2\. When using a proportional font in your graph, the tab characters or the sequence \\tt will line-up legend elements. Note that the tabs inserted are relative to the start of the current legend element! Since RRDtool 1.3 is using Pango for rending text, you can use Pango markup. Pango uses the xml ssppaann tags for inline formatting instructions.: A simple example of a marked-up string might be: Blue text is cool! The complete list of attributes for the span tag (taken from the pango documentation): ffoonntt__ddeesscc A font description string, such as "Sans Italic 12"; note that any other span attributes will override this description. So if you have "Sans Italic" and also a style="normal" attribute, you will get Sans normal, not italic. ffoonntt__ffaammiillyy A font family name ffaaccee Synonym for font_family ssiizzee Font size in 1024ths of a point, or one of the absolute sizes 'xx-small', 'x-small', 'small', 'medium', 'large', 'x-large', 'xx-large', or one of the relative sizes 'smaller' or 'larger'. If you want to specify a absolute size, it's usually easier to take advantage of the ability to specify a partial font description using 'font_desc'; you can use font_desc='12.5' rather than size='12800'. ssttyyllee One of 'normal', 'oblique', 'italic' wweeiigghhtt One of 'ultralight', 'light', 'normal', 'bold', 'ultrabold', 'heavy', or a numeric weight vvaarriiaanntt 'normal' or 'smallcaps' ssttrreettcchh One of 'ultracondensed', 'extracondensed', 'condensed', 'semicondensed', 'normal', 'semiexpanded', 'expanded', 'extraexpanded', 'ultraexpanded' ffoorreeggrroouunndd An RGB color specification such as '#00FF00' or a color name such as 'red' bbaacckkggrroouunndd An RGB color specification such as '#00FF00' or a color name such as 'red' uunnddeerrlliinnee One of 'none', 'single', 'double', 'low', 'error' uunnddeerrlliinnee__ccoolloorr The color of underlines; an RGB color specification such as '#00FF00' or a color name such as 'red' rriissee Vertical displacement, in 10000ths of an em. Can be negative for subscript, positive for superscript. ssttrriikkeetthhrroouugghh 'true' or 'false' whether to strike through the text ssttrriikkeetthhrroouugghh__ccoolloorr The color of crossed out lines; an RGB color specification such as '#00FF00' or a color name such as 'red' ffaallllbbaacckk 'true' or 'false' whether to enable fallback. If disabled, then characters will only be used from the closest matching font on the system. No fallback will be done to other fonts on the system that might contain the characters in the text. Fallback is enabled by default. Most applications should not disable fallback. llaanngg A language code, indicating the text language lleetttteerr__ssppaacciinngg Inter-letter spacing in 1024ths of a point. ggrraavviittyy One of 'south', 'east', 'north', 'west', 'auto'. ggrraavviittyy__hhiinntt One of 'natural', 'strong', 'line'. To save you some typing, there are also some shortcuts: bb Bold bbiigg Makes font relatively larger, equivalent to ii Italic ss Strike through ssuubb Subscript ssuupp Superscript ssmmaallll Makes font relatively smaller, equivalent to tttt Monospace font uu Underline SSEEEE AALLSSOO rrdgraph gives an overview of how rrrrddttooooll ggrraapphh works. rrdgraph_data describes DDEEFF,CCDDEEFF and VVDDEEFF in detail. rrdgraph_rpn describes the RRPPNN language used in the ??DDEEFF statements. rrdgraph_graph page describes all of the graph and print functions. Make sure to read rrdgraph_examples for tips&tricks. AAUUTTHHOORR Program by Tobias Oetiker This manual page by Alex van den Bogaerdt with corrections and/or additions by several people 1.4.8 2013-05-23 RRDGRAPH_GRAPH(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdgraph Rpn (`third-party/rrdtool-1.4.8/doc/rrdgraph_rpn.txt`) RRDGRAPH_RPN(1) rrdtool RRDGRAPH_RPN(1) NNAAMMEE rrdgraph_rpn - About RPN Math in rrdtool graph SSYYNNOOPPSSIISS _R_P_N _e_x_p_r_e_s_s_i_o_n:=_v_n_a_m_e|_o_p_e_r_a_t_o_r|_v_a_l_u_e[,_R_P_N _e_x_p_r_e_s_s_i_o_n] DDEESSCCRRIIPPTTIIOONN If you have ever used a traditional HP calculator you already know RRPPNN (Reverse Polish Notation). The idea behind RRPPNN is that you have a stack and push your data onto this stack. Whenever you execute an operation, it takes as many elements from the stack as needed. Pushing is done implicitly, so whenever you specify a number or a variable, it gets pushed onto the stack automatically. At the end of the calculation there should be one and only one value left on the stack. This is the outcome of the function and this is what is put into the _v_n_a_m_e. For CCDDEEFF instructions, the stack is processed for each data point on the graph. VVDDEEFF instructions work on an entire data set in one run. Note, that currently VVDDEEFF instructions only support a limited list of functions. Example: "VDEF:maximum=mydata,MAXIMUM" This will set variable "maximum" which you now can use in the rest of your RRD script. Example: "CDEF:mydatabits=mydata,8,*" This means: push variable _m_y_d_a_t_a, push the number 8, execute the operator _*. The operator needs two elements and uses those to return one value. This value is then stored in _m_y_d_a_t_a_b_i_t_s. As you may have guessed, this instruction means nothing more than _m_y_d_a_t_a_b_i_t_s _= _m_y_d_a_t_a _* _8. The real power of RRPPNN lies in the fact that it is always clear in which order to process the input. For expressions like "a = b + 3 * 5" you need to multiply 3 with 5 first before you add _b to get _a. However, with parentheses you could change this order: "a = (b + 3) * 5". In RRPPNN, you would do "a = b, 3, +, 5, *" without the need for parentheses. OOPPEERRAATTOORRSS Boolean operators LLTT,, LLEE,, GGTT,, GGEE,, EEQQ,, NNEE Pop two elements from the stack, compare them for the selected condition and return 1 for true or 0 for false. Comparing an _u_n_k_n_o_w_n or an _i_n_f_i_n_i_t_e value will result in _u_n_k_n_o_w_n returned ... which will also be treated as false by the IIFF call. UUNN,, IISSIINNFF Pop one element from the stack, compare this to _u_n_k_n_o_w_n respectively to _p_o_s_i_t_i_v_e _o_r _n_e_g_a_t_i_v_e _i_n_f_i_n_i_t_y. Returns 1 for true or 0 for false. IIFF Pops three elements from the stack. If the element popped last is 0 (false), the value popped first is pushed back onto the stack, otherwise the value popped second is pushed back. This does, indeed, mean that any value other than 0 is considered to be true. Example: "A,B,C,IF" should be read as "if (A) then (B) else (C)" Comparing values MMIINN,, MMAAXX Pops two elements from the stack and returns the smaller or larger, respectively. Note that _i_n_f_i_n_i_t_e is larger than anything else. If one of the input numbers is _u_n_k_n_o_w_n then the result of the operation will be _u_n_k_n_o_w_n too. LLIIMMIITT Pops two elements from the stack and uses them to define a range. Then it pops another element and if it falls inside the range, it is pushed back. If not, an _u_n_k_n_o_w_n is pushed. The range defined includes the two boundaries (so: a number equal to one of the boundaries will be pushed back). If any of the three numbers involved is either _u_n_k_n_o_w_n or _i_n_f_i_n_i_t_e this function will always return an _u_n_k_n_o_w_n Example: "CDEF:a=alpha,0,100,LIMIT" will return _u_n_k_n_o_w_n if alpha is lower than 0 or if it is higher than 100. Arithmetics ++,, --,, **,, //,, %% Add, subtract, multiply, divide, modulo AADDDDNNAANN NAN-safe addition. If one parameter is NAN/UNKNOWN it'll be treated as zero. If both parameters are NAN/UNKNOWN, NAN/UNKNOWN will be returned. SSIINN,, CCOOSS,, LLOOGG,, EEXXPP,, SSQQRRTT Sine and cosine (input in radians), log and exp (natural logarithm), square root. AATTAANN Arctangent (output in radians). AATTAANN22 Arctangent of y,x components (output in radians). This pops one element from the stack, the x (cosine) component, and then a second, which is the y (sine) component. It then pushes the arctangent of their ratio, resolving the ambiguity between quadrants. Example: "CDEF:angle=Y,X,ATAN2,RAD2DEG" will convert "X,Y" components into an angle in degrees. FFLLOOOORR,, CCEEIILL Round down or up to the nearest integer. DDEEGG22RRAADD,, RRAADD22DDEEGG Convert angle in degrees to radians, or radians to degrees. AABBSS Take the absolute value. Set Operations SSOORRTT,, RREEVV Pop one element from the stack. This is the _c_o_u_n_t of items to be sorted (or reversed). The top _c_o_u_n_t of the remaining elements are then sorted (or reversed) in place on the stack. Example: "CDEF:x=v1,v2,v3,v4,v5,v6,6,SORT,POP,5,REV,POP,+,+,+,4,/" will compute the average of the values v1 to v6 after removing the smallest and largest. AAVVGG Pop one element (_c_o_u_n_t) from the stack. Now pop _c_o_u_n_t elements and build the average, ignoring all UNKNOWN values in the process. Example: "CDEF:x=a,b,c,d,4,AVG" TTRREENNDD,, TTRREENNDDNNAANN Create a "sliding window" average of another data series. Usage: CDEF:smoothed=x,1800,TREND This will create a half-hour (1800 second) sliding window average of x. The average is essentially computed as shown here: +---!---!---!---!---!---!---!---!---> now delay t0 <---------------> delay t1 <---------------> delay t2 <---------------> Value at sample (t0) will be the average between (t0-delay) and (t0) Value at sample (t1) will be the average between (t1-delay) and (t1) Value at sample (t2) will be the average between (t2-delay) and (t2) TRENDNAN is - in contrast to TREND - NAN-safe. If you use TREND and one source value is NAN the complete sliding window is affected. The TRENDNAN operation ignores all NAN-values in a sliding window and computes the average of the remaining values. PPRREEDDIICCTT,, PPRREEDDIICCTTSSIIGGMMAA Create a "sliding window" average/sigma of another data series, that also shifts the data series by given amounts of of time as well Usage - explicit stating shifts: CDEF:predict=,...,,n,,x,PREDICT CDEF:sigma=,...,,n,,x,PREDICTSIGMA Usage - shifts defined as a base shift and a number of time this is applied CDEF:predict=,-n,,x,PREDICT CDEF:sigma=,-n,,x,PREDICTSIGMA Example: CDEF:predict=172800,86400,2,1800,x,PREDICT This will create a half-hour (1800 second) sliding window average/sigma of x, that average is essentially computed as shown here: +---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!---!---> now shift 1 t0 <-----------------------> window <---------------> shift 2 <-----------------------------------------------> window <---------------> shift 1 t1 <-----------------------> window <---------------> shift 2 <-----------------------------------------------> window <---------------> Value at sample (t0) will be the average between (t0-shift1-window) and (t0-shift1) and between (t0-shift2-window) and (t0-shift2) Value at sample (t1) will be the average between (t1-shift1-window) and (t1-shift1) and between (t1-shift2-window) and (t1-shift2) The function is by design NAN-safe. This also allows for extrapolation into the future (say a few days) - you may need to define the data series whit the optional start= parameter, so that the source data series has enough data to provide prediction also at the beginning of a graph... Here an example, that will create a 10 day graph that also shows the prediction 3 days into the future with its uncertainty value (as defined by avg+-4*sigma) This also shows if the prediction is exceeded at a certain point. rrdtool graph image.png --imgformat=PNG \ --start=-7days --end=+3days --width=1000 --height=200 --alt-autoscale-max \ DEF:value=value.rrd:value:AVERAGE:start=-14days \ LINE1:value#ff0000:value \ CDEF:predict=86400,-7,1800,value,PREDICT \ CDEF:sigma=86400,-7,1800,value,PREDICTSIGMA \ CDEF:upper=predict,sigma,3,*,+ \ CDEF:lower=predict,sigma,3,*,- \ LINE1:predict#00ff00:prediction \ LINE1:upper#0000ff:upper\ certainty\ limit \ LINE1:lower#0000ff:lower\ certainty\ limit \ CDEF:exceeds=value,UN,0,value,lower,upper,LIMIT,UN,IF \ TICK:exceeds#aa000080:1 Note: Experience has shown that a factor between 3 and 5 to scale sigma is a good discriminator to detect abnormal behavior. This obviously depends also on the type of data and how "noisy" the data series is. This prediction can only be used for short term extrapolations - say a few days into the future- Special values UUNNKKNN Pushes an unknown value on the stack IINNFF,, NNEEGGIINNFF Pushes a positive or negative infinite value on the stack. When such a value is graphed, it appears at the top or bottom of the graph, no matter what the actual value on the y-axis is. PPRREEVV Pushes an _u_n_k_n_o_w_n value if this is the first value of a data set or otherwise the result of this CCDDEEFF at the previous time step. This allows you to do calculations across the data. This function cannot be used in VVDDEEFF instructions. PPRREEVV((vvnnaammee)) Pushes an _u_n_k_n_o_w_n value if this is the first value of a data set or otherwise the result of the vname variable at the previous time step. This allows you to do calculations across the data. This function cannot be used in VVDDEEFF instructions. CCOOUUNNTT Pushes the number 1 if this is the first value of the data set, the number 2 if it is the second, and so on. This special value allows you to make calculations based on the position of the value within the data set. This function cannot be used in VVDDEEFF instructions. Time Time inside RRDtool is measured in seconds since the epoch. The epoch is defined to be "Thu Jan 1 00:00:00 UTC 1970". NNOOWW Pushes the current time on the stack. TTIIMMEE Pushes the time the currently processed value was taken at onto the stack. LLTTIIMMEE Takes the time as defined by TTIIMMEE, applies the time zone offset valid at that time including daylight saving time if your OS supports it, and pushes the result on the stack. There is an elaborate example in the examples section below on how to use this. Processing the stack directly DDUUPP,, PPOOPP,, EEXXCC Duplicate the top element, remove the top element, exchange the two top elements. VVAARRIIAABBLLEESS These operators work only on VVDDEEFF statements. Note that currently ONLY these work for VVDDEEFF. MAXIMUM, MINIMUM, AVERAGE Return the corresponding value, MAXIMUM and MINIMUM also return the first occurrence of that value in the time component. Example: "VDEF:avg=mydata,AVERAGE" STDEV Returns the standard deviation of the values. Example: "VDEF:stdev=mydata,STDEV" LAST, FIRST Return the last/first non-nan or infinite value for the selected data stream, including its timestamp. Example: "VDEF:first=mydata,FIRST" TOTAL Returns the rate from each defined time slot multiplied with the step size. This can, for instance, return total bytes transferred when you have logged bytes per second. The time component returns the number of seconds. Example: "VDEF:total=mydata,TOTAL" PERCENT, PERCENTNAN This should follow a DDEEFF or CCDDEEFF _v_n_a_m_e. The _v_n_a_m_e is popped, another number is popped which is a certain percentage (0..100). The data set is then sorted and the value returned is chosen such that _p_e_r_c_e_n_t_a_g_e percent of the values is lower or equal than the result. For PERCENTNAN _U_n_k_n_o_w_n values are ignored, but for PERCENT _U_n_k_n_o_w_n values are considered lower than any finite number for this purpose so if this operator returns an _u_n_k_n_o_w_n you have quite a lot of them in your data. IInnffinite numbers are lesser, or more, than the finite numbers and are always more than the _U_n_k_n_o_w_n numbers. (NaN < -INF < finite values < INF) Example: "VDEF:perc95=mydata,95,PERCENT" "VDEF:percnan95=mydata,95,PERCENTNAN" LSLSLOPE, LSLINT, LSLCORREL Return the parameters for a LLeast SSquares LLine _(_y _= _m_x _+_b_) which approximate the provided dataset. LSLSLOPE is the slope _(_m_) of the line related to the COUNT position of the data. LSLINT is the y-intercept _(_b_), which happens also to be the first data point on the graph. LSLCORREL is the Correlation Coefficient (also know as Pearson's Product Moment Correlation Coefficient). It will range from 0 to +/-1 and represents the quality of fit for the approximation. Example: "VDEF:slope=mydata,LSLSLOPE" SSEEEE AALLSSOO rrdgraph gives an overview of how rrrrddttooooll ggrraapphh works. rrdgraph_data describes DDEEFF,CCDDEEFF and VVDDEEFF in detail. rrdgraph_rpn describes the RRPPNN language used in the ??DDEEFF statements. rrdgraph_graph page describes all of the graph and print functions. Make sure to read rrdgraph_examples for tips&tricks. AAUUTTHHOORR Program by Tobias Oetiker This manual page by Alex van den Bogaerdt with corrections and/or additions by several people 1.4.8 2013-05-23 RRDGRAPH_RPN(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdinfo (`third-party/rrdtool-1.4.8/doc/rrdinfo.txt`) RRDINFO(1) rrdtool RRDINFO(1) NNAAMMEE rrdinfo - extract header information from an RRD SSYYNNOOPPSSIISS rrrrddttooooll iinnffoo _f_i_l_e_n_a_m_e [----ddaaeemmoonn _a_d_d_r_e_s_s] DDEESSCCRRIIPPTTIIOONN The iinnffoo function prints the header information from an RRD in a parsing friendly format. Check rrdcreate if you are uncertain about the meaning of the individual keys. _f_i_l_e_n_a_m_e The name of the RRRRDD you want to examine. ----ddaaeemmoonn _a_d_d_r_e_s_s Address of the rrdcached daemon. If specified, a "flush" command is sent to the server before reading the RRD files. This allows rrrrddttooooll to return fresh data even if the daemon is configured to cache values for a long time. For a list of accepted formats, see the --ll option in the rrdcached manual. rrdtool info --daemon unix:/var/run/rrdcached.sock /var/lib/rrd/foo.rrd EEXXAAMMPPLLEE This is the output generated by running iinnffoo on a simple RRD which contains two data sources and one RRA. Note that the number after the _l_a_s_t___u_p_d_a_t_e keyword is in seconds since 1970. The string NNaaNN stands for _*_U_N_K_N_O_W_N_* data. In the example it means that this RRD has neither minimum nor maximum values defined for either of its data sources. filename = "random.rrd" rrd_version = "0001" step = 300 last_update = 955892996 header_size = 2872 ds[a].type = "GAUGE" ds[a].minimal_heartbeat = 600 ds[a].min = NaN ds[a].max = NaN ds[a].last_ds = "UNKN" ds[a].value = 2.1824421548e+04 ds[a].unknown_sec = 0 ds[b].type = "GAUGE" ds[b].minimal_heartbeat = 600 ds[b].min = NaN ds[b].max = NaN ds[b].last_ds = "UNKN" ds[b].value = 3.9620838224e+03 ds[b].unknown_sec = 0 rra[0].cf = "AVERAGE" rra[0].pdp_per_row = 1 rra[0].cdp_prep[0].value = nan rra[0].cdp_prep[0].unknown_datapoints = 0 rra[0].cdp_prep[1].value = nan rra[0].cdp_prep[1].unknown_datapoints = 0 EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS The following environment variables may be used to change the behavior of "rrdtool info": RRRRDDCCAACCHHEEDD__AADDDDRREESSSS If this environment variable is set it will have the same effect as specifying the "--daemon" option on the command line. If both are present, the command line argument takes precedence. AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDINFO(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdlast (`third-party/rrdtool-1.4.8/doc/rrdlast.txt`) RRDLAST(1) rrdtool RRDLAST(1) NNAAMMEE rrdlast - Return the date of the last data sample in an RRD SSYYNNOOPPSSIISS rrrrddttooooll llaasstt _f_i_l_e_n_a_m_e [----ddaaeemmoonn _a_d_d_r_e_s_s] DDEESSCCRRIIPPTTIIOONN The llaasstt function returns the UNIX timestamp of the most recent update of the RRD. _f_i_l_e_n_a_m_e The name of the RRRRDD that contains the data. ----ddaaeemmoonn _a_d_d_r_e_s_s Address of the rrdcached daemon. If specified, a "flush" command is sent to the server before reading the RRD files. This allows rrrrddttooooll to return fresh data even if the daemon is configured to cache values for a long time. For a list of accepted formats, see the --ll option in the rrdcached manual. rrdtool last --daemon unix:/var/run/rrdcached.sock /var/lib/rrd/foo.rrd EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS The following environment variables may be used to change the behavior of "rrdtool last": RRRRDDCCAACCHHEEDD__AADDDDRREESSSS If this environment variable is set it will have the same effect as specifying the "--daemon" option on the command line. If both are present, the command line argument takes precedence. AAUUTTHHOORR Russ Wright 1.4.8 2013-05-23 RRDLAST(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdlastupdate (`third-party/rrdtool-1.4.8/doc/rrdlastupdate.txt`) RRDLASTUPDATE(1) rrdtool RRDLASTUPDATE(1) NNAAMMEE rrdlastupdate - Return the most recent update to an RRD SSYYNNOOPPSSIISS rrrrddttooooll llaassttuuppddaattee _f_i_l_e_n_a_m_e [----ddaaeemmoonn _a_d_d_r_e_s_s] DDEESSCCRRIIPPTTIIOONN The llaassttuuppddaattee function returns the UNIX timestamp and the value stored for each datum in the most recent update of an RRD. _f_i_l_e_n_a_m_e The name of the RRRRDD that contains the data. ----ddaaeemmoonn _a_d_d_r_e_s_s Address of the rrdcached daemon. If specified, a "flush" command is sent to the server before reading the RRD files. This allows rrrrddttooooll to return fresh data even if the daemon is configured to cache values for a long time. For a list of accepted formats, see the --ll option in the rrdcached manual. rrdtool lastupdate --daemon unix:/var/run/rrdcached.sock /var/lib/rrd/foo.rrd EENNVVIIRROONNMMEENNTT VVAARRIIAABBLLEESS The following environment variables may be used to change the behavior of "rrdtool lastupdate": RRRRDDCCAACCHHEEDD__AADDDDRREESSSS If this environment variable is set it will have the same effect as specifying the "--daemon" option on the command line. If both are present, the command line argument takes precedence. AAUUTTHHOORR Andy Riebs 1.4.8 2013-05-23 RRDLASTUPDATE(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdresize (`third-party/rrdtool-1.4.8/doc/rrdresize.txt`) RRDRESIZE(1) rrdtool RRDRESIZE(1) NNAAMMEE rrdresize - alters the size of an RRA and creates a new .rrd file SSYYNNOOPPSSIISS rrrrddttooooll rreessiizzee _f_i_l_e_n_a_m_e _r_r_a_-_n_u_m GGRROOWW_|SSHHRRIINNKK _r_o_w_s DDEESSCCRRIIPPTTIIOONN The rreessiizzee function is used to modify the number of rows in an RRRRAA. _f_i_l_e_n_a_m_e the name of the RRRRDD you want to alter. _r_r_a_-_n_u_m the RRRRAA you want to alter. You can find the number using rrrrddttooooll iinnffoo. GGRROOWW used if you want to add extra rows to an RRA. The extra rows will be inserted as the rows that are oldest. SSHHRRIINNKK used if you want to remove rows from an RRA. The rows that will be removed are the oldest rows. _r_o_w_s the number of rows you want to add or remove. NNOOTTEESS The new .rrd file, with the modified RRAs, is written to the file rreessiizzee..rrrrdd in the current directory. TThhee oorriiggiinnaall ..rrrrdd ffiillee iiss nnoott mmooddiiffiieedd. It is possible to abuse this tool and get strange results by first removing some rows and then reinserting the same amount (effectively clearing them to be Unknown). You may thus end up with unknown data in one RRA while at the same timestamp this data is available in another RRA. AAUUTTHHOORR Alex van den Bogaerdt 1.4.8 2013-05-23 RRDRESIZE(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdrestore (`third-party/rrdtool-1.4.8/doc/rrdrestore.txt`) RRDRESTORE(1) rrdtool RRDRESTORE(1) NNAAMMEE rrdrestore - Restore the contents of an RRD from its XML dump format SSYYNNOOPPSSIISS rrrrddttooooll rreessttoorree _f_i_l_e_n_a_m_e_._x_m_l _f_i_l_e_n_a_m_e_._r_r_d [----rraannggee--cchheecckk|--rr] DDEESSCCRRIIPPTTIIOONN The rreessttoorree function reads the XML representation of an RRD and converts it to the native RRRRDD format. _f_i_l_e_n_a_m_e_._x_m_l The name of the XXMMLL file you want to restore. _f_i_l_e_n_a_m_e_._r_r_d The name of the RRRRDD to restore. ----rraannggee--cchheecckk|--rr Make sure the values in the RRAs do not exceed the limits defined for the various data sources. ----ffoorrccee--oovveerrwwrriittee|--ff Allows RRRRDDttooooll to overwrite the destination RRRRDD. AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDRESTORE(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdthreads (`third-party/rrdtool-1.4.8/doc/rrdthreads.txt`) RRDTHREADS(1) rrdtool RRDTHREADS(1) NNAAMMEE rrdthreads - Provisions for linking the RRD library to use in multi-threaded programs SSYYNNOOPPSSIISS Using librrd in multi-threaded programs requires some extra precautions, as the RRD library in its original form was not thread- safe at all. This document describes requirements and pitfalls on the way to use the multi-threaded version of librrd in your own programs. It also gives hints for future RRD development to keep the library thread-safe. Currently only some RRD operations are implemented in a thread-safe way. They all end in the usual ""_r"" suffix. DDEESSCCRRIIPPTTIIOONN In order to use librrd in multi-threaded programs you must: � Link with _l_i_b_r_r_d___t_h instead of _l_i_b_r_r_d (use "-lrrd_th" when linking) � Use the ""_r"" functions instead of the normal API-functions � Do not use any at-style time specifications. Parsing of such time specifications is terribly non-thread-safe. � Never use non *"_r" functions unless it is explicitly documented that the function is tread-safe. � Every thread SHOULD call "rrd_get_context()" before its first call to any "librrd_th" function in order to set up thread specific data. This is not strictly required, but it is the only way to test if memory allocation can be done by this function. Otherwise the program may die with a SIGSEGV in a low-memory situation. � Always call "rrd_error_clear()" before any call to the library. Otherwise the call might fail due to some earlier error. NNOOTTEESS FFOORR RRRRDD CCOONNTTRRIIBBUUTTOORRSS Some precautions must be followed when developing RRD from now on: � Only use thread-safe functions in library code. Many often used libc functions aren't thread-safe. Take care in the following situations or when using the following library functions: � Direct calls to "strerror()" must be avoided: use "rrd_strerror()" instead, it provides a per-thread error message. � The "getpw*", "getgr*", "gethost*" function families (and some more "get*" functions) are not thread-safe: use the *"_r" variants � Time functions: "asctime", "ctime", "gmtime", "localtime": use *"_r" variants � "strtok": use "strtok_r" � "tmpnam": use "tmpnam_r" � Many others (lookup documentation) � A header file named _r_r_d___i_s___t_h_r_e_a_d___s_a_f_e_._h is provided that works with the GNU C-preprocessor to "poison" some of the most common non-thread-safe functions using the "#pragma GCC poison" directive. Just include this header in source files you want to keep thread- safe. � Do not introduce global variables! If you really, really have to use a global variable you may add a new field to the "rrd_context" structure and modify _r_r_d___e_r_r_o_r_._c, _r_r_d___t_h_r_e_a_d___s_a_f_e_._c and _r_r_d___n_o_n___t_h_r_e_a_d___s_a_f_e_._c � Do not use "getopt" or "getopt_long" in *"_r" (neither directly nor indirectly). "getopt" uses global variables and behaves badly in a multi- threaded application when called concurrently. Instead provide a *_r function taking all options as function parameters. You may provide argc and **argv arguments for variable length argument lists. See "rrd_update_r" as an example. � Do not use the "rrd_parsetime" function! It uses lots of global variables. You may use it in functions not designed to be thread-safe, like in functions wrapping the "_r" version of some operation (e.g., "rrd_create", but not in "rrd_create_r") CCUURRRREENNTTLLYY IIMMPPLLEEMMEENNTTEEDD TTHHRREEAADD SSAAFFEE FFUUNNCCTTIIOONNSS Currently there exist thread-safe variants of "rrd_update", "rrd_create", "rrd_dump", "rrd_info", "rrd_last", and "rrd_fetch". AAUUTTHHOORR Peter Stamfest 1.4.8 2013-05-23 RRDTHREADS(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdtool (`third-party/rrdtool-1.4.8/doc/rrdtool.txt`) RRDTOOL(1) rrdtool RRDTOOL(1) NNAAMMEE rrdtool - Round Robin Database Tool SSYYNNOOPPSSIISS rrrrddttooooll -- [workdir]| _f_u_n_c_t_i_o_n DDEESSCCRRIIPPTTIIOONN OOVVEERRVVIIEEWW It is pretty easy to gather status information from all sorts of things, ranging from the temperature in your office to the number of octets which have passed through the FDDI interface of your router. But it is not so trivial to store this data in an efficient and systematic manner. This is where RRRRDDttooooll comes in handy. It lets you _l_o_g _a_n_d _a_n_a_l_y_z_e the data you gather from all kinds of data-sources (DDSS). The data analysis part of RRDtool is based on the ability to quickly generate graphical representations of the data values collected over a definable time period. In this man page you will find general information on the design and functionality of the Round Robin Database Tool (RRDtool). For a more detailed description of how to use the individual functions of RRRRDDttooooll check the corresponding man page. For an introduction to the usage of RRDtool make sure you consult the rrdtutorial. FFUUNNCCTTIIOONNSS While the man pages talk of command line switches you have to set in order to make RRRRDDttooooll work it is important to note that RRRRDDttooooll can be remotely controlled through a set of pipes. This saves a considerable amount of startup time when you plan to make RRRRDDttooooll do a lot of things quickly. Check the section on Remote_Control further down. There is also a number of language bindings for RRDtool which allow you to use it directly from Perl, python, Tcl, PHP, etc. ccrreeaattee Set up a new Round Robin Database (RRD). Check rrdcreate. uuppddaattee Store new data values into an RRD. Check rrdupdate. uuppddaatteevv Operationally equivalent to uuppddaattee except for output. Check rrdupdate. ggrraapphh Create a graph from data stored in one or several RRDs. Apart from generating graphs, data can also be extracted to stdout. Check rrdgraph. dduummpp Dump the contents of an RRD in plain ASCII. In connection with restore you can use this to move an RRD from one computer architecture to another. Check rrddump. rreessttoorree Restore an RRD in XML format to a binary RRD. Check rrdrestore ffeettcchh Get data for a certain time period from a RRD. The graph function uses fetch to retrieve its data from an RRD. Check rrdfetch. ttuunnee Alter setup of an RRD. Check rrdtune. llaasstt Find the last update time of an RRD. Check rrdlast. iinnffoo Get information about an RRD. Check rrdinfo. rrrrddrreessiizzee Change the size of individual RRAs. This is dangerous! Check rrdresize. xxppoorrtt Export data retrieved from one or several RRDs. Check rrdxport. fflluusshhccaacchheedd Flush the values for a specific RRD file from memory. Check rrdflushcached. rrrrddccggii This is a standalone tool for producing RRD graphs on the fly. Check rrdcgi. HHOOWW DDOOEESS RRRRDDTTOOOOLL WWOORRKK?? Data Acquisition When monitoring the state of a system, it is convenient to have the data available at a constant time interval. Unfortunately, you may not always be able to fetch data at exactly the time you want to. Therefore RRRRDDttooooll lets you update the log file at any time you want. It will automatically interpolate the value of the data-source (DDSS) at the latest official time-slot (interval) and write this interpolated value to the log. The original value you have supplied is stored as well and is also taken into account when interpolating the next log entry. Consolidation You may log data at a 1 minute interval, but you might also be interested to know the development of the data over the last year. You could do this by simply storing the data in 1 minute intervals for the whole year. While this would take considerable disk space it would also take a lot of time to analyze the data when you wanted to create a graph covering the whole year. RRRRDDttooooll offers a solution to this problem through its data consolidation feature. When setting up an Round Robin Database (RRRRDD), you can define at which interval this consolidation should occur, and what consolidation function (CCFF) (average, minimum, maximum, total, last) should be used to build the consolidated values (see rrdcreate). You can define any number of different consolidation setups within one RRRRDD. They will all be maintained on the fly when new data is loaded into the RRRRDD. Round Robin Archives Data values of the same consolidation setup are stored into Round Robin Archives (RRRRAA). This is a very efficient manner to store data for a certain amount of time, while using a known and constant amount of storage space. It works like this: If you want to store 1'000 values in 5 minute interval, RRRRDDttooooll will allocate space for 1'000 data values and a header area. In the header it will store a pointer telling which slots (value) in the storage area was last written to. New values are written to the Round Robin Archive in, you guessed it, a round robin manner. This automatically limits the history to the last 1'000 values (in our example). Because you can define several RRRRAAs within a single RRRRDD, you can setup another one, for storing 750 data values at a 2 hour interval, for example, and thus keep a log for the last two months at a lower resolution. The use of RRRRAAs guarantees that the RRRRDD does not grow over time and that old data is automatically eliminated. By using the consolidation feature, you can still keep data for a very long time, while gradually reducing the resolution of the data along the time axis. Using different consolidation functions (CCFF) allows you to store exactly the type of information that actually interests you: the maximum one minute traffic on the LAN, the minimum temperature of your wine cellar, the total minutes of down time, etc. Unknown Data As mentioned earlier, the RRRRDD stores data at a constant interval. Sometimes it may happen that no new data is available when a value has to be written to the RRRRDD. Data acquisition may not be possible for one reason or other. With RRRRDDttooooll you can handle these situations by storing an _*_U_N_K_N_O_W_N_* value into the database. The value '_*_U_N_K_N_O_W_N_*' is supported through all the functions of the tool. When consolidating a data set, the amount of _*_U_N_K_N_O_W_N_* data values is accounted for and when a new consolidated value is ready to be written to its Round Robin Archive (RRRRAA), a validity check is performed to make sure that the percentage of unknown values in the data point is above a configurable level. If not, an _*_U_N_K_N_O_W_N_* value will be written to the RRRRAA. Graphing RRRRDDttooooll allows you to generate reports in numerical and graphical form based on the data stored in one or several RRRRDDs. The graphing feature is fully configurable. Size, color and contents of the graph can be defined freely. Check rrdgraph for more information on this. Aberrant Behavior Detection by Jake Brutlag RRRRDDttooooll provides the building blocks for near real-time aberrant behavior detection. These components include: � An algorithm for predicting the value of a time series one time step into the future. � A measure of deviation between predicted and observed values. � A mechanism to decide if and when an observed value or sequence of observed values is _t_o_o _d_e_v_i_a_n_t from the predicted value(s). Here is a brief explanation of these components: The Holt-Winters time series forecasting algorithm is an on- line (or incremental) algorithm that adaptively predicts future observations in a time series. Its forecast is the sum of three components: a baseline (or intercept), a linear trend over time (or slope), and a seasonal coefficient (a periodic effect, such as a daily cycle). There is one seasonal coefficient for each time point in the period (cycle). After a value is observed, each of these components is updated via exponential smoothing. This means that the algorithm "learns" from past values and uses them to predict the future. The rate of adaptation is governed by 3 parameters, alpha (intercept), beta (slope), and gamma (seasonal). The prediction can also be viewed as a smoothed value for the time series. The measure of deviation is a seasonal weighted absolute deviation. The term _s_e_a_s_o_n_a_l means deviation is measured separately for each time point in the seasonal cycle. As with Holt-Winters forecasting, deviation is predicted using the measure computed from past values (but only at that point in the seasonal cycle). After the value is observed, the algorithm learns from the observed value via exponential smoothing. Confidence bands for the observed time series are generated by scaling the sequence of predicted deviation values (we usually think of the sequence as a continuous line rather than a set of discrete points). Aberrant behavior (a potential failure) is reported whenever the number of times the observed value violates the confidence bands meets or exceeds a specified threshold within a specified temporal window (e.g. 5 violations during the past 45 minutes with a value observed every 5 minutes). This functionality is embedded in a set of related RRRRAAss. In particular, a FAILURES RRRRAA logs potential failures. With these data you could, for example, use a front-end application to RRRRDDttooooll to initiate real-time alerts. For a detailed description on how to set this up, see rrdcreate. RREEMMOOTTEE CCOONNTTRROOLL When you start RRRRDDttooooll with the command line option '--' it waits for input via standard input (STDIN). With this feature you can improve performance by attaching RRRRDDttooooll to another process (MRTG is one example) through a set of pipes. Over these pipes RRRRDDttooooll accepts the same arguments as on the command line and some special commands like qquuiitt,, ccdd,, mmkkddiirr and llss. For detailed help on the server commands type: rrdtool help cd|mkdir|pwd|ls|quit When a command is completed, RRDtool will print the string '"OK"', followed by timing information of the form uu::_u_s_e_r_t_i_m_e ss::_s_y_s_t_e_m_t_i_m_e. Both values are the running totals of seconds since RRDtool was started. If an error occurs, a line of the form '"ERROR:" _D_e_s_c_r_i_p_t_i_o_n _o_f _e_r_r_o_r' will be printed instead. RRRRDDttooooll will not abort, unless something really serious happens. If a wwoorrkkddiirr is specified and the UID is 0, RRDtool will do a chroot to that workdir. If the UID is not 0, RRDtool only changes the current directory to wwoorrkkddiirr. RRRRDD SSeerrvveerr If you want to create a RRD-Server, you must choose a TCP/IP Service number and add them to _/_e_t_c_/_s_e_r_v_i_c_e_s like this: rrdsrv 13900/tcp # RRD server Attention: the TCP port 13900 isn't officially registered for rrdsrv. You can use any unused port in your services file, but the server and the client system must use the same port, of course. With this configuration you can add RRDtool as meta-server to _/_e_t_c_/_i_n_e_t_d_._c_o_n_f. For example: rrdsrv stream tcp nowait root /opt/rrd/bin/rrdtool rrdtool - /var/rrd Don't forget to create the database directory /var/rrd and reinitialize your inetd. If all was setup correctly, you can access the server with Perl sockets, tools like netcat, or in a quick interactive test by using 'telnet localhost rrdsrv'. NNOOTTEE:: that there is no authentication with this feature! Do not setup such a port unless you are sure what you are doing. RRRRDDCCAACCHHEEDD,, TTHHEE CCAACCHHIINNGG DDAAEEMMOONN For very big setups, updating thousands of RRD files often becomes a serious IO problem. If you run into such problems, you might want to take a look at rrdcached, a caching daemon for RRDtool which may help you lessen the stress on your disks. SSEEEE AALLSSOO rrdcreate, rrdupdate, rrdgraph, rrddump, rrdfetch, rrdtune, rrdlast, rrdxport, rrdflushcached, rrdcached BBUUGGSS Bugs? Features! AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDTOOL(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdtune (`third-party/rrdtool-1.4.8/doc/rrdtune.txt`) RRDTUNE(1) rrdtool RRDTUNE(1) NNAAMMEE rrdtune - Modify some basic properties of a Round Robin Database SSYYNNOOPPSSIISS rrrrddttooooll ttuunnee _f_i_l_e_n_a_m_e [----hheeaarrttbbeeaatt|--hh _d_s_-_n_a_m_e:_h_e_a_r_t_b_e_a_t] [----mmiinniimmuumm|--ii _d_s_-_n_a_m_e:_m_i_n] [----mmaaxxiimmuumm|--aa _d_s_-_n_a_m_e:_m_a_x] [----ddaattaa--ssoouurrccee--ttyyppee|--dd _d_s_-_n_a_m_e:_D_S_T] [----ddaattaa--ssoouurrccee--rreennaammee|--rr _o_l_d_- _n_a_m_e:_n_e_w_-_n_a_m_e] [----ddeellttaappooss _s_c_a_l_e_-_v_a_l_u_e] [----ddeellttaanneegg _s_c_a_l_e_-_v_a_l_u_e] [----ffaaiilluurree--tthhrreesshhoolldd _f_a_i_l_u_r_e_-_t_h_r_e_s_h_o_l_d] [----wwiinnddooww--lleennggtthh _w_i_n_d_o_w_-_l_e_n_g_t_h] [----aallpphhaa _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r] [----bbeettaa _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r] [----ggaammmmaa _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r] [----ggaammmmaa--ddeevviiaattiioonn _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r] [----ssmmooootthhiinngg--wwiinnddooww _f_r_a_c_t_i_o_n_-_o_f_-_s_e_a_s_o_n] [----ssmmooootthhiinngg--wwiinnddooww--ddeevviiaattiioonn _f_r_a_c_t_i_o_n_-_o_f_-_s_e_a_s_o_n] [----aabbeerrrraanntt--rreesseett _d_s_- _n_a_m_e] DDEESSCCRRIIPPTTIIOONN The tune option allows you to alter some of the basic configuration values stored in the header area of a Round Robin Database (RRRRDD). One application of the ttuunnee function is to relax the validation rules on an RRRRDD. This allows to fill a new RRRRDD with data available in larger intervals than what you would normally want to permit. Be very careful with tune operations for COMPUTE data sources. Setting the _m_i_n, _m_a_x, and _h_e_a_r_t_b_e_a_t for a COMPUTE data source without changing the data source type to a non-COMPUTE DDSSTT WILL corrupt the data source header in the RRRRDD. A second application of the ttuunnee function is to set or alter parameters used by the specialized function RRRRAAss for aberrant behavior detection. _f_i_l_e_n_a_m_e The name of the RRRRDD you want to tune. ----hheeaarrttbbeeaatt|--hh _d_s_-_n_a_m_e:_h_e_a_r_t_b_e_a_t modify the _h_e_a_r_t_b_e_a_t of a data source. By setting this to a high value the RRD will accept things like one value per day. ----mmiinniimmuumm|--ii _d_s_-_n_a_m_e:_m_i_n alter the minimum value acceptable as input from the data source. Setting _m_i_n to 'U' will disable this limit. ----mmaaxxiimmuumm|--aa _d_s_-_n_a_m_e:_m_a_x alter the maximum value acceptable as input from the data source. Setting _m_a_x to 'U' will disable this limit. ----ddaattaa--ssoouurrccee--ttyyppee|--dd _d_s_-_n_a_m_e:_D_S_T alter the type DDSSTT of a data source. ----ddaattaa--ssoouurrccee--rreennaammee|--rr _o_l_d_-_n_a_m_e:_n_e_w_-_n_a_m_e rename a data source. ----ddeellttaappooss _s_c_a_l_e_-_v_a_l_u_e Alter the deviation scaling factor for the upper bound of the confidence band used internally to calculate violations for the FAILURES RRRRAA. The default value is 2. Note that this parameter is not related to graphing confidence bounds which must be specified as a CDEF argument to generate a graph with confidence bounds. The graph scale factor need not to agree with the value used internally by the FAILURES RRRRAA. ----ddeellttaanneegg _s_c_a_l_e_-_v_a_l_u_e Alter the deviation scaling factor for the lower bound of the confidence band used internally to calculate violations for the FAILURES RRRRAA. The default value is 2. As with ----ddeellttaappooss, this argument is unrelated to the scale factor chosen when graphing confidence bounds. ----ffaaiilluurree--tthhrreesshhoolldd _f_a_i_l_u_r_e_-_t_h_r_e_s_h_o_l_d Alter the number of confidence bound violations that constitute a failure for purposes of the FAILURES RRRRAA. This must be an integer less than or equal to the window length of the FAILURES RRRRAA. This restriction is not verified by the tune option, so one can reset failure-threshold and window-length simultaneously. Setting this option will reset the count of violations to 0. ----wwiinnddooww--lleennggtthh _w_i_n_d_o_w_-_l_e_n_g_t_h Alter the number of time points in the temporal window for determining failures. This must be an integer greater than or equal to the window length of the FAILURES RRRRAA and less than or equal to 28. Setting this option will reset the count of violations to 0. ----aallpphhaa _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r Alter the intercept adaptation parameter for the Holt-Winters forecasting algorithm. This parameter must be between 0 and 1. ----bbeettaa _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r Alter the slope adaptation parameter for the Holt-Winters forecasting algorithm. This parameter must be between 0 and 1. ----ggaammmmaa _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r Alter the seasonal coefficient adaptation parameter for the SEASONAL RRRRAA. This parameter must be between 0 and 1. ----ggaammmmaa--ddeevviiaattiioonn _a_d_a_p_t_i_o_n_-_p_a_r_a_m_e_t_e_r Alter the seasonal deviation adaptation parameter for the DEVSEASONAL RRRRAA. This parameter must be between 0 and 1. ----ssmmooootthhiinngg--wwiinnddooww _f_r_a_c_t_i_o_n_-_o_f_-_s_e_a_s_o_n Alter the size of the smoothing window for the SEASONAL RRRRAA. This must be between 0 and 1. ----ssmmooootthhiinngg--wwiinnddooww--ddeevviiaattiioonn _f_r_a_c_t_i_o_n_-_o_f_-_s_e_a_s_o_n Alter the size of the smoothing window for the DEVSEASONAL RRRRAA. This must be between 0 and 1. ----aabbeerrrraanntt--rreesseett _d_s_-_n_a_m_e This option causes the aberrant behavior detection algorithm to reset for the specified data source; that is, forget all it is has learnt so far. Specifically, for the HWPREDICT or MHWPREDICT RRRRAA, it sets the intercept and slope coefficients to unknown. For the SEASONAL RRRRAA, it sets all seasonal coefficients to unknown. For the DEVSEASONAL RRRRAA, it sets all seasonal deviation coefficients to unknown. For the FAILURES RRRRAA, it erases the violation history. Note that reset does not erase past predictions (the values of the HWPREDICT or MHWPREDICT RRRRAA), predicted deviations (the values of the DEVPREDICT RRRRAA), or failure history (the values of the FAILURES RRRRAA). This option will function even if not all the listed RRRRAAss are present. Due to the implementation of this option, there is an indirect impact on other data sources in the RRD. A smoothing algorithm is applied to SEASONAL and DEVSEASONAL values on a periodic basis. During bootstrap initialization this smoothing is deferred. For efficiency, the implementation of smoothing is not data source specific. This means that utilizing reset for one data source will delay running the smoothing algorithm for all data sources in the file. This is unlikely to have serious consequences, unless the data being collected for the non-reset data sources is unusually volatile during the reinitialization period of the reset data source. Use of this tuning option is advised when the behavior of the data source time series changes in a drastic and permanent manner. EEXXAAMMPPLLEE 11 "rrdtool tune data.rrd -h in:100000 -h out:100000 -h through:100000" Set the minimum required heartbeat for data sources 'in', 'out' and 'through' to 10'000 seconds which is a little over one day in data.rrd. This would allow to feed old data from MRTG-2.0 right into RRDtool without generating *UNKNOWN* entries. EEXXAAMMPPLLEE 22 "rrdtool tune monitor.rrd --window-length 5 --failure-threshold 3" If the FAILURES RRRRAA is implicitly created, the default window-length is 9 and the default failure-threshold is 7. This command now defines a failure as 3 or more violations in a temporal window of 5 time points. AAUUTTHHOORR Tobias Oetiker 1.4.8 2013-05-23 RRDTUNE(1) --- ### Third Party Rrdtool 1.4.8 Doc Rrdtutorial (`third-party/rrdtool-1.4.8/doc/rrdtutorial.txt`) RRDTUTORIAL(1) rrdtool RRDTUTORIAL(1) NNAAMMEE rrdtutorial - Alex van den Bogaerdt's RRDtool tutorial DDEESSCCRRIIPPTTIIOONN RRDtool is written by Tobias Oetiker with contributions from many people all around the world. This document is written by Alex van den Bogaerdt to help you understand what RRDtool is and what it can do for you. The documentation provided with RRDtool can be too technical for some people. This tutorial is here to help you understand the basics of RRDtool. It should prepare you to read the documentation yourself. It also explains the general things about statistics with a focus on networking. TTUUTTOORRIIAALL IImmppoorrttaanntt Please don't skip ahead in this document! The first part of this document explains the basics and may be boring. But if you don't understand the basics, the examples will not be as meaningful to you. Sometimes things change. This example used to provide numbers like "0.04" in stead of "4.00000e-02". Those are really the same numbers, just written down differently. Don't be alarmed if a future version of rrdtool displays a slightly different form of output. The examples in this document are correct for version 1.2.0 of RRDtool. Also, sometimes bugs do occur. They may also influence the outcome of the examples. Example speed4.png was suffering from this (the handling of unknown data in an if-statement was wrong). Normal data will be just fine (a bug in rrdtool wouldn't last long) but special cases like NaN, INF and so on may last a bit longer. Try another version if you can, or just live with it. I fixed the speed4.png example (and added a note). There may be other examples which suffer from the same or a similar bug. Try to fix it yourself, which is a great excercise. But please do not submit your result as a fix to the source of this document. Discuss it on the user's list, or write to me. WWhhaatt iiss RRRRDDttooooll?? RRDtool refers to Round Robin Database tool. Round robin is a technique that works with a fixed amount of data, and a pointer to the current element. Think of a circle with some dots plotted on the edge. These dots are the places where data can be stored. Draw an arrow from the center of the circle to one of the dots; this is the pointer. When the current data is read or written, the pointer moves to the next element. As we are on a circle there is neither a beginning nor an end, you can go on and on and on. After a while, all the available places will be used and the process automatically reuses old locations. This way, the dataset will not grow in size and therefore requires no maintenance. RRDtool works with Round Robin Databases (RRDs). It stores and retrieves data from them. WWhhaatt ddaattaa ccaann bbee ppuutt iinnttoo aann RRRRDD?? You name it, it will probably fit as long as it is some sort of time- series data. This means you have to be able to measure some value at several points in time and provide this information to RRDtool. If you can do this, RRDtool will be able to store it. The values must be numerical but don't have to be integers, as is the case with MRTG (the next section will give more details on this more specialized application). Many examples below talk about SNMP which is an acronym for Simple Network Management Protocol. "Simple" refers to the protocol. It does not mean it is simple to manage or monitor a network. After working your way through this document, you should know enough to be able to understand what people are talking about. For now, just realize that SNMP can be used to query devices for the values of counters they keep. It is the value from those counters that we want to store in the RRD. WWhhaatt ccaann II ddoo wwiitthh tthhiiss ttooooll?? RRDtool originated from MRTG (Multi Router Traffic Grapher). MRTG started as a tiny little script for graphing the use of a university's connection to the Internet. MRTG was later (ab-)used as a tool for graphing other data sources including temperature, speed, voltage, number of printouts and the like. Most likely you will start to use RRDtool to store and process data collected via SNMP. The data will most likely be bytes (or bits) transferred from and to a network or a computer. But it can also be used to display tidal waves, solar radiation, power consumption, number of visitors at an exhibition, noise levels near an airport, temperature on your favorite holiday location, temperature in the fridge and whatever your imagination can come up with. You only need a sensor to measure the data and be able to feed the numbers into RRDtool. RRDtool then lets you create a database, store data in it, retrieve that data and create graphs in PNG format for display on a web browser. Those PNG images are dependent on the data you collected and could be, for instance, an overview of the average network usage, or the peaks that occurred. WWhhaatt iiff II ssttiillll hhaavvee pprroobblleemmss aafftteerr rreeaaddiinngg tthhiiss ddooccuummeenntt?? First of all: read it again! You may have missed something. If you are unable to compile the sources and you have a fairly common OS, it will probably not be the fault of RRDtool. There may be pre-compiled versions around on the Internet. If they come from trusted sources, get one of those. If on the other hand the program works but does not give you the expected results, it will be a problem with configuring it. Review your configuration and compare it with the examples that follow. There is a mailing list and an archive of it. Read the list for a few weeks and search the archive. It is considered rude to just ask a question without searching the archives: your problem may already have been solved for somebody else! This is true for most, if not all, mailing lists and not only for this particular one. Look in the documentation that came with RRDtool for the location and usage of the list. I suggest you take a moment to subscribe to the mailing list right now by sending an email to with a subject of "subscribe". If you ever want to leave this list, just write an email to the same address but now with a subject of "unsubscribe". HHooww wwiillll yyoouu hheellpp mmee?? By giving you some detailed descriptions with detailed examples. I assume that following the instructions in the order presented will give you enough knowledge of RRDtool to experiment for yourself. If it doesn't work the first time, don't give up. Reread the stuff that you did understand, you may have missed something. By following the examples you get some hands-on experience and, even more important, some background information of how it works. You will need to know something about hexadecimal numbers. If you don't then start with reading bin_dec_hex before you continue here. YYoouurr ffiirrsstt RRoouunndd RRoobbiinn DDaattaabbaassee In my opinion the best way to learn something is to actually do it. Why not start right now? We will create a database, put some values in it and extract this data again. Your output should be the same as the output that is included in this document. We will start with some easy stuff and compare a car with a router, or compare kilometers (miles if you wish) with bits and bytes. It's all the same: some number over some time. Assume we have a device that transfers bytes to and from the Internet. This device keeps a counter that starts at zero when it is turned on, increasing with every byte that is transferred. This counter will probably have a maximum value. If this value is reached and an extra byte is counted, the counter starts over at zero. This is the same as many counters in the world such as the mileage counter in a car. Most discussions about networking talk about bits per second so lets get used to that right away. Assume a byte is eight bits and start to think in bits not bytes. The counter, however, still counts bytes! In the SNMP world most of the counters are 32 bits. That means they are counting from 0 to 4294967295. We will use these values in the examples. The device, when asked, returns the current value of the counter. We know the time that has passes since we last asked so we now know how many bytes have been transferred ***on average*** per second. This is not very hard to calculate. First in words, then in calculations: 1. Take the current counter, subtract the previous value from it. 2. Do the same with the current time and the previous time (in seconds). 3. Divide the outcome of (1) by the outcome of (2), the result is the amount of bytes per second. Multiply by eight to get the number of bits per second (bps). bps = (counter_now - counter_before) / (time_now - time_before) * 8 For some people it may help to translate this to an automobile example. Do not try this example, and if you do, don't blame me for the results! People who are not used to think in kilometers per hour can translate most into miles per hour by dividing km by 1.6 (close enough). I will use the following abbreviations: m: meter km: kilometer (= 1000 meters). h: hour s: second km/h: kilometers per hour m/s: meters per second You are driving a car. At 12:05 you read the counter in the dashboard and it tells you that the car has moved 12345 km until that moment. At 12:10 you look again, it reads 12357 km. This means you have traveled 12 km in five minutes. A scientist would translate that into meters per second and this makes a nice comparison toward the problem of (bytes per five minutes) versus (bits per second). We traveled 12 kilometers which is 12000 meters. We did that in five minutes or 300 seconds. Our speed is 12000m / 300s or 40 m/s. We could also calculate the speed in km/h: 12 times 5 minutes is an hour, so we have to multiply 12 km by 12 to get 144 km/h. For our native English speaking friends: that's 90 mph so don't try this example at home or where I live :) Remember: these numbers are averages only. There is no way to figure out from the numbers, if you drove at a constant speed. There is an example later on in this tutorial that explains this. I hope you understand that there is no difference in calculating m/s or bps; only the way we collect the data is different. Even the k from kilo is the same as in networking terms k also means 1000. We will now create a database where we can keep all these interesting numbers. The method used to start the program may differ slightly from OS to OS, but I assume you can figure it out if it works different on your's. Make sure you do not overwrite any file on your system when executing the following command and type the whole line as one long line (I had to split it for readability) and skip all of the '\' characters. rrdtool create test.rrd \ --start 920804400 \ DS:speed:COUNTER:600:U:U \ RRA:AVERAGE:0.5:1:24 \ RRA:AVERAGE:0.5:6:10 (So enter: "rrdtool create test.rrd --start 920804400 DS ...") WWhhaatt hhaass bbeeeenn ccrreeaatteedd?? We created the round robin database called test (test.rrd) which starts at noon the day I started writing this document, 7th of March, 1999 (this date translates to 920804400 seconds as explained below). Our database holds one data source (DS) named "speed" that represents a counter. This counter is read every five minutes (this is the default therefore you don't have to put "--step=300"). In the same database two round robin archives (RRAs) are kept, one averages the data every time it is read (e.g., there's nothing to average) and keeps 24 samples (24 times 5 minutes is 2 hours). The other averages 6 values (half hour) and contains 10 such averages (e.g. 5 hours). RRDtool works with special time stamps coming from the UNIX world. This time stamp is the number of seconds that passed since January 1st 1970 UTC. The time stamp value is translated into local time and it will therefore look different for different time zones. Chances are that you are not in the same part of the world as I am. This means your time zone is different. In all examples where I talk about time, the hours may be wrong for you. This has little effect on the results of the examples, just correct the hours while reading. As an example: where I will see "12:05" the UK folks will see "11:05". We now have to fill our database with some numbers. We'll pretend to have read the following numbers: 12:05 12345 km 12:10 12357 km 12:15 12363 km 12:20 12363 km 12:25 12363 km 12:30 12373 km 12:35 12383 km 12:40 12393 km 12:45 12399 km 12:50 12405 km 12:55 12411 km 13:00 12415 km 13:05 12420 km 13:10 12422 km 13:15 12423 km We fill the database as follows: rrdtool update test.rrd 920804700:12345 920805000:12357 920805300:12363 rrdtool update test.rrd 920805600:12363 920805900:12363 920806200:12373 rrdtool update test.rrd 920806500:12383 920806800:12393 920807100:12399 rrdtool update test.rrd 920807400:12405 920807700:12411 920808000:12415 rrdtool update test.rrd 920808300:12420 920808600:12422 920808900:12423 This reads: update our test database with the following numbers time 920804700, value 12345 time 920805000, value 12357 etcetera. As you can see, it is possible to feed more than one value into the database in one command. I had to stop at three for readability but the real maximum per line is OS dependent. We can now retrieve the data from our database using "rrdtool fetch": rrdtool fetch test.rrd AVERAGE --start 920804400 --end 920809200 It should return the following output: speed 920804700: nan 920805000: 4.0000000000e-02 920805300: 2.0000000000e-02 920805600: 0.0000000000e+00 920805900: 0.0000000000e+00 920806200: 3.3333333333e-02 920806500: 3.3333333333e-02 920806800: 3.3333333333e-02 920807100: 2.0000000000e-02 920807400: 2.0000000000e-02 920807700: 2.0000000000e-02 920808000: 1.3333333333e-02 920808300: 1.6666666667e-02 920808600: 6.6666666667e-03 920808900: 3.3333333333e-03 920809200: nan 920809500: nan Note that you might get more rows than you expect. The reason for this is that you ask for a time range that ends on 920809200. The number that is written behind 920809200: in the list above covers the time range from 920808900 to 920809200, EXCLUDING 920809200. Hence to be on the sure side, you receive the entry from 920809200 to 920809500 as well since it INCLUDES 920809200. You may also see "NaN" instead of "nan" this is OS dependent. "NaN" stands for "Not A Number". If your OS writes "U" or "UNKN" or something similar that's okay. If something else is wrong, it will probably be due to an error you made (assuming that my tutorial is correct of course :-). In that case: delete the database and try again. The meaning of the above output will become clear below. TTiimmee ttoo ccrreeaattee ssoommee ggrraapphhiiccss Try the following command: rrdtool graph speed.png \ --start 920804400 --end 920808000 \ DEF:myspeed=test.rrd:speed:AVERAGE \ LINE2:myspeed#FF0000 This will create speed.png which starts at 12:00 and ends at 13:00. There is a definition of a variable called myspeed, using the data from RRA "speed" out of database "test.rrd". The line drawn is 2 pixels high and represents the variable myspeed. The color is red (specified by its rgb-representation, see below). You'll notice that the start of the graph is not at 12:00 but at 12:05. This is because we have insufficient data to tell the average before that time. This will only happen when you miss some samples, this will not happen a lot, hopefully. If this has worked: congratulations! If not, check what went wrong. The colors are built up from red, green and blue. For each of the components, you specify how much to use in hexadecimal where 00 means not included and FF means fully included. The "color" white is a mixture of red, green and blue: FFFFFF The "color" black is all colors off: 000000 red #FF0000 green #00FF00 blue #0000FF magenta #FF00FF (mixed red with blue) gray #555555 (one third of all components) Additionally you can (with a recent RRDtool) add an alpha channel (transparency). The default will be "FF" which means non-transparent. The PNG you just created can be displayed using your favorite image viewer. Web browsers will display the PNG via the URL "file:///the/path/to/speed.png" GGrraapphhiiccss wwiitthh ssoommee mmaatthh When looking at the image, you notice that the horizontal axis is labeled 12:10, 12:20, 12:30, 12:40 and 12:50. Sometimes a label doesn't fit (12:00 and 13:00 would be likely candidates) so they are skipped. The vertical axis displays the range we entered. We provided kilometers and when divided by 300 seconds, we get very small numbers. To be exact, the first value was 12 (12357-12345) and divided by 300 this makes 0.04, which is displayed by RRDtool as "40 m" meaning "40/1000". The "m" (milli) has nothing to do with meters (also m), kilometers or millimeters! RRDtool doesn't know about the physical units of our data, it just works with dimensionless numbers. If we had measured our distances in meters, this would have been (12357000-12345000)/300 = 12000/300 = 40. As most people have a better feel for numbers in this range, we'll correct that. We could recreate our database and store the correct data, but there is a better way: we do some calculations while creating the png file! rrdtool graph speed2.png \ --start 920804400 --end 920808000 \ --vertical-label m/s \ DEF:myspeed=test.rrd:speed:AVERAGE \ CDEF:realspeed=myspeed,1000,\* \ LINE2:realspeed#FF0000 Note: I need to escape the multiplication operator * with a backslash. If I don't, the operating system may interpret it and use it for file name expansion. You could also place the line within quotation marks like so: "CDEF:realspeed=myspeed,1000,*" \ It boils down to: it is RRDtool which should see *, not your shell. And it is your shell interpreting \, not RRDtool. You may need to adjust examples accordingly if you happen to use an operating system or shell which behaves differently. After viewing this PNG, you notice the "m" (milli) has disappeared. This it what the correct result would be. Also, a label has been added to the image. Apart from the things mentioned above, the PNG should look the same. The calculations are specified in the CDEF part above and are in Reverse Polish Notation ("RPN"). What we requested RRDtool to do is: "take the data source myspeed and the number 1000; multiply those". Don't bother with RPN yet, it will be explained later on in more detail. Also, you may want to read my tutorial on CDEFs and Steve Rader's tutorial on RPN. But first finish this tutorial. Hang on! If we can multiply values with 1000, it should also be possible to display kilometers per hour from the same data! To change a value that is measured in meters per second: Calculate meters per hour: value * 3600 Calculate kilometers per hour: value / 1000 Together this makes: value * (3600/1000) or value * 3.6 In our example database we made a mistake and we need to compensate for this by multiplying with 1000. Applying that correction: value * 3.6 * 1000 == value * 3600 Now let's create this PNG, and add some more magic ... rrdtool graph speed3.png \ --start 920804400 --end 920808000 \ --vertical-label km/h \ DEF:myspeed=test.rrd:speed:AVERAGE \ "CDEF:kmh=myspeed,3600,*" \ CDEF:fast=kmh,100,GT,kmh,0,IF \ CDEF:good=kmh,100,GT,0,kmh,IF \ HRULE:100#0000FF:"Maximum allowed" \ AREA:good#00FF00:"Good speed" \ AREA:fast#FF0000:"Too fast" Note: here we use another means to escape the * operator by enclosing the whole string in double quotes. This graph looks much better. Speed is shown in km/h and there is even an extra line with the maximum allowed speed (on the road I travel on). I also changed the colors used to display speed and changed it from a line into an area. The calculations are more complex now. For speed measurements within the speed limit they are: Check if kmh is greater than 100 ( kmh,100 ) GT If so, return 0, else kmh ((( kmh,100 ) GT ), 0, kmh) IF For values above the speed limit: Check if kmh is greater than 100 ( kmh,100 ) GT If so, return kmh, else return 0 ((( kmh,100) GT ), kmh, 0) IF GGrraapphhiiccss MMaaggiicc I like to believe there are virtually no limits to how RRDtool graph can manipulate data. I will not explain how it works, but look at the following PNG: rrdtool graph speed4.png \ --start 920804400 --end 920808000 \ --vertical-label km/h \ DEF:myspeed=test.rrd:speed:AVERAGE \ CDEF:nonans=myspeed,UN,0,myspeed,IF \ CDEF:kmh=nonans,3600,* \ CDEF:fast=kmh,100,GT,100,0,IF \ CDEF:over=kmh,100,GT,kmh,100,-,0,IF \ CDEF:good=kmh,100,GT,0,kmh,IF \ HRULE:100#0000FF:"Maximum allowed" \ AREA:good#00FF00:"Good speed" \ AREA:fast#550000:"Too fast" \ STACK:over#FF0000:"Over speed" Remember the note in the beginning? I had to remove unknown data from this example. The 'nonans' CDEF is new, and the 6th line (which used to be the 5th line) used to read 'CDEF:kmh=myspeed,3600,*' Let's create a quick and dirty HTML page to view the three PNGs: Speed Speed in meters per second
Speed in kilometers per hour
Traveled too fast? Name the file "speed.html" or similar, and look at it in your web browser. Now, all you have to do is measure the values regularly and update the database. When you want to view the data, recreate the PNGs and make sure to refresh them in your browser. (Note: just clicking reload may not be enough, especially when proxies are involved. Try shift-reload or ctrl-F5). UUppddaatteess iinn RReeaalliittyy We've already used the "update" command: it took one or more parameters in the form of "