Testy kompatibility meracej platformy MONICA

Maji

link na oficialnu stranku projektu Wand Maji

Maji is an implementation of an IPFIX meter, developed by the WAND Research Group at the University of Waikato, New Zealand. Maji has been developed in close consultation with the IPFIX draft specifications (some of which are now published RFCs) but it is not a feature-complete implementation at this stage. Maji does not provide any form of useful IPFIX collector, nor does it come with any tools for processing the IPFIX records produced by the meter.

- v slovniku Monici je Maji exporterom

Instalacia

Potrebne balicky:

• libtrace 3.0.4 alebo novsi
• libconfuse (pripadne libconfuse-dev)
• libsqlite3 (pripadne libsqlite3-dev) - volitelne, ale vyzadovane pri SQL exporte
• libsctp (pripadne libsctp-dev) - volitelne, vyzadovane pri SCTP exporte
• libwandevent 2.0.0

1.) instalacia libtrace

apt-get install libtrace3 libtrace3-dev libtrace-tools

2.) instalacia libconfuse a libwandevent

• stiahnut balicky z http://savannah.nongnu.org/download/confuse/confuse-2.7.tar.gz a http://research.wand.net.nz/software/libwandevent/libwandevent-2.0.0.tar.gz
• v oboch pripadoch rozbalit a:

./configure
make
make install

• balicky sa nainstalovali do /usr/local/lib preto treba zadat

echo /usr/local/lib >> /etc/ld.so.conf
ldconfig

3.) instalacia libsqlite 3 a libsctp

sudo apt-get install libsctp1 libsqlite3-0 libsqlite0

4.) instalacia maji

• stiahnut z http://research.wand.net.nz/software/maji/maji-1.0.1.tar.gz
• rozbalit a:

./configure
make
make install 

Konfiguracia Maji

Priklad konfiguracneho suboru example.config je sucastou dodavky maji:

# Simple template for measuring packets and bytes using the 5-tuple as the
# flow key
template
{
keys = {"sourceIPv4Address", "destinationipv4address",
"protocolIdentifier",
"sourceTransportPort", "destinationTransportPort"}
elements = {"packettotalcount", "octettotalcount" }  
}

# Export expired flows and option template records every 60 seconds
report_freq = 60
# This instance of maji will have an observation domain id of 1
domain_id = 1 
# Location of the installed libwandipfix information element modules
module_location = {"/usr/local/maji/info_elements" }


# Connect to a TCP collector listening on port 4739 via localhost
output {
uri="tcp:localhost:4739"
options = ""
}

Struktura konfiguracnych suborov je dana kniznicou libconfuse, ktora ma vlastnu specificku syntax. Konfiguracne moznosti a ich syntax su nasledovne:

• report_freq:

Defines how often, in seconds, IPFIX flow records should be
        exported for expired flows. Expired flows will not be dealt
       with outside of these reporting intervals. In addition, this
       defines how often option template records will be exported.

       Syntax is as follows:

       report_freq = <seconds>

• domain_id:

Within IPFIX, measurement data from multiple "Observation
           Domains" may be accumulated by a single collector. This option
          specifies which Observation Domain id number should be 
          applied to the flow records exported by this meter.
          
          Syntax is as follows:

          domain_id = <id>

• module_location:

Specifies the directories where the information element
            modules have been installed so that maji can locate them.
           By default, the standard elements will all be installed
           into /usr/local/maji/info_elements - if you
           have installed maji elsewhere, you'll need to change this
           option to match that location. 

           Multiple module locations may be specified if required,
           for instance if user-defined information elements have
           been defined in a separate location.

           Avoid using relative paths when providing a module
           location.

           Syntax is as follows:

           module_location = {
              "/module/location/one", "/module/location/two"
           }

• template

Describes an IPFIX template. Templates consist of two sets of 
         information elements, one describing the flow key and the other
         describing what should be measured per flow. The example config
         defines a template using the standard 5-tuple as a key and
         counts packets and bytes for each flow.

         When defining a template, the information element names must be
         exactly the same as the name of the source file where the 
         element is defined in the info_elements directory. For example, 
         the sourceIPv4Address element is defined in sourceIPv4Address.cc.
         
         The syntax for this option is (note the placement of braces and
         double quotes) :

         template
         {
               keys = { "key_element_one", "key_element_two",
         "key_element_three" }
         elements = {"per_flow_measure_one", 
         "per_flow_measure_two" }
         }

• option_template:

Describes an IPFIX option template. Rather than measuring
            a distinct flow, option templates measure meta-data about
           the IPFIX metering process. Option template records are
           sent once per reporting interval, regardless of how many
           flows might have expired during that interval. The syntax
           for option templates is more or less identical to 
           regular templates.

           Note that we use the term "scope" instead of "keys" for
           defining the keys for an option template. This is to 
           maintain consistency with the IPFIX specifications. 
           For all intents and purposes, scope and keys are 
           interchangable terminology.

           The syntax for this option is:

           option_template
           {
              scope = { "option_key_one", "option_key_two" }
               elements = {"option_measure_one", "option_measure_two" }
           }

• output:

Specifies an exporter to be used for outputting IPFIX records.
        The uri option describes which exporter to use, along with a 
       base option (if required). The options field can be used to provide
       getopt-style command line arguments to the exporter. More details
       on the options accepted by each of the exporters is provided below
       in the "Supported Export Formats" section.

       Multiple exporters may be specified using separate "output" config
       options.

       Syntax:

       output {
          uri="<export format>:<base location or port number>"
      options="<exporter exclusive command line arguments>"
       }   

Podporovane formaty exportu

Maji podporuje niekolko formatov pre exportovanie IPFIX zaznamov. Kazdy typ exportu ma rozne moznosti konfiguracie:

• Standard Output

a.k.a. writing IPFIX records to the terminal, mainly 
   useful for debugging and checking that things are working correctly. Takes
   no options.

   Syntax:
      output {
      uri="stdout:"
      options = ""
   }

• SQLite

writes the IPFIX records to an SQLite3 database. You must have
   libsqlite3 installed to be able to use this exporter. The database will be
   named after the time when the database was created. Each template will 
   create a separate table within the database. The table names will be derived
   from the information elements used by the templates. I warn you now, the
   table names are not pretty!

   The SQLite exporter takes options that allow the user to specify a 
   frequency for rotating the database file, i.e. closing the existing
   database and creating a new one. Doing so allows you to keep separate 
   databases for different time periods. The -R option can be specified as
   either "h" or "d" to describe an hourly or daily base rotation respectively.
   The -C option specifies how many times to rotate within the base rotation
   period, e.g. -R d -C 4 would rotate 4 times daily, i.e. every six hours. 
   All rotation times are calculated from the start of the day, not when the
   meter was started so the above example would always rotate at 0000, 0600,
   1200 and 1800 hrs.

   The intended location of the SQLite database files is specified as part
   of the uri, after the first colon. Avoid using a relative path when
   specifying the location.

   Syntax:
      output {
      uri="sql:<location to store database files>"
      options="-R <'h' or 'd'> -C <times to rotate>"
   }

• UDP

writes IPFIX records to the network via UDP. This exporter requires
   that a listening collector be available to connect to. Maji comes bundled 
   with a simple UDP collector that will merely print IPFIX records to a 
   terminal. An alternative collector is required to do anything more 
   sophisticated with the received records.
   
   The IP address and port of the collector should be specified as part of the
   uri as shown below.

   The UDP exporter supports two additional options: -r which specifies how
   frequently template definitions should be retransmitted in seconds and -p
   which specifies that templates should be retransmitted after a certain
   number of IPFIX messages have been exported. These have default values of
   600 seconds and 200 messages respectively. 
   
   The message-based retransmit is designed to be used in conjunction with the
   timer-based one. Retransmission will occur every N seconds or every M 
   messages, whichever occurs first. Once a retransmission is performed, both
   the timer and the message count is reset to zero. In addition, if the -p
   option is not specified, message-based retransmission will not occur at all
   and only the timer will be used. There is no way to disable timer-based
   retransmission.   

   Syntax:
      output {
      uri="udp:<collector ip>:<collector port>"
      options="-r <retransmit seconds> -p <retransmit messages>"
   }

• TCP

writes IPFIX records to the network via TCP. This exporter requires
   that a listening collector be available to connect to. Maji comes bundled 
   with a simple TCP collector that will merely print IPFIX records to a 
   terminal. An alternative collector is required to do anything more 
   sophisticated with the received records.
   
   The IP address and port of the collector should be specified as part of the
   uri as shown below.
   
   The TCP exporter takes no additional options.

   Syntax:
      output {
      uri="tcp:<collector ip>:<collector port>"
      options=""
   }

• SCTP

writes IPFIX records to the network via SCTP. This exporter requires
   that a listening collector be available to connect to. Maji comes bundled
   with a simple SCTP collector that will merely print IPFIX records to a
   terminal. An alternative collector is required to do anything more
   sophisticated with the received records.

   The IP address and port of the collector should be specified as part of the
   uri as shown below.

   The SCTP exporter takes no additional options.

   As per the recently published IETF draft "IPFIX Export per SCTP Stream",
   http://www.ietf.org/internet-drafts/draft-ietf-ipfix-export-per-sctp-stream-01.txt,
   the SCTP exporter included with Maji uses a separate SCTP stream for each
   defined data template. All option templates, however, are exported using a
   single dedicated stream. Ultimately, this should have little apparent 
   effect for a user, though.

   Syntax:
      output {
      uri="sctp:<collector ip>:<collector port>"
      options=""
   }

• Stdout

writes IPFIX records directly to the terminal via stdout. 
   Primarily intended for use in testing and debugging Maji as, aside from
   being able to manually inspect IPFIX records, it has little practical use.

   The stdout exporter takes no additional options.

   Syntax:
      output {
      uri="stdout:"
      options=""
   }

Informacne elementy

Vsetky podporovane informacne elementy su ulozene v

<install directory>/info_elements

Maji kolektory

Nasledujuce kolektory su sucastou dodavky Maji:

• simple_udp_reader - UDP collector
• simple_tcp_reader - TCP collector
• simple_sctp_reader - SCTP collector

Spustanie Maji kolektorov

simple_<protocol>_reader -c <directory> [-p <port>] 

Vysvetlivky:

• -p port

• -c directory

Spustanie Maji

 maji -c <maji config file> <libtrace uri> 

Libtrace uri - kde ma Maji zachytavat, alebo odkial citat pakety. Detaily podporovanych trace formatov a ako ma uri pre kazdy format vyzerat sa nachadzaju tu: Podporovane formaty Libtrace.

-c cesta ku konfiguracnemu suboru.

Maji moze byt spusteny aj ako daemon spustenim 'majid' namiesto 'maji'.

pozn.: pouzivatelia by mali spustat maji radsej z nainstalovaneho adresara (default: /usr/local/maji/bin) ako zo zdrojoveho adresara.

Experiment 13.04. 2010 - potvrdenie kompatibility s JXColl-3.2

Stručná charakteristika

• Účel: testovanie kompatibility nastroja Maji s meracou platformou BasicMeter
• HW: localhost
• OS: Ubuntu GNU/Linux 9.04 server (Jaunty)
• SW: Maji 1.0.1beta, JXColl-3.2

Tento experiment potvrdil kompatibilitu nastroja Maji v pozicii exportera a kolektora JXColl-3.2.

Scenar experimentu

Scenar experimentu spocival v stiahnuti iso suboru debianu: http://ftp.antik.sk/debian-cd/current-live/i386/iso-cd/debian-live-504-i386-standard.iso a pozorovani, ci Maji nadviaze spojenie s JXColl-3.2 a tento bude bez problemov prijimat IPFIX pakety a ukladat data o nich do databazy.

• test.config: Konfiguracny subor Maji.
• jxcoll.conf: Konfiguracny subor JXColl-3.2.

spustenie Maji: ./maji -c ~/install/maji/maji-1.0.1/test.config int:eth0

spustenie JXColl: jxcoll

Vystupy experimentu

• majijxcolldbs13-04: Vystup nastoja Maji.

• jxcollmajidbs13-04: Vystup kolektora JXColl-3.2.

• majisql13-04.out: Vystup databazy postgresql.

• majijxcoll13-04.png:
  

Zaver experimentu

Experiment mozeme pokladat za uspesny, pretoze potvrdil kompatibilitu nastroja Maji a kolektora JXColl-3.2. A teda Maji moze byt v pripade potreby pouzity ako nahrada exportera mybeem.