MonALISA Grid Monitoring
Menu mode: dynamic | fixed
  HOME       CLIENTS       REPOSITORIES       DOWNLOADS       LOOKING GLASS       FAST DATA TRANSFER  
Last update on:
Dec 03, 2015

Uptime: 142 days, 10h, 44m
Number of requests: 4032791
since 28 October 2005
ApMon Python User Guide

ApMon Python User Guide


Chapter 1. Installation

The ApMon archive contains the following files in the ApMon module:

  • apmon.py - main ApMon module. It can be instantiated by users to send data.
  • ProcInfo.py - procedures to monitor the system and a given application.
  • README
  • example/*.py - a set of examples with the usage of the ApMon module.
  • example/*.conf - a set of sample destination files, for url/file configuration.
  • ApMon_doc/*.html - HTML documentation

Chapter 2. Using ApMon

1. ApMon Initialization

In the Python version, the ApMon features are available as methods of a class called ApMon. The ApMon.pm module defines the ApMon class that has to be instantiated in one of the following ways:

  • passing as parameter a location from where the configuration will be loaded. This should be a string that represents either a filename, or an URL address (if it starts with "http://"). The ApMon class will start a second thread that will verify periodically the configuration file/webpage for any changes. This verification is done in a sepparate thread because there could be long waiting times for the downloading and the resolving of the destination hosts for the udp datagrams. This way, the sendParam* functions will not block. You can also pass an array of file/url strings in order to initialize ApMon.
  • passing as parameter a tuple, containg as elements directly the destinations (host[:port][ pass]). ApMon will send datagrams to all valid given destinations (i.e.hosts that can be resolved), with default options.
  • passing as parameter to the constructor a hash. In this case, the keys will be destinations and the corresponding values will be references to another hash in which the key is a configuration option and the value is the value of that option. Note that in this case, the options should not be preceded by xApMon_ and options should be True/False instead of on/off as in the configuration file.

2.  Sending Datagrams

To send user parameters to MonALISA, you have the following set of functions:

        sendParameters ( clusterName, nodeName, params);
     

Use this function to send a set of parameters to all given destinations. The default cluster an node names will be updated with the values given here. If afterwards you want to send more parameters, you can use the shorter version of this function, sendParams. The parameters to be sent can be eiter a list, or a hash. If list, it should have an even length and should contain pairs like (paramName, paramValue). paramValue can be a string, an integer or a float.

        sendParams (params);
     

Use this to send a set of parameters without specifying a cluster and a node name. In this case, the default values for cluster and node name will be used. See the sendParameters function for more details.

        sendTimedParameters (clusterName, nodeName, time, params);
     

Use this instead of sendParameters to set the time for each packet that is sent. The time is in seconds from Epoch. If you use the other function, the time for these parameters will be sent by the MonALISA service that receives them. Note that it is recommended to use the other version unless you really need to send the parameters with a different time, since the local time on the machine might not be synchronized to a time server. The MonALISA service sets the correct real time for the packets that it receives.

Please see the examples/*.py files for examples of using these functions.

3. Configuring ApMon with the aid of the API

The behaviour of ApMon can be configured not only from the configuration files or webpages, but also with the aid of the API.

If you want to disable temporarily sending of background monitoring information, and to enable it afterwards, you can use:

        enableBgMonitoring (bool)
     

To force sending monitoring information with background monitoring disabled, you can use the following function:

        sendBgMonitoring ();
     

4. Automated Job Monitoring

To monitor jobs, you have to specify the PID of the parent process for the tree of processes that you want to monitor, the working directory, the cluster and the node names. If work directory is "", no information will be retrieved about disk:

        addJobToMonitor (pid, workDir, clusterName, nodeName);
     

To stop monitoring a job, just call:

        removeJobToMonitor (pid);
     

5. Logging

To change the log-level of the ApMon module, you can either set it in the configuration file or use the following function:

        setLogLevel(level);
     

with the following valid log-levels: "DEBUG", "NOTICE", "INFO", "WARNING", "ERROR", "FATAL"

From the configuration file the log-level can be set as follows:

        xApMon_loglevel = <level>
     

e.g.,

        xApMon_loglevel = FINE
     

6. Maximum number of messages

To set the maxim number of messages that can be sent to a MonALISA service, per second, you can use the follwing function:

                          setMaxMsgRate(rate);
                 

By default, it is 50. This is a very large number, and the idea is to prevent errors from the user. One can easily put in a for loop, without any sleep, some sendParams calls that can generate a lot of unnecessary network load.

You can put this line in the config file:

                          xApMon_maxMsgRate = 30
                 

The datagrams with general system information are only sent if system monitoring is enabled, at greater time intervals (one datagram with general system information for each 2 system monitoring datagrams).

7. Recreate the ApMon object

If you have to recreate the ApMon object over and over again, then you should use the following function when you fininshed using a instance of the ApMon:

                          free();
                 

This will delete the temporary file and terminate the background processes. The UDP socket will not be closed, but if you will reuse ApMon afterwards, it will not open another socket.

8. Restrictions

The maximum size of a datagram is specified by the constant MAX_DGRAM_SIZE; by default it is 8192B and the user should not modify this value as some hosts may not support UDP datagrams larger than 8KB.

Chapter 3. How to Write a Simple Python Program with ApMon

In this section we show how the ApMon API can be used to write a simple program that sends monitoring datagrams. The source code for this short tutorial (slightly modified) can be found in the simple_send.py file under the examples/ directory.

The program generates values for a few parameters and sends them to the MonALISA destinations specified in the 'dest_2.conf' file; this action is repeated in 20 iterations.

With this example program we'll illustrate the steps that should usually be taken to write a program with ApMon:

1. Import the ApMon module (and possibly other necessary modules):

      import apmon
      import time
   

2. Initialize ApMon and possibly set some options (in this example, we disabled the background job/system monitoring and the periodical reloading of the configuration file, and also set the loglevel). These options could have also been set from a configuration file.

      # Initialize ApMon specifying that it should not send information about the system. 
      # Note that in this case the background monitoring process isn't stopped, in case you may
      # want later to monitor a job.
      apm = apmon.ApMon('dest_2.conf')
      apm.setLogLevel("INFO");
      apm.confCheck = False
      apm.enableBgMonitoring(False)
   

3. Send the datagrams. We used here two functions: sendParameters, which specifies the cluster name and the node name (which will be cached in the ApMon object), and sendParams, which uses the names that were memorized at the call of the first function.

      for i in range(1,20):
      # you can put as many pairs of parameter_name, parameter_value as you want
      # but be careful not to create packets longer than 8K.
          apm.sendParameters("SimpleCluster", "SimpleNode", {'var_i': i, 'ar_i^2': i*i})
          f = 20.0 / i
          # send in the same cluster and node as last time
          apm.sendParams({'var_f': f, '5_times_f': 5 * f, 'i+f': i + f})
          print "simple_send-ing for i=",i 
          time.sleep(1)
   

This, and other documents, can be downloaded from http://monalisa.cacr.caltech.edu/

For questions about MonALISA, write at <support@monalisa.cern.ch>.