NAV Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Writing ipdevpoll plugins =====

This is a short introduction for getting you started writing plugins for ipdevpoll.
Before you start you should read [blueprint:ipdevpoll].

==== Step 1 - The plugin skeleton ====

Let's jump straight to the code!

<code python>
# -*- coding: utf-8 -*-

from twisted.internet import defer
from nav.ipdevpoll import Plugin, FatalPluginError

class Example(Plugin):
    """
    Example plugin for ipdevpoll.
    """
    def __init__(self, *args, **kwargs):

        Plugin.__init__(self, *args, **kwargs)
        self.deferred = defer.Deferred()

    @classmethod
    def can_handle(cls, netbox):
        return True

    @defer.deferredGenerator
    def handle(self):
        self.logger.debug("Running the example plugin")	
	return

    def error(self, failure):
        """
        Return a failure to the ipdevpoll-deamon
        """
        if failure.check(defer.TimeoutError):
            # Transform TimeoutErrors to something else
            self.logger.error(failure.getErrorMessage())
            # Report this failure to the waiting plugin manager (RunHandler)
            exc = FatalPluginError("Cannot continue due to device timeouts")
            failure = Failure(exc)
        self.deferred.errback(failure)
</code>

Let's go through the code line by line.

We start by importing the the defer object from twisted, since all plugins are run asynchronously,
and we need to tell the scheduler that we can be run asynchronously. We then import the base plugin-class
and an ''FatalPluginError''-exception which we can exit the plugin with if something horrible happens during
the plugin run.

We can now define our own plugin. All plugins must inherit from the base ''Plugin''-class. We will see what this provides
us with later in the tutorial.

Now we have to create a init-method for initializing our plugin. We first make sure our base class gets initialized properly,
then we create our deferred-object so we can do async operations.

Next up is the ''can_handle'' method. The method is provided with a django instance of the netbox we are currently asked to
poll information from. This is a netbox-object collected with ''select_related(depth=2)'', so you can go out two levels from
it without blocking on database-access. This is very important as we *never* want to block on IO-access.
In the example we want to poll from all netboxes, so we just return True, but if we wanted to only handle routers for instance,
we could do a ''return netbox.category.id in ['GW', 'GSW']''.

We now define our ''handle''-method, this is were we do all our work. We decorate the method as a deferredGenerator. This allows
us to write async handling in a more sequential matter, and keeps the code a bit cleaner. You can read more about deferredGenerators here.
For now we will just log that we started the plugin then exit the plugin by returning.

Last up in the skeleton is our error-handler. We will use this later when we poll the boxes. This error handler checks if we got a timeout
trying to connect to the netbox, and return the failure.

That's the skeleton, now let's do some real work!

==== Step 2 - Polling data from the netbox ====

We use SNMP to talk to the devices we monitor, so we'll need something to ask for. This is defines through MIBs.
ipdevpoll includes a wrapper around mibs allowing us to use then in a more pythonic and clear way. See the adding mibs guide on
how to add more mibs to the system (TODO: Write that guide). We do include a number of mibs though, they are accessed through the ''nav.mibs''-module.

In this tutorial we will collect the ''ifDescr''-field from all the interfaces on the netboxes we poll from, let's get to it!

First of we must include the mib we are about to use, in this case ''IF-MIB'', so we add the proper import statement to our code.
<code python>
from nav.mibs.if_mib import IfMib
</code>

We are now ready to poll some data from the device. As before, code first, then the explanation.

<code python>
    @defer.deferredGenerator
    def handle(self):
        self.logger.debug("Running the example plugin")
        
        ifmib = IfMib(self.job_handler.agent)
        df = self.ifmib.retrieve_column('ifDescr')
        df.addErrback(self.error)
        dw = defer.waitForDeferred(df)
        yield dw
        
        result = dw.getResult()
        
        for (ifindex,),row in result.items():
            self.logger.debug("%s has ifDescr %s" % (ifindex, row['ifDescr']))
            
        return
</code>

First of we create our MIB-retriever object by calling its constructor with our job_handler's agent.
This agent does the actual talking to the device, and the mib-class translates OID's for us, so we can
use their textual names.

We then ask the mib-retriever to create an deferred-object (an asynchronous action) for us that ask the device for the column 'ifDescr',
then we ask our handler to wait for that deferred to finish and return it's result, and we yield the "waiter". This tells python
to jump back to this location once the waiting is over (the result is ready). We grab the result from the "waiter" and loop over it.
You might notice that ''ifindex'' comes out of nowhere. This is automaticly added as the table ''ifDescr'' belongs to is indexed by ''ifindex''. The indexes
are returned as a tuple, since a lot of tables have multiple indexes. Then we just print out our results.

Not to hard is it? :-)

But there is not much use in just printing our result, which leads us to step 3.

==== Step 3 - Storing data to the database ====

Storing to the database is handled by our job_handler, and should *never* be done by the plugin itself. This brings us to the concept of ShadowClasses. ShadowClasses are explained in
the ipdevpoll-specification metioned earlier in the tutorial. Read it again if you don't know what they are.

First of we need to get a hold of our ShadowClasses. They are accessed in the nav.ipdevpoll.storage-module. So we import it
<code python>
from nav.ipdevpoll import shadows
</code>

Now we are ready!

<code python>
    @defer.deferredGenerator
    def handle(self):
        # <Snip!> This part is the same are before
        netbox = self.job_handler.container_factory(shadows.Netbox, key=None)
        for (ifindex,),row in result.items():
            self.logger.debug("%s has ifDescr %s" % (ifindex, row['ifDescr']))
            interface = self.job_handler.container_factory(storage.Interface, key=ifindex)
            interface.netbox = netbox
            interface.ifindex = ifindex
            interface.ifdescr = row['ifDescr']
            
        return
</code>

So what's new here.
First of we need a reference to the device we are working on. This is done by asking
for a netbox-instance with key=None from the container_factory of our job_handler.
We need this so the storage-system can derive the foreignkeys on the interfaces we
want to update.
We then create a shadowed instance of the Interface-model by calling our
job_handler's container_factory-method with the class we want an instance of, and an key that
is unique for that object during this run. We set the ifindex and netbox of this object to the ifindex we
just collected and the netbox-object we got from the container_factory; this makes the storagesystem find the correct row in the database to update,
and we set our new ifdescr-attribute. And we're done! When the job is ower, the storage system
will parse all the objects created by the container_factory and update or insert as needed.


That's pretty much it, we'll try to write about creating more advanced plugins later, but that should get you started at least.


TODO: database-lookups inside the plugin, deletion of objects, testing