backendprocesses
Differences
This shows you the differences between two versions of the page.
| Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
|
backendprocesses [2010/11/18 14:01] morten rewrite intro |
backendprocesses [2011/04/08 13:10] faltin [Back-end processes in NAV] |
||
|---|---|---|---|
| Line 3: | Line 3: | ||
| NAV has a number of back-end processes. This page attempts to give an overview of them. | NAV has a number of back-end processes. This page attempts to give an overview of them. | ||
| - | The following figure used to complements this document, but is getting a bit outdated (it hasn't been updated since NAV 3.2). | + | The figure below complements this document. Unfortunately the figure is getting a bit outdated (it hasn't been updated since NAV 3.2): |
| + | * getDeviceData is now replaced with ipdevpoll | ||
| + | * iptrace does not exist anymore (the job is done by ipdevpoll) | ||
| + | * snmptrapd is a new daemon process | ||
| {{architecture1.png?800|The NAV processes}} | {{architecture1.png?800|The NAV processes}} | ||
| Line 21: | Line 24: | ||
| * [[backendprocesses#collecting_statistics|cricket]] (includes makecricketConfig, Cricket collector and cleanrrds) | * [[backendprocesses#collecting_statistics|cricket]] (includes makecricketConfig, Cricket collector and cleanrrds) | ||
| * [[#eventengine|eventengine]] | * [[#eventengine|eventengine]] | ||
| - | * [[#getdevicedata|getDeviceData]] | + | * [[#ipdevpoll|ipdevpoll]] |
| - | * [[#iptrace|iptrace]] | + | |
| * [[#logengine|logengine]] | * [[#logengine|logengine]] | ||
| * [[#mactrace|mactrace]] | * [[#mactrace|mactrace]] | ||
| Line 36: | Line 38: | ||
| ====== Building the network model ====== | ====== Building the network model ====== | ||
| - | ===== getDeviceData ===== | + | ===== ipdevpoll ===== |
| - | + | ||
| - | + | ||
| ==== Key information ==== | ==== Key information ==== | ||
| - | ^ Process name | getDeviceData | | + | ^ Process name | ipdevpoll | |
| - | ^ Alias | gDD / the snmp data collector | | + | |
| ^ Polls network | Yes | | ^ Polls network | Yes | | ||
| ^ Brief description | Collects SNMP data from equipment in the netbox table and stores data regarding the equipment in a number of tables. Does not build topology. | | ^ Brief description | Collects SNMP data from equipment in the netbox table and stores data regarding the equipment in a number of tables. Does not build topology. | | ||
| - | ^ Depends upon | Seed data must be filled in the netbox table, either by the [[seedessentials|Edit Database tool]] or by the autodiscovery contrib || | + | ^ Depends upon | Seed data must be filled in the netbox table, using the [[seedessentials|Seed Database tool]] | |
| - | ^ Updates tables | netbox, netboxsnmpoid, netboxinfo, device, module, gwport, gwportprefix, prefix, vlan, swport, swportallowedvlan, netbox_vtpvlan | | + | ^ Updates tables | netbox, netboxsnmpoid, netboxinfo, device, module, gwportprefix, prefix, vlan, interface, swportallowedvlan | |
| - | ^ Run mode | Daemon process. Thread based. | | + | ^ Run mode | Daemon process | |
| - | ^ Default scheduling | Initial data collection for new netboxes is done every 5 minutes. Update polls on existing netboxes is done every 6 hrs. Collection of certain OIDs for the netbox may deviate from this interval; i.e. the moduleMon OID is polled every hour. | | + | ^ Default scheduling | Polling is organized into jobs in ''ipdevpoll.conf'', so is job scheduling. | |
| - | ^ Config file | getDeviceData.conf | | + | ^ Config file | ipdevpoll.conf | |
| - | ^ Log files | getDeviceData.log og getDeviceData/getDeviceData-stderr.log | | + | ^ Log files | ipdevpoll.log | |
| - | ^ Programming language | Java | | + | ^ Programming language | Python | |
| - | ^ Further doc | [[http://metanav.uninett.no/static/reports/tigaNAV.pdf|tigaNAV report chapter 5]]| | + | ^ Further doc | | |
| Line 61: | Line 59: | ||
| ==== Details ==== | ==== Details ==== | ||
| - | * Initial OID classification \\ When gDD detects a new box that has a valid snmp read community (regardless of category), he will start initial OID classifiation. This is done by testing the netbox against all OIDs in the snmpoid table and in turn populating the netboxsnmpoid table. Testing is done based on attributes in snmpoid table, see reference to further doc for details. Frequency will be set based on the snmpoid.defaultfreq. | + | * jobs and plugins \\ All ipdevpoll's work is done by plugins. Plugins are organized into jobs, and jobs are scheduled for each active IP device individually. |
| - | + | * inventory job \\ Polls for inventory information every 6 hours (by default). Inventory information includes interfaces, serial numbers, modules, VLANs and prefixes. | |
| - | * Plugin-based architecture \\ gDD has a plugin based architecture. Plugins fall into two types; device plugins and data plugins: | + | * profiler job \\ Runs every 5 minutes, profiling devices if deemed necessary. NAV has an internal list of SNMP OIDs that are tested for compatibility with each device. This is used to create a sort of profile that says what the device supports - the profile is typically used to produce a Cricket configuration that will collect statistics from proprietary OIDs. |
| - | * Device plugins collects data with SNMP. Each device plugin is geared towards a particular type of equipment, supporting a particular subset of OIDs. See further doc for details. | + | * logging job \\ Runs every 30 minutes and collects log-like information from devices. At the time being, only the arp plugin runs, collecting ARP caches from routers. ARP data is logged to a table, and aids in topology detection and client machine tracking. |
| - | * Data plugins updates NAVdb with data fed from the device plugins. A particular data plugin is responsible for a particular table (or set of tables) in the database. See further doc for details. | + | |
| - | + | ||
| - | * Module monitor \\ The module monitor is a data plugin within gDD. It has the dedicated function of detecting outage of modules in operating netboxes. When a module is detected down a moduleDown event is posted on the event queue (eventq). | + | |
| - | + | ||
| - | + | ||
| - | ===== iptrace ===== | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | + | ||
| - | ==== Key information ==== | + | |
| - | + | ||
| - | ^ Process name | iptrace | | + | |
| - | ^ Alias | IP-to-mac collector / arplogger| | + | |
| - | ^ Polls network | Yes | | + | |
| - | ^ Brief description | Collects arp data from routers and stores this information in the arp table. | | + | |
| - | ^ Depends upon | The routers (GW / GSW) must be in the netbox table. To assign prefixes to arp entries, [[#getDeviceData|getDeviceData]] must have done router data collection. | | + | |
| - | ^ Updates tables | arp | | + | |
| - | ^ Run mode | cron | | + | |
| - | ^ Default scheduling | every 30 minutes (0,30 * * * *). No threads | | + | |
| - | ^ Config file | None | | + | |
| - | ^ Log file | None | | + | |
| - | ^ Programming language | Perl | | + | |
| - | ^ Further doc | [[http://metanav.uninett.no/static/reports//NAVMe.pdf|NAVMe report ch 4.5.8]] (Norwegian) | | + | |
| - | + | ||
| - | + | ||
| - | + | ||
| - | ==== Details ==== | + | |
| - | + | ||
| - | * iptrace understands proxy arp and will not store arp entries that are "false". | + | |
| - | * iptrace ignores routers that are known to be down (fixed in NAV 3.1). | + | |
| - | * The command line tool [[commandlinetools#navclean.py|navclean.py]] offers a means of deleting old arp (and cam) entries. | + | |
| Line 111: | Line 74: | ||
| ^ Polls network | Yes | | ^ Polls network | Yes | | ||
| ^ Brief description | Collects mac addresses behind switch table data for all switches (cat GSW, SW, EDGE). The process also checks for spanning tree blocked ports. | | ^ Brief description | Collects mac addresses behind switch table data for all switches (cat GSW, SW, EDGE). The process also checks for spanning tree blocked ports. | | ||
| - | ^ Depends upon | [[#getDeviceData|getDeviceData]] must have created the swport tables for the switches. | | + | ^ Depends upon | [[#ipdevpoll|ipdevpoll]] must have created the swport tables for the switches. | |
| ^ Updates tables | cam (mac adresses), netboxinfo (CDP neighbors), swp_netbox (the candidate list for the physical topology builder), swportblocked (switch ports that are blocked by spannning for a given vlan). | | ^ Updates tables | cam (mac adresses), netboxinfo (CDP neighbors), swp_netbox (the candidate list for the physical topology builder), swportblocked (switch ports that are blocked by spannning for a given vlan). | | ||
| ^ Run mode | cron | | ^ Run mode | cron | | ||
| Line 346: | Line 309: | ||
| * See [[ThresholdMonitor]] | * See [[ThresholdMonitor]] | ||
| - | |||
| - | ===== moduleMon ===== | ||
| - | |||
| - | |||
| - | ==== Key information ==== | ||
| - | ^ Process name | getDeviceData data plugin moduleMon | | ||
| - | ^ Alias | The module monitor | | ||
| - | ^ Polls network | Yes | | ||
| - | ^ Brief description | A plugin to gDD. A dedicated OID is polled. If this is a HP switch, a specific HP OID is used (oidkey hpStackStatsMemberOperStatus), similarly for 3Com (oidkey 3cIfMauType). For other equipment the genereric moduleMon OID is used. For 3com and HP the OID actually tells us if a module is down or not. For the generic test we (in lack of something better) check if an arbitrary ifindex on the module in question responds. If the module has no ports, no check is done. | | ||
| - | ^ Depends upon | The switch or router to be processed by gDD with apropriate data in module and gwport/swport. | | ||
| - | ^ Updates tables | posts moduleMon events on the eventq. Sets in addition the boolean module.up value. | | ||
| - | ^ Run mode | daemon, a part of gDD. | | ||
| - | ^ Default scheduling | Depends on the defaultfreq of the moduleMon OID (equivalently for the HP and 3com OIDs) Defaults to 1 hour. | | ||
| - | ^ Config file | see gDD | | ||
| - | ^ Log file | see gDD | | ||
| - | ^ Programming language | Java | | ||
| - | ^ Further doc | Not much. | | ||
| Line 425: | Line 371: | ||
| ^ Config file | alertengine.cfg | | ^ Config file | alertengine.cfg | | ||
| ^ Log file | alertengine.log og alertengine.err.log | | ^ Log file | alertengine.log og alertengine.err.log | | ||
| - | ^ Programming language | perl | | + | ^ Programming language | Python | |
| ^ Further doc | [[http://metanav.uninett.no/static/reports/NAVMore.pdf|NAVMore report ch 3.7 and 3.8]] (Norwegian). | | ^ Further doc | [[http://metanav.uninett.no/static/reports/NAVMore.pdf|NAVMore report ch 3.7 and 3.8]] (Norwegian). | | ||
| Line 488: | Line 434: | ||
| - | ===== The snmptrapd ===== | + | ===== snmptrapd ===== |
| Line 528: | Line 474: | ||
| ^ Config file | None | | ^ Config file | None | | ||
| ^ Log file | cricket-changelog | | ^ Log file | cricket-changelog | | ||
| - | ^ Programming language | perl | | + | ^ Programming language | python | |
| ^ Further doc | [[howtoconfigurecricket|How to configure Cricket addons in NAV v3]] | | ^ Further doc | [[howtoconfigurecricket|How to configure Cricket addons in NAV v3]] | | ||
backendprocesses.txt · Last modified: 2015/09/24 12:48 by morten
Page Tools
