NAV Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.

====== The history of NAV ======


===== 1999 - NAV v1 =====

The development of NAV started  at [[http://www.ntnu.no|NTNU]] in 1999. Prior to that NTNU, like so
many others, had acquired a traditional network management system (HP
Openview) that hopefully would fulfil our needs. After thorough
investigation and adjustment of the system we found however that it
lacked functionality in many areas. We decided to implement a basic
system ourself.

The name NAV came about later, in the initial stages we called our
system //NettInfo//. The first tasks we solved were:

  * A rudimentary collection system that gathered router port data using SNMP.

We introduced MRTG at the time and found it tedious to manually update
the MRTG config file, thus we made:

  * An automated system for generating the MRTG configs based on the actual router ports in use.

At the time we also developed a separate system, called //Falkeblikk//:

  * A basic status monitoring that used ping to verify availability of routers, switches and servers. Detected outages where reported using  SMS (or email). (//Falkeblikk// was in 2001 replaced by //Live// and in 2002 replaced by //pping//).
 
Further we expanded to cover:

  * An ARP data collector and database giving us historical information on IP and MAC address usage.

  * Web based reports with data on routers, router ports, switches and switch ports.

  * The first version of the network map showing traffic load on a manually laid out network graph.

Project participants in 1999: Vidar Faltinsen (project leader),
Trygve Lunheim, Tor-Arne Kvaløy, Eric Sandnes, 
Stig Venås, Gro-Anita Vindheim, Knut-Helge Vindheim.

===== 2000 =====

The development at the time was focused around perl and php. In 2000
we introduced java in an upgraded version of the traffic map (vlanplot). We
also introduced mySQL for our database. Further:

  * We evolved our collection system around an important design principle; we would use seed files and SNMP to build our model of  the running network. In other words; we would not put resources into doing auto discovery. Our initial understanding of topology required port naming standards on the configured routers and switches in the network.

  * We improved our web front end, introducing a basic authentication and authorization scheme supporting three levels of access (public, restricted and secret).

  * The initial event and alert engine, named Trapdetect, was implemented. Trapdetect incorporated an snmp trap demon. Trapdetect parsed incoming traps and sent alarms to users using email or SMS. (Trapdetect is in NAV v3 replaced by event engine, alert engine and alert profiles).

  * We moved from MRTG to RRD and Cricket. We automated the process of making the Cricket configuration tree.

Project participants in 2000: 
John Magne Bredal, Kristian Eide, 
Vidar Faltinsen (project leader),
Trygve Lunheim, 
Gro-Anita Vindheim, Knut-Helge Vindheim.


===== 2001 - NAV v2 =====

In early 2001 UNINETT gained interest in our system. They asked
whether it could be used on other universities and colleges as well.
At the time the system was tightly coupled with the NTNU network
design and the local naming standards used for equipments and router/
switch ports.

We decided to take the challenge and with funding from UNINETT the
process of making a more general network management system
started. This was when our name came about; Network Administration
Visualized or NAV. The system running at NTNU at the time was referred
to as NAV version 1 (or NTNU NAV).

The 2001 development project (called NAVMe) redesigned the NAV
database from scratch introducing many improvements. We also moved
from mySQL to postgreSQL with the advantages that imply (cascading
deletes etc).  The collection system was revitalized and we made a
general report generator.

A crucial result was the general topology detector. We no longer based
topology detection on port naming standards, at least not on the
switch layer. We introduced a general CAM collector gathering mac
address to switch port data. By complementing this with CDP data we
were able to understand the physical network topology on layer
2. Further we gathered vlan information including trunk port data and
spanning tree states. We also derived the actual scope of the various
vlans based on the actual live status in the running network.
 
We adopted CVS as our code revision system.  (CVS was replaced by
subversion in 2003).

The result of project NAVMe was NAV v 2.0. NAV v2.0 was successfully
installed at NTNU and at the University of Tromsø in late 2001.

In summary NAV version 2.0 incorporated:

  * A general collection system building a model of the running network

  * A status monitor (//Live//) reporting network outage (using topology information to report on root cause)

  * A general report generator introducing an easy to expand set of reports (equipment types, software version, IP address usage, vlan scope, switch port state etc).

  * An automated Cricket config builder allowing "maintenance free" gathering of network statistics, including octet, packet and error counters.

  * A graphical drill-down display of the network topology also visualizing traffic and CPU load.

  * A machine tracker with historical data on the whereabouts of machines (IP and mac adresses) in the network (pinpointed down to the connected switch port).

  * A general RRD threshold monitor sending alarms on rising and falling thresholds (using hysteresis).

  * An event and alert system (//Trapdetect//) with the ability to interpret internal and external (SNMP trap) events and sending alerts using SMS or email to NAV users. NAV users could configure their alert profile using a web based front end.
 
Project participants in 2001: 
John Magne Bredal, Kristian Eide, 
Vidar Faltinsen (project leader),
Sigurd Gartmann, 
Gro-Anita Vindheim, Knut-Helge Vindheim.


===== 2002 =====


With continued contribution from NTNU and UNINETT we kept on with our
work in two follow-up projects named NAVMore and NAVRun. The former
focused on further development of NAV, while the latter introduced an
installation system making NAV easily available to UNINETT members;
i.e. Norwegian universities and colleges.

The development was now done in two parallel tracks; version 2.1 and
version 3.

Version 2.1 introduced improvements to the existing version 2.0 and
was geared for the 2002 distribution of NAV to UNINETT members. New
functionality in NAV version 2.1 included:

  * Improvement in the topology detection, supporting a larger set of equipment types.

  * A Cisco syslog parser and database.

  * A tabular display of the IP address scope usage.

By the end of 2002 all four Norwegian universities and a total of six
colleges where running NAV v2.1. Complementing our documentation pages
and our mailing lists we introduced a web based bug reporter system
(using php bugtracker). We further established a scheme for delivering
patches to our system.  At the time distribution was done through ftp
with a closed user group.

In the summer of 2002 we radically expanded the development group.
The main effort was on the new NAV version 3.  Unfortunately
development of NAV version 3 has been more time demanding than
expected. In fact the version 3 results produced in 2002 have not been
deployed until the beta release of NAV v3 in early 2004.

Nevertheless, the NAV v3 2002 results included:

  * A sorted, tabular display of RRD data.

  * Cricket/RRD collection of server statistics (i.e. disk usage, cpu load, number of users etc).

  * A new status monitor, pping, working in parallel (thread based, implemented in python).

  * A service monitor detecting outages of services. Initially supporting ssh, http, imap, pop, smtp, smb, rpc, dns, dc. The monitor is plugin based, allowing for easily expansion of new services to monitor. 

  * A general event system, the event engine, replaces trapDetect. Event engine parses an event queue and has built in mechanisms for correlating events and reporting based on network topology. With event engine SNMP is no longer the only input possibility for events. Further, the task of processing alerts to users is separated from the more general work of coping with events. The event engine also reduces the complexity needed in the basic monitors (status, threshold, service etc), as they no longer need to worry about network topology issues.

  * A new alert engine and a new alert profile system focused on delivering alarms to NAV users based on their configured profile. Alert engine and alert profile is //very// general. Where trapdetect had many constraints as to how the NAV user could adjust his profile, alert profile allows the user to configure //almost anything//. Further, it is easy to support more alarm types in the future. Initially support for email and SMS was implemented.

Project participants in 2002: 
John Magne Bredal, Kristian Eide, 
Vidar Faltinsen (project leader),
Sigurd Gartmann, 
Erik Gorset, Steinar Hamre, Magnus Thanem Nordseth, Daniel Sandvold,
Andreas Åkre Solberg, 
Stian Søiland, Knut-Helge Vindheim, Morten Vold, Arne Øslebø.


===== 2003 =====

In 2003 focus has been on finalizing NAV version 3.0. Once again NTNU and UNINETT are supporting development, and a large group of students and staff are involved in development.  This years project was called
tigaNAV. Some important decisions are made:

  * Adopt English as our language in every aspect of the system (no more Norwegian). This would include the database (with names on tables and fields), the code (with comments, and ideally the names of all variables), the web interface (of course) and all documentation, including the project report.

  * Adopt python as the preferred programming language for the user interface. Introduce a templating mechanism, thus abstracting the look and feel from the content programming.  Cheetah was chosen as our templating engine. In effect we wanted a more consistent (and more professional) user interface.

  * Adopt a general and powerful authentication and authorization mechanism. Privileges may be granted to account groups based on a "who, what, where" principle.

  * Replace CVS with subversion. Subversion allows atomic commits and history is kept in cases where files or directories are renamed or moved.

  * The monolithic installation approach used for NAV v2.1 is left. The source code repository is restructured and a build system is established. Installation layout is also restructured adopting a typical UNIX like directory structure.

Further many significant changes is done in the underlying collection
system. One important goal is to improve NAV's ability to support
new equipment types:

  * An OID database is introduced containing information on all SNMP OIDs NAV used in the collection process.  The new collection engine, getDeviceData, uses the OID database. When new equipment is added to NAV it is tested against all the OIDs and classified accordingly.

The new collection system (getDeviceData) uses a plug in-based architecture, working
in threads. We allow for fine-grained control of collection intervals.
In effect the NAV database is kept more up-to-date with changes in
the managed network (in version 2 updates were done on a nightly
basis).

Finally there are important functional improvements:

  * A more general operational status page with status on all operational events (eventually).

  * An integrated message system to inform NAV users of scheduled outage, special faults and other operational events.

  * The IP device center presenting all information on a device in one web page with links to reports, statistics, switch port data etc.

  * The network explorer introducing a graphical display of the network layout on a per vlan basis.

  * An RRD browser with the ability to gather different statistics on the same page or even in the same graph.

  * Device management with the ability to track milestone events of physical devices from order and arrival through the operational stages.

Also worth mentioning; NAV has so far been presented on numerous occasions/
conferences in Norway. In 2003 NAV is for the first time
presented internationally:

  * For the Cisco Works team (with the Cisco Works product manager and director of engineering) at the Cisco headquarters in San Jose, California.  March 2003.

  * On the 21st NORDUnet Networking Conference in Reykjavik, Iceland. August 2003.

Project participants in 2003: 
John Magne Bredal, Kristian Eide, 
Vidar Faltinsen (project leader),
Sigurd Gartmann, 
Bjørn Ove Grøtan, Hans Jørgen Hoel, Erlend Mjåvatten, 
Magnus Thanem Nordseth, Andreas Åkre Solberg, Magnar Sveen,  
Stian Søiland, Gro-Anita Vindheim, Morten Vold, Arne Øslebø.



===== 2004 =====

With most of the NAV version 3 development projects left behind, the focus in
2004 was to finish off unfinished work, fixing bugs and testing the
system thoroughly. In terms of new functionality, a conservative model was followed.
Our focus was on completing NAV3 and thus permanently leaving NAV2 behind.

Many hours were put into further
improving the collection system (getDeviceData). We also made important enhancements
to event engine, the traffic map, network explorer and the message system.

An important decision of releasing NAV under GPL is made.  In the
process many arguments are put forward, the most important being the
new opening for distributing of code.  It is our hope that a more wide
spread deployment of NAV will in return give interesting code
contributions back to the project.

The process of preparing the code for general availability was on the
agenda in the second quarter in 2004. In parallel the code was beta
tested on an increasingly set of Norwegian universities and colleges.

Documentation was improved and translated to English, and a new wiki web site
(based on MoinMoin) was established, also allowing for a more distributed contribution
of documentation.

A total of 8 NAV 3.0 beta releases was released in 2004, beta4  (May 11) being the first GPL release 
and beta7 (July 27) being the first with a changelog (which has been updated since). 

Project participants in 2004: John Magne Bredal, Kristian Eide, Vidar
Faltinsen, Sigurd Gartmann, Hans Jørgen Hoel, Magnus Thanem Nordseth,
Andreas Åkre Solberg, Magnar Sveen,
Stian Søiland, Gro-Anita Vindheim, Morten Vold (project leader), Arne Øslebø.

===== 2005 - NAV v3.0 =====

2005 was a relatively slow year for NAV. NTNU and UNINETT did not have so much resources to put into the
project. At the start of the year we introduced beta9, followed by beta10 in February. Focus was on bug fixing the most vital components.
In June we finally announced our first release candidate (rc1), followed by a second in July. 

NAV 3.0.0 was finally released on September 16 2005 with the following highlights:

  * Completely new web interface - new look
  * New authentication and authorization system
  * New Java based snmp collector: getDeviceData
  * New database design
  * New event- and alarm system (event engine, alert engine, alert profiles)
  * Module monitor
  * New message system
  * New tool Network discovery, improved vlanplot (the network map)

For further details on the improvements made in the 2005 releases, we refer to the 
[[http://svn.itea.ntnu.no/repos/nav/navme/tags/3.0.0/CHANGES|3.0 change log with its history]].
 
Project participants in 2005: John Magne Bredal, Kristian Eide, Vidar
Faltinsen, Gro-Anita Vindheim, Morten Vold (project leader).


===== 2006 - NAV v3.1 =====

2006 has been a definite upswing for NAV. Project development is now
supported by the UNINETT
[[https://openwiki.uninett.no/gigacampus:about|GigaCampus programme]] with an
overall goal of getting NAV up and running at most major universities
and university colleges in Norway within 2008. Morten Brekkevold
(previously Vold) is devoted full time to NAV, now being employed by
UNINETT.  Vidar Faltinsen made a comeback as project leader.  John
Magne Bredal at NTNU devoted 40% of his time working with NAV.  We
also recruited a student, Stein Magnus Jodal, to work on NAV as a
summer intern, and we kept him on for the fall.

The Universities of Tromsø and Bergen began sponsoring a FreeBSD port
of NAV, while Tromsø offered to contribute their own code to NAV.  The
Debian developer Morten Werner Olsen at The University of Oslo was
commisioned by UNINETT to maintain a Debian NAV package.  A SuSE
package was maintained by NTNU and Centos RPMs were contributed by
Alexander Krapivin at MSU, Moscow, Russia.

A strategy of targeting new feature releases twice a year was
established (with bugfix releases in between).  NAV 3.1 was released
in June and 3.2, with an original schedule of October, was released in
February 2007.

NAV gained the capability to actively block switch ports through the
addition of the NTNU-contributed Arnold tool.  Adding the ability to
collect high speed traffic counters (64 bits) made it possible to
reduce the Cricket collector frequency for gigabit switch ports from
every 1 minute to every 5 minutes, thus removing the need to have a
separate configuration tree for gigabit ports. The IP Device Center (later renamed to IP Device Info) was introduced.

We established a project page for NAV at SourceForge.  This was used
for bug tracking and hosting source code releases.  A month after the
3.1 release, we had seen approximately 160 downloads from around the
world (at the same time, the 3.0 version had been downloaded 674
times).

Project participants in 2006: John Magne Bredal, Vidar Faltinsen
(project leader), Stein Magnus Jodal, Morten Brekkevold.


===== 2007 - NAV 3.2/3.3 =====

The NAV year of 2007 opened with the belated 3.2 release in February.
The 3.2 release also marked our first step towards realizing a new
long-term goal: Reduce the number of dependencies and programming
languages utilized in NAV.

The 3.2 release replaced the messages system with separate maintenance
and message systems, rewritten from scratch, after many usability
complaints from users.  The Perl SMS daemon was rewritten in Python,
adding support for SMS dispatcher plugins.  The Ranked Statisdtics feature
was introduced. Much of the
behind-the-scenes HTML and CSS was cleaned up and restructured. The
release also brought numerous small improvements to NAV overall.

October 1st brought the NAV 3.3 feature release.  This release added
the long-awaited plugin based SNMP trap daemon, which enabled NAV to
translate traps into NAV events and alerts.  Browsing and searching of
device history was greatly improved.  The web interface received a
major facelift.  Switch/router interface names replaced port numbers
as identifiers in most parts of the web interface. A router port
view was added to IP Device Center (Info).

The 2007 development project saw Morten Brekkevold take over as the
project leader.  Kristian Klette was taken on as a new student intern,
and started working on a Traffic Map replacement, based on the Prefuse
visualization library.

Project participants in 2007: John Magne Bredal, Vidar Faltinsen,
Stein Magnus Jodal, Kristian Klette, Jostein Gogstad, Morten Brekkevold (project
leader).

===== 2008 - NAV 3.4 =====

In May 2008 NAV 3.4 was released. IPv6 support was added for the machine
tracker, subnet collection and subnet matrix. The LDAP authentication
was improved.

In the summer and autumn there was ongoing development for v3.5.

===== 2009 - NAV 3.5 =====

In January NAV 3.5 was released:
  * A new and improved IP Device Info now replaced IP Device Center.
  * Alert Profiles and Alert Engine were rewritten in Python. 
  * vlanplot was replaced by netmap. 
  * Network explorer was rewritten in Python. 
  * A new Radius tool was introduced that was able to track 802.1X authenticated users on the network
  * Report was improved and local reports were allowed
  * HP switch switch stack support was report do to many problems

===== 2010 - NAV 3.6/3.7 =====

NAV 3.6 was a major release for NAV as it introduced the forth generation SNMP collection system - ipdevpoll.
ipdevpoll would thus replace getDeviceData. The firs beta version of 3.6 came in May and the final release (after 6 beta version) 
was launched in October. NAV 3.6 included:

  * New SNMP collector: ipdevpoll 
  * Major changes to the dataves with a new common interface table (leaving the separate router interface and switxh port tables behind)
  * New backend daemon mailin: For receiving external alarms as email, parsing them and posting events on the event queue
  * Enhancements to report system: CSV export, monthly availability report and more
  * Several subsystems rewritten: machine tracker, dev history, makecricketconfig (from perl to python)
  * Support  “ignore resolved alerts” in Alert Profiles

NAV 3.7 came just a month after 3.6, in November, introducing more results from summer students:

  * Geomap -  a geographical map using OpenStreetMap data.
  * A sudo (mechaism allowing NAV administrators to manage user’s alert profiles and other setup
  * Introducing AD LDAP support

===== 2011 - NAV 3.8/3.9/3.10 =====

In February NAV 3.8 was out with:

  * New tool portAdmin that allows for configuring vlan value and port descriptions for switch ports.
  * The seedDB tool was rewritten
  * A report for inactive devices was included

In May 3.9 came with:

  * Threshold manager: New GUI to set threshold alarms - finally in place  
  * New tool Mac Watch: Generate alarm when a mac address appears on the network
  * Improved the physical topology detection ("topology" replaces "networkdiscovery topology")
  * Support for Extreme Switches
  * pynetsmp/NetSNMP support for ipdevpoll - significant performance impact


In December 3.10 came with:

  * Port link status monitoring with ipdevpoll - limit to topology ports if you like
  * Cricket statistics for environmental sensors in network equipment and Weathergoose
  * Generate alarm when a redundant power or fan in a switch fails (Cisco and HP)
  * Manage alarms from UPS: supports UPS-MIB (RFC 1628), APC, Eaton and MGE
  * Improved vlan topology detection ("topology" now entirely replaces "networkdiscovery")
  * Parallel DNS lookups increases performance 
  * The status monitor (pping) now supports IPv6
  * Alerts when SNMP agent stops responding

In November 2011 a NAV reference committee was established. The reference committee consists
of NAV power users from dominant Norwegian universities and university colleges. The reference
committee will give advice to future developments in NAV and help UNINETT in making our priorities.
The committee has members from these institutions: NTNU, UiT, UiO, UiA, HiVolda.



 ===== 2012 - NAV 3.11/3.12 =====

In March NAV 3.11 was released with the following improvements:

  * LLDP support for improved topology detection.
  * getBoksMacs replaced by ipdevpoll plugins in `topo` job.
  * Q-BRIDGE-MIB now also used when collecting switch forwarding tables,
    meaning improved machine tracking data for switch vendors like HP, Juniper
    and Alcatel.
  * The Machine Tracker's IP search form has consolidated the `To` and `From`
    IP address search fields into a single `IP Range` search field with
    extended search syntax for ranges of IP addresses.  See LP#643544 for
    details.
  * ipdevpoll can now be made to run single job configurations from
    ipdevpoll.conf, to run in the foreground, and to log to stderr instead of
    its log file.
  * ipdevpoll will reset its log levels from logging.conf upon receiving a
    SIGHUP signal.
  * ipdevpoll will log the currently active polling jobs upon receiving a
    SIGUSR1 signal.


In June 2012 the number of full time NAV-developers doubled! 


In September NAV 3.12 was out with:

  * Rewrite of new Netmap using open standards as HTML5, Javascript (D3JS) and SVG (Scalable vector graphics) 
  * A cleaner and more responsive Ip Device Info
  * An improved navbar search
  * Room information including devices and a powerful netbox interface overview
  * IPv6 traffic counters
  * An improved device poller
  * Greater use of django in favor of mod_python
NAV Wiki

User Tools

Site Tools

Page Tools
