NAV Wiki

User Tools

Site Tools

Trace:
devel:hacking

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
devel:hacking [2009/05/27 08:53]
morten add radius to list of namespaces 		devel:hacking [2014/11/05 10:57] (current)
morten
Line 1: Line 1:
 ====== Hacker'​s guide to NAV ====== ====== Hacker'​s guide to NAV ======
  
-If you are contributing code to Network Administration Visualized,​ +This document has moved to https://​nav.uninett.no/​doc/​dev/​hacking/hacking.html
-please read this first. +
- +
- +
-====== Contributing to NAV ====== +
-Originally, NAV was a closed source project, initiated by the +
-Norwegian University of Science and Technology (NTNU), and eventually +
-sponsored by UNINETT on behalf of the Norwegian higher education +
-community. ​ In 2004, however, NTNU and UNINETT started distributing +
-NAV under the GNU General Public License, making it a truly free +
-software system. +
- +
-While UNINETT and NTNU are still the main contributors to NAV, +
-developing NAV to support the needs of the Norwegian higher education +
-community, contributions from third parties is highly appreciated. +
- +
-We communicate mainly through mailing lists, +
-[[https://​launchpad.net/nav/​|Launchpad]],​ and the wiki and Mercurial +
-repositories hosted at http://​metanav.uninett.no/ ​.  At times, UNINETT +
-also arranges workshops and gatherings for its customers: Norwegian +
-universities,​ university colleges and research institutions.  +
- +
-To contribute:​ +
- +
-Go to http://​metanav.uninett.no/​ and +
- +
-  * Join the mailing lists. ​ The //nav-dev// mailing list in +
-    particular is for discussing NAV development. ​ So far, this is a +
-    low traffic list. We can only hope this will change ;-) +
-  * Get a copy of the latest development sources by cloning the +
-    Mercurial repository at http://​metanav.uninett.no/​hg/​default/​. +
-    Most new development takes place on this branch. +
-  * Take a look at the [[:​navprojects|project reports from previous +
-    development projects at NTNU]] (NAVMe, NAVMore, tigaNAV and +
-    others) - design specifications and other useful bits of historic +
-    NAV information is mostly to be found in these. ​ Unfortunately,​ +
-    some of the oldest project documentation is in Norwegian only.  Do +
-    not hesitate to ask for help on the mailing lists. +
- +
-If you wish to contribute code to the project, see the [[#​submitting patches]] section. +
- +
-====== Directory layout ====== +
-A rough guide to the source tree: +
- +
-| conf/      | Files related to the autoconf build system. | +
-| doc/       | User and developer documentation,​ SQL scripts and example NAV configuration files. | +
-| tools/ ​    | Tool scripts for the build and release processes. | +
-| contrib/ ​  | User contributed NAV tools. NAV doesn'​t depend on these, and any maintenance of them is left up to the original developers. ​ We do not offer support for these tools. | +
-| packages/ ​ | Stuff to help packaging NAV for various platforms, such as RedHat, CentOS, FreeBSD, Debian and soforth. | +
-| src/       | Source code to Java subsystems of NAV (here for historic reasons). | +
-| subsystem/ | Source code to the rest of NAV - most of it Python. ​ NAV is loosely divided into subsystems, and each one of these has its own subdirectory in here. | +
-| subsystem/​lib-python/​ | Python libraries & APIs.  Please check what's already there before you roll your own. | +
-| subsystem/​lib-perl/​ | Perl libraries & APIs.  Only a single piece of Perl code remains in NAV; once this has been replaced, this directory will cease to exist. | +
-| subsystem/​webfront/​ | Python libraries for the web interface and front-page handler modules for mod_python. | +
- +
- +
-====== Development languages and frameworks ====== +
-For historic reasons, different parts of NAV are written in different +
-programming languages (Perl, Java, PHP and Python). ​ This has been +
-unfortunate in may ways, not at least for the sake of consistency and +
-maintenance,​ but fortunately we have a long-term goal of reducing the +
-number of languages and dependencies. ​ The last few years we've spent +
-a significant amount of time rewriting parts that were written in Perl +
-and PHP to Python, which is the language we are currently gravitating +
-towards. +
- +
-Currently (as of February 2009), major parts of NAV are written in +
-Python and Java, while only a single Perl program remains +
-(''​makecricketconfig.pl''​). +
- +
-  * When contributing patches to existing code, or plugins to existing +
-    subsystems, use the language that subsystem was written in. +
-  * When writing entirely new subsystems, the following rules apply: +
-    - If your subsystem is a new tool for the web interface, use +
-      Python and Django. ​ The legacy parts of the web system interface +
-      directly with ''​mod_python'',​ using Cheetah for HTML templating. +
-      Anything written in 2008 and later uses Django. ​ There is also +
-      [[devel:​django_introduction|a guide for interfacing Django +
-      applications with the legacy web code]]. +
-    - If your subsystem is a new back-end tool/​daemon,​ please use +
-      Python. ​ The NAV Python API is more complete than for any of the +
-      other languages, and you will receive a lot for free. +
- +
-If these rules are not followed, your patches will not be accepted +
-into NAV (but if they are really good, we will consider including them +
-in the //contrib// directory). +
- +
-If **YOU** are willing to invest in porting some of the existing +
-Java/Perl code to Python, then you will be celebrated as a NAV hero! +
- +
- +
-====== Coding style ======  +
-Much of the legacy NAV code was written without using any coding style +
-guidelines. ​ This has resulted in some chaotic combination of styles, +
-which we hope to reduce in the future. ​ For new code, please follow +
-these guidelines:​ +
- +
-  * For Java code, please refer to SUN's "Code conventions for the +
-    Java Programming Language":​ http://​java.sun.com/​docs/​codeconv/​ +
-  * For Python code, please refer to PEP-8, "Style Guide for Python +
-    Code" http://​www.python.org/​doc/​peps/​pep-0008/​ +
- +
-If you see violations of these guidelines, don't hesitate to fix them. +
-If you fix file-wide indentation problems etc., please submit this as +
-a separate patch to make your other patches look clean and readable. +
- +
-===== Python boilerplate headers ===== +
-We will generally only accept code into NAV if it is licensed under +
-GPL v2, but we may make individual exceptions for code licensed under +
-compatible licenses. ​ Each Python source code file should contain the +
-following boilerplate at the top: +
- +
-<code python>​ +
-# +
-# Copyright (C) 2008,2009 Somebody +
-# +
-# This file is part of Network Administration Visualized (NAV). +
-# +
-# NAV is free software: you can redistribute it and/or modify it under the +
-# terms of the GNU General Public License version 2 as published by the Free +
-# Software Foundation. +
-# +
-# This program is distributed in the hope that it will be useful, but WITHOUT +
-# ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or +
-# FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for +
-# more details. ​ You should have received a copy of the GNU General Public +
-# License along with NAV. If not, see <​http://​www.gnu.org/​licenses/>​. +
-# +
-</​code>​ +
- +
-If a file uses non-ASCII characters, it **must** be encoded as UTF-8, and an +
-encoding statement should be inserted at the top:  +
- +
-<code python>​ +
-# -*- coding: utf-8 -*- +
-</​code>​ +
- +
-====== Database connections ====== +
-NAV uses PostgreSQL as its database backend. ​ Namespaces (schemas) are +
-employed to logically group tables and relations. ​ NAV versions prior +
-to 3.5 employed separate PostgreSQL databases instead of namespaces. +
- +
-The namespaces currently in use are: +
- +
-| ''​manage'' ​     | The core knowledge database of NAV, containing all sorts of information about the monitored IP Devices, events, alerts, network topology and machine tracking data. | +
-| ''​profiles'' ​   | Contains NAV user accounts and groups, user preferences and alert profiles. | +
-| ''​logger'' ​     | Anything related to NAV's syslog parser/​browser system. | +
-| ''​arnold'' ​     | The port detention system Arnold stores it's data here. | +
-| ''​radius'' ​     | Radius accounting logs, updated directly by FreeRadius'​ PostgreSQL module. | +
- +
-**NOTE**: The following is Python-specific,​ more info should be added for +
-the other languages used in NAV. +
- +
- +
-===== Raw SQL ===== +
- +
-To obtain a connection to the NAV database, use the API accordingly,​ +
-e.g.: +
- +
-<code python>​ +
-import nav.db +
-# Get a connection to the NAV database +
-connection = nav.db.getConnection('​default'​) +
-</​code>​ +
- +
-The above code will open a connection to NAV's database, or, if a +
-previous connection with these parameters is already open, returns the +
-already existing connection from a connection cache. +
- +
-The ''​default''​ parameter is there for legacy reasons; it specifies +
-the name of a subsystem. ​ The ''​db.conf''​ file allows configuration of +
-separate database users for each subsystem (known as a ''​script''​ in +
-''​db.conf''​) of NAV.  The default ''​db.conf''​ file specifies a +
-database user for a subsystem called ''​default'',​ and also specifies +
-the same database user for all known subsystem names. ​ At present, +
-using a subsystem name that is not configured in ''​db.conf''​ will +
-raise an exception in ''​nav.db.getConnection()''​. +
- +
-===== Django models ===== +
-NAV 3.5 and on includes Django models for the most widely used +
-database tables. ​ If no SQL magic is needed to perform your database +
-voodoo, it is recommended that you use these models, located in the +
-module ''​nav.models''​. ​ You do not need to explicitly establish a db +
-connection to use these models, as Django takes care of all that. +
- +
- +
-====== Legacy web code ====== +
-When making changes to NAV's legacy web code, a few special +
-considerations need to be made. +
- +
-Legacy web code interfaces directly with +
-[[http://​www.modpython.org/​|mod_python]],​ and uses +
-[[http://​www.cheetahtemplate.org/​|Cheetah for HTML templating]]. +
- +
-===== Cheetah templates ===== +
-Global Cheetah templates are found in +
-''​subsystem/​webfront/​nav/​web/​templates/'',​ while local templates are +
-located in the subdirectories of their respective subsystems. ​ All +
-Python modules compiled from Cheetah templates should be installed in the +
-''​nav.web.templates''​ package. +
- +
- +
-===== Database connections ===== +
-Use the ''​nav.db.getConnection()''​ call to open or retrieve an +
-existing database connection. ​ All NAV web modules share the same +
-interpreter and namespace per Apache process, which also means that +
-database connections will be shared between the modules running in +
-each process. ​ Therefore, the following conventions apply for +
-connections obtained from ''​nav.db.getConnection()'':​ +
- +
-  * **Do not, under any circumstances**,​ retain references to a database +
-    connection between client requests. ​ Make sure to retrieve a new +
-    connection at the start of each request cycle - the API will cache +
-    connections between requests, and will automagically re-open +
-    broken connections. ​ As the connection is shared between several +
-    modules, retained references may be invalid in the next request +
-    cycle. +
-  * **Do not explicitly close database connections.** ​ Although the API +
-    will try to reopen any closed or broken connections,​ you create +
-    extra overhead, and you don't play nice with the other web +
-    modules. +
-  * **The obtained connections will use an isolation level of //read +
-    committed//​**,​ i.e. no autocommits. ​ Be careful to commit the +
-    current transaction if you modify any data.  A mod_python +
-    cleanuphandler will try to automatically commit all open +
-    transactions as the request cycle ends, but this may change +
-    in the future, so you must not rely on it. +
-  * **Do not change the isolation level of a connection** without +
-    restoring it to its original value before the end of the request +
-    cycle. +
- +
-====== Writing new web code ====== +
-If you are writing a new web application / tool for NAV, please use +
-the Django framework. ​ [[devel:​django_introduction|Here'​s a quick +
-primer]]. +
- +
-====== Version Control ====== +
-NAV uses [[http://​www.selenic.com/​mercurial/​|Mercurial]] for +
-distributed version control. ​ Official repositories are located at +
-http://​metanav.uninett.no/​hg/​ . +
- +
-===== Guide to the repository jungle ===== +
-The official repositories represent three types of branches +
- +
-==== Unstable (default) ==== +
-New, bleeding edge development occurs on the +
-//​[[http://​metanav.uninett.no/​hg/​default/​|default]]//​ branch, which is +
-considered unstable (although we try to always keep it buildable). +
- +
-==== Feature branches ==== +
-New features that take a while (and a lot of changesets) to implement +
-and test will often be published as separate feature branches. ​ For +
-all intents and purposes, the feature branches will look like the +
-//default// branch with some added feature. ​ They will merge changes +
-from the //default// branch regularly. ​ Once a feature is considered +
-"​ready",​ the feature branch will be merged onto the default branch. +
- +
-==== Stable (series) ==== +
-Once we are nearing a new series release of NAV (such as 3.5 or 3.6), +
-a new [[http://​metanav.uninett.no/​hg/​series/​|series branch]] is +
-created from the //default// branch. ​ Once this branch is stabilized,​ +
-the first version is tagged and released. ​ After this point, we accept +
-only bug fixes in this branch. ​ Further point releases in this series +
-are tagged on this branch, and all changes are merged back onto the +
-//default// branch. +
- +
-When someone writes a patch for a bug, this should usually be +
-committed to the latest active series branch which is affected by the +
-bug.  Once a new series is released, we do not usually maintain the +
-older series branches. ​ We may push bug fixes to these branches, but +
-we are unlikely to create a new point release from it. +
- +
-===== Push access ===== +
-Push access to the official repositories is limited to developers +
-employed or commisioned by UNINETT. +
- +
-====== Submitting patches ====== +
-Unless you are submitting one-off fixes for bugs and small issues, +
-please take the time to discuss your change proposals on the +
-//nav-dev// mailing list.  This will increase the chances of having +
-your patches accepted. +
- +
-Base your patches on the relevant Mercurial branches. ​ If you are +
-submitting a patch for an issue that affects the latest stable series, +
-base your patch on that series branch. ​ If you are submitting patches +
-containing new features, base them on the default branch. +
- +
-There are three common options for submitting patches: +
- +
-  * The best way to submit your patches would be using Mercurial. Publish +
-    your own Mercurial branch, and mail its URL to the //nav-dev// mailing +
-    list. +
-  * If unable to host a public Mercurial branch, export your changes +
-    as a Mercurial bundle and attach it to an email addressed to the +
-    //nav-dev// mailing list.   +
-  * If you have a single patch to submit, attach it to an email +
-    addressed to the //nav-dev// mailing list.  Please **do not +
-    patchbomb** the mailing list with multiple emails. +
devel/hacking.1243414429.txt.gz · Last modified: 2009/05/27 08:53 by morten

Page Tools

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki