This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision | ||
devel:hacking [2009/02/09 14:22] morten |
devel:hacking [2014/11/05 10:57] (current) morten |
||
---|---|---|---|
Line 1: | Line 1: | ||
- | :!: **This information is currently being updated, check back often.** | ||
====== Hacker's guide to NAV ====== | ====== Hacker's guide to NAV ====== | ||
- | If you are contributing code to Network Administration Visualized, | + | This document has moved to https://nav.uninett.no/doc/dev/hacking/hacking.html |
- | please read this first. | + | |
- | + | ||
- | + | ||
- | ===== Participating in the community ===== | + | |
- | Originally, NAV was a closed source project, initiated by the | + | |
- | Norwegian University of Science and Technology (NTNU), and eventually | + | |
- | sponsored by UNINETT on behalf of the Norwegian higher education | + | |
- | community. In 2004, however, NTNU and UNINETT started distributing | + | |
- | NAV under the GNU General Public License, making it a truly free | + | |
- | software system. | + | |
- | + | ||
- | While NTNU and UNINETT are still the main contributors to NAV, | + | |
- | developing NAV to support the needs of the Norwegian higher education | + | |
- | community, contributions from third parties is highly appreciated. | + | |
- | + | ||
- | We communicate mainly through mailing lists, | + | |
- | [[https://launchpad.net/nav/|Launchpad]], and the wiki and Mercurial | + | |
- | repositories hosted at http://metanav.uninett.no/ . At times, UNINETT | + | |
- | also arranges workshops and gatherings for its customers: Norwegian | + | |
- | universities, university colleges and research institutions. | + | |
- | + | ||
- | To contribute: | + | |
- | + | ||
- | Go to http://metanav.uninett.no/ and | + | |
- | + | ||
- | * Join the mailing lists. The //nav-dev// mailing list in | + | |
- | particular is for discussing NAV development. So far, this is a | + | |
- | low traffic list. We can only hope this will change ;-) | + | |
- | * Get a copy of the latest development sources by cloning the | + | |
- | Mercurial repository at http://metanav.uninett.no/hg/default/. | + | |
- | Most new development takes place on this branch. | + | |
- | * Take a look at the [[:navprojects|project reports from previous | + | |
- | development projects at NTNU]] (NAVMe, NAVMore, tigaNAV and | + | |
- | others) - design specifications and other useful bits of historic | + | |
- | NAV information is mostly to be found in these. Unfortunately, | + | |
- | some of the oldest project documentation is in Norwegian only. Do | + | |
- | not hesitate to ask for help on the mailing lists. | + | |
- | + | ||
- | If you wish to contribute code to the project, please tell us about it | + | |
- | on the nav-dev mailing list. It is always a good idea to check if | + | |
- | someone is already working on something similar, and to get some | + | |
- | helpful tips on how to integrate your code with the rest of the | + | |
- | project. If you already went ahead and wrote a patch, announce it on | + | |
- | nav-dev and provide a link to the patch so it can be studied for | + | |
- | possible inclusion into NAV. | + | |
- | + | ||
- | ===== Directory layout ===== | + | |
- | A rough guide to the source tree: | + | |
- | + | ||
- | | conf/ | Files related to the autoconf build system. | | + | |
- | | doc/ | User and developer documentation, SQL scripts and example NAV configuration files. | | + | |
- | | tools/ | Tool scripts for the build and release processes. | | + | |
- | | contrib/ | User contributed NAV tools. NAV doesn't depend on these, and any maintenance of them is left up to the original developers. We do not offer support for these tools. | | + | |
- | | packages/ | Stuff to help packaging NAV for various platforms, such as RedHat, CentOS, FreeBSD, Debian and soforth. | | + | |
- | | src/ | Source code to Java subsystems of NAV (here for historic reasons). | | + | |
- | | subsystem/ | Source code to the rest of NAV - most of it Python. NAV is loosely divided into subsystems, and each one of these has its own subdirectory in here. | | + | |
- | | subsystem/lib-python/ | Python libraries & APIs. Please check what's already there before you roll your own. | | + | |
- | | subsystem/lib-perl/ | Perl libraries & APIs. Only a single piece of Perl code remains in NAV; once this has been replaced, this directory will cease to exist. | | + | |
- | | subsystem/webfront/ | Python libraries for the web interface and front-page handler modules for mod_python. | | + | |
- | + | ||
- | + | ||
- | ===== Development languages and frameworks ===== | + | |
- | For historic reasons, different parts of NAV are written in different | + | |
- | programming languages (Perl, Java, PHP and Python). This has been | + | |
- | unfortunate in may ways, not at least for the sake of consistency and | + | |
- | maintenance, but fortunately we have a long-term goal of reducing the | + | |
- | number of languages and dependencies. The last few years we've spent | + | |
- | a significant amount of time rewriting parts that were written in Perl | + | |
- | and PHP to Python, which is the language we are currently gravitating | + | |
- | towards. | + | |
- | + | ||
- | Currently (as of February 2009), major parts of NAV are written in | + | |
- | Python and Java, while only a single Perl program remains | + | |
- | (''makecricketconfig.pl''). | + | |
- | + | ||
- | When contributing patches to existing code, or plugins to existing | + | |
- | subsystems, use the language that subsystem was written in. | + | |
- | + | ||
- | When writing entirely new subsystems, the following rules apply: | + | |
- | + | ||
- | * If your subsystem is a new tool for the web interface, use Python | + | |
- | and Django. The legacy parts of the web system interface directly | + | |
- | with ''mod_python'', using Cheetah for HTML templating. Anything | + | |
- | written in 2008 and later uses Django. There is also | + | |
- | [[devel:django_introduction|a guide for interfacing Django | + | |
- | applications with the legacy web code]]. | + | |
- | + | ||
- | * If your subsystem is a new back-end tool/daemon, please use | + | |
- | Python. The NAV Python API is more complete than for any of the | + | |
- | other languages, and you will receive a lot for free. | + | |
- | + | ||
- | If these rules are not followed, your patches will not be accepted | + | |
- | into NAV (but if they are really good, we will consider including them | + | |
- | in the //contrib// directory). | + | |
- | + | ||
- | If **YOU** are willing to invest in porting some of the existing | + | |
- | Java/Perl code to Python, then you will be celebrated as a NAV hero! | + | |
- | + | ||
- | + | ||
- | ===== Coding style ===== | + | |
- | Much of the legacy NAV code was written without using any coding style | + | |
- | guidelines. This has resulted in some chaotic combination of styles, | + | |
- | which we hope to reduce in the future. For new code, please follow | + | |
- | these guidelines: | + | |
- | + | ||
- | * For Java code, please refer to SUN's "Code conventions for the | + | |
- | Java Programming Language": http://java.sun.com/docs/codeconv/ | + | |
- | * For Python code, please refer to PEP-8, "Style Guide for Python | + | |
- | Code" http://www.python.org/doc/peps/pep-0008/ | + | |
- | + | ||
- | FIXME Add info about header guidelines for Python files. | + | |
- | + | ||
- | If you see violations of these guidelines, don't hesitate to fix them. | + | |
- | If you fix file-wide indentation problems etc., please submit this as | + | |
- | a separate patch to make your other patches look clean and readable. | + | |
- | + | ||
- | + | ||
- | ===== Database connections ===== | + | |
- | NOTE: The following is Python-specific, more info should be added for | + | |
- | the other languages used in NAV. | + | |
- | + | ||
- | NAV uses PostgreSQL as its database backend. Namespaces (schemas) are | + | |
- | employed to logically group tables and relations. NAV versions prior | + | |
- | to 3.5 employed separate PostgreSQL databases instead of namespaces. | + | |
- | + | ||
- | The namespaces currently in use are: | + | |
- | + | ||
- | | ''manage'' | The core knowledge database of NAV, containing all sorts of information about the monitored IP Devices, events, alerts, network topology and machine tracking data. | | + | |
- | | ''profiles'' | Contains NAV user accounts and groups, user preferences and alert profiles. | | + | |
- | | ''logger'' | Anything related to NAV's syslog parser/browser system. | | + | |
- | | ''arnold'' | The port detention system Arnold stores it's data here. | | + | |
- | + | ||
- | To obtain a connection to the NAV database, use the API accordingly, | + | |
- | e.g.: | + | |
- | + | ||
- | <code python> | + | |
- | import nav.db | + | |
- | # Get a connection to the NAV database | + | |
- | connection = nav.db.getConnection('default') | + | |
- | </code> | + | |
- | + | ||
- | The above code will open a connection to NAV's database, or, if a | + | |
- | previous connection with these parameters is already open, returns the | + | |
- | already existing connection from a connection cache. | + | |
- | + | ||
- | The ''default'' parameter is there for legacy reasons; it specifies | + | |
- | the name of a subsystem. The ''db.conf'' file allows configuration of | + | |
- | separate database users for each subsystem (known as a ''script'' in | + | |
- | ''db.conf'') of NAV. The default ''db.conf'' file specifies a | + | |
- | database user for a subsystem called ''default'', and also specifies | + | |
- | the same database user for all known subsystem names. At present, | + | |
- | using a subsystem name that is not configured in ''db.conf'' will | + | |
- | raise an exception in ''nav.db.getConnection()''. | + | |
- | + | ||
- | ===== Legacy web code ===== | + | |
- | When making changes to NAV's legacy web code, a few special | + | |
- | considerations need to be made. | + | |
- | + | ||
- | Legacy web code interfaces directly with | + | |
- | [[http://www.modpython.org/|mod_python]], and uses | + | |
- | [[http://www.cheetahtemplate.org/|Cheetah for HTML templating]]. | + | |
- | + | ||
- | ==== Cheetah templates ==== | + | |
- | Global Cheetah templates are found in | + | |
- | ''subsystem/webfront/nav/web/templates/'', while local templates are | + | |
- | located in the subdirectories of their respective subsystems. All | + | |
- | Python modules compiled from Cheetah templates should be installed in the | + | |
- | ''nav.web.templates'' package. | + | |
- | + | ||
- | + | ||
- | ==== Database connections ==== | + | |
- | Use the ''nav.db.getConnection()'' call to open or retrieve an | + | |
- | existing database connection. All NAV web modules share the same | + | |
- | interpreter and namespace per Apache process, which also means that | + | |
- | database connections will be shared between the modules running in | + | |
- | each process. Therefore, the following conventions apply for | + | |
- | connections obtained from ''nav.db.getConnection()'': | + | |
- | + | ||
- | * **Do not, under any circumstances**, retain references to a database | + | |
- | connection between client requests. Make sure to retrieve a new | + | |
- | connection at the start of each request cycle - the API will cache | + | |
- | connections between requests, and will automagically re-open | + | |
- | broken connections. As the connection is shared between several | + | |
- | modules, retained references may be invalid in the next request | + | |
- | cycle. | + | |
- | * **Do not explicitly close database connections.** Although the API | + | |
- | will try to reopen any closed or broken connections, you create | + | |
- | extra overhead, and you don't play nice with the other web | + | |
- | modules. | + | |
- | * **The obtained connections will use an isolation level of //read | + | |
- | committed//**, i.e. no autocommits. Be careful to commit the | + | |
- | current transaction if you modify any data. A mod_python | + | |
- | cleanuphandler will try to automatically commit all open | + | |
- | transactions as the request cycle ends, but this may change | + | |
- | in the future, so you must not rely on it. | + | |
- | * **Do not change the isolation level of a connection** without | + | |
- | restoring it to its original value before the end of the request | + | |
- | cycle. | + | |
- | + | ||
- | ===== Writing new web code ===== | + | |
- | If you are writing a new web application / tool for NAV, please use | + | |
- | the Django framework. [[devel:django_introduction|Here's a quick | + | |
- | primer]]. | + |