====== Introduction to using Django with NAV ====== Since the release of 3.4.0 parts of NAV have been using the Python web framework [[http://www.djangoproject.com/|Django]], starting with the new IP Device Info app. This introduction to using Django with NAV assumes that the reader is familiar with Django, i.e. have read the [[http://www.djangoproject.com/documentation/tutorial01/|Django tutorial]] and knows his way around the [[http://www.djangoproject.com/documentation/|Django documentation]]. ===== nav.django package ===== As NAV has not used Django from the very beginning, NAV does not strictly follow the Django convention of multiple apps with their own models, views, etc. To plug NAV into the Django framework some glue is needed. This glue and other common Django-related code is located in the ''nav.django'' Python package, which is located in ''subsystem/lib-python/nav/django''. ==== Settings ==== The usual Django settings file is located at ''nav.django.settings''. The end-users do not need to modify this settings file when deploying NAV, as all options are derived from existing NAV configuration in the files ''nav.conf'' and ''db.conf''. Note that there is no ''INSTALLED_APPS'' setting. NAV does not use this setting at all, due to a different file organization than what Django expects. Following from this, Django does not have a concept of what is an app in NAV. An app is just defined in the minds of the developers. This also means that NAV to a very little degree may take advantage of the ''django-admin.py'' executable, including features like ''syncdb'' for creating database tables for new apps. ==== URL configuration ==== The root URL configuration for all Django-related applications in NAV is located in ''nav.django.urls''. It is not only a module, but also a package. The ''%%__init__.py%%'' file in the package imports all submodules it can find in the directory, calls ''get_urlpatterns()'' on all the submodules, and combines the results into one ''urlpatterns'' list, as all Django URL configurations does. When creating a new Django app, create a new submodule for your app in the ''subsystem/your_app/nav/django/urls/'' folder, run ''make install'' in your app to install the module into the ''nav.django.urls'' package, and the NAV installation will find your app. If somebody else installs URLconf glue into ''nav.django.urls'' they will not affect the URL configuration for your app, unless they use exactly the same name for the submodule as you did. For details, check out the ''ipdevinfo'' subsystem and its ''Makefile.in''. === nav.django.urls.urlbuilder === There is one special submodule named ''nav.django.urls.urlbuilder''. This is simply a Django replacement for the archaic ''nav.web.urlbuilder''. It is used by Django apps to link to non-Django apps, like the report subsystem, through the use of the ''url'' template tag or the ''reverse()'' function, instead of using the archaic ''nav.web.urlbuilder''. From your code's point of view, the URL configurations in ''nav.django.urls.urlbuilder'' is used as any other Django URL configuration in NAV. ==== Shortcuts ==== Since before Django, NAV has been using the Cheetah template system. To enable the use of Django templates for Django apps, while still integrating with the existing template hierarchy of NAV, some shortcut functions have been created in ''nav.django.shortcuts''. At the time of writing, the shortcuts are ''render_to_response()'', ''object_list()'', and ''object_detail()''. ''render_to_response()'' is analogous to the well known ''django.shortcuts.render_to_response()'', and ''object_list()'' and ''object_detail()'' are analogous to functions in ''django.views.generic.list_detail''. The difference between the original functions from Django and the ones provided in ''nav.django.shortcuts'' is that the NAV versions take an additional first argument, namely ''cheetah_template_func''. ''cheetah_template_func'' is assumed to be a Cheetah template function, which returns a Cheetah template with a ''content_string'' variable. The shortcuts takes the content which Django normally would have returned, and inserts it into the ''content_string'' variable of the Cheetah template. In other words, a Django template is rendered as usual, and then the result are wrapped into a Cheetah template. ==== Context processors ==== At the time of writing, ''nav.django.context_processors'' only contains one context processor: ''debug''. If Django is run in debug mode, this context processor appends a ''sql_queries'' variable to the context of all templates, containing all SQL queries executed to generate the current page. This is useful for optimizing the use of the Django ORM. ===== nav.models package ===== The Django convention places models in the apps where they belong the most. In NAV, all models are located centrally in the ''nav.models'' package. Usually, one creates the models first and then generates the database from the models using the previously mentioned ''syncdb'' feature. Since NAV existed long before Django, our models was created for an existing legacy database through the use of Django's database introspection and numerous hours of hard manual labor to clean up the models, adding ''%%__unicode__()%%'' methods, etc. ==== Modules ==== The models was split in as many modules as possible, only limited by interdependencies between the models. This resulted in the following seven modules: * ''cabling'': models for mapping switch ports to physical rooms. * ''event'': models related to event and alert queues. * ''manage'': all models from the old ''manage'' database which were not possible to split out to own modules, including the most important models like ''Netbox'', ''SwPort'', and ''GwPort''. * ''msgmaint'': models related to the message and maintenance subsystems. * ''oid'': models related to OID mappings. * ''rrd'': models related to RRD files and data sources. * ''service'': models related to services. Of these the ''manage'' module is by far the most used and probably of the highest quality. All users of the models should have a look through their definitions to be familiar with how they are mapped to the legacy database using the ''db_column'' and ''db_table'' arguments, and what methods like ''get_absolute_url()'' and ''%%__unicode__()%%'' are available to the user. ==== DB API ==== In ''%%nav.models.__init__%%'' the ''DJANGO_SETTINGS'' environment variable is set to ''nav.django.settings''. As a result of this, one can simply import the models one wants to work with and just use them, without regard to database connections, etc. An example follows: >>> from nav.models.manage import Netbox >>> Netbox.objects.filter(sysname__endswith='uninett.no').count() 189L >>> n = Netbox.objects.filter(sysname__endswith='uninett.no')[0] >>> n >>> n.up_since datetime.datetime(2008, 5, 30, 15, 36, 50, 350000) >>> n.organization >>> ===== The workings of a Django app in NAV ===== To create a new NAV tool/Django app, the easiest approach is probably to start with copying an existing app, like ''subsystem/ipdevinfo/''. This section explains what the different parts of the ''ipdevinfo'' app is, and how it is plugged into the rest of NAV, by following the life cycle of a HTTP request and response. ==== File structure ==== Using the ''ipdevinfo'' app as an example. The ''nav/'' folder contains all runnable Python code, and nothing else. It is installed to the Python path. ''nav/web/ipdevinfo/'' (''nav.web.ipdevinfo'' Python package after installation) contains all the Django code for the app, including views, forms, etc., but not models, as they are centralized in ''subsystem/lib-python'' in the ''nav.models'' package. The URLconf for the apps URL scope is also located in ''nav/web/ipdevinfo/''. For Django to find it, it must be referenced in the root URLconf as mentioned in the ''nav.django'' section above. In addition ''nav/web/templates/'' contains a Cheetah template (not runnable Python code, but it will be compiled to a runnable Python file during installation) which is used to wrap around the Django templates. The Django templates themselves are not runnable Python code, but HTML files with some additional syntax, and are thus available directly in the ''templates/'' folder, not within the ''nav/'' folder. The Django templates are installed to a ''templates/'' folder next to the ''python/'' folder where all installed Python code goes. Typically this is ''/usr/local/nav/lib/templates/''. Further, the file ''ipdevinfo.tool'' defines the name, description, icon and URL of the IP Device Info tool in NAV's toolbox. The file ''htaccess'' is the only file which is installed into the ''ipdevinfo/'' folder in the web server's document root. It simply states that all URLs starting with ''http://nav.example.com/ipdevinfo/'' are to be handled by a Python program, namely Django's mod_python handler, using ''nav.django.settings'' as configuration. In other words, the htaccess file should be identical for all Django apps in NAV. Finally, the folder ''media/'' contains all static media which is needed by the app. Typically, this should be ''media/style/'' for CSS-files, ''media/js/'' for JavaScript, and ''media/images/'' for images. Files should typically be named after the app to avoid name collisions with other apps when installed, i.e. ''media/style/ipdevinfo.css'' and subfolders like ''media/images/ipdevinfo/''. Configuration files should be in the ''config/'' folder in the source tree, and installed into ''$NAVHOME/etc/appname/''. How to install everything mentioned here is defined in ''Makefile.in''. ==== Request/response life cycle ==== - A user enters the URL http://nav.example.com/ipdevinfo/ in her browser, which then issues a HTTP request for the URL. - Apache receives the request, looks up the ''/ipdevinfo/'' subfolder within its document root which contains the mentioned ''.htaccess'' file stating that this URL scope is controlled by Django using ''nav.django.settings'' as configuration. - The request is passed to Django's mod_python handler, which uses the root URLconf ''nav.django.urls'', as configured in ''nav.django.settings'', to identify what Django view should process the request. - During lookup in ''nav.django.url'', ''nav.web.ipdevinfo.urls'' is identified as responsible for the ''/ipdevinfo/'' URL scope. - The matching part of the URL, that is ''/ipdevinfo/'' is stripped and the rest is passed to ''nav.web.ipdevinfo.urls'' for further lookup. - As the URL now is empty, it matches the ''^$'' pattern in ''nav.web.ipdevinfo.urls'' and the ''search'' view, as imported from ''nav.web.ipdevinfo.views'' is called. - The ''search'' view does its job, i.e. either creating a search form, or if a query already has been issued, looking up IP devices matching the query. - Finally, the view calls ''nav.django.shortcuts.render_to_response()'' with the following arguments: a Cheetah template, a Django template, the form, the query string, the search results and a list of error mesages if any. - ''render_to_response()'' uses Django's own ''render_to_response()'' implementation to render the Django template. Then the result is inserted into the ''content_string'' variable of the given Cheetah template. Finally, the result of the Cheetah template is extracted and inserted into a Django ''HttpResponse'', which is returned to the ''search()'' view. - The ''search()'' view returns the ''HttpResponse'' object which it gets from ''render_to_response()''. - The response is passed to the end user. In addition to the steps mentioned here, Django middleware may plug into the life cycle and do work at inbetween most of these steps, except those happening inside the view function. This is not NAV specific in any way, so the reader is referenced to the Django documentation for futher details. ===== Future enhancements ===== The integration between NAV and Django is rather new and basic. Given increased usage of Django in NAV one should strive to follow Django's conventions and, where applicable, use more of Django's capabilities instead of NAV's old homebrewn solutions. All NAV developers are encouraged to sketch out ideas for improvements below. ==== (DONE) Upgrading to Django 1.0 ==== Django 1.0 is targeted for release September 2 2008. To jump from the version NAV are currently using (SVN r7534) to 1.0, some changes have to be made to the Django code in NAV. [[http://code.djangoproject.com/wiki/BackwardsIncompatibleChanges]] contains details of the about 30 backwards incompatible changes done to Django between our current version and Django 1.0. IMHO, we should do this jump before the release of NAV 3.5.0 in September 2008. This was done in [[https://bugs.launchpad.net/nav/+bug/260554|LP#260554]]. ==== mod_python and Django sessions ==== NAV uses sessions for keeping track of logged in users. Currently, we are not using ''django.contrib.sessions'', but are still using a session library developed specifically for NAV. This session library attaches session objects to ''mod_python'''s request objects. For now, the easiest way to get hold of the NAV session in views is to use ''request._req.session'', where ''request._req'' is the traditional ''mod_python'' request object. In the future, one should look into using Django's authentication and session frameworks. ==== Base template using Django templates ==== As more parts of NAV are (re)implemented in Django, one should look into replacing the wrapping of Django templates in Cheetah templates with pure Django templates throughout the stack. Even if this may require maintenance of base templates in both Cheetah and Django for quite some time, it would result in much more flexibility with regard to page titles, bread crumb paths, etc. when using Django templates. Proposal in [[devel:blueprints:django_template]] ==== Unit testing ==== NAV should certainly start using unit testing. Tests for ''nav.web.ipdevinfo'' could for example go into ''nav.web.ipdevinfo.tests'', but this should be synchronized with Django conventions, and one must also find a way to run the tests, most probably through a custom test runner or something. The point is, one needs to give this some thought, and the sooner the better. New Django apps in NAV should really have unit tests from the start.