This shows you the differences between two versions of the page.
Both sides previous revision Previous revision Next revision | Previous revision Next revision Both sides next revision | ||
devel:hacking [2011/02/03 13:33] morten [Running tests] |
devel:hacking [2012/11/14 13:27] norangshol [Javascript] fixed a typo: browser -> browsers |
||
---|---|---|---|
Line 25: | Line 25: | ||
To contribute: | To contribute: | ||
- | Go to http://metanav.uninett.no/ and | + | Go to http://nav.uninett.no/ and |
* Join the mailing lists. The //nav-dev// mailing list in | * Join the mailing lists. The //nav-dev// mailing list in | ||
Line 31: | Line 31: | ||
low traffic list. We can only hope this will change ;-) | low traffic list. We can only hope this will change ;-) | ||
* Get a copy of the latest development sources by cloning the | * Get a copy of the latest development sources by cloning the | ||
- | Mercurial repository at http://metanav.uninett.no/hg/default/. | + | Mercurial repository at http://nav.uninett.no/hg/default/. |
Most new development takes place on this branch. | Most new development takes place on this branch. | ||
* Take a look at the [[:navprojects|project reports from previous | * Take a look at the [[:navprojects|project reports from previous | ||
Line 66: | Line 66: | ||
long-term goal to rewrite the remaining Java backend code to Python. | long-term goal to rewrite the remaining Java backend code to Python. | ||
- | Currently (as of February 2011), NAV consists mostly of Python code, | + | Currently (as of September 2012), NAV consists mostly of Python code, |
- | with a few remaining backend systems written in Java (there's also the | + | with one remaining backend systems written in Java (eventEngine). |
- | Netmap Java Applet). | + | |
* We will only accept new code written in Python (except when it | * We will only accept new code written in Python (except when it | ||
Line 126: | Line 125: | ||
</code> | </code> | ||
- | ====== Database connections ====== | + | ===== Javascript ===== |
+ | |||
+ | When writing javascript code try to focus on modules not pages. If the code is html-related, it should take selectors or objects as input and concern itself solely about those. This makes for much easier testing and reuse. And of course - write the tests first. | ||
+ | |||
+ | When the module is done you write a controller for the page that plugs the needed plugins to the page elements. This should fail gracefully if the needed elements are not present. | ||
+ | |||
+ | NAVs javascript uses [[http://requirejs.org/|require.js]] - use this to create modules and specify dependencies. | ||
+ | |||
+ | Pro tip is to create ''require_config.dev.js'' in ''media/js/'' and add the following configuration to requirejs: | ||
+ | <code>require.urlArgs = "bust=" + (new Date()).getTime();</code> This makes sure your not using cached resources in your browser when developing, which browsers loves to do! See [[http://requirejs.org/docs/api.html#config-urlArgs|config-urlArgs]] in requirejs documentation for «details». The ''require_config.dev.js'' is added in global HG ignore. | ||
+ | ====== Database ====== | ||
NAV uses PostgreSQL as its database backend. Namespaces (schemas) are | NAV uses PostgreSQL as its database backend. Namespaces (schemas) are | ||
employed to logically group tables and relations. NAV versions prior | employed to logically group tables and relations. NAV versions prior | ||
Line 139: | Line 148: | ||
| ''radius'' | Radius accounting logs, updated directly by FreeRadius' PostgreSQL module. | | | ''radius'' | Radius accounting logs, updated directly by FreeRadius' PostgreSQL module. | | ||
- | **NOTE**: The following is Python-specific, more info should be added for | + | ===== Connecting to the database (Python) ==== |
- | the other languages used in NAV. | + | ==== Raw SQL ==== |
- | + | ||
- | + | ||
- | ===== Raw SQL ===== | + | |
To obtain a connection to the NAV database, use the API accordingly, | To obtain a connection to the NAV database, use the API accordingly, | ||
Line 166: | Line 172: | ||
using a subsystem name that is not configured in ''db.conf'' will cause ''nav.db.getConnection()'' to revert to using the ''default'' name. | using a subsystem name that is not configured in ''db.conf'' will cause ''nav.db.getConnection()'' to revert to using the ''default'' name. | ||
- | ===== Django models ===== | + | ==== Django models ==== |
NAV 3.5 and on includes Django models for most database tables. If no | NAV 3.5 and on includes Django models for most database tables. If no | ||
SQL magic is needed to perform your database voodoo, it is recommended | SQL magic is needed to perform your database voodoo, it is recommended | ||
Line 174: | Line 180: | ||
The models are defined in modules of the ''nav.models'' package. | The models are defined in modules of the ''nav.models'' package. | ||
+ | |||
+ | ===== Changing the schema ==== | ||
+ | |||
+ | The baseline schema is located in ''sql/baseline'' - the ''syncdb.py'' script | ||
+ | is responsible for running this when creating a new database. To make a schema | ||
+ | change, you **do not** change the baseline, but go to the ''sql/changes'' | ||
+ | directory and create a new schema change script there. | ||
+ | |||
+ | Schema change scripts as numbered, using the following pattern: | ||
+ | |||
+ | * ''sc.<major>.<minor>.<point>.sql'' | ||
+ | |||
+ | The ''<major>'' and ''<minor>'' numbers usually correspond to the major and | ||
+ | minor number of the next NAV release. The ''<point>'' number is a sequence id | ||
+ | - pick the next free number when creating a schema change script. | ||
+ | |||
+ | Remember these points when creating a schema change script: | ||
+ | |||
+ | * Create separate change scripts for unrelated schema changes. | ||
+ | * Remember to write SQL to //migrate// existing data, if necessary. | ||
+ | * Do not use transactional statements - the ''syncdb.py'' script will take | ||
+ | care of that. | ||
+ | |||
+ | To apply your change scripts, just run ''syncdb.py''. It will look inside the | ||
+ | ''schema_change_log'' table to see which change scripts have already been | ||
+ | applied, and it will detect your new change script and apply this to the | ||
+ | database. | ||
+ | |||
+ | :!: When changing the schema, don't forget to update the Django models in the | ||
+ | ''nav.models'' package. An integration test exists to verify that the Django | ||
+ | models can at least be used to run proper SELECTs against the database. | ||
====== Legacy web code ====== | ====== Legacy web code ====== | ||
Line 247: | Line 284: | ||
NAV uses [[http://www.selenic.com/mercurial/|Mercurial]] for | NAV uses [[http://www.selenic.com/mercurial/|Mercurial]] for | ||
distributed version control. Official repositories are located at | distributed version control. Official repositories are located at | ||
- | http://metanav.uninett.no/hg/ . | + | http://nav.uninett.no/hg/ . |
===== Guide to the repository jungle ===== | ===== Guide to the repository jungle ===== | ||
Line 254: | Line 291: | ||
==== Unstable (default) ==== | ==== Unstable (default) ==== | ||
New, bleeding edge development occurs on the | New, bleeding edge development occurs on the | ||
- | //[[http://metanav.uninett.no/hg/default/|default]]// branch, which is | + | //[[http://nav.uninett.no/hg/default/|default]]// branch, which is |
considered unstable (although we try to always keep it buildable). | considered unstable (although we try to always keep it buildable). | ||
Line 267: | Line 304: | ||
==== Stable (series) ==== | ==== Stable (series) ==== | ||
Once we are nearing a new series release of NAV (such as 3.5 or 3.6), | Once we are nearing a new series release of NAV (such as 3.5 or 3.6), | ||
- | a new [[http://metanav.uninett.no/hg/series/|series branch]] is | + | a new [[http://nav.uninett.no/hg/series/|series branch]] is |
created from the //default// branch. Once this branch is stabilized, | created from the //default// branch. Once this branch is stabilized, | ||
the first version is tagged and released. After this point, we accept | the first version is tagged and released. After this point, we accept | ||
Line 324: | Line 361: | ||
</code> | </code> | ||
- | ===== Hudson/Jenkins ===== | ||
- | We use //Hudson// (soon-to-be //Jenkins//) for Continuous Integration | ||
- | of NAV. All the automated tests are run each time new changesets are | ||
- | pushed to the MetaNAV repositories. Hudson also runs pylint to create | ||
- | stats on code quality. | ||
- | Our Hudson installation is available on http://metanav.uninett.no/hudson . | + | ===== Javascript testing ===== |
+ | |||
+ | Testing of javascript is in its infancy in NAV. We are currently using [[http://busterjs.org/|buster.js]] as testing toolkit. | ||
+ | |||
+ | To install buster.js install [[http://nodejs.org/|node]] and then: | ||
+ | <code> | ||
+ | npm install -g buster | ||
+ | </code> | ||
+ | |||
+ | As we use [[http://requirejs.org/|require.js]] you need the AMD module of buster aswell. Install it in the /media/js directory: | ||
+ | <code> | ||
+ | npm install buster-amd | ||
+ | </code> | ||
+ | |||
+ | To run the tests you need to | ||
+ | - Start a buster server by typing ''buster-server'' | ||
+ | - Capture browsers by pointing browsers to the buster-server (default localhost:1111) | ||
+ | - Go to /media/js | ||
+ | - Run the tests by typing ''buster-test'' | ||
+ | |||
+ | All tests are located under ''media/js/tests/''. Create new tests there. For syntax, assertions and related stuff take a look at the tests already there and [[http://busterjs.org/docs/|the buster docs]]. | ||
+ | ===== Jenkins ===== | ||
+ | |||
+ | We use //Jenkins// (formerly //Hudson//) for Continuous Integration testing of | ||
+ | NAV. All the automated tests are run each time new changesets are pushed to | ||
+ | the NAV repositories. Jenkins also runs pylint to create stats on code | ||
+ | quality. | ||
+ | |||
+ | Our Jenkins installation is available on http://nav.uninett.no/jenkins . | ||
+ | |||
+ | ===== Tips and tricks ===== | ||
+ | |||
+ | ===== Make fixtures for integration testing ===== | ||
+ | |||
+ | <code> | ||
+ | from django.core import serializers | ||
+ | from nav.models.manage import Netbox | ||
+ | |||
+ | fixtures = serializers.serialize("xml", Netbox.objects.all()[:2]) | ||
+ | </code> | ||
+ | |||
+ | Fixtures can so be used in your integration tests by extending | ||
+ | the test case DjangoTransactionTestCase in nav.tests.cases | ||
+ | |||
+ | See nav.tests.integration.l2trace_test for an example on applying | ||
+ | fixtures for your particular test case. | ||
+ | |||
+ | Also keep in mind you have to make sure you have the model | ||
+ | dependency in correct order when importing. | ||
+ | Example: Netbox contains a location to a Room where it is located, | ||
+ | you have to make sure Room's are imported first before importing | ||
+ | Netbox's | ||
+ | |||
+ | See https://docs.djangoproject.com/en/dev/topics/serialization/ | ||
+ | |||
+ | TODO: Be able to use [[https://docs.djangoproject.com/en/dev/ref/django-admin/#dumpdata-appname-appname-appname-model|django-admin's management command: dumpdata]] | ||
+ | to create fixtures. | ||
Line 355: | Line 444: | ||
addressed to the //nav-dev// mailing list. Please **do not | addressed to the //nav-dev// mailing list. Please **do not | ||
patchbomb** the mailing list with multiple emails. | patchbomb** the mailing list with multiple emails. | ||
- |