Openpolis Labhttp://lab.openpolis.it/2013-12-09T10:45:00+01:00Writeable Foreign keys in Django Rest Framework2013-12-09T10:45:00+01:00guglielmotag:lab.openpolis.it,2013-12-09:writeable-foreign-keys-in-django-rest-framework.html<p><a class="reference external" href="http://django-rest-framework.org/">Django Rest Framework</a> is a django-based framework to produce browseable APIs.</p>
<p>The documentation and the tutorial are pretty much clear, but implementing a writeable <span class="caps">API</span>
for related fields can be tricky.</p>
<p>RelatedField is read-only.
PrimaryKeyRelatedFields, HyperlinkedRelatedField, SlugRelatedField, can be used as writeable, but the
related items must already exist. It seems they’re more appropriate to handle aggregation relations (i.e. User and Groups)</p>
<p>Nested relationships can be used to handle composition relations (i.e. a Place and its Acronyms).
The quirk is that the get_identity() method must be re-written in the nested serializer, because the default one uses
the id field to determine the identity of the items, which may not have, and which you surely don’t have when creating new items.</p>
<p>A working example with Places and Acronyms can be found <a class="reference external" href="http://gist.github.com/guglielmo/7851650">here</a>.</p>
<p>Given a Place instance with just one acronym, the <span class="caps">JSON</span> representation would be:</p>
<pre class="literal-block">
{
"_self": "http://localhost:8003/maps/places/roma-comune",
"slug": "roma-comune",
"name": "Roma",
"acronyms": [
"RM"
]
}
</pre>
<p>PUTting the following json payload to the resource:</p>
<pre class="literal-block">
{
"slug": "roma-comune",
"name": "Roma",
"acronyms": [
"RM", "ROM"
]
}
</pre>
<p>or PATCHing the even shorter one:</p>
<pre class="literal-block">
{
"acronyms": [
"RM", "ROM"
]
}
</pre>
<p>The final result would be that the <strong><span class="caps">ROM</span></strong> acronym would be added to the Place resource.</p>
Using Mapnik inside a virtualenv on OSX2013-10-20T10:30:00+02:00guglielmotag:lab.openpolis.it,2013-10-20:using-mapnik-inside-a-virtualenv-on-osx.html<p><a class="reference external" href="https://github.com/mapnik/mapnik/wiki/Mapnik2">Mapnik</a> is a tool to produce maps out of gis data.</p>
<p>Since installation in a virtualenv through <tt class="docutils literal">pip install mapnik2</tt> fails, due to compilation
problems (boost library cannot be located correctly), the following workaround can be used.</p>
<ul>
<li><p class="first">Install Mapnik through macports:</p>
<pre class="literal-block">
sudo port install mapnik @2.2.0_0+python27
</pre>
<p>This will install both the libraries and the python bindings, as global packages in:</p>
<pre class="literal-block">
/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/
</pre>
</li>
<li><p class="first">Link mapnik directories into the virtualenv’s site_package directory:</p>
<pre class="literal-block">
workon <virtualenv>
cdsitepackages
ln -s /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mapnik .
ln -s /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/mapnik2 .
</pre>
</li>
</ul>
Using Tkinter inside a virtualenv2013-10-19T10:30:00+02:00guglielmotag:lab.openpolis.it,2013-10-19:using-tkinter-inside-a-virtualenv.html<p><a class="reference external" href="https://wiki.python.org/moin/TkInter">Tkinter</a> is usually installed on <span class="caps">OSX</span> trhough <a class="reference external" href="http://www.macports.org/">macports</a> as a global package:</p>
<pre class="literal-block">
sudo port install py27-tkinter
</pre>
<p>Within a virtualenv the package will not be imported, as global packages are not included by default:</p>
<pre class="literal-block">
>>> import Tkinter
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk/Tkinter.py", line 39, in <module>
import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named _tkinter
</pre>
<p>The package consists of a single library (_tkinter.so) that can be <em>installed</em> in the virtual environment,
just by symlinking it into the virtualenv’s site_package directory:</p>
<pre class="literal-block">
workon <virtualenv>
cdsitepackages
ln -s /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/_tkinter.so .
</pre>
<p>Yes, it’s kind of an hack, but it works.</p>
Fixing the “can’t start the AWT” bug for Tomcat62013-06-25T14:30:00+02:00guglielmotag:lab.openpolis.it,2013-06-25:fixing-the-cant-start-the-awt-bug-for-tomcat6.html<p>After a Java upgrade on the <span class="caps">OSX</span>, Tomcat6 is not starting properly. The log shows a Java error:</p>
<pre class="code literal-block">
2013-06-27 14:45:40.300 jsvc[57146:203] Apple AWT Java VM was loaded on first thread -- can't start AWT.
Jun 27, 2013 2:45:40 PM org.apache.catalina.startup.Bootstrap initClassLoaders
SEVERE: Class loader creation threw exception
java.lang.InternalError: Can't start the AWT because Java was started on the first thread. Make sure StartOnFirstThread is not specified in your application's Info.plist ...
</pre>
<p>To have it properly start through <tt class="docutils literal">tomcatctl</tt>, you need to add the <tt class="docutils literal">awt.headless</tt> option to <tt class="docutils literal">JAVA_OPTS</tt>.
This can be done, by modifying the <tt class="docutils literal">/opt/local/share/java/tomcat6/conf/setenv.local</tt> file,
and appending this line to it:</p>
<pre class="literal-block">
export JAVA_OPTS="$JAVA_OPTS -Djava.awt.headless=true"
</pre>
How to configure bash completion in ubuntu2013-04-11T16:30:00+02:00guglielmotag:lab.openpolis.it,2013-04-11:how-to-configure-bash-completion-in-ubuntu.html<p>Working almost always in the terminal of a linux box, the <em>autocompletion</em> feature enhances your speed and comfort, a lot.</p>
<p>Under Ubuntu, autocompletion is installable, if not already installed, with:</p>
<pre class="literal-block">
apt-get install bash_completion
</pre>
<p>The <tt class="docutils literal"><span class="pre">~/.bashrc</span></tt> config file should contain these lines:</p>
<pre class="literal-block">
if [ -f /etc/bash_completion ] && ! shopt -oq posix; then
. /etc/bash_completion
fi
</pre>
<p>If things are set this way, then you will be able to autocomplete regularly (and save a lot of time).</p>
<dl class="docutils">
<dt>Note:</dt>
<dd>In <tt class="docutils literal">/etc/bash_completion.d</tt> you can find the rather cryptic scripts enabling autocomplete patterns inside various unix contexts.</dd>
</dl>
How to install solr on tomcat7, over Ubuntu 12.042013-04-10T15:30:00+02:00guglielmotag:lab.openpolis.it,2013-04-10:how-to-install-solr-on-tomcat7-over-ubuntu-1204.html<div class="section" id="tomcat7">
<h2>Tomcat7</h2>
<p>Ubuntu 12.04 comes with <a class="reference external" href="http://tomcat.apache.org/">tomcat</a> easily installable as a package (you need jdk):</p>
<pre class="literal-block">
apt-get install default-jdk tomcat7 tomcat7-admin tomcat7-user tomcat7-docs
</pre>
<p>Configure access to the web-admin in <tt class="docutils literal"><span class="pre">/etc/tomcat7/tomcat-users.xml</span></tt>:</p>
<pre class="literal-block">
<tomcat-users>
<role rolename="manager-gui"/>
<user username="manager" password="S3krEt" roles="manager-gui"/>
</tomcat-users>
</pre>
<p>The tomcat management console is now accessible, through authentication at <tt class="docutils literal"><span class="pre">http://<span class="caps">HOST</span>:8080/manager/html</span></tt></p>
<p>Please, do change username and password in your <tt class="docutils literal"><span class="pre">tomcat-users.xml</span></tt>.</p>
</div>
<div class="section" id="solr">
<h2>Solr</h2>
<p>It can be useful to have different, isolated instances of <a class="reference external" href="http://lucene.apache.org/solr/">solr</a> running, corresponding to different solr versions
(i.e. 1.2, 1.4, 3.4, …). One of the instances could be a <em>multicore</em> solr instance, able to host multiple indexes under a single app.</p>
<p>This is possible by creating different <strong>contexts</strong> inside <tt class="docutils literal">/etc/tomcat7/Catalina/localhost/</tt>.</p>
<p><strong>open_parlamento.xml</strong>:</p>
<pre class="literal-block">
<?xml version="1.0" encoding="utf-8"?>
<Context docBase="/home/solr/open_parlamento/apache-solr-1.4.0.war" debug="0" crossContext="true" >
<Environment name="solr/home" type="java.lang.String" value="/home/solr/open_parlamento" override="true" />
</Context>
</pre>
<p><strong>open_politici.xml</strong>:</p>
<pre class="literal-block">
<?xml version="1.0" encoding="utf-8"?>
<Context docBase="/home/solr/open_politici/apache-solr-1.2.0.war" debug="0" crossContext="true" >
<Environment name="solr/home" type="java.lang.String" value="/home/solr/open_politici" override="true" />
</Context>
</pre>
<p><strong>multicore.xml</strong>:</p>
<pre class="literal-block">
<?xml version="1.0" encoding="utf-8"?>
<Context docBase="/home/solr/multicore/apache-solr-3.4.0.war" debug="0" crossContext="true" >
<Environment name="solr/home" type="java.lang.String" value="/home/solr/multicore/cores" override="true" />
</Context>
</pre>
<p>Create this directory tree:</p>
<pre class="literal-block">
/home/solr
|- open_parlamento
| |- apache-solr-1.4.0.war
| |- data
| | |- index
| | | |- ...
| |- conf
| |- admin-extra.html
| |- elevate.xml
| |- italian-stopwords.txt
| |- mapping-ISOLatin1Accent.txt
| |- protwords.txt
| |- schema.xml
| |- scripts.conf
| |- solrconfig.xml
| |- spellings.txt
| |- stopwords.txt
| |- synonyms.txt
|- open_polis
| |- apache-solr-1.2.0.war
| |- ...
|- multicore
|- apache-solr.3.4.0.war
|- cores
| |- solr.xml
| |- app1
| | |- conf
| |- app2
| | |- conf
|- data
|- app1
| |- index
|- app2
|- index
</pre>
<dl class="docutils">
<dt>Note:</dt>
<dd>You need to download the different versions of the solr war from their web site, and you may eventually need to
download libraries, as pointed in the <tt class="docutils literal">solrconfig.xml</tt> file to handle advanced tasks, so <em>your mileage may vary</em>.</dd>
</dl>
<p><strong>solr.xml</strong> is the configuration file that points to the various cores in the multicore instance:</p>
<pre class="literal-block">
<?xml version="1.0" encoding="UTF-8" ?>
<solr persistent="false" sharedLib="lib">
<cores adminPath="/admin/cores" shareSchema="true">
<core name="app1" instanceDir="app1" dataDir="${solr.data.dir:../data}/app1" />
<core name="app2" instanceDir="app2" dataDir="${solr.data.dir:../data}/app2" />
</cores>
</solr>
</pre>
<p>The <tt class="docutils literal">conf</tt> directories in <tt class="docutils literal">cores/appX</tt> are standard solr conf directories, with all files needed, according to
the solr version.</p>
</div>
<div class="section" id="permissions">
<h2>Permissions</h2>
<p>Tomcat is run by the <tt class="docutils literal">tomcat7</tt> user, so all the <tt class="docutils literal">data</tt> directories need to be writeable to this user:</p>
<pre class="literal-block">
chgrp -R tomcat7 /home/solr/open_parlamento/data
chmod -R g+w /home/solr/open_parlamento/data
</pre>
</div>
<div class="section" id="restart">
<h2>Restart</h2>
<p>Is needed whenever a tomcat or solr configuration changes. Under ubuntu:</p>
<pre class="literal-block">
service tomcat7 restart
</pre>
<p>The applications are visible under <tt class="docutils literal"><span class="pre">http://<span class="caps">HOST</span>:8080/APP_NAME</span></tt> where <tt class="docutils literal">APP_NAME</tt> is the name of the context file (<tt class="docutils literal">open_parlamento</tt>, <tt class="docutils literal">open_politici</tt>) or the name of the core (app1, app2).</p>
</div>
OpenSource in the UK2013-03-20T11:16:00+01:00guglielmotag:lab.openpolis.it,2013-03-20:opensource-in-the-uk.html<p>The <span class="caps">UK</span> government issued a beta version of the <a class="reference external" href="https://www.gov.uk/service-manual">Service Design Manual</a>, stating guidelines on
how to design digital services according to the <a class="reference external" href="https://www.gov.uk/service-manual/digital-by-default">Digital by Default Service Standard</a>.</p>
<p>Setting a digital service standard is part of the <a class="reference external" href="http://publications.cabinetoffice.gov.uk/digital/">Governmental Digital Strategy</a>, which aims at producing public
digital services, “so good, that people prefer to use them”.</p>
<p>The manual is an interesting reading in itself, but the <a class="reference external" href="https://www.gov.uk/service-manual/making-software/open-source.html">Open Source</a> section is a must.</p>
<p>Basically, the <span class="caps">UK</span> government is suggesting to use open source software
for operating systems, networking software, web servers, databases and programming languages.</p>
<p>The choice is based on the quality of the overall development and maintenance process, not on economic grounds.</p>
Bureaucrats like Github2013-03-15T11:00:00+01:00guglielmotag:lab.openpolis.it,2013-03-15:bureaucrats-like-github.html<p>A <a class="reference external" href="http://radar.oreilly.com/2013/03/github-government-bureaucat-open-source.html?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+oreilly%2Fradar%2Fatom+(O'Reilly+Radar)">post</a> on <a class="reference external" href="http://radar.oreilly.com">Oreilly Radar</a> informs us that <a class="reference external" href="http://www.github.com">Github</a> is trending as the social-repository of choice for governments and public institutions
who want to publish data, standards and source code.</p>
<p>Some remarkable examples:</p>
<ul class="simple">
<li><span class="caps">US</span> - <a class="reference external" href="https://github.com/WhiteHouse">https://github.com/WhiteHouse</a></li>
<li><span class="caps">NASA</span> - <a class="reference external" href="https://github.com/nasa">https://github.com/nasa</a></li>
<li><span class="caps">UK</span> - <a class="reference external" href="https://github.com/alphagov">https://github.com/alphagov</a></li>
<li>Canada - <a class="reference external" href="https://github.com/pwgsc">https://github.com/pwgsc</a></li>
<li>Finlandia - <a class="reference external" href="https://github.com/avoinministerio">https://github.com/avoinministerio</a></li>
<li>Buenos Aires - <a class="reference external" href="https://github.com/gcba">https://github.com/gcba</a></li>
</ul>
<p>Github also destined an evangelist to the task.
The objective: change the way governments actually face (and solve) <span class="caps">ICT</span> problems, studying similar solutions in the social repository,
and implementing from scratch only when really required.</p>
BLOG TODO LIST2012-09-17T19:02:00+02:00danieleftag:lab.openpolis.it,2012-09-17:blog-todo-list.html<p><strong><span class="caps">UPDATED</span></strong> by guglielmo Nov, 11th, 2012</p>
<ul class="simple">
<li>refine blog deploy flow (make/fab file) - <span class="caps">OK</span></li>
<li>make a theme - <span class="caps">OK</span> (sortof)</li>
<li>add links</li>
<li>disqus - <span class="caps">OK</span></li>
<li>google analytics - not <span class="caps">OK</span></li>
</ul>