Operations

Management tasks and other operations can be performed within the shell of deployed web services, executing python manage.py ... within the container, using docker.

docker ps # processes in given environment
docker exec -it web /bin/bash # shell into the service

Operations are here described only with the python manage.py, so that they’re useable in the development environment as well.

Setup

Once the stack has been deployed, then the machine needs to be setup.

python manage.py migrate
python manage.py createsuperuser --username admin --email guglielmo@depp.it
python manage.py collectstatic --noinput
python manage.py compilemessages -l en

Data import

see Data Ingestion.

Web application modifications

Whenever the web application source code changes, the docker image needs to be rebuilt, then the stack re-deployed on the remote server.

This is automated with gitlab-ci, and the only operation needed is a push to the develop branch.

The test suite will be launched, the flake8 syntax check will be executed, the security check of the python packages in requirements will be checked, and then the image will be re-built and re-deployed on the staging server.

The deployment on the production server is tied to the master branche, but is not automated (must be performed manually from the gitlab pipeline page).

Solr schema modifications

If a solr field is changed, added or removed, then the schema needs to be changed, and eventually the data need to be re-indexed in the running Solr index.

python manage rebuild_solr_schema > compose/solr/schema.xml

Warning

django-haystack assigns the field type text_en to CharField types, so all of relevant fields used in raw queries need to be assigned to another type manually.

This means that the scheme produced with rebuild_solr_schema cannot be directly overwritten, but modifications need to be performed manually.

Failure to do so will produce a faulty index and the similarities queries will not work, jeopardising the import procedures.

Development

The new schema is already linked in core config path: .. code-block:: bash

ln -sf compose/solr/schema.xml /var/lib/solr/cores/opdm/conf/managed-schema

Production/Staging

Copy the schema in the correct position within the running container:

docker cp compose/solr/schema.xml\
    oc2_solr_1:/opt/solr/server/solr/mycores/opdm/conf/

Solr need to be restarted, or the core reloaded.

TODO: put this under gitlab-ci, so that modifications to the managed schema are deployed in staging or production

Requirements modifications

Whenever security issues are raised by safety, requirements need to be upgraded with:

./compile-requirements --upgrade
./install-requirements

Tests need to be executed locally, and then changes can be committed and pushed, to go through the deployment pipelines.