running in vagrant
==================

A development server can be started in the vagrant virtualization environment. A ``Vagrantfile`` is provided in the root folder of the runmacs repository.
When inside the root folder of the repository, vagrant can be started by using the command::

    vagrant up

On the first run, this command will download a virtual machine image and install all requirements.
This may take a while to complete.

starting the server
-------------------

After completion, you can log into the freshly created virtual machine by using::

    vagrant ssh

Inside the virtual machine, it is advisable to use ``tmux`` to create multiple terminal sessions.
To run a full macsserver, you need a mongodb database, a varnish http cache server (these two should be started automatically) and at least one instance of the authserver and the data server.
These two should run inside the user ``macsserver`` and can be started as follows (in two separate sessions)::

    sudo su macsserver
    cd /vagrant
    python -m runmacs.processor.authserver.authserver --port=21001

and::

    sudo su macsserver
    cd /vagrant
    python -m runmacs.processor.frontend.frontendServer --port=20001

Now, on the host machine, a fresh macsserver instance should be visible at http://localhost:9080 .
You can login to the macsserver as any of the users described in ``/provisioning/roles/mongodb/files/populate_database.py``.

loading data into the server
----------------------------

However, this instance will not have any data loaded.
To fill it with data, some data and collections have to be placed in the ``/data`` and ``/collections`` folders of the repository (or any other folder if you modify the ``~/.macs/processor.json`` configuration file of the macsserver user inside the virtual machine).
Before any files placed in the ``/data``-folder will be recognized, they must be listed in a collection file.
Collection files must be named ``/collections/collection_<id>.yaml`` where ``<id>`` is replaced by any integer number starting from 0.
A collection file can be pre filled with data using the ``collection_tool`` in ``/runmacs/processor/tools/``, e.g. by the following command::

    collection_tool -s /vagrant/data -o collection_0.yaml

The fields ``name`` and ``description`` of the resulting file should be filled manually.
Note that the collection tool allows to append to existing files using the ``-i`` option without destroying the existing contents.

Now, as all input data has been set up, the database can be filled by running the ``productBuilder``::

    python -m runmacs.processor.productBuilder --production --crawl --collection 0

This will scan though all files listed in collection 0 and create imported and derived products from it.

flushing the cache
------------------
The macsserver features two levels of caching, on the frontend side, there is the varnish http cache.
This is mostly an in-memory cache and is mostly a side effect of using varnish as a load balancer, but also helps to speed up successive identical requests.
However, if you update the database, this cache will delay the visibility of database updates and must be flushed if you want to see them immediately.
This is done most easily by restarting varnish as the ``vagrant`` user::

    sudo service varnish restart

Another level of caching is the file based data cache.
This is intended for static (i.e. "never" changing data) and is managed by the macsserver software itself.
In the vagrant virtual machine, this data is located at ``/home/macsserver/macsserver_cache``.
This cache sometimes needs to be flushed during development, if the variable data of newly created products changes.
You can either flush single datasets by searching for the folder with the corresponding dataset oid or you can delete the entire contents of this folder such that all products will be recomputed.

preheating the cache
--------------------

An empty file level cache can be very slow, especially for previewing data.
In fact, some previews will never be generated by the web interface and must be generated manually.
To facilitate the processes of filling (or "heating") the cached, there is the ``cacheHeater`` which can be run by::

    python -m runmacs.processor.cacheHeater --check-all