Amethyst 1.6 Archiving

[Leon van Heerden]

Amethyst doesn't allow direct access to the Operating System for security reasons. This means that there is no way to backup the system. To allow a copy of the recording and system configuration to be stored off site, Amethyst has 2 main methods of archiving to another device. This archive contains a copy of the recordings, configuration and database, in an encrypted form. These files are not used to restore a system, but rather accessed directly on this device using the AmtListen utility.

The 2 Archiving destinations can be used individually or at the same time for redundancy.

1. Physical Drive
In Amethyst 1.5 and older an internal IDE drive bay was supplied with the server and from Amethyst 1.6 onward an external USB is supplied. The IDE drive was replaced with an USB drive on older installations that were upgraded from 1.5 to 1.6 when the IDE drives filled up. Here are the instructions to add a new Physical drive.

• The Physical drive is formatted with a FAT32 (vfat) file system that can be accessed on a Windows PC.
• The Amethyst server can format a clean hard drive. If the drive has contained a file system, or partition prior to being connected to the server, Amethyst will not be able to format the drive.
• The drive can be formatted on a Windows PC before you connect it to the Amethyst server.

2. Network Location
From Amethyst 1.5, an option to archive to a NFS share was implemented. In version 1.6.4.10 an option to configure this share as a normal Windows Share was added. This allows a facility to configure a remote server, share location and login and password.

• This login that is used by the Amethyst server needs to have write access to the share location
• For security we recommend that all other users access to this location is limited to read only access. This prevents malicious or accidental deletion of files.
• Note that if the location is filled 100% and this location is on the main Operating System drive of the network server, the Shared server will run into problems. Make sure that this location always has space.
• This location can be included in the standard backup processes, since this location can be one of the clients server.

How does the Archiving work?
Every time a recording is processed into Amethyst it is added to a change log with a check-sum. Every night at 21:00 the archive process copies each file that is listed in this file to the archive drive(s) and the check-sum is verified. If dual archiving is enabled, the file is copied to each drive individually. The Network and local drive processes runs independatlly, which means if one process fails, the other process is not affected and can continue running.
When the archive process starts, a log entry is generated in the Amethyst Web Admin logs. Any errors that may occur during the archive process will also be logged after this "start" log entry. Once the archive process is complete, it will write a "stop" log entry to the Amethyst Web Admin log.

A folder is created for each day that has recordings or a database. The folder name is the date in the format YYYYMMDD. If more than 10000 files has to be copied to the folder, a new folder is created with the name YYYYMMDD-2. If this folder also exceeds 10000 files, another folder is created with YYYYMMDD-3 and so on.

Archive steps:

• Access to the Archive locations is checked. If it doesn't have access, the process stops and an error is logged in the Amethyst Web Admin logs.
• Then the Database is backed up and encrypted. This encrypted file is then split into 1 gig files and copied to the Archive location(s). It will be copied into the folder for the day the archive is started on.
• The change log is then processed. The Change log contains all the recordings processed since the last archive. It is sorted and broken up into a groups of 5000 recordings. The Archive process copies 5000 files to each archive location. Each recording is check-summed before and after it has been copied to compare with the original check-sum. If a check-sum fails, a log entry is created, but the process continues. All the files that fails are held back to re-try.
  After each batch of 5000 files the archive location is unmounted to ensure that everything is synced to the drive. Then the location is mounted again and the next 5000 files is copied.
• The configuration files are archived into an AMT file that Datatex can use to retrieve the configuration. This file is then copied to the Archive location(s).
• The log files are then also archived into an Archive format called ".AMT" and copied to the Archive location(s). This file can be used to upload the logs back onto the server if it is required for investigations.
• Lastly the change log file is copied to the Archive location(s). This allow manual verification of files that were copied.

Files are written to the Archive location.