Archive

[danielrichman]

The eventual plan is to have an archive daemon (in addition to a garbage collection daemon that then cleans up after it, and cleans up other stuff; #107) that will, a bit after a flight has ended, pack up all the relevant documents into a flat file or two and put it somewhere, thereby reducing the load on the database/making them easier to back up/making viewing that data super fast/whatever. Then the plan was to have it add a 'shell' document with some aggregated stats (number of lines uploaded by who, etc.) so that reduce/stats views may still work over all the data.

This issue mainly holds TODOs/notes on things that will need to be cleaned up. There's not much point importing old data into the database and then archiving it, we may as well take the old dumps of RJH's SQL and the old habitat database and convert them straight to the eventual archive format. Therefore the creation of the archive daemon will probably coincide with cleaning up old data, hence the list.

• listener_telemetry speed km/h in android app uploads <= version 0.1.6. Add speed to listener_telemetry #283
• listener_information keys (specifically, client version) in dl_fldigis below (not yet updated to latest version)
• listener_telemetry data.client in android app uploads <= version 0.1.6
• data in habitat_old database
• (maybe) old payload_telemetry keys
• data in RJH SQL dump