===============================================================================
=                                                                             =
=                                 Ocarina 6.0                                 =
=                                                                             =
===============================================================================

My main goal for Ocarina 6.x is to plan out all of my actions before writing
code.  In the past I was adding features as I thought of them before thinking
out how everything works together, and this made Ocarina difficult to maintain
because I had no overall plan.  This document aims to fix that.

I will also create unit tests as I add features so bugs can be found faster.
Unit tests will be created for each module (file) in my backend library code.



Files:
	$HOME/.ocarina{-debug}/
		album.db
		artist.db
		genre.db
		groups.idx
		library.db
		playlists.lst
		track.db
	/usr/bin/
		ocarina
	/usr/lib/ocarina/
	ocarina/
		design.txt
	ocarina/include/
		audio.h
		database.h
		database.hpp
		file.h
		filter.h
		groups.h
		idle.h
		index.h
		library.h
		playlist.h
		prefs.h
		print.h
		version.h
	ocarina/lib/
		audio.cpp
		database.cpp
		file.cpp
		filter.cpp
		groups.cpp
		idle.cpp
		index.cpp
		library.cpp
		playlist.cpp
		prefs.cpp



Install:
	Ocarina will be compiled into a single executable placed under
	/usr/bin/.  Any extra files needed to run will be placed under
	/usr/lib/ocarina/.



Printing: (include/print.h>
	Sometimes text needs to be printed to the screen so users (or debuggers)
	know what is going on.

API:
	void print(string fmt, ...)
		Print text to the screen
	void dprint(string fmt, ...)
		Print text to the screen when debugging is enabled



Versioning: (include/version.h)
	This file contains a simple function for returning a string stating
	the current version.

API:
	const char *get_version();
		Returns a string describing the current version.



Audio: (lib/audio.cpp)
	This file will introduce an "audio" namespace containing all of the
	functions interacting with gstreamer.  This will create a wrapper
	namespace that will be easier to work with than using raw gstreamer
	functions.

	The audio layer will also control the "pause after N tracks" feature
	so songs can be loaded without neeting to pass in a "begin playback"
	flag every time.

- Internal:
	Set up a message bus to look for end-of-stream and error messages so
	the next song can be played.  This function should call the play
	function after loading a track and after checking the "pause after N"
	count.

- API:
	audio :: init(argc, argv)
		Initialize the gstreamer layer and reload the track that was
		last loaded before shutdown.  Please only pass --gst-* options
		for argv.
	audio :: load_file(filepath)
		Loads a file path but does not begin playback.
	audio :: play()
		Begin playback
	audio :: pause()
		Pause playback
	audio :: seek_to(X)
		Seek to a position X seconds into the track
	audio :: stop()
		pause()
		seek_to(0)
	audio :: pause_after(N)
		Pause after N tracks, pass a negative number to disable.
	audio :: position()
		Return the number of seconds that the song has played
	audio :: duration()
		Return the duration of the current song in seconds



On-disk files: (lib/file.cpp)
	Data will be stored in the user's home directory according to the
	XDG / freedesktop.org specification.  This means storing data in
	$XDG_DATA_HOME/ocarina{-debug|-test}/ and storing configuration in
	$XDG_CONFIG_HOME/ocarina{-debug|-test}/.  In addition, I will support
	importing data from Ocarina 5.10 for conversion to the new format.

	In theory my file format will not change often, so it should be
	possible to use multiple Ocarina versions with the same data.  However,
	should the format need to change I will only support forward
	compatibility.  This means that downgrades will not be possible after
	a file format change.  To keep the code even cleaner, I will only
	support updating from the previous file format version.  This means
	that legacy support will be removed after the first file format change.

	Items should be written to a file with either a space or new line
	separating multiple values.

- Notation:
	File << aaaaa << bbbbb << endl is translated into "aaaaa bbbbb\n"

- File version:
	#define FILE_VERSION 0

- Hint where the file is located:
	enum FileLocHint {
		FILE_TYPE_CONFIG,
		FILE_TYPE_DATA,
		FILE_TYPE_LEGACY,
	}

- Open mode:
	enum OpenMode {
		OPEN_READ,
		OPEN_WRITE,
		NOT_OPEN,
	}

- File:
	class File : std::fstream {
	private:
		unsigned int version;
		OpenMode mode;
		FileLocHint hint;
		string filepath;

	public:
		File(string, FileLocHint);
		~File();
		const char *get_filepath();
		const unsigned int get_version();
		bool exists();
		bool open(OpenMode);
		bool close();
		string getline();
	}

- File format:
	File << FILE_VERSION << endl;
	File << <OTHER_DATA>;

- API:
	File : File(string filepath, FileLocHint hint)
		Resolve filepath to one of:
			XDG_{CONFIG|DATA}_HOME/ocarina/filepath
			XDG_{CONFIG|DATA}_HOME/ocarina-debug/filepath
			XDG_{CONFIG|DATA}_HOME/ocarina-test/filepath
			$HOME/.ocarina/
			$HOME/.ocarina-debug/

	File : ~File()
		Close the file stream if it is open.

	const char *get_filepath()
		Return the full filepath to the file.

	const unsigned int get_version()
		Return the file version number.

	bool File : exists()
		Return true if the file exists in the filesystem.
		Return false otherwise.

	bool File : open(OpenMode mode)
		When opening a file for reading (mode == OPEN_READ),
			- Return false if the file does not exist
			- Open the file
			- Read in version from the start of the file
			- Return true

		When opening a file for writing (mode == OPEN_WRITE),
			- Return false if the file has FILE_TYPE_LEGACY set
			- Create missing directories as needed
			- Write version information to the start of the file
			- Return true

		Return false if the file is already open.
		Return false if there are any other errors.

	bool File : close()
		Close a file after IO.

	string File : getline();
		Read an entire line from the file and return it to the caller.
		In theory a return value optimization will occur so returning
		a string by value won't be a problem.



Idle queue: (lib/idle.cpp)
	The idle queue is used to schedule tasks to run at a later time.  Idle
	tasks must inherit from the IdleBase class so that multiple templated
	types can be placed on the same idle queue.

- IdleBase:
	class IdleBase {
		IdleBase();
		~IdleBase();
		virtual void run() = 0;
	};

- IdleTask:
	template <class T>
	class IdleTask : IdleBase {
	private:
		void (*func)(T *);
		T *data;

	public:
		IdleTask(void (*)(T *), T *);
		void run();
	};

- Queue:
	deque(IdleTask *> idle_queue;
	float queued = 0.0
	float serviced = 0.0

- API:
	template <class T>
	void idle :: schedule(void (*)(T *), T *);
		Schedule a function to run later.
		queued++
	bool idle :: run_task()
		Run the next task on the queue.  Return true if a task was
		found, and false otherwise.
		scheduled++, reset to zero if idle queue is empty.



Database: (lib/database.cpp)
	Ocarina 5.x created a different save file format for each type of
	data that needed to be stored (preferences, library paths, playlists).
	I intend to unify everything into a generic file format that can be
	accessed through a generic database interface.  The database code will
	be in charge of printing the "valid" bit for each DatabaseEntry so that
	child classes do not need to call into the parent class.  If valid ==
	true, the DatabaseEntry will be streamed out followed by a newline.  If
	valid == false the database will print the next entry in the vector.

	Modules should inherit from the DatabasEntry class and implement their
	own read() and write() functions.  The "valid" field will be stored
	before these functions are called, and the entry will be skipped if
	valid is set to false.

	The Database class is a templated class, so code could potentially
	get messy.  Normal class declarations can still exist in the file
	include/database.h and member functions can be written in the file
	include/database.hpp, which will be included by database.h.  Any
	function not relying on a template can be written in lib/database.cpp.

- DatabaseEntry:
	class DatabaseEntry { /* let database modify valid flag */
	private:
		bool valid;
	public:
		virtual File &operator>>(File &) = 0;
		virtual File &operator<<(File &) = 0;
		friend class Database;
	};

	File << <CHILD_CLASS_DATA>

- Database:
	template <class T>
	class Database {
	private:
		unsigned int _size;  /* Number of valid rows */
		File filename;
		vector<T> db;
	public:
		Database::Database(filename);
		void load();
		void save();
		unsigned int insert(T);
		void delete(unsigned int);
		const unsigned int &size();
		const T &operator[](unsigned int);
	};

	File << db.size() << endl
	File << INDEX_0 << db[INDEX_0].valid << db[INDEX_0] << endl;
	File << Index_1 << db[INDEX_1].valid << db[INDEX_1] << endl;
	...

- API:
	Database.Database(filename);
		Initializes database to use ~/.ocarina{-debug}/filename.  Pass
		an empty string if you do not want this database to be saved.
	Database.load();
		Reads data from file.  Call after static initialization of
		Ocarina to ensure idle tasks are configured so loading can
		happen at a later time.
	Database.save();
		Saves data to file.
	Database.insert(T &);
		Adds a new item to the db, returns the id of the item
	Database.delete(unsigned int index);
		Mark db[index] as invalid (quick deletion)
	Database.size();
		Returns number of valid rows in the database
	Database.operator[unsigned int index]
		Return a reference to db[index]



Index: (lib/index.cpp)
	An inverted index allows me to map multiple values to a single key.

- Index:
	class Index {
	private:
		map<string, set<int>>
		File filename;
	public:
		Index::Index(filename);
		void load();
		void save();
		void insert(key, int);
		void delete(key, int);
		void replace(key, int);
		const set<string> &keys();
		const set<int> &operator[](string);
	};

	File << key << endl;
	File << map[key].size() << int_0 << int_1 << ... << int_n << endl;

- API:
	Index.Index(filename);
		Initializes an index using ~/.ocarina{-debug}K/filename.  Pass
		an empty string if you do not want this index to be saved.
	Index.load();
		Reads data from a file.  Call after static initialization of
		Ocarina to ensure idle tasks are configured
	Index.save();
		Saves data to file
	Index.insert(key, int);
		1) If key does not exist, create it.
		2) Add int to the list
		3) Index.save()
	Index.delete(key, int);
		1) Remove int from the set of values associated with key
		2) Do not delete key if set is empty
		3) Index.save()
	Index.replace(key, int)
		1) map[key].clear()
		2) insert(key, int)
	Index.keys();
		Return a set of index keys
	Index.operator[](string key);
		Return the set associated with key



Filter: (lib/filter.cpp)
	Filtering is used to generate a subset of songs for easier searching.

- Index:
	map<string, string> lowercase_cache;
	map<string, set<string>> substring_cache;
	Index filter_index("");

- Parsing:
	1) Convert the provided string into a list of words, using whitespace
	   and the following characters as delimiters: \/,;()_~+"

	For each word:
	    2) Check the lowercase_cache to see if we have seen the word before,
	       a) If we have, return the stored string
	       b) Convert the string to lowercase and strip out remaining
	          special characters.  Add the result to the lowercase_cache;
	    3) Check the substring_cache to see if we have seen the word before,
	       a) If we have, use the substring set returned
	       b) Break the word into substrings from the front only.  For
	          example:  "dalek" would contain the substrings
	          {d, da, dal, dale, dalek}.  Add to the substring cache.

- API:
	filter :: add(string, track_id);
		Parses the string and adds the track_id to the index.
	void filter :: search(string, set<track_id> &);
		Parse the string and fill in the set with matching tracks.



Groups: (lib/group.cpp)
	Groups are going to be a new feature in Ocarina 6 and can compare
	directly to Gmail-style labels.  Ocarina 6 will create dynamic groups
	that cannot be deleted by the user based on library status.  Similar
	to the library, groups should exist in their own namespace.

- Index:
	Index group_idx(groups.idx)

- Default groups:
	All music
		All tracks are added to this group
	Library
	Banned Songs
		These groups are mutually exclusive.  A track is either
		in the Library or the Banned Songs group

- API
	group :: init();
		Initialize the index
	group :: list();
		return group_idx.keys();
	group :: get_tracks(name):
		return group_idx[name]
	group :: add(name, track_id)
		group_idx.insert(name, track_id);
	group :: del(name, track_id)
		grou_idx.delete(name, track_id)



Library: (lib/library.cpp)
	The library manages databases containing track information added by the
	user.  Ocarina 6 splits the library into multiple database tables for
	storing content.  The library will exist in a library namespace to
	to make functions and classes more unique.

	When a library : Track is created, it should be added to the "Library"
	group if it is NOT a member of the banned songs group.

- Album:
	class library :: Album : public DatabaseEntry {
		string name;
		short  year;
	};

	File << year << name

- Artist:
	class library :: Artist : public DatabaseEntry {
		string name;
	};

	File << name

- Genre:
	class library :: Genre : public DatabaseEntry {
		string name;
	};

	File << name

- Path:
	class library :: Path : public DatabaseEntry {
		string root_path;
		bool enabled;
	};

	File << enabled << root_path

- Track:
	class library :: Track : public DatabaseEntry {
		unsigned int artist_id;
		unsigned int album_id;
		unsigned int genre_id;
		unsigned int library_id;

		short track;
		short last_year;
		short last_month;
		short last_day;
		unsigned int play_count;
		unsigned int length;
		string title;
		string length_str;
		string filepath;
	};

	File << artist_id << album_id << genre_id << library_id << track << last_year
	File << last_year << last_month << last_day << play_count << length << endl
	File << title << endl;
	File << filepath << endl;

- Track: /* This struct lies outside the library namespace */
	struct Track {
		library :: Album   *album;
		library :: Artist  *artist;
		library :: Genre   *genre;
		library :: Library *library;
		library :: Track   *track;
	};

- Databases:
	Database<library :: Album>   album_db(album.db);
	Database<library :: Artist>  artist_db(artist.db);
	Database<library :: Album>   genre_db(genre.db);
	Database<library :: Library> library_db(library.db);
	Database<library :: Track>   track_db(track.db);

- Updating algorithm:
	set<pair<lib_id, track_path>> known_tracks;

	1) For each track currently in the library, check if the track exists
	   in the filesystem.
	   1a) If the track does exist, add to the known_tracks map.
	   1b) Else, mark track invalid.
	2) For each file in the scan directory, check if (lib_id, track_path)
	   exists in the known_tracks map.
	   2a) If the file is in the map, do nothing.
	   2b) Else, add track to the library, to the groups "All Music" and
	       "Library", and then to the filter index.
	3) Save all databases

	The taglib library should be used for finding artist, album, etc. tags
	for each track.

	Use idle tasks for step 2 to break up tagging new files into chunks.
	This way the user will still be able to use Ocarina and scanning can
	happen while idle.

- API
	library :: init();
		Initialize all databases
	library :: add_path(string dir);
		Add new row to paths table, update
	library :: del_path(unsigned int lib_id);
		Invalidate a path row
	library :: update_path(lib_id);
		Update the given library path, if valid.
	const Database<LibraryEntry> &library :: get_db();
		Returns the database containing library information.
	struct Track library :: resolve(track_id)
		Fill out a Track structure for the provided track_id



Playlist: (lib/playlist.cpp)
	Playlists are a list of songs that the user has configured to play.  I
	will create a pool of playlists that will be filled by user actions.

	Playlists will be put on a "deck" that is used to give an order to the
	next songs played.  When deck :: next() is called, find the first
	playlist with PL_ENABLED set and call that playlists next() function.

	When a playlist is empty, remove it from the deck.

- Flags:
	enum playlist_flags {
		PL_ENABLED (1 << 0),
		PL_RANDOM  (1 << 1),
		PL_DRAIN   (1 << 2),
	};

- Playlist:
	class Playlist : public Database {
	private:
		database<track_id> tracks; /* Keyed on track id */
		unsigned int cur;
		unsigned int flags;
	public:
		Playlist();
		void add(vector<track_id> &);
		void del(vector<track_id> &);
		void set_flag(playlist_flags);
		const unsigned int get_flags();
		unsigned int size()

		File &operator<<(File &);
		File &operator>>(File &);

		void sort();
		void next();
	}

	File << flags << cur << tracks[0] << tracks[1] << ... << tracks[N];

- Deck:
	list<Playlist> deck;
	File << deck[0] << endl;
	File << deck[1] << endl;
	File << deck[N] << endl;

- API
	deck :: init();
		Read in the playlist file
	deck :: new();
		Adds a new playlist to the deck
	deck :: rm(N)
		Removes playlist N from the deck
	Playlist *deck :: get(N)
		Return playlist N from the deck

- TODO										<<<<<
	What if each playlist has its own playlist_id for tracks?  This would
	allow for simpler removals, since I won't need to search for a track id.
	I can easily create a function for mapping a list of playlist_ids to
	track_ids...



Preferences: (lib/prefs.cpp)
	Preferences make use of a special index where the set<int> is always
	size 1.  Preferences will be in the prefs namespace.

- Index:
	Index prefs(prefs.idx);

- API:
	prefs :: set(string, val);
		prefs.replace(string, val);
	prefs :: get(string)
		return prefs[string].begin()



Future work:
	I want to set reasonable expectations for Ocarina 6 so that I don't
	have to spend a large amount of time coding before releasing something
	to the wild.  This section will be a list of features that I want, but
	should be deferred to a future release so basic support can be coded.

	Hint:  If feature B depends on A, implement A in 6.x and B in 6.x+1

	- New default groups: (6.1)
		Unplayed tracks

	- Categories: (6.1)
		Use these to make "groups of groups" for better organization.
		Different categories can include Album, Artist and Genere
		dynamic groups in addition to user created groups (see below)

		The Artist, Album and Genre "tables" can be used to populate
		these categories.

	- User created song groups: (6.2)
		Basic add and remove features can be implemented using the
		Library and Banned Songs groups.  This will give me a chance
		to test saving groups on a small scale before letting users
		create anything they want.

	- Save a user's playlist as a group: (6.2)

	- Library defragment: (6.1)
		Ocarina 6.0 will leave holes in the library when tracks are
		deleted, potentially leading to fragmentation and larger-than-
		needed file sizes.  A "defragment" utility can be created to
		clean up unused slots.

		To help with fixing groups, a mapping of (old values) ->
		(new values) should be kept.

	- Fix track durations: (6.1)
		Some tracks in my library are tagged with the wrong duration,
		so fix them as they are played.

	- Track tag editor: (6.2)
		Make a pop-up window for editing the tags of a track.  Be sure
		to update the library information and the on-disk file.

	- Album art: (6.1)

	- Playlist custom sorting: (6.1)
		Click column headers to choos sort order
		Keep a list of fields that the user has selected and place new
		fields in the front of this list.  Use a recursive stable sort
		to do the sorting.