ocarina/design.txt

===============================================================================
=                                                                             =
=                                 Ocarina 6.0                                 =
=                                                                             =
===============================================================================

My main goal for Ocarina 6.x is to plan out all of my actions before writing
code.  In the past I was adding features as I thought of them before thinking
out how everything works together, and this made Ocarina difficult to maintain
because I had no overall plan.  This document aims to fix that.

I will also create unit tests as I add features so bugs can be found faster.


Files:
	ocarina/
		design.txt
	ocarina/gui/
	ocarina/include/
		database.h
		database.hpp
		file.h
		group.h
		index.h
		library.h
		playlist.h
		prefs.h
	ocarina/lib/
		database.cpp
		file.cpp
		group.cpp
		index.cpp
		library.cpp
		playlist.cpp
		prefs.cpp
	ocarina/tests/

	$HOME/.ocarina{-debug}/
		album.db
		artist.db
		genre.db
		groups.idx
		library.db
		track.db



On-disk files: (lib/file.cpp)
	I use the disk to store data between sessions, this could include
	library state and user preferences.  In theory, file formats do not
	change often so updating between file formats should be possible.
	Supporting all previous file formats can create a lot of clutter in
	the code, so I will ONLY support updating from the previous file-format
	version.  Ocarina 5.x used two different numbers to represent the
	current file format (library = 2 and playlist = 3).  I want to unify
	this into a single number shared across all files for simplicity, and
	then create a class to read and write data on disk.

	Items should be written to a file with either a space or new line
	separating multiple values.

- Notation:
	File << aaaaa << bbbbb << endl is translated into "aaaaa bbbbb\n"

- File version:
	#define FILE_VERSION 4

- Open mode:
	enum OpenMode {
		OPEN_READ,
		OPEN_WRITE,
	}

- File:
	class File {
	private:
		union {
			ifstream in;
			ofstream out;
		} f;
		unsigned int version;
		OpenMode mode;
		string filepath;
	public:
		File(filepath);
		open(OpenMode);
		const File &operator<<(File &);
		const File &operator>>(File &);
		getline(string &);
	}

- File format:
	File << FILE_VERSION << <OTHER_DATA>

- API:
	File : File(filepath)
		Resolve filepath to ~/.ocarina{-debug}/filepath
	File : open(mode)
		Open a file for either reading or writing.
		If reading: Read in version from start of file
		Else write version to file
	File : operator<<()
		Write data to out
	File : operator>>()
		Read data from in;
	File : getline(string);
		getline(f.in, string) to read an entire line



Database: (lib/database.cpp)
	Ocarina 5.x created a different save file format for each type of
	data that needed to be stored (preferences, library paths, playlists).
	I intend to unify everything into a generic file format that can be
	accessed through a generic database interface.  The database code will
	be in charge of printing the "valid" bit for each DatabaseEntry so that
	child classes do not need to call into the parent class.  If valid ==
	true, the DatabaseEntry will be streamed out followed by a newline.  If
	valid == false the database will print the next entry in the vector.

	Modules should inherit from the DatabasEntry class and implement their
	own read() and write() functions.  The "valid" field will be stored
	before these functions are called, and the entry will be skipped if
	valid is set to false.

	The Database class is a templated class, so code could potentially
	get messy.  Normal class declarations can still exist in the file
	include/database.h and member functions can be written in the file
	include/database.hpp, which will be included by database.h.  Any
	function not relying on a template can be written in lib/database.cpp.

- DatabaseEntry:
	class DatabaseEntry { /* let database modify valid flag */
	private:
		bool valid;
	public:
		virtual File &operator>>(File &) = 0;
		virtual File &operator<<(File &) = 0;
		friend class Database;
	};

	File << <CHILD_CLASS_DATA>

- Database:
	template <class T>
	class Database {
	private:
		unsigned int _size;  /* Number of valid rows */
		File filename;
		vector<T> db;
	public:
		Database::Database(filename);
		void load();
		void save();
		unsigned int insert(T);
		void delete(unsigned int);
		const unsigned int &size();
		const T &operator[](unsigned int);
	};

	File << db.size() << endl
	File << INDEX_0 << db[INDEX_0].valid << db[INDEX_0] << endl;
	File << Index_1 << db[INDEX_1].valid << db[INDEX_1] << endl;
	...

- API:
	Database.Database(filename);
		Initializes database to use ~/.ocarina{-debug}/filename
	Database.load();
		Reads data from file.  Call after static initialization of
		Ocarina to ensure idle tasks are configured.
	Database.save();
		Saves data to file.
	Database.insert(T &);
		Adds a new item to the db, returns the id of the item
	Database.delete(unsigned int index);
		Mark db[index] as invalid (quick deletion)
	Database.size();
		Returns number of valid rows in the database
	Database.operator[unsigned int index]
		Return a reference to db[index]



Index: (lib/index.cpp)
	An inverted index allows me to map multiple values to a single key.

- Index:
	class Index {
	private:
		map<string, set<int>>
		File filename;
	public:
		Index::Index(filename);
		void load();
		void save();
		void insert(key, int);
		void delete(key, int);
		void replace(key, int);
		const set<string> &keys();
		const set<int> &operator[](string);
	};

	File << key << endl;
	File << map[key].size() << int_0 << int_1 << ... << int_n << endl;

- API:
	Index.Index(filename);
		Initializes an index using ~/.ocarina{-debug}/filename
	Index.load();
		Reads data from a file.  Call after static initialization of
		Ocarina to ensure idle tasks are configured
	Index.save();
		Saves data to file
	Index.insert(key, int);
		1) If key does not exist, create it.
		2) Add int to the list
		3) Index.save()
	Index.delete(key, int);
		1) Remove int from the set of values associated with key
		2) Do not delete key if set is empty
		3) Index.save()
	Index.replace(key, int)
		1) map[key].clear()
		2) insert(key, int)
	Index.keys();
		Return a set of index keys
	Index.operator[](string key);
		Return the set associated with key



Library: (lib/library.cpp)
	The library manages databases containing track information added by the
	user.  Ocarina 6 splits the library into multiple database tables for
	storing content.  The library will exist in a library namespace to
	to make functions and classes more unique.

	When a library : Track is created, it should be added to the "Library"
	group if it is NOT a member of the banned songs group.

- Album:
	class library :: Album : public DatabaseEntry {
		string name;
		short  year;
	};

	File << year << name

- Artist:
	class library :: Artist : public DatabaseEntry {
		string name;
	};

	File << name

- Genre:
	class library :: Genre : public DatabaseEntry {
		string name;
	};

	File << name

- Path:
	class library :: Path : public DatabaseEntry {
		string root_path;
		bool enabled;
	};

	File << enabled << root_path

- Track:
	class library :: Track : public DatabaseEntry {
		unsigned int artist_id;
		unsigned int album_id;
		unsigned int genre_id;
		unsigned int library_id;

		short track;
		short last_year;
		short last_month;
		short last_day;
		unsigned int play_count;
		unsigned int length;
		string title;
		string length_str;
		string filepath;
	};

	File << artist_id << album_id << genre_id << library_id << track << last_year
	File << last_year << last_month << last_day << play_count << length << endl
	File << title << endl;
	File << filepath << endl;

- Track: /* This struct lies outside the library namespace */
	struct Track {
		library :: Album   *album;
		library :: Artist  *artist;
		library :: Genre   *genre;
		library :: Library *library;
		library :: Track   *track;
	};

- Databases:
	Database<library :: Album>   album_db(album.db);
	Database<library :: Artist>  artist_db(artist.db);
	Database<library :: Album>   genre_db(genre.db);
	Database<library :: Library> library_db(library.db);
	Database<library :: Track>   track_db(track.db);

- Updating algorithm:
	set<pair<lib_id, track_path>> known_tracks;

	1) For each track currently in the library, check if the track exists
	   in the filesystem.
	   1a) If the track does exist, add to the known_tracks map.
	   1b) Else, mark track invalid.
	2) For each file in the scan directory, check if (lib_id, track_path)
	   exists in the known_tracks map.
	   2a) If the file is in the map, do nothing.
	   2b) Else, add track to the library and to the "All Music" and
	       "Library" and "Unplayed Tracks" groups
	3) Save all databases

	The taglib library should be used for finding artist, album, etc. tags
	for each track.

	Use idle tasks for step 2 to break up tagging new files into chunks.
	This way the user will still be able to use Ocarina and scanning can
	happen while idle.

- API
	library :: init();
		Initialize all databases
	library :: add_path(string dir);
		Add new row to paths table, update
	library :: del_path(unsigned int lib_id);
		Invalidate a path row
	library :: update_path(lib_id);
		Update the given library path, if valid.
	const Database<LibraryEntry> &library :: get_db();
		Returns the database containing library information.
	struct Track library :: resolve(track_id)
		Fill out a Track structure for the provided track_id



Groups: (lib/group.cpp)
	Groups are going to be a new feature in Ocarina 6 and can compare
	directly to Gmail-style labels.  Ocarina 6 will create dynamic groups
	that cannot be deleted by the user based on library status.  Similar
	to the library, groups should exist in their own namespace.

- Index:
	Index group_idx(groups.idx)

- Default groups:
	All music
		All tracks are added to this group
	Library
	Banned Songs
		These groups are mutually exclusive.  A track is either
		in the Library or the Banned Songs group
	Unplayed tracks
		Tracks with a play count of 0

- API
	group :: init();
		Initialize the index
	group :: list();
		return group_idx.keys();
	group :: get_tracks(name):
		return group_idx[name]
	group :: add(name, track_id)
		group_idx.insert(name, track_id);
	group :: del(name, track_id)
		grou_idx.delete(name, track_id)



Preferences: (lib/prefs.cpp)
	Preferences make use of a special index where the set<int> is always
	size 1.  Preferences will be in the prefs namespace.

- Index:
	Index prefs(prefs.idx);

- API:
	prefs :: set(string, val);
		prefs.replace(string, val);
	prefs :: get(string)
		return prefs[string].begin()



Playlist: (lib/playlist.cpp)
	A playlist is a simple list of songs that can be played either randomly
	or in a user-defined order.  It would probably be best to use a linked
	list or vector to represent playlists, rather than creating a SQLite
	table.  I will be able to easily rearrange tracks in the playlist this
	way.  This will also make it easier to deal with playlist renames and
	reordering by the user.

- API
	/* Playlist management */
	new_playlist();
	del_playlist(playlist);
	add_to_playlist(playlist, songid);
	rm_from_playlist(playlist, songid);
	playlist_size(playlist)
	set_flag(playlist, flag)

- Flags
	PL_ENABLED (1 << 0)
	PL_RANDOM  (1 << 1)
	PL_DRAIN   (1 << 2)



Future work:
	I want to set reasonable expectations for Ocarina 6 so that I don't
	have to spend a large amount of time coding before releasing something
	to the wild.  This section will be a list of features that I want, but
	should be deferred to a future release so basic support can be coded.

	Hint:  If feature B depends on A, implement A in 6.x and B in 6.x+1

	- Categories: (6.1)
		Use these to make "groups of groups" for better organization.
		Different categories can include Album, Artist and Genere
		dynamic groups in addition to user created groups (see below)

		The Artist, Album and Genre "tables" can be used to populate
		these categories.

	- User created song groups: (6.2)
		Basic add and remove features can be implemented using the
		Library and Banned Songs groups.  This will give me a chance
		to test saving groups on a small scale before letting users
		create anything they want.

	- Save a user's playlist as a group: (6.2)

	- Library defragment: (6.1)
		Ocarina 6.0 will leave holes in the library when tracks are
		deleted, potentially leading to fragmentation and larger-than-
		needed file sizes.  A "defragment" utility can be created to
		clean up unused slots.

		To help with fixing groups, a mapping of (old values) ->
		(new values) should be kept.