anna
/
ocarina
1
0
Fork 0
Code Issues 7 Pull Requests Releases Wiki Activity

design: Design out the index class 

I made a few modifications to the database, and introduced an
"IndexEntry" database type to do the exact same thing :)

Signed-off-by: Anna Schumaker <schumaker.anna@gmail.com>
This commit is contained in:
Anna Schumaker 2013-11-23 13:21:17 -05:00 committed by Anna Schumaker
parent 3a47fc1549
commit c2bc99ad9b
6 changed files with 321 additions and 345 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

529
design.txt
View File

@ -39,10 +39,8 @@ Files:
groups.h
idle.h
idle.hpp
index.h
library.h
playlist.h
prefs.h
print.h
version.h
ocarina/lib/
@ -52,10 +50,8 @@ Files:
filter.cpp
groups.cpp
idle.cpp
index.cpp
library.cpp
playlist.cpp
prefs.cpp
@ -231,6 +227,7 @@ Database: (lib/database.cpp)
public:
bool valid;
virtual const std::string &primary_key() = 0;
virtual void write(File &) = 0;
virtual void read(File &) = 0;
virtual void print() = 0;
@ -238,24 +235,34 @@ Database: (lib/database.cpp)
File << <CHILD_CLASS_DATA>
- Flags:
enum DatabaseFlags {
DB_NORMAL,
DB_UNIQUE,
}
- IndexEntry:
class IndexEntry : public DatabaseEntry {
public:
string key;
vector<unsigned int> values;
const std::string &primary_key();
void write(File &);
void read(File &);
void print();
};
File << key << endl;
File << values.size() << values[0] << .. << values[N] << endl;
- Database:
template <class T>
class Database {
private:
vector<T> db;
map<std::string, unsigned int> keys;
unsigned int _size; /* Number of valid rows */
File file;
DatabaseFlags flags;
public:
Database::Database(filename, flags);
void load();
void save();
void clear();
void print();
unsigned int insert(T);
@ -265,7 +272,8 @@ Database: (lib/database.cpp)
unsigned int first();
unsigned int last();
unsigned int next();
unsigned int next(unsigned int &);
unsigned int find(const std::string &);
T &operator[](unsigned int);
};
@ -285,6 +293,11 @@ Database: (lib/database.cpp)
void Database :: save();
Saves the database to disk.
void Database :: clear();
This function exists only if CONFIG_DEBUG is enabled.
Clear the database contents in-memory, but do NOT write
to disk.
void Database :: print()
This function exists only If CONFIG_DEBUG is enabled.
Following a similar format for writing to disk, print the
@ -292,9 +305,9 @@ Database: (lib/database.cpp)
template <class T>
unsigned int Database :: insert(T &);
Adds a new item to the db, returns the id of the item.
if DB_UNIQUE is set, check if the item already exists in the
DB and return it's ID instead of adding a new item.
Adds a new item to the db and marks it as valid. A reverse
mapping is also created into the keys map to map the primary
key back to the id of the newly added item.
void Database :: delete(unsigned int index);
Mark db[index] as invalid (quick deletion).
@ -306,20 +319,106 @@ Database: (lib/database.cpp)
Return db.size().
unsigned int Database :: first();
Return the id to the first valid row or return db.size()
if there are no valid rows.
Return the id to the first valid row.
unsigned int Database :: last();
Return the id of the last valid row or return db.size()
if there are no valid rows.
Return the id of the last valid row.
unsigned int Database :: next(unsigned int &id)
Return the id of the next valid row or return db.size()
if there are no remaining valid rows.
template <class T>
T &Database :: find(const std::string &key);
Search for primary key "key" in the database. The reverse
mapping should be used to make this operation faster. Throw
an empty exception if the key is not found in the mapping.
template <class T>
T &Database :: operator[unsigned int index]
Return a reference to db[index].
Return a reference to db[index]. If index is out of range,
throw an empty exception.
Filter: (lib/filter.cpp)
Filtering is used to generate a subset of songs displayed by the UI to
that users can choose from. The inverted index is generated at startup
so there is no need for a remove() function, since it will be wiped
the next time the application starts.
- Index:
Database<database :: IndexEntry> filter_index;
map<string, string> lowercase_cache;
unsigned int lowercase_cache_hits;
- Parsing:
1) Convert the provided string into a list of words, using whitespace
and the following characters as delimiters: \/,;()_-~+"
For each word:
2) Check the lowercase_cache to see if we have seen the word before,
a) If we have, return the stored string
b) Convert the string to lowercase and strip out remaining
special characters. Add the result to the lowercase_cache;
- API:
void filter :: add(string, track_id);
Parse the string into substrings following the "Parsing"
section (above). Add each (substring, track_id) pair to the
filter_index.
To generate substrings, iterate over the word starting from
the front. For example: "dalek" would contain the substrings
{d, da, dal, dale, dalek}.
void filter :: search(string, set<track_id> &);
Parse the string into substrings following the "Parsing"
section (above). We want to find track_ids that match ALL
substrings, so take the intersection of all sets returned by
the filter_index for a given substring.
void filter :: print_cache_stats();
Print cache hit and size information.
void filter :: get_index();
Return the index storing all the filter data.
(Only available if -DCONFIG_TEST is set)
Groups: (lib/group.cpp)
Groups are going to be a new feature in Ocarina 6 and can compare
directly to Gmail-style labels. Ocarina 6 will create dynamic groups
that cannot be deleted by the user based on library status. Similar
to the library, groups should exist in their own namespace.
In Ocarina 6.0, groups are a wrapper around a specific index. Future
releases will store user-defined groups in a file on disk.
- Index:
Database<database :: IndexEntry> group_db
- Default groups:
All music
All tracks are added to this group
Library
Banned Songs
These groups are mutually exclusive. A track is either
in the Library or the Banned Songs group
- API
void group :: add(name, track_id)
group_idx.insert(name, track_id);
void group :: del(name, track_id)
grou_idx.delete(name, track_id)
void void group :: list(list<string> &);
return group_idx.keys();
void group :: get_tracks(name):
return group_idx[name]
@ -389,161 +488,6 @@ Idle queue: (lib/idle.cpp)
Index: (lib/index.cpp)
An inverted index allows me to map multiple values to a single key.
Keys are tracked separately from the rest of the map so they can be
found and iterated over without writing ugly code.
- Index:
class Index {
private:
map<string, set<unsigned int>> index;
set<string> keys;
File file;
public:
Index::Index(filename);
void load();
void save();
void print();
void insert(key, unsigned int);
void remove(key);
void remove(key, unsigned int);
const set<string>::iterator keys_begin();
const set<string>::iterator keys_end();
const set<unsigned int> &operator[](string);
};
File << keys.size() << endl;
File << key << endl;
File << map[key].size() << int_0 << int_1 << ... << int_n << endl;
- API:
Index :: Index(filename);
Initializes an index using ~/.ocarina{-debug}K/filename. Pass
an empty string if you do not want this index to be saved.
void Index :: load();
Reads data from a file. Call after static initialization of
Ocarina to ensure idle tasks are configured
void Index :: save();
Saves data to file
void Index :: print();
This function exists only if CONFIG_DEBUG is enabled.
Following a similar format for writing to disk, print the
index to the console in a human-readable format.
void Index :: print_keys();
This function exists only if CONFIG_DEBUG is enabled.
Print the database keys to the console in a human-readable
format.
void Index :: insert(key, unsigned int);
1) If key does not exist, create it.
2) Add int to the set for the given key
void Index :: remove(key);
Remove a key from the index;
void Index :: remove(key, unsigned int);
1) Remove int from the set of values associated with key
2) If the set is empty, remove the key
const set<string>::iterator void Index :: keys_begin()
Return an iterator pointing to the beginning of the keys set.
const set<string>::iterator void Index :: keys_end()
Return an iterator pointing to the end of the keys set.
const set<unsigned int> &Index :: operator[](string key);
Return the set associated with key
Filter: (lib/filter.cpp)
Filtering is used to generate a subset of songs displayed by the UI to
that users can choose from. The inverted index is generated at startup
so there is no need for a remove() function, since it will be wiped
the next time the application starts.
- Index:
Index filter_index("");
map<string, string> lowercase_cache;
unsigned int lowercase_cache_hits;
- Parsing:
1) Convert the provided string into a list of words, using whitespace
and the following characters as delimiters: \/,;()_-~+"
For each word:
2) Check the lowercase_cache to see if we have seen the word before,
a) If we have, return the stored string
b) Convert the string to lowercase and strip out remaining
special characters. Add the result to the lowercase_cache;
- API:
void filter :: add(string, track_id);
Parse the string into substrings following the "Parsing"
section (above). Add each (substring, track_id) pair to the
filter_index.
To generate substrings, iterate over the word starting from
the front. For example: "dalek" would contain the substrings
{d, da, dal, dale, dalek}.
void filter :: search(string, set<track_id> &);
Parse the string into substrings following the "Parsing"
section (above). We want to find track_ids that match ALL
substrings, so take the intersection of all sets returned by
the filter_index for a given substring.
void filter :: print_cache_stats();
Print cache hit and size information.
void filter :: get_index();
Return the index storing all the filter data.
(Only available if -DCONFIG_TEST is set)
Groups: (lib/group.cpp)
Groups are going to be a new feature in Ocarina 6 and can compare
directly to Gmail-style labels. Ocarina 6 will create dynamic groups
that cannot be deleted by the user based on library status. Similar
to the library, groups should exist in their own namespace.
In Ocarina 6.0, groups are a wrapper around a specific index. Future
releases will store user-defined groups in a file on disk.
- Index:
Index group_idx()
- Default groups:
All music
All tracks are added to this group
Library
Banned Songs
These groups are mutually exclusive. A track is either
in the Library or the Banned Songs group
- API
void group :: add(name, track_id)
group_idx.insert(name, track_id);
void group :: del(name, track_id)
grou_idx.delete(name, track_id)
void void group :: list(list<string> &);
return group_idx.keys();
void group :: get_tracks(name):
return group_idx[name]
Library: (lib/library.cpp)
The library manages databases containing track information added by the
user. Ocarina 6 splits the library into multiple database tables for
@ -553,14 +497,24 @@ Library: (lib/library.cpp)
When a library : Track is created, it should be added to the "Library"
group if it is NOT a member of the banned songs group.
- Databases:
enum DB_Type {
DB_ALBUM,
DB_ARTIST,
DB_GENRE,
DB_LIBRARY,
DB_TRACK,
};
- Album:
class library :: Album : public DatabaseEntry {
public:
string name;
short year;
unsigned int year;
unsigned int artist_id;
};
File << year << name
File << artist_id << year << name
- Artist:
class library :: Artist : public DatabaseEntry {
@ -578,8 +532,8 @@ Library: (lib/library.cpp)
File << name
- Path:
class library :: Path : public DatabaseEntry {
- Library:
class library :: Library : public DatabaseEntry {
public:
string root_path;
bool enabled;
@ -590,30 +544,31 @@ Library: (lib/library.cpp)
- Track:
class library :: Track : public DatabaseEntry {
public:
unsigned int library_id;
unsigned int artist_id;
unsigned int album_id;
unsigned int genre_id;
unsigned int library_id;
short track;
short last_year;
short last_month;
short last_day;
unsigned int track;
unsigned int last_year;
unsigned int last_month;
unsigned int last_day;
unsigned int play_count;
unsigned int length;
bool banned;
string title;
string length_str;
string filepath;
};
File << artist_id << album_id << genre_id << library_id << track << last_year
File << last_year << last_month << last_day << play_count << length << banned << endl
File << library_id << artist_id << album_id << genre_id << track << last_year
File << last_month << last_day << play_count << length << banned << endl
File << title << endl;
File << filepath << endl;
- Track: /* This struct lies outside the library namespace */
struct Track {
- Song:
struct Song {
library :: Album *album;
library :: Artist *artist;
library :: Genre *genre;
@ -650,42 +605,39 @@ Library: (lib/library.cpp)
happen while idle.
- Testing:
A test library should be created to test adding tags and anything else.
The command `arecord -d 10 </dev/zero | lame - -b 32 -h silence.mp3`
will create a 10 second long, silent mp3 file. Do something with this
command and figure out how to apply tags!
The script tests/library/gen_library.sh will create a sample library
in the /tmp/ directory for testing purposes. All the track files are
complete silence, but the script will fake up tags for each file.
- API
library :: init();
void library :: init();
Initialize databases and read files from disk. Fill out
groups and prepare filter as tracks are read.
library :: add_path(string dir);
Add new row to paths table, update
bool library :: add_path(string dir);
If dir is not a directory:
return false
library :: del_path(unsigned int lib_id);
Invalidate a path row and all tracks owned by that path
Add new row to the library_db table, begin an update only
on the new path.
return true
library :: update_path(lib_id);
Update the given library path, if valid.
void library :: del_path(unsigned int lib_id);
Invalidate a library_db row and all tracks owned by that path
struct Track library :: resolve(track_id)
Fill out a Track structure for the provided track_id
void library :: update_path(lib_id);
Update the given library_db row, if valid.
const Database<library :: Album> &library :: get_albums();
Return the album database.
struct Song library :: lookup(track_id)
Fill out a Song structure for the provided track_id
const Database<library :: Artist> &library :: get_artists();
Return the artist database.
#ifdef CONFIG_DEBUG
void library :: print_db(DB_Type);
Print the database corresponding to DB_Type
const Database<library :: Genre> &library :: get_genres();
Return the genre database.
const Database<library :: Library> &library :: get_libraries();
Return the library database.
const Database<library :: Track> &library :: get_tracks();
Return the track database.
void library :: reset();
Clear all databases, returning the library to an empty state.
endif /* CONFIG_DEBUG */
@ -767,6 +719,88 @@ Playlist: (lib/playlist.cpp)
- API
void playlist :: init();
Read in the playlist file
* Playlist *playlist :: add();
Add a new playlist to the deck, return the created playlist
to the caller.
* void playlist :: remove(Playlist *);
Remove the provided playlist from the deck. The pointer will
be unusable after calling this function.
void playlist :: move(Playlist *, unsigned int);
Move the playlist to the provided location in the deck.
* trackid_t playlist :: next()
Return the next trackid from the top playlist on the playlist
deck (id = deck[0].next()). If the top playlist is now empty,
remove it.
void playlist :: prev()
Keep a playlist :: Playlist recent(PL_ENABLED)
Whenever next() is called, add the returned track to the front
of this playlist, reset recent.cur to 0.
When prev() is called, return recent.next();
trackid_t playlist :: Playlist :: next()
If PL_RANDOM is set:
Randomly pick a value between 1 and size(). Increment
the cur pointer by this value, taking into account any
roll over.
Else:
cur += 1, if cur == size(): cur = 0;
if PL_DRAIN is set:
Remove the trackid pointed to by cur from the list and
return its value.
return list[cur]
void playlist :: Playlist :: set_flag(flag);
Set the user-requested flag
void playlist :: Playlist :: clear_flag(flag);
Clear the user-requested flag
const unsigned int playlist :: Playlist :: get_flags();
return flags
bool playlist :: Playlist :: play_row(unsigned int id);
Call this fuction to play a song from the playlist. id matches
up to the index in the playlist to play. Return true if the
selected row should be removed from the playlist, false
otherwise.
string playlist :: Playlist :: get_length()
Calculate the length of the playlist and return a string
in mm:ss format with the results.
unsigned int playlist :: Playlist :: size()
Return the number of tracks in your playlist
unsigned int playlist :: Playlist :: current_index()
Return the current index into the playlist
unsigned int playlist :: Playlist :: filter(string text)
Set the current filter text
unsigned int playlist :: Playlist :: is_visible(id)
Check if the playlist id is visible based on playlis text
* unsigned int playlist :: Playlist :: add_track(trackid_t)
Add a new track to the playlist
* unsigned int playlist :: Playlist :: rm_track(playlistid_t)
Remove a row from the playlist
Audio: (lib/audio.cpp)
This file will introduce an "audio" namespace containing all of the
functions interacting with gstreamer. This will create a wrapper
@ -775,7 +809,9 @@ Audio: (lib/audio.cpp)
The audio layer will also control the "pause after N tracks" feature
so songs can be loaded without neeting to pass in a "begin playback"
flag every time.
flag every time. The remaining tracks counter will only be decremented
when a song finishes through the end-of-stream message passed by the
gst pipeline.
- Internal:
Set up a message bus to look for end-of-stream and error messages so
@ -784,51 +820,50 @@ Audio: (lib/audio.cpp)
count.
- API:
audio :: init(argc, argv)
void audio :: init(argc, argv)
Initialize the gstreamer layer and reload the track that was
last loaded before shutdown. Please only pass --gst-* options
for argv.
audio :: load_file(filepath)
Loads a file path but does not begin playback.
void audio :: play()
Begin or resume playback.
audio :: play()
Begin playback
void audio :: pause()
Pause playback.
audio :: pause()
Pause playback
audio :: seek_to(X)
void audio :: seek_to(int)
Seek to a position X seconds into the track
audio :: stop()
void audio :: stop()
pause()
seek_to(0)
audio :: pause_after(N)
void audio :: next()
Call the playlist :: next() function to get the next trackid,
and load that file into the gstreamer pipeline. Do not change
the state of the pipeline (if nothing is playing yet, don't
call play()).
void audio :: previous()
Call the playlist :: previous() function to iterate backwards
through the recently played playlist. Load the returned trackid
without changing the pipeline state.
trackid audio :: current_trackid()
Return the trackid of the currently playing song.
unsigned int audio :: position()
Return the number of seconds that the song has played.
unsigned int audio :: duration()
Return the duration of the current song in seconds.
void audio :: pause_after(unsigned int)
Pause after N tracks, pass a negative number to disable.
audio :: position()
Return the number of seconds that the song has played
audio :: duration()
Return the duration of the current song in seconds
Preferences: (lib/prefs.cpp)
Preferences make use of a special index where the set<int> is always
size 1. Preferences will be in the prefs namespace.
- Index:
Index prefs(prefs.idx);
- API:
prefs :: set(string, val);
prefs.replace(string, val);
prefs :: get(string)
return prefs[string].begin()
unsigned int audio :: pause_count()
Return the number of tracks that will be played before
playback pauses.
@ -924,3 +959,7 @@ Future work:
- Extra testing ideas: (6.1)
- Run tests through valgrind to help find memory leaks
- Combine earlier tests into a single file
- Exceptions: (6.1)
- Don't return error codes, throw exceptions like C++ is
designed to do.

47
design/database.txt
View File

@ -34,6 +34,7 @@ Database: (lib/database.cpp)
public:
bool valid;
virtual const std::string &primary_key() = 0;
virtual void write(File &) = 0;
virtual void read(File &) = 0;
virtual void print() = 0;
@ -41,24 +42,34 @@ Database: (lib/database.cpp)
File << <CHILD_CLASS_DATA>
- Flags:
enum DatabaseFlags {
DB_NORMAL,
DB_UNIQUE,
}
- IndexEntry:
class IndexEntry : public DatabaseEntry {
public:
string key;
vector<unsigned int> values;
const std::string &primary_key();
void write(File &);
void read(File &);
void print();
};
File << key << endl;
File << values.size() << values[0] << .. << values[N] << endl;
- Database:
template <class T>
class Database {
private:
vector<T> db;
map<std::string, unsigned int> keys;
unsigned int _size; /* Number of valid rows */
File file;
DatabaseFlags flags;
public:
Database::Database(filename, flags);
void load();
void save();
void clear();
void print();
unsigned int insert(T);
@ -68,7 +79,8 @@ Database: (lib/database.cpp)
unsigned int first();
unsigned int last();
unsigned int next();
unsigned int next(unsigned int &);
unsigned int find(const std::string &);
T &operator[](unsigned int);
};
@ -100,9 +112,9 @@ Database: (lib/database.cpp)
template <class T>
unsigned int Database :: insert(T &);
Adds a new item to the db, marks it as valid, and returns the
id of the item. if DB_UNIQUE is set, check if the item already
exists in the DB and return it's ID instead of adding a new item.
Adds a new item to the db and marks it as valid. A reverse
mapping is also created into the keys map to map the primary
key back to the id of the newly added item.
void Database :: delete(unsigned int index);
Mark db[index] as invalid (quick deletion).
@ -114,17 +126,22 @@ Database: (lib/database.cpp)
Return db.size().
unsigned int Database :: first();
Return the id to the first valid row or return db.size()
if there are no valid rows.
Return the id to the first valid row.
unsigned int Database :: last();
Return the id of the last valid row or return db.size()
if there are no valid rows.
Return the id of the last valid row.
unsigned int Database :: next(unsigned int &id)
Return the id of the next valid row or return db.size()
if there are no remaining valid rows.
template <class T>
T &Database :: find(const std::string &key);
Search for primary key "key" in the database. The reverse
mapping should be used to make this operation faster. Throw
an empty exception if the key is not found in the mapping.
template <class T>
T &Database :: operator[unsigned int index]
Return a reference to db[index].
Return a reference to db[index]. If index is out of range,
throw an empty exception.

4
design/filter.txt
View File

@ -5,7 +5,7 @@
filter.cpp
== Depends ==
index
database
Filter: (lib/filter.cpp)
Filtering is used to generate a subset of songs displayed by the UI to
@ -14,7 +14,7 @@ Filter: (lib/filter.cpp)
the next time the application starts.
- Index:
Index filter_index("");
Database<database :: IndexEntry> filter_index;
map<string, string> lowercase_cache;
unsigned int lowercase_cache_hits;

4
design/groups.txt
View File

@ -7,7 +7,7 @@
groups.idx
== Depends ==
index
database
Groups: (lib/group.cpp)
Groups are going to be a new feature in Ocarina 6 and can compare
@ -19,7 +19,7 @@ Groups: (lib/group.cpp)
releases will store user-defined groups in a file on disk.
- Index:
Index group_idx()
Database<database :: IndexEntry> group_db
- Default groups:
All music

2
design/gui.txt
View File

@ -3,7 +3,7 @@
*
== Depends ==
idle file
idle file audio library
Gui: (ocarina/*)
The GUI will be written in C++ using gtkmm3 for (hopefully) cleaner code.

80
design/index.txt
View File

@ -1,80 +0,0 @@
== Files ==
ocarina/include/
index.h
ocarina/lib/
index.cpp
== Depends ==
idle file
Index: (lib/index.cpp)
An inverted index allows me to map multiple values to a single key.
Keys are tracked separately from the rest of the map so they can be
found and iterated over without writing ugly code.
- Index:
class Index {
private:
map<string, set<unsigned int>> index;
set<string> keys;
File file;
public:
Index::Index(filename);
void load();
void save();
void print();
void insert(key, unsigned int);
void remove(key);
void remove(key, unsigned int);
const set<string>::iterator keys_begin();
const set<string>::iterator keys_end();
const set<unsigned int> &operator[](string);
};
File << keys.size() << endl;
File << key << endl;
File << map[key].size() << int_0 << int_1 << ... << int_n << endl;
- API:
Index :: Index(filename);
Initializes an index using ~/.ocarina{-debug}K/filename. Pass
an empty string if you do not want this index to be saved.
void Index :: load();
Reads data from a file. Call after static initialization of
Ocarina to ensure idle tasks are configured
void Index :: save();
Saves data to file
void Index :: print();
This function exists only if CONFIG_DEBUG is enabled.
Following a similar format for writing to disk, print the
index to the console in a human-readable format.
void Index :: print_keys();
This function exists only if CONFIG_DEBUG is enabled.
Print the database keys to the console in a human-readable
format.
void Index :: insert(key, unsigned int);
1) If key does not exist, create it.
2) Add int to the set for the given key
void Index :: remove(key);
Remove a key from the index;
void Index :: remove(key, unsigned int);
1) Remove int from the set of values associated with key
2) If the set is empty, remove the key
const set<string>::iterator void Index :: keys_begin()
Return an iterator pointing to the beginning of the keys set.
const set<string>::iterator void Index :: keys_end()
Return an iterator pointing to the end of the keys set.
const set<unsigned int> &Index :: operator[](string key);
Return the set associated with key