Completely replace old interpreter with Faye and update docs

README.md

@@ -2,7 +2,7 @@
 
 ![repl_demo](https://raw.githubusercontent.com/NGnius/mps/master/extras/demo.png)
 
-A language all about iteration to play your music files.
+Sort, filter and analyse your music to create great playlists.
 This project implements the interpreter (mps-interpreter), music player (mps-player), and CLI interface for MPS (root).
 The CLI interface includes a REPL for running scripts.
 The REPL interactive mode also provides more details about using MPS through the `?help` command.
@@ -47,7 +47,7 @@ One day I'll add pretty REPL example pictures and some script files...
 **Music Playlist Script (MPS) is technically a query language for music files.** It uses an (auto-generated) SQLite3 database for SQL queries and can also directly query the filesystem. Queries can be modified by using filters, functions, and sorters built-in to MPS (see mps-interpreter's README.md).
 
 ### Is MPS a scripting language?
-**Yes**. It evolved from a simple query language into something that can do arbitrary calculations. Whether it's Turing-complete is still unproved, but it's powerful enough to do what I want it to do.
+**Yes**. It evolved from a simple query language into something that can do arbitrary calculations. Whether it's Turing-complete is still unproven, but it's powerful enough for what I want it to do.
 
 
 ## License

mps-interpreter/README.md

@@ -2,13 +2,10 @@
 
 All necessary components to interpret and run a MPS script.
 
-MpsInterpretor uses a non-standard Iterator implementation,
+MpsFaye is the new interpreter, which replaces the old MpsInterpretor and MpsRunner types.
-so it is recommended to use MpsRunner to execute a script.
+MpsDebugger can be used to run scripts with a custom debug harness.
 Since MPS is centered around iterators, script execution is also done by iterating.
 
-MpsInterpretor is misspelt to emphasise that it behaves strangely:
-after every MPS statement, a None item is returned even when the script is not complete.
-MpsRunner wraps MpsInterpretor so that this behaviour is hidden when iterating.
 
 ```rust
 use std::io::Cursor;
@@ -18,7 +15,7 @@ let cursor = Cursor::new(
 "files(folder=`~/Music/`, recursive=true)" // retrieve all files from Music folder
 );
 
-let interpreter = MpsRunner::with_stream(cursor);
+let interpreter = MpsFaye::with_stream(cursor);
 
 // warning: my library has ~3800 songs, so this outputs too much information to be useful.
 for result in interpreter {
@@ -96,7 +93,7 @@ Keep only items which are do not duplicate another item, or keep only items whoe
 
 ### Functions
 Similar to most other languages: `function_name(param1, param2, etc.);`.
-These always return an iterable which can be manipulated.
+These always return an iterable which can be manipulated with other syntax (filters, sorters, etc.).
 Functions are statements of the format `function_name(params)`, where "function_name" is one of the function names (below) and params is a valid parameter input for the function.
 Each function is responsible for parsing it's own parameters when the statement is parsed, so this is very flexible.
 E.g. `files(folder="~/Music/", recursive=true);` is valid function syntax to execute the files function with parameters `folder="~/Music/", recursive=true`.
@@ -104,7 +101,7 @@ E.g. `files(folder="~/Music/", recursive=true);` is valid function syntax to exe
 
 #### sql_init(generate = true|false, folder = "path/to/music");
 
-Initialize the SQLite database connection using the provided parameters. This must be performed before any other database operation (otherwise the database will already be connected with default settings).
+Initialize the SQLite database connection using the provided parameters. This must be performed before any other database operation (otherwise the database will already be connected with default settings). This returns an empty iterable (contains zero items).
 
 #### sql("SQL query here");
 
@@ -140,19 +137,19 @@ Explicitly reset an iterable. This useful for reusing an iterable variable.
 
 #### interlace(iterable1, iterable2, ...);
 
-Combine multiple iterables in an interleaved pattern. This is a variant of union(...) where the first item in iterable1, then iterable2, ... is returned, then the second item, etc. until all iterables are depleted. There is no limit on the amount of iterables which can be provided as parameters.
+Combine multiple iterables in an interleaved pattern. This is a variant of union(...) where the first item in iterable1, then iterable2, ... is returned, then the second item, etc. until all iterables are depleted. There is no limit to the amount of iterables which can be provided as parameters.
 
 #### union(iterable1, iterable2, ...);
 
-Combine multiple iterables in a sequential pattern. All items in iterable1 are returned, then all items in iterable2, ... until all provided iterables are depleted. There is no limit on the amount of iterables which can be provided as parameters.
+Combine multiple iterables in a sequential pattern. All items in iterable1 are returned, then all items in iterable2, ... until all provided iterables are depleted. There is no limit to the amount of iterables which can be provided as parameters.
 
 #### intersection(iterable1, iterable2, ...);
 
-Combine multiple iterables such that only items that exist in iterable1 and iterable2 and ... are returned. The order of items from iterable1 is maintained. There is no limit on the amount of iterables which can be provided as parameters.
+Combine multiple iterables such that only items that exist in iterable1 and iterable2 and ... are returned. The order of items from iterable1 is maintained. There is no limit to the amount of iterables which can be provided as parameters.
 
 #### empty();
 
-Empty iterator. Useful for deleting items using replacement filters.
+Empty iterator containing zero items. Useful for deleting items using replacement filters.
 
 #### empties(count);
 

mps-interpreter/src/debug.rs

@@ -0,0 +1,48 @@
+use std::iter::Iterator;
+
+use super::tokens::MpsTokenReader;
+use super::{MpsError, MpsFaye, MpsItem};
+
+/// Wrapper for MpsFaye with a built-in callback function for every iteration of the interpreter.
+pub struct MpsDebugger<'a, 'b, T>
+where
+    T: MpsTokenReader,
+{
+    interpreter: MpsFaye<'a, T>,
+    transmuter: &'b dyn Fn(
+        &mut MpsFaye<'a, T>,
+        Option<Result<MpsItem, MpsError>>,
+    ) -> Option<Result<MpsItem, MpsError>>,
+}
+
+impl<'a, 'b, T> MpsDebugger<'a, 'b, T>
+where
+    T: MpsTokenReader,
+{
+    /// Create a new instance of MpsDebugger using the provided interpreter and callback.
+    pub fn new(
+        faye: MpsFaye<'a, T>,
+        item_handler: &'b dyn Fn(
+            &mut MpsFaye<'a, T>,
+            Option<Result<MpsItem, MpsError>>,
+        ) -> Option<Result<MpsItem, MpsError>>,
+    ) -> Self {
+        Self {
+            interpreter: faye,
+            transmuter: item_handler,
+        }
+    }
+}
+
+impl<'a, 'b, T> Iterator for MpsDebugger<'a, 'b, T>
+where
+    T: MpsTokenReader,
+{
+    type Item = Result<MpsItem, MpsError>;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        let next_item = self.interpreter.next();
+        let transmuted_next = (self.transmuter)(&mut self.interpreter, next_item);
+        transmuted_next
+    }
+}

mps-interpreter/src/faye.rs

@@ -3,14 +3,14 @@ use std::io::Read;
 use std::iter::Iterator;
 
 use super::lang::{MpsLanguageDictionary, MpsLanguageError, MpsOp};
-use super::tokens::{MpsToken, MpsTokenizer};
+use super::tokens::{MpsToken, MpsTokenReader, MpsTokenizer};
 use super::MpsContext;
 use super::MpsError;
 use super::MpsItem;
 
 const DEFAULT_TOKEN_BUFFER_SIZE: usize = 16;
 
-pub enum MpsDebuggableEvent {
+pub enum MpsInterpreterEvent {
     FileEnd,
     StatementComplete,
     NewStatementReady,
@@ -19,26 +19,26 @@ pub enum MpsDebuggableEvent {
 /// The script interpreter.
 pub struct MpsFaye<'a, T>
 where
-    T: crate::tokens::MpsTokenReader,
+    T: MpsTokenReader,
 {
     tokenizer: T,
     buffer: VecDeque<MpsToken>,
     current_stmt: Box<dyn MpsOp>,
     vocabulary: MpsLanguageDictionary,
-    callback: &'a dyn Fn(&mut MpsFaye<'a, T>, MpsDebuggableEvent) -> Result<(), MpsError>,
+    callback: &'a dyn Fn(&mut MpsFaye<'a, T>, MpsInterpreterEvent) -> Result<(), MpsError>,
 }
 
 #[inline]
-fn empty_callback<'a, T: crate::tokens::MpsTokenReader>(
+fn empty_callback<'a, T: MpsTokenReader>(
     _s: &mut MpsFaye<'a, T>,
-    _d: MpsDebuggableEvent,
+    _d: MpsInterpreterEvent,
 ) -> Result<(), MpsError> {
     Ok(())
 }
 
 /*impl <T> MpsFaye<'static, T>
 where
-    T: crate::tokens::MpsTokenReader,
+    T: MpsTokenReader,
 {
     /// Create a new interpreter for the provided token reader, using the standard MPS language.
     #[inline]
@@ -64,12 +64,11 @@ impl<'a, R: Read> MpsFaye<'a, MpsTokenizer<R>> {
 
 impl<'a, T> MpsFaye<'a, T>
 where
-    T: crate::tokens::MpsTokenReader,
+    T: MpsTokenReader,
 {
     #[inline]
     pub fn with_standard_vocab(token_reader: T) -> Self {
-        let mut vocab = MpsLanguageDictionary::default();
+        let vocab = MpsLanguageDictionary::standard();
-        super::interpretor::standard_vocab(&mut vocab);
         Self::with_vocab(vocab, token_reader)
     }
 
@@ -84,7 +83,7 @@ where
     pub fn with(
         vocab: MpsLanguageDictionary,
         token_reader: T,
-        debugger: &'a dyn Fn(&mut MpsFaye<'a, T>, MpsDebuggableEvent) -> Result<(), MpsError>,
+        callback: &'a dyn Fn(&mut MpsFaye<'a, T>, MpsInterpreterEvent) -> Result<(), MpsError>,
     ) -> Self {
         Self {
             tokenizer: token_reader,
@@ -93,34 +92,36 @@ where
                 context: Some(MpsContext::default()),
             }),
             vocabulary: vocab,
-            callback: debugger,
+            callback: callback,
         }
     }
 
+    // build a new statement
     #[inline]
-    fn new_statement(
+    fn new_statement(&mut self) -> Option<Result<Box<dyn MpsOp>, MpsError>> {
-        tokenizer: &mut T,
+        while !self.tokenizer.end_of_file() && self.buffer.is_empty() {
-        buffer: &mut VecDeque<MpsToken>,
+            let result = self.tokenizer.next_statement(&mut self.buffer);
-        vocab: &MpsLanguageDictionary,
-    ) -> Option<Result<Box<dyn MpsOp>, MpsError>> {
-        while !tokenizer.end_of_file() && buffer.is_empty() {
-            let result = tokenizer.next_statement(buffer);
             match result {
                 Ok(_) => {}
-                Err(e) => return Some(Err(error_with_ctx(e, tokenizer.current_line()))),
+                Err(e) => return Some(Err(error_with_ctx(e, self.tokenizer.current_line()))),
             }
         }
-        if buffer.is_empty() {
+        if self.buffer.is_empty() {
+            let callback_result = (self.callback)(self, MpsInterpreterEvent::FileEnd);
+            match callback_result {
+                Ok(_) => {}
+                Err(e) => return Some(Err(e)),
+            }
             return None;
         }
-        let result = vocab.try_build_statement(buffer);
+        let result = self.vocabulary.try_build_statement(&mut self.buffer);
         let stmt = match result {
             Ok(stmt) => stmt,
-            Err(e) => return Some(Err(error_with_ctx(e, tokenizer.current_line()))),
+            Err(e) => return Some(Err(error_with_ctx(e, self.tokenizer.current_line()))),
         };
         #[cfg(debug_assertions)]
-        if !buffer.is_empty() {
+        if !self.buffer.is_empty() {
-            panic!("Token buffer was not emptied! (rem: {:?})", buffer)
+            panic!("Token buffer was not emptied! (rem: {:?})", self.buffer)
         }
         Some(Ok(stmt))
     }
@@ -128,7 +129,7 @@ where
 
 impl<'a, T> Iterator for MpsFaye<'a, T>
 where
-    T: crate::tokens::MpsTokenReader,
+    T: MpsTokenReader,
 {
     type Item = Result<MpsItem, MpsError>;
 
@@ -141,28 +142,25 @@ where
                 None => {
                     // current_stmt has terminated
                     if self.tokenizer.end_of_file() {
-                        // notify reached end of file
+                        // always try to read at least once, in case stream gets new data (e.g. in a REPL)
-                        let callback_result = (self.callback)(self, MpsDebuggableEvent::FileEnd);
+                        let result = self.tokenizer.next_statement(&mut self.buffer);
-                        match callback_result {
+                        match result {
                             Ok(_) => {}
-                            Err(e) => return Some(Err(e)),
+                            Err(e) => {
+                                return Some(Err(error_with_ctx(e, self.tokenizer.current_line())))
+                            }
                         }
-                        // nothing left to return
-                        return None;
                     } else {
                         // notify old statement is complete
                         let callback_result =
-                            (self.callback)(self, MpsDebuggableEvent::StatementComplete);
+                            (self.callback)(self, MpsInterpreterEvent::StatementComplete);
                         match callback_result {
                             Ok(_) => {}
                             Err(e) => return Some(Err(e)),
                         }
+                    }
                     // build next statement
-                        let result = Self::new_statement(
+                    let result = self.new_statement();
-                            &mut self.tokenizer,
-                            &mut self.buffer,
-                            &self.vocabulary,
-                        );
                     let mut stmt = match result {
                         Some(Ok(stmt)) => stmt,
                         Some(Err(e)) => return Some(Err(e)),
@@ -173,7 +171,7 @@ where
                     self.current_stmt = stmt;
                     // notify new statement is ready
                     let callback_result =
-                            (self.callback)(self, MpsDebuggableEvent::NewStatementReady);
+                        (self.callback)(self, MpsInterpreterEvent::NewStatementReady);
                     match callback_result {
                         Ok(_) => {}
                         Err(e) => return Some(Err(e)),
@@ -183,7 +181,6 @@ where
         }
     }
 }
-}
 
 fn error_with_ctx<T: std::convert::Into<MpsError>>(error: T, line: usize) -> MpsError {
     let mut err = error.into();

mps-interpreter/src/interpretor.rs

@@ -1,150 +1,6 @@
-use std::collections::VecDeque;
+use super::lang::MpsLanguageDictionary;
-use std::fs::File;
-use std::iter::Iterator;
-use std::path::Path;
 
-use super::lang::{MpsLanguageDictionary, MpsLanguageError, MpsOp};
+/// Builder function to add the standard statements parsers for MPS interpreters.
-use super::tokens::MpsToken;
-use super::MpsContext;
-use super::MpsError;
-use super::MpsItem;
-
-/// The script interpreter.
-/// Use MpsRunner for a better interface.
-pub struct MpsInterpretor<T>
-where
-    T: crate::tokens::MpsTokenReader,
-{
-    tokenizer: T,
-    buffer: VecDeque<MpsToken>,
-    current_stmt: Option<Box<dyn MpsOp>>,
-    vocabulary: MpsLanguageDictionary,
-    context: Option<MpsContext>,
-}
-
-pub fn interpretor<R: std::io::Read>(stream: R) -> MpsInterpretor<crate::tokens::MpsTokenizer<R>> {
-    let tokenizer = crate::tokens::MpsTokenizer::new(stream);
-    MpsInterpretor::with_standard_vocab(tokenizer)
-}
-
-impl<T> MpsInterpretor<T>
-where
-    T: crate::tokens::MpsTokenReader,
-{
-    pub fn with_vocab(tokenizer: T, vocab: MpsLanguageDictionary) -> Self {
-        Self {
-            tokenizer,
-            buffer: VecDeque::new(),
-            current_stmt: None,
-            vocabulary: vocab,
-            context: None,
-        }
-    }
-
-    pub fn with_standard_vocab(tokenizer: T) -> Self {
-        let mut result = Self {
-            tokenizer,
-            buffer: VecDeque::new(),
-            current_stmt: None,
-            vocabulary: MpsLanguageDictionary::default(),
-            context: None,
-        };
-        standard_vocab(&mut result.vocabulary);
-        result
-    }
-
-    pub fn context(&mut self, ctx: MpsContext) {
-        self.context = Some(ctx)
-    }
-
-    pub fn is_done(&self) -> bool {
-        self.tokenizer.end_of_file() && self.current_stmt.is_none()
-    }
-}
-
-impl MpsInterpretor<crate::tokens::MpsTokenizer<File>> {
-    pub fn standard_file<P: AsRef<Path>>(path: P) -> std::io::Result<Self> {
-        let file = File::open(path)?;
-        let tokenizer = crate::tokens::MpsTokenizer::new(file);
-        let mut result = Self {
-            tokenizer,
-            buffer: VecDeque::new(),
-            current_stmt: None,
-            vocabulary: MpsLanguageDictionary::default(),
-            context: None,
-        };
-        standard_vocab(&mut result.vocabulary);
-        Ok(result)
-    }
-}
-
-impl<T> Iterator for MpsInterpretor<T>
-where
-    T: crate::tokens::MpsTokenReader,
-{
-    type Item = Result<MpsItem, MpsError>;
-
-    fn next(&mut self) -> Option<Self::Item> {
-        let mut is_stmt_done = false;
-        let result = if let Some(stmt) = &mut self.current_stmt {
-            let next_item = stmt.next();
-            if next_item.is_none() {
-                is_stmt_done = true;
-            }
-            next_item.map(|item| item.map_err(|e| error_with_ctx(e, self.tokenizer.current_line())))
-        } else {
-            /*if self.tokenizer.end_of_file() {
-                return None;
-            }*/
-            //println!("try get next statement");
-            // build new statement
-            let token_result = self
-                .tokenizer
-                .next_statement(&mut self.buffer)
-                .map_err(|e| error_with_ctx(e, self.tokenizer.current_line()));
-            match token_result {
-                Ok(_) => {}
-                Err(x) => return Some(Err(x)),
-            }
-            if self.tokenizer.end_of_file() && self.buffer.is_empty() {
-                return None;
-            }
-            let stmt = self.vocabulary.try_build_statement(&mut self.buffer);
-            match stmt {
-                Ok(mut stmt) => {
-                    #[cfg(debug_assertions)]
-                    if !self.buffer.is_empty() {
-                        panic!("Token buffer was not emptied! (rem: {:?})", self.buffer)
-                    }
-                    stmt.enter(self.context.take().unwrap_or_default());
-                    self.current_stmt = Some(stmt);
-                    let next_item = self.current_stmt.as_mut().unwrap().next();
-                    if next_item.is_none() {
-                        is_stmt_done = true;
-                    }
-                    next_item.map(|item| {
-                        item.map_err(|e| error_with_ctx(e, self.tokenizer.current_line()))
-                    })
-                }
-                Err(e) => {
-                    Some(Err(e).map_err(|e| error_with_ctx(e, self.tokenizer.current_line())))
-                }
-            }
-        };
-        if is_stmt_done {
-            self.context = Some(self.current_stmt.take().unwrap().escape());
-        }
-        result
-    }
-}
-
-fn error_with_ctx<T: std::convert::Into<MpsError>>(error: T, line: usize) -> MpsError {
-    let mut err = error.into();
-    err.set_line(line);
-    err
-}
-
-/// Builder function to add the standard statements of MPS.
 pub(crate) fn standard_vocab(vocabulary: &mut MpsLanguageDictionary) {
     vocabulary
         // filters

mps-interpreter/src/lang/dictionary.rs

@@ -45,6 +45,12 @@ impl MpsLanguageDictionary {
             vocabulary: Vec::new(),
         }
     }
+
+    pub fn standard() -> Self {
+        let mut new = Self::new();
+        crate::interpretor::standard_vocab(&mut new);
+        new
+    }
 }
 
 impl Default for MpsLanguageDictionary {

mps-interpreter/src/lib.rs

@@ -1,12 +1,9 @@
 //! All necessary components to interpret and run a MPS script.
 //!
-//! MpsInterpretor uses a non-standard Iterator implementation,
+//! MpsFaye is the new interpreter, which replaces the old MpsInterpretor and MpsRunner types.
-//! so it is recommended to use MpsRunner to execute a script.
+//! MpsDebugger can be used to run scripts with a custom debug harness.
 //! Since MPS is centered around iterators, script execution is also done by iterating.
 //!
-//! MpsInterpretor is misspelt to emphasise that it behaves strangely:
-//! after every MPS statement, a None item is returned even when the script is not complete.
-//! MpsRunner wraps MpsInterpretor so that this behaviour is hidden when iterating.
 //!
 //! ```
 //! use std::io::Cursor;
@@ -16,7 +13,7 @@
 //! "files(folder=`~/Music/`, recursive=true)" // retrieve all files from Music folder
 //! );
 //!
-//! let interpreter = MpsRunner::with_stream(cursor);
+//! let interpreter = MpsFaye::with_stream(cursor);
 //!
 //! // warning: my library has ~3800 songs, so this outputs too much information to be useful.
 //! for result in interpreter {
@@ -94,7 +91,7 @@
 //!
 //! ## Functions
 //! Similar to most other languages: `function_name(param1, param2, etc.);`.
-//! These always return an iterable which can be manipulated.
+//! These always return an iterable which can be manipulated with other syntax (filters, sorters, etc.).
 //! Functions are statements of the format `function_name(params)`, where "function_name" is one of the function names (below) and params is a valid parameter input for the function.
 //! Each function is responsible for parsing it's own parameters when the statement is parsed, so this is very flexible.
 //! E.g. `files(folder="~/Music/", recursive=true);` is valid function syntax to execute the files function with parameters `folder="~/Music/", recursive=true`.
@@ -102,7 +99,7 @@
 //!
 //! ### sql_init(generate = true|false, folder = "path/to/music");
 //!
-//! Initialize the SQLite database connection using the provided parameters. This must be performed before any other database operation (otherwise the database will already be connected with default settings).
+//! Initialize the SQLite database connection using the provided parameters. This must be performed before any other database operation (otherwise the database will already be connected with default settings). This returns an empty iterable (contains zero items).
 //!
 //! ### sql("SQL query here");
 //!
@@ -138,19 +135,19 @@
 //!
 //! ### interlace(iterable1, iterable2, ...);
 //!
-//! Combine multiple iterables in an interleaved pattern. This is a variant of union(...) where the first item in iterable1, then iterable2, ... is returned, then the second item, etc. until all iterables are depleted. There is no limit on the amount of iterables which can be provided as parameters.
+//! Combine multiple iterables in an interleaved pattern. This is a variant of union(...) where the first item in iterable1, then iterable2, ... is returned, then the second item, etc. until all iterables are depleted. There is no limit to the amount of iterables which can be provided as parameters.
 //!
 //! ### union(iterable1, iterable2, ...);
 //!
-//! Combine multiple iterables in a sequential pattern. All items in iterable1 are returned, then all items in iterable2, ... until all provided iterables are depleted. There is no limit on the amount of iterables which can be provided as parameters.
+//! Combine multiple iterables in a sequential pattern. All items in iterable1 are returned, then all items in iterable2, ... until all provided iterables are depleted. There is no limit to the amount of iterables which can be provided as parameters.
 //!
 //! ### intersection(iterable1, iterable2, ...);
 //!
-//! Combine multiple iterables such that only items that exist in iterable1 and iterable2 and ... are returned. The order of items from iterable1 is maintained. There is no limit on the amount of iterables which can be provided as parameters.
+//! Combine multiple iterables such that only items that exist in iterable1 and iterable2 and ... are returned. The order of items from iterable1 is maintained. There is no limit to the amount of iterables which can be provided as parameters.
 //!
 //! ### empty();
 //!
-//! Empty iterator. Useful for deleting items using replacement filters.
+//! Empty iterator containing zero items. Useful for deleting items using replacement filters.
 //!
 //! ### empties(count);
 //!
@@ -240,6 +237,7 @@
 //!
 
 mod context;
+mod debug;
 mod errors;
 mod faye;
 mod interpretor;
@@ -249,17 +247,18 @@ pub mod lang;
 pub mod music;
 //mod music_item;
 pub mod processing;
-mod runner;
+//mod runner;
 pub mod tokens;
 
 pub use context::MpsContext;
+pub use debug::MpsDebugger;
 pub use errors::MpsError;
-pub use faye::MpsFaye;
+pub use faye::{MpsFaye, MpsInterpreterEvent};
-pub use interpretor::{interpretor, MpsInterpretor};
+//pub use interpretor::{interpretor, MpsInterpretor};
 pub use item::MpsItem;
 //pub(crate) use item::MpsItemRuntimeUtil;
 //pub use music_item::MpsMusicItem;
-pub use runner::MpsRunner;
+//pub use runner::MpsRunner;
 
 #[cfg(test)]
 mod tests {}

mps-interpreter/tests/single_line.rs

@@ -1,5 +1,4 @@
 use mps_interpreter::tokens::{MpsToken, MpsTokenizer, ParseError};
-use mps_interpreter::MpsError;
 use mps_interpreter::*;
 use std::collections::VecDeque;
 use std::io::Cursor;
@@ -55,7 +54,7 @@ fn execute_single_line(
     let cursor = Cursor::new(line);
 
     let tokenizer = MpsTokenizer::new(cursor);
-    let interpreter = MpsInterpretor::with_standard_vocab(tokenizer);
+    let interpreter = MpsFaye::with_standard_vocab(tokenizer);
 
     let mut count = 0;
     for result in interpreter {

mps-player/src/controller.rs

@@ -18,7 +18,11 @@ pub struct MpsController {
 }
 
 impl MpsController {
-    pub fn create<F: FnOnce() -> MpsPlayer<T> + Send + 'static, T: MpsTokenReader>(
+    pub fn create<
+        'a,
+        F: FnOnce() -> MpsPlayer<'a, T> + Send + 'static,
+        T: MpsTokenReader + 'static,
+    >(
         player_gen: F,
     ) -> Self {
         let (control_tx, control_rx) = channel();
@@ -42,7 +46,11 @@ impl MpsController {
         }
     }
 
-    pub fn create_repl<F: FnOnce() -> MpsPlayer<T> + Send + 'static, T: MpsTokenReader>(
+    pub fn create_repl<
+        'a,
+        F: FnOnce() -> MpsPlayer<'a, T> + Send + 'static,
+        T: MpsTokenReader + 'static,
+    >(
         player_gen: F,
     ) -> Self {
         let (control_tx, control_rx) = channel();

mps-player/src/player.rs

@@ -5,22 +5,22 @@ use rodio::{decoder::Decoder, OutputStream, OutputStreamHandle, Sink};
 
 use m3u8_rs::{MediaPlaylist, MediaSegment};
 
-use mps_interpreter::{tokens::MpsTokenReader, MpsItem, MpsRunner};
+use mps_interpreter::{tokens::MpsTokenReader, MpsFaye, MpsItem};
 
 use super::PlaybackError;
 
 /// Playback functionality for a script.
 /// This takes the output of the runner and plays or saves it.
-pub struct MpsPlayer<T: MpsTokenReader> {
+pub struct MpsPlayer<'a, T: MpsTokenReader + 'a> {
-    runner: MpsRunner<T>,
+    runner: MpsFaye<'a, T>,
     sink: Sink,
     #[allow(dead_code)]
     output_stream: OutputStream, // this is required for playback, so it must live as long as this struct instance
     output_handle: OutputStreamHandle,
 }
 
-impl<T: MpsTokenReader> MpsPlayer<T> {
+impl<'a, T: MpsTokenReader + 'a> MpsPlayer<'a, T> {
-    pub fn new(runner: MpsRunner<T>) -> Result<Self, PlaybackError> {
+    pub fn new(runner: MpsFaye<'a, T>) -> Result<Self, PlaybackError> {
         let (stream, output_handle) =
             OutputStream::try_default().map_err(PlaybackError::from_err)?;
         Ok(Self {
@@ -201,14 +201,14 @@ impl<T: MpsTokenReader> MpsPlayer<T> {
 #[cfg(test)]
 mod tests {
     use super::*;
-    use mps_interpreter::MpsRunner;
+    use mps_interpreter::MpsFaye;
     use std::io;
 
     #[allow(dead_code)]
     //#[test]
     fn play_cursor() -> Result<(), PlaybackError> {
         let cursor = io::Cursor::new("sql(`SELECT * FROM songs JOIN artists ON songs.artist = artists.artist_id WHERE artists.name like 'thundercat'`);");
-        let runner = MpsRunner::with_stream(cursor);
+        let runner = MpsFaye::with_stream(cursor);
         let mut player = MpsPlayer::new(runner)?;
         player.play_all()
     }
@@ -216,7 +216,7 @@ mod tests {
     #[test]
     fn playlist() -> Result<(), PlaybackError> {
         let cursor = io::Cursor::new("sql(`SELECT * FROM songs JOIN artists ON songs.artist = artists.artist_id WHERE artists.name like 'thundercat'`);");
-        let runner = MpsRunner::with_stream(cursor);
+        let runner = MpsFaye::with_stream(cursor);
         let mut player = MpsPlayer::new(runner)?;
 
         let output_file = std::fs::File::create("playlist.m3u8").unwrap();

mps-player/src/player_wrapper.rs

@@ -11,17 +11,17 @@ use super::PlaybackError;
 /// This allows for message passing between the player and controller.
 ///
 /// You will probably never directly interact with this, instead using MpsController to communicate.
-pub struct MpsPlayerServer<T: MpsTokenReader> {
+pub struct MpsPlayerServer<'a, T: MpsTokenReader + 'a> {
-    player: MpsPlayer<T>,
+    player: MpsPlayer<'a, T>,
     control: Receiver<ControlAction>,
     event: Sender<PlayerAction>,
     playback: Sender<PlaybackAction>,
     keep_alive: bool,
 }
 
-impl<T: MpsTokenReader> MpsPlayerServer<T> {
+impl<'a, T: MpsTokenReader + 'a> MpsPlayerServer<'a, T> {
     pub fn new(
-        player: MpsPlayer<T>,
+        player: MpsPlayer<'a, T>,
         ctrl: Receiver<ControlAction>,
         event: Sender<PlayerAction>,
         playback: Sender<PlaybackAction>,
@@ -146,7 +146,7 @@ impl<T: MpsTokenReader> MpsPlayerServer<T> {
         self.on_end();
     }
 
-    pub fn spawn<F: FnOnce() -> MpsPlayer<T> + Send + 'static>(
+    pub fn spawn<F: FnOnce() -> MpsPlayer<'a, T> + Send + 'static>(
         factory: F,
         ctrl_tx: Sender<ControlAction>,
         ctrl_rx: Receiver<ControlAction>,

src/help.rs

@@ -10,7 +10,7 @@ Similar to most other languages: function_name(param1, param2, etc.)
 These always return an iterable which can be manipulated.
 
  sql_init(generate = true|false, folder = `path/to/music`)
-    Initialize the SQLite database connection using the provided parameters. This must be performed before any other database operation (otherwise the database will already be connected with default settings).
+    Initialize the SQLite database connection using the provided parameters. This must be performed before any other database operation (otherwise the database will already be connected with default settings). Returns an empty iterable.
 
  sql(`SQL query here`)
     Perform a raw SQLite query on the database which MPS auto-generates. An iterator of the results is returned.
@@ -37,13 +37,13 @@ These always return an iterable which can be manipulated.
     Explicitly reset an iterable. This useful for reusing an iterable variable.
 
  interlace(iterable1, iterable2, ...)
-    Combine multiple iterables in an interleaved pattern. This is a variant of union(...) where the first item in iterable1, then iterable2, ... is returned, then the second item, etc. until all iterables are depleted. There is no limit on the amount of iterables which can be provided as parameters.
+    Combine multiple iterables in an interleaved pattern. This is a variant of union(...) where the first item in iterable1, then iterable2, ... is returned, then the second item, etc. until all iterables are depleted. There is no limit to the amount of iterables which can be provided as parameters.
 
  union(iterable1, iterable2, ...)
-    Combine multiple iterables in a sequential pattern. All items in iterable1 are returned, then all items in iterable2, ... until all provided iterables are depleted. There is no limit on the amount of iterables which can be provided as parameters.
+    Combine multiple iterables in a sequential pattern. All items in iterable1 are returned, then all items in iterable2, ... until all provided iterables are depleted. There is no limit to the amount of iterables which can be provided as parameters.
 
  intersection(iterable1, iterable2, ...);
-    Combine multiple iterables such that only items that exist in iterable1 and iterable2 and ... are returned. The order of items from iterable1 is maintained. There is no limit on the amount of iterables which can be provided as parameters.
+    Combine multiple iterables such that only items that exist in iterable1 and iterable2 and ... are returned. The order of items from iterable1 is maintained. There is no limit to the amount of iterables which can be provided as parameters.
 
  empty()
     Empty iterator. Useful for deleting items using replacement filters.

src/main.rs

@@ -1,4 +1,4 @@
-//! A language all about iteration to play your music files.
+//! Sort, filter and analyse your music to create great playlists.
 //! This project implements the interpreter (mps-interpreter), music player (mps-player), and CLI interface for MPS (root).
 //! The CLI interface includes a REPL for running scripts.
 //! The REPL interactive mode also provides more details about using MPS through the `?help` command.
@@ -43,7 +43,7 @@
 //! **Music Playlist Script (MPS) is technically a query language for music files.** It uses an (auto-generated) SQLite3 database for SQL queries and can also directly query the filesystem. Queries can be modified by using filters, functions, and sorters built-in to MPS (see mps-interpreter's README.md).
 //!
 //! ## Is MPS a scripting language?
-//! **Yes**. It evolved from a simple query language into something that can do arbitrary calculations. Whether it's Turing-complete is still unproved, but it's powerful enough to do what I want it to do.
+//! **Yes**. It evolved from a simple query language into something that can do arbitrary calculations. Whether it's Turing-complete is still unproven, but it's powerful enough for what I want it to do.
 //!
 
 mod channel_io;
@@ -54,13 +54,13 @@ mod repl;
 use std::io;
 use std::path::PathBuf;
 
-use mps_interpreter::MpsRunner;
+use mps_interpreter::MpsFaye;
 use mps_player::{MpsController, MpsPlayer, PlaybackError};
 
 #[allow(dead_code)]
 fn play_cursor() -> Result<(), PlaybackError> {
     let cursor = io::Cursor::<&'static str>::new("sql(`SELECT * FROM songs JOIN artists ON songs.artist = artists.artist_id WHERE artists.name like 'thundercat'`);");
-    let runner = MpsRunner::with_stream(cursor);
+    let runner = MpsFaye::with_stream(cursor);
     let mut player = MpsPlayer::new(runner)?;
     player.play_all()
 }
@@ -82,7 +82,7 @@ fn main() {
                 std::fs::File::open(&script_file2)
                     .unwrap_or_else(|_| panic!("Abort: Cannot open file `{}`", &script_file2)),
             );
-            let runner = MpsRunner::with_stream(script_reader);
+            let runner = MpsFaye::with_stream(script_reader);
 
             let player = MpsPlayer::new(runner).unwrap();
             if let Some(vol) = volume {

src/repl.rs

@@ -4,7 +4,7 @@ use std::io::{self, Write};
 
 use console::{Key, Term};
 
-use mps_interpreter::MpsRunner;
+use mps_interpreter::MpsFaye;
 use mps_player::{MpsController, MpsPlayer};
 
 use super::channel_io::{channel_io, ChannelWriter};
@@ -48,7 +48,7 @@ pub fn repl(args: CliArgs) {
     let (writer, reader) = channel_io();
     let volume = args.volume.clone();
     let player_builder = move || {
-        let runner = MpsRunner::with_stream(reader);
+        let runner = MpsFaye::with_stream(reader);
 
         let player = MpsPlayer::new(runner).unwrap();
         if let Some(vol) = volume {