#SpahQL: Query, manipulate and manage JSON data effortlessly. [github]: http://github.com/danski/spahql [js_minified]: https://raw.github.com/danski/spahql/master/spahql-min.js [spahql_getting_started]: #getting_started [selection_queries]: #selecting_data [modifying_data]: #modifying_data [spahql_listeners]: #listening_for_changes [assertion_queries]: #making_assertions [literals_and_sets]: #literals [comparison_operators]: #comparisons [object_equality]: #object_equality [strategies]: #spahql_strategies #Core concepts Think of SpahQL like jQuery, but instead of handling DOM elements, it handles JSON data. Instead of CSS selectors, you use SpahQL queries. It's a querying system for JSON data, aware of hashes, arrays, strings and booleans. You can use SpahQL to fetch deeply-nested data, traverse large trees, and to query for data based on conditions, and to make assertions about data. #Install SpahQL has no dependencies - all you need is SpahQL itself. ##Browser-based apps Download the [latest minified JS][js_minified] and include it in your project. ##Node.js / CommonJS apps SpahQL is available through NPM, so installation is painless. npm install spahql Once installed, require it like any other CommonJS dependency. var spahql = require('spahql'); #Getting started Using SpahQL starts out with you putting your data into a *SpahQL Database*. A *database* isn't a special object - it's just a regular SpahQL object holding the root data. Let's start out with an example - the state for a basic Twitter app UI. var data = { "user": { "logged_in": true, "name": "John Doe", "handle": "johndoe", "avatar": { "small": "https://myapp.com/avatar-small/johndoe.png", "large": "https://myapp.com/avatar-large/johndoe.png" } } }, "draft_status": "The meaning of life is", "active_tab": "timeline", "timeline": [ { "type": "status", "status": "FFFFFFFUUUUUUUUUUUU", "user": { "name": "Rage Guy", "handle": "rageguy", "avatar": { "small": "https://myapp.com/avatar-small/f7u12.png", "large": "https://myapp.com/avatar-large/f7u12.png" } } }, ... ], "mentions": null "direct_messages": null } In this state we've got the user's profile available to us for display, we know that the "timeline" tab is open and populated with some tweets, and we know that the user hasn't loaded any mentions or direct messages just yet. We also know that the user has typed something into the status field but has not yet saved it. We'll be using this example data below to explore SpahQL's capabilities. To start using this data with SpahQL, we need to put it in a _SpahQL database_: var db = SpahQL.db(data); #Selecting data Now that we've got a _SpahQL Database_ assigned to the db variable, we can start to pull data from it using SpahQL _selection queries_. We call the db object the _root_. The query syntax is a little like XPath. Every item in your database can be considered to have a unique *path*, and you can query for that path specifically, or perform more advanced actions such as recursion and filtering. To select items from the database, use the select method. This will return a new SpahQL object containing your results. var user = db.select("/user"); user.length; //-> 1 user.value(); //-> {"logged_in": true, "name": "John Doe", "handle": "johndoe" ... } In the above example, we queried for the path /user which pulled the key "user" from the data. We can also chain keys together: var avatar_large = db.select("/user/avatar/large"); avatar_large.value(); //-> "https://myapp.com/avatar-large/johndoe.png" The select method returns a SpahQL object, so we can scope queries to results we already have: var avatars = db.select("/user/avatar"); var avatar_large = avatars.select("/large") avatar_large.value(); //-> "https://myapp.com/avatar-large/johndoe.png" Much like XPath, SpahQL supports recursion with a double-slash anywhere in the path. To find all avatars, no matter where they appear in the state, we'd do this: var all_avatars = db.select("//avatar"); This will return a set containing multiple results from several places in the original db object: all_avatars.length; //-> 2 all_avatars.paths(); //-> ["/user/avatar", "/timeline/0/user/avatar"] all_avatars.values(); //-> ["https://myapp.com/avatar-large/johndoe.png", "https://myapp.com/avatar-large/f7u12.png"] Notice that the second path returned by all_avatars.paths() starts with /timeline/0. The key 0 refers to the first item an array, and this is how SpahQL handles arrays in general. var second_tweet_in_timeline = db.select("/timeline/1"); The * (asterisk) character works as a wildcard in paths, allowing you to pull every value from an object without recursion. To grab all tweets from the timeline: var timeline_tweets = db.select("/timeline/*") timeline_tweets.paths(); //-> ["/timeline/0", "/timeline/1", "/timeline/2", ...] We can also _filter_ the results at any point in the query. Here's an example where we filter the timeline for all tweets from a given user, and take the actual text of each tweet as the value: var tweets_from_bob = db.select("/timeline/*[/user/handle == 'bob']/status"); In the above, we took all objects from the timeline (/timeline/*) and filtered the list with an assertion ([/user/handle == 'bob']) - then we picked the tweet text from the remaining items (/status). Note that the contents of the filter were scoped to the object being filtered. This is fine for basic cases, but what if you need to compare the handle of each user to something else stored in the database? Let's add a field to the root object, for handling searches: db.set("show_only_from_handle", "coolguy99"); db.select("/show_only_from_handle").value(); //-> "coolguy99" Now to filter the tweets based on this new bit of data, we can use the $ (dollar sign) to scope any part of a filter to the root data: var tweets_filtered = db.select("/timeline/*[/user/handle == $/show_only_from_handle]/status"); And voila, we've filtered one part of the state based on the contents of another, and selected some data from within. Filters may be chained together to produce logical AND gates. Here we'll pull all users who have both a large and a small avatar available: var users_with_both_avatars = db.select("//user[/avatar/small][/avatar/large]"); #Working with results #Modifying data SpahQL objects provide a set of methods for modifying their data values. SpahQL always maintains _strict pointer equality_ to the original database data, so be aware that calling these methods will result in alterations being made directly to the object you originally passed to SpahQL.db(your_data). Most destructive methods apply only to the first item in a SpahQL result set, and have a partner method which applies to the entire set. For instance, here are the replace and replaceAll methods - just two of the many methods SpahQL offers for easy data editing: db.select("//user").replace("This string will replace the first user in the set"); db.select("//user").replaceAll("NO USERS FOR YOU"); #Listening for changes SpahQL objects are able to dispatch events when certain paths are changed, using an event-bubbling model similar to the HTML DOM. db.listen(function(db, path, subpaths) { console.log("Something in the DB was modified. Modified paths: "+subpaths.join(",")); }) The above code listens for changes to the database as a whole. You may scope listeners to certain paths using either of the following methods: db.listen("/user", function(user, path, subpaths) { console.log("User was modified: ", user.value()); }) db.select("/user").listen(function(user, path, subpaths) { console.log("User was modified: ", user.value()); }); The callback function always receives three arguments; result, a SpahQL object containing the data found at the path on which you registered the listener, path, the path on which you registered the listener (allowing you to assign a single listener function cabable of responding to multiple changes), and subpaths, an array of paths within the path that were detected as having been modified. db.listen("/user", function(user, path, subpaths) { console.log("Subpaths modified on user ("+path+"): ", subpaths.join(",")); }); db.select("/user").set({handle: "modified-handle", newobject: {foo: "bar"}}); // -> prints the following to console: // Subpaths modified on user (/user): /handle,/newobject,/newobject/foo #Properties Properties are like imaginary paths on objects in your database. They allow you to make more interesting assertions about your data. Each property uses the .propertyName syntax and may be used in any path query: Use .type When you need to know what type of data is at any given path. Returns the object type as 'object', 'array', 'string', 'number', 'boolean' or 'null': results = db.select("/timeline/.type"); results.value() //-> 'Array' The type property lets you query for all paths matching more precise criteria: // Find all arrays everywhere. var all_arrays = db.select("//[/.type == 'array']") Use .size when you need to know about the amount of data in an object. Returns the object's size if it is a String (number of characters), Array (number of items) or Object (number of keys): var timeline_is_empty = db.assert("/timeline/.size < 1"); //-> false, timeline contains items Use .explode when you need to break an object down into components. Returns the object broken into a set that may be compared to other sets. Strings are exploded into a set of characters. Arrays and objects do not support this property - use the wildcard * character instead. // Does the user's handle contain a, b and c? results = db.assert("/user/handle/.explode }>{ {'a','b','c'}") #Making assertions We've already seen how assertion queries can be used as filters in [selection queries][selection_queries]. Assertions can also be used on their own using SpahQL's assert method. Since the entity on either side of the comparison operator could contain one or more results (or no results at all), all comparisons in SpahQL are _set_ comparisons. Assertions are run through the assert method on the state: result = db.assert(myQuery) //-> true or false. Assertions don't have to use comparisons: db.assert("/user"); //-> true, since /user exists and has a truthy value db.assert("/flibbertygibbet"); //-> false, since /flibbertygibbet doesn't exist, or is false or null Much like selections, assertions can be scoped to a set of results you already have available: db.select("/user").assert("/handle"); //-> true, since /user/handle exists #Comparisons SpahQL's set arithmetic uses the following operators for comparing values. To learn how values are compared, see [Object equality][object_equality]. ##Set equality == Asserts that both the left-hand and right-hand sets have a 1:1 relationship between their values. The values do not have to be in the same order. ##Set inequality != Asserts that the sets are not identical under the rules of the == operator. ##Subset of }<{ Asserts that the left-hand set is a subset of the right-hand set. All values present in the left-hand set must have a matching counterpart in the right-hand set. ##Superset of }>{ Asserts that the left-hand set is a superset of the right-hand set. All values present in the right-hand set must have a matching counterpart in the left-hand set. ##Joint set }~{ Asserts that the left-hand set contains one or more values that are also present in the right-hand set. ##Disjoint set }!{ Asserts that the left-hand set contains no values that are also present in the right-hand set. ##Rough equality =~ Asserts that one or more values from the left-hand set are *roughly equal* to one or more values from the right-hand set. See [Object equality][object_equality]. ##Greater than (or equal to) >= and > Asserts that one or more values from the left-hand set is greater than (or equal to) one or more values from the right-hand set. ##Less than (or equal to) <= and < Asserts that one or more values from the left-hand set is less than (or equal to) one or more values from the right-hand set. #Literals SpahQL does support literals - strings, integers, floats, true, false and null may all be used directly in SpahQL queries. Strings may use single or double quotes as you prefer. Because all SpahQL comparisons compare sets to one another, all literals count as sets containing just one value. As such, the following basic comparisons work just as you'd expect: db.assert("/user/handle == 'johndoe'") //-> true db.assert("//user/handle == 'johndoe'") //-> false. The left-hand set contains more than one item. You may use set literals in SpahQL assertions. A set literal is wrapped in {} mustaches: db.assert("//user/handle }~{ {'johndoe', 'anotherguy'}") //-> true. The left set is a joint set with the right. Set literals may combine numbers, strings, booleans and even selection queries: // a set containing all the handles, plus one arbitrary one. {"arbitrary_handle", //user/handle} Sets may not be nested - in the above example, SpahQL flattens the set literal to contain all the results of querying for //user/handle and one other value, "arbitrary_handle". Ranges are also supported in set literals: {"a".."c"} // a set containing "a", "b" and "c" {"A".."Z"} // a set containing all uppercase letters {"Aa".."Ac"} // a set containing "Aa", "Ab", "Ac" {0..3} // a set containing 0, 1, 2 and 3. {"a"..9} // COMPILER ERROR - ranges must be composed of objects of the same type. {"a"../foo/bar} // COMPILE ERROR - ranges do not support path lookup. #Object equality There are two kinds of equality in SpahQL. *Strict* equality is applied with the == and other major operators, while *rough* equality is applied when using some of the more lenient operators such as =~. ##Strict equality The equality of objects is calculated based on their type. Firstly, for two objects to be equal under strict equality (==) they must have the same base type. ###Object equality The objects being compared must contain the same set of keys, and the value of each key must be the same in each object. If the value is an object or an array, it will be evaluated recursively. ###Array equality The arrays must each contain the same values in the same order. If any value is an array or object, it will be evaluated recursively. ###Number, String, Bool, null The objects must be of equal type and value. ##Rough equality Under rough equality (=~) the rules are altered: ###Rough String equality Strings are evaluated to determine if the left-hand value matches the right-hand value, evaluating the right-hand value as a regular expression e.g. "bar" =~ "^b" returns true but "bar" =~ "^a" returns false ###Rough Number equality Numbers are evaluated with integer accuracy only (using Math.floor, numeric.floor or an equivalent operation) ###Rough Array equality Arrays behave as if compared with the joint set operator. ###Rough Object equality Objects are roughly equal if both hashes contain one or more keys with the same corresponding values. Values are compared using strict equality. ###Rough Boolean and Null equality Booleans and Null objects are evaluated based on truthiness rather than exact equality. false =~ null is true but true =~ false is false. When using inequality operators <, =<, >, >=: * **Strings** are evaluated based on alphanumeric sorting. "a" <= "b" returns true but "z" >= "a" returns false. * **Numbers** are evaluated, as you'd expect, based on their native values. * **Arrays, Objects, Booleans, null** are not compatible with these operators and will automatically result in false being returned. #SpahQL Strategies _Strategies_ are a mechanism provided by SpahQL allowing you to define a queue of asynchronous actions to be run in order against a SpahQL object, provided that the value of the query result matches the criteria you specify. Pattern-wise, they're somewhere between a macro and a stored procedure. Strategies are managed using the Strategiser class. var state = SpahQL.db({a: {aa: "a.aa.val", bb: "a.bb.val"}, b: {bb: "b.bb.val", cc: "b.cc.val"}}); var strategiser = new SpahQL.Strategiser(); Strategies are objects which define a set of target paths, a condition which must be met for the strategy to run, and an action to take against the matched paths. // Add a strategy to the strategiser... strategiser.addStrategy( // which will take action on /aa and /b/cc, but only if the assertion "/b/bb" returns true {"paths": ["/aa", "/b/cc"], "if": "/b/bb"}, // with a named category "reduce", // when triggered, the strategy will be called function(results, root, attachments, strategy) { // make changes to the matched results results.deleteAll(); // signal that the strategiser can advance to the next strategy in the queue strategy.done(); } ); Strategies must specify the key _path_ or _paths_, a path or array of paths for the strategy to modify. Strategies may optionally use the key _if_ or _unless_, containing a SpahQL assertion whose expectation must be met for this strategy to be included. When we execute the strategies against a target SpahQL object, _path_, _paths_, _if_ and/or _unless_ will be evaluated relative to the target. Strategies also specify an _action_, a function containing the strategy's behaviour. It receives the arguments _results_, a SpahQL instance containing matches for the _path_, _root_, the original target SpahQL instance, _attachments_, an arbitrary object you may pass in when you execute the strategies, and _strategy_, an object containing flow control functions allowing you to signal that the strategy has completed. Specifying multiple paths using the _paths_ key is equivalent to registering multiple strategies each with the same expectation and action - the action function will be called once for each query specified in the _paths_ array and calling strategy.done() will advance the queue to the next path in this strategy, or to the next strategy. Execution is as follows: strategiser.run(target, category, attachments, callback); When applied to the above example: // Clone the State first to run the strategies without modifying the original // Run the strategies in the "reduce" category // Pass {foo: "bar"} as an attachment that will be available to all the strategies // Pass a callback function which will receive the modified SpahQL and the attachments strategiser.run(state.clone(), "reduce", {foo: "bar"}, function(clone, attachments) { console.log(clone.select("/aa").length()); //-> 0, as the above strategy deleted this value });