Mark Starkman

Musings of a long time computer geek

Lately, I’ve had a desire to use CoffeeScript to create a one page application, utilizing Backbone.js.  So I decided to take this opportunity to learn Node.js and use the Express framework for the application server.  While doing this I realized that I was going to have a lot of CoffeeScript code that would be used on the client side.  For this, I wanted a Ruby on Rails 3.1.x style asset pipeline.  I found a library called connect-assets, which was inspired by Rails 3.1.x asset pipeline and the sprockets gem.  Also, I needed to pick a client side template engine.  There are many options to choose from, and I decided to use Eco templates.

I chose Eco templates for two reasons.  First, they utilize CoffeeScript for their processing.  Secondly, they can be complied to JavaScript on the server.  To me, this is a benefit over many of the template engines that exist that require some sort of markup to be sent to the client and then compiled to be utilized by Backbone.js.  The other benefit of using the Eco templates is that I can also use them as the template engine to render the HTML views trough Express.

The connect-assets library really seemed to do what I was looking for.  It combines all of you JavaScript and CoffeeScript per the directives that you specify.  It defers to Stylus to handle all of the CSS processing.  Under the hood it uses a secondary library, snockets, to process all of the JavaScript and CoffeeScript.  However, it does not handle Eco templates natively.  Also, it will not allow you to provide multiple file extensions to handle processing a single file through multiple processors, for example template.jst.eco.  Here is the code that I came up with to handle processing Eco templates and provide them in a global JST object.  The JST object is created in a similar way that Jammit, by DocumentCloud, does it.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

assets = require 'connect-assets'
eco = require 'eco'


assets.jsCompilers.eco =
  match: /\.eco$/
  compileSync: (sourcePath, source) ->
    fileName = path.basename sourcePath, '.eco'
    directoryName = (path.dirname sourcePath).replace "#{__dirname}/assets/templates", ""
    jstPath = if directoryName then "#{directoryName}/#{fileName}" else fileName
    """
    (function() {
      this.JST || (this.JST = {});
      this.JST['#{jstPath}'] = #{eco.precompile source}
    }).call(this);
    """

This code does work for my needs and gets me the global JST object with all of the Eco templates compiled to JavaScript and stored in the JST object.  You will just need to put the following code into your view to have connect-assets render the correct JavaScript.

1

<%- js '/templates/index'  %>

Ultimately, this code and the processing of multiple file extensions should be handled in snockets.  This may be something that I will contribute to the project in the future.

mongoDB affords you the ability to store documents within a single collection that can each have their own schema. This is a drastic change if you have a background in relational databases. In a relational database, you have a static schema per table and you cannot deviate from without changing the structure of the table.

In the example below, I have defined a Person class and a PersonWithBirthDate class which derives from the Person class. These can both be stored in the same collection in a mongoDB database. In this case, assume the documents are stored in the Persons collection.

1
2
3
4
5
6
7
8
9
10

public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

public class PersonWithBirthDate : Person
{
    public DateTime DateOfBirth { get; set; }
}

You can easily retrieve and of the documents and have the mongoDB C# Driver deserialze the document into the PersonWithBirthDate class. However, you will run into issues if you query the Persons collection and try to have the driver deserialize the data into the Person class. You will receive an error that there are elements that cannot be deserialized.

This easily fixable by adding the [BsonIgnoreExtraElements] attribute to the Person class. You can see the modified class below. This will instruct the driver to ignore any elements that it cannot deserialize into a corresponding property. With the attribute any document in the Persons collection can be deserailized to the Person class without error.

1
2
3
4
5
6
7
8
9
10
11

[BsonIgnoreExtraElements]
public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
}

public class PersonWithBirthDate : Person
{
    public DateTime DateOfBirth { get; set; }
}

There is a gotcha that I recently found while trying to implement a scenario similar to the one above. In the source code for the mongo C# driver, the attribute is defined in a way that it can be inherited to the child classes when applied to the parent class (BsonIgnoreExtraElementsAttribute.cs). However, when the attribute is read, it ignores the inheritance of the attribute (BsonClassMap.cs) and does not get applied to the child classes. I agree with this implementation, but it’s a little confusing if you review the source code for the definition of the [BsonIgnoreExtraElements] attribute. Even with this inconsistency, all you need to do is to apply the [BsonIgnoreExtraElements] to each class that may read a document from a collection where there are extra elements.

Introduction

Implementing a many-to-many relationship in a relational database is not as straight forward as a one-to-many relationship because there is no single command to accomplish it. The same holds true for implementing them in mongoDB. As a matter of fact you cannot implement any type of relationship in mongoDB via a command. However, having the ability to store arrays in a document does allow you to store the data in a way that is fast to retrieve and easy to maintain and provides you the information to relate two documents in your code.

Modeling in relational database

In the past, I have modeled many-to-many relationships in relational databases using a junction table. A junction table is just the table that stores the keys from the two parent tables to form the relationship. See the example below, where there is a many-to-may relationship between the person table and the group table. The person_group table is the junction table.

Modeling in mongoDB

Using the schema-less nature of mongoDB and arrays, you can accomplish the same data model and create short query times with the appropriate indexes. Basically, you can store an array of the ObjectId’s from the group collection in person collection to identify what groups a person belongs to. Likewise, you can store an array of the ObjectId’s from the person collection in the group document to identify what persons belong to a group. You can also store DBRef’s in the array, but that would only be necessary if your collection names will change over time.

This is how some person documents would look in mongoDB.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

db.person.insert({
  "_id": ObjectId("4e54ed9f48dc5922c0094a43"),
  "firstName": "Joe",
  "lastName": "Mongo",
  "groups": [
    ObjectId("4e54ed9f48dc5922c0094a42"),
    ObjectId("4e54ed9f48dc5922c0094a41")
  ]
});

db.person.insert({
  "_id": ObjectId("4e54ed9f48dc5922c0094a40"),
  "firstName": "Sally",
  "lastName": "Mongo",
  "groups": [
    ObjectId("4e54ed9f48dc5922c0094a42")
  ]
});

This is how some group documents would look in mongoDB.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

db.groups.insert({
  "_id": ObjectId("4e54ed9f48dc5922c0094a42"),
  "groupName": "mongoDB User",
  "persons": [
    ObjectId("4e54ed9f48dc5922c0094a43"),
    ObjectId("4e54ed9f48dc5922c0094a40")
  ]
});

db.groups.insert({
  "_id": ObjectId("4e54ed9f48dc5922c0094a41"),
  "groupName": "mongoDB Administrator",
  "persons": [
    ObjectId("4e54ed9f48dc5922c0094a43")
  ]
});

The documents above show that “Joe Mongo” belongs to the “mongoDB User” and “mongoDB Administrator” groups. Similarly, “Sally Mongo” only belongs to the “mongoDB User” group. Effectively, these two arrays make up the data that is stored in the person_group table from the relational database example.

If you choose, you can just create the appropriate array on either the person documents or the group documents, but that would make your queries somewhat more complicated.

Getting the data

The following queries will show you how you can query the data without having to use joins as you would in a relational database.

1
2
3
4
5
6
7
8
9
10
11

// Get all persons in the "mongoDB User" group
db.person.find({"groups": ObjectId("4e54ed9f48dc5922c0094a42")});

// Get all persons in the "mongoDB Administrator" group
db.person.find({"groups": ObjectId("4e54ed9f48dc5922c0094a41")});

// Get all groups for "Joe Mongo"
db.groups.find({"persons": ObjectId("4e54ed9f48dc5922c0094a43")});

// Get all groups for "Sally Mongo"
db.groups.find({"persons": ObjectId("4e54ed9f48dc5922c0094a40")});

In order to improve the performance of the queries above, you should create indexes on the person.groups field and the groups.persons field. This can be accomplished by using the following commands.

1
2
3

db.person.ensureIndex({"groups": 1});

db.groups.ensureIndex({"persons": 1});

Summary

In general it’s pretty straight forward to be able to implement a many-to-many relationship in mongoDB. I have shown one method of doing this that closely resembles how it’s done in a relational database. One thing to keep in mind when it comes to data modeling, there is no silver bullet and you should always create the most appropriate data model that meets the needs of how you data will be queried.