Evaluating CouchDB

Building a game catalog with CouchDB

Revisiting a recurring example in this blog, this tutorial shows how to build a game catalog using CouchDB
. While cataloging board games could easily be made to fit into a SQL relational structure, exploring NoSQL gives flexibility into how we represent various entities that need to be referenced. The goal of this tutorial is to provide a scenario where one may consider NoSQL, identify key required functionality, and evaluate the different approach taken by CouchDB.

Why CouchDB

CouchDB is a document store, meaning we treat all objects similarly in our database as documents. CouchDB was chosen because it is mature, open source, easy to setup and has a relatively shallow learning curve. It also provides a REST interface, and a built in admin tool called Fauxton. Data views are built using MapReduce statements, and provides CouchDB with the indexing functionality it needs to provide speedy lookups and queries. CouchDB can even handle authentication and authorization, which means you can potentially use this as a free-standing backend REST service. These features make CouchDB an attractive solution to those new to the NoSQL world.

Data Set

For this example, we will create a data set that is comprised of various types of components that we might want to catalog. There may be things like game systems, which are a collection of various other components to form a playable game (like Dungeons & Dragons). We may have kits, that contain many pieces necessary to play the game, like dice. We also have other content like books, and potentially things like, counters, figurines, card decks, homebrew games, and so on. Each of these different categories of items have different attributes, and may have many to many relationships with any other component.

For example, we can have the following object with a many to many relationship:

{
  "name": "Dungeons & Dragons",
  "type": "game-system",
  "vendor": "Wizards of the Coast",
  "contents": [
    {"type":"dice", "name":"d20", "color":"black", "sides":20},
    "type":"book", "name":"Dungeon Master's Guide", "vendor":"Wizards of the Coast"},
    {"type":"kit", "name":"Dungeons & Dragons Starter Set", "vendor":"Wizards of the Coast", "contents":[
      {"type":"dice", "name":"d20", "color":"black", "sides":20}
      {"type":"dice", "name":"d4", "color":"red", "sides":4}
    ]}
    {
  ]
}

You’ll notice that the top level object (Dungeons & Dragons) is considered a game-system, which we can think of as a collection of various game pieces we may use to play. You’ll see contents like dice and the Dungeon Master’s guide, which are single items. We also have a kit, which can belong to the game system, but also have it’s own components, like dice, which also have a relationship directly with the game-system.

The SQL Relational Way

We certainly can represent this in a relational SQL database. We would have a table of game-systems, kits, and dice, books, and any other future component. In order to express these relationships, we need to build linking tables in order for us to represent our many to many relationships. We can make something like a game-systems-dice table. Our game-systems table will have a one to many relationship to this table, which in turn, will have a many-to-one relationship with other components. We will need to do this for each relationship we need to define between an entity that has contents. In this example, this would result in 11 tables, which is somewhat manageable, but consider the fact that as we add more entities, we have to now build out additional linking tables for each potential relationship. Suddenly, this problem begins to grow exponentially!

Is NoSQL the answer?

This hopefully serves as an example of the pros and cons of a NoSQL approach. Most non-graph NoSQL databases aren’t designed to handle relational data. In fact, that’s one of the distinguishing features of most NoSQL databases. But, they do give us the benefit of not having to define and maintain these relationships up front, allowing us to define these relationships as we need them.

Goals

  • Create an efficient view
  • Support the equivalent of a SQL many-to-many relationship join
  • Present views of a composite object

Prerequisites

The following tutorial assumes you have installed Docker
and can make curl requests.

Setup

  1. Run docker run -d -p 5984:5984 --name couchdb klaemo/couchdb
    to start an instance of CouchDB 2.0 running locally on port 5984
  2. Open your browser to http://localhost:5984/_utils
    to access the Fauxton admin panel.
  3. Navigate to ‘Setup’ > ‘Configure Single Node’ and specify a username and password. Leave the IP and port.

Adding Data

The following can be completed via curl requests against the REST API, or in the Fauxton admin dashboard.

  1. Create A database

    1. ‘Databases’>’Create Database’> name it “catalog”
  2. Insert some documents.

    1. ‘All Documents’>’+’>’New Doc’
  3. Here is some starting data. Typically, you’d let CouchDB create the _id
    , but this makes it easier for us to quickly create some relationships by copy and pasting.

Dice:

{
  "_id": "735174c9bbfec0115fa1e65e630035ed",
  "type": "dice",
  "name": "d20 Black",
  "sides": "20",
  "appearance": "black"
 }
{
  "_id": "735174c9bbfec0115fa1e65e63011d30",
  "_rev": "1-6a4c2db6ebe043ab37bd4ff06b55de97",
  "type": "dice",
  "name": "d4 Red",
  "sides": "4",
  "appearance": "red"
 }

Kit:

{
  "_id": "735174c9bbfec0115fa1e65e6301486e",
  "type": "kit",
  "name": "Dungeons & Dragons Starter Set",
  "vendor": "Wizards of the Coast",
  "contents": [
    {
      "qty": 2,
      "id": "735174c9bbfec0115fa1e65e630035ed"
    },
    {
      "qty": 4,
      "id": "735174c9bbfec0115fa1e65e63011d30"
    }
  ]
}

Game Systems:

{
  "_id": "735174c9bbfec0115fa1e65e63016382",
  "name": "Dungeons & Dragons",
  "type": "game-system",
  "vendor": "Wizards of the Coast",
  "contents": [
    {
      "qty": 3,
      "id": "735174c9bbfec0115fa1e65e630035ed"
    },
    {
      "qty": 5,
      "id": "735174c9bbfec0115fa1e65e63011d30"
    },
    {
      "qty": 1,
      "id": "735174c9bbfec0115fa1e65e6301486e"
    }
  ]
}
{
  "_id": "735174c9bbfec0115fa1e65e63019d6e",
  "name": "Pathfinder",
  "type": "game-system",
  "vendor": "Pazio",
  "contents": [
    {
      "qty": 10,
      "id": "735174c9bbfec0115fa1e65e630035ed"
    },
    {
      "qty": 4,
      "id": "735174c9bbfec0115fa1e65e63011d30"
    }
  ]
}

Retrieving Data

  1. Query our documents

    1. All documents are automatically displayed in the admin panel at http://localhost:5984/_utils/#/database/catalog/_all_docs
      and available via REST API with curl http://localhost:5984/catalog/_all_docs
    2. By default, CouchDB only returns the IDs and metadata. In the Fauxton admin panel, select the “Documents” checkbox up top to include the related document. Or append ?include_docs=true
      to your curl request.

This alone doesn’t provide much functionality. CouchDB doesn’t know how to best present or index this information.

Creating A Simple View

A view, is similar to a SQL table, in the fact that it provides CouchDB a way to group together data. This view will search for all documents that are a game-system
and emit them from a map function.

  1. Create a view by selecting “Design Documents”>”+”>”New View”
  2. Create a new design document to aggregate our views. In the input box labeled _design/
    , name your document “contents”.
  3. Name the index ‘game-systems’
  4. Create the following map function
function (doc) {
  if(doc.type === 'game-system'){
    emit(doc._id, null);
  }
}

This function iterates over all documents, and if they’re of type ‘game-system’ then it emits it’s key. This view has automatically created an index on the key (the game-system id). Query this view via curl http://localhost:5984/catalog/_design/contents/_view/game-systems?include_docs=true.
Now all game-systems can be queried as one would view a table in SQL. But, since CouchDB doesn’t have a concept of relationships, we only see the ids referencing our contents, not a joined view where we could see their details.

Build a More Complex View

Next, build a view that displays the contents of each document. This will be used to return one side of our many to many relationship, and allow us to build a composite view of our object.

  1. Create a new view, this time indexing all of our components and their ‘parent’. The index will be an array of the parent ID, and an index, and the value will be the component object (instead of an ID)
  2. Name the view ‘owners’ and use the following map function:
function (doc) {
  if(doc.name && doc.contents) {
    for (var i in doc.contents) {
      var content = doc.contents[i];
      emit([doc._id, i], {"_id": content.id, "content": content});
    }
  }
}

This is our “Join” statement. It returns a row for each component in each object that has ‘contents’. The map function first checks if there are contents. It then loops through each element and extracts the content. It emits key value pairs as follows: The key is an array of the owner’s ID, and the index (to ensure uniqueness), the value is an object containing an _id
(which will become apparent soon), and the entire content object.

The following query fetches all the contents for the D&D game-system: curl -X GET localhost:5984/catalog/_design/contents/_view/owners?include_docs=true -G --data-urlencode start_key='["735174c9bbfec0115fa1e65e63001edc",""]' --data-urlencode end_key='["735174c9bbfec0115fa1e65e63001edc",{}]'

The start_key and end_key allow you to filter your result. In this example, we’re interested in all primary keys associated with the Id of the D&D game-system.

If you query without include_docs=true
, you’ll only see the id of the content, as value.content.id
. When you include_docs, CouchDB uses the _id
field in the value to determine which document to attach. This provides us with the remaining information needed to create our composite object.

Summary

Our Goals

The three goals to accomplish were:

  • Create an efficient view
  • Support the equivalent of a SQL many-to-many relationship join
  • Present views of a composite object

We built a view using a CouchDB design document. This is defined by a map function that emits key-value pairs for CouchDB to index. Future updates to the index are updated on write, leading to quick look ups.

Using another view, we create a map function that identifies all documents that have contents
that references other documents. We create an index on the parent document’s IDs, and store the value of the contents. CouchDB provides the document related to the contents entry. We can pass query parameters to retrieve all contents for a particular document.

Unfortunately, this still requires two REST requests to fetch our data. Potentially List Functions
could be used to combine multiple objects. But, could prove to be difficult to handle things like paging and sorting.

So, CouchDB is unable to completely satisfy the last goal. Potentially, a service could be built that would handle the 2 REST requests and create the composite object.

稿源:Recursive Chaos (源链) | 关于 | 阅读提示

本站遵循[CC BY-NC-SA 4.0]。如您有版权、意见投诉等问题,请通过eMail联系我们处理。
酷辣虫 » 后端存储 » Evaluating CouchDB

喜欢 (0)or分享给?

专业 x 专注 x 聚合 x 分享 CC BY-NC-SA 4.0

使用声明 | 英豪名录