RegisterSign In

Application Development using the Simple Database API

This document is an introduction on how to develop an application on a MckoiDDB installation using the Simple Database API. If you have not yet installed MckoiDDB you should follow the Quick Start Guide before continuing.

What is the Simple Database API?

Before a developer can use MckoiDDB to store and query data, the system must be told about the data models that are going to be used by the client applications. MckoiDDB can support many different database types (such as graphs, relational, file systems, etc) through the flexibility of its customizable data model design. In MckoiDDB, a data model is a definition of some data consistency rules specific to the model, and an application API that bridges between the data objects used by the client application and the structural representation of the data as it is represented internally to the system.

The Simple Database API is one such data model. This data model can create and manipulate elementary structures such as tables with indexes, maps (associative arrays), and arbitarily sized string and data objects (files). The Simple Database API also provides a set of consistency rules that can gracefully handle concurrent changes to tables.

In this guide we will demonstrate how to set up a Simple Database instance on a MckoiDDB installation and then show how a Java applications can store and query information in the database using the application API.

How to set up a Simple Database

Before we start writing any Java code I will show how to create a Simple Database instance on a root server in the network. To create the new instance we will use the 'console' script to access the administration console. The console can also be accessed by running the following command;

  java -cp ..\..\lib\mckoiddb.jar com.mckoi.runtime.AdminConsole -netpassword [the network password]

From the administration console prompt enter the following command;

add path com.mckoi.sdb.SimpleDatabase testdb to 127.0.0.1:3500

The above example assumes you have set up a single machine installation on your local machine as described in the Quick Start Guide. If you are running MckoiDDB over a network, 127.0.0.1:3500 would instead be the IP address of the machine you have set as the root server. After running this command you will have a Simple Database instance path named 'testdb'.

If you'd like to check the current paths you have set up on your network you can use the 'show paths' command from the console prompt which should now include the following report;

MckoiDDB> show paths
Root server: 127.0.0.1:3500

+Name:            testdb
 Consensus Class: com.mckoi.sdb.SimpleDatabase
 Status:          114 bytes (in 0 tables 0 files)

Connecting to the Simple Database

To use a Simple Database, the client must create a MckoiDDBClient using MckoiDDBClientUtils, and then connect to the database by creating an SDBSession. The MckoiDDBClient object is used to encapsulate general information about the network. The SDBSession object is the client interface for accessing a Simple Database.

The following is an example of connecting to our test Simple Database instance;

import com.mckoi.data.*;
import com.mckoi.network.*;
import com.mckoi.sdb.*;
import java.io.IOException;
import java.util.Properties;

class MyConnectApp {
  public static void main(String[] args) {
    // This is the server we need to connect to get onto the network.
    Properties prop = new Properties();
    prop.setProperty("manager_address", "127.0.0.1:3500");
    prop.setProperty("network_password", [your network password]);

    // Connect to the network,
    MckoiDDBClient client;
    try {
      client = MckoiDDBClientUtils.connectTCP(prop);
    }
    catch (IOException e) {
      e.printStackTrace();
      return;
    }

    // Create an SDBSession object on our named path.
    SDBSession session = new SDBSession(client, "testdb");
    
    // ... Now we can interact with the database
    
  }
}

If you are connecting to a MckoiDDB installation over a network, the manager_address IP address in the above example will need to be set to the location of the Manager server on your network. If this code fails to connect then you should check that the network.conf file of all the machine nodes in the network includes the IP address of the client machine in the connect_whitelist property.

Transactions and Concurrency Control

Before we move on to some more example code, a quick overview of the transaction system in MckoiDDB should be covered.

In MckoiDDB, a transaction is a fully isolated snapshot 'view' of the data. To make a permanent change to the data, first a transaction object (SDBTransaction) is created, then the desired modification is made to the view of the data provided by the transaction object, and finally the transaction is committed which publishes the update allowing future transactions to see the changes. MckoiDDB's transactions are fully isolated, which means it is impossible for one transaction object to see changes made in other concurrent transactions. Also, MckoiDDB provides optimistic concurrency control which ensures that read and writes will never cause lock contentions.

When a transaction tries to commit a change that conflicts with an update made by another transaction (for example, when two concurrent clients try to delete the same row in a table), the commit function of the second transaction will raise a com.mckoi.network.CommitFaultException. The first transaction to commit will succeed. When a commit fault happens, any changes made to the transaction's snapshot of the data are rolled back.

The Simple Database API offers two types of primary data structures; Files and Tables, each which support their own concurrency control models.

For SDBFile objects, concurrency conflicts will happen at commit when;

For SDBTable objects, a commit fault happens when;

Note that Simple Database tables do not have a constraint rule that will cause an update to fail when rows are inserted into a table. Row inserts alone can never cause a commit fault.

Files in a Simple Database

As mentioned in the previous section, the Simple Database API provides a basic file-system data structure. An SDBFile is a binary object that can grow and shrink to any size and data can be written to and read from any position. The Mckoi Simple Database file object is very similar to Java's java.io.RandomAccessFile but with one important difference - data stored in an SDBFile can be efficiently shifted forwards and backwards by any amount. The 'shift' function makes it very easy to support dynamic structured data objects such as sorted list/sets and maps over the top of the the File object. Examples of this feature are shown below.

The following example creates a com.mckoi.sdb.SDBFile and inserts some data into the file;

...
    // Create a transaction
    SDBTransaction transaction = session.createTransaction();
    // Make sure the file 'BinaryFile' is created and fetch it
    boolean created = transaction.createFile("BinaryFile");
    SDBFile binary_file = transaction.getFile("BinaryFile");
    // Move to start of the file and insert 3 integers, {1, 2, 3}
    binary_file.position(0);
    binary_file.putInt(1);
    binary_file.putInt(2);
    binary_file.putInt(3);

    // Commit the change,
    try {
      transaction.commit();
    }
    catch (CommitFaultException e) {
      // This will only happen if another transaction has
      // concurrently created, deleted or modified the file.
      System.out.println("Oops, we failed to commit.");
    }
...

The next example wraps an SDBFile with a com.mckoi.data.StringData object. StringData allows free manipulation of a persistent string object stored in the database. You might use this object to store a variable sized text document, for example.

...
    SDBTransaction transaction = session.createTransaction();
    boolean created = transaction.createFile("StringFile");
    SDBFile string_file = transaction.getFile("StringFile");

    // Wrap the file with a StringData object and add "Hello World"
    // to the end and then insert "Goodbye World " at the start.
    StringData string_ob = new StringData(string_file);
    string_ob.append("Hello World");
    string_ob.insert(0, "Goodbye World ");

    // Print the complete string,
    System.out.println(string_ob.toString());

    // The output will be; "Goodbye World Hello World"

    try {
      transaction.commit();
    }
    catch (CommitFaultException e) {
      System.out.println("Oops, we failed to commit.");
    }
...

SDBFile can also support sorted lists and maps. The following example creates a persistent String->String associative array by wrapping com.mckoi.data.PropertySet over the top of an SDBFile object.

...
    boolean created = transaction.createFile("MyPropertySet");
    SDBFile property_file = transaction.getFile("MyPropertySet");

    // Wrap the file with the PropertySet class and make some
    // associations.
    PropertySet properties = new PropertySet(property_file);
    properties.setProperty("Hello", "world");
    properties.setProperty("Foo", "bar");
    properties.setProperty("Austin", "Powers");

    // Output a property,
    System.out.println("Austin = " + properties.getProperty("Austin"));

    // The output will be; "Austin = Powers"

...

As you can see, these file objects are very expressive and can be used to implement all sorts of other more complex persistent data structures. Internally, the Simple Database file objects are actually just exposing a similar structure that is used as the primitive building blocks of all other persistent data structures in MckoiDDB. Do not be afraid to create as many files as you need as small or large as necessary. They are designed to be efficient and light weight.

One drawback to using the Simple Database files is the simple concurrency control model they have. As mentioned in the Transactions and Concurrency Control section above, if the same file is updated by concurrent transactions then a commit fault will occur on the second transaction that tries to commit changes to the file. This can cause updates to fail and need to be retried often for files that change frequently. For more fine grained concurrency control using the Simple Database API, tables are a more appropriate structure for handling frequent changes in a distributed environment.

Table Structures in a Simple Database

In the Simple Database API, tables are represented by the com.mckoi.sdb.SDBTable object. The SDBTable object is able to store string values in the cell elements of the table, supports indexes on columns (SDBIndex), and provides a cursor interface for traversing rows and indexes (RowCursor). Simple Database tables have a more complex commit model than Simple Database files, permitting concurrent inserts, updates and deletes to the same table object provided the changes do not conflict.

In the following example we create a new SDBTable called 'BooksWithRobots' and set up the columns and indexes of the table and populates it with some data.

...
    // Create an SDBSession object on our named path.
    SDBSession session = new SDBSession(client, "testdb");
    // Create a new transaction,
    SDBTransaction transaction = session.createTransaction();
    // Create the table 'BooksWithRobots'
    boolean created = transaction.createTable("BooksWithRobots");
    // If we created a new table then we populate it with data
    if (created) {
      SDBTable robot_books = transaction.getTable("BooksWithRobots");
      // Add the columns and indexes of the table,
      robot_books.addColumn("name");
      robot_books.addColumn("author");
      robot_books.addColumn("year");
      robot_books.addIndex("author");
      robot_books.addIndex("year");

      // Insert 4 rows,
      robot_books.insert();
      robot_books.setValue("name", "The Hitchhiker's Guide to the Galaxy");
      robot_books.setValue("author", "Douglas Adams");
      robot_books.setValue("year", "1979");
      robot_books.complete();
      robot_books.insert();
      robot_books.setValue("name", "I, Robot");
      robot_books.setValue("author", "Isaac Asimov");
      robot_books.setValue("year", "1950");
      robot_books.complete();
      robot_books.insert();
      robot_books.setValue("name", "2001 : A Space Odyssey");
      robot_books.setValue("author", "Authur C. Clarke");
      robot_books.setValue("year", "1968");
      robot_books.complete();
      robot_books.insert();
      robot_books.setValue("name", "The Rest of the Robots");
      robot_books.setValue("author", "Isaac Asimov");
      robot_books.setValue("year", "1964");
      robot_books.complete();
    }

    // Commit the changes,
    try {
      transaction.commit();
    }
    catch (CommitFaultException e) {
      // In this example, this could only happen if a
      // concurrent transaction also created a table called
      // 'BooksWithRobots'
      System.out.println("Oops, we failed to commit.");
    }
...

The following example queries the author index and prints a list of all the books in the table from the author "Isaac Asimov".

...
    // Create a new transaction,
    SDBTransaction transaction = session.createTransaction();
    // Fetch the 'BooksWithRobots' table,
    SDBTable robot_books = transaction.getTable("BooksWithRobots");
    // Fetch the author index,
    SDBIndex index = robot_books.getIndex("author");
    // Find the sub-index of all entries that are 'Isaac Asimov' from the
    // author index.
    index = index.sub("Isaac Asimov", true, "Isaac Asimov", true);
    // Print the name and year of the books,
    for (SDBRow row : index) {
      System.out.print(row.getValue("name"));
      System.out.print(" (");
      System.out.print(row.getValue("year"));
      System.out.println(")");
    }
...

Where to go for more information

For more details about the Simple Database API, check out the Simple Database Java Docs. You can also follow the documentation link at the top of any page for more technical information about MckoiDDB.

The text on this page is licensed under the Creative Commons Attribution 3.0 License. Java is a registered trademark of Oracle and/or its affiliates.
Mckoi is Copyright © 2000 - 2017 Diehl and Associates, Inc. All rights reserved.