RegisterSign In
By Aris Setyawan (arissety) on Jun 9, 2010 2:26 PM.
Configuration to maximaze performance

Are MckoiDDB replication and partition configurable? What is the best configuration of a cluster with 4 nodes, a node have 4 core CPU?
By Tobias Downer (toby) on Jun 9, 2010 6:12 PM.
I'd recommend following the [[MckoiDDB Network Guide]] and tell me how you get on. If things aren't clear, I'm very interested in the feedback and will improve things. Right now MckoiDDB has the code for managing data but doesn't include anything like batch processing/analytics, so those 4 cores will go to waste. That's not to say you can't run your own applications on the nodes that interacts with MckoiDDB that can use up the processor potential.
By Aris Setyawan (arissety) on Jun 9, 2010 7:01 PM.
OK, I will try it.

Can I choose a partition column in a table? Can I have a small table that just replicated, not partitioned? That is because the goal of partitioning the database tables is to ensure that the most frequent transactions are single-sited.

So, currently, the efficient cluster configuration is all node have single thread processor? How about allowing partition in a node, so we can configure the amount of partition depends on the processor core we have?
By Tobias Downer (toby) on Jun 9, 2010 8:10 PM.
You shard your data by creating a new path instance, the individual path instance's commit operation happens under a lock. You can create multiple path instances per root server, so you can make full use of your processor cores if you partition your data over many path instances.

MckoiDDB will happily make use of multiple cores, and it is a multi-threaded application. The reason I mentioned you won't get much benefit from having multiple cores is because of the IO bound nature of these sorts of data storage systems. Once your data starts to become really large and your queries complex enough that a significant number of operations are occurring outside the bounds of a cache, threads will be waiting on disk seeks more so than anything else. That's where my comment came from.
By Aris Setyawan (arissety) on Jun 9, 2010 11:11 PM.
As you explained in another topic forum that, "...As far as persistent data is concerned, RAM is only used for caches. Certainly though, this is not hard wired into the design and I'm interested in exploring more memory oriented persistence especially in regards to the low seek time world of SSD technology."

How interested you in this topic?
If we change to the memory oriented persistence (RAM), can we explore the benefits of using partition in a multi-threaded node?

Currently, there are in memory DBMS (but some of them still using logging in update) that have very low latency BUT they don't scale.
By Tobias Downer (toby) on Jun 10, 2010 1:23 PM.
Do you mean a DB based on volatile DRAM storage? Removing the latency from disk seeks would allow for all sorts of opportunities. You would be able to scale individual partitions to support a large amount of commits with low latency.

I'm very interested in moving MckoiDDB in a direction that supports storage mechanisms that don't have the seek overhead of disks. Unfortunately this sort of hardware is a little bit exotic at the current time. But I'm very confident MckoiDDB will be able to support this technology well because of the flexible way consistency rules can be defined (ie. you make compromises in your data model to improve throughput for high latency hardware). With better technology you can make less compromises.
By Aris Setyawan (arissety) on Jun 10, 2010 3:50 PM.
Yes, like used by some in-memory database. RAM is cheap and have bigger size. As long as Mckoi snapshot replicated, it will make sure durability.
Please sign in or register to post in this topic.
The text on this page is licensed under the Creative Commons Attribution 3.0 License. Java is a registered trademark of Oracle and/or its affiliates.
Mckoi is Copyright © 2000 - 2020 Diehl and Associates, Inc. All rights reserved.