Major Objects

Overview

• Major Objects
  • Use Major Objects for fast in-memory handling of large amount of data that is thread-safe but must be persisted
  • We must support complex objects with simple keys, crud, and fast lookup by multiple keys.
  • Our most useful containers are vector, set (key in object) and map (<key,value> pair). Set can give us almost every positive feature, when used to store the PersistentIDObject class.
  • Use an unordered_set of const pointers to objects derived from PersistentIDObject
  • The default container should index by db_id primary key
  • Always use the db_id for foreign keys
  • Other containers can be created with alternate keys using object members; just define new hash functions.
• PersistentIDObject
  • Add a dirty flag to all objects, set to true on any change that must be persisted
  • Use an internal in-memory counter to generate the next db_id for a newly created object
  • This means that when creating new objects, there is NO NEED to access db, VERY IMPORTANT!
  • Use delayed-write tactics to write all dirty objects on idle time
• Memory Model
  • Use a Datastore manager (aka "MemoryModel") to hold sets
  • It can look up objects by any key, and strip away const to return a mutable object. NOTE that the user must not damage the key values!
  • Derive a class from the memory model for persistence; it can use any persistence method (local, remote, sql, nosql, etc.).
  • Make sure that the base MemoryModel class is concrete not abstract, thread-safe and self-contained; this makes parallel calculations trivial, helps scalability, etc.

Delayed delete pattern

           1) all deletion work should be done in one place:
               bool SqliteLocalModel::deleteBrokerAccount(BrokerAccount& ba)
           2) deletion check should happen in delayed write check:
               if (pau->bDirtyOrDeleted())
                   bNeeded = true;
               ----
               if (pa->bDeleted())
                   deleteBrokerAccount(*pa);
               else...
           3) to dynamically delete an object: 
               ba.setDeleted();
           4) include deleted status in active check:
               bool bActive() const { return b_active_ && !bDeleted(); }