Common Questions
What is BigObject?
BigObject is a Smart Data Analytics platform for interactive query and real-time computations. It is built to store and compute data for high performance purpose. It is a highly optimized engine and supports tree logic for complex data pattern analytics.
Does BigObject support OLTP (On-Line Transaction Processing)?
No. BigObject is not designed for OLTP. It is never aimed to replace relational databases in transactional applications.
Does BigObject support durability?
Yes. Recoverability can be achieved by snapshot and WAL.
What type of data model does BigObject support, relational, hierarchical or NoSQL?
BigObject,in general, supports multifaceted data models, including relational model (i.e., tables), hierarchical model (i.e., trees) and key-value model (i.e., semi-structured data), which are mutually transformable. The key-value model is often used for the raw datasets where schemas cannot be determined in advance or might be altered dynamically. The relational model defines the relational algebra and provides record-level query capability, while the hierarchical model is designed for efficient computation in complex group-level queries or for fast access to intermediate and resulted datasets.
Does BigObject support column-based tables?
Yes
Is BigObject an in-memory database platform?
Yes or no. It differs from many commonly known in-memory databases, which take advantage of the speed of physical memory (RAM) and are thus limited by its size (RAM size or SWAP space size to be precise). By assuming memory size is always bigger than the data size, BigObject, on the other end, takes advantage of entire memory address space, where data are mapped (i.e., mmap) at the moment being first referenced, but loaded on demand while the data values are actually needed in computing. With today’s Linux running on 64-bit CPU to supports 48-bit address space, the size of (virtual) memory space for one node is as big as 256TB, while the RAM size remains gigabytes as 4GB, 8GB, 16GB, 32GB, 64GB, 128GB, etc.
How does BigObject’s memory approach work?
As answered in the previous question, BigObject manages a large memory space of data backed up by memory-mapped files. This memory mapping approach simulates BigObject’s abstract model, called In-Data Computing (formerly known as In-Place Computing) model, where data resides in an infinite and persistent memory space, ready to compute, assuming that 264 is considerably infinite for most tractable problems. However, this memory mapping approach does not guarantee efficient computation due to potential page faults. In order to reduce page faults, it is important to organize data in a way such that relevant or associative data elements are kept adjacent in memory. This is the principle of data locality. BigObject implements several flexible locality techniques to organize tables and trees according to applications.
What is In-Data Computing model?
In-Data Computing is an abstract model where data are stored and computed in an infinite and persistent memory space. You can download our paper for more information.
How does BigObject's concurrency control work?
BigObject adopts a simple but efficient approach for concurrency control - read-write lock at the object level (i.e., table and tree) for performance reason.
What is the difference between FIND statements and SELECT?
The idea behind BigObject FIND statements is to find interested entities or groups, which are ‘group-by’ by default. On the other end, ‘SELECT’ statement focuses on record level.
Data Collection Questions
How does BigObject import data?
There are several ways.
1. REST API (default 9090): write a client program that issues SQL insert statements via BigObject REST API.
2. Streaming API (default 9091): upload a csv or an Avro file via TCP connection.
3. MySQL protocol (default 3306): Use MySQL client via MySQL protocol.
4. Uploader: upload a csv file to BigObject from BigObject shell.
How does BigObject extract data from an existing database?
There are several ways.
1. FluentD/Embulk: Run FluentD/Embulk to collect data from different database, one time or incrementally.
2. MySQL: Use MySQL as the staging (or primary) database to sync data into BigObject. MySQL offers the replication feature for many relational databases.
Does BigObject store my data?
Yes. Data is persistent in BigObject space. BigObject stores data in a special ‘Big Table’ format (.bt). If you need to move or backup your data, just copy and paste the .bt and the namespace file and you can get it in another BigObject server.
Is BigObject a storage engine for databases?
No. BigObject is a standalone platform with built-in database aimed for real-time analytics. BigObject includes its own storage engine and file spaces which are highly optimized in memory and executed.
Answers not found?
Contact us!
