In-Memory Database
In-Memory Database Definition
An in-memory database (IMDB) stores computer data in a computer’s main memory instead of a disk drive to produce quicker response times. Accessing data stored in memory eliminates the time needed to query data from a disk. In-memory databases are used by applications that depend on rapid response times and real-time data management. Industries that benefit from in-memory databases include telecommunications, banking, travel and gaming. An in-memory database is also referred to as a main memory database (MMDB), real-time database (RTDB) or in-memory database system (IMDS).
FAQs
What Is an In Memory Database?
An in-memory database keeps all its data in the random access memory (RAM) of a computer. Only the main memory is accessed when querying data. This allows for faster access of that data than a disk-based system.
The downside is the volatility of RAM. The data is lost when an in-memory database crashes. The development of non-volatile random access memory (NVRAM) can help in-memory databases maintain data after a power loss or crash. Flash is one example, but a major drawback is the limit to how many times flash memory can be erased and rewritten. NVRAM chips are being developed that provide a more persistent memory than flash.
How Does an In-Memory Database Work?
Data storage in an in-memory database relies on a computer’s random access memory (RAM) or main memory instead of traditional disk drives. Data is loaded into an in-memory database in a compressed and non-relational format. The data is in a directly usable format without the barrier of compression or encryption. It allows for direct navigation from index to row or column and is a read-only system.
The speed of an in-memory database is made possible by lack of translation and caching. The data is used in the same form as the application that contains it. Data access is managed by an in-memory database management system.
An in-memory database system can also act as an read-only analytic database that stores historical data on metrics for business intelligence (BI) applications. This eliminates data indexing, which can reduce IT costs. Multi-core servers, 64-bit computing and lower RAM prices have made in-memory analytics more common.
Why Use an In-Memory Database?
Applications that manage vast quantities of data and require rapid response times can benefit from in-memory database architecture. The data analytics industry increasingly relies on in-memory database systems.
The benefits of an in-memory database include:
- Faster transactions
- No translation
- Multi-user concurrency
In-memory databases are commonly used for:
- Real-time banking, retail, advertising, medical device analytics, machine learning and billing/subscriber applications
- Online interactive gaming
- Geospatial/GIS processing
- Processing of streaming sensor data
- Developing embedded software systems
- Applications in transport systems, network switches and routers
- Fulfilling the requirements of e-commerce applications
In-Memory Database vs Traditional Database?
An in-memory database comparison to a traditional disk database includes speed, volume and volatility. An in-memory database keeps all data in the main memory or RAM of a computer. A traditional database retrieves data from disk drives.
In-memory databases are faster than traditional databases because they require fewer CPU instructions. They also eliminate the time it takes to access data from a disk.
In-memory databases are more volatile than traditional databases because data is lost when there is a loss of power or the computer’s RAM crashes. Data can be more easily restored from the disks of a traditional database.
Traditional databases are formatted by the disk drives on which the data is read and written. When one part of a traditional database refers to another part, a different block must be read from the disk. With an in-memory database, different parts of the database can be managed through direct pointers.
In-memory databases allow for real-time analysis and reporting of data.
Traditional databases store redundant data as the system creates a copy of the data for each component added to the system.
How to Implement an In-Memory Database?
To avoid the risk of losing data in a power outage or a computer crash, enhance an in-memory database with non-volatile random access memory (NVRAM). Flash is commonly used, despite its high cost and limitation in number of times memory can be rewritten. This can help in-memory databases maintain data after a power loss or crash. Flash is one example, but a major drawback is the limit to how many times flash memory can be erased and rewritten.
Reading a database from flash is faster than using a disk drive. That is why NVRAM chips are being developed that provide a more persistent memory than flash.
Does HEAVY.AI Offer an In-Memory Database?
HEAVY.AI is the world's fastest open source SQL engine and the HEAVY.AI platform is a powerful accelerator of third-party analytic apps like Apache Arrow, which will play a pivotal role in the evolution of in-memory computing. HEAVY.AI is working to integrate Arrow as an open source SQL engine and as an integrated part in end-to-end Machine Learning workflows.