In-memory processing

In computer science, in-memory processing (PIM) is a computer architecture for processing data stored in an in-memory database.^[1] In-memory processing improves the power usage and performance of moving data between the processor and the main memory.^[2] Older systems have been based on disk storage and relational databases using Structured Query Language, which are increasingly regarded as inadequate to meet business intelligence (BI) needs. Because stored data is accessed much more quickly when it is placed in random-access memory (RAM) or flash memory, in-memory processing allows data to be analyzed in real time, enabling faster reporting and decision-making in business.^[3][4]

Disk-based business intelligence

Data structures

With disk-based technology, data is loaded on to the computer's hard disk in the form of multiple tables and multi-dimensional structures against which queries are run. Disk-based technologies are relational database management systems (RDBMS), often based on the structured query language (SQL), such as SQL Server, MySQL, Oracle and many others. RDBMS are designed for the requirements of transactional processing. Using a database that supports insertions and updates as well as performing aggregations, joins (typical in BI solutions) are typically very slow. Another drawback is that SQL is designed to efficiently fetch rows of data, while BI queries usually involve fetching of partial rows of data involving heavy calculations.

To improve query performance, multidimensional databases or OLAP cubes - also called multidimensional online analytical processing (MOLAP) - are constructed. Designing a cube is an elaborate and lengthy process, and changing the cube's structure to adapt to dynamically changing business needs may be cumbersome. Cubes are pre-populated with data to answer specific queries and although they increase performance, they are still not suitable for answering ad-hoc queries.^[5]

Information technology (IT) staff spend substantial development time on optimizing databases, constructing indexes and aggregates, designing cubes and star schemas, data modeling, and query analysis.^[6]

Processing speed

Reading data from the hard disk is much slower (possibly hundreds of times) when compared to reading the same data from RAM. Especially when analyzing large volumes of data, performance is severely degraded. Though SQL is a very powerful tool, complex queries take a relatively long time to execute and often result in bringing down the performance of transactional processing. In order to obtain results within an acceptable response time, many data warehouses have been designed to pre-calculate summaries and answer specific queries only. Optimized aggregation algorithms are needed to increase performance.

In-memory processing tools

Memory processing can be accomplished via traditional databases such as Oracle, IBM Db2 or Microsoft SQL Server or via NoSQL offerings such as in-memory data grid like Hazelcast, Infinispan, Oracle Coherence or ScaleOut Software. With both in-memory database and data grid, all information is initially loaded into memory RAM or flash memory instead of hard disks. With a data grid processing occurs at three order of magnitude faster than relational databases which have advanced functionality such as ACID which degrade performance in compensation for the additional functionality. The arrival of column centric databases, which store similar information together, allow data to be stored more efficiently and with greater compression ratios. This allows huge amounts of data to be stored in the same physical space, reducing the amount of memory needed to perform a query and increasing processing speed. Many users and software vendors have integrated flash memory into their systems to allow systems to scale to larger data sets more economically. Oracle has been integrating flash memory into the Oracle Exadata products for increased performance. Microsoft SQL Server 2012 BI/Data Warehousing software has been coupled with Violin Memory flash memory arrays to enable in-memory processing of data sets greater than 20TB.^[7]

Users query the data loaded into the system’s memory, thereby avoiding slower database access and performance bottlenecks. This differs from caching, a very widely used method to speed up query performance, in that caches are subsets of very specific pre-defined organized data. With in-memory tools, data available for analysis can be as large as a data mart or small data warehouse which is entirely in memory. This can be accessed quickly by multiple concurrent users or applications at a detailed level and offers the potential for enhanced analytics and for scaling and increasing the speed of an application. Theoretically, the improvement in data access speed is 10,000 to 1,000,000 times compared to the disk.^{[citation needed]} It also minimizes the need for performance tuning by IT staff and provides faster service for end users.

Advantages of in-memory processing technology

Certain developments in computer technology and business needs have tended to increase the relative advantages of in-memory technology.^[8]

• Hardware becomes progressively cheaper and higher-performing, according to Moore's law. Computing power doubles every two to three years while decreasing in costs. CPU processing, memory and disk storage are all subject to some variation of this law. Also hardware innovations such as multi-core architecture, NAND flash memory, parallel servers, and increased memory processing capability, in addition to software innovations such as column centric databases, compression techniques and handling aggregate tables, have all contributed to demand for in-memory products.^[9]
• The advent of 64-bit operating systems, which allow access to far more RAM (up to 100 GB or more) than the 2 or 4 GB accessible on 32-bit systems. By providing Terabytes (1 TB = 1,024 GB) of space for storage and analysis, 64-bit operating systems make in-memory processing scalable. The use of flash memory enables systems to scale to many Terabytes more economically.
• Increasing volumes of data have meant that traditional data warehouses are no longer able to process the data in a timely and accurate way. The extract, transform, load (ETL) process that periodically updates data warehouses with operational data can take anywhere from a few hours to weeks to complete. So, at any given point of time data is at least a day old. In-memory processing enables instant access to terabytes of data for real time reporting.
• In-memory processing is available at a lower cost compared to traditional BI tools, and can be more easily deployed and maintained. According to Gartner survey,^{[citation needed]} deploying traditional BI tools can take as long as 17 months. Many data warehouse vendors are choosing in-memory technology over traditional BI to speed up implementation times.
• Decreases in power consumption and increases in throughput due to a lower access latency, and greater memory bandwidth and hardware parallelism.^[10]

Application in business

A range of in-memory products provide ability to connect to existing data sources and access to visually rich interactive dashboards. This allows business analysts and end users to create custom reports and queries without much training or expertise. Easy navigation and ability to modify queries on the fly is of benefit to many users. Since these dashboards can be populated with fresh data, users have access to real time data and can create reports within minutes. In-memory processing may be of particular benefit in call centers and warehouse management.

With in-memory processing, the source database is queried only once instead of accessing the database every time a query is run, thereby eliminating repetitive processing and reducing the burden on database servers. By scheduling to populate the in-memory database overnight, the database servers can be used for operational purposes during peak hours.

Adoption of in-memory technology

With a large number of users, a large amount of RAM is needed for an in-memory configuration, which in turn affects the hardware costs. The investment is more likely to be suitable in situations where speed of query response is a high priority, and where there is significant growth in data volume and increase in demand for reporting facilities; it may still not be cost-effective where information is not subject to rapid change. Security is another consideration, as in-memory tools expose huge amounts of data to end users. Makers advise ensuring that only authorized users are given access to the data.

See also

• System on a chip
• Network on a chip

References

1. Ghose, S. (November 2019). "Processing-in-memory: A workload-driven perspective" (PDF). IBM Journal of Research and Development. 63 (6): 3:1–19. doi:10.1147/JRD.2019.2934048. S2CID 202025511.
2. Chi, Ping; Li, Shuangchen; Xu, Cong; Zhang, Tao; Zhao, Jishen; Liu, Yongpan; Wang, Yu; Xie, Yuan (June 2016). "PRIME: A Novel Processing-in-Memory Architecture for Neural Network Computation in ReRAM-Based Main Memory". 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA). Seoul, South Korea: IEEE. pp. 27–39. doi:10.1109/ISCA.2016.13. ISBN 978-1-4673-8947-1.
3. Plattner, Hasso; Zeier, Alexander (2012). In-Memory Data Management: Technology and Applications. Springer Science & Business Media. ISBN 9783642295744.
4. Zhang, Hao; Gang Chen; Beng Chin Ooi; Kian-Lee Tan; Meihui Zhang (July 2015). "In-Memory Big Data Management and Processing: A Survey". IEEE Transactions on Knowledge and Data Engineering. 27 (7): 1920–1948. doi:10.1109/TKDE.2015.2427795.
5. Gill, John (2007). . Business Intelligence Journal. 12 (2): 58–62. Archived from the original on 2015-09-24.
6. Earls, A (2011). (PDF). Tableau. Archived from the original (PDF) on 2012-04-25.
7. (PDF). Microsoft. Archived from the original (PDF) on 2013-03-09. Retrieved 2013-06-01.
8. "In_memory Analytics". yellowfin. p. 6.
9. Kote, Sparjan. . Archived from the original on April 24, 2011.
10. Upchurch, E.; Sterling, T.; Brockman, J. (2004). "Analysis and Modeling of Advanced PIM Architecture Design Tradeoffs". Proceedings of the ACM/IEEE SC2004 Conference. Pittsburgh, PA, USA: IEEE. p. 12. doi:10.1109/SC.2004.11. ISBN 978-0-7695-2153-4. S2CID 9089044.