Scratchpad memory (SPM), also known as scratchpad, scratchpad RAM or local store in computer terminology, is an internal memory, usually high-speed, used for temporary storage of calculations, data, and other work in progress. In reference to a microprocessor (or CPU), scratchpad refers to a special high-speed memory used to hold small items of data for rapid retrieval. It is similar to the usage and size of a scratchpad in life: a pad of paper for preliminary notes or sketches or writings, etc. When the scratchpad is a hidden portion of the main memory then it is sometimes referred to as bump storage.
In some systems[a] it can be considered similar to the L1 cache in that it is the next closest memory to the ALU after the processor registers, with explicit instructions to move data to and from main memory, often using DMA-based data transfer.[1] In contrast to a system that uses caches, a system with scratchpads is a system with non-uniform memory access (NUMA) latencies, because the memory access latencies to the different scratchpads and the main memory vary. Another difference from a system that employs caches is that a scratchpad commonly does not contain a copy of data that is also stored in the main memory.
Scratchpads are employed for simplification of caching logic, and to guarantee a unit can work without main memory contention in a system employing multiple processors, especially in multiprocessor system-on-chip for embedded systems. They are mostly suited for storing temporary results (as it would be found in the CPU stack) that typically wouldn't need to always be committing to the main memory; however when fed by DMA, they can also be used in place of a cache for mirroring the state of slower main memory. The same issues of locality of reference apply in relation to efficiency of use; although some systems allow strided DMA to access rectangular data sets. Another difference is that scratchpads are explicitly manipulated by applications. They may be useful for realtime applications, where predictable timing is hindered by cache behavior.
Scratchpads are not used in mainstream desktop processors where generality is required for legacy software to run from generation to generation, in which the available on-chip memory size may change. They are better implemented in embedded systems, special-purpose processors and game consoles, where chips are often manufactured as MPSoC, and where software is often tuned to one hardware configuration.
Fairchild F8 of 1975 contained 64 bytes of scratchpad.
The TI-99/4A has 256 bytes of scratchpad memory on the 16-bit bus containing the processor registers of the TMS9900[2]
Cyrix 6x86 is the only x86-compatible desktop processor to incorporate a dedicated scratchpad.
SuperH, used in Sega's consoles, could lock cachelines to an address outside of main memory for use as a scratchpad.
Sony's PS1's R3000 had a scratchpad instead of an L1 cache. It was possible to place the CPU stack here, an example of the temporary workspace usage.
Adapteva's Epiphany parallel coprocessor features local-stores for each core, connected by a network on a chip, with DMA possible between them and off-chip links (possibly to DRAM). The architecture is similar to Sony's Cell, except all cores can directly address each other's scratchpads, generating network messages from standard load/store instructions.
Sony's PS2Emotion Engine includes a 16 KB scratchpad, to and from which DMA transfers could be issued to its GS, and main memory.
Cell's SPEs are restricted purely to working in their "local-store", relying on DMA for transfers from/to main memory and between local stores, much like a scratchpad. In this regard, additional benefit is derived from the lack of hardware to check and update coherence between multiple caches: the design takes advantage of the assumption that each processor's workspace is separate and private. It is expected this benefit will become more noticeable as the number of processors scales into the "many-core" future. Yet because of the elimination of some hardware logics, the data and instructions of applications on SPEs must be managed through software if the whole task on SPE can not fit in local store.[3][4][5]
Many other processors allow L1 cache lines to be locked.
Most digital signal processors use a scratchpad. Many past 3D accelerators and game consoles (including the PS2) have used DSPs for vertex transformations. This differs from the stream-based approach of modern GPUs which have more in common with a CPU cache's functions.
NVIDIA's 8800GPU running under CUDA provides 16 KB of scratchpad (NVIDIA calls it Shared Memory) per thread-bundle when being used for GPGPU tasks. Scratchpad also was used in later Fermi GPU (GeForce 400 series).[6]
Ageia's PhysX chip includes a scratchpad RAM in a manner similar to the Cell; the theory of this specific physics processing unit is that a cache hierarchy is of less use than software managed physics and collision calculations. These memories are also banked and a switch manages transfers between them.
Intel's Knights Landing processor has a 16 GB MCDRAM that can be configured as either a cache, scratchpad memory, or divided into some cache and some scratchpad memory.
Some architectures such as PowerPC attempt to avoid the need for cacheline locking or scratchpads through the use of cache control instructions. Marking an area of memory with "Data Cache Block: Zero" (allocating a line but setting its contents to zero instead of loading from main memory) and discarding it after use ('Data Cache Block: Invalidate', signaling that main memory didn't receive any updated data) the cache is made to behave as a scratchpad. Generality is maintained in that these are hints and the underlying hardware will function correctly regardless of actual cache size.
Shared L2 vs Cell local storesedit
Regarding interprocessor communication in a multicore setup, there are similarities between the Cell's inter-localstore DMA and a shared L2 cache setup as in the Intel Core 2 Duo or the Xbox 360's custom powerPC: the L2 cache allows processors to share results without those results having to be committed to main memory. This can be an advantage where the working set for an algorithm encompasses the entirety of the L2 cache. However, when a program is written to take advantage of inter-localstore DMA, the Cell has the benefit of each-other-Local-Store serving the purpose of BOTH the private workspace for a single processor AND the point of sharing between processors; i.e., the other Local Stores are on a similar footing viewed from one processor as the shared L2 cache in a conventional chip. The tradeoff is that of memory wasted in buffering and programming complexity for synchronization, though this would be similar to precached pages in a conventional chip. Domains where using this capability is effective include:
Pipeline processing (where one achieves the same effect as increasing the L1 cache's size by splitting one job into smaller chunks)
Extending the working set, e.g., a sweet spot for a merge sort where the data fits within 8×256 KB
Shared code uploading, like loading a piece of code to one SPU, then copy it from there to the others to avoid hitting the main memory again
It would be possible for a conventional processor to gain similar advantages with cache-control instructions, for example, allowing the prefetching to the L1 bypassing the L2, or an eviction hint that signaled a transfer from L1 to L2 but not committing to main memory; however, at present no systems offer this capability in a usable form and such instructions in effect should mirror explicit transfer of data among cache areas used by each core.
^Some older systems used a hidden part of main storage, referred to as bump storage, as scratchpad. In other systems, e.g., UNIVAC 1107, all addressable registers were held in scratchpad.
Referencesedit
^Steinke, Stefan; Lars Wehmeyer; Bo-Sik Lee; Peter Marwedel (2002). "Assigning Program and Data Objects to Scratchpad for Energy Reduction" (PDF). University of Dortmund. Retrieved 3 October 2013.: "3.2 Scratchpad model .. The scratchpad memory uses software to control the location assignment of data."
^J. Lu, K. Bai, A. Shrivastava, "SSDM: Smart Stack Data Management for Software Managed Multicores (SMMs)", Design Automation Conference (DAC), June 2–6, 2013
^K. Bai, A. Shrivastava, "Automatic and Efficient Heap Data Management for Limited Local Memory Multicore Architectures", Design Automation and Test in Europe (DATE), 2013
^K. Bai, J. Lu, A. Shrivastava, B. Holton, "CMSM: An Efficient and Effective Code Management for Software Managed Multicores", CODES+ISSS, 2013
^Patterson, David (September 30, 2009). "The Top 10 Innovations in the New NVIDIA Fermi Architecture, and the Top 3 Next Challenges" (PDF). Parallel Computing Research Laboratory & NVIDIA. Retrieved 3 October 2013.
^Jia, Zhe; Tillman, Blake; Maggioni, Marco; Scarpazza, Daniele P. (December 7, 2019). Dissecting the Graphcore IPU Architecture via Microbenchmarking(PDF) (Technical report). Citadel Enterprise Americas, LLC. arXiv:1912.03413.
External linksedit
Rajeshwari Banakar, Scratchpad Memory : A Design Alternative for Cache. On-chip memory in Embedded Systems // CODES'02. May 6–8, 2002
April 10, 2024
scratchpad, memory, this, article, needs, additional, citations, verification, please, help, improve, this, article, adding, citations, reliable, sources, unsourced, material, challenged, removed, find, sources, news, newspapers, books, scholar, jstor, october. This article needs additional citations for verification Please help improve this article by adding citations to reliable sources Unsourced material may be challenged and removed Find sources Scratchpad memory news newspapers books scholar JSTOR October 2013 Learn how and when to remove this template message Scratchpad memory SPM also known as scratchpad scratchpad RAM or local store in computer terminology is an internal memory usually high speed used for temporary storage of calculations data and other work in progress In reference to a microprocessor or CPU scratchpad refers to a special high speed memory used to hold small items of data for rapid retrieval It is similar to the usage and size of a scratchpad in life a pad of paper for preliminary notes or sketches or writings etc When the scratchpad is a hidden portion of the main memory then it is sometimes referred to as bump storage In some systems a it can be considered similar to the L1 cache in that it is the next closest memory to the ALU after the processor registers with explicit instructions to move data to and from main memory often using DMA based data transfer 1 In contrast to a system that uses caches a system with scratchpads is a system with non uniform memory access NUMA latencies because the memory access latencies to the different scratchpads and the main memory vary Another difference from a system that employs caches is that a scratchpad commonly does not contain a copy of data that is also stored in the main memory Scratchpads are employed for simplification of caching logic and to guarantee a unit can work without main memory contention in a system employing multiple processors especially in multiprocessor system on chip for embedded systems They are mostly suited for storing temporary results as it would be found in the CPU stack that typically wouldn t need to always be committing to the main memory however when fed by DMA they can also be used in place of a cache for mirroring the state of slower main memory The same issues of locality of reference apply in relation to efficiency of use although some systems allow strided DMA to access rectangular data sets Another difference is that scratchpads are explicitly manipulated by applications They may be useful for realtime applications where predictable timing is hindered by cache behavior Scratchpads are not used in mainstream desktop processors where generality is required for legacy software to run from generation to generation in which the available on chip memory size may change They are better implemented in embedded systems special purpose processors and game consoles where chips are often manufactured as MPSoC and where software is often tuned to one hardware configuration Contents 1 Examples of use 2 Alternatives 2 1 Cache control vs scratchpads 2 2 Shared L2 vs Cell local stores 3 See also 4 Notes 5 References 6 External linksExamples of use editFairchild F8 of 1975 contained 64 bytes of scratchpad The TI 99 4A has 256 bytes of scratchpad memory on the 16 bit bus containing the processor registers of the TMS9900 2 Cyrix 6x86 is the only x86 compatible desktop processor to incorporate a dedicated scratchpad SuperH used in Sega s consoles could lock cachelines to an address outside of main memory for use as a scratchpad Sony s PS1 s R3000 had a scratchpad instead of an L1 cache It was possible to place the CPU stack here an example of the temporary workspace usage Adapteva s Epiphany parallel coprocessor features local stores for each core connected by a network on a chip with DMA possible between them and off chip links possibly to DRAM The architecture is similar to Sony s Cell except all cores can directly address each other s scratchpads generating network messages from standard load store instructions Sony s PS2 Emotion Engine includes a 16 KB scratchpad to and from which DMA transfers could be issued to its GS and main memory Cell s SPEs are restricted purely to working in their local store relying on DMA for transfers from to main memory and between local stores much like a scratchpad In this regard additional benefit is derived from the lack of hardware to check and update coherence between multiple caches the design takes advantage of the assumption that each processor s workspace is separate and private It is expected this benefit will become more noticeable as the number of processors scales into the many core future Yet because of the elimination of some hardware logics the data and instructions of applications on SPEs must be managed through software if the whole task on SPE can not fit in local store 3 4 5 Many other processors allow L1 cache lines to be locked Most digital signal processors use a scratchpad Many past 3D accelerators and game consoles including the PS2 have used DSPs for vertex transformations This differs from the stream based approach of modern GPUs which have more in common with a CPU cache s functions NVIDIA s 8800 GPU running under CUDA provides 16 KB of scratchpad NVIDIA calls it Shared Memory per thread bundle when being used for GPGPU tasks Scratchpad also was used in later Fermi GPU GeForce 400 series 6 Ageia s PhysX chip includes a scratchpad RAM in a manner similar to the Cell the theory of this specific physics processing unit is that a cache hierarchy is of less use than software managed physics and collision calculations These memories are also banked and a switch manages transfers between them Intel s Knights Landing processor has a 16 GB MCDRAM that can be configured as either a cache scratchpad memory or divided into some cache and some scratchpad memory Movidius Myriad 2 a vision processing unit organized as a multicore architecture with a large multiported shared scratchpad Graphcore has designed an AI accelerator based on scratchpad memories 7 Alternatives editCache control vs scratchpads edit Some architectures such as PowerPC attempt to avoid the need for cacheline locking or scratchpads through the use of cache control instructions Marking an area of memory with Data Cache Block Zero allocating a line but setting its contents to zero instead of loading from main memory and discarding it after use Data Cache Block Invalidate signaling that main memory didn t receive any updated data the cache is made to behave as a scratchpad Generality is maintained in that these are hints and the underlying hardware will function correctly regardless of actual cache size Shared L2 vs Cell local stores edit Regarding interprocessor communication in a multicore setup there are similarities between the Cell s inter localstore DMA and a shared L2 cache setup as in the Intel Core 2 Duo or the Xbox 360 s custom powerPC the L2 cache allows processors to share results without those results having to be committed to main memory This can be an advantage where the working set for an algorithm encompasses the entirety of the L2 cache However when a program is written to take advantage of inter localstore DMA the Cell has the benefit of each other Local Store serving the purpose of BOTH the private workspace for a single processor AND the point of sharing between processors i e the other Local Stores are on a similar footing viewed from one processor as the shared L2 cache in a conventional chip The tradeoff is that of memory wasted in buffering and programming complexity for synchronization though this would be similar to precached pages in a conventional chip Domains where using this capability is effective include Pipeline processing where one achieves the same effect as increasing the L1 cache s size by splitting one job into smaller chunks Extending the working set e g a sweet spot for a merge sort where the data fits within 8 256 KB Shared code uploading like loading a piece of code to one SPU then copy it from there to the others to avoid hitting the main memory againIt would be possible for a conventional processor to gain similar advantages with cache control instructions for example allowing the prefetching to the L1 bypassing the L2 or an eviction hint that signaled a transfer from L1 to L2 but not committing to main memory however at present no systems offer this capability in a usable form and such instructions in effect should mirror explicit transfer of data among cache areas used by each core See also editCPU cache NUMA MPSoCNotes edit Some older systems used a hidden part of main storage referred to as bump storage as scratchpad In other systems e g UNIVAC 1107 all addressable registers were held in scratchpad References edit Steinke Stefan Lars Wehmeyer Bo Sik Lee Peter Marwedel 2002 Assigning Program and Data Objects to Scratchpad for Energy Reduction PDF University of Dortmund Retrieved 3 October 2013 3 2 Scratchpad model The scratchpad memory uses software to control the location assignment of data The TI 99 4A internal architecture www unige ch Retrieved 2023 03 08 J Lu K Bai A Shrivastava SSDM Smart Stack Data Management for Software Managed Multicores SMMs Design Automation Conference DAC June 2 6 2013 K Bai A Shrivastava Automatic and Efficient Heap Data Management for Limited Local Memory Multicore Architectures Design Automation and Test in Europe DATE 2013 K Bai J Lu A Shrivastava B Holton CMSM An Efficient and Effective Code Management for Software Managed Multicores CODES ISSS 2013 Patterson David September 30 2009 The Top 10 Innovations in the New NVIDIA Fermi Architecture and the Top 3 Next Challenges PDF Parallel Computing Research Laboratory amp NVIDIA Retrieved 3 October 2013 Jia Zhe Tillman Blake Maggioni Marco Scarpazza Daniele P December 7 2019 Dissecting the Graphcore IPU Architecture via Microbenchmarking PDF Technical report Citadel Enterprise Americas LLC arXiv 1912 03413 External links editRajeshwari Banakar Scratchpad Memory A Design Alternative for Cache On chip memory in Embedded Systems CODES 02 May 6 8 2002 Retrieved from https en wikipedia org w index php title Scratchpad memory amp oldid 1211301072, wikipedia, wiki, book, books, library,
