The Turing GPU

The Turing GPU

Looking at the Turing GPU, there is a lot of stuff you can recognize, but there certainly have been fundamental block changes in the architecture, the SM (Streaming Multiprocessor) clusters have separated, core separated isolated blocks, something the Volta GPU architecture also shows as familiarity. Mind you that the base building block for all Turing GPUs will be the TU102, that is the flagship GPU that will be used on the GeForce RTX 2080 Ti. The TU102 is the GPU / graphics processor used on the GeForce (gosh I really have the tendency to write GTX nearly automatically when typing this article) RTX 2080. So the TU104 is a more simplified revision of the TU102, but shares the very same architecture.

Turing TU102 GPU Specifications

The GPU TU102 as the primary example is massive, it counts 18.6 billion transistors localized onto a 754mm2 die. In comparison, Pascal had close to 12 billion transistors on a die size of 471mm2. Gamers will immediately look at the shader processors, the Quadro RTX 8000 has 4608 of them enabled and since everything with bits is in multitudes of eight and while looking at the GPU die photos; it has 72 SMs (streaming multiprocessors) each holding 64 cores = 4608 Shader processors. This means A fully enabled GPU has 576 Tensor cores, 36 Geometry units, and 96 ROP units. This GPU is fabbed on an optimized 12nm TSMC FinFET+ node. We've placed the main specifications in a table overview.

Turing architecture

We'll try and be brief here, but Turing is a new and completely overhauled architecture that actually has gotten a new SM (Streaming multiprocessor) design. As I mentioned it has a bit of everything, but mostly it hints towards Volta. Two SMs are included per TPC (= Texture / Processor Cluster - a group made up of several SMs). Each SM has a total of 64 FP32 Cores and 64 INT32 Cores. Now before you get all confused yes that is radically different from Pascal (GeForce series 1000)who have one SM per TPC and 128 FP32 Cores per SM. The Turing SM architecture supports parallel execution of FP32 and INT32 operations, independent thread scheduling similar to the Volta GV100 GPU. That's also described as concurrent execution of FP32 and INT32 operation. Each Turing SM holds eight Turing Tensor Cores. With that out of the way, have a peek at the block diagram below.

Each Turing SM is partitioned into four processing blocks, each holds 16 FP32 Cores, 16 INT32 Cores, two Tensor Cores, one warp scheduler, and one dispatch unit. Each block includes a new L0 instruction cache and a 64 KB register file. The four processing blocks share a combined 96 KB L1 data cache/shared memory. Traditional graphics workloads partition the 96 KB L1/shared memory as 64 KB of dedicated graphics shader RAM and 32 KB for texture cache and register file spill area. Compute workloads can divide the 96 KB into 32 KB shared memory and 64 KB L1 cache, or 64 KB shared memory and 32 KB L1 cache.

Concurrent Execution of Floating Point (fp32) and Integer Instructions (int32)

Turing’s SM initiates a new unified architecture for shared memory, L1, and texture caching. This unified design allows the L1 cache to leverage resources, increasing its hit bandwidth by 2x per TPC compared to Pascal, and allows it to be reconfigured to grow larger when shared memory allocations are not using all the shared memory capacity. The Turing L1 can be as large as 64 KB in size, combined with a 32 KB per SM shared memory allocation, or it can reduce to 32 KB, allowing 64 KB of allocation to be used for shared memory. Turing’s L2 cache capacity has also been increased. Combining the L1 data cache with the shared memory reduces latency and provides higher bandwidth than the L1 cache implementation used previously in Pascal GPUs. NVIDIA claims these changes in SM enable Turing to achieve 50% improvement in delivered performance per CUDA core.

Caches and ROPs

Turing GPUs add larger and faster L2 caches in addition to the new GDDR6 memory subsystem. The TU102 GPU ships with 6 MB of L2 cache, double the 3 MB of L2 cache that was offered in the prior generation GP102 GPU used in the TITAN Xp. TU102 also provides significantly higher L2 cache bandwidth than GP102. Like prior generation, NVIDIA GPUs, each ROP partition in Turing contains eight ROP units and each unit can process a single-color sample. A full TU102 chip contains 12 ROP partitions for a total of 96 ROPs.

Graphics memory - GDDR6

Allow me to quickly inject a paragraph here. Another difference in-between Volta and Turing is graphics memory. HBM2 is a bust for consumer products, at least it seems and feels that way. The graphics industry at this time is clearly favoring the new GDDR6. It’s easier and cheaper to fab and add, and at this time can even exceed HBM2 in performance. The previous GeForce GTX 1080 with the latest GDDR5X memory could run 11 Gbps, often tweakable towards the 12 Gbps range. GDDR6 graphics memory will be faster and more energy efficient. The memory is advancing on GDDR5X (Graphics Double Data Rate (DDR)) but with a memory bandwidth of 14 Gbit/s it offers almost twice as much as GDDR5 (not GDDR5X)offers. In the near future, GDDR6 could transfer data at 16Gbps (bits per second), which is twice as fast as regular GDDR5. The GeForce RTX 2070 (8GB 256-bit), 2080 (8GB 256-bit) and 2080 Ti (11GB 352-bit) series will be paired with 14 Gbps GDDR6.