| .. SPDX-License-Identifier: (GPL-2.0+ OR MIT) | |
| ========================= | |
| FWSEC (Firmware Security) | |
| ========================= | |
| This document briefly/conceptually describes the FWSEC (Firmware Security) image | |
| and its role in the GPU boot sequence. As such, this information is subject to | |
| change in the future and is only current as of the Ampere GPU family. However, | |
| hopefully the concepts described will be useful for understanding the kernel code | |
| that deals with it. All the information is derived from publicly available | |
| sources such as public drivers and documentation. | |
| The role of FWSEC is to provide a secure boot process. It runs in | |
| 'Heavy-secure' mode, and performs firmware verification after a GPU reset | |
| before loading various ucode images onto other microcontrollers on the GPU, | |
| such as the PMU and GSP. | |
| FWSEC itself is an application stored in the VBIOS ROM in the FWSEC partition of | |
| ROM (see vbios.rst for more details). It contains different commands like FRTS | |
| (Firmware Runtime Services) and SB (Secure Booting other microcontrollers after | |
| reset and loading them with other non-FWSEC ucode). The kernel driver only needs | |
| to perform FRTS, since Secure Boot (SB) has already completed by the time the driver | |
| is loaded. | |
| The FRTS command carves out the WPR2 region (Write protected region) which contains | |
| data required for power management. Once setup, only HS mode ucode can access it | |
| (see falcon.rst for privilege levels). | |
| The FWSEC image is located in the VBIOS ROM in the partition of the ROM that contains | |
| various ucode images (also known as applications) -- one of them being FWSEC. For how | |
| it is extracted, see vbios.rst and the vbios.rs source code. | |
| The Falcon data for each ucode images (including the FWSEC image) is a combination | |
| of headers, data sections (DMEM) and instruction code sections (IMEM). All these | |
| ucode images are stored in the same ROM partition and the PMU table is used to look | |
| up the application to load it based on its application ID (see vbios.rs). | |
| For the nova-core driver, the FWSEC contains an 'application interface' called | |
| DMEMMAPPER. This interface is used to execute the 'FWSEC-FRTS' command, among others. | |
| For Ampere, FWSEC is running on the GSP in Heavy-secure mode and runs FRTS. | |
| FWSEC Memory Layout | |
| ------------------- | |
| The memory layout of the FWSEC image is as follows:: | |
| +---------------------------------------------------------------+ | |
| | FWSEC ROM image (type 0xE0) | | |
| | | | |
| | +---------------------------------+ | | |
| | | PMU Falcon Ucode Table | | | |
| | | (PmuLookupTable) | | | |
| | | +-------------------------+ | | | |
| | | | Table Header | | | | |
| | | | - version: 0x01 | | | | |
| | | | - header_size: 6 | | | | |
| | | | - entry_size: 6 | | | | |
| | | | - entry_count: N | | | | |
| | | | - desc_version:3(unused)| | | | |
| | | +-------------------------+ | | | |
| | | ... | | | |
| | | +-------------------------+ | | | |
| | | | Entry for FWSEC (0x85) | | | | |
| | | | (PmuLookupTableEntry) | | | | |
| | | | - app_id: 0x85 (FWSEC) |----|----+ | | |
| | | | - target_id: 0x01 (PMU) | | | | | |
| | | | - data: offset ---------|----|----|---+ look up FWSEC | | |
| | | +-------------------------+ | | | | | |
| | +---------------------------------+ | | | | |
| | | | | | |
| | | | | | |
| | +---------------------------------+ | | | | |
| | | FWSEC Ucode Component |<---+ | | | |
| | | (aka Falcon data) | | | | |
| | | +-------------------------+ | | | | |
| | | | FalconUCodeDescV3 |<---|--------+ | | |
| | | | - hdr | | | | |
| | | | - stored_size | | | | |
| | | | - pkc_data_offset | | | | |
| | | | - interface_offset -----|----|----------------+ | | |
| | | | - imem_phys_base | | | | | |
| | | | - imem_load_size | | | | | |
| | | | - imem_virt_base | | | | | |
| | | | - dmem_phys_base | | | | | |
| | | | - dmem_load_size | | | | | |
| | | | - engine_id_mask | | | | | |
| | | | - ucode_id | | | | | |
| | | | - signature_count | | look up sig | | | |
| | | | - signature_versions --------------+ | | | |
| | | +-------------------------+ | | | | | |
| | | (no gap) | | | | | |
| | | +-------------------------+ | | | | | |
| | | | Signatures Section |<---|-----+ | | | |
| | | | (384 bytes per sig) | | | | | |
| | | | - RSA-3K Signature 1 | | | | | |
| | | | - RSA-3K Signature 2 | | | | | |
| | | | ... | | | | | |
| | | +-------------------------+ | | | | |
| | | | | | | |
| | | +-------------------------+ | | | | |
| | | | IMEM Section (Code) | | | | | |
| | | | | | | | | |
| | | | Contains instruction | | | | | |
| | | | code etc. | | | | | |
| | | +-------------------------+ | | | | |
| | | | | | | |
| | | +-------------------------+ | | | | |
| | | | DMEM Section (Data) | | | | | |
| | | | | | | | | |
| | | | +---------------------+ | | | | | |
| | | | | Application | |<---|----------------+ | | |
| | | | | Interface Table | | | | | |
| | | | | (FalconAppifHdrV1) | | | | | |
| | | | | Header: | | | | | |
| | | | | - version: 0x01 | | | | | |
| | | | | - header_size: 4 | | | | | |
| | | | | - entry_size: 8 | | | | | |
| | | | | - entry_count: N | | | | | |
| | | | | | | | | | |
| | | | | Entries: | | | | | |
| | | | | +-----------------+ | | | | | |
| | | | | | DEVINIT (ID 1) | | | | | | |
| | | | | | - id: 0x01 | | | | | | |
| | | | | | - dmemOffset X -|-|-|----+ | | |
| | | | | +-----------------+ | | | | | |
| | | | | +-----------------+ | | | | | |
| | | | | | DMEMMAPPER(ID 4)| | | | | | |
| | | | | | - id: 0x04 | | | | Used only for DevInit | | |
| | | | | | (NVFW_FALCON_ | | | | application (not FWSEC) | | |
| | | | | | APPIF_ID_DMEMMAPPER) | | | |
| | | | | | - dmemOffset Y -|-|-|----|-----+ | | |
| | | | | +-----------------+ | | | | | | |
| | | | +---------------------+ | | | | | |
| | | | | | | | | |
| | | | +---------------------+ | | | | | |
| | | | | DEVINIT Engine |<|----+ | Used by FWSEC | | |
| | | | | Interface | | | | app. | | |
| | | | +---------------------+ | | | | | |
| | | | | | | | | |
| | | | +---------------------+ | | | | | |
| | | | | DMEM Mapper (ID 4) |<|----+-----+ | | |
| | | | | (FalconAppifDmemmapperV3) | | | |
| | | | | - signature: "DMAP" | | | | | |
| | | | | - version: 0x0003 | | | | | |
| | | | | - Size: 64 bytes | | | | | |
| | | | | - cmd_in_buffer_off | |----|------------+ | | |
| | | | | - cmd_in_buffer_size| | | | | | |
| | | | | - cmd_out_buffer_off| |----|------------|-----+ | | |
| | | | | - cmd_out_buffer_sz | | | | | | | |
| | | | | - init_cmd | | | | | | | |
| | | | | - features | | | | | | | |
| | | | | - cmd_mask0/1 | | | | | | | |
| | | | +---------------------+ | | | | | | |
| | | | | | | | | | |
| | | | +---------------------+ | | | | | | |
| | | | | Command Input Buffer|<|----|------------+ | | | |
| | | | | - Command data | | | | | | |
| | | | | - Arguments | | | | | | |
| | | | +---------------------+ | | | | | |
| | | | | | | | | |
| | | | +---------------------+ | | | | | |
| | | | | Command Output |<|----|------------------+ | | |
| | | | | Buffer | | | | | |
| | | | | - Results | | | | | |
| | | | | - Status | | | | | |
| | | | +---------------------+ | | | | |
| | | +-------------------------+ | | | |
| | +---------------------------------+ | | |
| | | | |
| +---------------------------------------------------------------+ | |
| .. note:: | |
| This is using an GA-102 Ampere GPU as an example and could vary for future GPUs. | |
| .. note:: | |
| The FWSEC image also plays a role in memory scrubbing (ECC initialization) and VPR | |
| (Video Protected Region) initialization as well. Before the nova-core driver is even | |
| loaded, the FWSEC image is running on the GSP in heavy-secure mode. After the devinit | |
| sequence completes, it does VRAM memory scrubbing (ECC initialization). On consumer | |
| GPUs, it scrubs only part of memory and then initiates 'async scrubbing'. Before this | |
| async scrubbing completes, the unscrubbed VRAM cannot be used for allocation (thus DRM | |
| memory allocators need to wait for this scrubbing to complete). |