Properly configuring MTRR

[sibok]

Hi,

while reading about plymouth and nvidia drivers I came around MTRR kernel settings. Quickly I run the following command which states uncachable for 3 MTRR registers:

cat /proc/mtrr

cat /proc/mtrr 
reg00: base=0x000000000 (    0MB), size=16384MB, count=1: write-back
reg01: base=0x400000000 (16384MB), size= 1024MB, count=1: write-back
reg02: base=0x0c0000000 ( 3072MB), size= 1024MB, count=1: uncachable
reg03: base=0x0bf700000 ( 3063MB), size=    1MB, count=1: uncachable
reg04: base=0x0bf800000 ( 3064MB), size=    8MB, count=1: uncachable

I went to make menuconfig -> Processor Type and Features -> MTRR support I've found MTRR cleanup support (which was disabled) so I enabled it.

-*- MTRR (Memory Type Range Register) support                                                                      │ │    
 │ │                                            [*]   MTRR cleanup support                                                                                         │ │    
 │ │                                            (0)     MTRR cleanup enable value (0-1) (NEW)                                                                      │ │    
 │ │                                            (1)     MTRR cleanup spare reg num (0-7) (NEW)                                                                     │ │    
 │ │                                            [ ] Intel MPX (Memory Protection Extensions)                                                                       │ │    
 │ │                                            [*] Intel Memory Protection Keys    

After reading /home/sys/usr/src/linux-4.19.5-gentoo/Documentation/x86/mtrr.txt I still don't know which values to place to MTRR cleanup enable value (0-1) (NEW) & MTRR cleanup spare reg num (0-7) (NEW) kernel config.

By now just placed value 1 to both options and compiled the kernel. It now shows the following in dmesg:
 

[    0.001933] MTRR default type: uncachable 
[    0.001933] MTRR fixed ranges enabled: 
[    0.001934]   00000-9FFFF write-back
[    0.001935]   A0000-BFFFF uncachable
[    0.001935]   C0000-E3FFF write-protect
[    0.001936]   E4000-EBFFF write-through
[    0.001936]   EC000-FFFFF write-protect
[    0.001936] MTRR variable ranges enabled: 
[    0.001937]   0 base 000000000 mask C00000000 write-back
[    0.001938]   1 base 400000000 mask FC0000000 write-back
[    0.001939]   2 base 0C0000000 mask FC0000000 uncachable
[    0.001940]   3 base 0BF700000 mask FFFF00000 uncachable
[    0.001940]   4 base 0BF800000 mask FFF800000 uncachable
[    0.001941]   5 disabled
[    0.001941]   6 disabled
[    0.001941]   7 disabled

[    0.002540] x86/PAT: Configuration [0-7]: WB  WC  UC- UC  WB  WP  UC- WT   
[    0.002611] total RAM covered: 16375M
[    0.002928] Found optimal setting for mtrr clean up
[    0.002929]  gran_size: 64K  chunk_size: 16M         num_reg: 7      lose cover RAM: 0G
[    0.003394] e820: update [mem 0xbf700000-0xffffffff] usable ==> reserved
[    0.003397] last_pfn = 0xbf680 max_arch_pfn = 0x400000000
[    0.003407] Scanning 1 areas for low memory corruption
[    0.003409] Base memory trampoline at [(____ptrval____)] 91000 size 24576
[    0.003414] BRK [0x02401000, 0x02401fff] PGTABLE
[    0.003415] BRK [0x02402000, 0x02402fff] PGTABLE
[    0.003416] BRK [0x02403000, 0x02403fff] PGTABLE
[    0.003451] BRK [0x02404000, 0x02404fff] PGTABLE
[    0.003596] BRK [0x02405000, 0x02405fff] PGTABLE
[    0.003602] BRK [0x02406000, 0x02406fff] PGTABLE

After compiling cat /proc/mtrr shows as follows:

reg00: base=0x000000000 (    0MB), size= 2048MB, count=1: write-back 
reg01: base=0x080000000 ( 2048MB), size= 1024MB, count=1: write-back
reg02: base=0x0bf700000 ( 3063MB), size=    1MB, count=1: uncachable
reg03: base=0x0bf800000 ( 3064MB), size=    8MB, count=1: uncachable
reg04: base=0x100000000 ( 4096MB), size= 4096MB, count=1: write-back
reg05: base=0x200000000 ( 8192MB), size= 8192MB, count=1: write-back
reg06: base=0x400000000 (16384MB), size= 1024MB, count=1: write-back

To me it looks like there are 2 registers unused (reg02 and reg03).

My system:

• Is running Gentoo-sources 4.19.5.
• i7-920 on an Asus P6T Deluxe.
• 16GB RAM (12GB on triple channel).
• Nvidia GTX-260 VGA (nvidia driver v340.107)

Looking into the BIOS I cannot find any MTR/MSR setting to switch from continuous to discrete.

Looking into my CPU flags it looks like it supports MTRR and PAT:

flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_ts
c arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 sse4_2 popcnt lahf_lm pti t
pr_shadow vnmi flexpriority ept vpid dtherm ida

 

lspci -v shows 3 "Memory at" lines but don't know how to identify which mtrr reg belongs to the VGA to setup MTRR in the kernel:

02:00.0 VGA compatible controller: NVIDIA Corporation GT200 [GeForce GTX 260] (rev a1) (prog-if 00 [VGA controller]) 
       Subsystem: ASUSTeK Computer Inc. GT200 [GeForce GTX 260]
       Flags: bus master, fast devsel, latency 0, IRQ 30
       Memory at fa000000 (32-bit, non-prefetchable) [size=16M]
       Memory at d0000000 (64-bit, prefetchable) [size=256M]
       Memory at f8000000 (64-bit, non-prefetchable) [size=32M]
       I/O ports at bc00 [size=128]
       [virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
       Capabilities: [60] Power Management version 3
       Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
       Capabilities: [78] Express Endpoint, MSI 00
       Capabilities: [100] Virtual Channel
       Capabilities: [128] Power Budgeting <?>
       Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
       Kernel driver in use: nvidia
       Kernel modules: nvidia

 

The following test throws 2 errors, maybe related to the 2 uncached registers:

fwts mtrr -  
 

Results generated by fwts: Version V17.02.00 (2017-02-16 02:00:35).

Some of this work - Copyright (c) 1999 - 2017, Intel Corp. All rights reserved.
Some of this work - Copyright (c) 2010 - 2017, Canonical.
Some of this work - Copyright (c) 2016 - 2017, IBM.

This test run on 02/12/18 at 02:31:29 on host Linux PauPC 4.19.5-gentoo #2 SMP Sat Dec 1 13:49:26 CET 2018 x86_64.

Command: "fwts mtrr -".
Running tests: mtrr.

mtrr: MTRR tests.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
MTRR overview
-------------
Reg 0: 0x0000000000000000 - 0x000000007fffffff (  2048 MB)   Write-Back
Reg 1: 0x0000000080000000 - 0x00000000bfffffff (  1024 MB)   Write-Back
Reg 2: 0x00000000bf700000 - 0x00000000bf7fffff (     1 MB)   Uncached
Reg 3: 0x00000000bf800000 - 0x00000000bfffffff (     8 MB)   Uncached
Reg 4: 0x0000000100000000 - 0x00000001ffffffff (  4096 MB)   Write-Back
Reg 5: 0x0000000200000000 - 0x00000003ffffffff (  8192 MB)   Write-Back
Reg 6: 0x0000000400000000 - 0x000000043fffffff (  1024 MB)   Write-Back

Test 1 of 3: Validate the kernel MTRR IOMEM setup.
FAILED [MEDIUM] MTRRIncorrectAttr: Test 1, Memory range 0x100000 to 0xbf67ffff (System RAM) has incorrect attribute Default (Most probably Uncached).
FAILED [MEDIUM] MTRRIncorrectAttr: Test 1, Memory range 0x100000000 to 0x43fffffff (System RAM) has incorrect attribute Default (Most probably Uncached).

Test 2 of 3: Validate the MTRR setup across all processors.
PASSED: Test 2, All processors have the a consistent MTRR setup.

Test 3 of 3: Test for AMD MtrrFixDramModEn being cleared by the BIOS.
SKIPPED: Test 3, CPU is not an AMD, cannot test.

===================================================================================================================================================================================
1 passed, 2 failed, 0 warning, 0 aborted, 1 skipped, 0 info only.
===================================================================================================================================================================================


1 passed, 2 failed, 0 warning, 0 aborted, 1 skipped, 0 info only.

Test Failure Summary
===================================================================================================================================================================================

Critical failures: NONE

High failures: NONE

Medium failures: 2
mtrr: Memory range 0x100000 to 0xbf67ffff (System RAM) has incorrect attribute Default (Most probably Uncached).
mtrr: Memory range 0x100000000 to 0x43fffffff (System RAM) has incorrect attribute Default (Most probably Uncached).

Low failures: NONE

Other failures: NONE

Test           |Pass |Fail |Abort|Warn |Skip |Info |
---------------+-----+-----+-----+-----+-----+-----+
mtrr           |    1|    2|     |     |    1|     |
---------------+-----+-----+-----+-----+-----+-----+
Total:         |    1|    2|    0|    0|    1|    0|
---------------+-----+-----+-----+-----+-----+-----+

 

I wonder:

• Why I can not see PAT enabled in dmesg? How can I make sure PAT is enabled?
• How should I configure MTRR to make use of the 7 registers?
• Why MTRR default type is uncachable?
• Whats does MTRR cleanup enable value mean at kernel config? I mean, what does the 0 and 1 value means?
• Setting enable_mtrr_cleanup mtrr_spare_reg_nr=1at cmdline (kernel line at grub.cfg) throughs a bunch of errors on dmesg.
• Also should I place mtrr_spare_reg_nr=2 (as cat /proc/mtrr shows) or mtrr_spare_reg_nr=2 (as dmesg states)?

Thanks in advanced! ?