Commit graph

37 commits

Author SHA1 Message Date
Asahi Lina
da9ceddeac memory: Map lowmem using 16K pages only
Turns out CTRR does not like working with huge pages, and just throws up
its hands in the air with an L2 address size fault if a huge page
overlaps the CTRR region.

Signed-off-by: Asahi Lina <lina@asahilina.net>
2022-09-27 06:02:28 +09:00
Asahi Lina
89a482fcab memory: Do not invalidate Normal-NC mappings
We probably don't need it, and the hypervisor doesn't like it.

Signed-off-by: Asahi Lina <lina@asahilina.net>
2022-08-17 17:17:22 +09:00
Hector Martin
849729f0c0 memory: Add GRE and nGRE mappings for experiments
Signed-off-by: Hector Martin <marcan@marcan.st>
2022-06-19 21:53:06 +09:00
Hector Martin
c95189ba2e memory: Normalize naming of Normal-NC memory attributes
Signed-off-by: Hector Martin <marcan@marcan.st>
2022-06-19 16:12:16 +09:00
Janne Grunau
7f4347ebe8 display: Reallocate framebuffer if required for wanted display mode
If an explicitly specified display mode exceeds the allocated
framebuffer allocate a new one from the top of RAM.

Note: macOS panics immediately with a realloced framebuffer.

Signed-off-by: Janne Grunau <j@jannau.net>
2022-05-30 23:01:01 +09:00
Hector Martin
fabe27e3f1 memory: Remap some carveouts as uncached
This fixes display DART real-time cache hits causing AMCC exceptions.

The relevant carve-outs have flags 0x60004016; 0x60004002 is used for
DCP which is non-realtime, so I'm guessing the '16' means we should map
it uncached.

Signed-off-by: Hector Martin <marcan@marcan.st>
2022-04-19 23:41:17 +09:00
Hector Martin
688c6b1494 memory: Dynamically allocate top-level page tables
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-12-17 21:46:55 +09:00
Hector Martin
bedcc905a3 gxf, smp: Allocate stacks dynamically
This significantly shrinks our .bss section

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-12-17 21:46:55 +09:00
Hector Martin
4b80041d6d mcc: New module to initialize the MCC memory controller.
This turns on the system level cache. The carveout unmapping also moves
here, and now it handles T8103/T6000 properly.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-11-16 21:09:48 +09:00
Hector Martin
8fa422c0d3 fb: Map the framebuffer uncached, and use a shadow FB for speed
Turns out AMCC on t600x throws errors when DISP0 real-time memory
requests hit the CPU cache, and then macOS panics.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-11-04 03:52:30 +09:00
Hector Martin
68aec75918 memory: Larger PA support & dynamically map MMIO
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-11-01 21:00:56 +09:00
Hector Martin
7ff48f6201 memory: Make the RAM base dynamic
For now we compute this as phys_base aligned down to a 4GiB boundary.
Hopefully that works for future SoCs too.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-11-01 13:39:40 +09:00
Hector Martin
72788291e6 memory: Map all RAM to the top, minus carveouts
This unbreaks dcp.py and other things that need to access reserved
regions. This way we don't have to start doing manual MMU maps, but
we're still safe from SErrors caused by hitting TZ carveouts.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-10-16 01:00:33 +09:00
Hector Martin
da024ffbd8 memory: Only map available RAM
We saw some crazy speculation running in the HV breaking things by
reading from invalid RAM, so let's actually map only what's available.
For now we do map all lowmem as we haven't seen SErrors there yet, but
we stop at the high boundary.

Fixes: #97

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-10-15 23:44:12 +09:00
Hector Martin
d53b40da48 memory: Add mmu_init_secondary() to init MMU for secondaries
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-09-15 23:24:37 +09:00
Hector Martin
52f432f0f6 memory: Add guard pages at the end of stacks
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-09-15 23:24:37 +09:00
Hector Martin
76dca8c891 memory: Unbork EL0 execution
We need to make m1n1 itself EL0-executable.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-05-22 02:48:17 +09:00
Hector Martin
b9ed00c6f3 memory: Initialize SPRR permissions
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-05-13 18:16:56 +09:00
Hector Martin
8af8dadee1 memory: Support L3 mappings, map m1n1 code as RX.
This replaces the old pagetable code with an adapted version of what
hv_vm.c does, which can handle block and page mappings more
intelligently.

Then, map the m1n1 code section as RX. This allows us to work in modes
where W^X is enforced.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-05-13 18:14:31 +09:00
Hector Martin
3e1ea2d503 memory: Add separate mappings for EL0 data access
This unborks stack and constant pool accesses from el0_call.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-05-13 17:36:02 +09:00
Sven Peter
9120cb8426 memory: allow to reinit and temporarily disable the MMU
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-05-11 15:48:40 +09:00
Hector Martin
4c043a0f97 memory: Move SCTLR/TCR defines to arm_cpu_regs.h
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-05-09 03:25:51 +09:00
Hector Martin
45a0054f63 Add build-time printf() argument checking and fix warnings
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-05-01 18:23:44 +09:00
Hector Martin
d63e84d2f7 fb: add fb_console_reserve_lines()
This is like fb_console_scroll(), but only scrolls if necessary to
ensure that enough free space is available at the end of the console.
This avoids spuriously scrolling during shutdown if we still have space.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-04-15 23:18:57 +09:00
Sven Peter
2873aed75a memory: speed hack: scroll fb console before shutting down the MMU
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-04-14 18:11:37 +09:00
Hector Martin
ce2038c59c memory: use tlbi vmalle1is
This works at both EL1 and EL2(with VHE) and seems to do the right
thing.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 22:39:26 +09:00
Hector Martin
16cff51bd4 exception,memory: s/EL2/EL1/
Since we're in VHE mode, we can pretend to be in EL1 - but this will
allow us to really run in EL1 if we want to in the future.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 20:31:36 +09:00
Hector Martin
8e4b99d75a memory: reformat and shorten MAIR_INDEX to MAIR_IDX
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 19:47:37 +09:00
Hector Martin
73a85be2b8 memory: prepare for EL0 support
Enable EL0 access to MMIO/etc, but not main RAM, because AArch64
architecturally enforces EL0w ^ EL2x.

Instead, create an alias of main RAM to grant EL0 full permissions,
at 0x80_0000_0000.

Grant EL0 full access to MMIO stuff, since EL2 will never execute
from there.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-24 21:48:12 +09:00
Sven Peter
f3eed1c7d7 MMU: correctly map MMIO/PCIe ranges
Previously all MMIO was mapped twice with different attributes
which may or may not lead to strange behaviour when the same
physical range is accessed from both mappings.

We now have a better idea which ranges require nGnRE and nGnRnE
and can just do it correctly instead.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-02-18 16:16:55 +09:00
Sven Peter
24938db17a MMU: use alle2 instead of vmalls12e1is to invalidate TLBs
I can't remember why I used vmalls12e1is but this leads to
the following bug:

  1. Load m1n1 with normal MMU setup
  2. Disable all mappings, recompile and chainload to that m1n1
  3. Everything will work fine for a while even though it should explode
     when enabling the MMU.

This happens becuse there are still stale TLB entries in some cache.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-02-15 01:37:39 +09:00
Hector Martin
96d133e854 mmu: map device memory as non-executable
This fixes the random SErrors after returning from a page fault. Turns
out the M1 was randomly deciding to speculate an instruction fetch from
address 0, triggering a fabric error.

Quoting the ARM ARM:

"Hardware does not prevent speculative instruction fetches from a memory
location with any of the Device memory attributes unless the memory
location is also marked as Execute-never for all Exception levels.

This means that to prevent speculative instruction fetches from memory
locations with Device memory attributes, any location that is assigned
any Device memory type must also be marked as execute-never for all
Exception levels. Failure to mark a memory location with any Device
memory attribute as execute-never for all Exception levels is a
programming error."

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-31 13:42:48 +09:00
Hector Martin
b65e2ca0e4 mmu: also set up high-VA translation
Just an alias

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Sven Peter
9242c820ae MMU: clean and invalidate all caches after shutdown
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-28 23:20:15 +09:00
Sven Peter
f244919c98 MMU: add initial support
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-28 23:20:15 +09:00
Hector Martin
344c84da17 clang-format everything
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 16:06:12 +09:00
Hector Martin
81aaf2ed35 Basic exceptions, irq, cache mgmt support
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-17 00:49:22 +09:00