Commit graph

706 commits

Author SHA1 Message Date
Sven Peter
95b1cb39fc utils.h: add poll32
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-04-11 05:26:29 +09:00
Sven Peter
bf3830ee6d utils.h: add dma_mb,dma_rmb,dma_wmb
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-04-11 05:26:29 +09:00
Sven Peter
8aa3c69a2d tunables: add tunables_apply_local_addr
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-04-08 20:32:12 +09:00
Sven Peter
a17e64f1e2 tunables: add proxy ops
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-04-08 20:32:12 +09:00
Sven Peter
d73276b0bc tunables: add initial version.
There are at least two types in the ADT related to USB,
but there's a decent chance that there are even more
required for other devices:

 * A simple tunable that applies to a whole device node
   and all its MMIO ranges specified in the "reg" property.
   This one seems to just be mask32.

 * A slightly more complex tunable that applies to a single
   MMIO range specified in the "reg" property. So far I've
   only seen 32 bit masks but the format looks like it should
   also support 8,16 and 64 bit masks.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-04-08 20:32:12 +09:00
Hector Martin
35f2b2435e kboot: reserve m1n1 in memory
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-15 02:26:13 +09:00
Hector Martin
efa4452a47 Fix libc header dependencies
Turns out we had a bunch of silly dependencies on libc headers that are
not included with freestanding compilers. Fix all this and change the
CFLAGS to exclude libc headers and only include the built-in compiler
path.

Add our own versions of assert.h, errno.h, limits.h, and move malloc.h
and string.h together into a new path used as -isystem, so these headers
can be included using #include <>.

Remove a bunch of other dependencies in third-party code.

Add a strnlen function.

Disable building the libfdt overlay code for now, as it needs a strtoul
implementation. We can throw that in if/when we decide to use overlays.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-08 01:31:14 +09:00
Hector Martin
ce2038c59c memory: use tlbi vmalle1is
This works at both EL1 and EL2(with VHE) and seems to do the right
thing.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 22:39:26 +09:00
Hector Martin
b8349819fe Implement calling code as EL1
This works by clearing HCR_EL2.TGE, and then doing essentially the same
thunk/return dance as for EL0 calls. However, since most EL1 exceptions
are not routed to EL2, we install hypercall vectors in EL1 to forward
them to EL2, and then short circuit the exception return to whatever
triggered the original exception.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 22:10:37 +09:00
Hector Martin
f669237c2a start.S: clean up some junk
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 21:27:25 +09:00
Hector Martin
16cff51bd4 exception,memory: s/EL2/EL1/
Since we're in VHE mode, we can pretend to be in EL1 - but this will
allow us to really run in EL1 if we want to in the future.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 20:31:36 +09:00
Hector Martin
4282cd0578 exception: size tables properly, work around overeager clang-format
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 19:52:53 +09:00
Hector Martin
d5acb000f2 kboot: add support for KASLR and bootloader randomness
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 19:47:39 +09:00
Hector Martin
8e4b99d75a memory: reformat and shorten MAIR_INDEX to MAIR_IDX
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-03-04 19:47:37 +09:00
Hector Martin
3872b1a7da exception: implement el0_call to run code as EL0
This lets us test register access and other features from EL0.

No serious attempt at security is made, but at least EL0 runs off of a
separate stack and can return to EL2 at any time with `brk`; we can
easily implement a guard mode to break straight to EL2 on exception
later if needed.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-24 21:48:12 +09:00
Hector Martin
73a85be2b8 memory: prepare for EL0 support
Enable EL0 access to MMIO/etc, but not main RAM, because AArch64
architecturally enforces EL0w ^ EL2x.

Instead, create an alias of main RAM to grant EL0 full permissions,
at 0x80_0000_0000.

Grant EL0 full access to MMIO stuff, since EL2 will never execute
from there.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-24 21:48:12 +09:00
Hector Martin
b5cbf7360e exception: handle PMC FIQs
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-18 17:55:55 +09:00
Hector Martin
cf640de6db utils.h: add BIT() macro
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-18 17:55:55 +09:00
Hector Martin
17aca31498 utils.h: move reg_{clr,set,mask} here from chickens.c
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-18 17:55:55 +09:00
Hector Martin
86a7d9c0f3 cpu_regs.h: s/SYS_/SYS_APL_/
This synchronizes the register naming convention with Linux

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-18 17:55:55 +09:00
Sven Peter
f3eed1c7d7 MMU: correctly map MMIO/PCIe ranges
Previously all MMIO was mapped twice with different attributes
which may or may not lead to strange behaviour when the same
physical range is accessed from both mappings.

We now have a better idea which ranges require nGnRE and nGnRnE
and can just do it correctly instead.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-02-18 16:16:55 +09:00
Sven Peter
870fead39c proxyclient: add ops for writeread{8,16,32,64}
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-02-18 16:12:28 +09:00
Sven Peter
1ccf2e2af5 utils.h: add writeread{8,16,32,64}
These functions all perform a store direcly followed by a load.
This is useful to e.g. useful to find busy bits which might
already be cleared a few cycles after a write.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-02-18 16:12:28 +09:00
Sven Peter
24938db17a MMU: use alle2 instead of vmalls12e1is to invalidate TLBs
I can't remember why I used vmalls12e1is but this leads to
the following bug:

  1. Load m1n1 with normal MMU setup
  2. Disable all mappings, recompile and chainload to that m1n1
  3. Everything will work fine for a while even though it should explode
     when enabling the MMU.

This happens becuse there are still stale TLB entries in some cache.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-02-15 01:37:39 +09:00
Hector Martin
d46d51d7ed exception: fix vtimer FIQ masking
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-11 01:35:23 +09:00
Hector Martin
063b9a1d27 exception: Add GUARD_SILENT feature to suppress logs
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-11 00:55:59 +09:00
Hector Martin
eab71ee92b fb, kboot: handle Retina flag correctly
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-06 22:27:45 +09:00
Hector Martin
5dbc82a9ee payload: add support for booting in-line payloads
Usage:

$ cat m1n1.macho Image.gz apple-j274.dtb initramfs.gz \
    > m1n1-payload.macho

That's it.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-06 16:44:10 +09:00
Hector Martin
93bad96311 minilzlib: fix support for non-CRC32 checksum types
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-06 16:44:10 +09:00
Hector Martin
61e2c36ef5 tinf, minzllib: support unknown input sizes
This can be used when the input file size is unknown: the decompression
functions will keep track of it and return it to the caller instead.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-06 16:44:10 +09:00
Hector Martin
0cd328214c exception: handle all 4 available timer FIQs
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-06 16:44:10 +09:00
Hector Martin
5aff13c404 proxy: add missing SPDX header
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-06 16:44:10 +09:00
Hector Martin
5e5627cd0b clang-format: Add include block configs, reformat
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-06 16:44:10 +09:00
Ariel Machado
476182e1f8 utils.h: Fix 'asm-operand-widths' warnings with clang
Signed-off-by: Ariel Machado <ariel@fe.up.pt>
2021-02-06 12:13:29 +09:00
Hector Martin
8691fcee8a kboot: fix fb debug print
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-05 03:49:20 +09:00
Hector Martin
234a511e35 kboot: add spin-table and SMP support
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-05 02:58:08 +09:00
Hector Martin
aaab2c6ca1 chickens: set WFI mode to clockgate only
This preserves CPU registers.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-05 02:58:08 +09:00
Hector Martin
e35ef6eb8c smp: fix IRQs on secondary CPUs
Seems one of the registers is some kind of system-level CPU spinup flag,
without which the CPU runs but the rest of the system thinks it's still
dead.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-05 02:58:08 +09:00
Hector Martin
cb7e7f7918 exception.c: print MPIDR with IRQs
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-02-05 02:58:08 +09:00
Hector Martin
96d133e854 mmu: map device memory as non-executable
This fixes the random SErrors after returning from a page fault. Turns
out the M1 was randomly deciding to speculate an instruction fetch from
address 0, triggering a fabric error.

Quoting the ARM ARM:

"Hardware does not prevent speculative instruction fetches from a memory
location with any of the Device memory attributes unless the memory
location is also marked as Execute-never for all Exception levels.

This means that to prevent speculative instruction fetches from memory
locations with Device memory attributes, any location that is assigned
any Device memory type must also be marked as execute-never for all
Exception levels. Failure to mark a memory location with any Device
memory attribute as execute-never for all Exception levels is a
programming error."

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-31 13:42:48 +09:00
Hector Martin
b65e2ca0e4 mmu: also set up high-VA translation
Just an alias

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
b2ef461114 get rid of another warning pragma
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
919e3fb64a exception: use 64bit print types
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
65c0632ae7 exception: do not reboot on SError
This may or may not go well depending on the problem... but let's see
what happens.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
201d7d9a1e exception: also print FAR_EL2
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
040cb9d6fd exception: enable SError
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
bbe47b2f75 Move Apple ARM register definitions to cpu_regs.h
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
7807dad7bc exception: add barriers around stuff
This CPU really likes to speculate...

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
1f99c17fef exception: clear L2C error status after exception
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
5dd63e41a1 exception: decode exception type on dumps
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
6dde40b05c exception: introduce recovery from synchronous exceptions
Use this to recover from address faults in the proxy. Also print out L2
subsystem fault info.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
d31eab5f87 chickens: do not disable L2 subsystem errors
These mean real problems, let's actually fix them instead of shoving
them under the carpet.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
4167a18a64 utils: move sys_reg and is_ecore() to utils.h
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 22:00:00 +09:00
Hector Martin
2df212bf5c adt: use ADT_FOREACH macros internally
This fixes some bugs too.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 03:15:00 +09:00
Hector Martin
dc2b7b3bb4 startup: clear bss section
iBoot does this for us, but this is for the benefit of chainload.py and
other dump loaders that may not.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 03:03:03 +09:00
Hector Martin
b673db78c3 minilzlib/xzstream.h: get rid of warning pragma
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 03:03:03 +09:00
Hector Martin
74307af0df kboot: add proper kernel boot & DT manipulation code
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 01:37:09 +09:00
Hector Martin
ad93d14ee5 chickens: clear SYS_L2C_ERR_STS
It's unclear why this is necessary, but it fixes some strange crashes

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 01:37:09 +09:00
Hector Martin
c057775ab9 adt: add ADT_FOREACH_PROPERTY
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 01:37:09 +09:00
Hector Martin
891bc5592e utils.h: fix msr/mrs macro expansion
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 01:37:09 +09:00
Hector Martin
46a46da111 string: add memmove, strrchr, strncmp
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 01:37:09 +09:00
Hector Martin
b82c0072e7 utils.h: mark reboot() noreturn
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 01:37:09 +09:00
Hector Martin
d280157289 libfdt: actually add libfdt
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-30 01:37:09 +09:00
Hector Martin
986c6730e9 Add heapblock and dlmalloc for managing memory
heapblock is a simple `sbrk` style implementation, also useful as an
"endless" decompression buffer. dlmalloc is used on top as a malloc
implementation.

This also changes how the Python side manages its heap. We still use a
python-side malloc implementation (since this is faster), and we put the
Python heap at the m1n1 heap + 128MB, without allocating it.
Hopefully this should never step on anything m1n1 neads, and avoids
having to manage freeing across Python script calls.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-29 16:25:15 +09:00
Hector Martin
f3d0a58f42 proxy: add mmu_shutdown call, use it for linux.py
I saw at least one SError crash on Linux after doing this, but can't
repro; unclear if related to the MMU changes or not...

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-29 00:36:46 +09:00
Hector Martin
c3c6e4e8f9 smp: add scaffolding and proxy funcs for cross-CPU calls
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-29 00:36:46 +09:00
Hector Martin
e97b104125 main: do not spin up SMP
This breaks reloads, we shouldn't do this until we need to.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Hector Martin
eae3d06e85 smp: fix totally broken stack setup
This worked by accident before...

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Hector Martin
11753f9983 start.S: probe the stack after init
This makes it obvious when the stack is broken.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Hector Martin
cffc423024 smp: use ADT data instead of hardcoded info
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Hector Martin
b4822eb02c adt: improve APIs for iterating through child nodes
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Hector Martin
db902ce01f wdt: move WDT disable code to wdt.c & use ADT regs
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Hector Martin
3733ea153f smp: fix broken spin-table routine
Only read target_cpu once, then keep a pointer our spin table.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Hector Martin
ba1162e322 adt: add an API for resolving reg properties
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 23:48:35 +09:00
Sven Peter
9242c820ae MMU: clean and invalidate all caches after shutdown
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-28 23:20:15 +09:00
Sven Peter
2f84b33ba8 MMU: add cache operations by set/way
this embeds a slightly modified file taken from
arm-trusted-firmware.

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-28 23:20:15 +09:00
Sven Peter
f244919c98 MMU: add initial support
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-28 23:20:15 +09:00
Sven Peter
63fad7e3e2 utils.h: add panic() macro
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-28 23:20:15 +09:00
Hector Martin
d762391a68 Add code to spin up secondary CPUs
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 16:06:12 +09:00
Hector Martin
65276fdd3f utils.h: add memory clobbers
We mostly use read32/write32 and friends to poke hardware registers,
which might have side effects on RAM, so just make everything a memory
clobber to avoid potential issues.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 16:06:12 +09:00
Hector Martin
0145830772 chickens: add firestorm chicken bits
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 16:06:12 +09:00
Hector Martin
344c84da17 clang-format everything
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-28 16:06:12 +09:00
Ariel Machado
51bf5c870a adt: replaces int to u32 and size_t for unsigned values.
Makefile: increases compiler warnings

Signed-off-by: Ariel Machado <ariel@fe.up.pt>
2021-01-25 19:18:30 +09:00
Hector Martin
cc34ace39a startup: Implement setting CPU chicken bits for Icestorm
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-25 19:11:54 +09:00
Hector Martin
58b643df77 clang-format a few missed things
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-25 17:25:49 +09:00
Hector Martin
a637b77816 Move minlzma.h -> src/minilzlib
This stops clang-format from running on this header.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-25 17:25:44 +09:00
Hector Martin
14db0224af Add config.h for build-time config options
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-25 17:24:29 +09:00
Hector Martin
674a82a2bb aic_test: UART IRQ testing stuff
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-23 22:42:59 +09:00
Hector Martin
63319ed348 utils: implement udelay()
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-23 22:42:25 +09:00
Hector Martin
009f04775e utils: add __assert_fail function to make assert() work
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-23 22:42:25 +09:00
Hector Martin
ee12d053a9 proxy: add XZ and GZ decompression functions and code
This embeds tinf and minlzma.

Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-23 22:42:23 +09:00
Ferdinand Bachmann
afc5813635 exceptions: fix clang warning for msr arg
Signed-off-by: Ferdinand Bachmann <theferdi265@gmail.com>
2021-01-23 22:09:22 +09:00
Ferdinand Bachmann
8511d9ff74 start.S: clang compat, don't use '\r' in asm files
Signed-off-by: Ferdinand Bachmann <theferdi265@gmail.com>
2021-01-23 22:09:22 +09:00
Ferdinand Bachmann
75c3cd39d4 start.S: don't use caller-saved registers accross calls
Signed-off-by: Ferdinand Bachmann <theferdi265@gmail.com>
2021-01-23 22:09:22 +09:00
Ferdinand Bachmann
6ecf5d9fb5 types.h: remove redundant definition of NULL (defined in stddef.h)
Signed-off-by: Ferdinand Bachmann <theferdi265@gmail.com>
2021-01-23 22:05:37 +09:00
Ferdinand Bachmann
274ea2baf2 exceptions: remove unused push_hi_regs symbol, make _exc_entry a function
Signed-off-by: Ferdinand Bachmann <theferdi265@gmail.com>
2021-01-23 22:05:24 +09:00
Hector Martin
31b6af14b2 exception.c: fix sp calculation in print_regs
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-18 01:57:27 +09:00
Hector Martin
81aaf2ed35 Basic exceptions, irq, cache mgmt support
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-17 00:49:22 +09:00
Ferdinand Bachmann
d49d0b781b adt: fix ignored qualifiers in adt_getprop_copy function prototype
Signed-off-by: Ferdinand Bachmann <theferdi265@gmail.com>
2021-01-16 05:12:48 +09:00
Sven Peter
3268aabb91 add missing const to src/adt.h
Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-16 02:48:22 +09:00
Sven Peter
4c4850935f change #include <string.h> to #include "string.h"
this is required to fix the following error when compiling with clang:

  CC    build/string.o
src/string.c:3:10: error: 'string.h' file not found with <angled> include; use "quotes" instead
         ^~~~~~~~~~
         "string.h"

Signed-off-by: Sven Peter <sven@svenpeter.dev>
2021-01-16 02:48:08 +09:00
Clay Freeman
b0b3548f1d Error codes should be returned as negative values
Signed-off-by: Clay Freeman <git@clayfreeman.com>
2021-01-15 12:52:34 +09:00
Hector Martin
5175c7a31b Chainloading, ADT support, misc fixes
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-15 04:04:12 +09:00
Hector Martin
b8fd8363cd Make UART proxy work
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-15 04:04:12 +09:00
Hector Martin
152c96a34a Import UART proxy
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-15 04:04:12 +09:00
Hector Martin
8af9f1eb95 Initial commit
Signed-off-by: Hector Martin <marcan@marcan.st>
2021-01-15 04:03:35 +09:00