A busy controller shouldn't be game-over for all controllers,
so keep trying on hitting -EBUSY.
This change brings the actual behavior of the routine in line
with what the descriptions says.
Fixes: 982388eaa9 ("nvme: Add NVM Express driver support")
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Moritz Fischer <moritzf@google.com>
Return -EBUSY if controller is found busy rather than -ENOMEM
and update the error message accordingly.
Fixes: 982388eaa9 ("nvme: Add NVM Express driver support")
Reviewed-by: Simon Glass <sjg@chromium.org>
Signed-off-by: Moritz Fischer <moritzf@google.com>
This name is a little confusing since it suggests that it sets up the
sibling block device. In fact it sets up a bootdev for it. Rename the
function to make this clearer.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Mattijs Korpershoek <mkorpershoek@baylibre.com>
When scanning fails it is useful to be able to decode what went wrong. Add
some debugging for this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Enable NVME and PCI NVMe drivers for SPL builds. Also enable PCI_PNP
for SPL which is required to auto configure the PCIe devices.
Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
U-Boot sets up devices ready for use, but coreboot does not. Enable this
so that NVMe works OK from coreboot.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
We need extensions to be set up before we start trying to boot any of the
bootdevs. Add a new priority before all the others for tht sort of thing.
Also add a 'none' option, so that the first one is not 0.
While we are here, comment enum bootdev_prio_t fully and expand the test
for the 'bootdev hunt' command.
Signed-off-by: Simon Glass <sjg@chromium.org>
We currently have an if_type (interface type) and a uclass id. These are
closely related and we don't need to have both.
Drop the if_type values and use the uclass ones instead.
Maintain the existing, subtle, one-way conversion between UCLASS_USB and
UCLASS_MASS_STORAGE for now, and add a comment.
Signed-off-by: Simon Glass <sjg@chromium.org>
Selecting this option can be handled in the Kconfig option itself, as it
is with BLK. Update this an drop the various 'select' clauses.
Signed-off-by: Simon Glass <sjg@chromium.org>
This option is fact really related to SPL. For U-Boot proper we always use
driver model for block devices, so CONFIG_BLK is enabled if block devices
are in use.
It is only for SPL that we have two cases:
- SPL_BLK is enabled, in which case we use driver model and blk-uclass.c
- SPL_BLK is not enabled, in which case (if we need block devices) we must
use blk_legacy.c
Rename the symbol to SPL_LEGACY_BLOCK to make this clear. This is
different enough from BLK and SPL_BLK that there should be no confusion.
Signed-off-by: Simon Glass <sjg@chromium.org>
Enable this option on all boards which support block devices. Drop the
related depencies on BLK since these are not needed anymore.
Disable BLOCK_CACHE on M5253DEMO as this causes a build error.
Signed-off-by: Simon Glass <sjg@chromium.org>
The nvme driver falsely assumed that the last entry on a page
of the prp-list always points to the next page of the prp-list.
This potentially can lead to the illegal creation of pages on
the prp-list with only a single entry. This change now ensures
that splitting the prp-list into multiple pages, behaves now as
required by the NVME-Spec.
Related to this, also the size of the memory allocation is adjusted
accordingly.
Signed-off-by: Alexander Sowarka <alexander.sowarka@aerq.com>
The brute-force controller disable method can end up racing controller
initialization and causing a crash when we shut down Apple ANS2 NVMe
controllers. Do a proper controlled shutdown, which does block until
things are quiesced properly. This is nicer in general for all
controllers.
Signed-off-by: Hector Martin <marcan@marcan.st>
Tested-by: Mark Kettenis <kettenis@openbsd.org> (firefly-rk3399)
The NVMe firmware in the macOS 13 beta blocks or crashes with u-boot's
current minimal RTKit implementation. It does not provide buffers for
the firmware's buffer requests. The ANS2 firmware included in macOS 11
and 12 tolerates this. The firmware included in the first macOS 13 beta
requires buffers for the crashlog and ioreport endpoints to function.
In the case of the NVMe the buffers are physical memory. Access to
physical memory is guarded by what Apple calls SART.
Import m1n1's SART driver (exclusively used for the NVMe controller).
Implement buffer management helpers for RTKit. These are generic since
other devices (none in u-boot so far) require different handling.
Signed-off-by: Janne Grunau <j@jannau.net>
Reviewed-by: Mark Kettenis <kettenis@openbsd.org>
Tested-by: Mark Kettenis <kettenis@openbsd.org>
Add a mask parameter to control the lookup of the PCI region from which
the mapping can be made.
Signed-off-by: Andrew Scull <ascull@google.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Evolve dm_pci_map_bar() to include an offset and length parameter. These
allow a portion of the memory to be mapped and range checks to be
applied.
Passing both the offset and length as zero results in the previous
behaviour and this is used to migrate the previous callers.
Signed-off-by: Andrew Scull <ascull@google.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Every time a nvme bus/port is scanned and a new device is detected,
we want to call device_probe() as it will give us a chance to run
additional post-processings for some purposes.
In particular, support for creating partitions on a device will be added.
Signed-off-by: AKASHI Takahiro <takahiro.akashi@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Add a driver for the NVMe storage controller integrated on
Apple SoCs. This NVMe controller isn't PCI based and deviates
from the NVMe standard in its implementation of the command
submission queue and the integration of an NVMMU that needs
to be managed. This commit tweaks the core NVMe code to
support the linear command submission queue implemented by
this controller. But setting up the submission queue and
managing the NVMMU controller is handled by implementing
the driver ops that were added in an earlier commit.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
Tested-on: firefly-rk3399
Tested-by: Mark Kettenis <kettenis@openbsd.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
Add a function to disable the NVMe controller. This will be used
to let the driver for the NVMe storage integrated on Apple SoCs
shutdown the NVMe controller such we can shutdown the NVMe
IOP controller in a clean way afterwards before handing control
to the OS.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
The NVMe storage controller integrated on Apple SoCs deviates
from the NVMe standard in two aspects. It uses a "linear"
submission queue and it integrates an NVMMU that needs to be
programmed for each NVMe command. Introduce driver ops such
that we can set up the linear submission queue and program the
NVMMU in the driver for this strange beast.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
Apple SoCs have an integrated NVMe controller that isn't connected
over a PCIe bus. In preparation for adding support for this NVMe
controller, split out the PCI support into its own file. This file
is selected through a new CONFIG_NVME_PCI Kconfig option, so do
a wholesale replacement of CONFIG_NVME with CONFIG_NVME_PCI.
Signed-off-by: Mark Kettenis <kettenis@openbsd.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
Tested on: Macbook Air M1
Tested-by: Simon Glass <sjg@chromium.org>
Calling 'nvme scan' followed by 'nvme detail' crashes U-Boot on Turris
Omnia with the following error:
undefined instruction
pc : [<0a000000>] lr : [<7ff80bfc>]
reloc pc : [<8a8c0000>] lr : [<00840bfc>]
sp : 7fb2b908 ip : 0000002a fp : 02000000
r10: 04000000 r9 : 7fb2fed0 r8 : e1000000
r7 : 0c000000 r6 : 03000000 r5 : 06000000 r4 : 01000000
r3 : 7fb30928 r2 : 7fb30928 r1 : 00000000 r0 : 00000000
Flags: nZCv IRQs off FIQs off Mode SVC_32
Code: 0f0fb4f0 0f0fb4f0 0f0fb4f0 0f0fb4f0 (f0f04b0f)
Resetting CPU ...
This happens when nvme_print_info() tries to return to the caller. It
looks like this error is caused by trying to allocate 8 KiB of memory
on the stack by the two uses of ALLOC_CACHE_ALIGN_BUFFER().
Use malloc_cache_aligned() to allocate this memory dynamically instead.
This fixes 'nvme detail' on Turris Omnia.
Note that similar change was applied to file drivers/nvme/nvme.c in past by
commit 2f83481dff ("nvme: use page-aligned buffer for identify command").
Signed-off-by: Pali Rohár <pali@kernel.org>
Signed-off-by: Marek Behún <marek.behun@nic.cz>
The current code invalidates the range after the read buffer since the
buffer pointer gets incremented in the read loop. Use a temporary
pointer to make sure we have a pristine pointer to invalidate the
correct memory range after read.
Fixes: 704e040a51 ("nvme: Apply cache operations on the DMA buffers")
Reviewed-by: Andre Przywara <andre.przywara@arm.com>
Signed-off-by: Stefan Agner <stefan@agner.ch>
Define LOG_CATEGORY for all uclass to allow filtering with
log command.
Signed-off-by: Patrick Delaunay <patrick.delaunay@foss.st.com>
Reviewed-by: Simon Glass <sjg@chromium.org>
A udevice's priv space is cleared in alloc_priv() in the DM core.
Don't do it again in its probe() routine.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
At present there is an offset of one added during the creation of
block device. This can be very confusing as we wanted to encode the
namespace id in the block device name but namespae id cannot be zero.
This changes to use the namespace id directly in the block device
name, eliminating the offset of one effectively.
Suggested-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
At present for each namespace there is a block device created for it.
There is no issue if the number of supported namespaces reported from
the NVMe device is only 1.
Since QEMU commit 7f0f1acedf15 ("hw/block/nvme: support multiple namespaces"),
the number of supported namespaces reported has been changed from 1
to 256, but not all of them are active namespaces. The actual active
one depends on the QEMU command line parameters. A common case is
that namespace 1 being active and all other 255 being inactive.
If a namespace is inactive, the namespace identify command returns a
zero filled data structure. We can use field NSZE (namespace size) to
decide whether a block device should be created for it.
Reported-by: Heinrich Schuchardt <xypron.glpk@gmx.de>
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
At present the block device creation happens in the NVMe uclass
driver post_probe() phase. In preparation to support multiple
namespaces, we should issue namespace identify before creating
block devices but that touches the underlying hardware hence it
is not appropriate to do such in the uclass driver post_probe().
Let's move it to driver probe() phase instead.
Signed-off-by: Bin Meng <bmeng.cn@gmail.com>
AQA (Admin Queue Attributes) register is a dword size with
lower word of ASQS, and higher word of ACQS.
The code set the variable aqa twice, but it is redundant.
Signed-off-by: Wesley Sheng <wesleyshenggit@sina.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Each prp is 8 bytes, calculate the number of prps
per page should just divide page size by 8
there is no need to minus 1
Signed-off-by: Wesley Sheng <wesleyshenggit@sina.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
writel() and co. already include the endian swap; doing the swap twice
is, er, unhelpful.
Tested on a P4080DS, which boots perfectly fine off NVMe with this.
Signed-off-by: David Lamparter <equinox@diac24.net>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At the moment the nvme_get_features() and nvme_set_features() functions
carry a (somewhat misleading) comment about missing cache maintenance.
As it turns out, nvme_get_features() has no caller at all in the tree,
and nvme_set_features' only user doesn't use a DMA buffer.
Mention that in the comment, and leave some breadcrumbs for the future,
should those functions attract more users.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Michael Trimarchi <michael@amarulasolutions.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At the moment nvme_read_completion_status() tries to invalidate a single
member of the cqes[] array, which is shady as just a single entry is
not cache line aligned.
The structure is dictated by hardware, and with 16 bytes is smaller than
any cache line we usually deal with. Also multiple entries need to be
consecutive in memory, so we can't pad them to cover a whole cache line.
As a consequence we can only always invalidate all of them - U-Boot just
uses two of them anyway. This is fine, as they are only ever read by the
CPU (apart from the initial zeroing), so they can't become dirty.
Make this obvious by always invalidating the whole array, regardless of
the entry number we are about to read.
Also blow up the allocation size to cover whole cache lines, to avoid
other heap allocations to sneak in.
Signed-off-by: Andre Przywara <andre.przywara@arm.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Reviewed-by: Michael Trimarchi <michael@amarulasolutions.com>
Tested-by: Neil Armstrong <narmstrong@baylibre.com>
There might be hardware configurations where 64-bit data accesses
to NVMe registers are not supported properly. This patch removes
the readq/writeq so always two 32-bit accesses are used to read/write
64-bit NVMe registers, similarly as it is done in Linux kernel.
This patch fixes operation of NVMe devices on RPi4 Broadcom BCM2711 SoC
based board, where the PCIe Root Complex, which is attached to the
system through the SCB bridge.
Even though the architecture is 64-bit the PCIe BAR is 32-bit and likely
the 64-bit wide register accesses initiated by the CPU are not properly
translated to a sequence of 32-bit PCIe accesses.
nvme_readq(), for example, always returns same value in upper and lower
32-bits, e.g. 0x3c033fff3c033fff which lead to NVMe devices to fail
probing.
This fix is analogous to commit 8e2ab05000 ("usb: xhci: Use only
32-bit accesses in xhci_writeq/xhci_readq").
Cc: Sylwester Nawrocki <s.nawrocki@samsung.com>
Cc: Nicolas Saenz Julienne <nsaenzjulienne@suse.de>
Cc: Matthias Brugger <mbrugger@suse.com>
Reviewed-by: Stefan Roese <sr@denx.de>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Stefan Agner <stefan@agner.ch>
We use 'priv' for private data but often use 'platdata' for platform data.
We can't really use 'pdata' since that is ambiguous (it could mean private
or platform data).
Rename some of the latter variables to end with 'plat' for consistency.
Signed-off-by: Simon Glass <sjg@chromium.org>
This construct is quite long-winded. In earlier days it made some sense
since auto-allocation was a strange concept. But with driver model now
used pretty universally, we can shorten this to 'auto'. This reduces
verbosity and makes it easier to read.
Coincidentally it also ensures that every declaration is on one line,
thus making dtoc's job easier.
Signed-off-by: Simon Glass <sjg@chromium.org>
This patch try to avoids eviction of dirty lines during DMA
transfer. The code right now execute the following step:
- allocate the buffer
- start a dma operation using the non-coherent dma buffer
- invalidate cache lines associated with the buffer
- read the buffer
This can lead to reading back not valid information, because the cache
controller could evict dirty cache lines belonging to the buffer *after*
the DMA operation has started to fill the DRAM.
In order to avoid this, a new invalidation is required *before* starting
the DMA operation. The patch just adds an invalidation before submitting
the DMA command.
Example below shows the nvme disk scan result without the following
patch
=> nvme scan
nvme_get_info_from_identify: nn = 544502629, vwc = 100,
sn = dev_0T, mn = `�\�, fr = t_part, mdts = 105
So, invalidating the cache before submitting the admin command,
fix the cpu read.
Cc: André Przywara <andre.przywara@arm.com>
Reported-by: Suniel Mahesh <sunil@amarulasolutions.com>
Signed-off-by: Michael Trimarchi <michael@amarulasolutions.com>
Signed-off-by: Jagan Teki <jagan@amarulasolutions.com>
Tested-by: Suniel Mahesh <sunil@amarulasolutions.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
At present dm/device.h includes the linux-compatible features. This
requires including linux/compat.h which in turn includes a lot of headers.
One of these is malloc.h which we thus end up including in every file in
U-Boot. Apart from the inefficiency of this, it is problematic for sandbox
which needs to use the system malloc() in some files.
Move the compatibility features into a separate header file.
Signed-off-by: Simon Glass <sjg@chromium.org>
These functions are CPU-related and do not use driver model. Move them to
cpu_func.h
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
This function belongs in time.h so move it over and add a comment.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Change the stack-allocated buffer for the identification command
to explicitly allocate page-aligned buffers. Even though the spec
seems to allow having admin queue commands on non page-aligned
buffers, it seems to not be possible on my i.MX8MQ board with a
a Silicon Power P34A80. Since all of the NVMe drivers I have seen
always do admin commands on a page-aligned buffer, which does work
on my system, it makes sense for us to do that as well.
Signed-off-by: Patrick Wildt <patrick@blueri.se>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
It's possible that the data cache for the buffer still holds data
to be flushed to memory, since the buffer was probably used as stack
before. Thus we need to make sure to flush it also on reads, since
it's possible that the cache is automatically flused to memory after
the NVMe DMA transfer happened, thus overwriting the NVMe transfer's
data. Also add a missing dcache flush for the prp list.
Signed-off-by: Patrick Wildt <patrick@blueri.se>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
