u-boot/drivers/pci
Maciej W. Rozycki a398a51ccc pci: Work around PCIe link training failures
Attempt to handle cases with a downstream port of a PCIe switch where
link training never completes and the link continues switching between
speeds indefinitely with the data link layer never reaching the active
state.

It has been observed with a downstream port of the ASMedia ASM2824 Gen 3
switch wired to the upstream port of the Pericom PI7C9X2G304 Gen 2
switch, using a Delock Riser Card PCI Express x1 > 2 x PCIe x1 device,
P/N 41433, wired to a SiFive HiFive Unmatched board.  In this setup the
switches are supposed to negotiate the link speed of preferably 5.0GT/s,
falling back to 2.5GT/s.

However the link continues oscillating between the two speeds, at the
rate of 34-35 times per second, with link training reported repeatedly
active ~84% of the time, e.g.:

02:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM2824 PCIe Gen3 Packet Switch [1b21:2824] (rev 01) (prog-if 00 [Normal decode])
[...]
	Bus: primary=02, secondary=05, subordinate=05, sec-latency=0
[...]
	Capabilities: [80] Express (v2) Downstream Port (Slot+), MSI 00
[...]
		LnkSta:	Speed 5GT/s (downgraded), Width x1 (ok)
			TrErr- Train+ SlotClk+ DLActive- BWMgmt+ ABWMgmt-
[...]
		LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis+, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
[...]

Forcibly limiting the target link speed to 2.5GT/s with the upstream
ASM2824 device makes the two switches communicate correctly however:

02:03.0 PCI bridge [0604]: ASMedia Technology Inc. ASM2824 PCIe Gen3 Packet Switch [1b21:2824] (rev 01) (prog-if 00 [Normal decode])
[...]
	Bus: primary=02, secondary=05, subordinate=09, sec-latency=0
[...]
	Capabilities: [80] Express (v2) Downstream Port (Slot+), MSI 00
[...]
		LnkSta:	Speed 2.5GT/s (downgraded), Width x1 (ok)
			TrErr- Train- SlotClk+ DLActive+ BWMgmt- ABWMgmt-
[...]
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis+, Selectable De-emphasis: -3.5dB
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
[...]

and then:

05:00.0 PCI bridge [0604]: Pericom Semiconductor PI7C9X2G304 EL/SL PCIe2 3-Port/4-Lane Packet Switch [12d8:2304] (rev 05) (prog-if 00 [Normal decode])
[...]
	Bus: primary=05, secondary=06, subordinate=09, sec-latency=0
[...]
	Capabilities: [c0] Express (v2) Upstream Port, MSI 00
[...]
		LnkSta:	Speed 2.5GT/s (downgraded), Width x1 (downgraded)
			TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
[...]
		LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance De-emphasis: -6dB
[...]

Make use of this observation then and attempt to detect the inability to
negotiate the link speed automatically, and then handle it by hand.  Use
the Data Link Layer Link Active status flag as the primary indicator of
successful link speed negotiation, but given that the flag is optional
by hardware to implement (the ASM2824 does have it though), resort to
checking for the mandatory Link Bandwidth Management Status flag showing
that the link speed or width has been changed in an attempt to correct
unreliable link operation (the ASM2824 does set it too).

If these checks indicate that link may not operate correctly, then poll
the Data Link Layer Link Active status flag along with the Link Training
flag for the duration of 200ms to see if the link has stabilised, that
is either that the Data Link Layer Link Active status flag has been set
or that Link Training has been inactive during at least the second half
of the interval.

If that has indicated failure, restrict the target speed to 2.5GT/s,
request a link retrain and check again if the link has stabilised.  If
that does not work either, then restore the original speed setting and
claim defeat, otherwise we are done.

NB interestingly enough with the ASM2824 vs PI7C9X2G304 configuration
referred above asking the ASM2824 to retrain with a higher target link
speed once the 2.5GT/s speed has been negotiated makes the two devices
successfully negotiate 5.0GT/s.  Lifting the 2.5GT/s speed restriction
would however prevent our workaround from working with an OS that issues
a reset and that is unaware of the problem.  This is because the devices
would then try to negotiate a higher link speed from scratch and fail,
while the sticky property of the Target Link Speed setting will keep the
2.5GT/s speed restriction across a reset.

Keep the 2.5GT/s speed restriction then, conservatively, if functional
once applied.

Signed-off-by: Maciej W. Rozycki <macro@orcam.me.uk>
Reviewed-by: Stefan Roese <sr@denx.de>
2022-01-14 12:26:42 -05:00
..
Kconfig arm: mvebu: a38x: serdes: Move non-serdes PCIe code to pci_mvebu.c 2022-01-14 07:47:57 +01:00
Makefile pci: Remove unused FSL_PCI_INIT code 2021-12-27 08:41:38 -05:00
pci-aardvark.c Prepare v2022.01-rc4 2021-12-20 17:12:04 -05:00
pci-emul-uclass.c dm: Use access methods for dev/uclass private data 2021-01-05 12:24:40 -07:00
pci-rcar-gen2.c dm: treewide: Rename ofdata_to_platdata() to of_to_plat() 2020-12-13 16:51:09 -07:00
pci-rcar-gen3.c pci: renesas: Fix BAR mapping on Gen3 2021-02-20 22:38:28 +01:00
pci-uclass.c pci: Skip configuring invalid P2P bridge devices 2021-10-14 19:45:07 -04:00
pci_auto.c pci: Work around PCIe link training failures 2022-01-14 12:26:42 -05:00
pci_auto_common.c pci: Fix printf format for regions 2021-10-14 19:45:07 -04:00
pci_common.c pci: Drop DM_PCI check from pci_common 2021-08-05 16:14:36 -04:00
pci_compat.c common: Drop log.h from common header 2020-05-18 21:19:18 -04:00
pci_gt64120.c pci: gt64120: Use PCI_CONF1_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pci_internal.h SPDX: Convert all of our single license tags to Linux Kernel style 2018-05-07 09:34:12 -04:00
pci_mpc85xx.c pci: mpc85xx: Use PCI_CONF1_EXT_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pci_msc01.c pci: msc01: Use PCI_CONF1_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pci_mvebu.c arm: mvebu: a38x: serdes: Move non-serdes PCIe code to pci_mvebu.c 2022-01-14 07:47:57 +01:00
pci_octeontx.c pci: pci_octeontx: Use PCIE_ECAM_OFFSET() macro 2022-01-12 14:21:24 -05:00
pci_rom.c video: Fix video on coreboot with the copy buffer 2021-03-27 15:04:30 +13:00
pci_sandbox.c dm: treewide: Rename ..._platdata variables to just ..._plat 2020-12-13 16:51:09 -07:00
pci_sh4.c common: Drop linux/delay.h from common header 2020-05-18 21:19:23 -04:00
pci_sh7751.c pci: sh7751: Fix access to config space via PCI_CONF1_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pci_sh7780.c pci: sh7780: Use PCI_CONF1_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pci_tegra.c pci: tegra: Use PCI_CONF1_EXT_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pci_x86.c dm: pci: Update the PCI read_config() method to const dev * 2020-02-05 19:33:45 -07:00
pcie_brcmstb.c pci: pcie-brcmstb: Use PCIE_ECAM_OFFSET() macro 2022-01-12 14:21:24 -05:00
pcie_dw_common.c drivers: pci: pcie_dw_common: fix Werror compilation error 2021-05-31 16:35:55 +08:00
pcie_dw_common.h pci: add common Designware PCIe functions 2021-04-15 10:43:17 +08:00
pcie_dw_meson.c pci: pcie_dw_meson: fix usb fail when pci link fails to go up 2021-10-07 10:14:50 +02:00
pcie_dw_mvebu.c pcie: designware: mvebu: do not configure ATU for IO when not used 2021-05-16 06:48:45 +02:00
pcie_dw_rockchip.c pci: pcie_dw_rockchip: Replace msleep occurences by udelay 2021-06-18 14:36:54 +08:00
pcie_dw_sifive.c drivers: pci: add pcie support for fu740 2021-05-31 16:35:54 +08:00
pcie_dw_ti.c pci: pcie_dw_ti: migrate to common Designware PCIe functions 2021-04-15 10:43:17 +08:00
pcie_ecam_generic.c RFC: arm: pci: Add PCI cam support to PCI-E ecam driver 2021-11-17 17:09:47 -05:00
pcie_ecam_synquacer.c pci: Add standard PCIe ECAM macros 2021-11-17 17:04:58 -05:00
pcie_fsl.c pci: fsl: Use PCI_CONF1_EXT_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pcie_fsl.h dm: pci: fsl: Correct the workaround of erratum A-007815 2020-10-23 16:52:09 +05:30
pcie_fsl_fixup.c treewide: convert bd_t to struct bd_info by coccinelle 2020-07-17 09:30:13 -04:00
pcie_imx.c pci: imx: Drop use of DM_PCI 2021-08-05 19:46:35 -04:00
pcie_intel_fpga.c common: Drop asm/global_data.h from common header 2021-02-02 15:33:42 -05:00
pcie_iproc.c pci: pcie_iproc: Use PCIE_ECAM_OFFSET() macro 2022-01-12 14:21:24 -05:00
pcie_layerscape.c common: Drop asm/global_data.h from common header 2021-02-02 15:33:42 -05:00
pcie_layerscape.h pci: layerscape: Remove the shadow SVR definitions 2021-02-08 14:01:18 +05:30
pcie_layerscape_ep.c pci: layerscape-ep: Add check of the PCIe controller enablement 2021-06-17 11:46:11 +05:30
pcie_layerscape_fixup.c fdt_support: Remove FDT_STATUS_FAIL_ERROR_CODE 2021-12-19 09:50:47 +01:00
pcie_layerscape_fixup_common.c pci: layerscape: Fix the LUT and msi-map mismatch issue 2021-11-09 17:18:23 +05:30
pcie_layerscape_fixup_common.h treewide: convert bd_t to struct bd_info by coccinelle 2020-07-17 09:30:13 -04:00
pcie_layerscape_gen4.c pci: layerscape: Fix the LUT and msi-map mismatch issue 2021-11-09 17:18:23 +05:30
pcie_layerscape_gen4.h pci: Drop dm.h inclusion from header file 2020-08-03 22:19:54 -04:00
pcie_layerscape_gen4_fixup.c fdt_support: Remove FDT_STATUS_FAIL_ERROR_CODE 2021-12-19 09:50:47 +01:00
pcie_layerscape_rc.c pci: layerscape: Fix the LUT and msi-map mismatch issue 2021-11-09 17:18:23 +05:30
pcie_mediatek.c pci: mediatek: Use PCI_CONF1_EXT_ADDRESS() macro 2022-01-12 14:21:24 -05:00
pcie_octeon.c mips: octeon: Add Octeon PCIe host controller driver 2021-04-28 10:05:12 +02:00
pcie_phytium.c pci: Add standard PCIe ECAM macros 2021-11-17 17:04:58 -05:00
pcie_rockchip.c pci: Add standard PCIe ECAM macros 2021-11-17 17:04:58 -05:00
pcie_uniphier.c pci: uniphier: Add UniPhier PCIe controller driver 2021-07-14 16:48:08 -04:00
pcie_xilinx.c pci: Add standard PCIe ECAM macros 2021-11-17 17:04:58 -05:00