Interesting SPI Routing with iCE40 FPGAs · Daniel Mangum

A few weeks ago I
posted
about how much fun I was having with the Fomu FPGA
development board while travelling. This project from Tim ‘mithro’
Ansell and Sean ‘xobs’
Cross is not new, but remains a favorite of mine
because of how portable it is — the entire board can fit in your USB port!

The Fomu includes a Lattice Semiconductor iCE40 UltraPlus
5K, which has
been a popular FPGA option over the past few years due to the reverse
engineered bitstream format and ability to
program it with a fully open source
toolchain (see updated repository
here). One of the more recent
production uses I have heard about is in the Oxide
Computer, which I discussed with Nathanael
Huffman on an episode
of Microarch Club and has also come up on
the Oxide & Friends
podcast.

One of the most interesting attributes of the Fomu is that, despite the fact
that it is programmed over USB, it doesn’t actually include any USB peripheral
hardware. Instead, in order to make the board small enough to fit in the USB
port, the USB core is implemented in RTL (Fomu uses
ValentyUSB) and is included in the
Foboot bootloader. Most FPGAs support
writing bitstreams directly into SRAM, meaning that it will have to be
reprogrammed after a power cycle, or storing a bistream in a separate flash IC
that the FPGA will read from when powered on. iCE40 FPGAs are no different, but
also include a Non-Volatile Configuration Memory (NVCM), which allows for
one-time programming of a bitstream in high-volume applications. From the iCE40
Programming and Configuration Technical
Note:

iCE40 components are configured for a specific application by loading a binary
configuration bitstream image, generated by the Lattice development system.
For high-volume applications, the bitstream image is usually permanently
programmed in the on-chip Non-volatile Configuration Memory. However, the
bitstream image can also be stored externally in a standard, low-cost
commodity SPI serial Flash PROM. The iCE40 component can automatically load
the image using the SPI Master Configuration Interface. Similarly, the iCE40
configuration data can be downloaded from an external processor,
microcontroller, or DSP using an SPI-like serial interface.

Given that the Fomu is primarily intended for prototyping and education,
utilizing the NVCM is unnecessary. However, it does include a
GD25Q16CEIGR
flash chip for persistent bitstream storage. Despite the Fomu hardware also
being open source, I hadn’t spent
much time looking at the
schematic
when developing with it in the past. However, while on the plane ride back from
my travels, I was reading some of the documentation for the device and became
curious about how the device supported both loading bitstreams into SRAM on the
iCE40 and into the external flash.

Other FPGA development boards, such as the Arty
A7 I have
previously written
about, support
configuring the FPGA’s SRAM by including a USB to Serial IC (the Arty A7 uses
the FTDI FT2232HQ) that can be used
to configure the FPGA directly, or configure the FPGA with a bitstream that
allows it to talk to the SPI flash, which is subsequently used to store the user
bitstream in flash. From the Arty A7 reference
manual:

When programming a nonvolatile flash device, a bitstream file is transferred
to the flash in a two-step process. First, the FPGA is programmed with a
circuit that can program flash devices, and then data is transferred to the
flash device via the FPGA circuit (this complexity is hidden from the user by
the Xilinx tools). This is called indirect programming. After the flash device
has been programmed, it can automatically configure the FPGA at a subsequent
power-on or reset event as determined by the mode jumper setting.

However, as previously mentioned, the Fomu’s USB core is implemented in RTL,
which presents a few challenges. Namely, the FPGA needs to be “bootstrapped” —
we cannot configure the FPGA or the SPI flash via USB without the USB core
already running on the FPGA. Fomu uses a bootloader,
Foboot, which includes gateware for a
RISC-V core
(VexRiscv)
and various peripherals, as well as
software
for USB and DFU suppport. If you buy a Fomu, Foboot will already be present on
the device. You can update Foboot using the
Booster
program, but if you somehow perform a faulty update, you could find yourself
with a bricked device that cannot be accesed over USB. In that case, you’ll need
to be able to access the components by some other means. Or, if you’re like me,
you might just be interested in loading alternative gateware and software onto
the device.

The Fomu hardware repository has a
nice image with some of the board’s test points labeled.

The first thing that I noticed was that there was only one “chip select”
(SPI_CS) test point. Both the FPGA and the flash IC can act as SPI
peripherals, and typically a CS line is required from the controller to each
peripheral in a multidrop configuration, or peripherals can be
daisy-chained
if supported. The iCE40 is particularly interesting in this scenario because, as
previously mentioned, it can act as a controller when reading from the flash IC,
or as a peripheral when being configured directly. When supplying power, the
iCE40 will act as a SPI controller by default and attempt to load a bistream
from the flash IC. If we want to program the flash directly via the test points,
then we need to keep the iCE40 from attempting to control it. This is possible
because the CRESET pin is also available on a test point.

As shown in the technical note, driving CRESET low will reset the device, and
configuration will not begin until the next rising edge. At this point it is
clear how we can go about programming the flash IC: hold the iCE40 in reset
while we program the flash using the exposed SPI test points. One plausible
assumption is that only the flash IC can be accessed over SPI via the test
points. In fact, we really don’t need direct access to the iCE40 because if the
flash IC can be programmed, then the iCE40 will load the persistent bitstream
when powered on, and a bricked device could be recovered.

Interestingly, the iCEstick evaluation
board made by Lattice Semiconductor, which uses an FTDI FT2232H in similar
fashion to the Arty A7, takes this to heart. In the user
guide, it states
the following under “Known Issue”.

This board is designed for SPI flash programming only. This would prevent the
iCE device from being accidentally NVCM programmed.

(This is also called out in the help output for
iceprog,
a programming tool included in Project IceStorm.)

As you would expect, this is reflected in the schematic from the same document.
The SPI chip select pins on the iCE40 and the flash IC have a pull-up resistor,
and the iCE_SS_B net allows for pulling the CS pin on the flash IC low for
programming, while the iCE40 SPI_SS_B pin drives it after boot when the FPGA
is acting as the controller.

However, as you may have noticed in the first image in this post, the note on
the Fomu schematic indicates that both the iCE40 and the flash IC can be
programmed via the test points. This complicates things a bit because, up until
now, each component only ever plays a single role:

• The iCE40 FPGA is always a SPI controller.
• The programmer (e.g. the machine connected to the test points) is always a
  SPI controller.
• The flash IC is always a SPI peripheral, either accessed by the programmer or
  the FPGA.

If we are configuring the iCE40 directly, the iCE40 now needs to act as a
peripheral, and the flash IC needs to no longer be driven by traffic on the SPI
nets. Though not labeled in the device image, opening the design in the KiCad
PCB viewer reveals the placement of the SPI_IO2 and SPI_IO3 (as well as the
CDONE, USB_P, and USB_N) test points.

SPI_IO3 is connected to the flash IC HOLD pin. This is in contrast to the
iCEstick, where the flash IC’s HOLD pin has a pull-up resistor (i.e. the
active low pin is never asserted). Because we have access to the HOLD pin on
the Fomu, we can drive it low to have the flash IC ignore all SPI traffic while
configuring the iCE40 directly. We also need to swap the pins used for the MOSI
and MISO signals, as SPI_MOSI is connected to the iCE40’s SPI_SO pin, and
SPI_MISO is connected to SPI_SI. When I looked at
fomu-flash, a utility to program Fomu
boards using a Raspberry Pi, it appeared as though that was exactly what was
happening.

        spiHold(spi);
        spiSwapTxRx(spi);
        fpgaResetSlave(fpga);

However, upon further inspection, spiHold() is not actually asserting the
HOLD pin, but rather issuing a
command.

void spiHold(struct ff_spi *spi) 🔥
void spiUnhold(struct ff_spi *spi) Tell us your thoughts in comments!

In looking up the spiHold() / spiUnhold() commands in the GD25Q16CEIGR
datasheet,
I observed that they correspond to “Deep Power-Down (DP)” (0xb9) and “Release
from Deep Power-Down and Read Device ID (RDI)” (0xab).

Executing the Deep Power-Down (DP) command is the only way to put the device
in the lowest consumption mode (the Deep Power-Down Mode). It can also be used
as an extra software protection mechanism, while the device is not in active
use, since in this mode, the device ignores all Write, Program and Erase
commands… Once the device has entered the Deep Power-Down Mode, all commands
are ignored except the Release from Deep Power-Down and Read Device ID (RDI)
command. These commands can release the device from this mode.

There are a few reasons why using the power down strategy rather than asserting
HOLD may be useful. First, the GD25Q16CEIGR supports Dual and Quad SPI, and in
the latter case the HOLD pin is reassigned as the final I/O pin (IO3). If
the goal is to configure the iCE40 directly, this may not be too much of a
problem, but the Quad Enable (QE) bit is set in a non-volatile status register,
so a previous interaction with the device where it was set could cause the
HOLD strategy to fail (with the Fomu specifically, I suspect that would also
cause issues any time the iCE40 attempted to control the flash IC). Perhaps a
more likely motivator is that it reduces the number of test points that need to
be accessed to configure the iCE40 directly. Only the 7 labeled in the initial
image (5V, GND, CRESET, SPI_MOSI, SPI_MISO, SPI_CLK, SPI_CS) are
necessary.

FPGAs are fascinating on many dimensions, but one of the most interesting is how
they are configured. As we can see with the Fomu and other iCE40 based
platforms, the capabilities of the FPGA can lead to some interesting board
design decisions.