Title here
Summary here
Detailed Description
Hardware Overview
The figure below illustrates the various hardware components that are located on the top-side (component side) of the 2x QSFP28 FMC.
The main components on the top-side of the mezzanine card are:
- 2x QSFP28 cages
- High Pin Count FMC Connector
- 2K EEPROM
- Jitter-attenuating clock multiplier ( Si5328 )
- 114.285MHz crystal
- I2C Level translators
- I/O Level translators
- 3.3VDC switching buck regulator
The figure below illustrates the various hardware components that are located on the bottom-side of the mezzanine card.
The main components on the bottom-side of the mezzanine card are:
- Bicolor user LEDs
- FMC Power indicator LED
- Test points for 12VDC and 3.3VDC power supplies
- Test points for QSFP modules
- Legend for QSFP testpoints
- Decoupling capacitors for QSFP modules
QSFP28 Cages
The QSFP28 cages can accommodate 2x QSFP, QSFP+ or QSFP28 modules. Each QSFP28 module provides four lanes of up to 25 Gbps each, supporting an aggregate data rate of up to 100 Gbps per port.
Jitter-attenuating Clock Multiplier
The 2x QSFP28 FMC features a jitter-attenuating clock multiplier ( Skyworks, Si5328 ), which generates two precision clocks with selectable frequencies ranging from 8kHz to 808MHz. Its wide frequency range and exceptional jitter performance support a variety of applications, including Synchronous Ethernet.
The Si5328 utilizes a 114.285MHz crystal and oscillator circuit to generate frequencies between 8kHz and 808MHz. The clock multiplication ratio can be programmed through an I2C interface. The device also has a clock input, connected to FPGA I/O pins (LA00_CC_P/N), which can receive a recovered clock from the FPGA gigabit transceivers. In Synchronous Ethernet applications, jitter attenuation can be applied to the recovered clock, which can then be directed to the clock outputs to drive the gigabit transceivers.
See the Clocks section for more information on the clock system.
EEPROM
The 2K EEPROM stores IPMI FRU data that can be read by the carrier board and contains the following information:
- Manufacturer name (Opsero Electronic Design Inc.)
- Product name
- Product part number
- Serial number
- Power supply requirements
The FRU data is read by some carrier boards to determine the correct VADJ voltage to apply to the mezzanine card. All Opsero FMC products have their EEPROMs programmed with valid FRU data to allow these carrier boards to correctly power them.
Erasing or writing over the contents of the EEPROM can corrupt the IPMI FRU
data making the mezzanine card unusable with carrier boards that require the
information. We recommend that you do not use the mezzanine card’s EEPROM for
non-volatile storage but instead use the storage options provided by the
carrier board. If you mistakenly erase or corrupt the contents of the EEPROM,
you can reprogram it using the Opsero FMC EEPROM Tool. For more information, see:
FMC EEPROM tool.
High Pin Count FMC Connector
The 2x QSFP28 FMC has a high pin count FMC (FPGA Mezzanine Card) connector for interfacing with an FPGA or SoC development board. The part number of this connector is Samtec, High pin count FMC connector, Module side, ASP-134488-01 . The pinout of this connector conforms to the VITA 57.1 FPGA Mezzanine Card Standard. For the pinout details, see the Pin configuration section. For more information on the FMC connector and the VITA 57.1 standard, see the Samtec page on VITA 57.1 .
I/O Interfaces
The FMC connector provides power to the 2x QSFP28 FMC and also presents the following I/O signals to the FPGA fabric of the development board:
- Gigabit serial links for the 2x QSFP28 slots (4 lanes per port, 8 lanes total)
- QSFP I/O signals (MODPRS_L, RESET_L, LPMODE, INT_L, MODSEL_L) for the 2x QSFP28 slots
- I2C for IPMI EEPROM
- I2C for each of the QSFP28 slots
- I2C for the clock multiplier
- LVDS recovered clock from the FPGA to drive the clock multiplier
- LVDS configurable clock from the clock multiplier
- Clock loss alarm from the clock multiplier
- Drive signals for the 2x bicolor user LEDs
- Reset signal for the I2C switch
The figure below illustrates the connections to the FMC connector.
Details on the I2C connections can be found in the I2C Buses section.
The level translators have been left out of the above diagram for clarity. Details can be found in the Level translation section.
Level translation
To support a wide range of I/O voltages (VADJ), the 2x QSFP28 FMC uses level translators for the QSFP I/O signals, the I2C bus signals, the LED drive signals and the clock loss alarm signal. The table below lists the devices used:
| Device | Purpose |
|---|---|
| TCA9416 | Level translation of the PL I2C bus. |
| SN74AVC4T245RSVR | Level translation of QSFP I/Os, LED drive signals and clock loss alarm. |
The gigabit serial links of the QSFP28 slots and the reference clocks connect to the gigabit transceivers, which are independent of the VADJ voltage and do not need voltage translation. The recovered clock signal (REC_CLK1_P/N) should be configured as an LVDS output in the FPGA and also does not require voltage translation.
Gigabit transceivers
The data channels between the QSFP28 modules and the FPGA operate over serial gigabit links at speeds of up to 25 Gbps per lane. Each QSFP28 module has four lanes, providing an aggregate bandwidth of up to 100 Gbps per port. The 2x QSFP28 FMC connects these serial links to gigabit transceivers in the FPGA or SoC on the development board. Each serial link consists of two differential pairs: one for transmission and one for reception. Port 0 connects to the first four gigabit transceivers on the FMC connector (DP0-3), and Port 1 connects to the next four gigabit transceivers (DP4-7).
| QSFP28 Port | Lane | Signal direction | FMC gigabit transceiver |
|---|---|---|---|
| 0 | 0 | FPGA to link partner | DP0_C2M_P/N |
| Link partner to FPGA | DP0_M2C_P/N | ||
| 1 | FPGA to link partner | DP1_C2M_P/N | |
| Link partner to FPGA | DP1_M2C_P/N | ||
| 2 | FPGA to link partner | DP2_C2M_P/N | |
| Link partner to FPGA | DP2_M2C_P/N | ||
| 3 | FPGA to link partner | DP3_C2M_P/N | |
| Link partner to FPGA | DP3_M2C_P/N | ||
| 1 | 0 | FPGA to link partner | DP4_C2M_P/N |
| Link partner to FPGA | DP4_M2C_P/N | ||
| 1 | FPGA to link partner | DP5_C2M_P/N | |
| Link partner to FPGA | DP5_M2C_P/N | ||
| 2 | FPGA to link partner | DP6_C2M_P/N | |
| Link partner to FPGA | DP6_M2C_P/N | ||
| 3 | FPGA to link partner | DP7_C2M_P/N | |
| Link partner to FPGA | DP7_M2C_P/N |
As the serial links connect to gigabit transceivers, they are independent of the VADJ voltage being used and do not need voltage translation.
QSFP I/O Signals
In addition to the high-speed serial links, QSFP28 modules have several management I/O signals used for configuration, status indication and fault management. On the 2x QSFP28 FMC, these I/O signals are connected through level translators to the FPGA I/O, allowing the FPGA to control and read them. The I/O signals and their functionality are listed in the table below:
| QSFP pin | Name | Direction | Function |
|---|---|---|---|
| 27 | MODPRS_L | QSFP to FPGA | Indicates module presence (active low) |
| 28 | INT_L | QSFP to FPGA | Indicates module interrupt (active low) |
| 8 | MODSEL_L | FPGA to QSFP | Selects the module for I2C communication (active low) |
| 9 | RESET_L | FPGA to QSFP | Resets the module (active low) |
| 31 | LPMODE | FPGA to QSFP | Sets the module to low power mode |
The QSFP I/O signals connect to the FMC pins listed in the table below:
| Port | Net Name | FMC pin |
|---|---|---|
| 0 | QSFP0_MODPRS_L_T | LA12_P |
| QSFP0_INT_L_T | LA12_N | |
| QSFP0_MODSEL_L_T | LA04_P | |
| QSFP0_RESET_L_T | LA04_N | |
| QSFP0_LPMODE_T | LA11_P | |
| 1 | QSFP1_MODPRS_L_T | LA05_P |
| QSFP1_INT_L_T | LA05_N | |
| QSFP1_MODSEL_L_T | LA15_P | |
| QSFP1_RESET_L_T | LA15_N | |
| QSFP1_LPMODE_T | LA11_N |
I2C Buses
The 2x QSFP28 FMC has four independent I2C buses: the FMC’s dedicated I2C bus for the IPMI EEPROM and three PL (programmable logic) I2C buses that connect to the 2x QSFP modules and the clock multiplier.
EEPROM I2C
A 2K EEPROM is located on the FMC card’s dedicated I2C bus. The FMC pins of the I2C bus are shown below, and it is up to the user to determine their corresponding connections to the FPGA/MPSoC on the carrier board being used.
| I2C bus signal | FMC pin name | FMC pin number |
|---|---|---|
| SCL (clock) | SCL | C30 |
| SDA (data) | SDA | C31 |
Be aware that on some carrier boards, the FMC I2C bus passes through an I2C MUX. On some boards it connects to
FPGA pins whereas on others it connects to PS pins. If you wish to communicate with the EEPROM or I/O expander, it is necessary
to check the schematic drawing of your carrier board to determine the structure of the I2C bus and to which
pins it connects.
CLK I2C
The clock multiplier I2C bus of the 2x QSFP28 FMC is implemented using two FPGA I/O pins (LA02_P/N) and enables communication between the FPGA and the clock multiplier. The I2C bus signals are connected to the FMC pins listed in the table below:
| Net Name | Description | FMC pin |
|---|---|---|
| CLK_I2C_SCL_T | I2C clock (SCL) | LA02_P |
| CLK_I2C_SDA_T | I2C data (SDA) | LA02_N |
The CLK I2C bus signals pass through a level translator to convert the FPGA I/O levels (VADJ) to 3.3VDC levels.
QSFP0/1 I2C
The 2x QSFP28 FMC has two independent I2C busses to enable communication between the FPGA and the QSFP modules. The I2C bus signals are connected to the FMC pins listed in the table below:
| Port | Net Name | Description | FMC pin |
|---|---|---|---|
| QSFP0 | QSFP0_I2C_SCL_T | I2C clock (SCL) | LA03_P |
| QSFP0_I2C_SDA_T | I2C data (SDA) | LA03_N | |
| QSFP1 | QSFP1_I2C_SCL_T | I2C clock (SCL) | LA17_CC_P |
| QSFP1_I2C_SDA_T | I2C data (SDA) | LA17_CC_N |
The QSFP0/1 I2C bus signals pass through a level translator to convert the FPGA I/O levels (VADJ) to 3.3VDC levels.
Clock signals
Refer to the Clocks section for more information about the clock related signals and how they connect to the jitter-attenuating clock multiplier.
Bicolor User LEDs
The 2x QSFP28 FMC features two bicolor (green/red) LEDs, one for each QSFP28 port, which can be driven by the FPGA and are visible on the bottom side of the mezzanine card. These LEDs provide the user or developer with programmable visible outputs that can be linked to specific signals for monitoring. Examples of such signals for monitoring include the QSFP I/Os (MODPRS_L, INT_L, RESET_L, LPMODE) or other QSFP module-specific indicators.
The drive pins for the user LEDs are routed through level translators to convert the FPGA I/O signal levels (VADJ) to 3.3VDC levels for driving the LEDs. The level translators have sufficient output current capacity to drive the LEDs directly.
The bicolor user LEDs connect to the FMC pins listed in the table below:
| Aligned with port | Net Name | FMC pin |
|---|---|---|
| 0 | QSFP0_GRN_LED_T | LA07_P |
| QSFP0_RED_LED_T | LA07_N | |
| 1 | QSFP1_GRN_LED_T | LA08_P |
| QSFP1_RED_LED_T | LA08_N |
Note that when the green and red signals for a single LED are asserted at the same time, the resulting color is amber.
Power Supplies
All power required by the 2x QSFP28 FMC is supplied by the development board through the FMC connector:
- +12VDC
- VADJ: +1.2VDC, +1.5VDC, +1.8VDC, +2.5VDC or +3.3VDC
- +3.3VAUX
The FPGA/MPSoC carrier board also supplies a 3.3VDC power supply, however this supply is not used by the 2x QSFP28 FMC.
The 12VDC Supply
The 12VDC supply is the main power source for the mezzanine card. It feeds a buck switching regulator ( TI, 3-16V 5A Buck Converter, TPS565247DRLR ) that generates 3.3VDC to power to both QSFP28 slots, the clock multiplier and the level translators.
VADJ Supply
The VADJ supply is the FPGA I/O power supply and it determines the voltage levels of the FMC I/Os. On the 2x QSFP28 FMC, the VADJ supply powers the level translators that allow the board to be used at any I/O voltage in the range of 1.2VDC to 3.3VDC.
The 3.3VAUX Supply
The 3.3VAUX supply is used to power the IPMI EEPROM and is independent of the main 3.3VDC supply so that the carrier board can read from the EEPROM without having to power up the entire board.
Power LED and testpoints
An LED indicates when both the power from the carrier board and the switching regulator are active, and it can be seen in the labelled bottom view of the board above. This LED is connected through a logic buffer to the POWER GOOD signal that is driven by the carrier board and is part of the Vita 57.1 FMC standard. The logic buffer is powered by the 3.3VDC that is generated by the switching regulator.
To aid hardware debug, there is a test point for the 12VDC and 3.3VDC (buck regulator) power supplies on the back side of the 2x QSFP28 FMC.
Clocks
The clock architecture of the 2x QSFP28 FMC is based on the jitter-attenuating clock multiplier ( Skyworks, Si5328 ). This clock multiplier operates in a free-running mode, generating a user-specified frequency ranging from 8kHz to 808MHz, synthesized by a crystal oscillator. In Synchronous Ethernet applications, it can also generate a jitter attenuated clock that is synchronous with link partner’s clock.
The figure below illustrates the clock connections on the 2x QSFP28 FMC.
Clock outputs
The Si5328 has two output clocks, both connected to the FMC connector’s GT reference clock inputs. These clocks are divided down separately from a common source, allowing them to be programmed to different frequencies while remaining synchronous.
The clock outputs are connected to the FMC pins listed in the table below:
| Si5328 pin | I/O standard | FMC pin |
|---|---|---|
| CLKOUT1 | LVDS | GBTCLK0_M2C_P/N |
| CLKOUT2 | LVDS | GBTCLK1_M2C_P/N |
Clock input
The Si5328 has two input clocks, but only one is connected on the 2x QSFP28 FMC. This clock input is connected to the FMC pins LA00_CC_P/N, enabling the FPGA to forward a recovered clock from the gigabit transceivers. The Si5328 can perform jitter attenuation on the recovered clock and forward the resulting clock to its outputs. This feature allows the 2x QSFP28 FMC to be used in Synchronous Ethernet applications.
The clock inputs are connected to the FMC pins listed in the table below:
| Si5328 pin | I/O standard | FMC pin |
|---|---|---|
| CLKIN1 | LVDS | LA00_CC_P/N |
Clock loss alarm
The Si5328 has a logic output INT_C1B that indicates loss-of-signal on the input clock CLKIN1 (the recovered clock). The signal goes HIGH when the device detects missing pulses on the input clock. The clock loss alarm passes through level translation and connects to FMC pin LA06_P.