PCI Express Multi-Channel DMA Interface 2014.12.15 UG-01160 Send Feedback Subscribe The PCI Express® DMA Multi-Channel Controller Example Design provides multi-channel support for the Stratix® V Avalon® Memory-Mapped (Avalon-MM) DMA for PCI Express IP Core. Up to eight external DMA controllers drive DMA descriptors to the PCI Express Multi-Channel DMA Avalon Streaming (Avalon-ST) sink interfaces. The PCI Express Multi-Channel DMA arbitrates between the descriptor queues, taking into account the request priority and the weight. The Data Mover is a module in the soft-logic bridge in the Stratix V Avalon-MM DMA for PCI Express IP Core. Figure 1: PCI Express Multi-Channel DMA Interface Example Design System-Level Block Diagram PCI Express Multi-Channel DMA Interface Src DMA Controller0 Sink Rd & Wr Descriptors0 Completion Status0 Sink Src Src DMA Controller7 Sink Completion Status7 Control Register Access Data Mover Rd & Wr Sub-Descriptors Sink Src Sink Rd & Wr Descriptors7 Avalon-MM DMA for PCI Express Completion Status Src Avalon Streaming 256 bits PCIe TLP Sink Src PCIe Hard IP G3X8 Mstr Slave Mstr Memory Slave Slave Avalon-MM 256 bits Avalon-MM 256 bits © 2014 Altera Corporation. All rights reserved. ALTERA, ARRIA, CYCLONE, ENPIRION, MAX, MEGACORE, NIOS, QUARTUS and STRATIX words and logos are trademarks of Altera Corporation and registered in the U.S. Patent and Trademark Office and in other countries. All other words and logos identified as trademarks or service marks are the property of their respective holders as described at www.altera.com/common/legal.html. Altera warrants performance of its semiconductor products to current specifications in accordance with Altera's standard warranty, but reserves the right to make changes to any products and services at any time without notice. Altera assumes no responsibility or liability arising out of the application or use of any information, product, or service described herein except as expressly agreed to in writing by Altera. Altera customers are advised to obtain the latest version of device specifications before relying on any published information and before placing orders for products or services. www.altera.com 101 Innovation Drive, San Jose, CA 95134 Preliminary ISO 9001:2008 Registered 2 UG-01160 2014.12.15 PCI Express Multi-Channel DMA Interface Figure 2: PCI Express Multi-Channel DMA Interface Internals Block Diagram Priority ID Dest Source Length End Orig. ID Sub-ID Dest Source Length Channel 0 Fragmentation Priority Sort 4X Channel 1 Fragmentation Priority Sort 4X Priority Queue 0 Priority Queue 1 Channel 7 Status to Controller Completion StatusCompletion Counter 16x StatusCompletion Counter 16x StatusCompletion Counter 16x Status Counters Fragmentation Priority Sort 4X Scheduler Descriptor to Data Mover Status from Data Mover Descriptor Removal Status Queue Slave Avalon-MM Slave Multi-Channel Control Registers (Abort, pause) Original ID 0 Mapped to > 0 - 7 Original ID 1 Mapped to > 8 - 15 Original ID 2 Mapped to > 16 - 23 Original ID 3 Mapped to > 24 - 31 Original ID 4 Mapped to > 32-39 Original ID 31 Mapped to > 248 - 255 Multi-Channel Adapter The PCI Express Multi-Channel DMA Interface Example Design performs the following functions: • • • • • • Stores up to four outstanding descriptors for each channel before deasserting the ready signal Divides each descriptor into subdescriptors with a payload size of 512 bytes Arbitrates between two priority queues based on the Channel Priority Weight Tracks the completion status of the 32 possible outstanding descriptors Forwards completion status to the DMA Controllers Removes descriptors from internal queues on successful completion and in response to a requests received on its Avalon-MM Control register interface • When multi-function support is enabled, maps function numbers to channel numbers. Altera Corporation Preliminary PCI Express Multi-Channel DMA Interface Send Feedback UG-01160 2014.12.15 Device Family Support 3 Note: The PCI Express Multi-Channel DMA Interface Example Design is preliminary in the current release. Both the programming model and top-level signals may change in subsequent releases. Device Family Support Table 1: Device Family Support Device Family Support Stratix V Final. The IP core is verified with final timing models. The IP core meets all functional and timing requirements for the device family and can be used in production designs. Other device families No support. Parameters The PCIe® Multi-Channel DMA Interface Example Design has three parameters that you can specify to customize its function. Table 2: Parameteres Parameter Name Value Description Number of channels 2, 4, 8 Specifies the number of channels. Channel priority weight 0, 4, 8 The arbiter uses the Channel priority weight in combina‐ tion with the descriptor priority bit to calculate the ratio of transmission between priority queues. Descriptor width 161, 169 Specifies the width of the descriptor. When you select 169, the low-order 8 bits specify the function number for variants with SR-IOV enabled. PCIe Multi-Channel DMA Interfaces The PCIe Multi-Channel DMA Interface Example Design interfaces receive DMA descriptors on up to eight Descriptor Controller Avalon-ST RX interfaces. It drives the selected descriptor to the Data Mover using its single Descriptor Avalon-ST TX interface. The Data Mover drives status to the PCIe MultiChannel DMA on the Descriptor Status Avalon-ST RX interface. The PCIe Multi-Channel DMA forwards status to the DMA controllers on its Descriptor Status Avalon-ST source interfaces. An Avalon-MM slave interface allows DMA controllers to remove descriptors from the PCIe Multi-Channel DMA internal queues. PCI Express Multi-Channel DMA Interface Send Feedback Preliminary Altera Corporation 4 UG-01160 2014.12.15 PCIe Multi-Channel DMA Descriptor Interfaces PCI Express Multi-Channel DMA Controller IP Core Descriptor Controller0 RX Descriptor Controller7 RX DescInstr0_i[168 or 160:0] DescInstrValid0_i DescInstrReady0_o DescInstr7_i[167:0] DescInstrValid7_i DescInstrReady7_o Completion Status0 DmaTxData0_o[31:0] DmaTxValid0_o Completion Status7 DmaTxData7_o[31:0] DmaTxValid7_o Control Register DmaRxData_o[159:0] DmaRxValid_o DmaRxReady_i Descriptor TX DmaTxData_i[31:0] DmaTxValid_i Descriptor Status RX MCSlaveChipSelect_i MCSlaveWrite_i MCSlaveAddress_i[13:0] MCSlaveWriteData_i[31:0] MCSlaveWaitRequect_o clk_i rstn_i Related Information Avalon Interface Specifications PCIe Multi-Channel DMA Descriptor Interfaces Table 3: Descriptor Controller RX Interface This is an Avalon-ST sink interface. It is synchronous to clk_i. Signal Name DescInstr<n>i[168 or 160:0] Direction Input Description Write descriptor or data moving instruction. When SRIOV is enabled, the bus has the following layout: • [168:161]: Function number • [160]: Priority • [159:0]: Descriptor Otherwise, the bus has the following layout: • [160]: Priority • [159:0] Descriptor This is the format that the Data Mover included in the Avalon-MM DMA for PCI Express requires. Altera Corporation Preliminary PCI Express Multi-Channel DMA Interface Send Feedback UG-01160 2014.12.15 Control Register Interface Signal Name Direction DescInstrValid<n> _i Input When asserted, indicates that DescInstr<n>_i is valid. DescInstrReady<n> _o Output When asserted, indicates that the channel can accept new descriptors. The ready latency on this interface is 0 cycles. The Multi-Channel DMA Controller stores data on the rising edge of clock whenever DescInstrReady<n> and DescInstrValid<n> are both asserted. 5 Description Table 4: Descriptor Status Interface This is an Avalon-ST source interface. Signal Name Direction DmaTxData<n>_o[31:0] Output Description Specifies the status for the descriptor ID specified. The following fields are defined: • [31:14]: Reserved • [13:12]: Status code • 2'b00: Descriptor completed successfully • 2'b01: Descriptor removal completed successfully • 2'b10: Descriptor partially removed • 2'b11: Descriptor removal failed • [11:8]: Reserved • [7:0]: Descriptor ID whose status is being reported DmaTxValid<n>_o Output When asserted, indicates that DmaTxData<n>_o[31:0] is valid. This interface does not have a ready input. The requesting controller must be able to store data whenever DmaTxValid<n> is asserted. Control Register Interface To cancel a pending descriptor, a DMA controller writes the descriptor ID to the Avalon-MM slave interface. The Mulit-Channel DMA Interface IP Core drives the status of this request on the Descriptor Status interface of the requesting controller. Table 5: Control Register Interface This is an Avalon-MM slave interface. It is synchronous to clk_i. Signal Name Direction MCSlaveChipSelect_i Input When asserted, the other signals in this interface are driving valid data. MCSlaveWrite_i Input Asserted to perform a write. MCSlaveAddress_i[13:0] Input Specifies the address of the Control Register AvalonMM slave interface. PCI Express Multi-Channel DMA Interface Send Feedback Description Preliminary Altera Corporation 6 UG-01160 2014.12.15 Descriptor TX Interface Signal Name MCSlaveWriteData_ i[31:0] Direction Input Description Specifies the ID for the of the descriptor being deleted. The status of this request is reported on the appropriate Control Status interface. Note: When removal of a descriptor is requested, internal logic searches the pipeline and buffers for the descriptor ID. It may not find the ID requested. The requesting controller must manage appropriately, based on the status reported on the Control Status interface. MCSlaveWaitRequest_o Input When asserted, this interface is not ready to accept commands. Descriptor TX Interface Table 6: Descriptor TX Interface to Data Mover This is an Avalon-ST source interface. It is synchronous to clk_i. Signal Name Direction Description Output The DMA Multi-Channel Controller drives descriptor data to the Data Mover on this Avalon-ST source interface. (It uses priority bit[160] internally and does not drive it to the Data Mover. DmaRxValid_o Output When asserted, the data on DmaRxData_o is valid. DmaRxReady_i Input DmaRxData_o[167 or 159:0] Altera Corporation When asserted, the Data Mover is ready to accept new descriptors. The ready latency on this interface is 3 cycles. Consequently, the DMA Multi-Channel Controller must stop driving valid data within 3 cycles of the deassertion of DmaRxReady_i. Preliminary PCI Express Multi-Channel DMA Interface Send Feedback UG-01160 2014.12.15 Example Design 7 Table 7: Descriptor Status Interface This is an Avalon-ST sink interface. the Signal Name DmaTxData_i[31:0] Direction Input Description The Data Mover drives status to the DMA MultiChannel Controller on this Avalon-ST interface. Specifies the status for the descriptor ID specified. The following fields are defined: • [31:14]: Reserved. All 0s. • [13:12]: Status cod.e • 2'b00: Descriptor completed successfully. • 2'b01: Descriptor removal completed successfully. • 2'b10: Descriptor partially removed. Part of the descriptor has already been processed. The controller must include logic to process partial completion data. • 2'b11: Descriptor removal failed. • [11:8]: Reserved. All 0s. • [7:0]: Descriptor ID whose status is being reported. DmaTxValid_i_i Input When asserted, indicates that DmaTxData_i[31:0] is valid. Example Design You can download a Qsys example design that illustrates the use of this component from the install directory, <install_dir>ip/altera/altera_pcie/altera_pcie_hip_256_avmm/channelizer/example_design/. This example design includes the following Qsys files: • mc_top_g3x8_4ch.qsys—This is the top-level Qsys file that connects the DUT and APPS components. • sriov_mcdma_app_g3x8_256b.qsys—This Qsys system connects an example Avalon-MM DMA for PCI Express with Single Root I/O Virtualization (SR-IOV) to read and write descriptor controllers and SRAM. • rddc_mc_256b.qsys—This Qsys system connects four read descriptor controllers the the PCI Express DMA Multi-Channnel Controller IP Core. • wrdc_mc_256b.qsys—This Qsys system connects four write descriptor controllers the the PCI Express DMA Multi-Channnel Controller IP Core. PCI Express Multi-Channel DMA Interface Send Feedback Preliminary Altera Corporation 8 UG-01160 2014.12.15 Revision History Figure 3: Qsys Sub-System with Four Write Descriptor Controllers and the DMA Multi-Channnel Controller Revision History Date 2014.12.15 Altera Corporation Version 14.1 Changes • Initial Release. Preliminary PCI Express Multi-Channel DMA Interface Send Feedback