Sub-LVDS Serial to CMOS Parallel
Sensor Bridge
October 2012
Reference Design RD1130
Introduction
The Sony Image Sensor Division has begun introducing image sensors with faster frame rates and more megapixels. Examples include the IMX136, a 2.38M pixel sensor (1080p) capable of frame rates of up 120 fps, and the
IMX104, a 720p sensor with frame rates up to 120 fps and a serial sub-LVDS interface. These sensors were introduced with CMOS parallel, sub-LVDS parallel, and sub-LVDS serial output formats. The output drivers of the sensor cannot handle the higher bandwidth requirements of the CMOS parallel output format. As a result, for higher
resolutions and frame rates, sub-LVDS output formats must be used. Since most ISP vendors only support CMOS
parallel inputs, a bridge device must convert the sub-LVDS serial interface to a CMOS parallel interface (Figure 1).
The Sub-LVDS Serial to CMOS Parallel Sensor Bridge for Sony Sensors can perform this conversion.
The Sony serial sub-LVDS interface requires the fewest signals to transmit an image. All future Sony cameras will
move to this serial interface. If you wish to use the sub-LVDS parallel interface, Lattice has a design solution for this
as well. Please refer to RD1122, Sub-LVDS-to-Parallel Sensor Bridge, for details.
Figure 1. Sub-LVDS Serial to CMOS Parallel Functional Overview
Sub-LVDS
Serial Interface
ISP with
CMOS Parallel
Interface
MachXO21200
Parallel Bus
(Frame, Line, Clock, etc.)
The sub-LVDS Serial to CMOS Parallel Sensor Bridge for Sony Sensors is targeted to Lattice FPGA devices. When
an external ISP device is used and only a bridge function is needed, the low-cost MachXO2™ is an ideal choice.
Serial sub-LVDS data from the IMX136 is delivered on both rising and falling clock edges (double data rate). The
reference design first converts the DDR input to use a single clock edge and reduces the clock rate using DDRx4
gearing. The output from the DDRx4 primitives are converted and aligned to 10 or 12 bit words. Words from each
of the serial lanes are then aligned to each other. Aligned words are parsed so that a 10 or 12 bit word is available
every pixel clock cycle.
© 2012 Lattice Semiconductor Corp. All Lattice trademarks, registered trademarks, patents, and disclaimers are as listed at www.latticesemi.com/legal. All other brand
or product names are trademarks or registered trademarks of their respective holders. The specifications and information herein are subject to change without notice.
www.latticesemi.com
1
rd1130_01.2
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Figure 2. Top Level Diagram
sony_serial2parallel
rstn
Word
Aligner
Deserializer
CH0_DO5
CH1_DO6
CH2_DO4
CH3_DO7
DCK
IDDRx4
pixdata
Word
Aligner
din
32
Lane
Aligner
DIV4
Parser
fv
lv
Word
Aligner
sclk
Word
Aligner
pixclk
pixclk_adj
pll_*bit_*lane
Design Package
The sub-LVDS Serial to CMOS Parallel Sensor Bridge for Sony Sensors design package is available free of charge
on the Lattice website at www.latticesemi.com. The design package contains everything you need to get started.
• /bitstream/* – Bitstreams for all serial device modes
• /doc/* – This document
• /impl/* – Lattice Diamond® 1.4 project targeted to the MachXO2-1200HC device in a 132-ball csBGA package
• /IPExpress/* – IPexpress™ modules for PLLs and iddrx4
• /models/* – Verilog simulation primitives
• /ngo/* – NGOs targeted to the MachXO2 device for all serial device modes
• /rtl/* – Top level design module and NGO module black box
• /sim/* – Aldec simulation environment files and simulation wizard script
• /testbench/* – Verilog simulation test bench
• /Readme.txt – Reiterates the basics of this document and how to get started
Top Level Design Module
The top level design module instantiates the deserializer module, PLL module and sony_serial2parallel NGO. The
deserializer module contains the IDDRX4 and CLKDIV4 primitives. A PLL is used to convert the divided-down
serial clock to the pixel clock. Access to the PLL is available at the top level so users can utilize it for any other
design-specific needs. The sony_serial2parallel NGO contains the serial-to-parallel bridge design targeted to the
MachXO2 device. The appropriate NGO and PLL are chosen automatically based on bus_width and lane_width
parameters when using the included Lattice Diamond project.
2
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Figure 3. Top Level RTL Diagram from Synplify Pro – RTL View
There are six possible configurations for the IMX136 in serial mode, all of which are supported by the bridge. The
mode can be configured using the Verilog parameters at the top level of the sony_serial_sensor_bridge.v design
file on lines 50 and 51.
• 10-bit, 1-lane
• 12-bit, 1-lane
• 10-bit, 2-lane
• 12-bit, 2-lane
• 10-bit, 4-lane
• 12-bit, 4-lane
Figure 4. Top Level Verilog Parameters for Bus and Lane Widths
3
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Top Level Verilog Parameters for Bus and Lane Widths
The top level pinout for the design consists of the serial sub-LVDS inputs from the IMX136, the parallel CMOS output bus, frame and line valid indictor pins and an output clock.
Table 1. Top Level Design Pinout
Parameter
Configurations
Description
bus_width
10, 12
Pixel bus width
lane_width
1, 2, 4
Number of data lanes used
Direction
I/O Type
rstn
Signal
Input
LVCMOS
CH0_DQ5
Input
LVDS
Lane 0 serial data
CH1_DQ6
Input
LVDS
Lane 1 serial data
CH2_DQ4
Input
LVDS
Lane 2 serial data (only used when lane_width==4)
CH3_DQ7
Input
LVDS
Lane 3 serial data (only used when lane_width==4)
Serial clock
DCK
Description
Sensor bridge reset
Input
LVDS
pixdata[bus_width-1:0]
Output
LVCMOS
fv
Output
LVCMOS
Frame valid
lv
Output
LVCMOS
Line valid
pixclk
Output
LVCMOS
Pixel clock
Pixel data
Gearing the data in the FPGA allows the design to internally run at a much slower clock rate, which increases performance, decreases power and decreases cost. Serial data from the IMX136 is converted to use a single clock
edge. It is also reduced to a lower clock speed, which can be calculated by the following equation:
sclk = input_clock * (1/4)
The pixel clock output is used to clock out the parallel data and is calculated with the following equation:
pixclk = (1/bus_width) * #_lanes * 2 * 4 * sclk
Table 2. sclk and pixclk Speeds for Device Modes
Device Mode
1080p Full HD mode
WUXGA All-pixel scan
mode
720p Full HD mode
Frame Rate
(Frames/Second)
Input Data Rate
(Mbps/Channel)
sclk (MHz)
pixclk (MHz)
30
445.5
55.6875
74.25
30
222.75
27.84375
74.25
60
445.5
55.6875
148.5
27
445.5
55.6875
74.25
54
222.75
27.84375
74.25
108
445.5
55.6875
148.5
30
222.75
27.84375
37.125
30
111.375
13.921875
37.125
60
445.5
55.6875
74.25
60
222.75
27.84375
74.25
120
445.5
55.6875
148.5
The active region of the image is found by using the sync codes embedded in each serial data lane by the Sony
IMX136. Sync codes are detected and control the pixdata output as well as the fv and lv (frame valid and line valid)
output indicators.
4
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Table 3. Sync Codes and Indicator States
Cycle 1
Sync Code
SAV (valid line)
Cycle 2
Cycle 3
Cycle 4
12-bit
10-bit
12-bit
10-bit
12-bit
10-bit
12-bit
10-bit
0xFFF
0x3FF
0x000
0x000
0x000
0x000
0x800
0x200
FV State LV State
1
1
EAV (valid line)
0xFFF
0x3FF
0x000
0x000
0x000
0x000
0x9D0
0x274
1
0
SAV (invalid line)
0xFFF
0x3FF
0x000
0x000
0x000
0x000
0xAB0
0x2AC
0
0
EAV (invalid line)
0xFFF
0x3FF
0x000
0x000
0x000
0x000
0xB60
0x2D8
0
0
pixdata, fv and lv are all edge-aligned with pixclk at the Lattice device output. pixclk_adj is the output clock to be
used by the ISP. By default, pixclk_adj is offset from pixclk by 180 degrees. This means that pixdata, fv and lv are
center-aligned with pixclk_adj at the output. The offset can be adjusted in IPexpress if desired by the user. To do
this, click on the pll_*bit_*lane.ipx file for your configuration and to adjust the phase on the CLKOS output.
Figure 5. pixclk_adj Phase Adjustment in IPexpress
5
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Reference Design NGO
The design NGO files contain a complete sensor bridge design for all serial configurations available on the IMX136.
Figure 6. NGO RTL Diagram from Synplify Pro – RTL View
Changing configuration parameters in the top level design file (sony_serial_sensor_bridge.v) will call the corresponding NGO automatically when using the included Diamond software project (sony_serial_sensor_bridge.ldf).
I/O for the NGO consists of the deserialized data bus and clocks, as well as the parallel data, clock, and control outputs.
6
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Table 4. NGO I/O Descriptions
Parameter
Configurations
Description
bus_width
10, 12
Pixel bus width
lane_width
1, 2, 4
Number of data lanes used
Signal
Direction
Description
rstn
Input
Sensor bridge reset
sclk
Input
Serial clock/4
pixclk
Input
Upconverted pixel clock from PLL. Based on sclk.
Deserialized sensor data input. I/O created to match
IPexpress generated output for iddrx4 core with 4 serial
inputs.
Lane 0:
{din[28],din[24],din[20],din[16],din[12],din[8],din[4],din[0]}
din[31:0]
Input
Lane 1:
{din[29],din[25],din[21],din[17],din[13],din[9],din[5],din[1]}
Lane 2:
{din[30],din[26],din[22],din[18],din[14],din[10],din[6],din[2]}
Lane 3:
{din[31],din[27],din[23],din[19],din[15],din[11],din[7],din[3]}
pixdata[bus_width-1:0]
Output
Pixel data
fv
Output
Frame valid
lv
Output
Line valid
Diamond Project Information
Included in the packaged design is the Lattice Diamond project file (sony_serial_sensor_bridge.ldf). The project is
pre-configured for the MachXO2-1200HC-6 in a 132-ball csBGA package. Configuring the top level design file for
your desired mode of operation can be performed by double clicking sony_serial_sensor_bridge.v under Input
Files in the File List.
7
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Figure 7. File List Tab in Lattice Diamond Software
Synthesis of the design and generation of a JEDEC bitstream can be performed by clicking on the Process tab.
Then check the JEDEC File checkbox under Export Files, right-click on JEDEC File, and click Rerun All. After
synthesis is complete, a *.JED bitstream file will be available in the *impl\sony_serial_sensor_bridge\ directory.
Figure 8. Process Tab in Lattice Diamond Software
8
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Simulation
The included test bench is capable of testing all lane and bus widths available on the IMX136 for serial sub-LVDS
mode. 1-lane 10-bit, 1-lane 12-bit, 2-lane 10-bit, 2-lane 12-bit, 4-lane 10-bit, and 4-lane 12-bit modes were all simulated for design verification. The easiest way to set up, access and run the simulation is through Lattice Diamond
and the pre-configured scripts (sony_serial_bridge_simulation.spf) provided in the *.ldf project. To do this, doubleclick on the script file under Script Files in the File List. Then click Finish in the Simulation Wizard. This will open
Aldec Active HDL Simulator. Follow the instructions for running a basic simulation in the simulator help if you are
unfamiliar with the environment.
Similar to configuring the serial device mode for synthesis, the design test bench parameters also need to be configured for the desired mode of operation. This can be done in the same way by opening the Verilog test bench
(sony_serial_sensor_bridge_tb.v) and modifying the Verilog parameters on lines 52 and 53.
Figure 9. Verilog Testbench Parameters
Figure 10. Simulation Screen Shot of Design Output
Device Support
The MachXO2-1200 device in the 132-ball csBGA package has been a proven solution for sensor bridge design
implementation. For this reason, it has been chosen as the default target device. Other devices and packages can
be targeted, but the pinout and design implementation choices are left up to the user. For best results, use the
device, package and pinout described below. With this pinout, no external termination resistors are required for the
sub-LVDS lanes.
9
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Table 5. Device Pinout
Signal
MachXO2, 132-Ball csBGA,
Speed Grade -6
rstn
C1
DCK_p
N6
DCK_n
P6
CH0_DQ5_p
M11
CH0_DQ5_n
P12
CH1_DQ6_p
P8
CH1_DQ6_n
M8
CH2_DQ4_p
P2
CH2_DQ4_n
N2
CH3_DQ7_p
M7
CH3_DQ7_n
N8
pixclk_adj
A11
fv
B7
lv
C4
pixdata_0
C6
pixdata_1
B3
pixdata_2
C11
pixdata_3
A12
pixdata_4
A7
pixdata_5
B5
pixdata_6
A9
pixdata_7
A10
pixdata_8
A2
pixdata_9
B12
pixdata_10
C12
pixdata_11
B13
Table 6. Design Performance
fMAX (MHz)
Device Family
MachXO2
Speed Grade -4
Speed Grade -5
Speed Grade -6
Configuration
sclk
pixclk
sclk
pixclk
sclk
pixclk
10-bit, 2-lane
110.412
131.216
104.384
154.488
123.778
163.639
12-bit, 2-lane
124.456
140.469
126.310
156.006
140.805
170.010
10-bit, 4-lane
51.222
131.216
69.099
151.860
78.456
168.947
12-bit, 4-lane
67.105
139.005
72.786
156.006
79.510
162.417
10
Sub-LVDS Serial to CMOS Parallel Sensor Bridge
Table 7. Resource Utilization
Device Family
MachXO2
Configuration
Registers
LUTs
EBRs
PLL
10-bit, 1-lane
322
345
2
1
12-bit, 1-lane
310
340
2
1
10-bit, 2-lane
482
526
3
1
12-bit, 2-lane
535
513
3
1
10-bit, 4-lane
923
1029
4
1
12-bit, 4-lane
950
891
4
1
Table 8. I/O Timing Analysis of Sub-LVDS Parallel Input Bus
Setup
Device Family
MachXO2
Hold
Setup
Hold
Setup
Hold
Speed Grade -4
Speed Grade 5
Speed Grade 6
0.222
0.222
0.222
0.337
0.254
0 .175
References
• Sony IMX136LQJ Data Sheet
Technical Support Assistance
Hotline: 1-800-LATTICE (North America)
+1-503-268-8001 (Outside North America)
e-mail: [email protected]
Internet: www.latticesemi.com
Revision History
Date
Version
June 2012
01.0
Initial release.
Change Summary
August 2012
01.1
Updated to include support of the built in WDR capability of the Sony
IMX136/104.
October 2012
01.2
Updated Introduction section.
11