MT9D115: 1/5-Inch SOC Digital Image Sensor Features 1/5-Inch System-On-A-Chip (SOC) CMOS Digital Image Sensor MT9D115 Datasheet, Rev. E For the latest datasheet, please visit www.onsemi.com Features • • • • • • • • • • • • • • • Table 1: Key Performance Parameters Parameter 2 Mp resolution (1600H x 1200V) 1/5-inch optical format Same or better image quality compared to MT9D112 Individual module ID support through one-time programmable (OTP) memory Surface fit lens correction (LC) to compensate for lens/small pixel vignetting and corner color variations Automatic functions: Exposure, white balance, black level offset correction, flicker detection and avoidance, color saturation control, defect identification and correction, aperture correction, and GPIO Programmable controls: Exposure, white balance, horizontal and vertical blanking, color, sharpness, gamma, lens shading correction, horizontal and vertical image flip, zoom, windowing, sampling rates, and GPIO 15 frames per second (fps) at 1600H x 1200V with moderate pixel clock frequency ( 64 MHz) to minimize baseband reception interference and 30 fps at 800H x 600V 2 x 2 pixel binning to improve low-light image quality Support for external LED or xenon flash On-chip phase-locked loop (PLL) to minimize the number of system clocks Low power modes to prolong battery life of portable devices Fail-safe I/Os with programmable output slew rate Industry standard two-wire serial interface for controls 10-bit parallel or MIPI serial interfaces for image data Pixel size Optical format Array format (active) Imaging area CRA Color filter array Scan mode Shutter Input clock range Output pixel clock maximum Output MIPI data rate maximum Max. Frame Rate Responsivity Signal-to-noise ratio Dynamic range Digital Analog Supply voltage I/O MIPI Power consumption Operating temperature range Package Value 1.75 m x 1.75 m 1/5-inch 1600H x 1200V = 1.92 Mp 2.8 mm x 2.10 mm: 3.50 mm diagonal (4:3 aspect ratio) 25° RGB Bayer Progressive Electronic rolling shutter (ERS) 6 – 54 MHz 85 MHz 512 Mb/s 15 fps full res 30 fps 800 x 600 0.65 V/Lux-sec (550 nm) 39 dB (MAX) 63.9 dB (pixel) 1.8 V (nominal) 2.8 V (nominal) 1.8 V or 2.8 V (nominal) 1.7-1.95V 196mW1 –30°C to 70°C (at junction) Bare die, CSP Notes: 1. Power consumption for typical voltages at 800 x 600 video mode Applications • Cellular phones • PC cameras • PDAs MT9D115 DS Rev. E Pub. 4/15 EN 1 ©Semiconductor Components Industries, LLC,2015 MT9D115: 1/5-Inch SOC Digital Image Sensor Ordering Information Ordering Information Table 2: Available Part Numbers Part Number Product Description Orderable Product Attribute Description MT9D115D00STCK25AC1-200 2 MP 1/5" SOC Die Sales, 200m Thickness MT9D115EB3STC-CR 2 MP 1/5" CIS SOC Chip Tray without Protective Film MT9D115W00STCK25AC1-750 2 MP 1/5" SOC Wafer Sales, 750m Thickness MT9D115 DS Rev. E Pub. 4/15 EN 2 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Table of Contents Table of Contents Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Ordering Information. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Functional Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Architecture Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Typical Connections. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Decoupling Capacitor Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8 Signal Descriptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 Control Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35 Camera Functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37 Optics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 Power Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41 Other Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .51 GPIO Control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53 Electrical Specifications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54 Appendix A- VDD_IO Current Addition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .59 Revision History. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .66 MT9D115 DS Rev. E Pub. 4/15 EN 3 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor List of Figures List of Figures Figure 1: MT9D115 Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6 Figure 2: Typical Configuration (connection) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .7 Figure 3: SOC Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10 Figure 4: Firmware Architecture. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 Figure 5: External Host Register Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 Figure 6: Two-Wire Serial Control Bus Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13 Figure 7: Sensor Core Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 Figure 8: Pixel Color Pattern Detail (Top Right Corner) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 Figure 9: Imaging a Scene . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16 Figure 10: Six Pixels in Normal and Column Mirror Readout Modes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 Figure 11: Six Rows in Normal and Row Mirror Readout Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 Figure 12: Eight Pixels in Normal and Column Skip 2X Readout Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 Figure 13: Pixel Readout (no skipping) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 Figure 14: Pixel Readout (x_odd_inc = 3, y_odd_inc = 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 Figure 15: Pixel Readout (x_odd_inc = 1, y_odd_inc = 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 Figure 16: Pixel Readout (x_odd_inc = 3, y_odd_inc = 3) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 Figure 17: Pixel Readout (x_odd_inc = 3, y_odd_inc = 1, x_bin = 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 Figure 18: Pixel Readout (x_odd_inc = 3, y_odd_inc = 3, x_ybin = 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 Figure 19: Valid Image Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 Figure 20: Pixel Data Timing Example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 Figure 21: Available Test Patterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26 Figure 22: IFP Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27 Figure 23: Gamma Correction Curve. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .30 Figure 24: Timing of Full Frame Data or Scaled Data Passing Through the FIFO . . . . . . . . . . . . . . . . . . . . . . . . . .33 Figure 25: Sequencer Finite State Machine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .36 Figure 26: CRA vs. Image Height. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 Figure 27: Power Application Sequence Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42 Figure 28: Internal Power-On Reset. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43 Figure 29: Hard Reset Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 Figure 30: Soft Reset Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45 Figure 31: Recommended Power Down Sequence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46 Figure 32: Hard Standby Mode Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48 Figure 33: Soft Standby Mode Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 Figure 34: Output Interface Timing Waveforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56 Figure 35: Recommended System and Test Setup for Minimum VDD_IO Power Consumption . . . . . . . . . . . . .59 Figure 36: RESET_BAR Pad Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .61 Figure 37: Recommended System and Test Setup for Minimum VDD_IO Power Consumption in the MT9D115 Sharing with Multiple Two-wire Serial Interface Devices62 Figure 38: Recommended System and Test Setup for Minimum VDD_IO Power Consumption in the MT9D115 Sharing with Multiple Two-wire Serial Interface Devices at Different IO Levels63 Figure 39: System Setup with Independent Voltage Sources for MT9D115 and Controller Chip . . . . . . . . . . . .64 Figure 40: Modifications to the Demo2 Sensor Head Board for VDD_IO Current Measurement. . . . . . . . . . . . .65 MT9D115 DS Rev. E Pub. 4/15 EN 4 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor List of Tables List of Tables Table 1: Table 2: Table 3: Table 4: Table 5: Table 6: Table 7: Table 8: Table 9: Table 10: Table 11: Table 12: Table 13: Table 14: Table 15: Table 16: Table 17: Table 18: Table 19: Table 20: Table 21: Table 22: Table 23: Table 24: Table 25: Table 26: Table 27: Table 28: Table 29: Table 30: Key Performance Parameters. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Available Part Numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2 Signal Description and Direction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9 List of Drivers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11 Two-Wire Serial Interface Timing Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 Row Address Sequencing (Sampling). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .23 Row Address Sequencing (Binning) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 Data Formats Supported by MIPI Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32 YCbCr Output Data Ordering. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 RGB Ordering in Default Mode . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 2-Byte RGB Format. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .34 Power Application Sequence Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42 POR Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43 Hard Reset . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 Soft Reset Signal Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45 Power Down Signal Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46 Hard Standby Signal Timing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .48 Soft Standby Signal Timing. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 Status of Signals During Different States. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .50 GPIO Related Registers and Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .53 Maximum Rating . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54 I/O Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .54 MIPI High-Speed Transmitter DC Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55 MIPI High-Speed Transmitter AC Characteristics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55 MIPI Low-Power Transmitter DC Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55 MIPI Low-Power Transmitter AC Characteristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .55 AC Electricals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .56 DC Electricals. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .57 Status of Signals During Standby State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .60 Typical Mismatch Current in VDD_IO Due to Mismatch in RESET_BAR level and VDD_IO Level . .61 MT9D115 DS Rev. E Pub. 4/15 EN 5 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Functional Description Functional Description The ON Semiconductor MT9D115 is a 1/5-inch 2 Mp CMOS digital image sensor with an integrated advanced camera system. This camera system features a microcontroller (MCU), a sophisticated image flow processor (IFP), MIPI and parallel output ports (only one output port can be used at a time). The microcontroller manages all functions of the camera system and sets key operation parameters for the sensor core to optimize the quality of raw image data entering the IFP. The IFP will be responsible for processing and enhancing the image. The entire system-on-a-chip (SOC) has superior low-light performance that is particularly suitable for PC camera applications. The MT9D115 features ON Semiconductor’s breakthrough low-noise CMOS imaging technology that achieves near-CCD image quality (based on signal-to-noise ratio and low-light sensitivity) while maintaining the inherent size, cost, and integration advantages of CMOS. The ON Semiconductor MT9D115 can be operated in its default mode or programmed for frame size, exposure, gain, and other parameters. The default mode output is a 800x600 image size at 30 frames per second (fps), assuming a 24 MHz input clock. It outputs 8-bit data, using the parallel output port. Architecture Overview The MT9D115 combines a 2 Mp sensor core with an IFP to form a stand-alone solution for both image acquisition and processing. Both the sensor core and the IFP have internal registers that can be controlled by the user. In normal operation, an integrated microcontroller autonomously controls most aspects of operation. The processed image data is transmitted to the host system either through the parallel or MIPI interface. Figure 1 shows the major functional blocks of the MT9D115. MT9D115 Block Diagram Sensor C ore Im age F low Processor (IF P ) Output Interface Formatter Figure 1: Pixel Array F IF O C olor Pipeline MIPI Parallel Stats Engine Internal R egister Bus POR R OM M icrocontroller SR AM T w o- W ire Serial IF System C ontrol MT9D115 DS Rev. E Pub. 4/15 EN M icrocontroller U nit (MCU) 6 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Typical Connections Typical Connections Figure 2: Typical Configuration (connection) MIPI Power TX I/O Power RPULL-UP5 Slave two-wire serial interface VDD_IO VDDIO_TX8 PLL Power VDD_PLL Digital Core Power Analog Power VDD VAA6 VAA_PIX6 SDATA SCLK SADDR Active LOW reset Standby mode External clock in (6–54 MHz) General purpose I/Os (FLASH, OE_BAR, DOUT_LSB[1:0]) DOUT[7:0] PIXCLK RESET_BAR To parallel camera port LINE_VALID STANDBY EXTCLK FRAME_VALID GPI0[3:0]3 DOUT_N OR4 DOUT_P VPP7 CLK_N To serial camera port2 CLK_P DGND GND_PLL GND_IO VDD_IO 0.1µF Notes: MT9D115 DS Rev. E Pub. 4/15 EN AGND VDDIO_TX/VDD 0.1µF VAA_PIX/VDD_PLL/VAA 0.1µF 1. This typical configuration shows only one scenario out of multiple possible variations for this sensor. 2. If a MIPI Interface is not required, the following pads must be left floating: DOUT_P, DOUT_N, CLK_P, and CLK_N. 3. The GPIO pads can serve multiple features that can be reconfigured. The function and direction will vary by applications. 4. Only one output mode (serial or parallel) can be used at any time. 5. ON Semiconductor recommends a resistor value of 1.5Kto VDD_IO for the two-wire serial interface RPULL-UP; Higher values can be used for slower transmission speed. 6. VAA and VAA_PIX can be tied together. Although separate decoupling capacitors are recommended for VAA and VAA_PIX, decoupling capacitors can be shared if one would like to reduce module size. 7. VPP is the OTP memory programming voltage and should be left floating during normal operation. 8. 1.8V supply is shared by MIPI interface and VDD to reduce the number of decoupling caps, and, subsequently, the module size. VDDIO_TX must be connected to a 1.8V power supply source, even though MIPI interface is not used. 9. ON Semiconductor recommends that 0.1F and 1F decoupling capacitors for each power supply are mounted as close as possible to the pad and that a 10F capacitor be placed nearby off-module. Actual values and results can vary depending on layout and design considerations. Please follow ON Semiconductor's recommended capacitor Recommendations. 10. VDD_PLL and VAA can share the same power source, in which case GND_PLL must be connected to GND. 11. Internal pull-up in RESET_BAR pin and can be left floating when not connected. 7 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Decoupling Capacitor Recommendations Decoupling Capacitor Recommendations It is important to provide clean, well-regulated power to each power supply. The customer is ultimately responsible for ensuring that clean power is provided for their own designs because hardware design is influenced by many factors, including layout, operating conditions, and component selection. The recommendations for capacitor placement and values listed below are based on the ON Semiconductor internal demo camera design and verified in hardware. ON Semiconductor recommends the following, in order of preference: 1. Mount 0.1F and 1F decoupling capacitors for each power supply as close as possible to the pad and place a 10F capacitor nearby off-module. 2. If module limitations allow for only six decoupling capacitors for a three-regulator design (VDD1V2 tied to external regulator), use a 0.1F and 1F capacitor for each of the three regulated supplies. ON Semiconductor also recommends placing a 10F capacitor for each supply off-module, but close to each supply. 3. If module limitations allow for only three decoupling capacitors, use a 1µF capacitor (preferred) or a 0.1µF capacitor for each of the three regulated supplies. ON Semiconductor also recommends placing a 10µF capacitor for each supply off-module but close to each supply. 4. Give priority to the VAA supply for additional decoupling capacitors. ON Semiconductor does not recommend inductive filtering components. Follow best practices when performing physical layout. Refer to technical note TN-09131. MT9D115 DS Rev. E Pub. 4/15 EN 8 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Signal Descriptions Signal Descriptions Table 3: Signal Description and Direction Name Type Description STANDBY Input Hardware standby EXTCLK Input External clock input Note SADDR Input Two-wire interface device select address SCLK Input Two-wire interface serial clock RESET_BAR Input Hardware reset CLK_N Output MIPI differential clock N 5 CLK_P Output MIPI differential clock P 5 DOUT_N Output MIPI differential data N 5 DOUT_P Output MIPI differential data P 5 DOUT[7:0] Output Parallel image data 1 FRAME_VALID Output Parallel pixel bus frame valid 1 4 LINE_VALID Output Parallel pixel bus line valid 1 PIXCLK Output Parallel pixel bus pixel clock 1 SDATA Bidirectional Two-wire interface serial data Bidirectional/Output 2 2 GPIO_3/OE_BAR Bidirectional/Output General-purpose I/O or LSB for Raw 10 data output during SOC Bypass General-purpose I/O or LSB for Raw 10 data output during SOC Bypass General-purpose I/O or output enable GPIO_2/FLASH Bidirectional/Output General-purpose I/O or flash control VAA Supply Analog core power source 2.8V nominal VAA_PIX Supply Analog core power source 2.8V nominal VDD Supply Digital core power source 1.8V nominal VDD_IO Supply Digital IO power source 1.8V or 2.8V nominal VDD_PLL Supply Digital PLL power source 2.8V nominal VDDIO_TX Supply Digital MIPI IO power source 1.8V nominal. GPIO_0/DOUT_LSB[0] GPIO_1/DOUT_LSB[1] Notes: MT9D115 DS Rev. E Pub. 4/15 EN Bidirectional/Output 2 2 6 1. In serial only mode, DOUT[7:0], PIXCLK, and GPIO[3:0] can be left floating by setting R0x0026[1] =1. If GPIO signals are required, DOUT[7:0] and PIXCLK must be tied to DGND and OE_BAR must be tied to VDD_IO. GPIO_3 should be configured as an input for OE_BAR function and set R0x001A[8] = 1. 2. GPIO can be left floating if not used and must be programmed as outputs. 3. Must be connected to VDD_IO, internal 100k ohms typical at 2.8V VDDIO used. 4. Can be left floating if not used. 5. Must be connected to VDD, even in designs where the MIPI interface is not used. 9 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Architecture The MT9D115 from ON Semiconductor is the third-generation, two-megapixel camera SOC. It is a microprocessor-based camera system that combines a sensor core with an image flow processor (IFP) to form a standalone solution that includes image acquisition and processing. Both the sensor core and IFP have internal registers that can be accessed by an external host. In normal operation, the integrated system microprocessor autonomously controls most aspects of operation. The image data is transmitted to the external host system either through a parallel bus or a serial MIPI interface (see Figure 3). Figure 3: SOC Block Diagram To external host Always ON Two-wire serial I/F slave GPIO Sensor Core 1KB data RAM Register Bus Master DMA Instruction bus Peripheral bus (SFR) MCU Input to IFP Interface IFP Math Coprocessor Sleep Unit 24KB code ROM Register bus (ICB) Memory data bus 1KB patch RAM Output from IFP Interface parallel MT9D115 DS Rev. E Pub. 4/15 EN 10 serial ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture The external host and the integrated microprocessor (MCU) can both access all the internal resources (RAM and registers). The external host always has higher priority. The following sections briefly describe the functionality of each key component of the system. Firmware The firmware implements all automatic camera functions, such as auto exposure (AE), auto white balance (AWB), and flicker detection/avoidance (FD), as well as control functions such as sequencer, mode/context, and histogram (see Table 4). The firmware consists of drivers, generally one driver for each major automatic or control function (see Figure 4). Table 4: List of Drivers ID Figure 4: Driver Name Description ID = 1 Sequencer Controls of camera main function ID = 2 AE Auto exposure ID = 3 AWB Auto white balance ID = 4 Flicker Detection Flicker detection and avoidance ID = 7 Mode/Context Context variables ID = 11 Histogram Reduce image flare and analyze image histogram Firmware Architecture Operation Driver Monitor Mode Control Sequencer Auto White Balance Auto Function Driver Flicker Avoidance Auto Exposure Hardware RAM MT9D115 DS Rev. E Pub. 4/15 EN STAT 11 IFP Core Output ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture External Host Interface (Two-Wire Slave-Only Interface) The MT9D115 will appear as a two-wire serial interface slave to the external host. Its base address is selectable by the external SADDR pin input (when SADDR = 0 then base address = 0x78; when SADDR = 1 then base address = 0x7A). There are 32K addressable 16bit registers (that is, the starting address of each register always falls into even addresses) within the MT9D115 but not all of them are being used (see Figure 5). Figure 5: External Host Register Map S O C 2 re g s 0x3 4 0 0 S O C 1 re g s 0x3 2 1 0 C O R E re g s 0x3 0 1 2 0x1 0 7 0 0x0 9 8C 0x0 1 0 2 0x0 0 0 0 G P IO re g s X D M A re g s R X_S S re g s S Y S C T L re g s 0x0 0 0 0 1K B U se r -lo a d a b le m e m o ry 1K B D rive r V a ria b le s Internal Memory Resource 16-bit The MT9D115 Register Reference provides detailed register explanations. Although most registers are self-explanatory, the next paragraphs contain enhanced information about XDMA registers are worth explaining here. The XDMA registers allow the external host to indirectly access the internal memory resources of the MT9D115, which include the firmware driver variables. To access the variables, use logical accesses provided by the XDMA registers. MT9D115 DS Rev. E Pub. 4/15 EN 12 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture The external host interface is implemented through a two-wire interface that enables direct read/write access to hardware registers and indirect access to firmware variables within the MT9D115. The interface is designed to be compatible with the MIPI alliance standard for Camera Serial Interface 2 (CSI-2) 1.0, which uses the electrical characteristics and transfer protocols of the two-wire serial interface specifications. The interface protocol uses a master/slave model in which a master controls one or more slave devices. The sensor acts as a slave device to the external host which acts as a master device. The master generates a clock (SCLK) that is an input to the sensor and used to synchronize the transactions at the interface. Data is transferred between the master and the slave on a bidirectional serial data bus (SDATA). Both SCLK and SDATA are pulled up to VDD_IO off-chip by a 1.5K resistor. Either the slave or master device can drive SDATA to LOW—the interface determines which device is allowed to drive SDATA at any given time. Figure 6: Two-Wire Serial Control Bus Timing Write Sequence tSDS tSRTS tSCLK tSRTH tSTPS tSHAW tAHSW tSDH tSTPH SCLK SDATA Write Start Write Address Bit 7 Write Address Bit 0 Read Sequence Register Value Bit 0 Register Value Bit 7 Ack Ack Stop tSDSR tSHAR tAHSR tSDHR SCLK SDATA Read Start MT9D115 DS Rev. E Pub. 4/15 EN Read Address Bit 7 Read Address Bit 0 Register Value Bit 7 Ack 13 Register Value Bit 0 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Table 5: Two-Wire Serial Interface Timing Data fEXTCLK = 14 MHz; VDD = 1.8V; VDD_IO = 1.8V; VAA = 2.8V; VAA_PIX = 2.8V; VDD_PLL = 2.8V; VDD_PHY = NA; TJ = 70°C; CLOAD = 68.5pF Symbol f SCLK t SCLK t r t SRTS Parameter Min Typ Max Unit Serial interface input clock frequency Conditions 100 – 400 kHz Serial interface input clock period 2.5 – 10 s SCLK duty cycle 33 50 50 % SCLK/SDATA rise time Start setup time – – 300 ns Master write to slave 600 – – ns t SRTH Start hold time Master write to slave 300 – – ns tSDH SDATA hold Master write to slave 5 – 900 ns tSDS SDATA setup Master write to slave 100 – – ns tSHAW SDATA hold to ack Master write to slave 150 – – ns tAHSW Ack hold to SDATA Master write to slave 150 – – ns tSTPS Stop setup time Master write to slave 300 – – ns tSTPH Stop hold time Master write to slave 600 – – ns tSHAR SDATA hold to ack Master read from slave 300 – – ns tAHSR Ack hold to SDATA Master read from slave 300 – – ns tSDHR SDATA hold Master read from slave 300 – 650 ns tSDSR SDATA setup Master read from slave 300 – – ns tR and tF are dependent on system-level parameters such as the value of pull-up resistor used, how the two-wire serial bus is routed, whether there are other devices on the serial bus, and the strength of the supply used to pull-up the serial bus. Note: Always-On Power Domain The always-on power domain (AOPD) provides an area of functionality that will always be active while power is applied to the MT9D115. The external host interface is located in the AOPD. The domain also includes miscellaneous clock and reset controls as well as configuration registers for the sensor core, processor core, clock configuration, and clock reset control. The user-loadable patch memory is also included in this domain. This memory will remain powered when the main core power is shut down using the standby command. Sensor Core The sensor core of the MT9D115 is a progressive-scan sensor that generates a stream of pixel data at a constant frame rate, qualified by LINE_VALID (LV) and FRAME_VALID (FV). The maximum pixel rate is 30 Mp/s, corresponding to a pixel clock rate of 63.25 MHz. See Figure 7 on page 15 for a block diagram of the sensor core. It includes a 2.0Mp active-pixel array. The timing and control circuitry sequences through the rows of the array, resetting and then reading each row in turn. In the time interval between resetting a row and reading that row, the pixels in the row integrate incident light. The exposure is controlled by varying the time interval between reset and readout. After a row is read, data from the columns are sequenced through an analog signal chain that provides offset correction and gain, and then through an ADC. The output from the ADC is a 10-bit value for each pixel in the array. MT9D115 DS Rev. E Pub. 4/15 EN 14 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture The pixel array contains optically active and light-shielded (dark) pixels. The dark pixels provide data for the offset-correction algorithms (black level control). The sensor core contains a set of control and status registers that can be used to control many aspects of the sensor behavior including the frame size, exposure, and gain settings. These registers are controlled by the firmware and can be accessed through a two-wire serial interface. Register values written to the sensor core can be overwritten by firmware. The output from the core is a Bayer pattern, where alternate rows are a sequence of either green and red pixels or blue and green pixels. The offset and gain stages of the analog signal chain provide per-color control of the pixel data. A flash strobe output signal is provided to allow an external xenon or LED light source to synchronize with the sensor exposure time. Figure 7: Sensor Core Block Diagram Control Registers PLL Timing and Control Active -Pixel Sensor (APS ) Array Gr and Gb Channel Analog Processing Gr and Gb Red and Blue ADC Gr and Gb Red and Blue Digital Processing 10-bit Data Out Red and Blue Channel Sensor Core Pixel Array The sensor core uses a Bayer color pattern (see Figure 8). The even-numbered rows contain green and red pixels. The odd-numbered rows contain blue and green pixels. Even-numbered columns contain green and blue pixels. Odd-numbered columns contain red and green pixels. MT9D115 DS Rev. E Pub. 4/15 EN 15 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Figure 8: Pixel Color Pattern Detail (Top Right Corner) Column Readout Direction .. . Black Pixels First Clear Pixel Row Readout Direction B G2 B G2 B G2 ... Gr R Gr R Gr R B Gb B Gb B G2 Gr R Gr R Gr R B Gb B Gb B G2 Default Readout Order By convention, the sensor core pixel array is shown with pixel (0,0) in the top right corner. This reflects the actual layout of the array on the die. When the sensor is operating in a system, the active surface of the sensor faces the scene (see Figure 9). When the image is read out of the sensor, it is read one row at a time, with the rows and columns sequenced. By convention, data from the sensor is shown with the first pixel read out in the case of the sensor core in the top left corner. Figure 9: Imaging a Scene Lens Scene Sensor (rear view) Row Readout Order Column Readout Order Pixel (0,0) Analog Processing Analog Readout Channel The sensor core features an analog readout channel, (see Figure 7 on page 15). The readout channel consists of a gain stage, a sample-and-hold stage with black level calibration capability, and a 10-bit ADC. MT9D115 DS Rev. E Pub. 4/15 EN 16 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Gain Options The MT9D115 provides per-color gain control as well as the option of global gain control. The per-color and global gain control can be used interchangeably. A WRITE to a global gain register is aliased as a WRITE of the same data to the four associated color-dependent gain registers. Integer digital gains in the range 0–7 can be programmed. A digital gain of 0 sets all pixel values to 0 (the pixel data will simply represent the value applied by the pedestal block). Gain settings are updated in every frame by the MCU auto functions such as AWB, AE, and FD. To make manual adjustments to gain settings, the MCU automatic exposure and automatic white balance adjustment features must be disabled. Integration Time The integration time (exposure) of the MT9D115 is controlled by variables. While coarse integration time controls the integration duration in terms of row times, fine integration time allows for sub-row times accuracy in terms of pixel clocks. Integration time is updated in every frame by the MCU auto feature. Disable the MCU auto features to make manual adjustments to integration time. Because of the basic operation of the Electronic Roller Shutter (ERS), it is not advisable to set an integration time that is greater than the frame time. It is not necessary to reprogram the frame time on the MT9D115 to make longer integration times available because the frame time adjusts automatically. However, long integration times increase the likelihood of image degradation because of increased accumulation of dark current. If the integration time is changed while FV is asserted for frame n, the first frame output using the new integration time is frame (n + 2). The sequence is as follows: 1. During frame n, the new integration time is held in the pending register. 2. At the start of frame (n + 1), the new integration time is transferred to the live register. Integration for each row of frame (n + 1) has been completed using the old integration time. 3. The earliest time that a row can start integrating using the new integration time is immediately after that row has been read for frame (n + 1). 4. When frame (n + 2) is read out, it will have been integrated using the new integration time. If the integration time is changed on successive frames, each value written will be applied for a single frame; the latency between writing a value and it affecting the frame readout remains at two frames. When the integration time and the gain are changed at the same time, the gain update is held off by one frame so that the first frame output with the new integration time also has the new gain applied. External Generated Master Clock If application does not use PLL, then the clock bypass bit in R0x0014 must be set before exiting soft standby state as follows: 1. Write 0x25F9 to R0x0014 to set clock bypass bit 2. Delay min. of 100 ms 3. Write 0x4028 to R0x0018 to exit from soft standby state 4. After successful exit from soft standby state, disable the clock bypass bit by writing 0x21F9 to R0x0014 MT9D115 DS Rev. E Pub. 4/15 EN 17 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture PLL-Generated Master Clock The PLL can generate a master clock signal whose frequency is up to 85 MHz (input clock from 6 MHz through 54 MHz). PLL Setup Because the input clock frequency is unknown, the sensor starts up with the PLL disabled. The PLL takes time to power up. During this time, the behavior of its output clock signal is not guaranteed. The PLL output frequency is determined by two constants, M and N, and the input clock frequency. Fin 2 M VCO = ----------------------------N+1 VCO PLL output frequency = ---------------P1 + 1 (EQ 1) Digital Processing Readout Options The sensor core supports different readout options to modify the image before it is sent to the IFP. The readout can be limited to a specific window of the original pixel array. For preview modes, the sensor core supports both skipping and pixel averaging in x and y directions. By changing the readout direction the image can be flipped in the vertical direction and/or mirrored in the horizontal direction. Window Size The image output size is set with registers x_addr_start, x_addr_end, y_addr_start, and y_addr_end. The edge pixels in the 1600 x 1200 array are present to avoid edge defects and should not be included in the visible window. Binning will change the image output size. Readout Modes Horizontal Mirror When the sensor is configured to mirror the image horizontally, the order of pixel readout within a row is reversed, so that readout starts from x_addr_end and ends at x_addr_start. Figure 10 shows a sequence of 6 pixels being read out with normal readout and reverse readout. The SOC corrects for this change in sensor core output. MT9D115 DS Rev. E Pub. 4/15 EN 18 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Figure 10: Six Pixels in Normal and Column Mirror Readout Modes LINE_VALID Normal readout DOUT[9:0] G0 (9:0) R0 (9:0) G1 (9:0) R1 (9:0) G2 (9:0) R2 (9:0) R2 (9:0) G2 (9:0) R1 (9:0) G1 (9:0) R0 (9:0) G0 (9:0) Reverse readout DOUT[9:0] Vertical Flip When the sensor is configured to flip the image vertically, the order in which pixel rows are read out is reversed, so that row readout starts from y_addr_end and ends at y_addr_start. Figure 11 shows a sequence of six rows being read out with normal readout and reverse readout. The SOC corrects for this change in sensor core output. Figure 11: Six Rows in Normal and Row Mirror Readout Modes FRAME_VALID Normal readout Row0 (9:0) Row1 (9:0) Row2 (9:0) Row3 (9:0) Row4 (9:0) Row5 (9:0) Row5 (9:0) Row4 (9:0) Row3 (9:0) Row2 (9:0) Row1 (9:0) Row0 (9:0) Reverse readout DOUT[9:0] Column and Row Skip The sensor core supports subsampling. Subsampling reduces the amount of data processed by the analog signal chain in the sensor and thereby allows the frame rate to be increased. This reduces the amount of row and column data processed and is equivalent to the skip2 readout mode provided by earlier ON Semiconductor image sensors. Set the proper image output and crop sizes before enabling subsampling. Figure 12: Eight Pixels in Normal and Column Skip 2X Readout Modes LINE_VALID Normal readout DOUT[9:0] G0 (9:0) R0 (9:0) G1 (9:0) R1 (9:0) G0 (9:0) R0 (9:0) G2 (9:0) R2 (9:0) G2 (9:0) R2 (9:0) G3 (9:0) R3 (9:0) LINE_VALID Column skip readout DOUT[9:0] MT9D115 DS Rev. E Pub. 4/15 EN 19 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Pixel Readouts Figures 13 through Figure 16 on page 22 show a sequence of data being read out with no skipping, with x_odd_inc = 3 and y_odd_inc = 1, with x_odd_inc = 1 and y_odd_inc = 3, and with x_odd_inc = 3 and y_odd_inc = 3. Figure 13: Pixel Readout (no skipping) X Incrementing Y Incrementing MT9D115 DS Rev. E Pub. 4/15 EN 20 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Figure 14: Pixel Readout (x_odd_inc = 3, y_odd_inc = 1) X Incrementing Y Incrementing Figure 15: Pixel Readout (x_odd_inc = 1, y_odd_inc = 3) X Incrementing Y Incrementing MT9D115 DS Rev. E Pub. 4/15 EN 21 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Figure 16: Pixel Readout (x_odd_inc = 3, y_odd_inc = 3) X Incrementing Y Incrementing Programming Restrictions when Skipping When skipping is enabled as a viewfinder mode, and the sensor is switched back and forth between full resolution and skipping, keep line_length_pck constant. This allows the same integration times to be used in each mode. When subsampling is enabled, it might be necessary to adjust the x_addr_end and y_addr_end settings. The values for these registers must correspond with rows/ columns that form part of the subsampling sequence. Use the following rules to make the adjustment. remainder = (addr_end – addr_start + 1 AND 4 (EQ 2) if (remainder == 0) addr_end = addr_end – 2 (EQ 3) Table 6 shows the row address sequencing for normal and subsampled (with y_odd_inc = 3) readout. The same sequencing applies to column addresses for subsampled readout. Because the subsampling sequence only reads half of the rows and columns, there are two possible subsampling sequences, depending upon the alignment of the start address. MT9D115 DS Rev. E Pub. 4/15 EN 22 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Table 6: Row Address Sequencing (Sampling) Normal Subsampled Sequence 1 Subsampled Sequence 2 0 1 2 3 4 5 6 7 0 1 No data No data 4 5 No data No data No data No data 2 3 No data No data 6 7 Binning The MT9D115 sensor core supports 2 x 1 and 2 x 2 analog binning (column binning, also called x-binning and row/column binning, also called xy-binning). Binning has many of the same characteristics as subsampling. However, because binning gathers image data from all pixels in the active window, rather than from a subset of pixels, it achieves superior image quality and avoids the aliasing artifacts that can be a characteristic side effect of subsampling. Enable binning by selecting the appropriate subsampling settings (x_odd_inc = 3 and y_odd_inc = 1 for x-binning, x_odd_inc = 3 and y_odd_inc = 3 for xy-binning) and setting the appropriate binning bit in read_mode register. As for subsampling, x_addr_end and y_addr_end might require adjustment when binning is enabled. The effect of the different subsampling settings is shown in Figure 17 and in Figure 18 on page 24. Figure 17: Pixel Readout (x_odd_inc = 3, y_odd_inc = 1, x_bin = 1) X incrementing Y incrementing MT9D115 DS Rev. E Pub. 4/15 EN 23 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Figure 18: Pixel Readout (x_odd_inc = 3, y_odd_inc = 3, x_ybin = 1) X incrementing Y incrementing Binning Limitations Binning requires a different sequencing of the pixel array and imposes different timing limits on the operation of the sensor. In particular, xy-binning requires two READ operations from the pixel array for each line of output data, which has the effect of increasing the minimum line blanking time. As a result, when xy-binning is enabled, some of the programming limits declared in the parameter limit registers are no longer valid. In addition, the default values for some of the manufacturer-specific registers need to be reprogrammed. None of these adjustments are required for x-binning. The sensor must be taken out of streaming mode before switching between binned and non-binned operation. The row addresses for various binning modes are shown in Table 7. Table 7: Row Address Sequencing (Binning) Normal Binning Sequence 1 Binning Sequence 2 0 1 2 3 4 5 6 7 0, 2 1, 3 No data No data 4, 6 5, 7 No data No data No data No data 2, 4 3, 5 No data No data 6, 8 7, 9 Raw Data Format The sensor core image data is read out in a progressive scan. Valid image data is surrounded by horizontal blanking and vertical blanking, (see Figure 19). The amount of horizontal blanking and vertical blanking is programmable. LV is HIGH during the shaded region of the figure. MT9D115 DS Rev. E Pub. 4/15 EN 24 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Figure 19: Valid Image Data P0,0 P0,1 P0,2................................P0,n-1 P0,n P1,0 P1,1 P1,2................................P1,n-1 P1,n 00 00 00 ............. 00 00 00 00 00 00 ............. 00 00 00 HORIZONTAL BLANKING VALID IMAGE Pm-1,0 Pm-1,1.........................Pm-1,n-1 Pm-1,n Pm,0 Pm,1.........................Pm,n-1 Pm,n 00 00 00 ............. 00 00 00 00 00 00 ............. 00 00 00 00 00 00 ..................................... 00 00 00 00 00 00 ..................................... 00 00 00 00 00 00 ............. 00 00 00 00 00 00 ............. 00 00 00 VERTICAL/HORIZONTAL BLANKING VERTICAL BLANKING 00 00 00 ............. 00 00 00 00 00 00 ............. 00 00 00 00 00 00 ................................ 00 00 00 00 00 00 ................................ 00 00 00 Raw Data Timing The sensor core output data is synchronized with the PIXCLK output. When LV is HIGH, one pixel’s data is output on the 10-bit DOUT output bus every PIXCLK period. By default, the PIXCLK signal runs at the same frequency as the master clock, and its falling edges occur one-half of a master clock period after transitions on LV, FV, and DOUT[9:0] (see Figure 20). This allows PIXCLK to be used as a clock to sample the data. PIXCLK is continuously enabled, even during the blanking period. Figure 20: Pixel Data Timing Example LINE_VALID PIXCLK Blanking DOUT0-DOUT9 MT9D115 DS Rev. E Pub. 4/15 EN Valid Image Data P0 (9:0) P1 (9:0) P2 (9:0) 25 P3 (9:0) Blanking P4 (9:0) Pn-1 (9:0) Pn (9:0) ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Input Interface to Image Flow Processor The input interface to the IFP is the front end of the IFP, in which it will choose between the sensor core output or a test pattern generator output. During normal operation, a stream of raw Bayer image data from the sensor is continuously fed into the IFP. For testing purposes, the test generator output is selected. The generator provides a selection of test patterns sufficient for basic testing of the IFP. Program variable (ID=7, Offset=0x66) followed by REFRESH command to the sequencer in order to access different test patterns (see Figure 21). Depending on which test pattern has been selected, the user might need to program additional registers in order to see the intended effects. Figure 21: Available Test Patterns Test Pattern Flat Field mode_common_ mode_settings test_mode (ID=7, Offset=0x66)=2 Color Bar mode_common_ mode_settings test_mode (ID=7, Offset=0x66)=3 Pseudo-Random Horizontal Stripes Example mode_common settings_test_mode (ID=7, Offset=0x66)=1 test_pxl_red (R0x0102) = 0x1ff test_pxl_g1 (R0x0104) = 0x1ff test_pxl_g2 (R0x0106) = 0x1ff test_pxl_blue (R0x0108) = 0x1ff Vertical Ramp Vertical Stripes MT9D115 DS Rev. E Pub. 4/15 EN Registers/Variables mode_common settings_test_mode (ID=7, Offset=0x66)=4 test_pxl_red (R0x0102) = 0x1ff test_pxl_g1 (R0x0104) = 0x17d test_pxl_g2 (R0x0106) = 0x000 test_pxl_blue (R0x0108) = 0x000 mode_common_mode settings_test_mode (ID=7, Offset=0x66)=5 mode_common settings_test_mode (ID=7, Offset=0x66)=6 test_pxl_red (R0x0102) = 0x1ff test_pxl_g1 (R0x0104) = 0x17d test_pxl_g2 (R0x0106) = 0x000 test_pxl_blue (R0x0108) = 0x000 26 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Image Flow Processor Most IFP functions can be controlled directly by the MCU. The external host will control it indirectly through the driver variables. IFP processing can be broken into three different phases: pixel reconstruction, color rendering/statistics collection, and digital scaling/output format. Figure 22 shows a simplified IFP block diagram and the operating color space at each processing phase. Figure 22: IFP Block Diagram Pixel Array ADC Raw Data Bypass Test Pattern Generator Black Level Subtraction Digital Gain Control, Shading Correction RAW 10 Pixel Reconstruction Defect Correction, Nosie Reduction, Color Interpolation 8-bit RGB RGB to YUV Statistics Engine Color Correction Color Kill Color Rendering/Statistics Collection Digital Scaling/Output Format Aperture 8-bit YUV Correction Scaler Gamma Correction (10-to-8 Lookup ) Output Formatting YUV to RGB 8-Bit RGB Output FIFO Output Format MIPI Serial Output Parallel Output Pixel Reconstruction (Lens Shading Correction) First Black Level Subtraction and Digital Gain Image stream processing starts with black level subtraction and multiplication of all pixel values by a programmable digital gain. Both operations can be independently set to separate values for each color channel (R, Gr, Gb, B). Independent color channel digital gain can be adjusted with registers. Independent color channel black level adjustments can also be made. If the black level subtraction produces a negative result for a particular pixel, the value of this pixel is set to “0.” MT9D115 DS Rev. E Pub. 4/15 EN 27 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Shading Correction (SC) Lenses tend to produce images whose brightness is significantly attenuated near the edges. Other factors also cause fixed pattern signal gradients in images captured by image sensors. The cumulative result of all factors is known as image shading. The MT9D115 has an embedded shading correction module that can be programmed to counter the shading effects on each individual R, Gb, Gr, and B color signal. The Correction Function Color-dependent solutions are calibrated using the sensor, lens system, and an image of an evenly illuminated, featureless grey calibration field. The color correction functions can be derived from the resulting image. The correction functions can then be applied to each pixel value to equalize the response across the image as follows: P corrected (row,col)=P sensor (row,col)*f(row,col) (EQ 4) where P are the pixel values and f is the color dependent correction functions for each color channel. Defect Correction The IFP performs continuous defect correction that can mask pixel array defects such as high dark-current (hot) pixels and pixels that are darker or brighter than their neighbors due to photoresponse nonuniformity. The module is edge-aware with exposure that is based on configurable thresholds. The thresholds are changed continuously based on the brightness of the current scene. Noise Reduction Noise reduction can be enabled or disabled. Thresholds can be set through register settings. Color Interpolation and Edge Detection In the raw data stream fed by the sensor core to the IFP, each pixel is represented by a 10-bit integer number, which can be considered proportional to the pixel's response to a one-color light stimulus, red, green, or blue, depending on the pixel's position under the color filter array. Initial data processing steps—up to and including the defect correction—preserve the one-color-per-pixel nature of the data stream. After the defect correction, the data stream must be converted to a three-colors-per-pixel stream appropriate for standard color processing. The conversion is done by an edge-sensitive color interpolation module. The module pads the incomplete color information available for each pixel with information extracted from an appropriate set of neighboring pixels. The algorithm used to select this set and extract the information seeks the best compromise between preserving edges and filtering out high frequency noise in flat field areas. The edge threshold can be set through register settings. Aperture Correction To increase image sharpness, a programmable 2D aperture correction (sharpening filter) is applied to color-corrected image data. The gain and threshold for 2D correction can be defined through variable settings. MT9D115 DS Rev. E Pub. 4/15 EN 28 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Color Rendering/Statistics Collection (Color Correction) Color Correction To achieve good color fidelity of the IFP output, interpolated RGB values of all pixels are subjected to color correction. The IFP multiplies each vector of three pixel colors by a 3 x 3 color correction matrix. The three components of the resulting color vector are all sums of three 10-bit numbers. Because such sums can have up to 12 significant bits, the bit width of the image data stream is widened to 12 bits per color (36 bits per pixel). The color correction matrix can be programmed by the user or automatically selected by the AWB algorithm implemented in the IFP. Ideally, color correction should produce output colors that are independent of the spectral sensitivity and color crosstalk characteristics of the image sensor. The optimal values of the color correction matrix elements depend on those sensor characteristics and on the spectrum of light incident on the sensor. The color correction variables can be adjusted through register settings. Image Cropping By configuring the cropped and output windows to various sizes, different zooming levels (for example 4X, 2X, and 1X) can be achieved. The location of the cropped window is also configurable so that panning is also supported. A separate cropped window is defined for context A and context B. In both contexts, the height and width definitions for the output window must be equal to or smaller than the cropped image. Gamma Correction The gamma correction curve (see Figure 23 on page 30) is implemented as a piecewise linear function with 19 knee points, taking 12-bit arguments and mapping them to 8-bit output. The abscissas of the knee points are fixed at 0, 64, 128, 256, 512, 768, 1024, 1280, 1536, 1792, 2048, 2304, 2560, 2816, 3072, 3328, 3584, 3840, and 4096. The 8-bit ordinates are programmable through IFP registers. The MT9D115 IFP includes a block for gamma correction that can adjust its shape based on brightness to enhance the performance under certain lighting conditions. Two custom gamma correction tables can be uploaded, one corresponding to a brighter lighting condition, the other corresponding to a darker lighting condition. At power-up, the IFP loads the two tables with default values. The final gamma correction table used depends on the brightness of the scene and can take the form of either uploaded tables or an interpolated version of the two tables. A single (non-adjusting) table for all conditions can also be used. MT9D115 DS Rev. E Pub. 4/15 EN 29 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Figure 23: Gamma Correction Curve Gamma Correction 300 Output RGB, 8-bit 250 200 150 0.45 100 50 0 0 1000 2000 3000 4000 Input RGB, 12-bit Color Kill A color kill circuit is included to remove high or low light color artifacts. It affects only pixels whose luminance exceeds a certain preprogrammed threshold. The U and V values of those pixels are attenuated proportionally to the difference between their luminance and the threshold. Digital Scaling/Output Format Special Effects Special effects like negative image, sepia, or black and white can be applied to the data stream at this point. These effects can be enabled and selected by registers. RGB to YUV Conversion For further processing, the data is converted from RGB color space to YUV color space. YUV Color Filter As an optional processing step, noise suppression by one-dimensional low-pass filtering of Y and/or UV signals is possible. A 3- or 5-tap filter can be selected for each signal. Image Scaling The IFP includes a scaler module to ensure that the size of images output by the MT9D115 can be tailored to the needs of all users. When enabled, this module performs rescaling of incoming images—shrinks them to arbitrarily selected width and height without reducing the field of view and without discarding any pixel values. The scaler performs pixel binning—divides each input image into rectangular bins corresponding to individual pixels of the desired output image, averages pixel values in these bins, and assembles the output image from the bin averages. Pixels lying on bin boundaries contribute to more than one bin average; their values are added to bin-wide sums of pixel values with fractional weights. The entire procedure preserves all image information that can be included in the downsized output image and filters out high frequency features that could cause aliasing. Use the image cropping and scaler module together to implement a digital zoom and pan. If the scaler is programmed to output images smaller than images coming from the sensor core, zoom effect can be produced by cropping the latter from their maximum MT9D115 DS Rev. E Pub. 4/15 EN 30 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture size down to the size of the output images. The ratio of these two sizes determines the maximum attainable zoom factor. For example, a 1600 x 1200 image rendered on a 250 x 150 display can be zoomed up to eight times, because 1280/160 = 1024/128 = 8. A panning effect can be achieved by fixing the size of the cropping window and moving it around the pixel array. YUV-to-RGB/YUV Conversion and Output Formatting The YUV data stream emerging from the scaling module can either exit the color pipeline as-is or be converted before exit to an alternative YUV or RGB data format. Color Conversion Formulas Y'U'V' This conversion is ITU-R BT.601 scaled to make YUV range from 0 through 255. This setting is recommended for JPEG encoding and is the most popular. Y = 0.299 R + 0.587 G + 0.114 B (EQ 5) U = 0.564 (B – Y + 128 (EQ 6) V = 0.713 (R – Y + 128 (EQ 7) There is an option where 128 is not added to U'V'. Y'Cb'Cr' Using sRGB Formulas The MT9D115 implements the sRGB standard. This option provides YCbCr coefficients for a correct 4:2:2 transmission. 16 < Y’< 235; 16 < Cb < 240; 16 < Cr < 240; and 0 RGB 255 Note: Y = (0.2126 R + 0.7152 G + 0.0722 B (219 256) + 16 (EQ 8) Cb = 0.5389 (B – Y (224 256) + 128 (EQ 9) Cr = 0.635 (R – Y (224 256) + 128 (EQ 10) Y'U'V' Using sRGB Formulas Similar to the previous set of formulas, but has YUV spanning a range of 0 through 255. Y = 0.2126 R' + 0.7152 G' + 0.0722 B' (EQ 11) U = 0.5389 B' – Y' + 128 = – 0.1146 R' – 0.3854 G' + 0.5 B' + 128 (EQ 12) V = 0.635 (R – Y + 128 = 0.5 R ' – 0.4542 G' – 0.0458 B ' + 128 (EQ 13) There is an option to disable adding 128 to U'V'. The reverse transform is as follows: MT9D115 DS Rev. E Pub. 4/15 EN R = Y + 1.5748 V – 128 (EQ 14) G = Y – 0.1873 (U – 128 – 0.4681 (V – 128) (EQ 15) B = Y + 1.8556 (U – 128) (EQ 16) 31 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Output Interface from IFP The output interface contains parallel image bus, MIPI serial bus interfaces, and walking 1s connectivity test generator. Parallel and MIPI Output The user can select to use either the serial MIPI output or the 10-bit parallel output to transmit the data. Only one of the output modes can be used at any time. The parallel output can be used with an output FIFO whose memory is shared with the MIPI output FIFO to retain a constant pixel output clock independent from the scaling factor. When scaling the image or skipping lines, the data would be generated in bursts and the pixel clock would turn on and off in intervals, which might lead to EMI problems. The output FIFO will group all active pixel data together so the pixel clock can be run at a constant speed. The MIPI output transmitter implements a serial differential sub-LVDS transmitter capable of up to 512 Mb/s. It supports multiple formats, error checking, and custom short packets. The MIPI clock is defined as half the data rate frequency. The virtual channel could be used by host MIPI RX circuits to differentiate between preview and capture image data if the FW changes the channel number when switching between contexts. The host cannot adequately time this switching of contexts and so cannot use channel number in this way without FW support. The hardware supports a configurable MIPI virtual channel in the MIPI Control register (R0x3400). Firmware provides two variables that allow the MIPI virtual channel to be changed according to context (preview/A, capture/B). The host can specify what channel is used for which context through these variables. Table 8: Data Formats Supported by MIPI Interface Note: MT9D115 DS Rev. E Pub. 4/15 EN Data Format Data Type YUV 422 8-bit 565RGB 555RGB 444RGB RAW8 RAW10 0x1E 0x22 0x21 0x20 0x2A 0x2B Data will be packed as RAW8 if the data type specified does not match any of the above data types. 32 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Output Format and Timing YUV/RGB Output YUV or RGB data can be output either directly from the output formatting block or through a FIFO buffer with a capacity of 1024 bytes. This size is large enough to hold one-fourth of a scan line at full resolution. Buffering of data is a way to equalize the data output rate when image scaling is used. Scaling produces an intermittent data stream consisting of short high-rate bursts separated by idle periods. High pixel clock frequency during bursts may be undesirable due to EMI concerns. Figure 24 shows the output timing of a YUV/RGB scan line when a scaled data stream is equalized by buffering or when no scaling takes place. The pixel clock frequency remains constant during each LV high period. Scaled data is output at a lower frequency than full size frames, which helps to reduce EMI. Figure 24: Timing of Full Frame Data or Scaled Data Passing Through the FIFO YUV/RGB Data Ordering The MT9D115 supports swapping YCbCr mode (see Table 9). Table 9: YCbCr Output Data Ordering Mode Data Sequence Default (no swap) Swapped CrCb Swapped YC Swapped CrCb, YC Yi Yi Cbi Cri Cbi Cri Yi Yi Cri Cbi Yi+1 Yi+1 Yi+1 Yi+1 Cri Cbi The RGB output data ordering in default mode is shown in Table 10. The odd and even bytes are swapped when luma/chroma swap is enabled. R and B channels are bitwise swapped when chroma swap is enabled. Table 10: RGB Ordering in Default Mode MT9D115 DS Rev. E Pub. 4/15 EN Mode (Swap Disabled) Byte D7D6D5D4D3D2D1D0 565RGB Odd Even R7R6R5R4R3G7G6G5 G4G3G2B7B6B5B4B3 33 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Architecture Table 10: RGB Ordering in Default Mode (continued) Mode (Swap Disabled) Byte D7D6D5D4D3D2D1D0 555RGB Odd Even Odd Even Odd Even 0 R7R6R5R4R3G7G6 G4G3G2B7B6B5B4B3 R7R6R5R4G7G6G5G4 B7B6B5B4 0 0 0 0 0 0 0 0 R7R66R5R4 G7G6G5G4B7B6B5B4 444xRGB x444RGB 10-Bit Bypass Output Raw 10-bit Bayer data from the sensor core can be output in bypass mode in two ways: 1. Using eight data output signals (DOUT[7:0]) and GPIO[1:0]. The GPIO signals are the lowest two bits of data. 2. Using only eight signals (DOUT[7:0]) and a special 8 + 2 data format, shown in Table 11. Table 11: 2-Byte RGB Format Byte Bits Used Bit Sequence Odd bytes 8 data bits D9D8D7D6D5D4D3D2 Even bytes 2 data bits + 6 unused bits 0 0 0 0 0 0 D 1D 0 FIFO During normal pipeline operation, the output data rate is determined by a number of factors: input image size, degree of scaling, and sensor operation mode. As these parameters change during normal sensor operation, output frequency changes. This output frequency may generate RF noise, interfering with the mobile device. By using an output FIFO to maintain a constant output clock frequency, noise is easily filtered out. The FIFO accumulates data and after a certain number of bytes are stored, it will output them in a single burst, making sure that the data rate within the burst remains constant. This approach utilizes a free-running clock, thus making possible minimal RF interference. MT9D115 DS Rev. E Pub. 4/15 EN 34 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Control Functions Control Functions Sequencer: Camera Operating System The sequencer is a finite state machine that controls the general operations of the camera and switching between operating modes. Camera operation is organized in states such as enter preview, preview, leave preview, enter capture, and capture. The sequencer carries out a number of commands such as run, go preview, go capture, refresh, and refresh mode. The current state of the sequencer is indicated in a variable seqr.state. To execute a command, the user must set a particular command number in the variable seqr.cmd. ON Semiconductor recommends that the external host monitor seqr.state to know when to change resolution or capture frames. Each state has its configuration; therefore, the user should set up the state configuration to customize the camera configuration before executing the corresponding program such as GO TO CAPTURE. A typical camera operating scenario is: 1. Configure mode variables after hardware reset. 2. Configure preview mode. 3. Execute sensor REFRESH program. 4. Run in preview until shutter button is pressed. 5. Capture a frame. MT9D115 DS Rev. E Pub. 4/15 EN 35 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Control Functions Figure 25: Sequencer Finite State Machine Refresh or Run PREVIEW Go capture ENTER PREVIEW LEAVE PREVIEW 0X0018[2] = 0 Refresh mode 0X0018[2] = 1 POR STANDBY INIT Go preview CHG MODE TO PREVIEW Go preview STANDBY CHG MODE TO CAPTURE Refresh mode STANDBY STANDBY Go capture 1 ENTER CAPTURE LEAVE CAPTURE Go preview or Go capture CAPTURE Refresh or Run Note: Any state except INIT can transition to Standby state by either hard standby or soft standby. Mode: Context Information The mode driver reduces integration efforts by managing most aspects of switching the two contexts. It remembers vital register values for each image acquisition context and loads these values to the appropriate registers in the IFP upon context switching. For the mode driver variables to take effect, the user changes the variable values in the mode driver (ID = 7). Upon the next mode change or sequencer's REFRESH command, these driver variable values will be loaded to the appropriate physical sensor core and IFP registers. The image processing will take the new values at the beginning of the next frame acquired. To control the output image size, the user can modify the mode driver variables such as output_width_A, output_height_A, output_width_B, and output_height_B. The mode driver will automatically apply any appropriate downscaling filter to achieve this output image size as well as update the watermark of the output FIFO. It is important to set up the sensor core to output an image equal to or larger than the crop window size, which in turn is equal to or larger than the desired output image size. MT9D115 DS Rev. E Pub. 4/15 EN 36 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Camera Functions Camera Functions Simple Rule-based Auto Exposure The AE algorithm performs automatic adjustments of the image brightness by controlling exposure time and analog gains of the sensor core as well as digital gains applied to the image. Auto exposure is implemented by a firmware (FW) driver that analyzes image statistics collected by the exposure measurement engine, makes a decision, and programs the sensor core and color pipeline to achieve the desired exposure. The measurement engine subdivides the image into 16 windows organized as a 4 x 4 grid. An AE algorithm mode is available: average brightness tracking (ABT). The ABT AE uses a constant average tracking algorithm where a target brightness value is compared to a current brightness value, and the gain and integration time are adjusted accordingly to meet the target requirement. AE Driver DRT exposure mode is activated during preview. This mode can also be enabled during video capture mode. The DRT exposure mode relies on the statistics engine to track speed and amplitude of the change of the overall luminance in the selected windows of the image. Backlight compensation is achieved by weighting the luminance in the center of the image higher than the luminance on the periphery. Other algorithm features include the rejection of fast fluctuations in illumination (time averaging), control of speed of response, and control of the sensitivity to the small changes. While the default settings are adequate in most situations, the user can program target brightness, measurement window, and other parameters. The driver calculates image brightness based on average luma values received from 16 programmable equal-size rectangular windows forming a 4 x 4 grid. In preview mode, 16 windows are combined in two segments: central and peripheral. The central segment includes four central windows. All remaining windows belong to the peripheral segment. Scene brightness is calculated as average luma in each segment taken with certain weights. The driver changes AE parameters (integration time, gains, and so on) to drive brightness to the programmable target. The value of the single step approach to the target value can be controlled. To avoid unwanted reaction of AE on small fluctuations of scene brightness or momentary scene changes, the AE driver uses a temporal filter for luma and a threshold around the AE luma target. The driver changes AE parameters only if the filtered luma is larger than the AE target step and pushes the luma beyond the threshold. Evaluative Algorithm A scene-evaluative AE algorithm is available for use in snapshot mode. The algorithm performs scene analysis and classification with respect to its brightness, contrast, and composure and then decides to increase, decrease, or keep the original exposure target. It makes the most difference for backlight and bright outdoor conditions. MT9D115 DS Rev. E Pub. 4/15 EN 37 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Camera Functions Accelerated Settling During Overexposure The AE speed is direction-dependent. Transitioning from overexposure to target can take more time than transitioning from underexposure. The AE driver has a mode that speeds up AE for overexposed scenes. The AE driver counts the number of AE windows whose average brightness is equal to or greater than some value, 250 by default. For a scene that has saturated regions, the average luma is underestimated because of signal clipping. The driver compensates underestimation by a factor that can be defined. Exposure Control To achieve the required amount of exposure, the AE driver adjusts the sensor integration time, gains, ADC reference, and IFP digital gains. In addition, ae_base_target (ID = 2, offset = 0x4F) is available for the user to adjust the overall brightness of the scene. Flicker Detection and Avoidance Flicker occurs when the integration time is not an integer multiple of the period of the light intensity. The automatic flicker detection block does not compensate for the flicker, but rather avoids it by detecting the flicker frequency and adjusting the integration time. To reject flicker, integration time is typically adjusted in increments of steps. The incremental step specifies the duration in row times equal to one flicker period. Thus, flicker is rejected if integration time is kept to an integral factor of the flicker period. Flicker cannot be avoided for integration times below the light intensity period (10ms for 50Hz environment). Flicker shows up in the image as horizontal bars that roll up or down. The MCU looks for these rolling bars using a thin horizontal window, which outputs luma average and is applied to 48 points in the upper half of the image. The MCU repeats the same sampling on the next frame; 48 samples from the previous frame are subtracted from corresponding samples from the current frame. Skipping more frames between subtraction can be set by a variable. The MCU then smooths the 48 sampling points, applies an amplitude threshold to avoid false detection, and looks at the resulting waveform. If flicker is present, the waveform should have a frequency within the search range. Assuming the flicker power is a sine wave, subtracting two frames results in: sin(wt) – sin(wt + a) = 2 sin(a 2) cos(wt + (a 2 (EQ 17) which is a cosine wave of the original frequency. MT9D115 DS Rev. E Pub. 4/15 EN 38 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Camera Functions Auto White Balance The MT9D115 has a built-in AWB algorithm designed to compensate for the effects of changing spectra of the scene illumination on the quality of the color rendition. The algorithm consists of two major parts: a measurement engine performing statistical analysis of the image and a driver performing the selection of the optimal color correction matrix, digital gains, and sensor core analog gains. While default settings of these algorithms are adequate in most situations, the user can reprogram the base color correction matrices, place limits on color channel gains, and control the speed of both matrix and gain adjustments. Flash To take a snapshot, the user must send a command that changes the context from preview to snapshot. The typical sequence of events after this command is: 1. The camera might turn on its LED flash, if it has one and is required to use it. With the flash on, the camera exposure and white balance are automatically adjusted to the changed illumination of the scene. 2. The camera captures one or more frames of desired size. A camera equipped with a xenon flash strobes while capturing images. When the images are captured, the camera automatically returns to context A and resumes running in preview mode. Note: This sequence of events can take up to 10 frames. Histogram: Dark Level Adjustments, Low Light and Tonal Controls The histogram driver continually works to reduce image flare and continually analyzes input image histogram and dynamically adjusts the black level. When flare is present (hence, the histogram does not contain dark tones), it causes the driver to subtract a higher black level, thus regaining the lost contrast. In certain situations, the scene may contain no dark tones without flare. The histogram driver cannot distinguish this situation and alters the black level just the same, causing the image to have more contrast, which looks acceptable in many situations. Besides black level adjustments and low light scene parameters, this driver also contains variables for the preloaded and custom gamma tables. MT9D115 DS Rev. E Pub. 4/15 EN 39 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Optics Optics Figure 26 shows the CRA versus image height. Figure 26: CRA vs. Image Height Image Height CRA vs. Image Height Plot 30.00 28.00 26.00 24.00 22.00 20.00 CRA (deg) 18.00 16.00 14.00 12.00 10.00 8.00 6.00 4.00 2.00 0.00 0 10 20 30 40 50 60 70 80 Image Height (%) Note: MT9D115 DS Rev. E Pub. 4/15 EN 90 100 110 CRA (%) (mm) (deg) 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 0.000 0.088 0.175 0.263 0.350 0.438 0.525 0.613 0.700 0.788 0.875 0.963 1.050 1.138 1.225 1.313 1.400 1.488 1.575 1.663 1.75 0.00 2.22 4.39 6.54 8.68 10.79 12.86 14.87 16.78 18.56 20.17 21.59 22.79 23.74 24.43 24.85 25.02 24.96 24.71 24.31 23.85 ON Semiconductor recommends a 670nm IR cut filter to achieve the best image quality; however, a 650nm IR cut filter is acceptable. 40 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Power Modes Power Application Sequence ON Semiconductor recommends the following sequence to maintain low power consumption during this process: Caution Applying power to analog supplies prior to applying digital and IO supplies or any other failure to follow the correct power up sequence may result in high current consumption and die heating. This can potentially result in performance and reliability issues. 1. Ensure that STANDBY is de-asserted and RESET_BAR is asserted. 2. Apply IO supply (VDD_IO) to the image sensor and wait for the IO supply to be stable. 3. Minimum of 1 ms after IO supply is stable, apply digital supply to the image sensor and wait. 4. Enable EXTCLK and wait for the EXTCLK signal to stabilize. 5. De-assert RESET_BAR for a minimum of 10 EXTCLK cycles. 6. After asserting the RESET_BAR, apply analog supplies (VAA, VAA_PIX, and VDD_PLL) to the image sensor. 7. After 6000 EXTCLK cycles from the end of step 6, the image sensor will be in soft standby state. 8. Communication with the sensor thorough two-wire serial interface can start 1 EXTCLK after step 7. In cases where the recommended procedure cannot be followed, the following condition would affect the sensor’s power consumption during the power application sequence: • When analog supplies are applied prior to the digital and IO supplies, high current consumption on the analog supplies may be present. MT9D115 DS Rev. E Pub. 4/15 EN 41 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Figure 27: Power Application Sequence Timing t0 STANDBY t4 RESET_BAR VDD_IO t1 t3 VDD t2 EXTCLK t5 VAA_, VAA_PIX, VDD_PLL t7 t6 Two-wire serial bus Table 12: Power Application Sequence Timing Parameters Symbol Min Typ Max Units Delay from stable RESET_BAR, STANDBY signals to VDDIO power start t0 0 – – ns Delay from stable VDD_IO to VDD start t1 1 – – ms Delay from stable VDD power to EXTCLK start t2 0 – – ns Delay from EXTCLK start to stable EXTCLK t3 1 – – EXTCLK RESET_BAR pulse width t4 10 – – EXTCLK Delay from RESET_BAR de-asserting to analog power supplies start t5 1 – – EXTCLK Delay from analog power stable to soft standby mode t6 6000 – – EXTCLK Delay from soft standby mode to first two-wire bus transaction t7 1 – – EXTCLK Power-On Reset The sensor includes a power-on reset feature that initiates a reset upon power-up. Even though this feature is included on the device, it is advised that the user still manually assert a hard reset upon power-up. The MT9D115’s POR circuit generates internal reset only and it will not generate the external reset signal through RESET_BAR. The POR circuit requires VDD ramp time to be less than 10µs. If the ramp time is longer than 10µs, the POR operation is not guaranteed and external reset must be used. The POR circuit will generate an internal reset when VDD falls below 1.25V (typical) for 1µs (typical) period, as shown in Figure 28 and described in Table 13. MT9D115 DS Rev. E Pub. 4/15 EN 42 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes The POR circuit reset and external RESET_BAR signals are gated together to generate an internal reset for MT9D115. Figure 28: Internal Power-On Reset t1 VTRIG_RISING VDD VTRIG_FALLING Table 13: POR Parameters Symbol t1 Definition Minimum VDD spike width below VTRIG_FALLING considered to be a reset Min Typ Max Unit – 1 – s VTRIG_RISING VDD rising trigger voltage 1.15 1.4 1.55 V VTRIG_FALLING VDD falling trigger voltage 1 1.25 1.45 V MT9D115 DS Rev. E Pub. 4/15 EN 43 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Hard Reset The MT9D115 enters the reset state when the external RESET_BAR signal is asserted LOW, as shown in Figure 29 and described in Table 14. All of the output signals will be in High-Z state except the MIPI outputs, which will be driven LOW. Figure 29: Hard Reset Operation t4 t1 t3 t2 EXTCLK RESET_BAR SDATA All Outputs Mode Table 14: Data Active After Programming by Host Processor Data Active Reset Entering Standby Mode and Two-wire serial interface is Ready Internal Boot Time Hard Reset Symbol Definition Min t1 RESET_BAR pulse width 100 – – t2 Active EXTCLK required after RESET_BAR asserted 10 – – t3 Active EXTCLK required before RESET_BAR de-asserted 10 – – t4 Maximum internal boot time – – 6000 MT9D115 DS Rev. E Pub. 4/15 EN 44 Typ Max Unit EXTCLK cycles ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Soft Reset The host processor can reset the MT9D115 using the two-wire serial interface by writing to SYSCTL 0x001A. Two types of soft reset are available. SYSCTL 0x001A[0] is used to reset the MT9D115, which is similar to external RESET_BAR signal. 1. Set SYSCTL 0x001A1:0] to 0x3 to initiate internal reset cycle. 2. Wait 6000 EXTCLK cycles. 3. Reset SYSCTL 0x001A[0] to 0x0 for normal operation. Figure 30: Soft Reset Operation t1 EXTCLK SCLK SDATA Mode Table 15: Write Soft Reset Command Registers Reset to Default Values Reseting Registers Soft Reset Signal Timing Symbol t1 MT9D115 DS Rev. E Pub. 4/15 EN Parameter Maximum soft reset time 45 Min Typ Max Unit – – 6000 EXTCLKs ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Power Removal Sequence ON Semiconductor recommends using the following method to remove the power supplies from the image sensor. 1. Put the image sensor into either hard standby or soft standby mode as described in “Standby Modes” on page 47. 2. Remove all digital and analog supplies. During step 2, the digital and analog supplies can be removed safely either one by one or at the same time, provided that step 1 is executed previously. To achieve a known power down state in the sensor, execute step 1 before removing any power supplies during the power removal process. Figure 31: Recommended Power Down Sequence t1 t2 t3 EXTCLK STANDBY VAA VDD VDD_IO STANDBY ASSERTED Table 16: Symbol STANDBY MODE Remove power supplies Power Down Signal Timing Parameter Min Typ Max Unit 1 frame + 40 – – EXTCLKs Active EXTCLK required after STANDBY asserted 10 – – EXTCLKs Power Supply Removal can be in any order and anytime after completion of t2 0 – – EXTCLKs t1 Standby entry complete t2 t3 MT9D115 DS Rev. E Pub. 4/15 EN 46 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Standby Modes The MT9D115 supports three standby modes: • Soft standby • Low leakage hard standby • High leakage hard standby For each mode, entry can be overridden by programming the standby_control register. Patch codes, PLL configurations, parallel and MIPI IO selections are retained in all standby modes. Soft Standby Soft standby can be enabled by register access; it disables the sensor core and most of the digital logic. The two-wire serial interface is still active and the MT9D115 can be programmed through register commands. All register settings and RAM content will be preserved. Soft standby can be performed in any sequencer state. Low Leakage Hard Standby Hard standby mode uses the STANDBY signal to shut down digital power (VDD) and ensure the lowest power consumption. All the two-wire serial interface settings and firmware variables except patch codes, PLL configuration, parallel/MIPI IO and context selections will be lost in this mode. Starting up from this mode is equivalent to power up and current context selections. The two-wire serial interface will be inactive and the sensor must be started up by de-asserting the STANDBY signal. High Leakage Hard Standby (Without Loss of Variable Data) High leakage hard standby mode (without the loss of variable data) can also be achieved. This mode stores the variables and state of the sensor before entering standby (similar to soft standby). The power consumption is lower than that of soft standby, with EXTCLK enabled, as internal clocks are turned off, and the two-wire serial interface will be inactive. Because the high leakage hard standby mode (without the loss of variable data) is also activated by STANDBY, the en_vdd_dis_soft register needs to be programmed to indicate the selection of this mode before STANDBY is asserted. De-asserting STANDBY will cause the sensor to come out of the standby mode. This also causes the sensor to resume operation from the state before the STANDBY signal was asserted. By default, asserting the STANDBY signal causes the hard standby mode described above. MT9D115 DS Rev. E Pub. 4/15 EN 47 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Hard Standby Mode The MT9D115 can enter hard standby mode by using external STANDBY signal, as shown in Figure 32. EXTCLK can be stopped to reduce the power consumption during hard standby mode. Two-wire serial interface and IFP block shut down even when EXTCLK is running during hard standby mode. Entering Standby Mode 1. Assert STANDBY signal (HIGH). Exiting Standby Mode 1. De-assert STANDBY signal (LOW). 2. Follow “Hard Reset” on page 44. Figure 32: Hard Standby Mode Operation t4 t1 t2 t3 EXTCLK STANDBY STANDBY ASSERTED Note: Table 17: Symbol STANDBY MODE EXTCLK Disabled EXTCLK Enabled In hard standby mode, EXTCLK is automatically gated off, and the two-wire serial interface is not active. Hard Standby Signal Timing Parameter Min Typ Max Unit 1 Frame + 40 – – EXTCLKs t1 Standby entry complete t 2 Active EXTCLK required after STANDBY asserted 10 – – EXTCLKs t 3 Active EXTCLK required before STANDBY de-asserted 10 – – EXTCLKs t 4 STANDBY time 1 Frame + 120 – – EXTCLKs t 5 Active EXTCLK required before RESET_BAR asserted 10 – – EXTCLKs MT9D115 DS Rev. E Pub. 4/15 EN 48 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Soft Standby Mode The MT9D115 can enter soft standby mode by writing to a SYSCTL register through the two-wire serial interface, as shown in Figure 33. EXTCLK can be stopped to reduce the power consumption during soft standby mode. However, since two-wire serial interface requires EXTCLK to operate, ON Semiconductor recommends that EXTCLK run continuously. If EXTCLK needs to be stopped, ON Semiconductor recommends using hardware standby mode. Entering Standby Mode 1. SYSCTL 0x0018[3] must be set to “1” to activate standby mode. This will generate an interrupt to the MCU when SYSCTL 0x0018[0] is set to “1.” 2. Set SYSCTL 0x0018[0] to “1” to initiate standby mode. 3. Check until SYSCTL 0x0018[14] changes to “1” to indicate MT9D115 is in standby mode. Exiting Standby Mode 1. Reset SYSCTL 0x0018[0] to “0.” 2. Check until SYSCTL 0x0018[14] changes to “0” to indicate the MT9D115 is out of standby mode. Figure 33: Soft Standby Mode Operation t4 t1 t2 t3 EXTCLK SDATA R0x0018[0] Mode Set R0x0018[0] = 1 Poll Standby R0x0018[14] Mode EXTCLK Disabled EXTCLK Enabled R0x0018[14] = 1 Table 18: Symbol Soft Standby Signal Timing Parameter Min Typ 1 Frame + 40 – Active EXTCLK required after soft standby activates 10 – – EXTCLKs t3 Active EXTCLK required before soft standby deactivates 10 – – EXTCLKs t4 Minimum standby time 1 Frame + 120 – – EXTCLKs t1 Standby entry complete (R0x18[14] = 1) t2 MT9D115 DS Rev. E Pub. 4/15 EN 49 Max Unit EXTCLKs ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Power Modes Table 19: Status of Signals During Different States Signal Reset Post-Reset DOUT[7:0] High-Z High-Z PIXCLK LV FV DOUT_N DOUT_P CLK_N CLK_P GPIO[3:0] High-Z High-Z High-Z 0 0 0 0 High-Z High-Z High-Z High-Z 0 0 0 0 High-Z SADDR SDATA SCLK Input Input Input Input I/O Input Note: MT9D115 DS Rev. E Pub. 4/15 EN Software Standby Low Power Hard Standby High-Z by default (configurable through OE_BAR High-Z by default or two-wire serial interface register) High-Z by default (configurable) High-Z by default High-Z by default (configurable) High-Z by default High-Z by default (configurable) High-Z by default 0 0 0 0 0 0 0 0 Depending on how Depending on how the system uses them as the system uses them DOUT_LSB1/DOUT_LSB0/FLASH/OE_BAR as DOUT_LSBs/FLASH/ OE_BAR Input Input Input Input Input Input Power Down X X X X X X X X X X = “Don’t Care.” 50 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Other Features Other Features One-time Programmable Memory (OTPM) The MT9D115 contains two bytes of OTPM, suitable for storing module identification that can be programmed during the module manufacturing process. Programming the OTP memory requires the use of a high voltage at the VPP pin. During normal operation, the VPP pin should be left floating. The OTPM can be accessed through the two-wire serial interface. Programming the OTPM Refer to TN-09-189: Programming OTP Memory to program the OTP Memory. Sequence of Signals for OTPM Operation Power Supplies RESET_BAR EXTCLK SCLK / SDATA VPP Information to be programmed to the register Initiate programming and poll status bit Read programmed values for status Reading the OTPM Reading the OTPM data requires the sensor to be fully powered and operational with its clock input applied. The data can be read through a register from the two-wire serial interface. MT9D115 DS Rev. E Pub. 4/15 EN 51 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Other Features GPIO and Output Enable Controls General Purpose I/Os The four GPIOs of the MT9D115 can be configured in multiple ways. Each of the I/Os can be used as a simple input/output that can be programmed from the host. The status of the GPIO is read at power up and can be used as a module ID to identify different module suppliers. In addition, module ID can be stored in the OTP memory of the sensor. Information on the OTP memory can be found in “One-time Programmable Memory (OTPM)” on page 51. If 10-bit RAW output is required, GPIO[1:0] can be configured as bit 1 and bit 0 (the LSBs) of a 10-bit data bus. GPIO[2] can be configured to output a flash pulse to trigger an external xenon or LED flash or a shutter pulse to control an external shutter. GPIO[3] can also be configured as an input to be used as an OE_BAR signal for the data bus. The general purpose inputs are enabled or disabled through register settings. Once enabled, all four inputs must be driven to valid logic levels by external signals. The state of the general purpose inputs can be read from a register. Output Enable Control When the parallel pixel data interface is enabled, its signals can be switched asynchronously between being driven and High-Z under signal or register control. MT9D115 DS Rev. E Pub. 4/15 EN 52 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor GPIO Control GPIO Control The MT9D115 has four general purpose I/O (GPIO) signals and that can be individually programmed to perform various functions. Table 20 describes the GPIO registers and variables. Overview of GPIO Signals • Extension of two lower data bits during 10-bit output mode • External flash control output • External OE_BAR control input for parallel port signals Table 20: GPIO Related Registers and Variables Map Address Bits Descriptions SYSCTL 0x001A [8] GPIO[3] Enable. GPIO[3] can be configured as an Output Enable pad. 0: GPIO[3] does not function as OE. 1: GPIO[3] functions as OE. SYSCTL 0x001E [6:4] Controls the output slew rate of GPIO signals. “111” programs the fastest slew rate. Refer to AC/DC specification in the data sheet for the numbers. Default value is “000” for slowest slew rate. SYSCTL 0x0024 [3:0] The state of GPIO signals during power on. Read-only. GPIO 0x1070 [12:9] GPIO data port. State of current GPIO signals can be read through this register. GPIO 0x1074 [12:9] GPIO Set command. When GPIO is configured as output, setting this register to “1” will write “1” to GPIO port. Use GPIO Clear command to clear. GPIO 0x1076 [12:9] GPIO Clear command. When GPIO is configured as output, setting this register to “1” will write “0” to GPIO port. Use GPIO Set command to set. GPIO 0x1078 [12:9] GPIO direction control for each GPIO signals. 0 = Output 1 = Input After power-on, GPIO ports are in input mode. MT9D115 DS Rev. E Pub. 4/15 EN 53 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Electrical Specifications Electrical Specifications Absolute Maximum Rating Caution Table 21: Table 21 shows stress ratings only, and functional operation of the device at these or any other conditions above those indicated in the product specification is not implied. Stresses above those listed may cause permanent damage to the device. Exposure to absolute maximum rating conditions for extended periods may affect device reliability. Maximum Rating Rating Symbol Parameter Min Max Unit VDD_MAX Core digital voltage –0.3 2.4 V VDD_IO_MAX I/O digital voltage –0.3 4.0 V Analog voltage –0.3 4.0 V Pixel supply voltage –0.3 4.0 V VDD_PLL_MAX PLL supply voltage –0.3 4.0 V VIH_MAX Input HIGH voltage –0.3 VDD_IO + 0.3 V VAA_MAX VAA_PIX_MAX VIL_MAX Input LOW voltage –0.3 – V T_OP Operating temperature (measured at junction) –30 75 °C T_ST Storage temperature –40 85 °C I/O Parameters Table 22: I/O Parameters fEXTCLK = 6–54 MHz, VDD = VDDIO_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; T = 25°C j Symbol Parameter Min VIH Input logic HIGH threshold Typ Max Unit Note VDD_IO – 0.3 – VDD_IO + 0.3 V – – – 0.6 0.3 – V V V VDD_IO = 2.8V VDD_IO = 1.8V At specified at IOH VIL Input logic LOW threshold VOH Output logic HIGH threshold – 0.1 – 0.1 VDD_IO – 0.3 VOL Output logic LOW threshold – – 0.3 V – CIN Input pins - (SCLK, SDATA) capacitance – 3.5 3.9 pF – CLOAD Output pins - (SDATA) load capacitance – – 30 pF – PUP_SDATA SDATA pull-up resistor – 1.5 – k – Note: MT9D115 DS Rev. E Pub. 4/15 EN VIH and VIL specifications apply to the over- and undershoot (ringing) present in the MCLK. 54 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Electrical Specifications MIPI Interface AC and DC Electrical Specifications Introduction The MIPI interface for the MT9D115 image sensor was designed to meet the MIPI Alliance Specification for D-PHY Version 1.0. Please refer to this document for details on the MIPI specification and usage. Electrical Specifications Table 23: MIPI High-Speed Transmitter DC Characteristics fEXTCLK = 48 MHz; VDD = VDD_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; Ambient temperature Parameter Description HS transmit static common mode voltage VCMTX |VCMTX (1,0)| VCMTX mismatch when output is Differential-1 or Differential-0 |VOD| HS transmit differential voltage |VOD| VOD mismatch when output is Differential-1 or Differential-0 VOHHS HS output high voltage Min Nom Max Units 150 200 250 mV 16 mV 200 270 mV 14 mV 140 ZOS Single-ended output impedance ZOS Single-ended output impedance mismatch 39 50 360 mV 64 Ohm 10 % . MIPI High-Speed Transmitter AC Characteristics Table 24: fEXTCLK = 48 MHz; VDD = VDD_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; Ambient temperature Parameter Description Min tR and tF 20%-80% rise time and fall time 150 Table 25: Nom Max Units ps MIPI Low-Power Transmitter DC Characteristics fEXTCLK = 48 MHz; VDD = VDD_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; Ambient temperature Parameter Description Min Nom Max Units VOH VOL ZOLP Thevenin output high level Thevenin output low level Output impedance of LP transmitter 1.1 -50 110 1.2 1.4 50 V mV Ohm Table 26: MIPI Low-Power Transmitter AC Characteristics fEXTCLK = 48 MHz; VDD = VDD_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; Ambient temperature Parameter Description TRLP/TFLP TREOT V/tSR CLOAD 15%-85% rise time and fall time 30%-85% rise time and fall time Slew rate @ CLOAD = 70pF Load capacitance MT9D115 DS Rev. E Pub. 4/15 EN Min 55 83 0 Nom Max Units 25 35 122 70 ns ns mV/ns pF ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Electrical Specifications AC Electricals Figure 34: Output Interface Timing Waveforms Table 27: AC Electricals fEXTCLK = 6–54 MHz, VDD = VDDIO_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; T = 25° j Symbol Parameter fEXTCLK tr tf DCEXTCLK tJITTER tCP External input clock frequency External input clock rise time External input clock fall time External input clock duty cycle External input clock jitter EXTCLK to PIXCLK propagation delay Pixel clock frequency Pixel clock rise time Pixel clock fall time Pixel clock to data valid Pixel clock to frame valid high Pixel clock to frame valid low Pixel clock to line valid high Pixel clock to line valid low Programmable slew = 7 fPIXCLK tRPIXCLK tFPIXCLK tPD tPFH tPLH tPFL tPLL PIXCLK pin slew rate Programmable slew = 4 Programmable slew = 0 MT9D115 DS Rev. E Pub. 4/15 EN Conditions From10% to 90% of Vp-p From10% to 90% of Vp-p Peak-to-peak Cload = 15pF Cload = 15pF VDD_IO = 2.8V, Cload = 45pF VDD_IO = 1.8V, Cload = 45pF VDD_IO = 2.8V, Cload = 45pF VDD_IO = 1.8V, Cload = 45pF VDD_IO = 2.8V, Cload = 45pF VDD_IO = 1.8V, Cload = 45pF 56 Min Typ Max Unit Note 6 40 5 2 2 50 - 54 5 5 60 500 45 MHz ns ns % ps ns 2 1 1 6 - 2 2 1.2 0.6 1 0.5 0.3 0.15 85 5 5 0.6*PIXCLK 0.6*PIXCLK 0.6*PIXCLK 0.6*PIXCLK 0.6*PIXCLK - MHz ns ns ns ns ns ns ns V/ns V/ns V/ns V/ns V/ns V/ns 3 4 5 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Electrical Specifications Table 27: AC Electricals f EXTCLK = 6–54 MHz, VDD = VDDIO_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; Tj = 25° Symbol Parameter Conditions Min Typ Max Unit Note Output pin slew rate Programmable slew = 7 VDD_IO = 2.8V, Cload = 45pF VDD_IO = 1.8V, Cload = 45pF VDD_IO = 2.8V, Cload = 45pF VDD_IO = 1.8V, Cload = 45pF VDD_IO = 2.8V, Cload = 45pF VDD_IO = 1.8V, Cload = 45pF - 1.6 0.8 1.25 0.55 0.3 0.15 - V/ns V/ns V/ns V/ns V/ns V/ns 5 Programmable slew = 4 Programmable slew = 0 Notes: Table 28: 1. Measured when the PLL is off. Specification not applicable when PLL is on, but input HIGH/LOW voltage should be within specification. 2. VIH and VIL specifications apply to the over- and undershoot (ringing) present in the MCLK. 3. Measurement done with PLL off. 4. Valid for Cload<20pF on PIXCLK, DOUT[9:0], LINE_VALID, and FRAME_VALID pads. Loads must be matched as closely as possible. 5. PLL is off and EXTCLK is 24MHz. DC Electricals Setup Conditions: fEXTCLK = 6-54MHz, VDD = VDD_IO_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; Tj = 25°C, unless stated otherwise Symbol Parameter VDD VDD_PLL VAA VAA_PIX VDD_IO Digital core supply voltage PLL supply voltage Analog supply voltage Pixel supply voltage Digital IO supply voltage VDDIO_TX VPP IDD_IO MIPI supply voltage OTPM supply voltage Digital IO supply current Condition For VDD_IO = 1.8V For VDD_IO = 2.8V VDD_IO =1.8V VDD_IO = 2.8V VDD_IO = 1.8V VDD_IO = 2.8V Context A Context B IDD_PLL IDD IAA IAA_PIX IDDIO_TX IDD IAA IAA_PIX IDDIO_TX MT9D115 DS Rev. E Pub. 4/15 EN PLL supply current Digital core supply current Analog supply current Pixel supply current MIPI supply current Digital core supply current Analog supply current Pixel supply current MIPI supply current PLL is OFF PLL is ON Operating in Parallel mode Context A Context B 57 Min Typ Max Unit 1.7 2.5 2.5 2.5 1.7 2.5 1.7 - 1.8 2.8 2.8 2.8 1.8 2.8 1.8 8 10 15 12 20 N/A 13 15 40 1.5 N/A 25 40 0.8 N/A 1.95 3.1 3.1 3.1 1.95 3.1 1.95 18 30 50 3 52 52 3 - V V V V V V V V mA mA mA mA mA mA mA mA mA mA mA mA mA mA Note s 1 2 3 3 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Electrical Specifications Table 28: DC Electricals (continued) Setup Conditions: fEXTCLK = 6-54MHz, VDD = VDD_IO_TX = VDD_IO = 1.8V; VAA = VAA_PIX = VDD_PLL = 2.8V; Tj = 25°C, unless stated otherwise Symbol Parameter IDD IAA IAA_PIX IDDIO_TX IDD IAA IAA_PIX IDDIO_TX Digital core supply current Analog supply current Pixel supply current MIPI supply current Digital core supply current Analog supply current Pixel supply current MIPI supply current Total Standby Current IHardStandby IHardStandby ISoftStandby ISoftStandby ISoftStandby Notes: MT9D115 DS Rev. E Pub. 4/15 EN Condition Operating in Serial mode Context A Context B STANDBY pin asserted, R0x0028=1 STANDBY pin asserted, R0x0028=0 R0x0018[0]=1 R0x0018[0]=1, R0x0028=0 R0x0018[0]=1, R0x0028=1 Min Typ Max Unit - 23 40 1.5 5 35 40 0.8 8 - 50 50 3 3 70 70 3 10 20 90 90 2 3 mA mA mA mA mA mA mA mA A A A mA mA Note s 4 5 7 6 7 7 1. 2. 3. 4. 5. VDD_IO current is dependent on the output data rate. PLL is on with 85 MHz output frequency setting. VDD_IO_TX current is only applicable in serial output mode. PLL is on with 480 MHz VCO frequency settings. Either with EXTCLK running or EXTCLK stopped and EXCLK pin is either pulled up or pulled down. Measured at Tj = 70°C. 6. EXTCLK is stopped and EXCLK pin is either pulled up or pulled down. 7. EXTCLK running at 27 MHz. 58 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Appendix A- VDD_IO Current Addition Appendix A- VDD_IO Current Addition Introduction This appendix describes the system requirements to achieve lowest power consumption in the VDD_IO domain during low power hardware standby mode in ON Semiconductor's MT9D115 CMOS digital image sensor. VDD_IO Current VDD_IO current is used to measure the power consumption of the IO pad ring of our sensor. Therefore, it is extremely system-dependent and greatly affected by the external conditions as current can flow out of the chip through the IO ring and into the system. In this document, we outline requirements at the system level to achieve minimum power consumption during low power hardware standby mode. To achieve the lowest VDD_IO current, it is critical to match VDD_IO level in the sensor and the controller. Otherwise, there can be elevated current drawn on the VDD_IO due to mismatch in termination voltage level on the sensor pins. In ON Semiconductor system design, the IO voltage between the sensor and the controller is matched by using the same voltage regulator, as shown in Figure 35. This is recommended for both the system design and setup to measure the VDD_IO power consumption. Figure 35: Recommended System and Test Setup for Minimum VDD_IO Power Consumption Voltage R egulator Vreg M T9D113 2. 2kΩ C ontroller GN D VIO_C ontroller V DD_IO S CLK H igh-Z S DATA H igh-Z R ESET_BAR, STANDBY D rive HIGH S ADDR , EXT C LK D rive LOW PIXC LK, F R AM E_VALID, LIN E_VALID, D OUT [7:0], GPIO[1:0] Input GPIO[3:2] GN D GN D MT9D115 DS Rev. E Pub. 4/15 EN 59 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Appendix A- VDD_IO Current Addition IO Pin States In addition to matching VDD_IO levels between the controller chip and the sensor, the control signals to the input pins of the sensor must be at a specified level to achieve minimal VDD_IO current draw. Table 29 shows the pin states recommended to achieve minimum VDD_IO current during hardware standby mode. It is always a good practice to measure the pin voltage during hard standby and try to match the recommended pin state. Table 29: Status of Signals During Standby State Signal State on Sensor Pin DOUT[7:0] High-Z by default (configurable through OE_BAR or twowire serial interface register) High-Z by default (configurable) High-Z by default (configurable) High-Z by default (configurable) Depending on how the system uses them as DOUT_LSB1/DOUT_LSB0/FLASH/OE_BAR 0 0 0 0 Input Input Input Input Input Input PIXCLK LINE_VALID FRAME_VALID GPIO[3:0] DOUT_N DOUT_P CLK_N CLK_P SADDR EXTCLK SDATA SCLK RESET_BAR STANDBY Notes: MT9D115 DS Rev. E Pub. 4/15 EN ON Semiconductor Test System Note No control 4 Pulled to GND 8, 9 Float 7 Pulled to GND 1, 6 High Z 3 Pulled to VDD_IO Level 2, 5 1. VIL specification for input signal applies. Refer to Table 20, “GPIO Related Registers and Variables,” on page 53. 2. VIH specification for input signal applies. Refer to Table 20, “GPIO Related Registers and Variables,” on page 53. 3. These pins are not directly connected to VDD_IO supply but through a voltage follower to the controller. 4. The pins on the controller connected to these sensor pins are input pins. 5. These pins are not directly connected to VDD_IO supply but through the controller. 6. These pins are not directly connected to GND but through the control signal on the controller. 7. These pins are floating in parallel mode operation. 8. Tie all the unused GPIO pins to DGND or VDD_IO level in the module. 9. If GPIO3 is connected to GPIO pin on the controller, program GPIO3 as an input before low power hard standby mode and drive GPIO3 externally from the controller to DGND level. 60 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Appendix A- VDD_IO Current Addition RESET_BAR with Internal Pull-UP The RESET_BAR pin has an internal pull-up device to VDD_IO (see Figure 36). Figure 36: RESET_BAR Pad Architecture Since RPULLUP is active devices connected in triode state, the value of RPULLUP is dependent on the difference between VDD_IO and Vpad Level (see Table 30). Table 30: Typical Mismatch Current in VDD_IO Due to Mismatch in RESET_BAR level and VDD_IO Level Conditions: VDD = VDD_IO_TX = 1.8V; VAA = VAA_PIX=VDD_PLL=2.8V; Tj=30°C, EXTCLK is off and tied to GND level. All other pin states and levels are set according to ON Semiconductor E2E on pin states. VDD_IO 1.80 2.80 Note: MT9D115 DS Rev. E Pub. 4/15 EN Vpad Min Typ Max 1.70 1.80 1.95 2.50 2.80 3.10 14 2 –13 61 2 –54 18 5 –17 67 7 –60 22 10 –21 72 12 –66 Unit A The above data is for engineering purposes only. These represent typical values that can be expected. 61 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Appendix A- VDD_IO Current Addition Recommended System and Test Setup with Multiple Serial Interface Slave Devices The recommended system and test setup shown in Figure 37 is only valid for a system where MT9D115 is the only two-wire serial interface slave device connected to the controller chip. As the SCLK and SDATA are to be pulled high to the VDD_IO level during the hardware standby to minimize the IO current consumption due to the data toggling in the two-wire serial interface, this design is not feasible if there is an additional slave device sharing the same two-wire serial interface on the controller chip. In such systems, it is recommended to have a voltage follower to isolate the pull-up resistors attached between the two-wire serial interface and VDD_IO supply of MT9D115 as shown in Figure 37. Otherwise, the data toggling from the two-wire serial interface transactions to other could cause leakage from the VDD_IO supply of MT9D115. Figure 37: Recommended System and Test Setup for Minimum VDD_IO Power Consumption in the MT9D115 Sharing with Multiple Two-wire Serial Interface Devices Voltage R egulator Vreg Voltage F ollow er M T9D113 C ontroller V DD_IO VIO_C ontroller SC LK X S DATA X R ESET_BAR, STANDBY D rive H IGH S ADDR, EXT C LK D rive LOW PIXC LK, F R AM E_VALID,LIN E_VALID, D OUT[7:0], GPIO[1:0] GN D GN D Input GPIO[3:2] GN D O th e r sla ve d e vice s In addition, in some system designs where the IO level of the additional devices sharing the same two-wire serial interface with MT9D115 is different from that of MT9D115, it is recommended that the IO voltage level between the controller and the MT9D115 be maintained to the same level by sourcing from the same voltage regular as shown in Figure 38 on page 63. MT9D115 DS Rev. E Pub. 4/15 EN 62 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Appendix A- VDD_IO Current Addition Figure 38: Recommended System and Test Setup for Minimum VDD_IO Power Consumption in the MT9D115 Sharing with Multiple Two-wire Serial Interface Devices at Different IO Levels Voltage R egulator Vreg1 Voltage F ollow er M T9D113 V DD _IO VIO_C ontroller SC LK X S DATA X R ESET_BAR, STANDBY D rive H IGH S ADDR, EXT C LK D rive LOW PIXC LK, F R AM E_VALID,LIN E_VALID, D OUT [7:0], GPIO[1:0] GN D GN D C ontroller Input GPIO[3:2] GN D Voltage R egulator O th e r sla ve d e vice s Vreg2 For a system which uses separate voltage regulators for MT9D115 sensor and the controller chip as shown in Figure 39, it is important to match the voltage levels between two regulators as close as possible. If the voltage levels are not matched, there can be additional current consumption in the VDD_IO domain connected to the sensor. This additional current is generated in the internal pull up resistor present in RESET_BAR pad and external resistors used for two-wire serial interface. The current leakage from the voltage level mismatch in RESET_BAR pad can be characterized as follows: IDD_IOmismatch = (Vreg2 - Vreg1)/ RpullupRESET_BAR@VDD_IO The internal pull up resistance is dependent on the VDD_IO level. Typical levels of mismatch current at different VDD_IO levels can be found in Table 30 on page 61. MT9D115 DS Rev. E Pub. 4/15 EN 63 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Appendix A- VDD_IO Current Addition Figure 39: System Setup with Independent Voltage Sources for MT9D115 and Controller Chip Voltage R egulator Voltage R egulator Vreg1 Vreg2 GND GND DUT 2.2kΩ V DD _IO VIO_C ontroller S CLK H igh-Z S DATA H igh-Z D rive H IGH R ESET_BAR, STANDBY S ADDR, EXT C LK D rive LOW PIXC LK, F R AM E_VALID, LIN E_VALID D OUT [7:0], GPIO[1:0] GND MT9D115 DS Rev. E Pub. 4/15 EN C ontroller Input GPIO[3:2] GND 64 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Appendix A- VDD_IO Current Addition Test Sequence for Measuring VDD_IO Current at Hard Standby Mode Follow the procedure defined within the sensor data sheet. The following test sequence takes the MT9D115 as a reference. Contact your local ON Semiconductor Applications Engineer for any specific product. To measure VDD_IO current at hard standby mode: 1. Supply PWRDN (also known as STANDBY) pin (J11 pin2) with 0 volt externally. 2. Supply the RESET_BAR pin (U22 pin4) with VDD_IO volt externally. 3. Connect an ammeter between IDD_IO (J9) and DGND. 4. Use jumpers J5, J17, and J20 to make sure GPIO pins are connected to either GND or VDD_IO. At this point, the part is in operating mode and it is possible to load an .INI file to get an image with DevWare. 5. Supply the PWRDN pin with VDD_IO volt externally and wait for one frame + 50 clock. At this stage, the part is in hard standby. 6. By removing J1, the EXTCLK is disconnected to the part and then connect EXTCLK pin (J1 pin2) to either DGND or VDD_IO. 7. Now the part is in hard standby with EXTCLK off. The user can read off the VDD_IO current from the ammeter at step 3 above. Figure 40: Modifications to the Demo2 Sensor Head Board for VDD_IO Current Measurement Modifications J9 jumper removed to monitor IDDIO by connecting Ammeter on Pins 1 and 2 To control the Reset pin externally, U22 was removed and a 10KOhm resistor connected to Pin 4, which is connected to a 2.8V external source J11 jumper removed to control the STANDBY or PWDN pin externally. A 10K resistor is connected for pullup on this pin. in 2 in operating= 0V, Hard Standby= 2.8V using external voltage source. J1 jumper for CLOCK Conclusion It is important from the system perspective to ensure IO levels between MT9D115 and | © 2009 Aptina Imaging Corporation | Aptina Confidential the controller chip at the same potential level to achieve minimum IO power consump©2008 Micron tion in MT9D115. A system configuration to achieve this is proposed in this document. In addition, suggestions on how to approach system designs with multiple two-wire serial interface devices and multiple IO level devices are provided. 2 MT9D115 DS Rev. E Pub. 4/15 EN 65 ©Semiconductor Components Industries, LLC,2015. MT9D115: 1/5-Inch SOC Digital Image Sensor Revision History Revision History Rev. E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4/15/15 • Updated “Ordering Information” on page 2 Rev. D . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3/26/15 • Converted to ON Semiconductor template • Removed confidential marking Rev. C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3/14/12 • Updated notes for Table 3, “Signal Description and Direction,” on page 9 • Updated Figure 6: “Two-Wire Serial Control Bus Timing,” on page 13 • Updated “Parallel and MIPI Output” on page 32 • Added Figure 31: “Recommended Power Down Sequence,” on page 46 • Added Table 16, “Power Down Signal Timing,” on page 46 • Updated Table 22, “I/O Parameters,” on page 54 • Updated Table 25, “MIPI Low-Power Transmitter DC Characteristics,” on page 55 • Updated Table 28, “DC Electricals,” on page 57 Rev. B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7/7/11 • Updated to Production • Updated “Features” on page 1 • Updated Table 1, “Key Performance Parameters,” on page 1 • Updated Table 2, “Available Part Numbers,” on page 2 • Deleted “Overview” • Added “Functional Description” on page 6 • Added “Architecture Overview” on page 6 • Updated Figure 11: “Six Rows in Normal and Row Mirror Readout Modes,” on page 19 Rev. A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3/16/10 • Initial release. ON Semiconductor and the ON logo are registered trademarks of Semiconductor Components Industries, LLC (SCILLC) or its subsidiaries in the United States and/or other countries. SCILLC owns the rights to a number of patents, trademarks, copyrights, trade secrets, and other intellectual property. A listing of SCILLC’s product/patent coverage may be accessed at www.onsemi.com/site/pdf/ Patent-Marking.pdf. SCILLC reserves the right to make changes without further notice to any products herein. SCILLC makes no warranty, representation or guarantee regarding the suitability of its products for any particular purpose, nor does SCILLC assume any liability arising out of the application or use of any product or circuit, and specifically disclaims any and all liability, including without limitation special, consequential or incidental damages. “Typical” parameters which may be provided in SCILLC data sheets and/or specifications can and do vary in different applications and actual performance may vary over time. All operating parameters, including “Typicals” must be validated for each customer application by customer’s technical experts. SCILLC does not convey any license under its patent rights nor the rights of others. SCILLC products are not designed, intended, or authorized for use as components in systems intended for surgical implant into the body, or other applications intended to support or sustain life, or for any other application in which the failure of the SCILLC product could create a situation where personal injury or death may occur. Should Buyer purchase or use SCILLC products for any such unintended or unauthorized application, Buyer shall indemnify and hold SCILLC and its officers, employees, subsidiaries, affiliates, and distributors harmless against all claims, costs, damages, and expenses, and reasonable attorney fees arising out of, directly or indirectly, any claim of personal injury or death associated with such unintended or unauthorized use, even if such claim alleges that SCILLC was negligent regarding the design or manufacture of the part. SCILLC is an Equal Opportunity/Affirmative Action Employer. This literature is subject to all applicable copyright laws and is not for resale in any manner. MT9D115 DS Rev. E Pub. 4/15 EN 66 ©Semiconductor Components Industries, LLC,2015 .