{
Choosing and using scientific cameras
}
KEITH BENNETT, PH.D. | HAMAMATSU PHOTONICS K.K. | FOCUS ON MICROSCOPY| 04.13.2014
Choosing and Using Scientific Cameras
1{
2{
Th i
The image problem
bl
3{
Real cameras are not perfect
4{
Know thyself
5{
The Living Image: Case Studies
g
g
Think in photons
Choosing and Using Scientific Cameras
1{ The image problem
2{ Think in photons
3{ Real cameras are not perfect
4{
Know thyself
5{
The Living Image: Case Studies
g
g
The Image
g Problem…
Courtesy: Prof. Jason Swedlow
University of Dundee, Scotland
O
Open Microscopy Environment
Mi
E i
t
4
The Image
g Problem…
A pretty picture?
A measurement?
A resource?
A reference?
Courtesy: Prof. Jason Swedlow
University of Dundee, Scotland
O
Open Microscopy Environment
Mi
E i
t
5
{1}
THE IMAGE PROBLEM
• Eyes can be fooled
LOOK
-
Not good at quantifying greys
Not objective
bj i
Emphasizes patterns and colors
Viewing environment
Viewing environment
CAREFULLY
• Screens are not capable of p
displaying full bit depth
• Image display can (and should be!) manipulated for on screen viewing p
g
{1}
THREE IDENTICAL IMAGES?
A
B
C
{1}
THREE IDENTICALLY DISPLAYED IMAGES!
A
B
C
200 photons
200 photons
500 photons
1000 photons
{1}
THREE DIFFERENT INTENSITIES?
THREE DIFFERENT DISPLAYS OF THE SAME INTENSITY!
1
{}
1000 photons
1000 photons
1000 photons
1000 photons
1000 h t
1000 photons
{1}
HISTOGRAM AND AREA STATISTICS
A
Peak (photons)
(p
)
Mean (photons)
B
200
47.0
C
500
117.4
1000
234.7
Choosing and Using Scientific Cameras
1{ The image problem
2{ Think in photons
3{ Real cameras are not perfect
4{
Know thyself
5{
The Living Image: Case Studies
g
g
{2}
THINKING IN PHOTONS
IS
THINKING IN
PHOTONS
REALLY
NECESSARY?
•
Aren’t ADU’s or grey levels good enough?
g
SCIENTIFIC CAMERAS SHOULD MEASURE PHOTONS
{2}
PHOTONS REALLY MATTER
Truth
100 photons peak
Looks similar, but
Looks
similar but
‐ The histogram is different
‐ Information is different
‐ Quantification different
ifi i diff
‐ Lower image contrast
Perfect camera Perfect
camera
Background = Peak/10
{2}
REMEMBER SHOT NOISE
{2}
WHAT’S
LIMITING
MY
SCIENCE?
THINKING IN PHOTONS
•
The information in an image is limited h i f
i i
i
i li i d
by the number of photons.
•
A perfect camera does not produce a perfect image, especially if photons are limited.
•
The minimum number of photons Th
i i
b
f h
needed depends upon the object imaged resolution and measurement
imaged, resolution and measurement requirements (i.e. your experiment).
{2}
PHOTONS REALLY MATTER
ARE YOU
CONVINCED?
Choosing and Using Scientific Cameras
1{
2{
The image h
problem
bl
Think in photons
3{ Real cameras are not perfect
4{
Know thyself
5{
The Living Image: Case Studies
g
g
3}
{3 REAL CAMERAS: T
: THINKING IN PHOTONS
HOW
DOES THIS
MAKE ME
A BETTER
MICROSCOPIST?
• Makes
Makes comparisons among cameras comparisons among cameras
meaningful. (ADUs are arbitrary)
• Brings relevance to your data.
• Kno
Knowing the number of photons ing the n mber of photons
and contrast in sample is key to picking the correct camera.
picking the correct camera.
{3}
REAL CAMERAS
IS
THINKING IN
PHOTONS
REALLY
NECESSARY?
•
Can’t we figure everything out from a camera specs (QE and electronic
camera specs (QE and electronic specs)?
[Hint: Maybe, but there’s a better way]
SCIENTIFIC CAMERAS SHOULD MEASURE PHOTONS
{
{3
REAL CAMERAS ARE NOT PERFECT
THE
WHAT
AND
HOW
•
•
•
•
•
•
The Gap
Electron multiplying CCDs (EMCCDs)
Simulations comparing perfect to product by spec
All pixels are not created equal
All pixels are not created equal
Actual product measurements
Camera noise & visualization
Why is a Why
is a
camera manufacturer proclaiming
that h
cameras are not perfect?
cameras are not perfect?
Because NO camera is perfect
&
B
Because understanding why d t di
h
matters to your science
to your science
{3} WHAT IS THE GAP?
The difference between the performance of an actual camera and a theoretically perfect camera
{
Perfect Camera
The GAP Actual Camera
UNDERSTANDING WHY WHY THERE IS A GAP ENABLES:
3
{}
• Appropriate camera selection
Appropriate camera selection
• Optimized camera usage
• Optimized experimental design • More reliable data analysis
Better
Results
{3}
THE GAP DEPENDS ON:
1 Sensor technology
1.
S
h l
2 Camera specs
2.
C
3. Input photon level
{
{
{
CCD
EMCCD
sCMOS
Quantum Efficiency
Camera Noise
Camera Noise
• Read noise
• Excess noise
• Photo‐response non‐uniformity Photo response non uniformity (PRNU)
Ultra low light
Low Light
h
Intermediate
High
{3} THE (HYPOTHETICAL) PERFECT CAMERA
100% QE
100% QE 0 e‐
read noise
read noise
Every photon is converted into one electron
{ Every photon is converted into one electron
y
g
y
p
y
{ Every electron is digitized exactly as expected every time
0% fixed {
pattern noise
tt
i
Every pixel and amplifier perform identically and predictably
yp
p
p
y
p
y
In a perfect camera, the
SNR of a single pixel is limited only
by the physics of photon statistics…
statistics
i.e. shot noise.
Signal to Noise Ratio (SNR)
Perfect Camera Signal to Noise Ratio
100
10
1
0.1
1
0
10
100
1000
Signal (Photons)
10000
100000
{3}
REAL CAMERAS ARE NOT PERFECT
IImagEM
EM X2 X2
EMCCD: Electron Multiplying CCD
ORCA‐Flash4.0 V2
ORCA
Fl h4 0 V2
Scientific CMOS Camera
ORCA‐R2
ORCA
R2
Cooled Interline CCD
: COMPARED
{3}BASIC SPECS: C
EMCCD
CMOS
{
{
{
CCD
Camera Name
Camera Name
ORCA‐R2
ORCA
R2 ImagEM x2
ORCA Flash4.0 V2
Flash4 0 V2
QE (550 nm)
70 %
90 %
72 % Read Noise Single Noise Single
Frame rms (e‐)
6
< 0.5 (M = 200)
1.5 Full Well Capacity (e‐)
,
18,000
Gain dependent
p
30,000
,
Dynamic Range
3000:1
Gain dependent
20,000:1
p
Bit Depth
16
16
16
Max pixel rate (Mps)
13
18
420
Pixel Size (m)
Pixel Size (m)
6.45 x 6.45
6.45 x 6.45
16 x 16
16 x 16
6.5 x 6.5
6.5 x 6.5
Pixel Number
1024 x 1344
512 x 512
2048 x 2048
{3} AMPLIFIERS
Important p
differences
CCD and sCMOS
EMCCD
{3}
ELECTRON MULTIPLYING CCDS (EMCCDS)
• A type of CCD: Frame transfer and At
f CCD F
t
f
d
back‐thinned for increased QE
• Frame transfer requires ~ 100s
• Serial devices where each pixel
Serial devices where each pixel’ss charge charge
is read out one at a time
• High voltage gain register on sensor for High voltage gain register on sensor for
on‐chip amplification. • Option to read out through EM circuitry O ti t
d t th
h EM i it
or non‐EM circuit (normal CCD mode)
{
}
EMCCD architecture
EMCCD architecture
{3}
CMOS AND CCD CMOS CCD AMPLIFIER NOISE
Output an exact multiple of the input
No noise broadening
Output is a multiple of the input
“Read noise” broadening
Width independent of signal p
g
level
CMOS
CMOS read noise: 1.5 e‐
d i 1 5 rms
EMCCD
{3} EMCCD AMPLIFIER NOISE DEPENDS ON SIGNAL
No electron:
No
electron:
‐ Very small noise
‐ beautiful blacks
Signal:
‐ Broad (excess noise)
‐ Long tail: larger apparent
contrast
Signal independent
‐ No excess noise
‐ Short tail
{3}
EMCCDS “DETECT” SINGLE PHOTONS, BUT
0.4 e‐ (!!)
Peak of 1e‐ output is ~0.4e‐!
Signal < (some) noise
Long tail SSymmetric distribution, with t i di t ib ti
ith
noise extending ~+2 (3 e‐) from mean.
Significant overlap
Quantization of ADC not included
{3}
EMCCDS CAN’T COUNT
Outputs from 1e‐ and 2e‐
overlap.
l
Peak output of 2e‐ input is ~ 1e‐
CMOS not so good either at very low light
2e‐ input, CMOS tail is shorter than EMCCD
{3}
EMCCD: SIGNAL DEPENDENT NOISE
EMCCD: S
Most probable output < mean.
Very long tail
2 = signal
Lots of overlap: 10e
Lots
of overlap: 10e‐ & 20e‐
Most probable output = mean
Short tail
2 = 1.5 e‐
CMOS clearly better
EMCCD
{3} EMCCD OUTPUT INCLUDING PHOTON SHOT NOISE
In simulated probability distribution p
y
functions for EMCCD, the output at p
high gain is not Poisson due to the electron multiplication process!
2 Photon Average Input
Gain = 200
10 Photon Average Input
Gain = 200
0.00025
0.0007
Probability [a.u.]
P
Prrobability [[a.u.]
0.0008
Long tail
0.0006
0 0005
0.0005
0.0004
0.0003
0.0002
0.0002
Long tail
0 00015
0.00015
0.0001
0.00005
0.0001
0
0
‐5
0
5
Photon equivalent
10
0
5
10
15
20
Photon equivalent
25
30
{3}
EMCCD VS. CMOS EMCCD CMOS AMPLIFIERS
• Stochastic
S h i EM amplification:
lifi i
– Very low noise without input
– Excess noise effectively doubles photoelectron shot noise (Fn2 = 2)
– Asymmetric output distribution
A
i
di ib i
• At low light, peak output is much below mean
• Long tail
Long tail
• CMOS – Noisier with no or very low input
N ii
ith
l i
t
– Noise independent of signal
{3}
ELECTRON MULTIPLYING CCDS
Are they Are
they
really what you thought?
{3}
Terms included:
SIMPLE (PIXEL) SNR EQUATION
Not included:
QE: Quantum Efficiency
Dark Noise: Dark current X time; S: Input Signal Photon Number (photon/pixel) considered negligible
: Noise Factor
Fn: Noise Factor (= 1 for CCD/sCMOS and √2 for EM‐CCD)
Photo response non uniformity: necessary for image SNR
Nr: Readout Noise
M: EM Gain M: EM Gain (=1 for CCD / CMOS)
(=1 for CCD / CMOS)
Ib: Background
{3} R
ELATIVE
SNR: DISPLAYS IMPERFECTIONS PERFECTLY
1
Relative S
SNR (rSNR))
0.9
0.8
0.7
0.6
Perfect
0.5
QE 70%
0.4
QE 50%
0.3
0.2
0.1
0
0.1
rSNR is the SNR for a
camera plotted relative to
the perfect camera
1
10
100
Signal (Photons)
1000
10000
rSNR shows differences
among cameras over full
range
g of signal
g
level
{3}R
EAD NOISE REDUCES RSNR SNR ONLY AT LOW LIGHT
Read Noise
Limited
Shot Noise
Limited
1.0
50% QE Limit
Re
elative SNR
R (rSNR)
09
0.9
0.8
0.7
0.6
0.5
QE 70%, Nr(ph) = 3
0.4
QE 50%, Nr(ph) = 3 0.3
0.2
0.0
1
10
100
1000
Signal (photons)
• Nr(ph): Read noise in photons
is the key low light parameter
Nr(ph) = Nr(e‐)/QE
• QE: always important
Nr (e‐) = 2.1 e‐
Nr (ph) = 3
Same Nr(ph)
0.1
0.1
70% QE Limit
10000
Nr (e‐) = 1.5 e‐
Nr (ph) = 3
{3} EMCCD : E: E
S
XCESS NOISE CREATES A GAP
SNR for CCD / CMOS
SNR for CCD / CMOS
QE P
SNR
QE P
SNR for EM‐CCD
SNR for EM
CCD
SNR
QE P
QE: Quantum Efficiency,
y
P:
Input Signal Photon Number,
M: EM Gain
Fn: Noise Factor
(assumes dark current and read noise are negligible)
M QE P
QE P
Fn 2
Fn M QE P
QEeff P
QEeff
QE QE
2
Fn
2
{3}
EMCCDs
• Stochastic EM amplification Native QE
adds excess noise
• Excess noise effectively lowers the SNR to a detector with ½ the SNR to a detector with ½
eQE
the QE
{
Effective QE in EMCCDs
ff
}
{3}
MIND THE GAP: PREDICTED PIXEL rSNR PERFORMANCE FOR
COMMON CAMERAS
2. {
The SNR of an EMCCD above 1 The
SNR of an EMCCD above 1
electron/pixel is comparable to a camera with QEeff =QE/2 due to excess noise from EM gain.
g
1
Re
elative SN
NR (rSNR
R)
1.{
A camera with the highest A
camera with the highest
SNR at the lowest light level may not be the best at higher light levels
g
g
0.9
0.8
1.
0.7
0.6
2.
05
0.5
P f t
Perfect
0.4
ORCA-R2 (CCD)
0.3
ImagEM X2 (EMCCD)
02
0.2
sCMOS Flash4.0
Flash4 0 (100 fps)
0.1
ImagEM X2 (BT CCD mode)
0
01
0.1
1
10
100
1000
Signal (Photon, no background)
10000
= 650 nm
ARE ALL
PIXELS THE
SAME?
• Offset non‐uniformity
• Photo response non‐
uniformity (PRNU)
• Dark signal non‐
uniformity (DSNU)
• Read noise distribution
{3}
ACCURATE MEASUREMENT OF THE NUMBER OF PHOTONS
OFFSET NON‐UNIFORMITY
Pixel to pixel variation of readings in the dark
+0.6
‐0.2
0.2
+1.0
If the zero is incorrect, then absolute measurement is also incorrect.
• Most noticeable in dark or low light conditions.
Most noticeable in dark or low light conditions
• Usually expressed as DN or e‐, rms.
• For scientific cameras, should be less than read noise.
{3}
ACCURATE MEASUREMENT OF THE NUMBER OF PHOTONS
PHOTO RESPONSE NON‐UNIFORMTIY
PRNU: pixel to pixel variation of the response to light (DN / photon)
‐ QE variation : conversion rate of photon to e‐
(may be spectrum dependent)
‐ Electronic gain variation: Conversion factor from e‐ to DN
‐15%
+22%
‐6.5%
If the unit length incorrect, then absolute measurement is also
g
,
incorrect.
• Most noticeable in higher light conditions.
• May have spatial pattern, stable over time.
• Usually expressed as % maximum.
U ll
d %
i
Mean: 11.9
{3}
ACCURATE MEASUREMENT OF THE NUMBER OF PHOTONS
TOTAL FIXED PATTERN NOISE
Total pixel‐to‐pixel variation in the accuracy of the measurement of the number of photons Includes
photons. Includes
• Offset non‐uniformity
• Photo‐response non‐uniformity
‐15%
+22%
‐6 5%
‐6.5%
Overall specification of the non‐uniformity measurement across the image sensor
Does not include:
• Errors in average QE
• Temporal noise (excess noise, read noise)
Temporal noise (excess noise read noise)
• Dark current and dark current shot
{3} D
ARK SIGNAL NON‐U
UNIFORMITY (DSNU)
Pixel‐to‐pixel variation in dark current
550 nm
850 nm
Offset : dark signal x exposure time.
Noise : (offset in e‐)
How big?
.
Which
technologies?
Correction
{
{
{
•
•
•
•
•
Proportional to exposure time.
Mean: 30157
Mean: 30508
Can be >100 e- / sec for a few pixels,
especially
for sensors
0C
s: 369.5
(1.2%)
s: 432 >
(1.4%)
For a given image sensor, a multiple of average dark current
Doubles for each ~8C increase in sensor temperature
Higher noise for high dark current pixels due to dark shot noise.
• Mainly sCMOS
• Identify high noise pixels and correct in image
• Dark shot noise can NOT be corrected.
{3} R
EAD NOISE UNIFORMITY: CCD & EMCCD
: CCD & EMCCD
CCDs and EMCCDs: All pixels are readout through the same amplifier and digitization
and EMCCDs: All pixels are readout through the same amplifier and digitization
circuits and therefore read noise is very uniform.
Median = spatial rms
Noise Histogram
10000000
Number of pixels (0.1 e‐ bin)
1000000
6 e‐
6
e (rms) (rms)
temporal
0.4 e‐
0
4 (rms) (
)
temporal)
100000
CCD
~8 Mpix / sec
10000
EMCCD. Gain EMCCD
Gain
dependent
1000
~18 Mpix / sec
100
10
1
0
2
4
6
8
10
12
14
Temporal read noise (e‐, rms)
16
18
20
{3}
READ NOISE UNIFORMITY: CMOS
: CMOS
CMOS: Each pixel has an independent amplifier and each column has an independent
CMOS:
Each pixel has an independent amplifier and each column has an independent
amplifier. Read noise is pixel dependent
“Median” < spatial rms. 1,000,000
Number of pixels 0.85 e‐ Median
100,000
1.51 e‐ spatial rms
10,000
1,000
(10 s/ row) = 420 Mpix/sec
100
10
0
1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16
Temporal read noise (e‐, rms)
{3}
READ NOISE
• CCDs: Uniform, readout speed dependent, relatively high.
• EMCCDs: Uniform, gain and readout speed dependent, very p
p
,
y
low with EM gain > ~50, but relatively high in “normal CCD” mode.
• sCMOS:
sCMOS: pixel dependent, little pixel dependent little
dependence on readout speed for a particular camera.
p
Things to keep in mind
i d
MEASURING
THE
REAL GAP
An in‐depth look at noise in CCD, EMCCD and CMOS
CCD, EMCCD and CMOS cameras
{3}A A
CLEARER WAY TO COMBINE CAMERA SPECIFICATIONS
• Single Frame rSNR p
Summarizes whole sensor performance
– QE
– Gain
– Noise: including spatial rms read noise, excess noise, dark shot noise
i d k h t i
– Fixed pattern noises, including offset non‐
uniformity and PRNU
– Saturation
ORCA R2 I
{3} ORCA‐R2 I
1{
1.
{
2.
NTERLINE CCD: P
CCD: PREDICTABLE AND ROBUST
PRNU is insignificant
PRNU is insignificant
Bright Image:
shot noise limited
Single frame read noise histogram has a Gaussian distribution Mean intensity: 17,300 e: 130.5ePRNU: not measurable
Read Noise
(Nr/QE)
10000
C
Count
1000
100
10
1
‐78
78 ‐60
60 ‐43
43 ‐25
25
‐8
8
10
Dark reading (ph)
27
45
62
80
=650 nm
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1
0.1 ‐
Shot Noise
& QE
58% QE Limit
10 100 1,000 signal (photon)
10,000 EMCCD: S
{3} EMCCD: S
1.{
OME SURPRISING RESULTS
• Cannot be removed during
manufacturing
• Must be calibrated by users for their
specific
p
spectrum.
p
• Individual pixel map required for
correction
850 nm
550 nm
Thickness variations from backback
thinning process causes spectrallydependent PRNU
Mean: 30157
s: 369.5 (1.2%)
Mean: 30508
s: 432 (1.4%)
Calculated Single Frame rSNR
1
0.9
0.8
07
0.7
rSNR
{
2.
The Gap for EMCCD in CCD mode becomes very wide due to PRNU
wide due to PRNU
0.6
0.5
0.4
03
0.3
eQE:
Q 95%
% ((gain off)
ff)
Nr/eQE = 8.4 photons
PRNU: 1.4%
0.2
0.1
0
10
100
1000
Photons
10000
100000
{3}
COMPLEX BEHAVIOR: A CLOSER LOOK AT EMCCD SNR WITH
HIGH AND LOW GAIN
-
1
90% QE Limit
0.9
Excess Noise
08
0.8
0.7
Relative
e SNR
Excess noise (eQE)
PRNU
Saturation
Hi h read
High
d noise
i
(34 e- @ M=5, 70 fps)
- Gain hard to measure
Saturation
Complex Behavior
Complex Behavior
0.6
0.5
0.4
0.3
Gain
Gain = 5
5
系列1
0.2
Gain = 400
系列2
0.1
0
0 01
0.01
01
0.1
1
10
100
Input photon number (photon)
1000
10000
ORCA FLASH4.0 V2 (
ORCA‐F
4 0 V2 (SCMOS): A V
CMOS): A VERY COMFORTABLE SWEET SPOT
The “Sweet Spot”
Shot Noise
& QE
1.0
0.9
0.8
0.7
06
0.6
0.5
0.4
0.3
0.2
0.1
0.0
70% QE Limit
70% QE Limit
rSNR
{3}
1
10
100
1,000
Signal (Photons @650 nm)
10,000
{3}
THE IMAGE SENSOR IS NOT THE CAMERA: PRNU IS SIGNIFICANT IN
“SCIENTIFIC ” CMOS IMAGE SENSORS
Bright Image
{
Corrected Data {
Raw Data PRNU ~2%
PRNU: ~2%
PRNU ~0 5%
PRNU: ~0.5%
Bright Image
Signal amplified and digitized in column‐parallel ADC.
Si
l
lifi d d di i i d i
l
ll l
FPGA provides offset and gain correction to the raw digitized signal.
CMOS: P
{3} CMOS: P
IXEL‐D
DEPENDENT READ NOISE
S
1,000,000
Rms read noise matches
single
g frame rSNR.
Numbe
er of pixels 0.85 e‐Median
100,000
Single Frame Read
Noise (measured)
1.51 e‐ rms
10,000
1,000
100
((10 s/ row) = 100 full fps
/
)
p
10
0
2
3
4
5
6
7
8
9 10 11 12 13 14 15 16
Temporal read noise (e‐, rms)
10000
1000
10
Does not fit a Gaussian
distribution, i.e. is not completely
1
modeled by a single “read noise.”
100
‐19.7
‐17.8
‐15.8
‐13.9
‐11.9
‐10.0
‐8.0
‐6.1
‐4.1
‐2.2
‐0.2
1.7
3.7
5.6
7.6
9.5
11.5
13.4
15.4
17.3
19.3
Numberr of pixels
Single Frame Dark Histogram
1
photon equivalent @ 650 nm
{3}
SCMOS: IMPROVING VISUAL IMAGE QUALITY
“NOISY” PIXEL FILTERING
Correction ON
A
AUTO LUT
T
Correction OFF
Map high noise pixels and selectively replace value with the average of the surrounding pixels.
Contro
olled LUT
• Improves contrast & “flicker” with “auto” LUT.
• Small difference with controlled LUT
• Affects only a very small number of pixels in frame
30 photons peak, ~10 photons avg.
{3}MANAGING READ NOISE
EMCCD
CMOS
{
{
{
CCD
Read noise expressed in photons is the key specification.
noise expressed in photons is the key specification
Nr (ph) = Nr (e‐)/QE
Specs
Data collection
Analog binning, optical matching
Use slowest clock speed possible Distribution
Use spatial or single Use spatial
or single
frame rms, not median rms
Optical matching
Optical matching
Use pixel noise filter when possible Visualization
Set lower threshold to a minimum of offset plus 0 to 3 noise standard deviations
Statistical
noise model
Poisson + uniform Poisson
+ uniform
Gaussian
Complicated, gain Complicated
gain
dependent
Poisson + pixel
Poisson
+ pixel‐
dependent Gaussian
WHAT ABOUT
IMAGES?
• Perfect and real cameras
• Visualization
• Histograms
• How many photons do you need?
{3}
Controllled LUT
AUTO LUTT
sCMOS
sCMOS:
Noise Correction ON
COMPARING CAMERAS: 1000 PHOTON PEAK
VISUALLY SIMILAR
EMCCD
CCD
Perfect
{3}
COMPARING CAMERAS: 100 PHOTON PEAK
CAMERA NOISE AND / OR VISUALIZATION MATTER
Contro
olled LUT
AUTO LUTT
sCMOS
sCMOS:
Noise Correction ON
EMCCD
CCD
Perfect
{3}
HARDER TO SEE IN THE DARK: 30 PHOTON PEAK
CAMERA & VISUALIZATION CRITICAL
Contrrolled LUT
AUTO LUTT
sCMOS
sCMOS:
Noise Correction ON
EMCCD
CCD
Perfect
H
ISTOGRAMS: :
MOST SIMILAR TO THE PERFECT CAMERA
3
{}
EMCCD
10
000 photons
100 photons
30 ph
hotons
sCMOS
Mean photons: ~35% of peak
CCD
Perfect
{3}H
OW MANY PHOTONS DO I I NEED WITH A PERFECT CAMERA?
Controlled LUT
AUTO LU
UT
30
100
1000
OW MANY PHOTONS DO I I NEED WITH A REAL CAMERA?
sCMOS
EMCCD
CCD
Perfect
?
Good enough?
Bad
Good enough?
Good
Good
?
Good
Good
Good
Good
Good
100 photon
ns
1000 pho
otons
Co
ontrolled LUT
30
0 photons
{3}H
sCMOS:
Noise Correction ON
Photons : peak intensity in whole, not zoomed, image
H
N
G
{3}
K
h
d
1{ Know what you want to do
OW TO
ARROW THE
AP
The number of photons required to “see” something depends upon what you want to see, and how clearly you want to see it, even with a perfect camera.
2{
Turn up the light carefully
3{
Vi li ti
Visualization matters
tt
4{
Use the right camera
Real cameras reduce image quality, however when there is enough R
l
d
i
lit h
h th
i
h
light, all scientific cameras work well
Monitor choice, ambient light, LUT settings all make a difference Gen II sCMOS cameras have comparable or better image quality than EMCCDs at light levels typically required for visual imaging
CHOOSING AND USING SCIENTIFIC CAMERAS
1{
2{
3{
The image h
problem
bl
Think in photons
Real cameras are not perfect
4{ Know thyself
5{
The Living Image: Case Studies
g
g
{4}
KNOW THYSELF
sample contrast
WHAT IS MOST
IMPORTANT
FOR YOUR
EXPERIMENT?
frame rate
resolution
l ti
accuracy
background
{4}CONSIDER THE ENTIRE SYSTEM
• Lightsheet microscopy (SPIM)
microscopy (SPIM)
TWO
EXAMPLES
• Single molecule localization microscopy
{4}
Light Sheet Micoscopy
Light Sheet Micoscopy
Just like Localization Microscopy LSM has many faces
Benefits
Better sectioning vs. widefield
Less photodamage vs. confocal
Fast acquisition of large samples
New developments
N d l
t
Multiple Cameras
Structured Illumination 75
LSFM ‐ Light Sheet Fluorescence Microscope
SPIM ‐ Single Plane Illumination Microscope
OPM ‐ Oblique Plane Microscopy
sTSLIM – Scanning Thin Sheet Laser Illumination
Microscopy
mSPIM – Multidirectional SPIM
{4}
MuVi SPIM and SIMView
MuVi‐SPIM and SIMView
Multiple illumination beams and cameras
Increased isotropy and axial resolution
axial resolution
Faster
Faster scanning with phase scanning with phase
or wavelength separation of offset beams
of offset
76
{4}
SCMOS IS >20X FASTER THAN EMCCDS
SPEED!
{4}
LIGHT SHEET MICROSCOPY
http://thelivingimage.hamamatsu.com/
http://player.vimeo.com/video/74253101
{4}
CRITICAL CAMERA
CHARACTERISTICS
HA AC
IS ICS
LIGHT SHEET MICROSCOPY
•
•
•
•
•
Large field of view (high pixel number)
High speed (data rate)
High speed (data rate)
Large dynamic range
Reasonably low noise
Reasonably low noise
Rolling shutter synchronized to sample scanning with variable speed
scanning with variable speed
Camera: ORCA Flash4.0 Scientific CMOS
{4}
LIGHT SHEET MICROSCOPY
Light sheet microscopy – matching the camera p
y
and optical system
http://www hamamatsu com/sp/sys/en/promotion/mp4/s Lightsheet en html
http://www.hamamatsu.com/sp/sys/en/promotion/mp4/s_Lightsheet_en.html
{4}
{
{
{
OPTIMALLY USING THE CAMERA FOR THE TASK
{4}
STANDARD PRACTICE IS NOT THE BEST PRACTICE: U
: USING EMCCD EMCCD
WITH GAIN YIELDS LEAST ACCURATE RESULTS
CCD QE: 100%, read noise = 1.8 ph, no background; No fixed pattern noise.
Adapted from: J. Chao et al (Ober Lab), Nat. Meth10, 2013) doi:10.1038/nmeth.2396
Ad
df
J Ch
l (Ob L b) N
M h10 2013) d i 10 1038/
h 2396
http://www.wardoberlab.com/
UNCORRECTED PRNU PRNU CAN LEAD TO LOCALIZATION BIAS
4
{}
Localization di t ib ti & bi
distribution & bias
Impact of PRNU on localization bias:
Alexa 647 simulation (3000 photons)
mEos2 simulation (750 photons)
0.5% PRNU: 1 – 2 nm @ 100 nm/ pixel
Courtesy: Zhen‐li Huang, Huazhong University of Science and Technology, (unpublished)
{4}
COMPENSATING READ NOISE VARIATION
Courtesy Prof. Joerg Bewersdorf, Yale University
IIncorporating pixel‐specific read noise into the Maximum Likelihood Probability Model eliminates and narrows the ti
i l
ifi
d i i t th M i
Lik lih d P b bilit M d l li i t
d
th
asymmetric distribution of localized molecules caused by higher read noise pixels.
Courtesy F. Huang, Bewersdorf Lab
{4}
MLE RECONSTRUCTION MUST USE A NOISE MODEL
INCLUDING CAMERA NOISE
Worst
Best
Note: MLE for EMCCDs are also difficult:
Simple and good
‐ Inaccurate gain
I
t
i
‐ Output PDF not Poisson
‐ Even at “high” light, the variance is 2X the mean signal (in photons). Courtesy: Zhen‐li Huang, Huazhong University of Science and Technology, (unpublished)
: CASE STUDIES
{4} SELECTING AND USING CAMERAS: C
{
{
{
{4}
Results
{
Accurate measurement of the distance
Accurate
measurement of the distance between two between two
fluorophores of different colors. distance ~0.77 nm using a dichroic beamsplitter to direct each color of light to separate halves of the CCD camera
halves of the CCD camera.
{
{
Camera
Correction
Measured PRNU maps for each color. Improved p
p
localization relative accuracy by ~2– 4 nm.
Details
Speed: 5 –
Speed:
5 50 s / measurement
50 s / measurement
Light: ~4,000 – 10,000 ph/ mol/frame
~105 ph / mol before bleaching
Imaging: Simultaneous 2 color
Simultaneous 2 color
Camera: Back‐thinned EM‐
CCD, gain off
Nature (2010)| doi:10.1038/nature09163
{4}
Cholera toxin B subunit Results
{
{
{
Camera
Correction
Details
scale bar: 1 m
Localization Microscopy with Minimal Bleaching. Plasma membrane dynamics for > 60 s (594 frames). 40% better 1 m
1 m
localization precision than “conventional” EMCCD localization p
Implemented detailed statistical EM noise model into maximum likelihood reconstruction probability model.
Speed: ~60s / reconstructed image
Light: ~100 photons /molecule frame
Mag: 630X
Camera: EM‐CCD, Gain ~1000
Courtesy of J. Chao et al (Ober Lab) Adapted from Nat Meth (2013) doi:10.1038/nmeth.2396
{4}
Localization Precision “conventional” EMCCD vs. sCMOS
Courtesy of F. Huang. Bewersdorf Lab, Yale C
t
fF H
B
d fL b Y l
Adapted from F. Huang et al., Nature Methods 10(7): 653‐658 (2013) {4}Y
MINIMIZING THE GAP: MATCHING THE CAMERA TO
OUR NEEDS
Light Requiired
Higher Accuracy
Better Resolution
L
S
l C
Lower Sample Contrast
S i ifi CMOS
Scientific CMOS
(BT)‐CCD
EMCCD Single
photon
EMCCD
EMCCD Speed / Field of View
Faster (or more pixels)
Choosing and Using Scientific Cameras
1{
2{
The image h
problem
bl
Think in photons
3{
Real cameras are not perfect
4{
Know thyself
5{ The Living Image
The Living Image
{5}
RESOURCES FOR MICROSCOPISTS
http://thelivingimage hamamatsu com
http://thelivingimage.hamamatsu.com
ACKNOWLEDGEMENTS
Prof Zhen‐li Huang Huazhong University of Science and Technology
Prof. Zhen‐li Huang, Huazhong
University of Science and Technology
F. Long et al, OPTICS EXPRESS 17741 (2012) Prof. Joerg Bewersdorf, Yale University
F. Huang et al., Nature Methods 10(7): 653‐658 (2013) Prof. Raimund Ober, Texas Southwestern University
J. Chao et al, Nat Meth (2013) doi:10.1038/nmeth.2396
Prof. Lars Hufnagel, EMBL
Dr. Philip Keller, Janelia Farms
Hamamatsu
Teruo Takahashi: simulations
Hiroyuki Kawai: camera measurements
oyu a a ca e a easu e e ts
Stephanie Fullerton: presentation guidance
Katsuhide Ito: Lightsheet microscopy
Eiji Toda: budget
Download
(look on The Living Image)
Keith Bennett
[email protected]
32 fps dynamics. 500 nm 32
f d
i
500
scale
Courtesy Vutara / Prof. Bewersdorf