Power E850

2Q 2015 Power Systems
Announcement Highlights
Thomas Prokop
Consulting I/T Specialist
There are 7.1 billion people on the planet
6 billion of them have access to mobile phones,
only 3.5 billion of them use a toothbrush
2
© 2014, 2015 International Business Machines Corporation
Security Breaches
The average data breach in the US costs more than $11M
Source: Ponemon Institute 2013 Cost of Cyber-Crime study
A typical security attack goes unnoticed for nearly 8 months
Source: https://www.mandiant.com/news/release/mandiant-releases-annual-threat-report-on-advanced-targetedattacks1/
42% of security breaches are caused by configuration errors
Source: IBM Security Services Cyber Security Intelligence Index, 7/2013
Lost or stolen credentials play a role in an estimated 76% of
network breaches
Source: http://www.verizonenterprise.com/resources/reports/rp_data-breach-investigations- report-2013_en_xg.pdf
© 2014, 2015 International Business Machines Corporation
Agenda
Power E880 and E870 enhancements
The New Power E880
Gen3 I/O Drawer enhancements
New PCIe adapters and SRIOV
Other info sprinkled in here and there
© 2014, 2015 International Business Machines Corporation
Power E870 & E880 servers
Power E870
•
•
•
•
Power E880
Increased performance and scale
Built-in PowerVM Enterprise Edition
Enterprise RAS
System Control Unit (2U)
Built-in Active Memory Mirroring for Hypervisor
More performance per-watt
8 PCIe adapter slots per node
Up to 4 PCIe I/O drawers per Node
Share resources in Power Enterprise Pool
Medium Software tier
PowerCare Services included
24x7 Warranty
8 to 80 cores @ 4.19 GHz
8 to 64 cores @ 4.0 GHz
256 to 8TB Memory
1 or 2 nodes (5U) per system
•
•
•
•
•
© 2014, 2015 International Business Machines Corporation
8 to 128 cores @ 4.35 GHz
Up to 192 cores @ 4 GHz
256 to 16TB Memory
1 to 4 nodes (5U) per system
Built-in initial Elastic CoD days
6
Power Enterprise Servers with POWER8 processors
GA = 5 June 2015
Power E880
192 cores @ 4.02 GHz
1 – 4 node, 4 – 8S (12c)
Up to 16 TB memory
GA = 4Q14
9119-MHE
128 cores @ 4.35 GHz
1 – 4 node, 4 – 8S (8c)
Up to 16 TB memory
1st & 2nd node
3rd & 4th node
80 cores @ 4.19 GHz
1 – 2 node, 4 – 8S (10c)
Up to 8* TB memory
Power E870
9119-MME
64 cores @ 4.02 GHz
1 – 2 node, 4 – 8S (8c)
Up to 8* TB memory
1st & 2nd node
* 8TB was originally SOD as only 4TB initially available.
8TB announced April 2015 with June 2015 GA date
7
© Copyright IBM Corporation 2015
POWER8 Enterprise E870
E870
Node 1
System
Control
Unit
Node 2
Node
32-core
4.02
GHz
40-core
4.19
GHz
1
Node
32
cores
40
cores
2
Nodes
64
cores
80
cores
* IBM Statement of Direction for 2015. For more details see the IBM Power E880 Announcement Letter released on October 6, 2014. IBM’s
statements regarding its plans, directions, and intent are subject to change or withdrawal without notice at IBM’s sole discretion.
© Copyright IBM Corporation 2015
POWER8 Enterprise E870 and E880
E870
Node 4
Node 1
System
Control
Unit
Node 2
Node 3
E880
Node
32-core
4.02
GHz
40-core
4.19
GHz
32-core
4.35
GHz
48-core
4.02
GHz*
1
Node
32
cores
40
cores
32
cores
48
cores
2
Nodes
64
cores
80
cores
64
cores
96
cores
3
Nodes
96
cores
144
cores
4
Nodes
128
cores
192
cores
* IBM Statement of Direction for 2015. Ann planned 2Q2015 GA 2Q2015
© Copyright IBM Corporation 2015
POWER8 Enterprise E870 and E880 rPerf
E870
Node 4
Node 1
System
Control
Unit
Node 2
Node 3
*Using
E880
Node
32-core
4.02
GHz
40-core
4.19
GHz
32-core
4.35
GHz
48-core
4.02
GHz*
1
Node
674.5
856
716
976.4
2
Nodes
1349
1711.9
1432.5
1952.9
3
Nodes
2148.8
2929.3
4
Nodes
2865
3905.8
SMT8
© Copyright IBM Corporation 2015
High-end system capacity increases with POWER8
Power E880 can provide similar capacity as the Power 795 with ½ the cores
192c
Up to 39% more capacity
128c
256c
128c 128c
64c
Power 595
3,906
80c
64c
64c 96c
----- Power 795 -----
Power 780 Power 795
Power E870
Power E880
rPerf performance for fully-configured systems. Power 795 rPerf using 64c LPARs
© 2014, 2015 International Business Machines Corporation
11
Enterprise upgrade paths (Keep System Serial Number)
New - Upgrade within E880 from
32-core node to 48-core nodes
These upgrades must be
ordered by Nov. 30, 2015
2014
POWER7 780
2015
2015
9179-MHC
2014
POWER7 780
POWER7+ 780
9179-MHB
One or two
nodes
9179-MHD
POWER6 570
9117-MMA
POWER7 770
POWER6 upgrades to POWER7+
POWER6+ upgrades to POWER7+
POWER7 upgrades to POWER7+
Three or four
nodes
POWER8 E880
2015
Enterprise Power High-end
9117-MMC
POWER7 770
POWER7+ 770
9117-MMB
9117-MMD
2014
One or two
nodes
POWER6 570
9117-MMA
Clients that have 9406-MMA
MUST upgrade to 9117-MMA prior to
upgrading to the B, C, or D models
12
*All statements regarding IBM's future direction and intent are subject to change or
withdrawal without notice, and represent goals and objectives only.
POWER8 E870
Enterprise Power Midrange
© Copyright IBM Corporation 2015
April 2015
Processor Core Activation Features
E870
E880
9119-MME
9119-MHE
Processor SCM feature
#core (c) GHz
#EPBA
32c 4.02 GHz
#EPBC
40c 4.19 GHz
#EPBB
32c 4.35 GHz
#EPBD
48c 4.0 GHz
Static activation
#EPBJ
#EPBL
#EPBK
#EPBM
Mobile-enabled (static)
activation
#EPBN
#EPBQ
#EPBP
#EPBR
Mobile activation
#EP2S
#EP2S
#EP2T
#EP2T
Power Integrated Facility
for Linux (IFL)
Quantity 4
#ELJ5*
#ELJ7*
#ELJ6*
#ELJ8*
Minimum of 8 cores must be activated (any of the above activations)
Notes:
Activations associated with Power IFL can only be used for Linux partitions
Static activations can be converted to mobile activations. There is a minimum number of static
activations required on the server.
It is less expensive to buy a mobile-enabled static activation and convert it to a mobile
activation than to buy a static activation and convert it to a mobile activation
14
© Copyright IBM Corporation 2015
Summary Power E880 / E870 PCIe Enhancements
• Additional PCIe Gen3 I/O Drawer Config
– Up to 4 drawers per node -- 2X increase in drawers (fulfilling SOD)
• Up to 192 adapters on 4-node Power E880
• Up to 96 adapters on 2-node Power E870
– Ability to configure less than 2 drawers per node
– Ability to configure ½ drawers
• Support for more PCIe adapters in System node
– 15 previously announced features now supported (fulfilling SOD)
– 4 new, additional PCIe adapter features added
• Support for more PCIe adapters in PCIe Gen3 I/O Drawer
– 4 previously announced features now supported (fulfilling SOD)
– 5 new additional PCIe adapter features added
15
© Copyright IBM Corporation 2015
PCIe Adapter Details - E870/E880
Adapters available on POWER7+ now also supported in system node
•
•
•
•
•
•
•
•
•
•
•
•
•
•
•
PCIe LP POWER GXT145 Graphics Accelerator (#5269)
PCIe LP 4Gb 2-Port Fibre Channel Adapter (#5276)
PCIe2 LP 2-Port 4X IB QDR Adapter 40Gb (#5283)
PCIe2 LP 2-port 10GbE SR Adapter (#5284)
PCIe2 LP 3D Graphics Adapter x1 (#EC41)
PCIe2 LP 4-Port (10GbE+1GbE) SR+RJ45 Adapter (#EN0T)
PCIe2 LP 4-port (10GbE+1GbE) Copper SFP+RJ45 Adapter (#EN0V)
PCIe LP 2-Port Async EIA-232 Adapter (#EN28)
PCIe2 LP 2-Port 10GbE RoCE SFP+ Adapter (#EC27)
PCIe LP 2-Port 1GbE TX Adapter (#5281)
PCIe LP 2-Port 1GbE SX Adapter (#5274)
PCIe LP 4-Port 10/100/1000 Base-TX Ethernet Adapter (#5271)
PCIe LP 10Gb FCoE 2-port Adapter (#5270)
PCIe2 4-port (10Gb FCoE & 1GbE) LR&RJ45 Adapter (#EN0N)
PCIe LP 4-Port Async EIA-232 Adapter (#5277)
27 PCIe
adapter
features
added April
2015
New adapters in system node
•
•
•
•
PCIe3 LPX 4-port 10GbE SFP+ Copper Adapter (#EN16)
PCIe3 LPX 4-port 10GbE SR Adapter (#EN18)
PCIe3 LP 2-port 10GbE NIC&RoCE SFP+ Copper Adapter (#EC37)
PCIe3 LP 2-port 10GbE NIC&RoCE SR Adapter (#EC2M)
Adapters available on POWER7+ now also supported in PCIe Gen3 I/O Drawer
•
•
•
•
PCIe2 4-Port (10GbE & 1GbE) SR&RJ45 Adapter (#5744)
10Gb FCoE PCIe Dual Port Adapter (#5708)
PCIe 380MB Cache Dual Port 3Gb SAS RAID Adapter (#5805)
PCIe2 4-port (10Gb FCoE & 1GbE) LR&RJ45 Adapter (#EN0M)
New adapters in PCIe Gen3 I/O Drawer
•
•
•
•
16
PCIe3 4-port 10GbE SFP+ Copper Adapter (#EN15)
PCIe3 4-port 10GbE SR Adapter (#EN17)
PCIe3 2-port 10GbE NIC&RoCE SFP+ Copper Adapter (#EC38)
PCIe3 2-port 10GbE NIC&RoCE SR Adapter (#EC2N)
© Copyright IBM Corporation 2015
Power EXP24S Drawer & HDD/SSD
EXP24S I/O Drawer (#5887)
I/O maximums
expanded in April
2015 announcement
2U Form factor
24 SFF-2 SAS bays
For SSD or for HDD
Connects to PCIe SAS adapters
Power
770
Power
780
Power
E870
Power
E880
Max qty EXP24S
126
126
128
168
Max qty HDD in EXP24S
3024
3024
3072
4032
If that isn’t enough, ask for more
Notes:
• max of 16 EXP24s per PCIe Gen3 I/O drawer. Rule for good cable management practices.
• it is a good practice to keep the PCIe Gen3 I/O drawer in the same rack as the EXP24S to
minimize SAS cable lengths. Shorter SAS cables are thinner and less expensive. Using the
8-inch rack extender for 7014-T42 rack for configs with lots of I/O cables is a good thing for
cable management.
• Earlier generation I/O drawers such as #5886 or #5802 or #EXP30 not supported
17
© Copyright IBM Corporation 2015
Power E850
18
© Copyright IBM Corporation 2015
Power E850
The most agile 4-socket system in the marketplace,
designed for big data with flexible capacity and guaranteed utilization
Redesigned 4-socket, 4U POWER8 system
Improved performance, RAS, and efficiency
2 – 4 POWER8 processors – up to 3.7GHz
Up to 2TB* memory – 2X increase over Power 750
11 PCIe Gen3 slots
Dual 10Gb ethernet ports (x8 slot)
2 Integrated SAS controllers
8 SFF (2.5”) SAS bays + 4 SSD (1.8”)
Up to 4 PCIe Gen3 I/O drawers
Active Memory Mirroring for Hypervisor (option)
CUoD, Power IFL & Elastic Capacity on Demand
8408-E8E
•
•
•
•
•
24 to 48 active cores at 3.0 GHz
20 to 40 active cores at 3.3 GHz
16 to 32 active cores at 3.7 GHz
Small Software tier
3 years of service (varies by country)
*Statement of Direction for 4TB of memory on Power E850 All statements regarding IBM's future direction and
intent are subject to change or withdrawal without notice, and represent goals and objectives only.
© 2014, 2015 International Business Machines Corporation
19
Seamless capacity for non-disruptive growth
Power E850 Capacity on Demand
• CoD benefits
– Quickly add additional processor and/or memory in non-disruptive
manner
– Enhances system reliability by providing processor sparing
– System growth through activations, even after processors withdrawn
– More granular price points, to match needs and growth
– Inactive cores not subject to maintenance
– Inactive cores are not subject to PowerVM licensing
• CoD offering
–
–
–
–
–
20
For all processor frequencies
Permanent (CUoD) and Temporary (Elastic, Utility, Trial) activations
Used for cores and memory greater than 50% of the server’s capacity
Consistent activation rules for all configurations
Very granular, grow in increments of just one processor core
© 2014, 2015 International Business Machines Corporation
Flexible, affordable, high-performance capacity for Linux applications
Power E850 Integrated Facility for Linux
Power IFL
• Simplify operations and reduce complexity by colocating applications on a more scalable and
reliable server
• Streamline access to data and applications via
secure, high-performance virtual networking
• Grow seamlessly and accelerate deployment of
new applications and services
• Reduce overhead by leveraging existing
production and disaster recovery infrastructure
• Possible to have an all IFL configuration if desired
21
© 2014, 2015 International Business Machines Corporation
Virtual stack consisting of :
4 x CUoD processor activations
32 GB CUoD memory
activations
4 x PowerVM for PowerLinux
license entitlement
Scales in increments of 4 cores
Available on :
Power E850
Power E870
Power E880
21
Power E850
3 GHz
3.35 GHz
3.7 GHz
8
8
8
8
• 16 – 32 core system
• 2, 3 or 4 sockets
• 8-core processor
modules
10
10
10
10
• 20 – 40 core system
• 2, 3 or 4 sockets
• 10-core processor
modules
© 2014, 2015 International Business Machines Corporation
12
12
12
• 24 – 48 core system
• 2, 3 or 4 sockets
• 12-core processor
modules
• Min active cores = 50% of system max
• Min active memory = 50% of installed
• Capacity on Demand & Power IFL
22
12
Power Enterprise Servers with POWER8 processors
• New E880 with up to 192 cores @
4GHz (1-4 nodes)
• E880 now supports up to 128 cores
@ 4.35GHz (3rd/4th node )
• Up to 16 TB memory
• Up to 16 PCIe I/O Expansion
drawers (0-4 per node)
• MES upgrades from E880 4.35GHz,
POWER7+ Power 780 & Power 770
192 cores @ 4 GHz
Power E880
9119-MHE
1 – 4 node, 4-16S (12c)
Up to 16 TB Memory
128 cores @ 4.35 GHz
1 – 4 node, 4-16S (8c)
Up to 16 TB Memory
80 cores @ 4.19 GHz
1 – 2 node, 4 - 8S (10c)
Up to 8TB Memory
Power E870
9119-MME
• Up to 8 TB memory (4TB per node w/ 128GB DIMM)
• Up to 8 PCIe I/O Expansion drawers (0-4 per node)
64 cores @ 4 GHz
1 – 2 node, 4-8S (8c)
Up to 8TB Memory
48 cores @ 3 GHz
2-4S (12c)
Up to 2TB* Memory
Power E850
8408-E8E
40 cores @ 3.35 GHz
2-4S (10c)
Up to 2TB* Memory
•
•
•
•
New Power E850 4-socket/4U system up to 3.7GHz
16 to to 48 cores with Capacity on Demand
8 SAS bays + 4SSD for local storage
Up to to 4 PCIe Expansion drawers
32 cores @ 3.7 GHz
2-4S (8c)
Up to 2TB* Memory
*Statement of Direction for 4TB of memory on Power E850. All statements
regarding IBM’s future plans and direction are for information only and are
subject to change without notice.
© 2014, 2015 International Business Machines Corporation
24
Power S824L
Power S824
Power E850
Power E870
(1 Node)
MTM
8247-42L
8286-42A
8408-E8E
9119-MME
Sockets
2
2
4
4-8
Max Cores / Frequency
20 @ 3.42 GHz
24 @ 3.02 GHz
12 @ 3.89 GHz
16 @ 4.15GHz
24 @ 3.55 GHz
32 @ 3.72 GHz
40 @ 3.35 GHz
48 @ 3.02 GHz
64 @ 4.02GHz
80 @ 4.2 GHz
Max Memory
1 TB
2 TB
2 TB*
8 TB
Max PCI slots
7 PCIe x8
4 PCIe x16
7 PCIe x8
4 PCIe x16
3 PCIe x8
8 PCIe x16
16 PCIe x16
SFF SAS bays
18
18
8
0
Integrated split backplane
Yes
Yes
Yes
No
Max PCIE I/O Drawers
2
2
4
8
LPARS
20 per core
20 per core
20 per core
20 per core
Height
4U
4U
4U
12U
Operating System
Linux
AIX, IBM i, Linux
AIX, Linux
AIX, IBM i, Linux
Active Memory Mirroring for
Hypervisor
No
No
Yes (option)
Yes (Standard)
PowerVM
Optional
Optional
Optional
PowerVM EE License
Entitlement included
Capacity on Demand
No
No
CUoD, Power IFL, Elastic,
Utility, Trial COD
CUoD, Elastic,
Utility, Trial
Power Enterprise Pools
No
No
No
Yes
Installation
Customer Set-up
Customer Set-up
Customer Set-up
IBM installed
Warranty
3yr (9x5 NBD)
3yr (9x5 NBD)
3yr warranty services included
(varies by country)
1yr (24x7 4hr)
*Statement of Direction for 4TB of memory on Power E850 All statements regarding IBM's future direction and intent are subject to change or withdrawal without notice, and represent goals and objectives only.
© 2014, 2015 International Business Machines Corporation
25
Midrange system capacity increases with POWER8
Power E850 offers over 2X more system capacity than POWER7 750
D. and up to 56% more system capacity than POWER7+ 750/760
48c
+49% more per-core
vs fastest POWER7+
48c
32c
32c
40c
32c
48c
32c
24c
----- Power 750 -----
----- Power 750 -----
----- Power 760 -----
---------- Power E850 ----------
© 2014, 2015 International Business Machines Corporation
26
Power E850
8408-E8E
Processor
Modules
2–4
CDIMM
slots
16 – 32
PCIe Gen3 slots
7 - 11
HMC ports
Serial port
USB-2 ports
USB-3 ports
Fans
DVD bay
4U
SFF-3
SAS bays
Op Panel
USB-3 ports
1.8-inch
SSD bays
27
© Copyright IBM Corporation 2015
E850 Processor Modules and Activations
More modules = more memory & more I/O
Number of
processor
modules
CDIMM
memory slots
PCIe slots in
system unit
Max PCIe
Gen3 I/O
Drawers
2
3
4
16
7
2
24
9
3
32
11
4
E850 offers CoD processor flexibility
1st & 2nd processor modules 100% permanently activated
3rd & 4th processor modules optionally activated
Elastic CoD, Utility CoD, Trial CoD
29
© Copyright IBM Corporation 2015
E850 Memory Bandwidth & Organization
Up to 32 total DDR ports
DRAM
Chips
Memory
Buffer
16-64*
GB
16MB
16-64*
GB
16MB
16-64*
GB
16-64*
GB
Up to 410 GB/s
Up to 8 high speed channels,
8 GHz per channel *
16MB
16-64*
GB
16MB
16-64*
GB
16MB
16MB
16-64*
GB
16MB
16MB
16-64*
GB
(2 bytes read + 1 byte write -- per channel)
Up to 192 GB/s sustained
Processor
Module
Up to 512 GB / Socket (with eight 64GB CDIMMs)
SOD for additional memory capacity
30
© Copyright IBM Corporation 2015
Memory Performance/Configuration Insights
• Can Mix different size CDIMMs
– Can not mix sizes within a pair
– Can mix different size pairs on a module
Memory slot
Memory slot
Memory slot
Memory slot
POWER8
module
Memory slot
Memory slot
Memory slot
Memory slot
• Always plug in pairs
– Minimum of four CDIMMs per processor module:
• 2-socket = minimum of eight DIMMs, 3-socket = 12 DIMMs, 4-socket = 16 DIMMs
– Minimum of 50% activation or 128GB activation – which ever is larger
•Performance rules of thumb U Performance disclaimer “it depends”,
but should apply to the majority of typical commercial client situations
– More DIMMs = more bandwidth. Thus for max memory performance fill all
DIMM slots. (this leaves no room for growth without parts on floor.)
• Note that for many applications which aren’t memory dependent or when the server
isn’t running high utilization, having some empty slots is probably not very significant.
– Mixing different size DIMMs. For max performance make all DIMMs the same
size.
• Impact of having different sizes is probably very modest for most systems
32
© Copyright IBM Corporation 2015
Memory Bandwidth per Socket for 4-Socket Servers
E850
POWER8
POWER7+
POWER7
POWER6
0
50
100
150
200
GB/Sec
33
© Copyright IBM Corporation 2015
Power 4-Socket IO Bandwidth
E850
POWER8
POWER7+
POWER7
POWER6
0
50
100
150
200
250
300
350
GB/Sec
35
© Copyright IBM Corporation 2015
Power E850 Block Diagram – 2 Processor Module
CEC Planar
8Gbps
Proc Mod 0
DMI-A
DMI-B
DMI-C
DMI-D
A0
A2
A1
Chip-0
X1
8B
CDIMM
DMI-A
DMI-B
DMI-C
DMI-D
A2
A1
A0
Chip-1
48 PCIe G3 lanes
36
RAID Backplane
Chip-0
PHB1 x8
PSI-0
FSI
8Gbps
DMI-A
DMI-B
DMI-C
DMI-D
Chip-0
X1
8B
CDIMM
DMI-A
DMI-B
DMI-C
DMI-D
Chip-1
PSI-1
48 PCIe G3 lanes
C5
FSP2 Card
2x1GE, 1 Serial
2 USB 2.0
Chip-0
PHB1
x8
PCIe Gn3 x16 FHHL slot C10
Chip-1
PHB1 x8
DVD
SAS Cntrl
RAID #2
Chip-0
PHB1 x8
Chip-1
PHB0 x16
DASD BP
8 SFF
4 1.8” SSD
SAS Cntrl
RAID #1
Proc Mod 1
A2
A1
A0
Chip-0
PHB0 x16
Chip-1
PHB1 x8
Op-panel
A0
A2
A1
Chip-0
PHB0 x16
Chip-1
PHB0 x16
PCIe Gn3 x16 FHHL slot C12
PCIe Gn3 x8 FHHL slot C11
C11 for LAN adapter for IBM Mfg/Test
S0P0
PCIe Gen3 x16 FHHL slot C8
PCIe Gen3 x16 FHHL slot C9
S0P1
PCIe Gn3 x8 FHHL slot C6
PEX S1P8
8732
PCIe Gn3 x8 FHHL slot C7
S1P9
TI TUSB7340 USB 3.0 (2F,2R)
IO Planar
© Copyright IBM Corporation 2015
Power E850 Block Diagram – 3 Processor Module
CEC Planar
8Gbps
CDIMM
Proc Mod 2
DMI-A
DMI-B
DMI-C
DMI-D
A0
A2
A1
Chip-0
X1
8B
DMI-A
DMI-B
DMI-C
DMI-D
A2
A1
A0
Chip-1
48 PCIe G3 lanes
Chip-1
PHB0 x16
8Gbps
Chip-0
PHB0 x16
Proc Mod 0
DMI-A
DMI-B
DMI-C
DMI-D
A0
A2
A1
Chip-0
X1
8B
CDIMM
DMI-A
DMI-B
DMI-C
DMI-D
A2
A1
A0
Chip-1
48 PCIe G3 lanes
DVD
37
SAS Cntrl
RAID #2
RAID Backplane
Chip-0
PHB1 x8
Chip-0
PHB1 x8
PSI-0
FSI
8Gbps
DMI-A
DMI-B
DMI-C
DMI-D
Chip-0
X1
8B
CDIMM
DMI-A
DMI-B
DMI-C
DMI-D
Chip-1
PSI-1
48 PCIe G3 lanes
C5
FSP2 Card
2x1GE, 1 Serial
2 USB 2.0
Chip-0
PHB1
x8
PCIe Gn3 x16 FHHL slot C10
Chip-1
PHB1 x8
DASD BP
8 SFF
4 1.8” SSD
SAS Cntrl
RAID #1
Chip-1
PHB0 x16
Op-panel
Proc Mod 1
A2
A1
A0
Chip-0
PHB0 x16
PCIe Gen3 x16 FHHL slot C4
Chip-1
PHB1 x8
PCIe Gen3 x16 FHHL slot C3
A0
A2
A1
Chip-0
PHB0 x16
Chip-1
PHB0 x16
PCIe Gn3 x16 FHHL slot C12
PCIe Gn3 x8 FHHL slot C11
C11 for LAN adapter for IBM Mgf/Test
S0P0
PCIe Gen3 x16 FHHL slot C8
PCIe Gen3 x16 FHHL slot C9
S0P1
PCIe Gn3 x8 FHHL slot C6
PEX S1P8
8732
PCIe Gn3 x8 FHHL slot C7
S1P9
TI TUSB7340 USB 3.0 (2F,2R)
IO Planar
© Copyright IBM Corporation 2015
Power E850 Block Diagram – 4 Processor Module
CEC Planar
8Gbps
CDIMM
Proc Mod 2
DMI-A
DMI-B
DMI-C
DMI-D
Chip-0
X1
8B
DMI-A
DMI-B
DMI-C
DMI-D
2B @ 5.33 Gbps
A0
A2
A1
A2
A1
A0
Chip-1
Proc Mod 3
A0
A2
A1
Chip-1
PHB0 x16
8Gbps
A0
A2
A1
X1
8B
DMI-A
DMI-B
DMI-C
DMI-D
38
RAID Backplane
Chip-0
PHB1 x8
PSI-0
FSI
Chip-0
PHB0 x16
8Gbps
DMI-A
DMI-B
DMI-C
DMI-D
Chip-0
X1
8B
CDIMM
DMI-A
DMI-B
DMI-C
DMI-D
Chip-1
PSI-1
48 PCIe G3 lanes
C5
FSP2 Card
2x1GE, 1 Serial
2 USB 2.0
PCIe Gen3 x16 FHHL slot C1
Chip-0
PHB1
x8
PCIe Gn3 x16 FHHL slot C10
PCIe Gen3 x16 FHHL slot C2
Chip-1
PHB1 x8
DVD
SAS Cntrl
RAID #2
Chip-1
PHB0 x16
DASD BP
8 SFF
4 1.8” SSD
Chip-0
PHB1 x8
Proc Mod 1
A2
A1
A0
Chip-0
PHB0 x16
PCIe Gen3 x16 FHHL slot C4
Chip-1
PHB1 x8
PCIe Gen3 x16 FHHL slot C3
A0
A2
A1
A2
A1
A0
Chip-1
48 PCIe G3 lanes
SAS Cntrl
RAID #1
Chip-1
Chip-1
PHB0 x16
Chip-0
CDIMM
48 PCIe G3 lanes
Proc Mod 0
DMI-A
DMI-B
DMI-C
DMI-D
8Gbps
DMI-A
DMI-B
DMI-C
DMI-D
Chip-0
PHB0 x16
CDIMM
Op-panel
X1
8B
A2
A1
A0
48 PCIe G3 lanes
DMI-A
DMI-B
DMI-C
DMI-D
Chip-0
Chip-0
PHB0 x16
Chip-1
PHB0 x16
PCIe Gn3 x16 FHHL slot C12
PCIe Gn3 x8 FHHL slot C11
C11 for LAN adapter for IBM Mgf/Test
S0P0
PCIe Gen3 x16 FHHL slot C8
PCIe Gen3 x16 FHHL slot C9
S0P1
PCIe Gn3 x8 FHHL slot C6
PEX S1P8
8732
PCIe Gn3 x8 FHHL slot C7
S1P9
TI TUSB7340 USB 3.0 (2F,2R)
IO Planar
© Copyright IBM Corporation 2015
E850
OS
41
© Copyright IBM Corporation 2015
E850 Supported OS Levels
If installing AIX LPAR with any I/O configuration:
– AIX V7.1 TL3 SP5 and APAR IV68444, or later
– AIX V7.1 TL2 SP7, or later (planned availability September 30, 2015)
– AIX V6.1 TL9 SP5 and APAR IV68443, or later
– AIX V6.1 TL8 SP7, or later (planned availability September 30, 2015)
If installing AIX Virtual-I/O-only LPAR:
– AIX V7.1 TL2 SP1, or later
– AIX V7.1 TL3 SP1, or later
– AIX V6.1 TL8 SP1, or later
– AIX V6.1 TL9 SP1, or later
If installing VIOS:
– VIOS 2.2.3.51 or later
If installing the Linux operating system:
• Big Endian
– Red Hat Enterprise Linux 7.1, or later
– Red Hat Enterprise Linux 6.6, or later
– SUSE Linux Enterprise Server 11 Service Pack 3 and later Service Packs
• Little Endian
– Red Hat Enterprise Linux 7.1, or later
– SUSE Linux Enterprise Server 12 and later Service Packs
– Ubuntu 14.04.2, or later
– Ubuntu 15.04
42
© Copyright IBM Corporation 2015
Power E850
• AIX license tier:
– small
• PVU
– 100 except for Power IFL cores which are 70
• HMC
– Optional unless doing temporary CoD or SR-IOV
• Firmware
– 8.3 or later required
43
© Copyright IBM Corporation 2015
E850
Adapters
44
© Copyright IBM Corporation 2015
Power E850 PCIe Adapters Supported
as of June 2015 (page 1) in System Unit
Ethernet NIC
Ethernet NIC
Ethernet NIC
Ethernet NIC
Ethernet NIC & RoCE
Ethernet NIC & RoCE
Ethernet NIC & RoCE
Ethernet NIC & SR-IOV
Ethernet NIC & SR-IOV
Ethernet NIC & SR-IOV & FCoE
Ethernet NIC & SR-IOV & FCoE
Ethernet NIC & SR-IOV & FCoE
Ethernet NIC & FCoE
Ethernet NIC
Ethernet NIC & RoCE
45
4-port 1GbE RJ45
2-port 10GbE
10GBase-T
RJ45
4-port 10GbE+1GbE
SR optical
4-port 10GbE+1GbE
Copper twinax
2-port 10GbE
SR optical (no NIM)
2-port 10GbE
SR optical
2-port 10GbE
Copper twinax
4-port 10GbE
SR optical
4-port 10GbE
Copper twinax
4-port 10GbE+1GbE SR optical+RJ45
4-port 10GbE+1GbE Copper Twinax+RJ45
4-port 10GbE+1GbE LR optical +RJ45
2-port 10GbE SR optical
2-port 10GbE
SR optical
2-port 40GbE
QSFP+
* Supported, but not orderable as new adapter
** Supported, but not orderable as new adapter in most countries
#5899
#EN0W
#EN0S
#EN0U
#EC30
#EC2N
#EN38
#EN17
#EN15
#EN0H
#EN0K
#EN0M
#5708 *
#5287 *
#EC3B
NEW
NEW
NEW
NEW
© Copyright IBM Corporation 2015
Power E850 PCIe Adapters Supported
as of June 2015 (page 2) in System Unit
46
Fibre Channel
Fibre Channel
Fibre Channel
2-port 8Gb
4-port 8Gb
2-port 16Gb
#5735
#5729
#EN0A
Communications
2-port Async RS232
#EN27
SAS RAID
SAS Tape/DVD
SAS RAID
SAS RAID
SAS RAID/Tape/DVD
4-port no-cache PCIe3 for SSD/HDD
4-port tape/DVD PCIe3
4-port huge-cache PCIe3 for SSD/HDD
4-port huge-cache PLUS PCIe3 for SSD/HDD
2-port no-cache PCIe1 for HDD or tape or DVD
#EJ0J
#EJ10
#EJ0L
#EJ14
#5901
* Supported, but not orderable as new adapter
** see withdrawal announcements
NEW
© Copyright IBM Corporation 2015
Power E850 PCIe Adapters Supported
as of June 2015 (page 3) in System Unit
47
InfiniBand (IB)
2-port QDR IB
SR optical #5285
Graphics
Graphics
2D graphics for general use
3D graphics for RHEL7
#5748
#EC42
USB
4-port USB-3
#EC46
© Copyright IBM Corporation 2015
More PCIe Adapters supported in PCIe Gen3 I/O Drawer
See the I/O drawer list. Includes
Older 1Gb Ethernet adapters:
#5767, #5768, #5717, #5769, #5744
Crypto card
48
#EJ28
© Copyright IBM Corporation 2015
E850
Storage
Backplane
49
© Copyright IBM Corporation 2015
Power E850 – SAS Bays
(Front View)
Hot plug fans for Memory/Proc/PCIe cards
DVD
4U
Eight
SFF-3 SAS bays
HDD and/or SSD
Hot plug
USB-3 ports
DVD bay
Hot plug
Operator
Panel
Four
1.8-inch SSD bays
Hot plug
HDD/SSD and DVD run by integrated SAS controllers. Pick one of 3 options:
A) split backplane, two controllers
B) dual controller with cache,
or C) dual controller without cache
50
© Copyright IBM Corporation 2015
Power E850 Storage Backplane
• All E850 have a storage backplane with
– 8 SFF-3 SAS bays (HDD and/or SSD) & 4 1.8-inch SAS bays (SSD)
– DVD bay
– Integrated SAS controller for above bays
• High performance, based on IBM PCIe Gen3 SAS technology
• Easy Tier function supported in all backplane flavors
• Can have both SSD and/or HDD in all backplane flavors
• Select one of three integrated SAS controller options
– #EPVN: Dual controllers with write cache
• Effectively up to 7GB write cache with compression
– #EPVP: Dual controllers with zero write cache
– #EPVQ: Split backplane (2 independent controllers without write cache)
• Each controller runs 4 SFF-3 bays and 2 1.8-inch SSD bays (split top/bottom)
– Can change the configuration and MES order a different backplane in the
field. Be careful to backup data if changing arrays.
51
© Copyright IBM Corporation 2015
E850 Storage Backplane Options
• Must select one of three SAS
controller options:
#EPVQ
#EPVP
#EPVN
6+6 SAS bays
12 SAS bays
12 SAS bays
2x (4 SFF-3 & 2 1.8”)
(8 SFF-3 & 4 1.8”)
(8 SFF-3 & 4 1.8”)
2 SAS controllers
0 GB cache
DVD bay
Dual SAS controlrs
0 GB cache
DVD bay
Dual SAS contrllrs
7.2** GB cache
DVD bay
$ 3000
$ 6000
$ 3000
AIX / Linux
Easy Tier Function
JBOD
RAID 0 / 1
RAID 5 / 6 / 10
RAID 5T2 (Easy Tier)
RAID 6T2 (Easy Tier)
RAID 10T2 (Easy Tier)
Split backplane
yes
yes
yes
yes
yes
no
no
Yes
Yes
yes
yes
no
yes
Yes
Yes
yes
Yes
no
yes
yes
no
yes
yes
yes
yes
yes
no
** 1.8GB physical write cache provides up to effectively 7.2GB with compression
USA IBM list price. Price is subject to change. Reseller prices will vary.
52
© Copyright IBM Corporation 2015
PCIe Gen3 IO
Expansion
Drawer
53
© Copyright IBM Corporation 2015
PCIe Gen3 I/O Expansion Drawer Announced 2014
Feat #EMX0
Front view
12 PCIe Gen3 slots
4U drawer
Full high PCIe slots
Hot plug PCIe slots
Modules not hot plug
Feat #EMXF
Rear view
Feat #EMXF
Fan-out Module
6 PCIe Gen3 Slots
4 x8 & 2 x16
Fan-out Module
6 PCIe Gen3 Slots
4 x8 & 2 x16
Use same Blind Swap Cassette (BSC) as used in
#5802/5877/5803/5873 I/O drawer
54
© Copyright IBM Corporation 2015
PCIe Gen3 I/O Expansion Drawer Announced 2014
2015: growth and more configuration flexibility
Feat #EMX0
Front view
Enterprise add 2015
•
•
•
•
Scale-out add
2015
12 PCIe
Gen3 slots
4Uincrease
drawerPCIe slot max
Double the max drawers per node
• Dramatically
Full
high
PCIe
slots
Allow just one drawer per node
• Very similar
config
rules
to Enterprise
Hot plug PCIe slots
Allow using “½” drawers
• Can use “½”
drawer for
entry
Modules
not
hotprice
plug
Support more kinds of PCIe adapters
Feat #EMXF
Rear view
Feat #EMXF
Fan-out Module
6 PCIe Gen3 Slots
4 x8 & 2 x16
Fan-out Module
6 PCIe Gen3 Slots
4 x8 & 2 x16
Use same Blind Swap Cassette (BSC) as used in
#5802/5877/5803/5873 I/O drawer
55
© Copyright IBM Corporation 2015
Min/Max PCIe I/O Drawer per E870/E880 Node 2014
One system node
One system node
+
1
0
2
0 or 2 PCIe Gen3 I/O Drawers in 2014
(max 4 fan-out modules per node)
59
© Copyright IBM Corporation 2015
Min/Max PCIe I/O Drawer per E870/E880 Node 2015
One system node
One system node
One system node
+
0
1
2x more drawers
PLUS
More flexibility
One system node
One system node
+
+
+
1
1
2
2
2
3
3
1
4
0, 1, 2, 3 or 4 PCIe Gen3 I/O Drawers in 2015
(max 8 fan-out modules per node)
Requires 8.3 firmware level available June 2015
60
© Copyright IBM Corporation 2015
Min/Max PCIe I/O Drawer per E870/E880 Node 2015
One system node
0
One system node
One system node
+
1
One system node
One system node
+
+
+
1
1
2
2
2
3
3
1
For even more flexibility – can choose to have “1/2” drawers.
Thus any of the drawers could have a single 6-slot fan-out module
4
0, ½, 1, 1½, 2, 2½, 3, 3½ or 4 PCIe Gen3 I/O Drawers in 2015
(max 8 fan-out modules per node)
Requires 8.3 firmware level available June 2015
61
© Copyright IBM Corporation 2015
PCIe Gen3 I/O Drawer E870/E880 PCIe Slot Counts 2015
Sample quantities
of PCIe drawers
on the server
PCIe slots per
1-node server
PCIe slots per
2-node server
PCIe slots per
3-node server
PCIe slots per 4node server
0
8
16
24
32
1
2
3
4
18
26
34
42
12 drwr + 6 node
12 drwr + 14 node
12 drwr + 22 node
12 drwr + 30 node
28
36
44
52
24 drwr + 4 node
24 drwr + 12 node
24 drwr + 20 node
24 drwr + 28 node
38
46
54
62
36 drwr + 2 node
36 drwr + 10 node
36 drwr + 18 node
36 drwr + 26 node
48
56
64
72
48 drwr + 0 node
48 drwr + 8 node
48 drwr + 16 node
48 drwr + 24 node
76
84
92
72 drwr + 4 node
72 drwr + 12 node
72 drwr + 20 node
96
104
112
96 drwr + 0 node
96 drwr + 8 node
96 drwr + 16 node
6
n/a
8
n/a
12
n/a
n/a
16
n/a
n/a
144
152
144 drwr + 0 node
144 drwr + 8 node
Not all possible quantities of drawers shown, nor are “1/2” drawers shown
62
n/a
192
192 drwr + 0 node
maximum
© Copyright IBM Corporation 2015
PCIe Gen3 I/O expansion drawer PCIe Slot Math
2U
1-socket
2U
2-socket
One filled
2U
2-socket
Two filled
4U
1-socket
***
4U
2-socket
One filled
4U
2-socket
Two filled
S812L
S822
S822, S822L
S814
S824
S824, S824L*
PCIe slots in
system unit
6
6
9
7
7
11
x16 slots in
system unit
2
2
4
2
2
4
Max PCIe Gen3
I/O Drawer
½
½
1
1
1
2
PCIe Gen3
Drawer slots
6
6
12
12
12
24
PCIe slots used
for Optical
Cable Adapter
2**
2**
4**
2
2
4
Total Max PCIe
slots
6-2+6 =
10
6-2+6 =
10
9-4+12 =
17
7-2+12 =
17
7-2+12 =
17
11-4+24 =
31
Nearly 2x more slots
Over 2x more
nearly 3x
* If S824L using a GPU, can not have I/O drawer
** 2U uses double-wide Optical Cable Adapter which uses two PCIe slots per fan-out module. Because of x16 slot location, can only use half
of the x16 slots for attaching an I/O drawer.
*** requires a 6-core or 8-core server. 4-core server doesn’t support I/O drawer
63
© Copyright IBM Corporation 2015
IBM Scale-Out Power Systems
Open Innovation to put data to work
Power Systems with POWER8
Built with open innovation to put data to work
What’s NEW for Scale-out Power Systems
Designed for
big data
Ann Apr. 28th
GA June 9th
• For workloads requiring highest single thread performance 8 core
fast offering to S822 and S822L, delivering 16 cores at 4.15 GHz
• Double Scale-Out Power S814 and S824L Systems memory
capacity
• Expanded Linux Distributions, delivering the promise of ease of
porting Red Hat Enterprise Linux 7.1 support little endian on Power Systems
and now orderable from IBM
Open
innovation
platform
• ( June Ann/GA ) NVIDIA K80 GPU accelerator available for S824L
Superior
cloud
economics
• For clients who want directly attached SSD/disk per VM with
minimal VIOS virtualization External PCIe Gen3 IO extension drawer
to all POWER8 scale-out
• Add flexibility to fit in different data center environment
Expanding S824L offering with Non-GPU acceleration version available
110V power supply available for S814 Rack mount form factor
(SOD ) Water Cooling features available for S822 and S882L
65
IBM – Page 65
© 2014 IBM Corporation
Scale-out Systems built with POWER8
•
Leadership Data / Analytics performance in consumable Scale out form factor;
Foundation for Big Data / Analytic solutions
•
Portfolio is complemented by a full commitment to support an open stack of software
-
•
Ubuntu, SUSE, RedHat, PowerKVM, and Open Stack
S812L
• 1-socket, 2U
• Linux only
New in
2Q15
66
IO Expansion Drawer
Offerings to compete head to head with x86 Linux
S822L
•
•
•
•
•
•
2-socket, 2U
Up to 24 cores
1 TB memory
9 PCI Gen3 slot
Linux only
NEBs option
• 8-core, 4.15 GHz
option
• PCIe Gen3 I/O
Drawer
• SOD: Water
cooling
S822
•
•
•
•
•
•
•
2-socket, 2U
Up to 20 cores
1 TB memory
9 PCIe Gen 3
AIX & Linux
PowerVM only
NEBs option
• 8-core, 4.15 GHz
option
• PCIe Gen3 I/O
Drawer
• SOD: Water
cooling
S814
•
•
•
•
•
•
•
1-socket, 4U
Up to 8 cores
512 GB memory
7 PCIe Gen 3
AIX, IBM i, Linux
PowerVM only
4core/P05 (IBM i)
• 1 TB memory
• 110 v power
• PCIe Gen3 I/O
Drawer
S824L
•
•
•
•
•
•
•
•
•
•
2-socket, 4U
Up to 24 cores
Linux
NVidia GPU
Full offering (GPU not
required)
2TB of memory
8/16-core 4.15GHz and
24-core 3.52GHz option
PowerKVM (2H15)
Nvidia K80 GPU
PCIe Gen3 I/O Drawer
S824
•
•
•
•
•
2-socket, 4U
Up to 24 cores
2 TB memory
11 PCIe Gen 3
AIX, IBM i,
Linux
• PowerVM only
• PCIe Gen3 I/O
Drawer
© 2014 IBM Corporation
New announcement POWER8 Linux Only Scale-Out Portfolio
Red Hat Enterprise Linux 7.1 support little endian on Power Systems and
now orderable from IBM
Available today. Red Hat Enterprise Virtualization for Power Systems
PCIe Gen3 I/O extension drawer– adds up to 10 PCIe Gen3 slots available
for clients who want directly attached SSD/disk per VM with minimal VIOS
virtualization ( except S824L with GPU installed )
S812L
•
8 core fast offering, delivering 16 cores at 4.15 GHz for
workloads requiring highest single thread performance
S822L
•
•
S824L
67
2 TB of Memory
Non GPU acceleration version available
• PowerVM support
•
8/16 core 4.15 GHz and 24 core 3.52GHz offering,
for workload requiring high processor/memory bandwidth
• Red Hat, SUSE Linux available
© 2014 IBM Corporation
Scale-Out POWER8 Linux Support Metrix
*w/o GPU
68
** no EXP24S external disk expansion drawer and no external PCIeGen3 I/O drawer expansion available
© 2014 IBM Corporation
New Announcement Features Detail
8-core 4.15GHz available for S822 and S822L
•
•
•
•
S822 : feature code #EPXL; processor entitlement: #EPYL
S822L : feature code #ELPF; processor entitlement #ELAF
S822 will allow 1 or 2 processors to be ordered.
Limitation
• High acoustics due to fan speed to cool
• Max memory 512GB; 16GB and 32GB DIMM's only
• Due to high cooling requirement, the following adapters are not offered to be installed within
system CEC, they can only be installed in external I/O drawer:
- EJ11, EJ0M, EL60, EL3B
- EC32, EL3B
- EC2M, EL40
- EC37, EL3X
900W 110V Power supply option available for S814 rack mounted
(#EB2L)
•
•
•
Allow 900W as initial order and MES for rack mount 41A
Allow switching tower to rack without changing power supply
Allow switching rack to tower when customer has 900W in rack or by changing 1400W power
supply to 900W
FDR InfiniBand adapter available for Power Scale-Out server
•
•
•
•
71
S812L and S822L – EL3D
S814 and S824 – EC33
S822 – EC32
S824L – EL50
© 2014 IBM Corporation
SR-IOV
72
© Copyright IBM Corporation 2015
POWER8 SR-IOV NIC Enhancements
• History
– Announced 2014 on POWER7+ 770/780 with latest levels of firmware and
software. Supported newest IMFC card and specific 4-port Ethernet
adapters
– POWER8 SR-IOV SODs provided 2014
– Announced Feb 2015 for Power E870/E880 in system node (not I/O
drawer) for two specific 4-port Ethernet adapters (#EN0J and #EN0L)..
• New news: April 2015
– Announcing capability for all POWER8 servers (E850 in May)
• Scale-out servers have specific slots which support SR-IOV
– Adding support for specific slots in PCIe Gen3 I/O drawer
– Adding additional PCIe Ethernet adapter with SR-IOV
capability
73
© Copyright IBM Corporation 2015
April 2015 SR-IOV
LPAR #1
LPAR #2
In specific SR-IOV capable PCIe slots
– Any E870/E880 system node slot
– 2 Slots of a 6-slot Fan-out Module
– 4 Slots of a Power S814 (1S 4U) or S812L (1S
2U)
– 8 Slots of a S824 or S824L (2-socket, 4U) with
both sockets populated. If only one socket
populated, then is 4 slots.
– 7 Slots of a S822 or S822L (2-socket, 2U) with
both sockets populated. If only one socket
populated, then 4 slots
– Any slots Power E850 system unit
LPAR #3
• Pre-req:for April SR-IOV enhancements:
–
–
–
–
–
–
–
–
–
–
–
AIX 6.1 TL9 SP5 and APAR IV68443, or later
IX 7.1 TL3 SP5 and APAR IV68444, or later
IBM i 7.1 TR10, or later
IBM i 7.2 TR2, or later
Red Hat Enterprise Linux 6.5, or later
Red Hat Enterprise Linux 7, or later
SUSE Linux Enterprise Server 11 SP3, or later
SUSE Linux Enterprise Server 12, or later
Ubuntu 15.04, or later
VIOS 2.2.3.51, or later
Firmware level 8.3 or later
Low profile
- multi-OS
Full high multi-OS
Low profile Linux only
Full high Linux only
PCIe2 4-port (10GbE+1GbE) SR Optical fiber
#EN0J *
#EN0H
#EL38
#EL56
PCIe2 4-port (10GbE+1GbE) copper twinax
#EN0L *
#EN0K
#EL3C
#EL57
PCIe2 4-port (10GbE+1GbE) LR Optical fiber
#EN0N
#EN0M
n/a
n/a
PCIe3 4-port 10GbE SR optical fiber
** #EN16
#EN15
n/a
n/a
PCIe3 4-port 10GbE copper twinax
** #EN18
#EN17
n/a
n/a
Specific PCIe adapters
* Not truly low profile. Adapter is available in Power E870/E880 system node, but not in 2U server
** SR-IOV announced Feb 2015 for Power E870/E880 system node. Now available in other POWER8 servers.
74
© Copyright IBM Corporation 2015
SR-IOV O. A Technology of Interest
LPAR #1
LPAR #2
• Single Root I/O Virtualization
LPAR #3
• Runs “closer to the silicon” potentially offering some performance
efficiencies
• Doesn’t require VIOS as a pre-req and thus can do simple
virtualization under PowerVM without VIOS U. BUT U. VIOS
continues to offer many additional advanced functions
• Architecturally can virtualize a resource like an Ethenet adapter and
allocate/provide a user-defined minimum level of bandwidth to a
partition U Quality of Service (QoS)
– Ethernet NIC announced. FCoE and FC not announced.
• Can use VIOS & SR-IOV together
75
© Copyright IBM Corporation 2015
SR-IOV Performance Efficiency
• Shared Ethernet Adapter Bridging
– Traffic flows between the physical adapter and
client partition through the hypervisor and
VIOS partition
– Within the VIOS the traffic flows between the
virtual adapter driver and physical adapter
driver through the Shared Ethernet Adapter
bridge support.
– Latency and CPU utilization overhead
VIOS LPAR
Shared Ethernet
Adapter Bridge
Adapter
Driver
Virtual
Adapter
Driver
LPAR
Virtual
Adapter
Driver
LPAR
Virtual
Adapter
Driver
Hypervisor
VIOS
Shared Ethernet Adapter Bridging
• Hypervisor copies packets between LPAR and
VIOS
• VIOS Shared Ethernet Adapter bridge function
and adapter drivers
LPAR
• SR-IOV Direct Access Adapter Sharing
– LPAR has direct access to the adapter
– Latency and CPU utilization on par with
adapter dedicated to an LPAR
– HMC/hypervisor configures, but the adapter is
the thing which knows it is virtualized
76
Adapter
Driver
LPAR
Adapter
Driver
SR-IOV
Direct Access Adapter Sharing
© Copyright IBM Corporation 2015
SR-IOV
(Single Root I/O Virtualization)
for Ethernet NIC
Virtualization with VIOS example
• Redundant VIOS with one hardware resource
• Minimum amount of bandwidth for Quality of Service (QoS)
VIOS - A
LPAR #2
QoS
Min of y%
Gen 2/3 *
LPAR #3
VIOS - B
78
• * For specific adapters with SR-IOV capable
electronics In PCIe Gen2/3 slots
• For specific Integrated Multifunction Cards
with SR-IOV capable electronics (770+/780+)
• Under latest 7.8 firmware (770+/780+) or
later firmware on POWER8 server
• With recent OS level software
© Copyright IBM Corporation 2015
SR-IOV
(Single Root I/O Virtualization)
Dual VIOS
• Redundant VIOS with redundant hardware resource
• Minimum amount of bandwidth for Quality of Service (QoS)
QoS
QoS
SRIOV virtualization +
VIOS virtualization
Gen 2/3
VIOS - A
LPAR #2
SRIOV virtualization +
VIOS virtualization
Min of y%
Gen 2/3
LPAR #3
VIOS - B
79
© Copyright IBM Corporation 2015
Open innovation to put data to
work in a waitless world
To find out more about how the IBM Power Systems platform can help
meet your enterprise innovation needs, visit: www.ibm.com/power
© 2014, 2015 International Business Machines Corporation
THANKS
© 2014, 2015 International Business Machines Corporation
81
CAPI
82
© 2014, 2015 International Business Machines Corporation
82