Today OSS notes # 1488135 and #1459005 was released by SAP. The first note explains SQL Server compression in general and how secondary index compression and page compression is used. The OSS note also has a new ABAP report MSSCOMPRESS attached which can be used to compress tables and indexes. One of the design criteria for secondary index compression was to have the same type of compression on the secondary indexes and the primary clustered indexes (that holds the data). OSS note 1459005 includes the correction instructions with all the changes necessary in SAP Basis to support secondary index compression. Therefore applying these corrections or moving to the Basis Support Package levels which are mentioned in the OSS note is a mandatory prerequisite before starting with secondary index compression and page compression

The minimum version of SAP Basis has to be Netweaver 7.00.

We’ll present more about the new report MSSCOMPRESS on this blog over the next weeks.

Have a nice weekend

SAP supports SQL Server Data Compression (row- and page-compression) for all applications based on ABAP application server as of NetWeaver (NW) Basis 7.00 and SQL Server 2008. Per default, SAP used to create tables with compression type ROW and indexes with compression type NONE. This decision was made in the past for the following reasons:

• Indexes were not compressed because of some issues with SAP BW and missing development and test resources at that time. These issues have been fixed over a year ago. Since then, customers can use index-compression with SAP NW. However, the default behavior of SAPINST was not changed.
• Page-compression was not used by default, because of the expected higher CPU load on the database server. These concerns have been resolved within the last 12 months. On productive customer systems the decreased number of I/Os always overcompensate the slightly higher CPU usage caused by page-compression. The overall SAP response time is much better for a page-compressed SQL Server database in comparison to a row-compressed database. If there is a resource bottleneck on the database server, it is typically the I/O system, not the CPU resources, which are causing the bottleneck.

Due to the great experience SAP made with productive page-compressed SQL Server databases, it was decided to change the default for new SAP NW ABAP installations to page-compression (for data and indexes).

How SAP changed the default compression type

In order to change the default behavior of an SAP installation, it was decided, not to change SAPINST. Instead, the default behavior of R3LOAD and the SAP Data Dictionary (DDIC) was changed.

The template file DDLMSS.TPL defines the compression type for data and indexes. This file is created during data export by SAP (to create the installation export DVDs) or by a customer (for a system copy or a DB/OS migration). DDLMSS.TPL is used by R3LOAD during SAP system installation or system copy. The newest version of R3LDCTL creates a template file for page-compression. The newest version of R3LOAD is able to use the template file DDLMSS.TPL for page-compression. SAP note 1505884 (R3load: compression for table / index) tells the necessary versions of R3LOAD and R3LDCTL. It also describes, how to change the template file DDLMSS.TPL manually.

One year ago, the Data Compression support for SAP DDIC was introduced by SAP note 1459005 (Enabling index compression for SQL Server). By setting the profile parameter dbs/mss/compression you can change the compression type, which is used by the DDIC when creating new tables. The default used to be row-compression. After applying the correction instruction of SAP note 1581700 (PAGE compression support for DDIC & Hom./Het. System Copy), the default is changed to page-compression. This correction will be included in Basis Support Packages of all SAP releases as of NW 7.00. This may result in an unexpected behavior, if you have not set the profile parameter dbs/mss/compression. By applying the according Basis Support Package, the default compression type (for newly created tables) will change from row-compression to page-compression. As a result, you may have tables with different compression type within the same database.

However, SAP works fine with different compression types. A few tables are neither created by R3LOAD nor by the DDIC, for example all tables created by report MSSPROCS. You can manually change the compression type for tables and indexes using the report MSSCOMPRESS as described in SAP note 1488135 (Database compression for SQL Server).

Installing future SAP products with page-compression

All future SAP products and Service Releases based on NW ABAP will create the database tables and all secondary indexes with page-compression out of the box. There is nothing you need to configure in SAPINST or in the SAP profiles.

Installing already released SAP products with page-compression

The official and by SAP supported way to install a page-compressed, older SAP release is to

• Install the SAP system with the default settings (row-compressed)
• Apply the corrections of SAP note 1581700
• Perform page-compression on all tables using report MSSCOMPRESS

However, you may speed-up this process as follows. Once SAPINST has copied the executables to the local disk, replace R3LOAD with the newest version. Modify the file DDLMSS.TPL as described in SAP note 1505884 to use page-compression. Then start the data load. Once the installation has finished, immediately apply the corrections of SAP note 1581700. Use report MSSCOMPRESS to detect non page-compressed tables and compress them. Since this procedure is not officially supported by SAP, it should only be used by an experienced SAP consultant.

Performing system copies with page-compression

If you follow all the instructions of the system copy note 888210 (NW 7.**: System copy supplementary note), then the target database will automatically be created page-compressed. The key points are

• Import the correction instructions of SAP note 1581700 on the source system
• Use the newest version of R3LDCTL.exe for the data export
• Use the newest version of R3LOAD.exe for the data import

However, this only works for a system copy using R3LOAD. Using simply detach/attach or backup/restore of the database will not change the data compression type of any table.

SAP encourages their customers to convert existing non-Unicode systems to Unicode. However, several SQL Server customers hesitated to perform a Unicode Conversion, because of the increased storage requirements of UCS-2. This issue has been finally solved with Unicode-compression, which is a new feature of SQL Server 2008 R2. A Unicode Conversion running on SQL Server 2008 R2 even results in a decreased database size. Therefore hardware restrictions are no longer an excuse to stay on non-Unicode. The space reduction is also documented in SAP note 1139642 (Hardware Requirements in Unicode Systems).

In recent conversations with customers and partners it turned out, that the SQL Server feature “Unicode-compression” was not fully understood. Although the feature contains the word “compression”, it is rather an improved encoding than a typical database compression feature (like page-compression). However, when doing a system copy, the database will be page-compressed by default. Therefore you will see space savings caused by page-compression in addition to savings caused by the improved encoding of Unicode-compression. As a result, the (used space inside the) database will shrink more than 50% during a Unicode Conversion.

To better understand Unicode-compression, let’s first have a look at the space requirements of UCS-2 in the past:

Space requirement for an SAP database up to SQL Server 2008

SAP always used the UCS-2 character encoding for SQL Server, which uses a 2 byte fixed length encoding per character. This resulted in a space usage of 18 bytes to store the string “Microsoft” (which consists of 9 characters) in a column with data type NVARCHAR(40). In the Stone Age SQL Server releases 6.0 and 6.5 it was even worse, since all strings were stored in fixed length data types. NCHAR(40) results in 80 bytes on the disk to store the string “Microsoft”.
With SQL Server 7.0 and 2000 a major improvement was to use variable length strings for SAP (but still fixed length 2 byte encoding for Unicode characters). In SQL Server 2005 we introduced the vardecimal storage format, which decreased the space usage for decimal data, but did not change anything for character data (the vardecimal storage format was only used by SAP BW).
SQL Server 2008 was a huge step forward to save disk space. The data compression types NONE, ROW and PAGE were introduced. The strongest compression type PAGE is a real database compression, which compresses whole database pages and therefore works for all data types. The compression type ROW implemented variable length storage of all numeric data types (including INT, not only DECIMAL). It had no effect on character data types. Finally, the compression type NONE guaranteed the exact same storage format as in older SQL Server releases. The only reason for having the compression type NONE is to enable an upgrade of SQL Server without changing the content of the user databases. There is no other benefit of using compression type NONE in comparison to ROW.

Unicode improvements as of SQL Server 2008 R2

One of the main demands for SQL Server 2008 R2 was to reduce the storage requirements for an SAP Unicode database. The natural way would have been to introduce UTF-8 character encoding. However, this would have resulted in a different data type (NVARCHAR for UCS-2 and VARCHAR for UTF-8). This was simply not acceptable for SAP and their customers. Changing the data type of existing SAP Unicode systems would have resulted in a complete data export and import of the SAP database. The downtime for this procedure could easily be several days for huge SAP databases. SAP required a transparent way to store Unicode data with UCS-2 as efficient as with UTF-8.
The new feature in SQL 2008 R2 called Unicode-compression meets these demands by SAP. The feature name contains the word “compression”, because it only works with compression type ROW and PAGE. As already mentioned, compression type NONE was merely introduced in order to keep the old storage format during a SQL Server upgrade.

Technical details of Unicode-compression

Unicode-compression is totally transparent for a SQL Server administrator. There is no switch to turn this feature on or off. There is no different data type. Strings are encoded much more efficient, in the best case 1 byte per character. In the worst case, the space usage for a string will never be higher than before (without Unicode-compression).
The trick is to have 2 different encoding schemes. Each string can be stored in one of the 2 schemes. After upgrading from SQL Server 2008 to SQL Server 2008 R2, all Unicode strings are stored in the old scheme using 2 bytes per character. Once you insert or update a row, the containing strings are encoded using the new scheme, having a variable length of bytes per character (1 byte for ASCII characters, up to 4 bytes for extremely seldom characters). For strings resulting in an even number of bytes, an additional zero byte is added. The new scheme is used, if it needs less bytes than the old one. Otherwise the string is stored using the old scheme. When reading a row, you can identify the encoding scheme simply by looking at the storage size of each string. If the storage size is an even number of bytes, then the old encoding scheme was used. For an odd number of bytes, it was the new encoding scheme.
You can directly see the improvements of the new encoding scheme by using DBCC PAGE:

00000000106AC060: 2302aa01 02000900 12005351 4c203230 †#.ª... ...SQL 20

00000000106AC070: 3038104d 6963726f 736f6674 00000000 †08.Microsoft....

00000000106AC080: 00000000 d8000000 0000†††††††††††††††....Ø.....

In the old encoding scheme it looked like this:

0000000000000000: 70000400 02000002 001d002f 00530051 †p........../.S.Q

0000000000000010: 004c0020 00320030 00300038 004d0069 †.L. .2.0.0.8.M.i

0000000000000020: 00630072 006f0073 006f0066 00740000 †.c.r.o.s.o.f.t..

0000000000000030: 00000000 000000db 00000000 00††††††††.......Û.....

The creation of database indexes is often a significant part in SAP BW process chains. Even on up-to-date database servers the index creation on a BW cube may take several hours. The obvious way to speed-up this procedure would be to increase parallelism by setting the SQL Server configuration option "max degree of parallelism" (MaxDop). However, this has an impact on all SQL statements, not only on the index creation. Therefore this is not recommended by SAP. By implementing SAP note 1454130 (Data dictionary support for parallel/online index) you can configure the degree of parallelism for index creation in SAP BW. Thereby SAP adds the optimizer hint MaxDop, which sets the parallelism for single SQL commands, independent from the general SQL Server configuration. SAP note 1654613 (SQL Server Parallelism for SAP BW) describes in detail, how to configure parallelism in SAP BW for queries and index creation.

Impact of Parallelism

By implementing the notes mentioned above, the maximum degree of parallelism for creating indexes in SAP BW is 8. This means that up to 8 CPU threads on the database server can be used to create a single index. The actual number of CPU threads is set by SQL Server at runtime. It depends on the available CPU and memory resources. Therefore you might see varying execution times for the BW process chains.

We have measured the impact of MaxDop for index creation on a test server with 16 CPU threads. The indexes were created on a cube with 100 million rows and a size of 15GB (page compressed). The performance scales-up quite well with the number of CPU threads used:

Impact of PAGE compression

Our experience on customer side showed an overall performance increase with PAGE compressed databases. This is caused by a reduced number of disk operations for the cost of a slightly increased CPU usage. However, index creation cannot benefit from PAGE compression. It is CPU intensive and not I/O intensive, if all data pages of the table fit into SQL Server data cache. Writing the data to disk is performed asynchronously by the lazy writer and the checkpoint thread. Writing the transaction log takes the same time for PAGE and ROW compressed tables.

Therefore you could expect a performance decrease for the index creation on a PAGE compressed table. On our test system we indeed saw a slightly performance decrease for the index creation job of about 10%. However, this impact is minimal compared with the impact of MaxDop.

Impact of ONLINE index creation

The idea of ONLINE index creation is to allow concurrent access to the table while the index is created. The ONLINE option does not speed-up the index creation, quite the contrary. It reduces the impact of the index creation on other processes accessing the same table. Typically no other processes access a BW cube when the indexes are created. Therefore you do not benefit from ONLINE index creation for SAP BW. In our test system we have measured a performance decrease of over 50% when using ONLINE index creation.

Therefore we recommend to turn this off for SAP BW systems. You can configure the ONLINE index settings in SAP transaction DBACOCKPIT. It is turned off by default.

SQL Server 2012 introduced an additional storage format for data. Besides the normal row-oriented store, there is a column-oriented store as well (like in SAP HANA). In order to make it as transparent as possible, the new column store got exposed to the Database Administrators and Applications as a new index type: non-clustered column-store index.
A column-oriented store is optimized for aggregating mass data. A row-oriented store can be used optimal when accessing just a single row. In SQL Server 2012, the column-oriented store is build side by side with the row-oriented store. SQL Server Query Optimizer can decide at compilation time of a query which store to use best.

SQL Server 2012 Column-store support has been integrated into SAP BW for the releases 7.00, 7.01, 7.02, 7.11, 7.30 and newer. It will be available with the next SAP BW support packages, which are scheduled by SAP for August/September 2012. See details below.
The general support of SQL Server 2012 for SAP products based on SAP NetWeaver (7.0x and 7.3x) is planned for Q3/2012. See SAP note 1651862 for details.

Benefits of the column-store for SAP BW

A column-store index has many benefits: The administration of SAP BW is much easier, since you do not need to maintain any BW aggregates any more. The disk space usage is much smaller when using column-store indexes and the BW query performance is much better.

Query Performance

We have seen significantly increased BW query performance when using a column-store index in our test lab. There was a wide spread of performance increase, dependent on the data and type of query. Some queries were up to 50 times faster, while others did hardly benefit from the column-store index. The majority of the BW queries we tested ran about 3 to 5 times faster. However, we expect even better results with real customer data. The data in our test cubes was often artificially created and did not have any relation between the different characteristics, which prevented optimal compression and performance.

Space Savings

The fact tables on SAP BW cubes have one database index per dimension. All these indexes are dropped, once we move the data into the column-oriented store by creating a column-store index. Dependent on the number of dimensions you save a lot of disk space when using a column-store index. We have seen space savings of 50% and more, even when comparing with fully PAGE compressed fact tables.

No need for aggregates

When defining a column-store index in SAP BW for a cube, all aggregates are deactivated. The definition of the aggregates still exists in SAP BW. However, the database tables for the aggregates are dropped. This results in further space savings and less administration efforts when using a column-store index

Usage of column-store indexes for SAP BW

The column-oriented store in SQL Server 2012 is read-only. Therefore you cannot modify the data of a table as long as a column-store index exists for this table. Before loading data into the table, you first have to drop the column-store index. After the data load you should re-create the column-store index in order to optimize the query performance again.
This behavior perfectly fits to the e-fact tables of SAP BW cubes. In SAP BW each cube has 2 fact tables. The F-fact table is optimized for data load, while the E-fact table is optimized for query performance. Data can be loaded the whole day into the F-fact table. However, the E-fact table is only modified during a process called BW cube compression, typically running at night.

Improved BW cube compression for the column-store

A column-store is dropped during BW cube compression (aka condense or collapse). At the end of the BW compression process, the column-store is automatically re-created again. You may argue that the runtime of BW cube compression increases when using a column-store index. However, the opposite is the case. When using the column-store, there are only 2 indexes on the E-fact table (the clustered, unique P-index and the CS-index). Without the column-store there are many additional indexes on the E-fact table, which have to be maintained during cube compression.

In order to speed-up the cube compression, additional improvements have been implemented in the BW compression code. Index creation is performed using SQL Server parallelism (per default 8 threads for creating the column-store index). A MERGE statement is now been used also in SAP BW 7.00 (This feature was down-ported from SAP BW 7.11). In addition, a SQL Server optimizer hint (HASH JOIN) is used in the MERGE statement.

Required SAP Support Package

The next SAP BW support packages will contain several improvements. The code for the index maintenance (cube activation, index check, index repair, update stats), cube compression and repartitioning was standardized for all SAP BW releases (7.00 to 7.30). Further new improvements have been implemented. This code base is also needed for using column-store indexes of SQL Server 2012 with SAP BW. The new code will be delivered in the next SAP support packages in August/September 2012:

* SP 10 is sufficient for using column-store indexes with SAP BW 7.11. However, SAP may not generally support SQL Server 2012 for BW 7.11 at the point in time when SP10 is released.

These SPs do not contain any user interface to define a column-store index. The next but one SP (e.g. SAP BW 7.00 SP30) will contain the report MSSCSTORE, which has the look and feel of MSSCOMPRESS. With MSSCSTORE you can view and modify existing cubes by creating the column-store index in dialog or in an SAP batch job. In the meanwhile you can use the lightweight report ZMSSCSTORE, which will be delivered using an SAP note.

New Index Layout

With the next SAP BW support package, the index layout for SAP BW cubes changes slightly on Microsoft SQL Server. All new cubes will be created with the new layout. All existing cubes can be adjusted by using SAP transaction RSA1 ("Manage cube" => "Performance" => "Repair DB indexes") or by SAP transaction RSRV ("All Elementary Tests" => "Database" => "Database Indexes of an InfoCube and Its Aggregates"). However, repairing the cubes is typically not necessary. The indexes will be adjusted automatically when creating indexes within a BW process chain.

New Index layout (without column-store)

There will be no P-index on the F-Fact table anymore. This index was helpful in SQL Server 7. In newer SQL Server releases the query optimizer is smart enough to use the existing single column indexes of the F-fact table. However, there are two exceptions: APO cubes and delta-write enabled cubes still have a P-index on the F-fact table.

The field order of the P-index (on the F-fact and E-fact table) has been changed. It now starts with the time dimension, which is used in almost all SAP BW queries.

There is no index for the unit dimension on the fact tables any more. In addition, the index for the packet dimension on the e-fact table for non-cumulative cubes was removed. This already had been introduced in SAP BW 7.11 and will become standard in SAP BW 7.0x, too.

For the dimension tables, single-column indexes had been introduced with SAP BW 7.11. This has also been down-ported to SAP 7.0x. These indexes can improve query performance. However, data load may slow down (during SID generation). Therefore a new feature was implemented: The single-column indexes of the dimension tables are dropped in a BW process chain at the same time as the indexes of the F-fact tables. They are re-created again at the end of the process chain.

Index layout with column-store

On the E-fact table there is a non-clustered column-store index on all table fields. The index is visible in the SAP data dictionary and has the name "CS". Besides the column-store index, there is only one other index on the E-fact table: The P-index on all dimensions of the cube. The P-index is a unique, clustered index, having the time dimension as first field.
A column-store index will only be created on basis cubes, not on aggregates. It will also not be created for transactional cubes. Of course, it will not be created when running on SQL Server releases older than SQL Server 2012.

Creating a cube with CS-index

You can either turn on column-store indexes globally for the BW system or define per cube whether a column-store index is used or not. If column-store indexes are not turned on globally, then creating a cube with a column-store index consists of the following steps:

1. Create and activate the cube as usual in transaction RSA1
2. Define the cube as a column-store cube in report ZMSSCSTORE
3. Create the column-store index by either running "Repair DB indexes" in RSA1 or RSRV or run a BW process chain which contains the process type "Create Index". Alternatively you can simply again activate the cube in the modeling view of RSA1.
   

Report ZMSSCSTORE

In report ZMSSCSTORE you can define the use of column-store indexes for an individual BW cube or all BW cubes. You can choose one of four different options and then press EXECUTE (F8).

Turn on CS-index for a single cube

The individual cube (CUM_BT) is marked to have a column-store index. However, the column-store index is not created at this point in time (you have to run RSA1, RSRV or a process chain later). The aggregates of this cube are immediately deactivated.

Turn on global flag

When creating or repairing the indexes of any cube in the future, a column-store index will be created. In addition, these cubes will be individually marked to have a column-store, too. When setting "Process all BW Cubes", the content of the field "Process single BW Cube:" is ignored. Be aware, that all aggregates of all cubes are immediately deactivatied.

Turn off global flag

The global flag for column-store indexes is turned off. This does not have any effect on cubes, which are already individually marked to have a column-store. The aggregates of any cube are not touched. Deactivated aggregates are not activated again.

Turn off CS-index for a single cube

The column-store definition for an individual cube is removed. However, this has no effect as long as the global flag is turned on. Deactivated aggregates of this cube are not activated again.

CS-index in SAP Data Dictionary

The column-store index is defined in SQL Server and in SAP Data Dictionary (DDIC). Due to the restrictions of the SAP DDIC, only the first 16 fields of a column-store index are visible in SAP DDIC. However, on the database there are always all table fields contained in the column-store index. The CS-Index is defined in SAP DDIC as "SQL Server Only" (Include MSS). Once there is a column-store index, all single-column indexes are dropped on the database. In SAP DDIC theses indexes still exists, but they are marked as "Not for SQL Server" (Exclude MSS)

Here is an example of the index definition in SE11 for an E-fact table having a column-store index:

A similar cube without column-store index looks like this:

Current restrictions and future features

There are a few restrictions you should be aware when using column-store indexes with SAP BW:

Cube compression

Cube Compression is fully aware of the column-store index. The index is automatically dropped at the beginning and re-created at the end of the cube compression. However, an administrator should regularly check for failed cube compression jobs. If it fails for any reason, the column-store index may not exist anymore on the database. The missing column-store has no functional impact, but will probably result in very bad BW query performance.

Repartitioning

The Full Repartitioning also works with column-store indexes. However, it is typically not needed. When having a column-store index, we typically do not see any additional performance benefit by partitioning the E-fact table by a time characteristics. A failed Repartitioning job may also result in a missing column-store index (also this is very unlikely).

Selective Deletion

The Selective Deletion is currently not aware of the column-store index. Therefore you should manually drop the indexes in RSA1 before running Selective Deletion and manually create them again in RSA1 afterwards. Be aware, that Data Archiving is also using Selective Deletion. Column-store support for Selective Deletion is currently in development.

Remodeling

The Remodeling is a very seldom activity in SAP BW systems. In order to use it, you first have to remove the column-store definition for affected cube and re-create the database indexes.

System Migration

Homogeneous and heterogeneous system migration is currently not aware of the column-store index. Column-store support for System Migration is currently in development.

Last Week SAP released Support Package Stack 5 for EHP1 of NetWeaver 7.3. Hereby all of the following equivalent SPS levels are now available:

• SPS 27 of SAP BW 7.0 (SAP NetWeaver 7.0)
• SPS 12 of SAP BW 7.01 (SAP enhancement package 1 for SAP NetWeaver 7.0)
• SPS 12 of SAP BW 7.02 (SAP enhancement package 2 for SAP NetWeaver 7.0)
• SPS 8 of SAP BW 7.3 (SAP NetWeaver 7.3)
• SPS 5 of SAP BW 7.31 (SAP enhancement package 1 for SAP NetWeaver 7.3)

These SP stacks are prerequisites for using SQL Server column-store with SAP BW. We already described the SQL Server column-store in a previous BLOG (http://blogs.msdn.com/b/saponsqlserver/archive/2012/05/23/sql-server-2012-column-store-support-for-sap-bw.aspx).

In the meanwhile we implemented several improvements for the column-store. These improvements are delivered by means of a correction instruction in SAP note 1771177. This note has been released today.

The following improvements have been made since we posted our last BLOG:

• Report MSSCSTORE. This report is completely new. It has nothing to do with the report ZMSSCSTORE, described in our last BLOG.
• There is now a system default for using the column-store. There is no need anymore, to define each single SAP BW cube, which uses the column-store.
• BW Selective Deletion fully supports the column-store.
• A failed BW Repartitioning does not result in a missing column-store index any more.
• SAP System Copy and System Migration now fully supports the column-store.
• The column-store can now also be used for BW Real-Time Cubes.

For using the column-store, you have to perform the following steps:

1. You first have to apply the minimum Support Package Stack as listed in the table above.
2. After that, you have to import the correction instruction of SAP note 1771177 using transaction SNOTE. The correction instruction contains report MSSCSTORE and several further improvements.
3. Finally you have to apply at least cumulative update 2 of SQL Server 2012 (or any newer CU or service pack of SQL Server 2012).

You can download SQL Server 2012 CU2 (SQLServer2012_RTM_CU2_2703275_11_0_2325_x64) from http://support.microsoft.com/hotfix/KBHotfix.aspx?kbnum=2703275&kbln=en-us. Service Pack 1 of SQL Server 2012 is scheduled for November 2012. Once it is available, we recommend to install the service pack rather than applying the CU.

We also published an SAP SCN document, which describes the usage of the column-store with SAP BW in detail. This document is also attached to SAP note 1771177 and to this BLOG.

SAP BW query performance depends on many factors: hardware, database configuration, BW configuration and last but not least BW cube design and BW query design. When running a query, there are several caches involved: disk caches, database caches and the BW OLAP cache. You should keep this in mind when comparing BW query performance of different configurations or even different systems. In the following we are discussing the configuration options, which are specific for Microsoft SQL Server.

Prerequisites

First of all you have to make sure that there is no bottleneck in the most important system resources: CPU, memory and I/O. You can configure the maximum number of CPU threads, which are used for a single database query (see below). However, SQL Server may reduce the actual number of used threads, if there are not enough free worker threads. Therefore the runtime of the same query can vary greatly, dependent on the current system load. A memory bottleneck on SQL Server may result in additional I/O. When there are sufficient CPU and memory resources, then repeatedly running queries are fully cached. In this case the performance of the I/O system is not crucial.

A huge part of the overall runtime of a BW query can be consumed on the SAP application server, not on the database server. Therefore the system resources on the application server are important, too. A simple BW query typically consists of 3 parts:

• A database query running against the f-fact table of the cube
• A parallel running database query against the e-fact table of the cube
• An aggregation of the two result sets running in the OLAP processor on the application server (this process has nothing to do with "BW Aggregates", see below)

If you have never run BW cube compression (see below), then all data is in the f-fact table. If you do run BW cube compression after each data load, then all data is in the e-fact table. In both cases, there is no need to aggregate the two results sets, which reduces the BW query runtime on the application server.

In SAP BW data is typically loaded into cubes using BW process chains. These chains contain steps for dropping/re-creating indexes and updating the database statistics (for small dimension tables). If one of these steps fails, then you may clearly see a BW query performance issue.

1. Number of requests in f-fact table

The f-fact table of a cube is optimized for data load (DTP: Data Transfer Process), request deletion and BW cube compression. On SQL Server, the f-fact table is partitioned by the data load request ID (as it is the case for most other database systems supported by SAP BW). Having a separate partition per request ID, the request deletion is very fast and does not consume considerable transaction log space. The BW cube compression also benefits from this, because part of the BW cube compression is the deletion of the compressed requests. However, BW query performance does not benefit from this kind of partitioning. Quite the contrary, having more than a few hundred partitions results in a decreased BW query performance. We ran a suite of queries, running against f-fact tables with the same data but different number of partitions:

Average query runtime compared with 10 partitions (in percent, lower number is better)

Since the partitions are created and dropped automatically on the f-fact table, a BW administrator has no direct influence on the number of partitions. However, there are two ways to reduce the number of partitions. First of all, you should keep the number of data loads (DTPs) into a cube as low as possible by avoiding small request. You can combine many small requests to a single, large one by loading the requests first into a Data Store Object (DSO). The following DTP from the DSO to the cube creates then fewer but larger requests in the cube.

Secondly, you can perform BW cube compression, which reduces the number of partitions again. For best query performance you should compress all requests anyway (which deletes all rows in the f-fact table). BW query performance is still good, if just most of the requests are compressed. Some customers keep new requests for at least one week uncompressed in the f-fact table. Thereby they can easily delete faulty requests, which were loaded into the cube by mistake.

2. BW cube compression

The e-fact table of a cube is optimized for query performance. The process "BW cube compression" moves (aggregates) single requests from the f-fact table to the e-fact table. Dependent on the kind of data, the total number of rows can dramatically be reduced by this aggregation. SAP generally recommends BW cube compression for performance reasons, see http://help.sap.com/saphelp_nw73/helpdata/en/4a/8f4e8463dd3891e10000000a42189c/content.htm. BW cube compression has further advantages for inventory cubes. As a side effect, it reduces the number of partitions on the f-fact tables.

For Microsoft SQL Server, we did not always benefit from BW cube compression in the past. Reason was the index layout using a heap for the e-fact table when having conventional b-tree indexes.

However, when using SQL Server 2012 column-store index, we strongly benefit from BW cube compression for BW query performance. The process of cube compression became much faster, although it contains an additional step: It fully re-organizes the column-store index. Since the creation of a column-store index scales up very well, we use 8 CPU threads for this by default. You can change the default by setting the RSADMIN parameter MSS_MAXDOP_INDEXING using report SAP_RSADMIN_MAINTAIN.

3. SQL Server compression

In typical customer scenarios, with sufficient CPU resources and a fast I/O system, the performance impact of using SQL Server PAGE compression is minimal. When CPU is a bottleneck, PAGE compression may result in reduced query performance. On the other hand, query performance increases by PAGE compression, if I/O is a bottleneck. Therefore you should simply keep SAP's default: PAGE compression.

In our suite of test queries we have clearly seen this expected behavior of PAGE compression. When the table already was in the SQL Server data cache, then the increased CPU usage resulted in slightly slower BW queries. On the other hand, the query performance was better, if the data head to be read from disk first.

Average query runtime compared with NONE compressed f-fact table (in percent, lower number is better)

The disk space savings of ROW compression was as expected. The additional space savings of PAGE compression was only moderate, because the f-fact table only contains numeric fields. Best compression ratios have been seen with string fields. A large number of partitions also results in an increased space usage per table.

Average disk space usage compared with NONE compressed f-fact table, 100 partitions
(in percent, lower number is better)

4. Degree of parallelism

Per default, SAP BW requests two SQL Server threads per query (by using a MaxDop hint). You can change this default behavior by setting the RSADMIN parameter MSS_MAXDOP_QUERY using report SAP_RSADMIN_MAINTAIN.

We measured the performance when running BW queries on BW compressed cubes with column-store index, dependent on the used degree of parallelism. Two suites of queries were running against two different cubes. The biggest improvement was seen when moving from 1 CPU thread to 2 threads (This is typically only the case for column-store indexes, not for conventional b-tree indexes). Increasing to 4 threads improved performance noticeably. A further increase to 8 threads did not have any impact in many cases:

Average speed increase of column-store indexes compared with Maxdop 1
(in factors, higher number is better)

The impact of MaxDop depends on many factors like cube design, query design, actual data and used hardware. However, in all cases we have cleary seen the negative impact when using only a single CPU thread per query. That is why you should never set MSS_MAXDOP_QUERY to 1.

When there is a (temporary) CPU bottleneck, SQL Server can reduce the actual number of used threads for a query. In extreme cases, this could end up in one thread, even when MSS_MAXDOP_QUERY is set to 4 or higher. In particular when running BW queries against multi-providers you can run into this issue. A BW multi-provider is a logical cube, which retrieves the data from multiple basis cubes at the same point in time. This results in many simultaneously running SQL queries: 2 SQL queries (one on the f-fact table and one on the e-fact table) per basis cube. When using MaxDop 4 for a multi-provider consisting of 4 basis cubes, SQL Server may need up to 32 threads (2 tables * 4 cubes * MaxDop 4 = 32 threads). On a database server with less than 32 CPU threads this can result in actually using MaxDop 1 for at least one SQL query. Keep in mind, that the response time of a BW query is determined by the slowest participating SQL query.

That is why the default value of MSS_MAXDOP_QUERY (=2) was chosen relatively low. On a database server with more than 64 CPU threads, you may increase MSS_MAXDOP_QUERY to 3 or 4 (dependent on your workload while BW queries are running).

For best BW query performance, one should avoid running BW queries during high workload. For example, BW cube compression can be very CPU intensive, since it includes the re-creation of the column-store index using 8 CPU threads by default.

5. SQL Server 2012 Column-store

Using SQL Server 2012 column-store results in 50% to 80% space savings on e-fact tables, even compared with SQL Server PAGE compression. Customers report an increased query performance of factor 3 to 7 in average (compared with conventional b-tree indexes on the same system). Some BW queries hardly benefit from the column-store, while a few queries are even 20 times faster. However, for SAP BW and SQL Server 2012, the column-store is only available for the e-fact table. Therefore you must perform BW cube compression, if you want to benefit from the column-store.

A column-store index is optimized for large tables having some million rows. It is described in detail in a white paper (http://scn.sap.com/docs/DOC-33129) and in SAP note 1771177. Cubes with small fact tables, having only a few thousand rows, do hardly benefit from the column-store. However, the query performance of such small cubes is typically not an issue. Therefore we recommend using the column-store for all of your BW cubes, independent of their (current) size. Keep in mind that BW cubes are typically growing by and by.

6. Partitioning of e-fact table

In SAP BW you can optionally (and manually) partition the e-fact table by a time characteristics. In contrast to the partitioning of the f-fact table, this partitioning is intended to increase query performance. In the past we have seen mixed results when testing the impact of partitioning for BW query performance on SQL Server. Therefore it was typically not worth to re-partition a cube. However, when using SQL Server column-store, we benefit from partitioning of large e-fact tables.

Our test suite running on a 100,000,000 row cube showed the following results with MaxDop 4:

Average speed increase (in factors, higher number is better)
compared with b-trees, non-partitioned (lines 1 and 2)
and compared with column-store, non-partitioned (lines 3 and 4)

For SQL Server column-store indexes we see consistent query performance improvements when using partitioning of the e-fact table. The performance improvements compared with non-partitioned column-store are moderate (factor 1.14 if data is already in cache). However, this is an additional performance increase compared with conventional b-trees. For example, when a non-partitioned column-store index was 6.78 times faster than a b-tree index, then the partitioned column-store index was 7.64 times faster.

A column-store index is optimized for large tables having some million rows. Internally, a column-store index is divided into segments. Each segment contains up to one million rows. When using 8 CPU threads for creating the column-store index, you typically see 8 segments (per column), which are not fully filled with one million rows. When using partitioning for small tables, you further decrease the average segment size of the column-store index. Having too small segments decreases the query performance. Therefore you should consider partitioning only for tables with at least some dozen of million rows.

To fully benefit from the performance improvements of the partitioned column-store, you have to apply the latest version of SAP note 1771177 first (which will be released in April 2013). Then you should re-create the indexes of existing, partitioned cubes. The new code improvements optimize column-store segment elimination, in addition to having partitions. Therefore, you have a performance benefit on BW cubes (containing a filter on a time characteristics), even when creating only a single partition.

7. BW Aggregates

A BW aggregate is neither a configuration option nor is it SQL Server specific. However, we want to discuss aggregates here, since they were the preferred means to increase BW query performance in the past. A BW aggregate is a copy of an existing BW basis cube with a restricted number of characteristics and/or applied filters. BW aggregates are optimized for one or a few BW queries. Therefore you typically have to create many aggregates in order to support all BW queries running against a single basis cube. Technically, a BW aggregate looks like a conventional cube. It has two fact tables, each of them having its own database indexes. Since BW aggregates are logical copies of the cube, they have to be manually loaded and compressed each time data is loaded into the basis cube.

On the contrary, a column-store index on a basis cube is maintained automatically. The size of the cube is decreasing when using the column-store, not increasing by creating additional copies of the cube. There is no need to create new aggregates to support new or ad-hoc BW queries when using the column-store. Therefore the column-store is the new preferred means to increase BW query performance. Once you define a column-store index on a BW cube for Microsoft SQL Server, all existing aggregates of this cube are deactivated. This is done due to the fact that the BW OLAP processor is not aware of the column-store. Therefore using an aggregate (which never has a column-store index) is avoided, once the basis cube has a column-store index.

Comparing BW query performance

The easiest way to measure the runtime of a BW query is using SAP transaction RSRT. The runtime of a BW query consist of many components: SQL Query runtime, BW OLAP processor runtime, time for transferring the results set to the BW application server and the BW client (for example BEx or BOBJ) and the rendering of the results on the client.

When comparing BW query runtime between different systems or different configurations you have to ensure, that you measure under the same conditions. For example, when running a SQL query (as part of a BW query), SQL Server may or may not perform physical I/O. By repeating the same BW query you can ensure that all data is already in the SQL Server buffer cache. Alternatively you could run the SQL command DBCC DROPCLEANBUFFERS to ensure that no data is in the buffer cache.

If you want to measure the database performance, then you should turn off the SAP OLAP cache. Otherwise you typically measure the cache performance (single threaded performance of SAP application server) rather than the database performance. In the worst case, the database is not accessed at all, if the OLAP cache can be used. You can turn off the OLAP cache on BW cube level using "InfoProvider Properties" in SAP transaction RSA1:

Alternatively you can turn off the OLAP cache on BW query level using SAP transaction RSRT. However, in productive customer systems the OLAP cache is typically turned on. So why should you turn it off for performance tests? There are two reasons for this: Firstly, the likelihood of a fully filled OLAP cache is in a test environment much higher than in a productive system. Therefore you would benefit much more from the OLAP cache in a test system, which results in unrealistic measurements. Secondly, you typically want to tune the slowest BW queries running under the worst conditions (when the OLAP cache does not contain fitting entries by chance).

Some BW queries are independent from database performance by nature. When there is a big result set with millions of rows, then a huge part of the runtime is consumed for transferring the result set from the database server to the SAP application server, and finally to the BW client. In this case you are measuring the network throughput rather than the database performance.

The FAQ document attached to this blog is the cumulative knowledge gained from hundreds of customers that have moved off UNIX/Oracle and DB2 to Windows and SQL Server in recent years.  Intel based servers provide all of the required Enterprise Features at a fraction of the cost of UNIX hardware. 

SAP provide fast, safe and fully documented and supported tools to export SAP databases from UNIX/Oracle or DB2 and import to SQL Server.  The FAQ provides information on how to speed up the data export/import process and how to address common questions that come up on migration projects.

Q1.  How long does it take to migrate from UNIX proprietary servers and Oracle or DB2 to standard Intel commodity servers and SQL Server?

A1.  There are few if any SAP databases anywhere in the world that would take longer to export than 24 hours.  Typically 2TB per hour can be achieved with modern Intel servers running many hundreds of R3load processes exporting data from UNIX/Oracle or DB2.  The performance limitation is the network which is solved by the use of 10 Gigabit Ethernet.  Databases 10-15TB can be exported/imported over a weekend without concern.  A typical SAP migration project to commission a new Intel server & SAN, migrate Dev, QAS and Production systems takes 2-3 months

Q2.  Is it possible to upgrade the SAP release or perform a Unicode Conversion during a migration to SQL Server?

A2.  Yes, the FAQ contains details on how to do this

Q3.  What certification is required to perform a OS/DB migration on a production system?

A3.  Only SAP certified consultants may perform OS/DB migrations on production systems.  SAP offer the course TADM70 SAP Certified Technology Associate - OS/DB Migration for SAP NetWeaver 7.30.

There are many certified System Integrators that perform migrations:

1. Accenture
2. RealTech www.realtech.com
3. HP
4. Abeam www.abeam.com
5. Connmove www.connmove.de/
6. BNW www.bnw.com.au

Recently we have released several new customer case studies with details about customers that have moved their systems from UNIX/Oracle or DB2 to SQL Server

SATS - a spin off from Singapore Airlines move ECC, BW, SRM off IBM pSeries/Oracle to reduce operating costs and improve performance.

“Aviation is a challenging industry because high fuel and labor costs have eroded the profits of our airline customers. As a result, there is always pressure on the service provider to lower prices,” says Lee Siew Kit, Vice President of Information Technology Services at SATS.  

http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=710000002537 

Telecommunications provider in New Zealand moves ECC, BW and EP off IBM pSeries to dramatically improve performance and reduce operating costs.  BW performance improves dramatically on SQL Server 2012.

“We had used Microsoft technologies for many years, and so we were excited about SQL Server,” says Martin. “It was good to see that advanced features in SQL Server 2012 such as page compression, backup compression, and column store and table partitioning are seamlessly integrated into SAP BW. We knew it would be the right fit for us.”

http://www.microsoft.com/casestudies/Case_Study_Detail.aspx?CaseStudyID=710000001762

Other recent customers moving off proprietary UNIX servers include:

Fletcher Insulation http://www.insulation.com.au/– IBM pSeries/Oracle to HP Blade servers running Windows 2008 R2 and SQL Server 2008 R2

Incitec Pivot Limited http://www.ipl.com.au/– IBM pSeries/Oracle to HP Blade servers running Windows 2008 R2 and SQL Server 2008 R2 (case study coming soon!)

Paperlinx http://www.paperlinx.com/ 

Other recent SAP on SQL Server case studies include:

Coca Cola South Africa run Africa and Indo-China bottling and beverage on SAP on SQL Server

http://www.microsoft.com/casestudies/Microsoft-SQL-Server-2008-R2-Enterprise/Dataflo/IT-Firm-Improves-Database-Performance-and-Helps-Bottler-Deliver-Products-Faster/710000001243

Balance Agri Nutrients upgrade to SQL Server 2012, compress databases and improve performance

http://www.microsoft.com/casestudies/Microsoft-SQL-Server-2012-Enterprise/Ballance-Agri-Nutrients/Manufacturer-Boosts-Database-Performance-Cuts-Storage-Costs-and-Supports-Growth/710000002468

Air Products move SAP off IBM Mainframe to standard Intel servers

http://www.microsoft.com/casestudies/Microsoft-SQL-Server-2008-R2-Enterprise/Air-Products/Manufacturer-Shrinks-Storage-Needs-by-75-Percent-with-Upgrade-Helps-Business-Grow/710000001725

Saudi Electricity move off IBM Mainframes to standard Intel servers.  Saudi Electricity case study is coming soon.  This will show how Saudi Electricity consolidated their entire SAP SQL Server landscape onto a 4 node DB cluster (4 x HP DL580 G7 512GB RAM) and consolidated all application servers onto 14 HP G8 blade servers.  With more than 700,000 SAPS, 15,000 logged on users, 5,000 active users this is one of the larger SAP infrastructures in the EMEA region

In recent years the capabilities of Intel and AMD commodity servers have grown exponentially.  Processing power (SAPS) has increased and the cost of RAM has decreased dramatically. SQL Server leverages these developments in hardware technologies with features that reduce storage and memory footprint such as SQL Server PAGE compression and SQL Server Column Store to deliver “in-memory” performance to SAP customers.  This blog discusses performance aspects of commodity hardware for SAP customers running SQL Server.  Following the guidance in this blog will ensure customers achieve the great performance today and in the future at a reasonable cost running on mainstream supportable technology platform. 

1.        SAP Sizing Process

In order to maximize performance we recommend to apply additional factors and considerations after conducting a SAP sizing analysis.  Some factors are not covered by SAP sizing techniques at all such as the use of SSD and large amounts of RAM. There are a number of approaches to sizing SAP systems:

Reference Based Sizing– ST03 and EarlyWatch data is analysed and compared to another known customer with similar workload and a known hardware configuration

SAP Quicksizer– SAP tool that calculates SAPS, DB size and RAM based on many inputs including business document volumes and number of users

T-Shirt– sizing based on the anticipated number of  SAP users where precise information is unavailable 

To deliver optimal performance the output of traditional sizing processes is the starting point for sizing SAP on SQL Server.  This blog now discusses additional factors that must be included to deliver optimal performance. 

2.        Why Purchase More Memory than Sizing Tools Calculate?

SAP running on Windows and SQL Server can deliver substantially better performance when three additional factors are taken into consideration during a SAP sizing exercise.  (A) Increased RAM (B) Ultra-Fast disk storage – Solid State Disk (SSD) or NAND and (C) 10 Gigabit networking.  Microsoft and SAP resources who support SAP on SQL Server customers and who run performance labs have repeatedly identified these three areas as possible bottlenecks.

Due to advancements in commodity hardware platforms and SQL Server making more efficient use of memory through compression technologies, it is now possible in many cases to store all or almost all data of an entire SAP database in memory. Today there are hundreds of SAP on SQL Server customers that have used SQL Server PAGE compression to shrink their databases by 80-88% and run these systems on servers with large amounts of physical memory.  In all cases these customers see almost zero reads to disk, near 100% cache hit ratio and dramatically improved performance. As expected, SQL Server is able to read data from SQL Server cache hundreds of times faster than from disk regardless of how much SAN cache or other technologies are deployed.  A disk IO operation requires the database to issue a command to the operating system rather than containing the request within the context of the database process. External IO operations are always vastly more “expensive” than a lookup in SQL Server buffer which is an internal operation.

SQL Server 2012 introduces a new Column Store Index feature that dramatically improves performance of OLAP type queries and also dramatically increases the compression efficiency.  Compression ratios on BW infocubes of over 95% are common. SQL Server 2012 Column Store index is already released and integrated into SAP BW. Column Store structures like SQL Server’s Column Store Index benefit from having the complete structures in-memory when data is being retrieved.  OLAP databases that are larger than the amount of physical memory available will show similar benefits when they are running on ultra-fast SSD or NAND storage.  Terradata, Vertica, HANA and other Column Store engines leverage in-memory column-store structures and in some cases mandate that these column-store structures are in-memory resident.

Using with large amounts of RAM, physical IO has been observed to almost cease within several hours after restarting SQL Server.  The buffering mechanism will take a few hours to fully populate a 1TB data cache.

3.        Ultra-Fast Solid State Disk Storage (SSD) 

SQL Server like other ACID compliant databases performs write ahead logging and check-pointing of in-memory data.  In order for a database to satisfy the ACID requirements there must always be writes to disk in case of some kind of infrastructure failure.  All DBMS including all ACID “in-memory” technologies available today must still write to a persistent storage (disk) before a transaction is committed. Therefore the INSERT, UPDATE and DELETE performance of all “in-memory” databases are still dependent on disk performance.   All ACID “in-memory” technologies must also periodically checkpoint or write the changed memory contents to the database persistence layer on disk.   If changes to the contents of an “in-memory” database were never written back to the disk persisted copy of the database, the shutdown, abort and recovery process would take an extraordinarily long time.  The DBMS would need to apply all of the changes accumulated into the transaction log to disk based persistence layer – a busy multi-terabyte database could take some hours to recover if the checkpoint interval was too long.   "in-memory" databases that are ACID compliant therefore must still write to persistent storage for at least two reasons (1) Writing Transaction Logs (2) Regular Checkpointing of changed data.

In traditional RDBMS the exchange between disk storage and memory when the size of the database exceeds that of the physical RAM can be speed up with SSD. Solid State Disk and NAND technologies change the response times of these mandatory log and checkpoint write operations from something measured in milliseconds to something measured in microseconds.  As discussed in section 2 of this blog there will be little if any physical read to disk as the majority of the database is cache resident.  SSD and NAND do also greatly accelerate read operations. 

FusionIO and other similar technologies are significantly different from SAN based SSD solutions in terms of overall concept. FusionIO (or similar) are direct local disks and do not rely on a highly performing Fibre Channel network, MPIO software and other components.  FusionIO disks plug directly into the PCIe slots on a server motherboard and are therefore in very close proximity to the CPU and RAM.  Latency is reduced due to the close proximity and direct connectivity between disk, CPU and RAM, this is especially important for SQL Server transaction logs and tempdb.  

Local SSDs and PCI devices cannot be shared disks.  This does not present a problem however as SQL Server 2012 AlwaysOn does not depend on shared SAN storage to deliver High Availability.  AlwaysOn is therefore the preferred method to achieve High Availability with SQL Server Log and/or Datafiles are storage entirely on local SSD disks such as FusionIO.

With the availability of 2TB and 3TB cards increasing numbers of customers are deploying both SQL Server datafiles and logfiles onto those cards.   FusionIO cards plug directly into the PCI bus within a server and are only available for Intel/AMD based servers.

Solid State Disk solutions that are deployed include the following:

1.     FusionIO – www.fusionio.com (FusionIO cards are resold by most hardware vendors such as HP, Cisco, Dell, IBM etc)

2.     Violin Memory - www.violin-memory.com

BW Loads and other write intensive operations greatly benefit from SSD/NAND technologies.

4.        10 Gigabit Network

3 tier systems are generally recommended for medium and large sized customers.  3 tier systems depend on an extremely low latency, reliable and high bandwidth network connection between the database tier (1^st tier) and the SAP application server tier (2^nd tier).  The connection between the SAP application server and the client PC (3^rd tier) is not particularly performance critical.

In recent years Microsoft and network card vendors have developed a large number of technologies to offload network processing onto the network card and to better distribute network processing within Windows.  These technologies are now proven and evolved on 10 Gigabit Network cards from Emulex, Intel and others.  Many of these features are not available on 1 Gigabit Network cards.  Serious problems such as described in Network Settings, Network Teaming, Receive Side Scaling (RSS) & Unbalanced CPU Load can occur on 1 Gigabit networks.  Modern good quality 10 Gigabit Network Cards do not have these issues. 

It is mandatory to deploy 10 Gigabit network between the Database and the SAP application tier for larger systems. This is detailed in How to Setup a Dedicated SAP Application Server to DB Server Network blog

5.        Recommended Hardware Configurations

SAP Note 1612283 - Hardware Configuration Standards and Guidance  provides comprehensive details about modern Intel server platforms.  As stated in Note 1612283 Intel based platforms account for 99%* of all servers shipped each year and this is driving huge R&D resulting in dramatic performance gains every 12-18 months.  SAPS and the amount of memory supported. 

Database Servers – Minimum Recommended Configurations:

2 Socket AMD or Intel with 384-768GB RAM, 10 Gigabit Network Card (up to ~40,000 SAPS as at May 2013)

4 Socket AMD or Intel with 1000-2000GB RAM, 10 Gigabit Network Card (up to ~70,000 SAPS as at May 2013)

8 Socket Intel with 2000-4000GB RAM, 10 Gigabit Network Card (up to ~135,000 SAPS as at May 2013)

*for full disclosure review SAP SD Benchmark site

SAP Application Server – Recommended Configuration:

2 Socket AMD or Intel with 384-768GB RAM, 10 Gigabit Network Card (up to ~40,000 SAPS as at May 2013)

SAP work processes on all operating systems are single threaded.  Therefore the maximum performance of many SAP business processes is governed by the maximum “thread” performance or SAPS/thread.  SAP Note 1612283 - Hardware Configuration Standards and Guidance  discusses this topic in more detail, but in general it is recommended to buy servers with higher clock speeds (higher GHz).  4 socket servers have slower clock speeds than 2 socket servers and are not recommended for use as a SAP application server.  Many SAP ABAP application servers can be consolidated to a single 2 socket server with or without virtualization.  The attachment to Note 1612283 has more details

*IDC Server tracker data.  Gartner data.

6.        What about “4GB per core” or “6GB per 1000 SAPS” Sizing Formula?

To deliver optimal performance on SAP on SQL Server systems it is recommended to disregard these old and out-dated approaches to sizing.  These formula do not take into consideration concepts such as SQL Server PAGE compression, column store and several other important concepts. 

These guidelines that were developed at a time when CPU (SAPS) and RAM were both scarce and expensive, neither condition is true with modern Intel hardware (this does however remain true for proprietary systems). 

Failure to have sufficient memory in a database server will significantly impact the overall performance of SAP on SQL Server. 

Given the rapid innovation in Intel/AMD processors, Windows and SQL Server any static rule such as 4GB per core is invalid.  As server costs decrease and new features are added to Windows and SQL Server this blog will contain the information regarding optimal hardware configuration. This is based on lab testing with various hardware vendors and experience from real customer systems.

7.        How much extra does it cost to achieve “in-memory” performance?

Most hardware vendors allow customers to configure servers online and obtain an initial quotation.

In the example below www.dell.com is randomly selected to configure a SAP application server as follows:

Dell R720 2 x Intel E5 2670, 384GB RAM (24 x 16GB), Intel 10G Network Card, Power supply + 3 years support = ~USD 10,000

This configuration would support in the range of 3-8 SAP ABAP application instances/servers dependent on their size and workload and is very highly performing.  Cutting the memory from 384GB to 96GB holding all other factors identical saves only $3,800 USD(as of June 2013).  Attempting to run more than 2-3 SAP ABAP application instances/servers on a 96GB RAM configuration server could lead to severe performance problems.  In addition the very powerful processors will be unable to reach significant utilization due to insufficient memory.

A similar situation is observed with a 4 socket Database server:

Dell R910 4 x Intel E7 4850 2Ghz, 1000GB RAM (64 x 16GB), Intel 10G NIC, HBA, Power supply + 3 years support = ~USD 32,000

Reducing the memory to 128GB lowers the cost by only $9,700 USD(as of June 2013). SQL Server will be unable to fully cache the database in memory, unable to provide enough cache for features like Column Store, unable to effectively leverage the huge processing power available and be unable to fully exploit additional features with only 128GB.  A powerful 4 socket 40 core 80 thread server with only 128GB RAM will be factors of 3-10x slower than the same server with 1TB of RAM (with a cost of less than $10,000 USD). 

Based on actual customer deployments we have observed:

1TB of RAM can fully cache a 1TB PAGE compressed database. 

1TB PAGE compressed ECC 6 EHP 6 equates to between 4.5-6.5TB of uncompressed data.  1TB BW with Column Store could equate to well over 7-8TB uncompressed. 

1TB of RAM would also be sufficient to provide near in-memory performance for compressed databases up to 2.5TB in size (as the cache size is still a very high percentage of physical DB size).  Very little disk IO would occur and cache hit ratios will still be around 99.5%.

A 2.5TB ECC 6.0 EHP 6 PAGE compressed is in excess of 10-12TB uncompressed (such as an Oracle 10g database) and much larger for BW systems with Column Store.

As long between 25% - 50%  ratio between memory and compressed database size we will be able to achieve near in-memory performance, as this is usually enough to cache the active (in-use) part of the database. Large parts of SAP databases contain legacy information that is rarely or never accessed. SQL Server only needs to cache the portion of the database that is active to achieve a high level of performance in contrast to other in-memory databases that have a hard requirement to cache the entire database or they will not operate.

8.        Summary

Modern Intel based servers are benefiting from the economies of scale and momentum of Research & Development. 

Intel and AMD release new generations of processors and chipsets that drive performance, scalability and reliability ever higher every 12-18 months.  Simultaneously the cost of hardware is reducing. 

Recommendations:

1.     Memory is very inexpensive – do not purchase a server with less than 384GB

2.     Leveraging the vast CPU processing power is highly dependent on sufficient memory

3.     10 Gigabit network provides many offload and balancing features in addition to extra bandwidth – always purchase 10 Gigabit network for the internal SAP network

4.     Non-rotating storage in form of SSD and NAND based solutions have shown considerable performance gains – medium to large size database servers are likely to greatly benefit from SSD/NAND technologies

9.        Frequently Asked Questions (FAQ)

Q1: What is meant by the term “in-memory” and what is the difference between latchless in-memory and disk block caching?

A1: The concept of a database cache in memory is nothing new and has existed for decades.  On disk a database is divided into discreet pages (or “blocks”).  Disk block caching is a mechanism to replicate these disk structures in memory whilst preserving consistency.  Latches are a mechanism used by disk block caching functions to preserve consistency and can be a source of latency if there is an attempt to read and write to the same block simultaneously.  At a very simple level a “latch” is a lock (mutual exclusion - mutex) on a DB page/block.  Latchless in-memory removes the concept of a “page” or block.  Removing the concept of dividing a database into blocks and storing data as an array in memory can have some performance advantages for some applications.  Exactly how much additional incremental performance this provides over and above a database that is already fully cache resident using traditional disk block caching depends on many factors such as the type of queries, degree of locking/blocking and application coding. 

SAP Business Suite and ERP systems are OLTP applications and hardly see latch contention since critical tables have index structures that prevent this type of contention. Only a small amount of CPU resources (much less than 20% of the CPU consumption of SQL Server) is spent for searching qualifying rows in the buffer pool. The effect of having the data latchless in-memory resident is therefore very unlikely to have any significant performance advantage.

SAP BW is an OLAP application and therefore greatly benefits from data presented in memory in a column-oriented structure. Whether the structure is one that is latchless or not has no measurable effect on performance. 

The next version of SQL Server will include a latchless in-memory optimized functionality called "in-Memory OLTP", which is fully embedded into the SQL Server RDBMS engine and all Enterprise Grade RDBMS features such as HA/DR functionalities.  

Microsoft are still performing testing and analysis to determine the benefit(s) of a latchless in-memory type technology on SAP applications.  Future blog posts will contain updates on this technology.

Q2: What about Unicode databases?  How does Unicode effect sizing?  

A2: Releases since SQL Server 2008 R2 are as efficient as or even more efficient than other DBMS for some languages.  Unicode does increase the CPU and memory requirements especially for the SAP application server.  There is no overhead in storing Unicode data on disk relative to other DBMS and SQL Server will often be more efficient at Unicode storage for non-latin characters  

Q3: How to calculate SAPS per core and SAPS per thread?

A3: All SAP benchmarks are publically available at http://www.sap.com/campaigns/benchmark/index.epx download the certificate and find the total SAPS number and divide by the number of threads.  The number of cores is not a useful measure as SAP workprocesses do not execute per core.  As at May 2013 SAP on SQL Server demonstrates the highest SAPS/thread of any platform on the SD 2 Tier EHP 5 Benchmark.

Q4: How SQL Server be clustered with FusionIO cards? Isn’t shared storage always required? 

A4: SQL Server 2012 and higher supports AlwaysOn. AlwaysOn does not require shared storage of any kind.  See earlier posts in this blog for more information on AlwaysOn

Q5.  Servers with very large amounts of physical memory are much more susceptible to memory errors and corruption.  Is it true that the chances of faulty memory chips corrupting a database increase greatly on servers with 2TB, 4TB or more RAM?

A5.  Yes, it is true that servers with huge amounts of physical RAM have a much higher chance of memory errors. 

The cause of such memory errors are due to many reasons, but the majority of the time the cause is due to:

(A) Manufacturing faults in memory modules (DIMMs) or attached controllers and interfaces

(B) Memory contents are altered in “Single Event Upsets”– these are radiation induced changes in memory contents

Intel based servers and Windows 2012 have sophisticated tools for detecting problem (A) – manufacturing errors.  Faulting memory modules can be deactivated and a warning issued in the server BIOS and Windows Event Log. 

The problem with “SEU”, problem (B) is much more difficult to deal with as there is no way to prevent background radiation from corrupting memory other than to locate servers behind lead shields or utilize expensive technologies used in satellites.  The solution is therefore to detect and contain such errors.  The technologies to do this are not new and have existed for some years.  It is only in the last 4-5 years that these features have been available on commodity hardware.

Fortunately Windows 2012 and SQL Server 2012 has functionalities that are seen on large UNIX servers and Mainframes.  Videos showing this technology on the HP DL980 are below:

With Hyper-V: http://h20324.www2.hp.com/SDP/Content/ContentDetails.aspx?ID=3432

Native OS: http://h20324.www2.hp.com/SDP/Content/ContentDetails.aspx?ID=3397

It is therefore an essential criteria for any “Enterprise Grade” in-memory DBMS to fully support and integrate these technologies at every layer of the solution from the system board up to the DBMS buffer pool.  Unless this condition is met and demonstrated then the DBMS is not suitable for large mission critical workloads.

Q6. Does SQL Server always require the amount of cache to be equal to or greater than the size of the database(s)?

A6. No.  SQL Server disk block caching algorithms in combination with highly effective PAGE compression and large amounts of physical RAM dramatically improve database request performance.  SQL Server stores database pages in memory in compressed format making the disk-block caching mechanism more efficient.  Near zero physical read IO is observed whenever the amount of SQL buffer cache is greater than 25%-50% of the total DB size. 

Based on extensive workload testing and analysis with lab and real customer systems it has been observed that:

(A) Many DB tables display absolutely no benefit from being latchless memory resident (B) A small proportion of DB tables will greatly benefit from being latchless memory resident.

The design approach for SQL Server is therefore based on the observable facts that only a small proportion of tables in a database benefit from latchless “in-memory” structures.  Therefore the next version of SQL Server includes an analysis tool [AMAR (Analyse Migrate and Report)] to analyse a workload and determine which tables benefit from converting from latched tables into latchless in-memory structures.  The analysis tool will also include a further step that allows a query to be natively compiled into a binary further improving query performance.  

In conclusion it should be noted that SQL Server differs from other “in-memory” DBMS technologies quite significantly in that there will be no hard requirement to load 100% of the database into physical RAM.   This is due to the fact that only a smaller number of critical tables benefit from being held as a latchless in memory structure while most of the tables in OLTP and OLAP databases do not show any benefit.  Unlike other "in-memory" DBMS SQL Server is licensed by cores and not the amount of RAM.  As DB sizes grow SQL Server unit cost decreases. Some in-memory solutions actually require more physical memory than the size of the database and will abort when they run out of RAM.  While RAM is inexpensive, some solutions are licensed by physical RAM.

A fixed requirement to load the entire database into memory is seen as a scalability constraint as customer databases continue to grow ever larger.  SQL Server will provide excellent performance for the majority of the DB tables that do not benefit from latchless in-memory technologies via highly efficient disk-block caching leveraging the vast amounts of low cost RAM available in commodity servers.  The smaller number of tables in a DB that do benefit from latchless in-memory technologies can be readily identified and converted to latchless in-memory objects selectively. 

Q7.  How to determine how much “Latching” is happening on a SAP on SQL Server system?

A7.  SQL Server has a comprehensive monitoring framework exposed in Windows Performance Monitor.  Open Perfmon.exe and create a Performance Counter Log and include CPU, Memory (free memory, page in/sec and page out/sec) and these counters for Latching. Leave the monitor recording data every 15 seconds for a week or so. 

It is also possible quickly quantify how much latching has been occurring on a system by adding up the PAGELATCH_EX and PAGELATCH_SH. ST04 -> Performance -> Wait Events.  Note: PAGEIOLATCH_EX/SH is a wait counter for disk performance and is not directly related to latching.

As seen in the screenshot below the “total wait” time for latching is tiny in comparison to the other causes of wait such as ASYNC_NETWORK_IO which is the time taken to transmit data from the DB server to the app server over the network.  The results below would be typical of most SAP systems and shows latching is an insignificant source of latency on a SAP system.   

Customers are welcome to post any questions about the results of their performance traces or waitstats in this blog post.  This process will provide the most accurate scientific proof about the impact of disk-block caching vs. latchless in-memory performance for SAP systems.

This document contains more information: Diagnosing and Resolving Latch Contention on SQL Server

The script attached to this blog will also help pinpoint latching

Links:

More information on Intel RAS features and SEU

http://www.intel.com/content/dam/www/public/us/en/documents/white-papers/platform-level-error-strategies-paper.pdf 

http://www.intel.com/content/dam/www/public/us/en/documents/solution-briefs/misson-critical-xeon-e7-microsoft-sql-server-brief.pdf

More information on Column Store

http://blogs.msdn.com/b/saponsqlserver/archive/2012/12/17/sql-server-column-store-updated-sap-bw-code.aspx
http://social.technet.microsoft.com/wiki/contents/articles/3540.sql-server-columnstore-index-faq.aspx
http://www.mssqltips.com/sqlservertip/2586/sql-server-2012-column-store-index-example/
http://en.wikipedia.org/wiki/Column-oriented_DBMS
http://www.databasejournal.com/features/mssql/understanding-new-column-store-index-of-sql-server-2012.html
http://blog.kejser.org/2012/07/04/how-do-column-stores-work/
http://msdn.microsoft.com/en-us/library/gg492088.aspx

More information on Latching

http://sqlserverperformance.wordpress.com/2009/12/21/great-resource-on-sql-server-wait-types/
http://www.fusionio.com/solutions/microsoft-sql-server/ -> download kit.

As you dear reader realized, there was not that much activity on our blog over the last few months. Reason for the lower output of blogs certainly has to do with work that the team is performing right now in different projects with SAP. One of the projects we were working on hard is SQL Server 2014. Something we want to go into details in a few articles on this blog site.

Overall SQL Server 2014 is a development which mainly focused in the following areas.

• Extending the capabilities of SQL Server’s Column Store Index as it got introduced with SQL Server 2012
• Introduction of In-memory-OLTP
• Introduction of a few functionalities which integrates SQL Server better into Windows Azure Infrastructure as aService (IaaS)  scenarios
• A new Cardinality Estimation for the SQL Server Query Optimizer
• Plus a larger list of smaller improvement which we as those apply will describe in the series of blogs

In September CTP2 of SQL Server 2014 got released. This is the last CTP which is going to be published before we intend SQL Server 2014 to the public in the first half of 2014. As usual all the preparations with SAP are in place to start the evaluation process as soon as SQL Server 2014 is released to the public. Details about the supported SAP releases will be discussed in a later blog entry. But it is not planned to be massively different from what is supported with SQL Server 2012.

When looking at some of the features as Column Store Indexes or our In-Memory-OLTP one will see differences to some of our competitors in this space. Especially to some that also have Column Store and/or in-memory functionality. Background for these differences have to do with some overarching principles or facts taken into account when developing new functionality for SQL Server.

As such can be list:

• Investment protection of millions of customers who invested into SQL Server. Features that are meaningful and open up new type of applications for RDBMS need to be integrated into the existing SQL Server RDBMS framework. Creating just another new engine is not acceptable for the most of customers and partners who invested a lot of money and resources into existing SQL Server engines.
• Ability to have SQL Server leveraging new functionality on existing commodity hardware – no hard requirement to buy new hardware or expensive appliances to leverage features like Column Store Indexes or In-Memory-OLTP
• Features of SQL Server should be usable on typical Infrastructure as aService (IaaS) Cloud providers. Hardware used by typical IaaS Service providers, like Microsoft with Windows Azure, is commodity hardware though not with huge amounts of CPUs and Terabytes of memory as we find it for very high-end DBMS servers at customer sites. The price points which make IaaS Services with e.g. Windows Azure so attractive are achieved by using hardware with a very excellent price/performance ratio. Today (around January 2014) this usually is 2-socket hardware with moderate memory of around 128GB to 256GB. The ability for our customers to leverage new SQL Server features in Virtual machines running on such type of hardware is very important for feature design in SQL Server
• Integration of new functionality into existing HA/DR frameworks is key. In order to enable consumption of new functionality implemented in SQL Server, ideally there are no changes required in HA/DR configurations for mission critical configurations. Nor should 3^rd party software which is used for operational and administrative process such as backup/restore require adaption or changes. 

Given these four items for principles, solutions as they showed up in SQL Server 2012 and will show up in SQL Server 2014 might look differently than the solutions presented by some of our competitors. Some competitive products were created without any installed customer base or do have a different customer base where the main hardware used is not commodity based, but more based on proprietary hardware with a complete different price point and also no strings attached to any IaaS Service offer.

That just as background on the features we delivered and are going to deliver. Now let’s look at the first functionality which will change in SQL Server 2014

SQL Server Column Store Index

As reported in this Blog location in several articles, SQL Server 2012 introduced a Column Store Index which is fully supported by SAP BW. In the order, these are the articles published so far:

• http://blogs.msdn.com/b/saponsqlserver/archive/2012/05/23/sql-server-2012-column-store-support-for-sap-bw.aspx
• http://blogs.msdn.com/b/saponsqlserver/archive/2012/12/17/sql-server-column-store-updated-sap-bw-code.aspx
• http://blogs.msdn.com/b/saponsqlserver/archive/2013/03/19/optimizing-bw-query-performance.aspx

Summarizing and giving a bit more background on what was and is going to be delivered with our Column Store Index functionality we look at the following characteristics:

• In order to make Column Store Indexes really efficient one needed to break away from representing the Column Store Index data in the usual 8KB page format in SQL Server’s bufferpool.
• As a result Column Store Index data is represented in its own memory area (different than SQL Server’s buffer pool) which is not organized in 8KB pages, but deal with larger variable Column Store Index segments in size of several MBs. Having such an independent addressable memory space allows the algorithms for searching through Column Store Index data to be more efficient than searches of pages in the regular bufferpool. The size of a particular segment is defined by the number of rows and the compression factor one can achieve.
• Since a Column Store index segment contains values of one column only and these segments are stored independent of each other on storage, only segments which relate to columns in the select list of statements need to be loaded in memory
• Whereas the Column Store Index data is still being stored on storage broken down into 8KB pages as BLOB data. Means there was no need for introducing a new storage type with all its related changes and eventual impact to the 3^rd party software used by SQL Server customers
• As with other memory segments, like SQL Server bufferpool, the memory area used for Column Store Indexes is NUMA oriented. Additionally the memory area used to store Column Store Index structures is using ‘large pages’. This improves efficiency in virtual address to physical address mapping by the underlying hardware.
• The way how table joins work got changed and a so called ‘batch mode’ introduced which increases efficiency of joins leveraging Column Store Index structures dramatically.
• Great performance out of Column Store Index structures is achieved as well with parallel scanning of segments of the columns. Something that works extremely scalable with the implementation in SQL Server. A fact that also was considered in the integration of Column Store Indexes into SAP BW, where queries are getting hinted to be executed in parallel (configurable within SAP BW – see articles listed earlier).
• In order to follow the principles to develop functionality for SQL Server, decision was taken not to require in-memory residency of the Column Store Index data. Or in other words, in opposite to some competitors, where ALL column oriented data needs to fit into the memory available, the solution implemented in SQL Server still relies on storing Column Store Index data on storage and loading from storage if needed. Means Column Store Index also can be used in IaaS cloud environments like Azure.

An excellent graphics of how the Column Store Indexes of SQL Server are structured and stored on storage can be found in graphics #1 in this document:

http://research.microsoft.com/pubs/193599/Apollo3%20-%20Sigmod%202013%20-%20final.pdf

Since SQL Server 2012 Column Store Indexes were released with SAP, there were many customers applying it already. Many customers tested it with competitive products underneath SAP BW workload and decided that SQL Server Column Store Indexes fulfilled most of their criteria of:

• As long as the SAP BW reports were database bound (significant time of the report was spent on the DBMS layer), Column Store Indexes as supported by SAP BW in SQL Server improved performance of reports significantly and sufficiently by factors
• Investments into new server hardware or memory upgrades were moderate or in most cases not necessary at all. Though SQL Server will add another cache to keep Column Store Data, the overall size of a BW cube is smaller. Hence less row oriented pages need to be kept in the regular buffer pool. Means the reduction of pages that involve the SAP BW cube will more or less compensate for the memory used by Column Store data. Whereas with some competitive products significant investments into infrastructure would have been necessary.
• High-Availability and Disaster Recovery scenarios did work as before and no new investments or upgrades into 3^rd party software were required.
• Especially in migrations from other platforms and older hardware, the results running SAP BW scenarios were similar to those of competitive DBMS platforms, however with a free choice of infrastructure.
• Customers leveraging SQL Server 2012 realized very substantial compression over and on top of SQL Server PAGE compression.  An example is a customer where a single BW cube was 100GB uncompressed, 30GB after SQL PAGE compression and only 4.5GB when converted to Column Store. Note: Experiences in Compression do vary and are dependent on the structure of the BW Cube and more important on the data in the Cube.

Hence there are quite a bit of positive experiences that we could collect. Some of those experiences also went into further extensions to SQL Server 2014. The extensions to SQL Server 2014 in regards to Column Store Indexes can be listed like:

• In general the range of datatypes and the precision of datatypes that are able to be stored in Column Store Indexes got changed in SQL Server 2014. We encountered issues with some application based on BW which were using datatypes with a precision which were not supported in SQL Server 2012 Column Store Indexes. With SQL Server 2014, this will not be the case anymore
• We enabled the Column Store Index to become a clustered index. That means that the data only would be available in a column oriented representation. This is different to the non-clustered Column Store Index we got so far, where row plus column oriented representation was stored.
• SQL Server enables modifications to the clustered Column Store Index. Unlike the non-clustered Column Store Index which is and remained read-only, the clustered Column Store Index is modifiable

These are basically the most important changes which will enable a wider range of scenarios to be covered with SQL Server 2014 Column Store Indexes. What remained unchanged is that the Column Store Index data has no requirement to be in-memory resident only, but as before is stored in the SQL Server data files.

How do modifications to a Column Store Index actually work?

As we meanwhile all learned, column oriented data representation have some downsides compared to row oriented representation. Not too surprising given that over the last 25 years where one spent hundreds of developer years to really tune the row oriented data representation in RDBMS systems to its absolute best. With the developer years yet spent into column oriented data representation, there still remain a few areas where column oriented representation lag in performance behind the row oriented in certain usage scenarios. Some of these cases are:

• Selecting single rows with all the columns in the select list. Column store constructs are optimized in scanning highly parallel through their segments in order to aggregate values of a few columns. Where column store construct usually are less performing is when just one single row is desired, but all columns of this one row. The way how most column stores are represented in memory, this requires more CPU cycles than just searching through a B-Tree index, finding the page with the row and then reading the complete row stored on that page.
• Another difficult case for column stores are modifications of a single or multiple rows. The way how the compression is achieved is a bit more complex than just the page dictionary compression applied on database pages storing row oriented data. Therefore changing a row or multiple rows could require several activities within the column segment where column values need to be changed. This effort would be too  massive  for often time critical modification operations. Therefore one needs to find another way to perform modifications.

How it works can be demonstrated nicely with the simple graphics below:

Additionally to just have the clustered index structure which includes the column oriented data, there also is a row oriented table aside which includes so called ‘deltas’. As another additional construct to the read-only non-clustered index is a ‘Delete Bitmap’. The different modification operations are handled like:

• Deletes of rows in the clustered Column Store Index: Just mark the row in the ‘Delete Bitmap’ as logically deleted. This is nothing special in a sense, since such a practice already is applied in the row oriented implementation as well.
• Changes (Updates) to rows in the clustered Column Store Index: Is a two-step process which marks a row in the existing Column Store Index as deleted and a second steps inserts the row with the changed column values into the delta store.
• Inserting a row: The row is going to be inserted into the delta store.

At a point in time the delta store is merged in the background with the column oriented structures. Means the rows move from a row oriented form into a column oriented form organized in segments and row groups. The merge usually is driven of certain thresholds of rows are hit in the delta store. Specifically for SAP BW a SQL command got introduced that would allow to trigger a merge after a Delta Loads to the F-Fact table of a BW cube.

Again more details of the implementation can be read in:

http://research.microsoft.com/pubs/193599/Apollo3%20-%20Sigmod%202013%20-%20final.pdf

A look at the space savings

Assuming a fact table of 100 million rows, of a SAP BW cube, we made the following space measurements with the different versions of SQL Server. The BW cube implementation against the different versions of SQL Server did change over time purely from an indexing schema in order to leverage functionality like SQL Server Column Store Indexes.  This also demonstrates to a great deal the progress in reduction of infrastructure investments like storage, especially for storage infrastructure. Or to keep such investments limited despite highly increasing data volumes.

As visible the Fact table occupied around 35GB with SQL Server 2005. The introduction of Page Dictionary compression and its subsequent usage by SAP NetWeaver applications provided a 50% reduction of volume already. Leveraging the read-only non-clustered Column Store Index of SQL Server reduced the space again dramatically to 6.3 GB. This reduction is also related to a drop of other non-clustered row oriented indexes against a typical Fact table. Using the same SAP BW implementation, but just using the clustered Column Store Index instead of the non-clustered one again reduces the volume since the data is now held in the column oriented representation only. This reduces the volume down to 2.4GB.

Quite impressive steps going from SQL Server 2005 even to e.g. SQL Server 2008 R2 which provides Page Dictionary compression (including UCS2 compression). But even more impressive what the changes to Column Store Indexes can achieve in SAP BW. Certainly boosted by the fact that one can rid of a lot of row oriented indexes in those cases.

As mentioned earlier compression rates can vary. Some of our customers observed better factors using SQL Server 2012 Column Store Index. But as with all the compression algorithms in the RDBMS systems, the factors of compression are majorly influenced by the nature of the data. This is true for the traditional PAGE dictionary compression of SQL Server (and other RDBMS) as well as the compression factors which can be achieved with storing data in a column oriented manner. In the case of SAP BW, the factor also is highly dependent on the number of SAP BW Aggregates which we usually drop when applying the Column Store Index to a fact table. Therefore numbers can vary.

This article only should give a small overview on SQL Server Column Store Indexes and its extensions in SQL Server 2014. Details about the implementation into SAP BW, releases covered, Support Packages required will, as always, be delivered by Martin in form of OSS notes or blog articles at a time when we are coming close to release.

In the next article, we will look into the new SQL Server Cardinality Estimation logic as it will be released with SQL Server 2014 and whether it is applicable for SAP NetWeaver architecture or not.

Analyzing incoming SAP support messages around Azure, we realized that a lot of the messages were centered around asking for information and pointers to documentation. So we decided to take a first shot on listing some of the documentation and give some points to start.

Question #1: What Azure Virtual Machines are certified for SAP workload on Azure?

There is one point of truth to answer this question. This is SAP Note #1928533 – ‘SAP Applications on Azure: Supported Products and Azure VM types’.

You will find a table of Azure VM types that are supported together with its sizing data (SAPS). As the table indicates, not all the VMs were benchmarked in 3-Tier configurations. Nevertheless, most of them can be used. Please note that some VM types were certified using Premium Storage for the database files only. This is the publication location that gets updated first when new VM types get certified with SAP.

Question #2: What SAP applications and architectures are supported on Azure?

Again, the first answer is to look at the SAP Note. Please read the SAP Note #1928533 – ‘SAP Applications on Azure: Supported Products and Azure VM types’. Please go through the note carefully since there is a first section listing the Windows and Linux OS versions supported as guest OS. The following two sections then list SAP applications for the guest OS of Windows and Linux. In each of the last two sections the supported databases are listed as well. It is obvious that Oracle DBMS is not supported for SUSE or Red Hat Linux. This is a side effect of Oracle not supporting these two Operating Systems as guest OS on Azure or other public clouds in general. Hence SAP is also not supporting this scenario.

As indicated in Note #1928533, there is another SAP Note 2145537 – ‘Support of SAP Business Objects BI platform on Microsoft Azure’ which lists more details on the Business Objects products that are supported on Azure. With one exception SAP Data Services has another note that unfortunately is a bit harder to find. The SAP Note to look at is: #2288344 – ‘EIM Applications on Microsoft Azure’. Please note that SAP Data Services does support several data sources on Azure, like SQL Azure Databases or Azure Blob Store. Please see SAP Note: #2324793 – ‘Release Notes for SAP Data Services 4.2 Support Pack 7 Patch 1 (14.2.7.1237)’.

Question #3: Where can we find documentation to implement SAP workload on Azure?

More details on planning and deployment can be found in these locations:

• https://azure.microsoft.com/documentation/articles/virtual-machines-windows-sap-get-started/
• https://azure.microsoft.com/documentation/articles/virtual-machines-linux-sap-get-started/

These two locations contain a lot of documentation about deployment of SAP workload in either Windows Guest OS or Linux Guest OS. In both areas, we are describing the deployments with the Azure Resource Management framework. In the Linux section, you also find a quick start guide for SAP HANA on Azure. Additionally to just the documentation, we also published JSON deployment scripts for SAP. Just to avoid misunderstandings. These scripts will not install VMs with SAP software pre-installed. But you could install a landscape of SAP VMs, including VNets, Internal load Balancer, etc, are going to be installed. Use this publication to explore more details on these templates: https://blogs.msdn.microsoft.com/saponsqlserver/2016/05/16/azure-quickstart-templates-for-sap/

The documents describing deployments in the classic deployment framework can be found here: https://azure.microsoft.com/documentation/articles/virtual-machines-windows-classic-sap-get-started/

Be aware that the documentation about the classic deployment did not get recent updates since we encourage customers to deploy using the Azure Resource Management framework and because a lot of the new functionalities are becoming available in Azure Resource Management first or even exclusively. So, the ‘classic’ documentation should only be referenced if you still have some older deployments. Also, be aware that the VM/application support matrix in these documentation locations can be outdated compared to SAP Note #1928533 – ‘SAP Applications on Azure: Supported Products and Azure VM types’. As mentioned earlier, the SAP note is the single point of truth in those regards.

Question #4: Is SAP HANA supported on Azure and for which scenarios?

There are several locations where support of SAP HANA is documented. Start with SAP Note #1928533 – ‘SAP Applications on Azure: Supported Products and Azure VM types’. In the Linux OS section you find a statement like this: ‘SAP HANA – only under the conditions described in SAP note 2316233 (SAP HANA on Microsoft Azure Large Instance Types) and in the Certified SAP HANA Hardware Directory at http://global.sap.com/community/ebook/2014-09-02-hana-hardware/enEN/iaas.html.’ Moving to SAP Note # 2316233 – ‘SAP HANA on Microsoft Azure (Large Instances)’ you can find out that the following scenarios are supported on Azure large Instance stamps:

• Suite on HANA, S/4HANA and OLTP workload with up to 3TB memory demand
• BW, BW/4HANA and OLAP workload up to 1.5TB with scale-out of up to 22TB

Moving to the Certified SAP HANA Hardware Directory, you can find this data:

If you click on Microsoft Azure the following information appears:

So in essence the GS5 VM type (G5-Series with Premium Storage) is supported for all non-production workloads with SAP HANA and production OLAP scenarios, like BW, BW/4HANA. Scale-out is not yet supported.

Question #5: Is there an impact on SAP NetWeaver Licenses when running on Azure?

Yes, you will need certain SAP kernel patch levels to be able to operate SAP NetWeaver systems on Azure without the license key expiring after a redeployment of a VM. The SAP Note describing the necessary kernel patches are:

• #2243692 – ‘Linux on Microsoft Azure (IaaS) VM: SAP license issues’
• #2035875 – Windows on Microsoft Azure: Adaption of your SAP License2035875 – ‘Windows on Microsoft Azure: Adaption of your SAP License’

Question #6: There seems to be some special steps necessary to enable monitoring for SAP ST06, OS07 and OS07N.

Yes, in order to get monitoring resource consumption data from the host that runs the virtualization layer, you need to setup additional monitoring within an Azure VM. The principles of the Monitoring and the data provided is explained in SAP Note #2178632 – ‘Key Monitoring Metrics for SAP on Microsoft Azure.’

For Windows, the way to configure and setup the monitoring is described in https://blogs.msdn.microsoft.com/saponsqlserver/2016/05/16/new-azure-powershell-cmdlets-for-azure-enhanced-monitoring/ . The description is only valid for the ARM deployment model. To deploy the SAP monitoring for the classical deployment model, please read the related sections in this Deployment Guide.
Please also regard SAP Note #1409604 – ‘Virtualization on Windows: Enhanced monitoring’ in order to find the correct release/path of the SAP Host Agent that supports the Azure ARM deployment model.

For Linux, the SAP Note #2191498 – ‘SAP on Linux with Azure: Enhanced Monitoring’ gives a starting point.

A blog with some help analyzing eventual issues when deploying the SAP Monitoring got published here: https://blogs.msdn.microsoft.com/saponsqlserver/2016/01/29/the-azure-monitoring-extension-for-sap-on-windows-possible-error-codes-and-their-solutions/

Question #7: Are there special support requirements when running SAP on Azure?

Yes, there are special support requirements. For running Windows as Guest OS, the major request is that you have a Premier Support contract with Microsoft. Details are listed in SAP Note #2015553 – ‘SAP on Microsoft Azure: Support prerequisites’. For SUSE Linux you are required to use SUSE Linux Enterprise Server for SAP Applications. For Red Hat you are required to leverage Red Hat Enterprise Linux for SAP Business Applications or Red Hat Enterprise Linux for SAP HANA. Both of these releases contain support with the Linux providers.

Question #8: Are there SAP notes covering the topic of Azure Standard and Premium Storage?

Yes, there is a SAP note that covers this topic. The SAP Note is # 2367194 – ‘Use of Azure Premium SSD Storage for SAP DBMS Instance’.

Question #9: Are there Single Sign On integrations between Azure and SAP SaaS services?

Absolutely, these SSO integrations do exist and are handled in the following SAP Notes:

• #2335666 – Microsoft Azure Integration with SuccessFactors SSO
• #2348735 – Tutorial: Azure ActiveDirectory integration with SuccessFactors

There is work on other integrations which should be released later this year.

Question #10: Are there any images of SAP on SAP CAL that can be deployed on Azure

Absolutely. There are images offered through SAP Cloud Appliance Library which can be deployed on Azure. Besides SAP HANA, SAP S/4HANA, BW/4HANA or some specialized offers like, SAP Connected Health Platform and Medical Research Insights, Cloud Appliance Library also offers a SAP IDES system on Windows and SQL Server for deployment. For details on how to deploy, these instances, please check out this blog: https://azure.microsoft.com/en-us/documentation/articles/virtual-machines-linux-sap-cal-s4h/ and this blog: https://blogs.msdn.microsoft.com/saponsqlserver/2016/08/03/__trashed/

At the moment, the images are getting deployed in the classical deployment framework with Azure. However, we are in the process of enabling deployments through Azure Resource Management as well.

Question #11: How does the SAP sizing work in Azure?

The base of sizing infrastructure for SAP workloads is usually the SAP throughput unit ‘SAPS’. This unit is derived out the SAP Standard Sales and Distribution benchmark (http://global.sap.com/campaigns/benchmark/measuring.epx). One of Microsoft’s obligations as infrastructure provider through Azure is to classify the SAPS of every Virtual machine type we certify for SAP usage. If you look into SAP Note #1928533 – ‘SAP Applications on Azure: Supported Products and Azure VM types’, you will see the SAPS throughput of each single VM in the column ‘2-Tier SAPS’. How you use these SAPS values to do sizing is described in more details here: https://blogs.msdn.microsoft.com/saponsqlserver/2015/12/01/new-white-paper-on-sizing-sap-solutions-on-azure-public-cloud/

Have fun

SAP recently released support for SQL Server 2016 for some SAP NetWeaver based systems, see https://blogs.msdn.microsoft.com/saponsqlserver/2016/10/04/software-logistics-on-sql-server-2016. SQL Server 2016 contains many new features which can be used by SAP. For SAP BW, particularly the improved columnstore features are relevant. An overview of SQL Server Columnstore in SQL Server 2014 is published here: https://blogs.msdn.microsoft.com/saponsqlserver/2015/03/24/concepts-of-sql-server-2014-columnstore. The most important columnstore improvements in SQL Server 2016 are discussed below:

Batch mode available for more operators and with maxdop 1

Since SQL Server 2014 there are three types of parallelism used for columnstore processing: Firstly, several SQL Server queries are running in parallel for a single SAP BW query (for example a SQL query against the f-fact table and another SQL query against the e-fact table). Secondly, a single SQL query can use many CPU threads at the same time (maxdop). Thirdly, a single CPU thread can process a set of rows, typically up to 900 rows (batch mode), rather than processing row by row (row mode). SQL Server 2016 can use the batch mode for much more operators, for example the sort operator. An overview is available at https://msdn.microsoft.com/en-us/library/dn935005.aspx.
In SQL Server 2014, single-threaded queries running under MAXDOP 1 or with a serial query plan cannot use batch mode. Therefore, we recommended a relatively low value for the SAP RSADMIN parameter MSS_MAXDOP_QUERY, which controls the maxdop setting used for BW queries (see https://blogs.msdn.microsoft.com/saponsqlserver/2013/03/19/optimizing-bw-query-performance). However, in SQL Server 2016 batch mode can be used with maxdop 1. Therefore, queries are still pretty fast, even when there is only one CPU thread available during high workload.

Columnstore uses vector instructions (SSE/AVX)

In addition to intra-query parallelism (maxdop) and batch mode, SQL Server 2016 uses another kind of parallelism: CPU vector instructions (AVX or SSE). You benefit most from this feature on most modern CPU generations. You can even use this feature in virtualized environments. More recent versions of virtualization layers used for private and public cloud like Azure, are supporting SSE/AVX instructions as well. SQL Server usage of SSE/AVX is described here: https://blogs.msdn.microsoft.com/bobsql/2016/06/06/how-it-works-sql-server-2016-sseavx-support/

Rowgroup Compression also merges rowgroups

In SQL Server 2014, you can perform columnstore rowgroup compression using the SQL command ALTER INDEX REORGANIZE. In SQL Server 2016, this SQL command performs additional optimizations: It also merges small rowgroups into a larger one for improving query performance. This is documented here: https://msdn.microsoft.com/en-us/library/dn935013.aspx. In SAP BW, there is no need to run this SQL command manually. This is automatically done during BW Cube Compression and when executing an Index Repair (within a BW Process Chain).

Parallel update of sampled statistics

Update Statistics on BW fact tables can be very time consuming, in particular for a BW Flat Cube (see https://blogs.msdn.microsoft.com/saponsqlserver/2015/03/27/columnstore-optimized-flat-cube-in-sap-bw). By default, an Update Statistics on the fact table(s) is performed during BW Cube Compression. SQL Server 2016 can use up to 16 CPU threads for running Update Statistics with default sample rate (vs. one thread in SQL Server 2014). See also https://blogs.msdn.microsoft.com/sqlserverstorageengine/2016/05/23/query-optimizer-additions-in-sql-server.

Writable Nonclustered Columnstore Index (NCCI)

In SQL Server 2016, you can create an additional, writable columnstore index on top of a table, which already has a primary key and other rowstore indexes. This feature is planned to be used in future SAP BW releases for implementing the columnstore on other BW objects besides the fact tables.
NCCIs are also useful for SAP ERP. SAP will not deliver NCCIs in standard ERP systems. However, customers can create a NCCI for some tables of their ERP system on their own. A detailed description and recommendations for such a project will be published in a BLOG next year.

This list of SQL Server 2016 features is by far not complete. For Example, we did not mention additional indexes on top of a Clustered Columnstore Index (CCI) or the new SQL Server Query Store. The intention of this blog was to give an overview of the features, which justify a database upgrade to SQL Server 2016 for SAP BW.

Based on customer feedback over the course of the last 12 months we implemented several improvements in SAP BW. The code changes improve SAP platform migrations of very large SAP BW instances from other DBMS to SQL Server significantly. Furthermore, they simplify and accelerate SAP BW process chains for very large BW systems. The most important improvements are related to columnstore rowgroup compression, which is described below. Other recent improvements will be discussed in a separate BLOG.

Customers, who use the columnstore on Microsoft SQL Server 2014 (or newer), can now benefit from performance improvements and additional functionality. The new code is delivered in upcoming SAP BW Support Packages (SPs) and as a correction instruction in SAP Note 2329540 – Rowgroup Compression Framework for SAP BW / SQL Server. For implementing this SAP note, you need a minimum SAP BW SP. Therefore, we increased the minimum, recommended SP and updated SAP Note SAP Note 2114876 – Release Planning SAP BW for SQL Server Columnstore.

BW Rowgroup Compression

A SQL Server columnstore is organized in rowgroups, having up-to one million rows per rowgroup (see https://blogs.msdn.microsoft.com/saponsqlserver/2015/03/24/concepts-of-sql-server-2014-columnstore for details). For best query performance, you should trigger a columnstore rowgroup compression at the end of all BW Process Chains, which load data into a cube. You can either use a BW Cube Compression or the BW Process Chain type “Index Repair” for this purpose. However, some customers had an issue with this.

1. When converting all cubes to columnstore, you had to add “Index Repair” to all existing process chains. Customers often wanted to avoid the pain of changing each and every process chain.
2. In some BW systems we migrated we were faced with hundreds of thousands of rowgroups on SQL Server. Since a single rowgroup can contain up-to one million rows, these rowgroups could theoretically cover some hundred billion rows. It turned out soon, that queries against columnstore related SQL Server system tables (sys.column_store_row_groups) did not scale too well. Therefore, reading the metadata from SQL Server was even slower than running a rowgroup compression. This resulted in an unexpected long runtime of the BW Process Chain type “Index Repair”. This issue does not occur on smaller systems with a reasonable number of DB partitions and rowgroups.

Simplified Process Chains

With the new Rowgroup Compression Framework of SAP Note 2329540, you can trigger a rowgroup compression also with the BW Process Chain type “Update Statistics”. Update Statistics is typically already included in a Process Chain, even when the Process Chain was originally created for a different database platform. A typical process chain for loading data into a cube looks then like this:

The new functionality is explicitly coded for BW Process Chains only. When running a manual Update Statistics (button “Refresh Statistics”) in SAP transaction RSA1, the columnstore rowgroups are not compressed. If you want to perform a manual rowgroup compression in SAP transaction RSA1, you still have to choose the button “Repair DB Indexes”

Faster Process Chains

It was not possible to change the SQL Server system table sys.column_store_row_groups (in SQL Server 2014 and 2016). However, we achieve now very good query performance on columnstore metadata by using a different SQL Server system table (sys.system_internals_partitions) instead. For implementing this we had to change huge parts of the BW code in several areas (BW index repair, BW index check, BW update statistics, BW cube compression…)

Reading the metadata using the new method is several thousand times faster on very large BW systems. This results in much faster process chains, which contain the process chain types “Index Repair” or “Cube Compression”. The only thing you have to do, is applying SAP Note 2329540. There is no need for configuring anything or for setting any RSADMIN parameter.

However, once SAP Note 2329540 is applied, you have various configuration options regarding columnstore rowgroup compression in SAP BW. The options are described in detail in the SAP Note. The major reason, why we implemented these options, was to be prepared for future changes in SAP BW and SQL Server. Furthermore, the new RSADMIN parameter standardizes already existing (undocumented) parameters. Without setting any RSADMIN parameter, you automatically get the preferred default behavior (which also depends on the SQL Server version).

Lately we were involved in a case where a customer moved a SAP ERP system from a 2-Tier configuration to a 3-Tier configuration in virtualized environments. This was done because the customer virtualized the system and could not provide VMs large enough on the new host hardware to continue with a 2-Tier SAP setup. Hence the ASCS and only dialog instance got moved into one VM. Whereas the DBMS server was in another VM hosted in the same private cloud. Testing the new configuration afterwards was positive in terms of functionality. However, performance wise some of the batch processes were running factors slower. Not a few percent points, but really factors apart from what those batch jobs took on run time in the 2-Tier configuration. So investigations went into several directions. Did something change on the DBMS settings? No, not really. Got a bit more memory, but no other parameters changed. ASCS and primary application server also configured the same way. Nothing really changed in configuration. Since the tests did run a single batch job, issues around network scalability could be excluded as well.

So next step was looking into SAP single record statistics (transaction STAD). It really showed rather high database times for the selects one of the slowest running batch jobs issued against the database. Means it looked like a lot of the run time was accumulated when trying to select data from the SQL Server DBMS. On the other side there was no reason why the DBMS VM should run slow. We talked about on single batch job that was running against the system when the tests ran. At the end we found out what the issue was. Correct, it had to do with networking latency that was added by moving from a 2-Tier configuration to a 3-Tier configuration. Got us thinking a bit about documenting and investigating a bit on this. Below you can read some hopefully interesting facts.

You got all the tools in SAP NetWeaver to figure it out

What does that mean? With the functionalities built into SAP NetWeaver you can measure what the DBMS system measures on response time for a query and on the other side the response time the application instance experiences. At least for SQL Server. The two functionalities we are talking about in specific are:

• DBACockpit – SQL Statements
• ST05 – Performance Trace

As implemented in DBACockpit for SQL Server, the data shown here:

Is taken directly from the SQL Server instance. Or to be more precise from a DMV called sys.dm_exec_query_stats. It measures a lot of data about every query execution and stores it I the DMV. However, within the SQL Server instance. So, the time starts ticking when the query enters the SQL Server instance and stops when the results are available to be fetched from the SQL Server client used by the application (like ODBC, JDBC, etc). Hence the elapsed time is recorded and measured purely within SQL Server usually without taking network latency into consideration.

On the other side, the SAP ST05 Performance Trace:

is measuring query execution times at a point where the query leaves the application server instance and the result set returns. Means in this case, the time spent in network communication between the SAP application instance and DBMS server is fully included in the measurement. And it is the time the SAP application instance experiences. In case of our batch job the time that can define the execution time majorly. Especially for batch jobs that spend larger quantities of time in interaction with the DBMS system.

Let’s measure a bit

In order to measure a bit and demonstrate the effects, we went a bit extreme by creating a job that spent nearly 100% of its time inter acting with the DBMS. And then we started to measure by performing the following steps:

• We flushed the SQL Server procedure cache with the T-SQL command dbcc freeproccache. Be aware this command is also dropping the content of the SQL Server DMV sys.dm_exec_query_stats.
• We then started ST05 with the appropriate filters to trace the execution of our little job
• We then started our little job. The run time was just around 10 seconds

Our little job read around 9000 Primary keys of the SAP ERP material master table MARA into an internal table. As next step, it looped over the internal table and selected row by row fully defining the primary key. However only the first 3 columns of every row got read.

Our first setup was to test with VMs that were running on rather old server hardware. Something around 6 or 7 year old processor generation. But the goal of this exercise is not really to figure out the execution times as such, but the ratio, we are getting between what we measure on the DBMS side and the SAP side in the different configurations.

Hence in a first test using our older hosts with VMs we saw this after a first run:

So, we see 89 microseconds average execution time per query on the SQL Server side. Well, the data as expected, was in in memory. The query is super simple since it accesses a row by the primary key. Just reading the first 3 columns of a relative wide table. So yes, very well doable.

As we are going to measure the timing of running this report on the SAP application side, we are going to introduce the notion of an ‘overhead factor’. We define:

‘Overhead factor’ = The factor of time that is spent on top of the pure SQL Server execution time for network transfer of the data and processing in the ODBC client until the time measurement in the SAP logic is reached. So, if it takes x microseconds elapsed time in SQL Server to execute the query and we measure 2 times x in ST05, the ‘overhead factor is 2.

As we are running in a 2-Tier configuration, expectation would be that we add a few microseconds into this network and processing time. Looking at the summarized ST05 statistics of our trace we were a bit surprised to see this:

We measure 362 microseconds in our 2-Tier configuration. Means compared to what SQL Server requires on time to execute our super simple query, the response time as measured from the SAP side is a rough 4 times larger. Or in other words our ‘overhead factor’ is a factor of 4.

The interesting question is now going to be how the picture is going to look in case of a 3-Tier system. The first test we are going to do is involving a DBMS VM and an SAP application server VM that are running on the same host server.

In this case the measurements in ST05 looked like:

With 544 microseconds in average. Means the ‘overhead factor’ accumulated to a factor of around 6.

In the second step let’s consider a scenario where the VMs are not hosted on the same server. The servers of the same type and same processor that we used for the next test were actually sitting in two different racks. Means we had at least two or three switches to have the traffic flowing between the VMs. This very fact was reflected in the ST05 trace immediately where we looked at this result:

Means our ‘overhead factor’ basically increased to a bit over 8.

So, let’s summarize the results in this table:

Thus, we can summarize that the move from 2-Tier to 3-Tier can add significant overhead time, especially if more and more network components get involved. For SAP jobs this can mean that the run time can increase by factors in extreme cases as we tested them here. The more resource consuming a SAP logic is on the application layer, the less the impact of the network will be.

What can you do, especially in case of virtualization?

• Try to leverage hardware for your private cloud that optimizes network traffic through architectures like SR-IOV (https://msdn.microsoft.com/en-us/windows/hardware/drivers/network/overview-of-single-root-i-o-virtualization–sr-iov- ). SR-IOV is also supported by VMWare in more recent releases. Please check individual documentation of your vSphere version.
• Especially for larger VMs of 16vCPUs or above use vRSS with the NIC drivers (https://technet.microsoft.com/en-us/library/dn383582(v=ws.11).aspx and http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/whitepaper/vmware-vsphere-pnics-performance-white-paper.pdf )

That assumes that the design principles from an infrastructure point of view are to have the compute infrastructure of the SAP DBMS layer and the application layer as close as possible together. ‘Together’ in the sense of distance, but also in the sense of having the least amount of network switches/routers and gateways in the network path that ideally is not miles in length.

Testing on Azure

The tests in Azure were conducted exactly with the same method and same report.

Tests within one Azure VNet

For the first test on Azure, we created a VNet and deployed our SAP system into it. Since the intent as well was to test a new network functionality of Azure, we used the DS15v2 VM type for all our exercises. The new functionality we wanted to test and demonstrate is called Azure Accelerated Networking and is documented here: https://azure.microsoft.com/en-us/documentation/articles/virtual-network-accelerated-networking-portal/

Executing our little report, in a 2-Tier configuration we measured:

60 microseconds to execute a single query in SQL Server. Looking at the ST05 values, we are experiencing 177 microseconds:

This is a factor of around 3.5 times between the pure time it takes on SQL Server and the time measured on the SAP instance that ran on the same VM as SQL Server.

For the first 3-Tier test, we deployed the two VMs, the VM running the database instance and the VM running the SAP application instance without the ‘Azure network acceleration’ functionality deployed. As mentioned the VMs are in the same Azure VNet.

As we perform the tests, the results measured in ST05 on the dedicated SAP instance look like:

Means we are looking into 570 microseconds which is nearly 3 times the time it takes with a 3-Tier configuration and around a factor of 9.5 times more time than SQL Server did take to work on a single query execution.

As a next step, we configured the new ‘Azure network acceleration’ functionality in both VMs. The database VM and the VM running the SAP instance were using the acceleration. Repeating the measurements, we were quite positively surprised experiencing a result like this:

We basically are down to 340ms per execution of the query measured on the SAP application instance. A dramatic reduction of the impact the network stack introduced in virtualized environments.

The factor of overhead compared to the pure time spent on the execution on SQL Server is reduced to a factor of under 6. Or you cut out a good 40% of communication time.

Additionally, we performed some other tests in 3-Tier SAP configurations in Azure where:

• We put the SAP instance VM and DBMS VM into an Azure Availability Set
• Where we deployed the SAP instance VM into a VM type that forced that VM onto another hardware cluster (SAP instance VM = GS4 and DBMS VM = DS15v2). See also this blog on Azure VM Size families: https://azure.microsoft.com/en-us/blog/resize-virtual-machines/

In all those deployments, without ‘Azure network optimization’ functionality, we did not observe any significant change from ST05 measurements. In all the cases the values the single execution values measured with ST05 ended between 530 and 560ms.

So in essence, we showed for the Azure side:

• Azure compared to on-premise virtualization does not add anymore overhead in communications.
• The new accelerated networking feature has a significant impact on the time spent in communications and data transport over the network.
• There is no significant impact having two Azure VMs communicating that run on different hardware clusters.

What are we missing? Yes, we are missing pure bare-metal to bare-metal measurements. The reason we left that scenario out is because most of our customers have at least one side of the 3-Tier architecture virtualized or even both sides. Take e.g the SAP landscape in Microsoft. Except the DBMS side of the SAP ERP system, all other components are virtualized. Means the Microsoft SAP landscape has a degree of 99% virtualization with a good portion of the virtualized systems already running on Azure. Since the tendency by customers is to virtualize their SAP landscapes more and more, we did not see any need to perform these measurements on bare-metal systems anymore.

1. Customers are increasingly moving large SAP systems in the 5-10TB range to Azure.  This press release is an example of one of many successful projects
2. With powerful new VMs, Accelerated Networking and Premium Storage it is now possible to import even huge size SAP databases on Azure
3. Based on experience from recent migrations we recommend using SQL Server 2014 SP2 CU2 or SQL 2016 SP1 (as at November 2016).  It is generally recommended to use the latest Service Pack and Cumulative update
4. A future blog will cover specific recommendations for BW migrations.  Some changes have been made to improve import performance on BW fact tables and other general recommendations.  These will appear in a blog shortly
5. Intel has released both the E5v4 and E7v4 processors.  2 socket commodity Intel servers now benchmark~120,000 SAPS and 4 socket Intel servers now benchmark ~220,000 SAPS
6. Customers still running on UNIX platforms are terminating these platforms and moving to more cost effective Intel systems with much better performance
7. Multiple customers have terminated the use of SAP BW Accelerator and replaced this with SQL Server’s built in Column Store.  The OS/DB Migration FAQ now includes a link to the procedure to remove BWA configuration from a BW system

SAP program MSSCOMPRESS is a tool for reducing disk space usage of SAP NetWeaver based systems. In addition, it is often used as a monitoring tool. MSSCOMPRESS has been released by SAP over 6 years ago. In the meanwhile, several improvements have been implemented. The latest version of MSSCOMPRESS is available as an attachment of SAP Note 1488135.

Overview

The basic features of MSSCOMPRESS are described in https://blogs.msdn.microsoft.com/saponsqlserver/2010/10/08/compressing-an-sap-database-using-report-msscompress. In this blog we want to describe the new features and functionalities. The user interface of MSSCOMPRESS has changed a little bit:

When starting MSSCOMPRESS or pressing the button “Refresh”, the following information is retrieved from SQL Server for all tables of the SAP database:

• Total table size
• Data and index size
• DB compression type of data and indexes
• Number of rows, indexes and partitions
• A checkbox, which flags those tables which use a HEAP
• Number of uncompressed rows in columnstore rowgroups
  (be aware, that this number may not be accurate for a few minutes after compressing the columnstore rowgroups of this particular table)

The SQL query for retrieving this information has been tuned several times in the past.

New filter options

Traditionally, you can filter the list of tables in MSSCOMPRESS using the Filter Options. The Name Filter now allows the wildcards star and question mark. If you want to see all tables of a particular SAP BW cube, you can use the filter /*/?CUBE* (see the example above). The list now includes an automatic sum of some columns. In this example, the sum of the table sizes is the total cube size.

In addition, you can now use SAP ALV (ABAP List Viewer) Filters. By right-clicking in the ALV list, you can set ALV Filters or sort the ALV list. This feature is particularly useful, when using MSSCOMPRESS as a monitoring tool. To avoid confusion, compressing “Filtered Tables” is disabled, once you use the traditional Filter Options and the new ALV Filters at the same time. However, you can still compress using “Selected Tables”.

DB compression type COLUMNSTORE_ARCHIVE

For B-trees (rowstore) you can choose the DB compression type (NONE, ROW or PAGE) separately for data (clustered index or heap) and indexes. For columnstore, you can now choose between DB compression types COLUMNSTORE and COLUMNSTORE_ARCHIVE. This feature has been added to support the full SQL Server functionality in SAP. However, we strongly recommend keeping the SAP default DB compression types: COLUMNSTORE for columnstore indexes and PAGE for rowstore. MSSCOMPRESS does not touch existing indexes by default, if the DB compression type is already correct. By choosing one of the “Force Data/Index/CS Rebuild” check boxes, an index rebuild is always executed, even when not changing the DB compression type.

Compress Rowgroups

This is a new feature in MSSCOMPESS, which actually has nothing to do with the DB compression type. It performs a rowgroup compression of columnstore indexes. For SAP BW, this tasks should be performed in SAP BW process chains, see https://blogs.msdn.microsoft.com/saponsqlserver/2016/11/14/simplified-and-faster-sap-bw-process-chains. There is no need to use MSSCOMPRESS for compressing columnstore rowgroups in SAP BW. However, one may want to create an additional Nonclustered Columnstore Index (NCCI) in SAP ERP on SQL Server 2016. A detailed description regarding this will be published in a BLOG next year. In such a scenario, you can use MSSCOMPRESS to schedule a rowgroup compression job each night.

By choosing “Compress Rowgroups” the mode of MSSCOMPRESS changes from DB compression (which performs an index REBULD) to rowgroup compression (which performs a columnstore index REORGANIZE). Therefore, all DB compression type options are greyed out. The Online and MAXDOP options are also greyed out, because a rowgroup compression is always performed online and single-threaded. MSSCOMPRESS only performs a rowgroup compression, if there are uncompressed rows in a columnstore index of the chosen table. By choosing “Force Reorganize”, a rowgroup compression will always be performed. This is useful as of SQL Server 2016, since the rowgroup compression also merges small rowgroups into one larger rowgroup. See https://blogs.msdn.microsoft.com/saponsqlserver/2016/11/11/sql-server-2016-improvements-for-sap-bw.

Summary

The newest version of MSSCOMPRESS is faster and provides additional filter options. SAP consultants “misuse” MSSCOMPRESS as a monitoring tool, which enables a quick look on all database tables of an SAP NetWeaver system.

Below is a list of updates, recommendations and new technologies for customers moving SAP applications onto Azure.

At the end of the blog is a checklist we recommend all customers follow with planning to deploy SAP environments on Azure.

1. High Performance & UltraPerformance Gateway

The UltraPerformance Gateway has been released.

This Gateway supports only Express Route Links and has a much higher maximum throughput.

The UltraPerformance Gateway is very useful on projects where a large amount of R3load dump files or database backups need to be uploaded to Azure.

https://azure.microsoft.com/en-us/documentation/services/vpn-gateway/

2. Accelerated Networking

A new feature to drastically increase the bandwidth between DS15v2 VMs running on the same Vnet has been released. The latency between VMs with Accelerated Networking is greatly reduced.

This feature is available in most data centers already and is based on the concept of SRIO-V.

Accelerated networking is particularly useful when running SAP Upgrades or R3load migrations. Both the DB server and the SAP Application servers should be configured for Accelerated networking.

Large 3 tier systems with many SAP Application servers will also benefit from Accelerated networking on the Database server however.

For example a highly scalable configuration would be a DS15v2 database server running SQL Server 2016 with Buffer Pool Extension enabled and Accelerated Networking and 6 D13v2 application servers:

Database Server: DS15v2 database server running SQL Server 2016 SP1 with Buffer Pool Extension enabled and Accelerated Networking. 110GB of memory for SQL Server cache (SQL Max Memory) and another ~200GB of Buffer Pool Extension

Application Server: 6 * D13v2 each with two SAP instances with 50 work processes and PHYS_MEMSIZE set to 50%. A total of 600 work processes (6 * D13v2 VMs * 50 work process per instance * 2 instances per VM = 600)

The SAPS value for such a 3 tier configuration is around 100,000 SAPS = 30,000 SAPS for DB layer (DS15v2) and 70,000 SAPS for app layer (6 x D13v2)

3. Multiple ASCS or SQL Server Availability Groups on a Single Internal Load Balancer

Prior to the release of multiple frontend IP addresses for an ILB each SAP ASCS required a dedicated 2 node cluster.

Example: a customer with SAP ECC, BW, SCM, EP, PI, SolMan, GRC and NWDI would need 8 separate 2 node clusters = total of 16 small VMs for the SAP ASCS layer.

With the release of the multiple ILB frontend IP address feature only 2 small VMs are now required.

A single Internal Load Balancer can now bind multiple frontend IP addresses. These frontend IP addresses can be listening on different ports such as the unique port assigned to each AlwaysOn Availability Group listener or the same port such as 445 used for Windows File Shares.

A script with the PowerShell commands to set the ILB configuration is available here

Note: It is now possible to assign a Frontend IP address to the ILB for the Windows Cluster Internal Cluster IP (this is the IP used by the cluster itself). Assigning the IP address of the Cluster to the ILB allows the cluster admin tool and other utilities to run remotely.

Up to 30 Frontend IP addresses can be allocated to a single ILB. The default Service Limit in Azure is 5. A support request can be created to get this limit increased.

The following PowerShell commands are used

New-AzureRmLoadBalancer

Add-AzureRmLoadBalancerFrontendIpConfig

Add-AzureRmLoadBalancerProbeConfig

Add-AzureRmLoadBalancerBackendAddressPoolConfig

Set-AzureRmNetworkInterface

Add-AzureRmLoadBalancerRuleConfig

4. Encrypted Storage Accounts, Azure Key Vault for SQL TDE Keys & Advanced Disk Encryption (ADE)

SQL Server supports Transparent Database Encryption (TDE). SQL Server keys can be stored securely inside the Azure Key Vault. SQL Server 2014 and earlier can retrieve keys from the Azure Key Vault with a free Connector utility. SQL Server 2016 onwards natively supports the Azure Key Vault. It is generally recommended to Encrypt a database before loading data with R3load as the overhead involved is only ~5%. Applying TDE after an import is possible, but this will take a lot of time on large databases. The recommended cipher is AES-256. Backups are encrypted on TDE systems.

Advanced Disk Encryption is a technology like “Bitlocker”. It is preferable not to use ADE on disks holding DBMS datafiles, temp files or log files and to secure SQL Server (or other DBMS) datafiles at rest with TDE (or the native DBMS encryption tool).

It is strongly recommended not to use both SQL Server TDE and ADE disks in combination. This may create a large overhead and is a scenario that has not been tested. ADE is useful for encrypting the OS Boot Disk

The Azure Platform now supports Encrypted Storage Accounts. This feature encrypts at-rest data on a storage account

5. Windows Server 2016 Cloud Witness

Windows Server 2016 will be generally available for SAP customers in Q1 2017 based on current planning.

One very useful feature is the Cloud Witness.

A general recommendation for Windows Clusters on Azure is:

1. Use Node & File Share Witness Majority with Dynamic Quorum

2. The File Share Witness should be in a third location (not in the same location as either primary or DR)

3. There should be independent redundant network links between all three locations (primary, DR, File Share Witness)

4. Systems that have a very high SLA may require the File Share Witness share to be highly available (thus requiring another cluster)

Until Windows 2016 there were several problems with this approach:

1. Many customers compromised the DR solution by placing the FSW in the primary site

2. Often the FSW was not Highly Available

3. The FSW required at least one additional VM leading to increased costs

4. If the FSW was Highly Available this required 2 VMs and software shared disk solutions like SIOS. This increases costs

Windows Server 2016 resolves this problem with the Cloud Witness:

1. The FSW is now a Platform as a Service (PaaS) role

2. No VM is required

3. The FSW is now automatically highly available

4. There is no ongoing maintenance, patching, HA or other activities

5. After the Cloud Witness is setup it is a Managed Service

6. The Cloud Witness can be in any Azure datacenter (a third location)

7. The Cloud Witness can be reached over standard internet links (avoiding the requirement for redundant independent links to the FSW)

6. Azure Site Recovery Azure-2-Azure (ASR A2A)

Azure Site Recovery is an existing technology that replicates VMs from a source to a target in Azure.

A new scenario will be released in Q1. The new scenario is Azure to Azure ASR.

Scenarios supporting replicating Hyper-V, Physical or VMWare to Azure are already Generally Available.

The key differentiators between Azure Site Recovery (ASR) and competing technologies:

Azure Site Recovery substantially lowers the cost of DR solutions. Virtual Machines are not charged for unless there is an actual DR event (such as fire, flood, power loss or test failover). No Azure compute cost is charged for VMs that are synchronizing to Azure. Only the storage cost is charged

Azure Site Recovery allows customers to perform non-disruptive DR Tests. ASR Test Failovers copy all the ASR resources to a test region and start up all the protected infrastructure in a private test network. This eliminates any issues with duplicate Windows computernames. Another important capability is the fact that Test Failovers do not stop, impair or disrupt VM replication from on-premise to Azure. A test failover takes a “snapshot” of all the VMs and other objects at a particular point in time

The resiliency and redundancy built into Azure far exceeds what most customers and hosters are able to provide. Azure blob storage stores at least 3 independent copies of data thereby eliminating the chances of data loss even in event of a failure on a single storage node

ASR “Recovery Plans” allow customers to create sequenced DR failover / failback procedures or runbooks. For example, a customer might create a ASR Recovery Plan that first starts up Active Directory servers (to provide authentication and DNS services), then execute a PowerShell script to perform a recovery on DB servers, then start up SAP Central Services and finally start SAP application servers. This allows “Push Button” DR

Azure Site Recovery is a heterogeneous solution and works with Windows and Linux and works well with SQL Server, Oracle, Sybase and DB2.

Additional Information:

1. To setup an ASCS on more than 2 nodes review SAP Note 1634991 – How to install an ASCS or SCS instance on more than 2 cluster nodes

2. SAP Application Servers in the DR site are not running. No compute costs incurred

3. Costs can be reduced further by decreasing the size of the DR SQL Servers. Use a smaller cheaper VM sku and upgrade to a larger VM if a DR event occurs

4. Services such as Active Directory must be available in DR

5. SIOS or BNW AFSDrive can be used to create a shared disk on Azure. SAP requires a shared disk for the ASCS as at November 2016

6. Costs can be reduced by removing HA from DR site (only 1 DB and ASCS node)

7. Careful planning around cluster quorum models / voting should be done. Explain cluster model to operations teams. Use Dynamic Quorum

Diagram showing DR with HA

7. Pinning Storage

Azure storage accounts can be “affinitized” or “pinned” to specific storage stamps. A stamp can be considered a separate storage device.

It is strongly recommended to use separate storage account for the DBMS files for each database replica.

In the example below the following configuration is deployed:

1. SQL Server AlwaysOn Node 1 uses Storage Account 1

2. SQL Server AlwaysOn Node 2 uses Storage Account 2

3. A support request was opened and Storage Account 1 was pinned to stamp 1, Storage Account 2 was pinned to stamp 2

4. In this configuration the failure of the underlying storage infrastructure will not lead to an outage.

8. Increase Windows Cluster Timeout Parameters

When running Windows Cluster on Azure VMs it is recommended to apply a hotfix to Windows 2012 R2 to increase the Cluster Timeout values to the defaults that are set on Windows 2016.

To increase the Windows 2012 R2 cluster timeouts to those defaulted in Windows 2016 please apply this KB https://support.microsoft.com/en-us/kb/3153887

No action is required on Windows 2016. The values are already correct

More information can be found in this blog

9. Use ARM Deployments, Use Dv2 VMs, Single VM SLA and Use Premium Storage for DBMS only

Most of the new enhanced features discussed in this blog are ARM only features. These features are not available on old ASM deployments. It is therefore strongly recommended to only deploy ARM based systems and to migrate ASM systems to ARM.

Azure D-Series v2 VM types have fast powerful Haswell processors that are significantly faster than the original D-Series.

All customers should use
Premium Storage for the Production DBMS servers and for non-production systems.

Premium Storage
should also be used for Content Server, TREX, LiveCache, Business Objects and other IO intensive non-Netweaver file based or DBMS based applications

Premium Storage is of no benefit on SAP application servers

Standard Storage can be used for database backups or storing archive files or interface files

More information can be found in SAP Note 2367194 – Use of Azure Premium SSD Storage for SAP DBMS Instance

Azure now offers a financially backed SLA for single VMs. Previously a SLA was only offered for VMs in an availability set. Improvements in online patching and reliability technologies allow Microsoft to offer this feature.

10. Sizing Solutions for Azure – Don’t Just Map Current VM CPU & RAM Sizing

There are a few important factors to consider when developing the sizing solution for SAP on Azure:

1. Unlike on-premises deployments there is no requirement to provide a large sizing buffer for expected growth or changed requirements over the lifetime of the hardware. For example when purchasing new hardware for an on-premises system it is normal to purchase sufficient resources to allow the hardware to last 3-4 years. On Azure this is not required. If additional CPU, RAM or Storage is required after 6 months, this can be immediately provisioned

2. Unlike most on-premises deployment on Intel servers Azure VMs do not use Hyper-Threading as at November 2016. This means that the thread performance on Azure VMs is significantly higher than most on-premises deployments. D-Series v2 have more than 1,500 SAPS/thread

3. If the current on-premises SAP application server is running on 8 CPU and 56GB of RAM, this does not automatically mean a D13v2 is required. Instead it is recommended to:

a. Measure the CPU, RAM, network and disk utilization

b. Identify the CPU generation on-premises – Azure infrastructure is renewed and refreshed more frequently than most customer deployments.

c. Factor in the CPU generation and the average resource utilization. Try to use a smaller VM

4. If switching from 2-tier to 3-tier configurations it is recommended to review this blog

5. Review this blog on SAP on Azure Sizing

6. After go live monitor the DB and APP servers and determine if they need to be increased or decreased in size

11. Fully Read & Review the SAP on Azure Deployment Guides

Before starting a project all technical members of the project team should fully review the SAP on Azure Deployment Guides

These guides contain the recommended deployment patterns and other important information

Ensure the Azure VM monitoring agents for ST06 as documented in Note 2015553 – SAP on Microsoft Azure Support prerequisites

SAP systems are not supported on Azure until this SAP Note is fully implemented

12. Upload R3load Dump Files with AzCopy, RoboCopy or FTP

The diagram below shows the recommended topology for exporting a system from an existing datacenter and importing on Azure.

SAP Migration Monitor includes built in functionality to transfer dump files with FTP.

Some customers and partners have developed their own scripts to copy the dump files with Robocopy.

AzCopy can be used and this tool does not need a VPN or ExpressRoute to be setup as AzCopy runs directly to the storage account.

13. Use the Latest Windows Image & SQL Server Service Pack + Cumulative Update

The latest Windows Server image includes all important updates and patches. It is recommended to use the latest available Windows Server OS available in the Azure Gallery

The latest DBMS versions and patches are recommended.

We do not generally recommend deploying SQL Server 2008 R2 or earlier for any SAP system. SQL Server 2012 should only be used for systems that cannot be patched to support more recent SQL Server releases.

SQL Server 2014 has been supported by SAP for some time and is in widespread deployment amongst SAP customers already both on-premises and on Azure

SQL Server 2016 is supported by SAP for SAP_BASIS 7.31 and higher releases and has already been successfully deployed in Production at several large customers including a major global energy company. Support for SQL 2016 for Basis 7.00 to 7.30 is due soon.

The latest SQL Server Service Packs and Cumulative updates can be downloaded from here.

Due to a change in incremental servicing policies the very latest SQL Server CU will only be available for download.

Previous Cumulative Updates can be downloaded from here

1966681 – Release planning for Microsoft SQL Server 2014

2201059 – Release planning for Microsoft SQL Server 2016

The Azure platform fully supports Windows, Suse 12 or higher and RHEL 7 or higher. Oracle, DB2, Sybase, MaxDB and Hana are all supported on Azure.

Many customers utilize the move from on-premises to Cloud to switch to a single support vendor and switch to Windows and SQL Server

Microsoft has released a  Database Trade In Program
that allows customers to trade in DB2, Oracle or other DBMS and obtain SQL Server licenses free of charge (conditions apply).

The Magic Quadrant for Operational Database Management Systems places SQL Server in the lead in 2015. This lead was further extended in the 2016 Magic Quadrant

14. Migration to Azure Pre-Flight Checklist

Below is a recommended Checklist for customers and partners to follow when migrating SAP applications to Azure.

1. Survey and Inventory the current SAP landscape. Identify the SAP Support Pack levels and determine if patching is required to support the target DBMS. In general the Operating Systems Compatibility is determined by the SAP Kernel and the DBMS Compatibility is determined by the SAP_BASIS patch level.

Build a list of SAP OSS Notes that need to be applied in the source system such as updates for SMIGR_CREATE_DDL. Consider upgrading the SAP Kernels in the source systems to avoid a large change during the migration to Azure (eg. If a system is running an old 7.41 kernel, update to the latest 7.45 on the source system to avoid a large change during the migration)

2. Develop the High Availability and Disaster Recovery solution. Build a PowerPoint that details the HA/DR concept. The diagram should break up the solution into the DB layer, ASCS layer and SAP application server layer. Separate solutions might be required for standalone solutions such as TREX or Livecache

3. Develop a Sizing & Configuration document that details the Azure VM types and storage configuration. How many Premium Disks, how many datafiles, how are datafiles distributed across disks, usage of storage spaces, NTFS Format size = 64kb. Also document Backup/Restore and DBMS configuration such as memory settings, Max Degree of Parallelism and traceflags

4. Network design document including VNet, Subnet, NSG and UDR configuration

5. Security and Hardening concept. Remove Internet Explorer, create a Active Directory Container for SAP Service Accounts and Servers and apply a Firewall Policy blocking all but a limited number of required ports

6. Create an OS/DB Migration Design document detailing the Package & Table splitting concept, number of R3loads, SQL Server traceflags, Sorted/Unsorted, Oracle RowID setting, SMIGR_CREATE_DDL settings, Perfmon counters (such as BCP Rows/sec & BCP throughput kb/sec, CPU, memory), RSS settings, Accelerated Networking settings, Log File configuration, BPE settings, TDE configuration

7. Create a “Flight Plan” graph showing progress of the R3load export/import on each test cycle. This allows the migration consultant to validate if tunings and changes improve r3load export or import performance. X axis = number of packages complete. Y axis = hours. This flight plan is also critical during the production migration so that the planned progress can be compared against the actual progress and any problem identified early.

8. Create performance testing plan. Identify the top ~20 online reports, batch jobs and interfaces. Document the input parameters (such as date range, sales office, plant, company code etc) and runtimes on the original source system. Compare to the runtime on Azure. If there are performance differences run SAT, ST05 and other SAP tools to identify inefficient statements

9. SAP BW on SQL Server. Check this blogsite regularly for new features for BW systems including Column Store

10. Audit deployment and configuration, ensure cluster timeouts, kernels, network settings, NTFS format size are all consistent with the design documents. Set perfmon counters on important servers to record basic health parameters every 90 seconds. Audit that the SAP Servers are in a separate AD Container and that the container has a Policy applied to it with Firewall configuration.

Content from third party websites, SAP and other sources reproduced in accordance with Fair Use criticism, comment, news reporting, teaching, scholarship, and research

On 4/10/2016, SAP released support for SQL Server 2016 for some SAP NetWeaver based systems. In this blog we will talk about how you can update your existing installations on SQL Server to SQL Server 2016, what you need to do when installing a new SAP NetWeaver based system on SQL Server 2016 and what to consider when upgrading an SAP NetWeaver based system which is already running on SQL Server 2016. We will also list all supported products and the required Support Package Stacks.

On 9/12/2016, SAP updated SAP Note 2201059 (Login required) and the Product Availability Matrix (PAM) to include most SAP products based on SAP NetWeaver 7.0 ABAP and higher. See below for an updated list.

SQL Server RDBMS DVD

Important: Please read SAP Note 2139358 (Login required) and SAP Note 2313067 (Login required) on more information about how and where to download the SQL Server 2016 RDBMS DVD.

Installation of SAP NetWeaver

The following SAP NetWeaver based products are currently supported on SQL Server 2016:

• SAP products based on SAP NetWeaver Java for releases SAP NetWeaver 7.0 or higher
• Update: SAP products based on SAP NetWeaver ABAP for releases SAP NetWeaver 7.0 and higher
• SAP NetWeaver PI for releases SAP NetWeaver 7.0 and higher
• SAP Solution Manager 7.2

The following SAP NetWeaver based products are not supported on SQL Server 2016:

• SAP Solution Manager 7.0 EHP1 and earlier releases of SAP Solution Manager
• SAP CRM 5.0 and SAP CRM 6.0 (also known as SAP CRM 2007)
• SAP SCM 5.0 and SAP SCM 5.1 (also known as SAP SCM 2007)
• SAP SRM 5.0
• SAP NetWeaver Developer Workplace is not supported on SQL Server 2016.

Before you start with the installation, please read the following SAP Notes:

1. Release planning Note – SAP Note 2201059 (Login required)
2. Setting up Note – SAP Note 2201060 (Login required)
3. Required SQL Server patches – SAP Note 62988 (Login required)
4. SWPM Note and required Kernel DVD – SAP Note 1680045 (Login required)
5. Central Note for SL Toolset – SAP Note 1563579 (Login required)

Requirements

1. Windows Server 2012 64-bit or higher
2. SQL Server 2016 Enterprise Edition

Preparation

• SAP Software Provisioning Manager
  • See SAP Note SAP Note 1680045 (Login required) on where to download the SWPM. Please make sure to always download the latest SWPM and the SWPM that matches the SAP NetWeaver version that you want to install (e.g. for SAP NetWeaver 7.0 based products you need to download 70SWPM)
• Kernel DVD
  • See SAP Note SAP Note 1680045 (Login required) on where to download the latest Kernel DVD.
  • For SAP products based on SAP NetWeaver 7.4 and higher, use the 7.45 DCK Kernel
  • For SAP products based on EHP1 for SAP NetWeaver 7.3 and lower use the 721_EXT Kernel DVD (only the EXT Kernel is supported)
• Java DVD
  • See SAP Note SAP Note 1680045 (Login required) on where to download the Java DVD for the product you want to install.
• ODBC driver
  • If you want to install a distributed SAP System (database and SAP application Server are running on different hosts), make sure that the latest ODBC driver for Microsoft SQL Server is installed on the host running your SAP application Server. See SAP Note 1902794 (Login required)
  • The ODBC driver can also be downloaded from the Microsoft Download Center

Installation

• Install your SAP system using
  • The SAP Software Provisioning Manager SAR archive (SWPM.SAR or 70SWPM.SAR)
  • The SAP Software Provisioning Manager Kernel DVD
  • The Java DVD as described in SAP Note 1680045 (Login required)
  • The DVDs that were originally shipped with the SAP product (Export DVD, language DVDs…)
• After the Installation upgrade your kernel to the latest 7.21 EXT/7.22 EXT Kernel or 7.45 Kernel. 7.22 EXT Kernel is recommended for SAP System based on SAP NetWeaver 7.31 or lower.
• ABAP/ABAP+JAVA: Connect your new SAP System to your SAP Solution Manager (only required if you do not install SAP Solution Manager)
• ABAP/ABAP+JAVA: Create a maintenance stack to implement at least (only required if you do not install SAP Solution Manager) SAP Business Suite 7i 2011
• • Update: SAP Business Suite 2005
    • SAP ERP 6.0 SPS 28 or higher
  • Update: SAP Business Suite 7 Support Release 1
    • SAP CRM 7.0 SPS 18 or higher
    • EHP4 for SAP ERP 6.0 SPS 18 or higher
    • SAP SCM 7.0 SPS 18 or higher
    • SAP SRM 7.0 SPS 19 or higher
  • Update: SAP Business Suite 7i 2010
    • EHP1 for SAP CRM 7.0 SPS 15 or higher
    • EHP5 for SAP ERP 6.0 SPS 15 or higher
    • EHP1 for SAP SCM 7.0 SPS 15 or higher
    • EHP1 for SAP SRM 7.0 SPS 15 or higher
  • SAP Business Suite 7i 2011
    • EHP2 for SAP CRM 7.0 SPS 16 or higher
    • EHP6 for SAP ERP 6.0 SPS 16 or higher
    • EHP2 for SAP SCM 7.0 SPS 16 or higher
    • EHP2 for SAP SRM 7.0 SPS 16 or higher
  • SAP Business Suite 7i 2013
    • EHP3 for SAP CRM 7.0 SPS 10 or higher
    • EHP7 for SAP ERP 6.0 SPS 10 or higher
    • EHP3 for SAP SCM 7.0 SPS 10 or higher
    • EHP3 for SAP SRM 7.0 SPS 10 or higher
  • SAP Business Suite 7i 2016
    • EHP4 for SAP CRM 7.0 SPS 01 or higher
    • EHP8 for SAP ERP 6.0 SPS 01 or higher
    • EHP4 for SAP SCM 7.0 SPS 01 or higher
    • EHP4 for SAP SRM 7.0 SPS 01 or higher
  • Update: SAP NetWeaver 7.0 SPS 33 or higher
  • Update: SAP NetWeaver 7.0 including EHP1 SPS 18 or higher
  • Update: SAP NetWeaver 7.0 including EHP2 SPS 18 or higher
  • Update: SAP NetWeaver 7.0 including EHP3 SPS 17 or higher
  • Update: SAP NetWeaver 7.1 SPS 20 or higher
  • Update: SAP NetWeaver 7.1 including EHP1 SPS 15 or higher
  • Update: SAP NetWeaver 7.3 SPS 14 or higher
  • SAP NetWeaver 7.3 including EHP1 SPS 17 or higher
  • SAP NetWeaver 7.4 SPS 12 or higher
  • SAP NetWeaver 7.5 SPS 01 or higher
• ABAP/ABAP+JAVA: Use the Software Update Manager (SUM) 1.0 SP17 or higher to implement the maintenance stack (only required if you do not install SAP Solution Manager)
  • DO NOT use SPAM to implement the support packages
  • DO NOT update SPAM manually but let the SUM update the SPAM as part of the maintenance stack implementation
• SAP Solution Manager 7.2 only: Use the Software Update Manager (SUM) 1.0 SP17 to install SAP Solution Manager 7.2 SR1 which already contains the required SPS 01

Post Steps

Configure your SQL Server as described in SAP Note 2312935 (Login required)

System Copy of SAP NetWeaver

You can also copy your SAP System that is running on an older SQL Server release to a machine running a new SQL Server with the following steps:

1. Make sure that your source system is at least on the support package stack as described in the installation section if this blog.
2. Copy your SAP system using
   1. The SAP Software Provisioning Manager SAR archive (SWPM.SAR or 70SWPM.SAR)
   2. The SAP Software Provisioning Manager Kernel DVD
   3. The Java DVD as described in SAP Note 1680045 (Login required)
   4. The DVDs that were originally shipped with the SAP product (Export DVD, language DVDs…)
3. After the Installation upgrade your kernel to the latest 7.21 EXT/7.22 EXT Kernel or 7.45 Kernel. 7.22 EXT Kernel is recommended for SAP System based on SAP NetWeaver 7.31 or lower.

The steps for a System Copy where the source SAP system is already running on SQL 2016 are the same.

Update or Upgrade of SAP NetWeaver

Only updates or upgrades that are supported by the SAP Software Update Manager are supported for SQL Server 2016. Please read SAP Note SAP Note 1563579 (Login required) to find the SAP Note for the latest SAP Software Update Manager that describes the supported update and upgrade scenarios.

As an example, the following upgrades are not supported by SAP Software Manager and are therefore not supported for SQL Server 2016:

1. Upgrade of SAP CRM 5.0 to SAP CRM 7.0 or EHP1 for SAP CRM 7.0
2. Upgrade of SAP SCM 5.0 to SAP SCM 7.0 or EHP1 for SAP SCM 7.0
3. Upgrade of SAP SRM 5.0 to SAP SRM 7.0 or EHP1 for SAP SRM 7.0

On 13/12 SAP updated SAP Note 1928533 (login required) to include support for SAP systems on Windows on virtual machines that are created using the Azure Resource Manager. The documentation on docs.microsoft.com was updated to reflect the change.

You can now use virtual machines that are created using the Azure Resource Manager for both Windows and Linux virtual machines.

We also published new quick start templates on github for setting up an ASCS/SCS cluster that supports multiple ASCS/SCS instances (aka multi SID). Please have a look at the HA guide for more information on how to use them. The HA guide also contains information on how to modify an already existing ASCS/SCS cluster to support more ASCS/SCS instances.

With the latest change of SAP Note 1928533 (login required) and the release of the new templates, you can now create SAP landscapes on Azure that use less virtual machines and thus further reduce your IT costs.

Please also have a look at the following documentation

Windows Guides

• Using SAP on Azure Virtual Machines (VMs)
• SAP NetWeaver on Azure Virtual Machines (VMs) – Planning and Implementation Guide
• SAP NetWeaver on Azure Virtual Machines (VMs) – Deployment Guide
• SAP NetWeaver on Azure Virtual Machines (VMs) – High-Availability Guide
• SAP NetWeaver on Azure Virtual Machines (VMs) – DBMS Deployment Guide

Linux Guides

• Using SAP on Azure Virtual Machines (VMs)
• SAP NetWeaver on Azure Virtual Machines (VMs) – Planning and Implementation Guide
• SAP NetWeaver on Azure Virtual Machines (VMs) – Deployment Guide
• SAP NetWeaver on Azure Virtual Machines (VMs) – DBMS Deployment Guide
• SAP SUSE quickstart guide
• SAP HANA on Azure quickstart guide
• SAP HANA on Azure Large Instances

Over the course of the last few months we implemented several improvements in SAP BW. We already blogged about two improvements separately:

• https://blogs.msdn.microsoft.com/saponsqlserver/2016/11/14/simplified-and-faster-sap-bw-process-chains
• https://blogs.msdn.microsoft.com/saponsqlserver/2016/11/25/improved-sap-compression-tool-msscompress

In this blog we want to describe a few, additional improvements

BW query performance of F4-Help

Some BW queries were not using intra-query parallelism, because the BW statement generator did not generate the MAXDOP N hint. This was particularly an issue, when using SQL Server 2014 columnstore. As of SQL Server 2016, queries on columnstore are still pretty fast, even when using MAXDOP 1. See https://blogs.msdn.microsoft.com/saponsqlserver/2016/11/11/sql-server-2016-improvements-for-sap-bw for details. The following SAP notes fix the issues with missing MAXDOP:

• SAP Note 2148094 – MAXDOP for inventory queries
• SAP Note 2371083 – Enable Maxdop for F4-Queries

BW query performance of 06-tables

The 06-tables are used as a performance improvement for BW queries with large IN-filters, for example the filter condition “WHERE COMPANY_CODE IN (1, 2, 3, … n)”. When the IN-clause contains 50 elements or more, a temporary 06-table (for example /BI0/0600000001) is created and filled with the elements of the IN-clause. Then an UPDATE STATISTICS is executed on the 06-table. Finally, the 06-table is joined with the (fact-) table rather than applying the IN-filter on the (fact-) table. After executing the BW-query, the 06-table is truncated and added to a pool of 06-tables. Therefore, the 06-tables can be reused for future BW queries.

Historically, the 06-tables do not have a primary key nor any other database index. Therefore, no DB statistics exists and an UPDATE STATISTICS command does not have any effect. However, SQL Server automatically creates a column statistics, if the 06-table contains at least 500 rows. Smaller 06-tables do not have a DB statistics, which might result in bad execution plans and long running BW queries.

This issue has not been analyzed for a long time, because it is self-healing: Once a 06-table ever had at least 500 rows, it has a column statistics. Therefore, the UPDATE STATISTICS works fine when the 06-table is reused, even if it contains now less than 500 rows.

As a workaround, you can simply disable the usage of 06-tables by setting the RSADMIN parameter MSS_EQSID_THRESHOLD = 999999999. Therefore, you have to apply the following SAP note first:

• SAP Note 2234040 – Set MSS_EQSID_THRESHOLD

As a matter of course, this workaround has some drawbacks. The SQL queries get more complicated when turning off the 06-tables. Therefore, we made a code change, which changes the structure of the 06-tables. They now contain a primary key and have regular DB statistics. For using this, you have to implement the following SAP note and delete the existing 06-tables using report SAP_DROP_TMPTABLES

• SAP Note 2375321 – Performance of 06-tables

This note is not generally available yet, because we would like to have some pilot customers first. You can apply as a pilot customer by opening an SAP message in component BW-SYS-DB-MSS.

BW Cube Compression

You will get best BW cube compression performance in SAP release 7.40 and newer.

• With SAP BW 7.40 SP8 the Flat Cube was introduced (see https://blogs.msdn.microsoft.com/saponsqlserver/2015/03/27/columnstore-optimized-flat-cube-in-sap-bw). Therefore, we had to implement the cube compression for the Flat Cube. At the same point in time, we added a missing feature for non-Flat Cubes, too: Using the SQL statement MERGE for the cube compression of inventory cubes. Before 7.40 SP8, the MERGE statement was only used for cumulative cubes (non-inventory cubes).
• The next major improvement was delivered in SAP BW 7.40 SP13 (and 7.50 SP1): We removed some statistics data collection during BW cube compression, which consumed up-to 30% of the cube compression runtime. These statistics had not been collected by other DB platforms since years. At the same time, we added a broader consistency check for cubes with columnstore index during BW cube compression. This check takes some additional time, but the overall performance of the BW cube compression improved as of BW 7.40 SP13.

For all BW releases we introduced a HASH JOIN and a HASH GROUP optimizer hint for cubes with columnstore indexes. We already had delivered a HASH JOIN hint in the past, which was counterproductive in some cases for rowstore cubes. Therefore we had removed that hint again. By applying the following SAP Note, the HASH JOIN hint will be used for columnstore tables only. Furthermore, you can configure it the hints using RSADMIN parameters:

• SAP Note 2367122 – HASH JOIN HINT for BW Cube Compression

The last step of the BW cube compression is the request deletion of compressed requests. Therefore, the partitions of the compressed requests are dropped. It turned out, that the partition drop of a table with more than 10,000 partitions takes very long. On the one hand, it is not recommended to have more than a few hundred partitions. On the other hand, we need a fast way to compress a BW cube with 10,000+ partitions, because BW cube compression is the only way to reduce the number of partitions. We released the following SAP notes, which speed-up partition drop when having several thousand of partitions:

• SAP Note 2369986 – performance improvement for cube compression on MSSQL
• SAP Note 2371646 – performance improvement for DB_DROP_TABLE on MSSQL
• SAP Note 2329540 – Rowgroup Compression Framework for SAP BW / SQL Server

DBACOCKPIT performance

For large BW systems with a huge amount of partitions and columnstore rowgroups, Single Table Analysis in SAP transaction DBACOCKPIT takes very long. Therefore, you should update the stored procedure sap_get_index_data, which is attached to the following SAP Note:

• SAP Note 2364523 – Bad performance when checking column store tables in DBACOCKPIT

Columnstore for realtime cubes

In the past we used the columnstore only for the e-fact table of SAP BW realtime cubes. Data is loaded into the e-fact table only during BW cube compression. The cube compression ensures that DB statistics are up-to-date and the columnstore rowgroups are fully compressed. The data load into the f-fact table in planning mode does not ensure this. Due to customer demands you can now use the columnstore for selected or all realtime cubes. For this have to apply the following note, which also enables realtime cubes to use the Flat Cube model:

• SAP Note 2371454 – Columnstore and Realtime Cubes

BW System Copy performance improvements

We increased the r3load import performance when using table splitting with columnstore by optimizing the report SMIGR_CREATE_DDL. With the new code, r3load will never load into a SQL Server HEAP any more. Hereby you can minimize the downtime for database migrations and system copies. Furthermore, we decreased the runtime of report RS_BW_POST_MIGRATION when using columnstore. For this, you have to apply the following SAP notes:

• SAP Note 2372455 – BW sys copy import into clustered index
  
• SAP Note 2329672 – long runtime of RS_BW_POST_MIGRATION in step 2)
  

Customers automatically benefit from all system copy improvements on SQL Server, when applying the new patch collection of the following SAP Note:

• SAP Note 1593998 – SMIGR_CREATE_DDL for MSSQL
  

In addition, you should also check the following SAP Note before performing a system copy or database migration

• SAP Note 888210 – NW 7.**: System copy (supplementary note)

Summary

The SAP BW code for SQL Server is permanently being improved. All optimizations for SQL Server columnstore are documented in the following note:

• SAP Note 2116639 – SQL Server Columnstore documentation