Ant’s HomePage
Security Matters
OS/390 Auditing
The following is a snapshot of a page from Software Europe’s website c. August 2000, for historical interest only.

How Much is Your Mainframe Software “Leaking”?


| ACF2 Administration | RACF Administration |
| OS/390 Auditing & Event Monitoring | Open Systems Event Monitoring |
| Other OS/390 Security | Security Matters |


The following is based on a paper
How Much is Your Mainframe Software "Leaking"?
by Hans Schoone, one of the three founding partners of
CONSUL Risk Management B.V.

If you'd like a hard or soft copy of the original document,
or details of any of the software solutions mentioned,
please contact ——————.

Statistics on 350 Penetration Tests

Introduction

At CONSUL Risk Management we have found that system penetration testing of the world's main large business systems reveals the extremely low resistance of real-life systems to penetration. Our experience in penetration testing has resulted in a 100% penetration success rate requiring between ten minutes and two days to first penetration, even in today's systems. Our further testing revealed an average of three leaks per day.

Leak classification showed:

A leak can be defined as security administration practices, program design, or program code that compromises the normal integrity of the operating environment and/or the security of the information stored on the computer system. While a leak can be intentional or inadvertent, most leaks are inadvertent. The danger with an inadvertent leak is that it can cause operational problems on the computer system or can provide a "hacker" with a means of penetrating the environment. Let's begin the detailed study with an example.

A Case History

While the following situation may never have happened to you, it has happened many times before.

Monday

1:00 p.m. - It certainly is a hot afternoon. This is really frustrating! When I finally have time to continue testing on my new user interface program, the systems slows down and eventually hangs. When will the computing center get its act together?

4:00 p.m. - My program almost works, I am only missing one line from the display.

4:10 p.m. - The system response slowly deteriorates.

4:20 p.m. - The system hangs. I am going home!

Tuesday

9:30 a.m. - I receive a phone call from systems programming. Do I know where this line of text containing my user id comes from? "Uh, well, yes, and it was missing from my display yesterday."

I hear an angry reply: "So you are the one bringing our system down each day!"

The scenario above is typical for the first inkling that your system contains a major integrity exposure. It also shows why it is wise to have separate development and production systems.

There is a fascinating myriad of ways that exist to penetrate operating system integrity and security. They range from accidentally overwriting critical operating system storage to providing a universal read access for the security database. A platform expert can use these to circumvent all applications' security and arbitrarily obtain or use cryptographic keys, read and change database files, insert phantom transactions, and remove traces of it in audit trails. The amount of information on these computer systems that have between 2,000 and 100,000 employees working on them is staggering -- it is expressed in hundreds of terabytes.

The "Forever" Programming Flaws

There are numerous programming flaws that have been useful in penetrating systems for over ten years now. It is important to note that these flaws are all in non-IBM applications. One was first reported to the vendor in 1985, another in 1987. In contrast, all programming flaws reported to IBM have been fixed within a few years. Following is a mind-boggling response in a 1985 letter from an MVS software product vendor:

"We have many users with very strong security demands, like the CIA, who accept the risk and find the level of security to be adequate."

Did anyone tell the CIA that nearly anybody can break into its multi-level secure systems with minimal effort? This leak still exists today.

Our experience with OS/390 software vendors is also reflected by experience with UNIX vendors:

"Many of the bugs discovered (approximately 40%) and reported in 1990 are still present in their exact form in 1995."

Penetration of 350 Systems

We will present here statistics of system penetration tests performed on over 350 mainframe systems running IBM's OS/390 and MVS operating systems with an installed base of 10 to 500 additional products from a wide variety of vendors. The companies in question comprise many major financial institutions and well- known organizations around the world. They can be expected to be more secure than others because they spend money specifically on security. Clearly this kind of information is rather sensitive. However, we believe that the situation is grave enough to warrant wider attention, certainly in view of the ever-increasing pressure to connect these systems to the Internet, the amount of money transferred daily through these systems, and the perceived increase of electronic fraud.

Our employees and other companies in this field have collected these over the years. The following are the overall results:

  1. All 350 system penetration tests have been "successful," meaning we have been able to penetrate the target system in every test.

  2. Penetration was achieved in less than two days in every case. The minimum was ten minutes, which is truly a shocking fact.

Besides this common overall statistic, the actual detail data collected differs in nature across the three information sources. Rather than trying to find a commonality, they are presented in sequence below.

CONSUL's Test Results

We will present here some statistics of system penetration tests that have been performed by CONSUL employee Guus Bonnes (formerly of IBM) and myself:

  1. All system penetration tests performed by CONSUL specialists have been "successful," meaning we have been able to penetrate every system in every situation.

  2. Penetration was achieved in less than two days in every case. The minimum was ten minutes.

  3. Penetration tests and audits varying from two days to five weeks surprisingly show a more or less constant average of three leaks found per day. Audits always involved running the Consul/Audit for RACF product [now called Tivoli zSecure Audit for RACF], which automates much of the technical aspects of an audit.

The statement that all penetration tests were "successful" needs to be clarified by identifying the amount of authority we needed to achieve Trusted Computing Base -- that is, the highest system authorization.

Below are the methods that resulted in the initial penetration. Note that this is not a statistic of all leaks found during testing. There are many more instances of improper security product implementation penetrations as well as penetrations as a result of other operating system leaks, like globally writable storage and Program Call (PC) routines. The table below just shows the ones that were actually used first because they were the easiest to exploit:

Leaks used for initial penetration

SVC, or Supervisor Calls, are routines that interface an unauthorized caller with authorized SVC code. SVCs are especially high risk because the binary SVC code is loaded into globally readable storage (CSA) and can be studied at leisure. The same is true for many of their more modern counterparts, the PC or Program Call routines. An added risk is that many OS/390 product vendors supply SVC routines to help their applications achieve their purposes.

Penetration testing does not stop at the first leak exploited. Typically, additional effort includes a technical audit of critical parts of the system, like SVC and PC routines. The table below details SVC technical audit statistics covering more than 40 SVC technical audits. We did not spend more than eight hours on any SVC. Note that these times cover visual inspection (source code review, disassembly, or hex browse), not exploitation testing. The table shows that 78% of the SVC routines reviewed contained a leak. Because most leaky SVC routines contain more than one leak, the average time needed per leak is actually rather low. Note that the analysis is skewed by the fact that the SVC routines reviewed were not selected randomly, but based on a suspicion that something was more likely to be wrong with these SVCs than with others.

SVC review

Average review time

Max. time to first leak

Fraction

No leak found

6h

-

22%

At least one leak

2h20m

2h

78%

Time per leak

1h

2h

-

A penetration method that has not been exploited in the statistics above, but is definitely on the increase, is exploiting a sniffer on the LAN to obtain the mainframe passwords.

Another statistic we only recently started collecting is the comparative security between multiple systems in a site. It turns out that the newest is usually the leakiest in terms of improper security rules, but often has fewer (but not yet 0) leaky SVCs.

As previously stated, we found 19% initial penetration successes as a result of improper security access rules. This is probably less than would be achieved unannounced because many sites check their security implementations just before a penetration test starts to try to positively influence the results.

ECHO Statistics

Kurt Meiser of ITSS (formerly Coopers & Lybrand) has rated the results of his penetration tests and technical audits into:

Hence the acronym "ECHO."

The figures below shows the results for the category "Authorization and Privileges" collected for 71 systems in 41 installations.

Independent Statistics

Another expert in the field, Peter Goldis, has done over 200 MVS/TSO studies where he always found at least one leaky SVC. He also did over 20 UNIX penetration tests that also were 100% "successful."

Yet another expert in the field, Finn Thunbo Christensen of IBM Danmark, stated that penetration is typically successful within one day and especially highlights the "subroutine with AC(1)" cause for leaks.

What Can We Do?

Finding these improper security implementations over and over again, we felt compelled to write software that can automatically detect and correct these situations. This software was brought to market in 1990 under the name Consul/RACF [now called Tivoli zSecure Admin].

We have continued to develop this product extensively and moved the penetration test knowledge into the current products, Consul/Audit for RACF and Consul/Audit for ACF2 [now called Tivoli zSecure Audit for RACF and Tivoli zSecure Audit for ACF2]. We have introduced another product named Consul/CVO (Command Verification Option) [now called Tivoli zSecure Command Verifier] to help prevent improper RACF implementations by verifying RACF command contents before the command is executed.

By far the largest factor in initial OS/390 system penetrations are SVCs of Independent Software Vendors (ISVs). The leaks found can be split fifty-fifty into programming errors and basic design flaws. What alarms us is that even modern software (e.g. year 2000 support packages) contains the same kind of problems as old software.

The long-term solution must come from better education of programmers and designers and the implementation of Peer Code Reviews. Theory on security can be learned from Amoroso's Fundamentals of Computer Security, but this ignores the paramount importance of software engineering techniques. A more balanced overview of computer security is found in Rita Summers' book, Secure Computing: Threats and Safeguards. Until and even after that occurs, there is a need to perform automated operating system and security environment audits to constantly look for leaks. We have designed Consul/Audit to check thousands of settings, check asset protection, and analyze storage protection and executable code. If a potential suspect setting or instruction is encountered, an audit observation is recorded and an audit priority is assigned. The audit priorities of SVC routines have proven to have a good correlation with SVC routines that, in fact, contain leaks.

Will Modern Software Help?

It is often claimed that programming in a higher-level language would diminish the chance of flaws. Our experience so far is that this has not been demonstrated. This observation may be supported by the following points:

  1. Operating systems written in C seem to have an even higher leak rate than operating systems written in assembler, as illustrated by the CERT Coordination Center's Internet site and through the listserv, bugtraq.

  2. PC operating systems like OS/2 and Windows NT which have large parts written in C fail more often than the mainframe systems. As illustrated in our case history at the beginning of this study document, unexplained "hangs" are often symptoms of major integrity leaks and thus hang frequency can be used as a contributing measure for system integrity.

  3. Code review of a higher level language does not at all guarantee safe expansion by the compiler. Large numbers of security leaks are generated by the incorrect assumption that one cannot write outside arrays.

On the other hand, this may also be due to design flaws in the basic operating system -- UNIX for example -- or quality control issues by the vendors. In fact, we suspect that properly designing and testing for security is of paramount importance and that the actual programming language is irrelevant.

This study supports the impression that mainframes are basically safer than a lot of more "modern" operating systems. The statistics show that it is practically always a site or third party vendor addition to the operating system that caused the leak, not a deficiency in the OS or its subsystems.

Conclusions

  1. The current state of technical security in the world's computer systems is appallingly bad.

  2. The technical insecurity encountered is due to improper security implementations, improper programming design, and errors in program coding.

  3. Site-specific customization and third party vendors cause most leaks. All flaws found in MVS itself have since been fixed.

  4. Newly educated programmers make exactly the same mistakes as their colleagues did 20 years ago.

  5. Audit tools based on real life experience and knowledge are needed to detect and correct the growing "technical insecurity." Consul/Audit for RACF, Consul/Audit for ACF2, and Consul/Enterprise Audit [now called Tivoli Compliance Insight Manager] are built from the knowledge gained in the penetration testing results described in this document.

[BACK TO TOP]


About the Author

Hans Schoone is one of the three founding partners of CONSUL Risk Management B.V.

After graduating in both electrical engineering and mathematics (with honors) at the Delft University of Technology in 1985/86, he worked for Inter Access Consultancy B.V. and later his own firm, CONSUL, and was hired as a systems programmer, technical auditor, performance expert, and security consultant by large companies. He has done MVS performance troubleshooting projects, system penetration testing and technical audits on MVS, VM, VMS, and MPE platforms; supervised penetration testing on Windows NT, routers and firewalls; and has written and given MVS security courses and introductions on Evaluation Criteria.

He is currently lead designer and product manager of the Consul/RACF and Consul/Audit products. He was project leader of a joint research effort with the Delft University of Technology called Audion, auditing in open networks. This research laid the foundation for the Consul/Enterprise Audit product.

He also participates in the permanent Dutch Informatics Society workgroup that reviews draft international Security Evaluation Standards. He has presented for the Dutch Informatics Society, EDPAA, RACF Security Expo, and workshops on Security Evaluation Criteria. He has written about operating system integrity, hacking, security evaluation, and performance issues.

[BACK TO TOP]


Copyleft & Creative Commons (cc) 2000–2008 Ant: This XHTML encoding is dual-licensed under both ―
GFDL
The GNU Free Documentation License
  Creative Commons License
A Creative Commons Attribution-Noncommercial-Share Alike 3.0 License
URL
http://homepage.mac.com/antallan/howmuch.html
History
Last updated Friday 8 August 2008

Made on a MacBuilt with BBEdit In Association with Amazon.co.uk Valid XHTML 1.0! Valid CSS!