Adding Monte-Carlo analysis to iWork Numbers
03/02/09 06:03 Filed in: iWork
Monte-Carlo analysis is not a new method of
analysis. Scientists, engineers and project
managers have been using it for years in many
different fields to solve problems with multiple
varying inputs. As a tool it has proved to be
incredibly adaptable and reliable as a problem
solving method. It is simple to understand, works
with models of virtually any type and gives a good
understanding of a problem. So the question is why
is it not used more?
The first problem seems to be that it is not well understood. I think that few engineers actually have a chance to learn about methods if they don't have access to the tools to actually use them. Monte-Carlo analysis can only really be used with a computer model capable of running a Monte-Carlo simulation. It can be done manually, and for complex models sometimes it is necessary to just perform a few runs and manually compare the difference (ref. 1). This is not much different to performing a finite element analysis manually though, it can be done but there is little benefit for all the work that goes in. There are far better ways to solve the problem if manual methods are going to be adopted.
Tools are available to the engineer to undertake Monte-Carlo style risk analysis. The problem is that they are not accessible and they come with an enterprise price. Combining the the cost per seat of the software, the training cost and the learning cost, the price of getting up and running with these tools prevents engineers just having a go. It becomes a serious proposition to look at using Monte-Carlo analysis on a project and yet so often some fairly small decision could be well informed using the method. The tools should encourage the playful, explorative search for a solution as well as being capable of performing accurately, tightly defined, high quality analysis necessary when solving a significant problem.
Monte-Carlo analysis is also prone to a credibility problem. The outputs an always statistical. If you ask your programmer when the project will be complete it is much easier to deal with the answer '4 weeks time' rather than 'there is a 67% chance that the project will be complete in 4 weeks time'. Whilst the statistical answer is more informative, it can also hide many problems. If the project over-runs beyond the 4 week deadline it is easy to say there was a problem with the deterministic answer of '4 weeks time'. You can't readily say whether the statistical answer is right or not though.
For a project manager this would be a serious issue, because without some form of validation of the method being used, you can not have any confidence about using the methodology. Validation of a method only really comes with its regular use; repeated use gives both direct evidence of its performance as well as the personal, empirical understanding of how effective a method is.
To help with these problems I have started to develop a tool that will put Monte-Carlo analysis, as well as other similar tools, into the hands of any engineer. When Apple updated iWork earlier this year, Numbers finally got an Applescript dictionary. This means that automated tools can be created which hook into Numbers. The aim is to create a complete suit of automated tools that allows engineers, scientists and managers to create a model using the freeform approach of Numbers and then analyses the model using simple, intuitive but powerful tools to explore the behaviour of the model.
The project is in an early Beta phase and I would like as many people to try it as possible. People use these tools in many different ways and I need to understand how people will use it to understand how to develop it further, without impacting on the creative use of the tool.
If you are interested then please have a look at the Risk Engine page, download the application and try it out in your models.
Thankyou for your support.
References
1. One is not Enough, 2008, Rocscience Inc., [http://www.rocscience.com/library/rocnews/winter2009/One-Is-Not-Enough.pdf]
The first problem seems to be that it is not well understood. I think that few engineers actually have a chance to learn about methods if they don't have access to the tools to actually use them. Monte-Carlo analysis can only really be used with a computer model capable of running a Monte-Carlo simulation. It can be done manually, and for complex models sometimes it is necessary to just perform a few runs and manually compare the difference (ref. 1). This is not much different to performing a finite element analysis manually though, it can be done but there is little benefit for all the work that goes in. There are far better ways to solve the problem if manual methods are going to be adopted.
Tools are available to the engineer to undertake Monte-Carlo style risk analysis. The problem is that they are not accessible and they come with an enterprise price. Combining the the cost per seat of the software, the training cost and the learning cost, the price of getting up and running with these tools prevents engineers just having a go. It becomes a serious proposition to look at using Monte-Carlo analysis on a project and yet so often some fairly small decision could be well informed using the method. The tools should encourage the playful, explorative search for a solution as well as being capable of performing accurately, tightly defined, high quality analysis necessary when solving a significant problem.
Monte-Carlo analysis is also prone to a credibility problem. The outputs an always statistical. If you ask your programmer when the project will be complete it is much easier to deal with the answer '4 weeks time' rather than 'there is a 67% chance that the project will be complete in 4 weeks time'. Whilst the statistical answer is more informative, it can also hide many problems. If the project over-runs beyond the 4 week deadline it is easy to say there was a problem with the deterministic answer of '4 weeks time'. You can't readily say whether the statistical answer is right or not though.
For a project manager this would be a serious issue, because without some form of validation of the method being used, you can not have any confidence about using the methodology. Validation of a method only really comes with its regular use; repeated use gives both direct evidence of its performance as well as the personal, empirical understanding of how effective a method is.
To help with these problems I have started to develop a tool that will put Monte-Carlo analysis, as well as other similar tools, into the hands of any engineer. When Apple updated iWork earlier this year, Numbers finally got an Applescript dictionary. This means that automated tools can be created which hook into Numbers. The aim is to create a complete suit of automated tools that allows engineers, scientists and managers to create a model using the freeform approach of Numbers and then analyses the model using simple, intuitive but powerful tools to explore the behaviour of the model.
The project is in an early Beta phase and I would like as many people to try it as possible. People use these tools in many different ways and I need to understand how people will use it to understand how to develop it further, without impacting on the creative use of the tool.
If you are interested then please have a look at the Risk Engine page, download the application and try it out in your models.
Thankyou for your support.
References
1. One is not Enough, 2008, Rocscience Inc., [http://www.rocscience.com/library/rocnews/winter2009/One-Is-Not-Enough.pdf]