Working paper - Why is there so little impact of system dynamics in the most important social questions?

#1

Jay Forrester, in his 2007 paper on System dynamics—the next fifty years, asked a question that has not yet been answered: “Why is there so little impact of system dynamics in the most important social questions?” We have long been working on that question and wish to offer a working paper on the subject for discussion and feedback, as this question is near and dear to the hearts of those who have long been in the system dynamics community.

The paper title is: The process must fit the problem: Integrating root cause analysis with the system dynamics modeling process for difficult problems.

Abstract - System dynamics has the theoretical potential to productively model any dynamic problem where entity flow can be aggregated without significant loss of information and to offer practical solution strategies based on the model. However, in practice, as Jay Forrester observed, the field is presently stagnated “on a rather aimless plateau… there is very little penetration into the big issues.” We argue the central reason is that for the more difficult problems, the present modeling process does not fit the problem because it lacks root cause analysis. This too often results in models that omit a problem’s root causes and therefore the correct high leverage points. The paper begins the conversation for filling this gap by presenting an educational example of a comprehensive process for integrating root cause analysis into the system dynamics modeling process.

The working paper may be found here. Only the first 8 1/2 pages are drafted. The rest is in detailed outline form. This is exactly the stage at which the young paper can benefit most from insightful comments. The direction of the paper is defined, but not the full argument and content.

We hope this will be a stimulating and productive forum discussion. Thanks to all!

Jack Harich and Montserrat Koloffon Rosas

#2

Jack, the paper link is not working.
Len

#3

Thanks Len, sorry it’s not working. I tested it beforehand and it worked. It’s also working now. I’m using Chrome. This is the URL:
http://www.thwink.org/sustain/publications/papers/index.htm#ProcessMustFit

I notice that in Chrome, it says thwink.org links are not secure. Perhaps your browser is preventing you from visiting such links? Just a guess. I’ve also just tested it in Firefox and it’s working there. Perhaps Bob the web wizard can shed some light on this…

#4

Re: change resistance and social force diagrams

The book below discusses why people have a Soldier Mindset and resist change, use faulty reasoning and don’t seek the truth. The media would have us believe political party or ideology determine what people believe. Soldier Mindset uses “directionally motivated reasoning” to determine what to believe.
A Soldier Mindset will never agree to the 5 Why process, or root cause analysis. You can not prove them wrong. They know what a beaver looks like, and that was what was eating the fish, and you can’t convince them otherwise.

Quote: “Research shows that showing people research doesn’t work.” John Sterman MIT

Recommendation: The Scout Mindset: Why Some People See Things Clearly and Others Don’t
by Julia Galef

#5

Hi Richard,

Thanks for your comment. First, only the roughed in outline spoke of “change resistance and social force diagrams.” The first eight pages did not. Those pages are what we are most looking for comments on.

For your comment to be useful, it would help to explain precisely where in our research it applies and how it applies. “Change resistance and social force diagrams” is too general. I really have no idea what you have in mind. Perhaps you can explain? Can you quote the material it applies to, and provide a reasoned argument for how it applies? We would really appreciate that.

Regarding: “A Soldier Mindset will never agree to the 5 Why process, or root cause analysis. You cannot prove them wrong. They know what a beaver looks like, and that was what was eating the fish, and you can’t convince them otherwise.” - If I understand what you’re trying to say correctly, this doesn’t apply to a process for system dynamics modelers. That is the audience for the paper, not the public at large, some of whom might be soldiers.

Regarding: “Quote: “Research shows that showing people research doesn’t work.” John Sterman MIT” – Can you put that quote in context? The vast majority of the people I know do believe information based on solid research. Perhaps the quote refers to the well-known cognitive psychology behavior of confirmation bias? If a person already believes something strongly, they tend to accept and seek out information supporting the belief and reject and avoid information refuting the belief.

I did look at the book. Thanks. Cool stuff. Perhaps the author has published a paper demonstrating and/or arguing how this theory applies to human behavior? Otherwise, it doesn’t seem that much different from thousands of other popular psychology books for personal growth or helping others. The blurb saying “The Scout sees more clearly, makes better judgements, and enjoys the freedom and thrill of discovery. Learn how to be a scout” tells me it’s a self-help book.

#6

“First, only the roughed in outline spoke of “change resistance and social force diagrams.” The first eight pages did not. Those pages are what we are most looking for comments on.”

OK, I guess I was wrong about the book’s concepts applying to your topic. Galef suggested in her book that the concepts are scalable and could be applied at a larger scale, so was wondering if maybe you might be able to apply them to your project. Guess not.
Take Care
Richard

#7

Ah, I see, thanks. Did Galef show specifically how the concept could be scaled, or offer any clues? Or was it just speculation? Our work deals with systemic change resistance, rather than individual change resistance or behavioral factors.

#8

When I built an SD model of climate change I showed that increasing CO2 in the atmosphere was the root cause, along with other greenhouse gases. That SD model was based on the Steffan-Boltzmann equation and many unit conversion values. That SD model was science based using physics equations. Next, one would need to build an SD model of how to control CO2 and other greenhouse gas emissions in order to reduce their concentration in the atmosphere. That would penetrate into a big issue.

An example of not identifying the root cause is how to help low income people. Providing food pantries and free lunches at schools does not result in taking action at the correct high leverage point. Actually, people suffering from food insecurity need free childcare, low cost housing and low cost transportation. Giving people a monthly stipend with no strings attached gives them the opportunity to improve their lives and eventually support themselves using a high leverage point. This was done in Stockton CA.

As you state, "society’s largest problems are of such scale and public interest that they must be addressed by governments.” Is government the system that lacks a root cause analysis? Does this result in omitting a problem’s root causes and therefore the correct high leverage point in the government system?

As you state, “Industry’s solution to its top wicked problem was continuous improvement…”. Government’s solution to its top wicked problems is continuous improvement of all kinds but the time frame of implementing improvements might be decades or centuries. The US government still uses the electoral college defined in the Constitution and as modified by the 12th amendment, adopted in 1804, that solidified the two political parties.

The history of RCA on pages 3 and 4 is irrelevant to the main question of your paper.

The message of this paper on page 5 in italics uses an “If-Then” format. An SD model or SD modeler that uses “If-Then” statements pre-determines the result. You are forcing a specific outcome without any RCA or allowing the dynamic system to show you the high leverage points. You are writing a paper about RCA and dictating to SD modelers that they must use RCA or they are going to omit the correct high leverage points.

My point: A better use of your time would be to take action and apply the process you are advocating, to one or more government systems that are trying to solve wicked problems.

On page 5, doctors use a process of elimination to diagnose and treat patient illness. Once they’ve eliminated potential problems then they focus on what’s left. That’s how they narrow in on the root cause. Again, on page 8, problem 1, doctors also triage a patient and treat immediate problems first, then get more tests and narrow in on the root cause.

One assumption you might be making is that SD models are the real world in the same way that Industry used RCA to change the real world. However, SD models are not the real world. SD models never include the root cause. SD models give you insights into why systems cause people to behave the way they do. Systems determine why people behave the way they do.

Conclusion

Instead of writing a paper about the process, apply your process to a real world wicked problems like climate change, pandemic, voting rights, system racism and many other social issues. Show me, don’t tell me.

#9

Hi
About the ‘aimless plateau’, Jay forester gave his own explanation: the general mediocre and superficial quality of the profession models. He advised too anybody really interested to cut all links to official or public organisations and to work on his own. This was reported by Jack Homer some years before.
Of course, the explanation of the poor SD impact on the real world depends on its author.
From my point of view the explanation does not come from root causes. Finding root causes will generate huge models with infinite boundaries and will lead to the big bang, 13 billions years earlier.
It comes from the lack of knowledge and experience of how the real world is functioning by the modelers who may be good in maths but have rarely worked within it. This lack of experience generates first bad and uncredible models but more importantly suspicion from the decision takers.
When I look at the models presented at an SD conference, they all look uncredible, proposing policies that should be applied but not that could be applied.
Regards.
JJ

#10

Hi Richard,

Thanks for the helpful feedback. Let me address some of your points:

“When I built an SD model of climate change, I showed that increasing CO2 in the atmosphere was the root cause, along with other greenhouse gases.” – The paper said “A root cause is the deepest cause in a causal chain (or the most basic cause in a feedback loop structure) that can be resolved.” You don’t seem to be using that definition. Tracing the causal chain of CO2 emissions, we find many deeper causes, like vehicle emissions, that have even deeper causes, like long commutes and drivers not taking public transport. The rising level of CO2 in the atmosphere is an intermediate cause.

“An example of not identifying the root cause is how to help low income people.” – The note in the paper said “Can anyone think of an actual case where a completed SD model was later discovered to not include the root cause(s)?” Are you describing a completed SD model was later discovered to not include the root cause(s)? Where exactly is this model?

“As you state, ‘society’s largest problems are of such scale and public interest that they must be addressed by governments.’ Is government the system that lacks a root cause analysis? Does this result in omitting a problem’s root causes and therefore the correct high leverage point in the government system?” – Nice question, but what does “Is government the system that lacks a root cause analysis?” mean? Are governments not using RCA or is RCA not being performed on why governments are unable to address this problem?

“As you state, ‘Industry’s solution to its top wicked problem was continuous improvement…’. Government’s solution to its top wicked problems is continuous improvement of all kinds but the time frame of implementing improvements might be decades or centuries. The US government still uses the electoral college defined in the Constitution and as modified by the 12th amendment, adopted in 1804, that solidified the two political parties.” – How is this relevant to the paper? What is your specific suggestion to improve the paper or what is the problem in the paper?

“The history of RCA on pages 3 and 4 is irrelevant to the main question of your paper.” – Thanks. If you are familiar with papers, you will realize that most have a literature review of some kind after the introduction, in order to educate the reader on the context of problem and where the paper’s research question and potential contribution fit into the literature. A paper on how to integrate RCA with SD modeling must therefore review RCA. What is it, what does it accomplish, how successful has this method been?

“The message of this paper on page 5 in italics uses an ‘If-Then’ format. An SD model or SD modeler that uses ‘If-Then’ statements pre-determines the result. You are forcing a specific outcome without any RCA or allowing the dynamic system to show you the high leverage points.” – I don’t understand what you are trying to say.

“You are writing a paper about RCA and dictating to SD modelers that they must use RCA or they are going to omit the correct high leverage points.” – Well, I don’t think the paper is “dictating” anything. However, it does present these premises: all causal problems arise from their root causes, difficult modeling problems are causal problems, and RCA is the only known method of reliably and efficiently finding and resolving root causes. Therefore, if a problem is so difficult that non-RCA-based methods are failing, then it would be prudent for the problem solver to use RCA. This is exactly why RCA is the foundational method for all large-scale industrial processes. The paper attempts to allow SD modelers to be just as successful. RCA is not perfect. People (especially those in training) do miss root causes on their first pass, so they try again. But RCA is orders of magnitude better than non-RCA methods like trial and error, expert judgement, statistical correlation, and so on, for difficult causal problems.

“My point: A better use of your time would be to take action and apply the process you are advocating, to one or more government systems that are trying to solve wicked problems.” – Thanks, but this makes little sense. Method papers are common, especially in the social sciences. Editors and readers are not rejecting them for the reason you use, because papers about potential new methods, or method changes, are extremely useful. Sometimes data precedes theory, and sometime theory precedes data. That’s the way scientific progress works.

“On page 5, doctors use a process of elimination to diagnose and treat patient illness. Once they’ve eliminated potential problems then they focus on what’s left. That’s how they narrow in on the root cause.” – This seems to argue that doctors are not using RCA but are using something else. You seem to be very resistant to the idea that RCA can be useful. RCA is the foundational method behind all formal methods of solving causal problems. The paper says “RCA is the systematic practice of finding, resolving, and preventing recurrence of the root causes of causal problems.” How does the process of elimination (a systematic practice) to find the root cause of an illness not fit this definition?

“Again, on page 8, problem 1, doctors also triage a patient and treat immediate problems first, then get more tests and narrow in on the root cause.” – Triage has nothing to do with RCA. It’s a problem management issue.

“One assumption you might be making is that SD models are the real world in the same way that Industry used RCA to change the real world.” – I don’t understand. Perhaps you are writing too fast?

“However, SD models are not the real world. SD models never include the root cause.” Models frequently contain root causes, in the same sense that models contain things like growth rates and population. A successful policy resolves the root causes and solves the problem, first in the model, and then the real world. Many SD models have done this, as it is usually their primary purpose. Of course, they usually don’t use root cause terminology.

“Conclusion. Instead of writing a paper about the process, apply your process to a real world wicked problems like climate change, pandemic, voting rights, system racism and many other social issues. Show me, don’t tell me.” – This repeats an earlier comment you made, starting with “My point.” See my reply to that.

Richard, I hope my replies help you to see a little deeper into the complex situation the paper attempts to deal with. It wasn’t easy to write.

Thanks again,

Jack Harich

#11

Hi JJ,

I appreciate your taking the time to look the paper over and share your feedback. My comments are below.

“About the ‘aimless plateau’, Jay forester gave his own explanation: the general mediocre and superficial quality of the profession models. He advised to anybody really interested to cut all links to official or public organizations and to work on his own. This was reported by Jack Homer some years before.” – Thanks. Why is the quality of SD models on difficult problems low? The paper argues it’s because modeling is not RCA driven.

“Of course, the explanation of the poor SD impact on the real world depends on its author. From my point of view the explanation does not come from root causes. Finding root causes will generate huge models with infinite boundaries and will lead to the big bang, 13 billion years earlier.” – You seem to be saying that RCA cannot help SD modelers solve problems, because that will generate huge unmanageable models. Why will this happen? You imply it’s because asking why successively will have no end. Let me explain why that’s not true.

The paper says “A root cause is the deepest cause in a causal chain (or the most basic cause in a feedback loop structure) that can be resolved.” The Five Whys example goes five layers deep to find the root cause. An SD model of the “Why did the machine stop” problem would be small or medium size. There would be no tendency to be infinitely large. Does this help you to see that RCA can be applied practically?

“It [the explanation of the poor SD impact on the real world] comes from the lack of knowledge and experience of how the real world is functioning by the modelers who may be good in math but have rarely worked within it. This lack of experience generates first bad and uncredible models but more importantly suspicion from the decision takers.” – I think you are saying that low modeler expertise in a problem area causes low quality models. The SD literature disagrees. It says that by following best practices, high quality models can be produced. Acquiring a high level of modelling expertise takes time and training. The record shows that many times modelers who were not already familiar with a problem have been able to apply best practices and produce good models that solved problems or allowed significant progress in the client’s eyes. Please see Sterman’s book, Jack Homer’s book, and many SDR articles for proof.

Thanks,

Jack Harich

#12

You might consider the possibility of reading The Scout Mindset and practicing some of the actions suggested by the author. Like all of us, including myself, we tend to default to the Soldier Mindset.
Thanks
Richard

#13

Hi Jack
Thank you for your answer. Technically I agree with you but mixing my experience in the real world will generate different conclusions. To agree with you globally would need that we share quite similar life experiences: I have been managing an SME (100 employees) for 40 years and been in the board of another family business (about 80 employees) for about the same time plus other experiences: a franchising system, a common central buying system for French vehicles renters, a consultancy in mathematical finance, a cinema production company. I have too a good mathematical background and more than 40 years of programming experience (building my own business information system) and 20 years of SD practice. The mixing of all these experiences generated conclusions that are impossible to explain, especially to people with different experiences. I have experimented this in past SD forums. The only solution is for each party to progress on its own and this is what I am doing and I wish that you will do it too.
I looked too at the common property laws, an interesting idea. But its implementation will be difficult because common property means no unique owner. Common goods are the property of everybody and nobody at the same time, generating a low level of motivation. This is why private management will always be more performant than public one.
Regards.
JJ

#14

Hi everybody
I took some time to think about the reasons of little impact of SD on social situations. Most of the time SD studies are about big social organisations (state agencies, busyness). To be implemented the people in charged of these organisations must collaborate closely with the study. Unfortunately, the decision power in these organisations is diluted, varying with the time (example of elections), often more officious and hidden than official. To resume most of time nobody has a real power of decision that is more or less collective. But, of course, you will find people who can decide to launch or buy a study whose cost and risk is negligeable compared with its implementation. If you do a such a study, you have no real decision-taker to talk with and the model even of high quality will never be implemented. Not to mention that changes are difficult to trigger in big organisations due to their inertia.
More than 10 years ago I worked with a consultant on a study and he told me that in more than 10 years of DS practice, it was the first time he had the occasion to talk to a decision taker!
To resume, SD as long as it will not be possible to be used by decision takers or problem owner, like spreadsheets for example, has no chance to succeed in social situations and will never be used in big social problems within big social organisations due to the impossibility to work with identified real decision takers.
JJ

#15

Some deep thinking here! Thanks.

Let me summarize your conclusion: “As long as government decision makers or problem owners do not personally use SD, SD will not succeed in solving big social problems within big social organisations.”

This implies that if a decision maker or problem owner does not know how to use a problem-solving tool, they must learn it themselves and apply it. They cannot do what is normally done: Hire consultants like Jack Homer to study the problem, or evaluate reports on the problem prepared by those outside government or by government problem solving projects. An example of the latter is the periodic IPCC reports on the climate change problem, prepared by thousands of scientists. These reports involve dozens of advanced tools (statistics, data analysis, ice core examination, carbon dating, econometric modeling, etc.) that are so specialized they are usually not personally used by political decision makers. That’s the purpose of specialized problem analysts.

Thus, I would conclude that while it would be helpful for government decision makers or problem owners to know SD, it is not a requirement for SD’s success on large social problems.

SPECIAL NOTE – The paper mentioned at the beginning of this thread is now complete. If runs 41 pages, which is too long. Now that we know what we want to say, we will shorten it in subsequent versions.

Jack Harich

#16

Hi Jack.
What I explained in my post is not specific to SD. Any complex method will generate the same effect.
Big organisations have many decision takers and it is difficult to reach a consensus and common understanding on the problem, allowing a common agreement on the way to act. Not to mention that in big organisations, the decision takers and the problem owners are not the same people.
I recently read some parts of a book “discrete-event simulation and system dynamics for management decision making” and on page 223 about SD modelling, one could find the same figure that exists in Sterman’s business dynamics page 87 and 88 in the paragraph ‘modelling is iterative’. What is written here is rather evident and not specific to SD. But I have never seen any paper in 20 years reporting an experience about such an iterative process simply because to have such a process there must be an initial implementation, something that is rarely done. The question is then: is Sterman too academical, ignoring the constraints of reality, and proposing methods impossible to apply, or are practitioners not able to apply it for different reasons (lack of knowledge, experience, time, too low fees, etc…) or are clients not really motivated to follow such a long process.
Probably all these reasons at the same time.
Sterman’s method seems totally logical but to be applied he should have added to my opinion, that it is better to start with a very simple model, easy to understand by all the decision takers, generating policies easy to apply even if totally not optimised and rustic with the possibility to see easily the results after a certain delay.
I happened to meet 40 years ago in Grenoble the man who was building with 7 modelers a model based on Forrester method whose purpose was to model the French economy, solving all the government’s questions: level of taxes, level of interest rate, investments etc… He was proud to announce that he had already 60 thousand equations written in Pascal and that he expected the model once finished to have 150 thousand. The model was running on a big mainframe. All the economic papers where writing papers about this model. I heard still about this model for a certain time but after no more news! It was clear that nobody was able to use that model, the author included. Now if instead of building such a monstruous model, one had built a very simple one, with one stock, extremely easy to use and accept to apply simple policies for 5 years, and see the results and modify the model and apply new policies the way that Sterman is explaining it, I think that after 40 years and 8 successive modifications, one would now have in France a model that does a pretty good job something that is certainly not the case. But was it possible to do this very simple task?
Was it possible to propose a very simple model proposing simple policies? Everybody would have said: I could tell the same thing and propose such policies without any model at all, and it is not reasonable to wait 40 years to at last have something effective. People want quick results given by sophisticated methods because they do not believe that a simple method can generate quick results. A consultant will be judged by how sophisticated his work is, and will never dare to propose a method that is simple and requires a long time to become effective. This is what I wanted to explain but probably was badly formulated in my preceding post.
JJ

#17

Wow, some very interesting thoughts!

Re: “Big organisations have many decision takers and it is difficult to reach a consensus and common understanding on the problem, allowing a common agreement on the way to act.”

Some struggle to make reasonably good decisions, and some do well. Those that do well, such as long-term industry leaders like Toyota or McKinsey Consultants, have an overall process that consistently works well for them. Thus, I don’t think the constraint is organization size. The limiting factor is process quality. Size is merely one more factor that must be well-managed. I have studied these processes to see what can be learned and applied to difficult large-scale social problems.

Re: “Modeling is iterative.” To me this mostly refers to model construction. An evolving model need not be implemented to learn most of how well it works. It can be run, inspected, and shown to experts. Laboratory experiments can be run. Natural real-world experiments can be run with past data. All this can serve as input to model improvement without “final” policy implementation. That’s the great benefit of simulation versus implementation. It’s a huge time and cost saver.

Sterman, Business Dynamics, p87 makes the point that “Iteration can occur from any step to any other step,” referring to a diagram with 5 steps. Bundled into step 5 is policy formulation, implementation (not stated but implied), and testing.

You write: “to have such an [iterative] process there must be an initial implementation.” The literature seems to disagree. In an iterative process, model iteration can occur due to insights from many steps, not just implementation.

Re: “it is better to start with a very simple model, easy to understand by all the decision takers, generating policies easy to apply even if totally not optimised and rustic with the possibility to see easily the results after a certain delay.” (Followed by a story about a very large model that was so large it ultimately failed and was abandoned.)

Great point, great story, I agree. This capitalizes on the well-known principle that well-running complex systems of any kind, technical or social, start simple and evolve to reach manageable complexity, which can take a long time. But countless cases exist where this principle was ignored, apparently due to hubris (unjustified excessive pride or confidence). In your story, the model builder’s high confidence was unjustified.

Re: “A consultant will be judged by how sophisticated his work is, and will never dare to propose a method that is simple and requires a long time to become effective.”

Sometimes yes, sometimes no. Here I think “method” means “policy recommendation.” There are many examples of simple policy changes that take a long time to become effective, such as universal education, curriculum changes, company or government hiring policies, John Maynard Keynes’s invention of fiscal policy to radically lessen the size of economic cycles, etc.

Another example is a company changing to the lean process, which is simply eliminating all forms of waste. Going lean take many years, especially since things often get worse before getting better. See this article.

A further example is a friend whose parents steered him to becoming a doctor. Simple decision. But it required 27 years of school-based education before his “final” graduation party, where the theme was “school is out forever!” :slight_smile:

Happy trails,
Jack Harich

#18

Hi Jack.
It seems that we do not understand Sterman’s 88 page the same way.
You think that after the arrow into the real world, the process is finished. I think that it leads to a new start.
One should have Sterman’s opinion about this diagram.
Of course, as a system dynamics practitioner who sells his work, it is much more attractive to propose a one-shot model without having to experiment with it and modify it to start again a new cycle, and where the job is finished even if it is not implemented.
Not selling SD but using it on my own problems, I will never agree to anyone selling SD (or teaching, to resume earning one’s life with SD). This makes all discussions impossible. A car sales-man will never be objective about the car he sells.
But this discussion may suggest that what SD people think about SD, maybe very different from what SD user think. And the facts prove that. SD is scarcely used in practice and fails to demonstrate in practice the capability it pretends to have and that it would have if it was properly used.
In the last SD conference, I have not found among more than 200 papers, not one exposing a model that is expected to be implemented, or has been implemented, with or without success.
I have always been surprised how the field considers that his job is to make models without bothering about its eventual implementation.
But I will finish here, having not had in 20 years the opportunity to share ideas with an end SD user.
I wish you all the best and thank you for your answers.
JJ

#19

Hi Jack
I saw that in your paper, you mention the world3.03 Vensim model. Are you interested or is anybody interested to know why I do not consider this model as reliable or just bug free? If anybody does not want to appear as considering this eventuality, he can contact me at sddp4119@gmail;com. JJ

#20

Bonjour, JJ.

I would certainly appreciate this discussion of the World3 model, as long as it’s done in an objective, constructive manner. One consideration would be to define two of your terms, reliable and bug-free. This subject differs so much from the original topic of this thread that I would suggest starting a new thread.

Merci, Jack Harich