Our previous two posts introduced and explored economic control. The first post, dated March 18th 2013, was an introduction to this principle. The second post, dated April 4th 2013, illustrated in more detail using the healthcare industry. As the third and final part of this series, we’ll explore how economic control can serve any business, and any industry.
Economic control charts find their own level, meaning they find the stable process (“level playing field”) within the instabilities (spikes etc.). The “tramlines” reveal the inherent stable process. There are strict, objective rules in the mathematics for how this is done. Once set up, no changes are made to the limits (“tramlines”) until the process improves, as determined by the chart. New points are added each day, week or month, without changing the limits. Modern software encourages the limits to be recalculated each time but this is wrong (like moving goalposts around).
The landmark text that introduced economic control in 1931, in a brilliant 300+ page development, used physics, mathematics, statistics and economics. Suddenly (on page 304) appears the seemingly simple formula: process average ± 3 standard deviations. This became known as the control chart. More correctly, it’s an economic control chart [1].
The important element of this deceptively simple formula (used in most industries for nearly 100 years now) is that the control limits are set economically (so that management actions will save and make most money). It is often said that control charts have a false-alarm rate of 0.27% but this is irrelevant. Nowhere in the original text does that number appear. The whole point is the limits are set economically. Further, the 0.27% is inexact in real processes, so it’s a red herring at best.
First impressions of the 3-sigma limits are often that they’re really wide and we should have a tighter standard. In fact they’re not “wide”. They are what they are. This error arises from missing the economic aspect.
It’s also often said that variations inside limits are random. They are not. All fluctuations (large or small) are caused by something(s). Now, the data within limits do often follow random patterns. So when someone says: “What if that point outside limits is just a false alarm?” the answer is there is no such thing. Since all variations have causes, a large one is by definition worth something. The hand-wringing comes because it is thought it may be pure chance. It isn’t. Now the causes that conspired to create a large number may have fallen into a perfect storm by chance, but they’re still real and there’s money to be made. This is a little like serendipity. No-one minds that we stumble into breakthroughs serendipitously. Same here.
The way to see this is to think of variations inside limits as having many causes, unknown. These will be impossible or expensive to figure out. So we use statistical design to do that. Variations outside limits will be economic to figure out and fix, ignore (e.g. month effects) or bake in (if good). In other words their cause(s) are easily found out and exploited for improvement.
Points outside limits (and/or a few other patterns) render the process unstable and are easy to figure out, then remove (or bake in if it’s a good spike), to make the process stable (standardized).
Here we’re not naïve that all processes can be made stable (standardized), such as if weather is involved in outside plant. Then we use standard workarounds. In general though, economic control gives a method to assess and standardize processes and their measurement systems.
If an improvement has just been implemented and a single point then crosses the limit (in the improvement direction) then it is a very big deal. This is where that crude, approximate rule of thumb of 0.27% comes in handy. If we’re plotting data monthly then we’d only expect to cross that limit roughly once every 62 years. So this is like a 62-year flood. (100/(0.27/2) x 12 = 61.7 years.) That information alone adds management fuel to the emerging improvement.
There is a deeper aspect to economic control. All of statistics (in the main techniques statisticians call parametric) is based on distribution theory. Averages, standard deviations, significance tests, regression etc. all are based on this. Certain mathematical requirements precede use of all these things. When we take courses in statistics we see those but read them much like the small-print in a legal contract. Most fundamental in that small-print is that the process be stable. If it is not, distribution theory breaks down. So, for example, even a simple pre-post test to see if something we did improved the process will be wrong if we ignore instabilities. “Wrong” is not strong enough a word – a better one would be arbitrary. Economic control is the only way to adjust so that we find the correct answer. This is profound for competitive advantage in a business.
This surprising comment that all of statistics* breaks down on unstable processes, is in the literature under the heading of analytic statistics. It is not well known even among qualified statisticians.
Industrial processes are almost always unstable, so statistics will not work unless the adjustments are made. The adjustments are simple, fast and follow rigorous rules that cannot be bent.
Anyone can do economic control charts. They are self-teaching devices where experience increases skill in usage. Common sense and a copy of the formulae/rules is enough to get started. Access to an expert speeds this learning curve and avoids stumbling through common mistakes.
This all happens very fast. Economic control does not slow the business but speeds it. If a business were able to keep up with economic control it would be moving fast indeed. With a little practice, firms will find the economic control charts ready and waiting each morning. Of course they are only used on a few big things that make most money fastest.
* Statisticians use a code here: i.i.d. ~ independent, identically distributed
REFERENCES:
1. Shewhart, Walter, A. Economic Control of Quality… Van Nostrand (1931)
Monday, April 15, 2013
Wednesday, April 3, 2013
04 . 02 . 13 | Healthcare Economic Control
Building on our previous post which introduced the principles of economic control, this set of charts shows the work that preceded the healthcare case described in Case Studies: Healthcare | Health at Home, Not Hospital.
The first of these shows how measurement error (meaning “noise” not “mistake”) was initially unstable but was quickly fixed by removing non-applicable cases. The chart’s spike revealed this flaw in the tracking systems and found it also throughout all the data (not just the spike). That stabilized measurement error (i.e. the data all then fell inside tighter tramlines). Calls for measurement perfection were advised against since it would have been uneconomic (i.e. a severe drain on resources) and often impossible. This economic aspect is one of the most valuable features of economic control.
The second chart shows that the nurses (and the patients they cared for) in the largest of 3 simultaneous statistical designs, are homogeneous in terms of chronic health events, similar to the retail stores in the first post. So this assured a “level playing field” for the study that followed.
The next pair of charts are a hybrid. On the top chart, the gray areas are measurement error (i.e. noise) and the outer limits are the process (i.e. chronic events). This gray area offers a simple way to always know measurement error will not get in the way. It is clear the gray portion is not obscuring the chronic events month by month. The gray is about a quarter of distance between the outer dotted lines. Since statisticians use squared (not linear) distances, only about a quarter2 = 1/16= 6 ¼% of the process is really obscured. A good rule of thumb here is 25% tops, but 10-15% preferred.
This is surprising, given the visual impression, so the square law clarifies.
The lower chart is also measurement error but looks at precision (i.e. how much measurement error varies). The top chart was measurement accuracy (i.e. how close to the true mark it gets and how well it discriminates process shifts).
This simple hybrid method allows processes to be improved and all questions about measurement error answered (really pre-empted) in real time, in the months ahead.
This case produced about a third improvement (against experimental prediction of a quarter) in a 3-month study plus a couple of months to solve implementation problems. The implementation population was double the size of the random sample used in the study.
The first of these shows how measurement error (meaning “noise” not “mistake”) was initially unstable but was quickly fixed by removing non-applicable cases. The chart’s spike revealed this flaw in the tracking systems and found it also throughout all the data (not just the spike). That stabilized measurement error (i.e. the data all then fell inside tighter tramlines). Calls for measurement perfection were advised against since it would have been uneconomic (i.e. a severe drain on resources) and often impossible. This economic aspect is one of the most valuable features of economic control.
The next pair of charts are a hybrid. On the top chart, the gray areas are measurement error (i.e. noise) and the outer limits are the process (i.e. chronic events). This gray area offers a simple way to always know measurement error will not get in the way. It is clear the gray portion is not obscuring the chronic events month by month. The gray is about a quarter of distance between the outer dotted lines. Since statisticians use squared (not linear) distances, only about a quarter2 = 1/16= 6 ¼% of the process is really obscured. A good rule of thumb here is 25% tops, but 10-15% preferred.
This is surprising, given the visual impression, so the square law clarifies. The lower chart is also measurement error but looks at precision (i.e. how much measurement error varies). The top chart was measurement accuracy (i.e. how close to the true mark it gets and how well it discriminates process shifts).
This simple hybrid method allows processes to be improved and all questions about measurement error answered (really pre-empted) in real time, in the months ahead.
This case produced about a third improvement (against experimental prediction of a quarter) in a 3-month study plus a couple of months to solve implementation problems. The implementation population was double the size of the random sample used in the study.
Tuesday, March 19, 2013
03 . 18 . 13 | Principles of Economic Control
Nobi’s large statistical designs (to test which of 20+ changes improve a business quickly, and quantify by how much) are well understood. Less well known is the novel way we integrate economic control.
Economic control charts just plot the measurement we’re improving, then place limits (that look like tramlines) showing the extremes the process will normally confine to. Some examples will speak best.
The chart above ensured a “level playing field” when randomizing retail stores to a statistical design for testing 11 changes in stores and marketing, bringing a 9.8% “comps” increase from 2 of the 11 (see also homepage Case Studies: Retail Sales | Innovation During a Recession).
The next chart shows an inventory reduction project that started when it did.
Usually there are gaps to close in initial implementation, which typically take 1-2 meetings using the scientific method and economic control. Although little or no adherence monitoring is needed in general, it is used more to close implementation gaps, in order to know what’s been going on at all times and places. A small, random sample of transactions is inspected for the changes being implemented. The work is completed with people in the trenches. This is quite subtle in the details but with good business advantage.
Economic control is essential when using statistical designs for rapid improvement. It’s simple to use, though deceptively clever. It is important to managers in guiding economic decisions, and to scientists in providing objectivity in unstable business data (where the rules of statistics break down).
Economic control charts just plot the measurement we’re improving, then place limits (that look like tramlines) showing the extremes the process will normally confine to. Some examples will speak best.
The chart above ensured a “level playing field” when randomizing retail stores to a statistical design for testing 11 changes in stores and marketing, bringing a 9.8% “comps” increase from 2 of the 11 (see also homepage Case Studies: Retail Sales | Innovation During a Recession).
The next chart shows an inventory reduction project that started when it did.
Usually there are gaps to close in initial implementation, which typically take 1-2 meetings using the scientific method and economic control. Although little or no adherence monitoring is needed in general, it is used more to close implementation gaps, in order to know what’s been going on at all times and places. A small, random sample of transactions is inspected for the changes being implemented. The work is completed with people in the trenches. This is quite subtle in the details but with good business advantage.
Economic control is essential when using statistical designs for rapid improvement. It’s simple to use, though deceptively clever. It is important to managers in guiding economic decisions, and to scientists in providing objectivity in unstable business data (where the rules of statistics break down).
Monday, November 26, 2012
11 . 29 . 12 | Continued Push Into Healthcare
2012 cases in healthcare numbered roughly a dozen in:
In most cases, widespread acceptance has required backing out precursors (such as HCC score) analytically, to show the findings do not change (and instead strengthen). No surprise here since the theory has been around since the 1920s but remains notoriously hard to grasp. These more practical exercises of showing users in their own language have been well received.
- Reducing hospitalizations
- Reducing re-admits
- Reducing exacerbations for disabled populations
- Increasing engagement in CM/DM and in wellness programs
- Improving treatments
- Streamlining utilization management
- Predictive modeling for CM/DM selection
In most cases, widespread acceptance has required backing out precursors (such as HCC score) analytically, to show the findings do not change (and instead strengthen). No surprise here since the theory has been around since the 1920s but remains notoriously hard to grasp. These more practical exercises of showing users in their own language have been well received.
Monday, November 19, 2012
11 . 19 . 12 | The Control Book
An ingenious management tool as client efforts expand to several statistical designs was suggested to us a while ago by a veteran CEO. It is a seemingly simple report, with one page for each project showing the improvement in the main measurement, annotated for actions taken.
In assembling the first edition for monthly update, the organization reaches agreement on the operational definition of the measurement. Discussions that take place after this discussion are more focused. The Control Book allows executive teams to manage the improvement effort in a short meeting every month. Financial information is also footnoted so that the overall ROI is clear at a glance every month. The idea is simple. Its execution is deceptively difficult but consumes little time. The tool also makes implementation straightforward whereas it can otherwise be elusive.
The reasons this simple tool can be difficult to introduce differ by organization and are best discovered by each client. Once in place and with a monthly forum for managing it, the tool is remarkably powerful and much liked. It is harder to accomplish than the statistical designs, mainly since sustained implementation is the hardest part of improvement.
The Control Book is the most effective way to manage implementation, short and long term.
In assembling the first edition for monthly update, the organization reaches agreement on the operational definition of the measurement. Discussions that take place after this discussion are more focused. The Control Book allows executive teams to manage the improvement effort in a short meeting every month. Financial information is also footnoted so that the overall ROI is clear at a glance every month. The idea is simple. Its execution is deceptively difficult but consumes little time. The tool also makes implementation straightforward whereas it can otherwise be elusive.
The reasons this simple tool can be difficult to introduce differ by organization and are best discovered by each client. Once in place and with a monthly forum for managing it, the tool is remarkably powerful and much liked. It is harder to accomplish than the statistical designs, mainly since sustained implementation is the hardest part of improvement.
The Control Book is the most effective way to manage implementation, short and long term.
Wednesday, August 1, 2012
08 . 01 . 12 | Nobi Presents At ISRN Event
Nobi Speaks at the 2012 Summer Institutes on Evidence-Based
Quality Improvement
Following the Network News article (3/20/12) Nobi was
invited to speak at the 2012 Summer Institutes on Evidence-Based Quality
Improvement Seminar held July 17th to the 21st in San
Antonio, TX.
Tracing the origins of the scientific method through the
1923 discovery of statistical design to pioneering work in application to CM/DM
generally, Nobi’s Kieron Dey provided case studies and fragments to illustrate
a new but long proven method for practical innovation in healthcare generally.
Aspects of management, science, statistics and economics
were summarized together with what will cause mainstream adoption from an
understanding of its prevention to date.
Insight into organizational dynamics and human nature that
are an essential aspect of the scientific method resonated with the 160
clinicians, nurses, physicians, administrators and graduate students in
attendance. These elements revealed how implementation (the hardest part) has
been made straightforward in delivering and sustaining results predicted by
studies. Central to the rapid cycle time for innovation the talk emphasized the
ability of orthogonal design to evaluate 20+ changes to treatments or clinical
models simultaneously without increasing sample size over RCT norms. The upshot
of thereby evaluating over a million potential treatment variants was
explained.
Advanced mathematical constructs employed in orthogonal
design were simplified. For example the meaning of orthogonality itself (the
device which allows cause and effect of 20+ interventions at once to be
established) was demonstrated visually with a newspaper. The surprising fact
that false alarm rate decreases when testing 20+ interventions was explained and
proven with actual data.
Further speeding safe research, the reason why single-shot
large orthogonal studies require no refining or validation testing was
demonstrated by case example (as is known from a full appreciation of the
subtending theory).
Interest continued by correspondence with attendees after
the conference including “I thought “yawn” ..but, found [orthogonal design
coupled with economic control]...the most interesting and provocative topics at
the entire conference.”
Monday, July 16, 2012
07 . 16 . 12 | Disability Study
Nobi is advising on statistical design in a study that began in
2010 to improve health thereby reducing hospitalizations for disabled people. Three
health insurance plans are participating, one of which is for disabilities due
to severe persistent mental illness (SPMI), by examining the effects of sets of
11 interventions differing by plan somewhat.
The study employs large orthogonal designs with a few dozen nurses (care managers) and the thousands of people they provide telephonic care to. Health outcomes, preventive measures and acute admits are being tracked for analysis.
Typically, these designs have reduced hospitalizations from 5-20% with notable findings such as novel ways to reduce falls. Because each hospitalization costs an average of $10,000 significant savings in the $1-10 million range per study implemented are on file.
Study results are expected in late 2012. Implementation will then follow using the interventions found helpful. Interventions explored range from medications to counseling and screening efforts for the SPMI project, to educating patients on fall risks and changes to the care model for the chronically physically ill patients.
The study employs large orthogonal designs with a few dozen nurses (care managers) and the thousands of people they provide telephonic care to. Health outcomes, preventive measures and acute admits are being tracked for analysis.
Typically, these designs have reduced hospitalizations from 5-20% with notable findings such as novel ways to reduce falls. Because each hospitalization costs an average of $10,000 significant savings in the $1-10 million range per study implemented are on file.
Study results are expected in late 2012. Implementation will then follow using the interventions found helpful. Interventions explored range from medications to counseling and screening efforts for the SPMI project, to educating patients on fall risks and changes to the care model for the chronically physically ill patients.
More Information Contact:
Subscribe to:
Posts (Atom)