Evidence-Based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals

UPDATE: 10/1/2115 years later- Still the BEST BOOK on security trading you can own.IMAGINE—That in one hand you held a bag full of returns from YOUR technical trading system........ In the other hand you held a bag full of RANDOM returns from the market over the same period.NOW IMAGINE—That there was a 500 year old scientific tool which would allow you to COMPARE each bag of returns to determine if YOUR trading system actually works better (makes more $$$) than a system driven by pure LUCK.The tool actually TELLS YOU IF YOU ARE “WINNING” SIMPLY BECAUSE OF LUCK or YOUR TRADING SYSTEM IS ADDING ANY VALUE, ANY VALUE AT ALL, TO YOUR TRADES.Aronson’s book explains, in thorough detail, THAT scientific method.###########. ORIGINAL REVIEW OF 12 YEARS AGO ############What you take-away from a reading of this book really depends on where you're coming from.For STATITICIANS with an interest in trading markets--You'll likely walk away with the feeling: "Yeah, that's what I've been thinking for years, nice to see someone took the time to debunk the TA myth."For TECHNICIANS (traders) with an interest in statistics--You'll likely walk away thinking "You gotta be kidding. There are a hundred good books which can show you how to use TA to make money. This book sucks."Aronson suggests that the truth does NOT lie in between-- He is firmly in the camp of the Statistician.But a close reading of this powerful book does not "close the door" on profitable TA, it simply confirms what every first-year MBA learns:"No OBJECTIVE black-box trading strategy CONSISTENTLY beats the market AVERAGES over the LONG TERM."But hard-core TA fans take heart. You will find something interesting in this book also. I'm certain Aronson would agree with the following:"Sure you can add COMPLEX rules to the black-box, and PERIODICALLY find runs of profitability with TA. If EXCESS returns exist only in the SHORT TERM, hey, that's good enough for me."For those not willing to take the time to digest the painstakingly presented statistical concepts, there will be little value in this book-- This is a serious study with lots of math. Its not hard math. But math best understood after fully internalizing a college level stats class.Even for those with a Stats or Econometrics degree statistics are tough--both computing and interpreting statistical data takes a little work. To complicate the issue, market-related statistics are fraught with half-truths, mind-bending math, and wall-street lore.This book goes a long way to put bogus TA lore to rest by presenting a clear, scientifically sound procedure to test Technical rules.For those seriously considering buying this book let me suggest that you find Aronson's website [...] and download and read Dr. Timothy Masters' .pdf "Monte-Carlo Evaluation of Trading Systems." The document, both in tone, and sophistication, mirrors Aronson's book.If you like Masters' 43 page doc--you will love Aronson's 500+ page book.The ReviewI break the book up into four parts, each with various degrees of usefulness depending on your background--ie: Technician or Statistician. Below, I'll simply give what I thought was the "money-quote" from each part, plus a couple of observations for those considering buying the book.***** Part 1: Chapter 1 - 31 pagesObjective Rules and Their Evaluation"The isolated fact that a rule earned 10 percent rate of return in a back test is meaningless. If many other rules earned over 30 percent on the same data, 10 percent would indicate inferiority, whereas if all other rules were barely profitable, 10 percent might indicate superiority." - Aronson, page 23Constructing Rules - Intro to bi-modal rule construction and trigger thresholdsData Transformation - Nice review of position-bias, log-differences and testing biasesBenchmarking Rules - Good review of why "Relative-Benchmarking" is importantBeating the Benchmark - Why a profitable back test is not conclusive proof of good rule***** Part 2: Chapters 2-3 - 130 pagesThe Illusory Validity of Subjective Technical AnalysisThe Scientific Method and Technical Analysis"Statistician Harry Roberts said that technical analysts fall victim to illusion of patters and trends for two possible reasons. First, the "usual method of graphing stock prices gives a picture of successive (price) levels rather than of price changes and levels can give an artificial appearance of pattern or trend. Second, chance behavior itself produces patterns that invite spurious interpretations""-- Aronson, page 83The Eye Deceives - Charting a random process and the representativeness heuristicSubjective vs. Objective -- Why its important to be able to "hard-code" a TA ruleThe Role of Logic - Why "Falsification" is more important than "Affirmation" in TAAstrology vs Astronomy - Pushing the TA boundaries from pseudo- to science***** Part 3: Chapter 4-7 - 230 pagesStatistical AnalysisHypothesis Tests and Confidence IntervalsData-Mining Bias: The Fool's Gold of Objective TATheories of Non-Random Price Motion"Informal data analysis is simply not up to the task of extracting valid knowledge from financial markets. The data blossoms with illusionary patterns whereas valid patterns are veiled by noise and complexity. Rigorous statistical analysis is far better suited to this difficult task." - Aronson, page 172Hypothesis Testing--Good review of probability and statistical inferenceThe Traditional Solution - Actually put your college-level stats knowledge to useThe Monte-Carlo Solution - Putting computer randomization and re-sampling to workThe Data-Mining Problem -- Why traditional MC solutions don't workInefficient Markets - How, where and why profitable TA rules should STILL exist***** Part 4: Chapter 8-9 - 100 pagesCase Study of Rule Data Mining for the S&P 500Case Study Results and the Future of TA"Few rule studies in popular TA apply significance tests of any sort. Thus, they do not address the possibility that rule profits may be due to ordinary sampling error. This is a serious omission, which is easily corrected by applying ordinary hypothesis tests." - Aronson. page 449The Operators - Reviews: channel-break-outs, moving averages, channel-normalizationThe Indicators -- Reviews: price, volume, breadth, spreads, yieldsThe Rules - Reviews: trends, inverse trends, reversions, divergenceThe Results - Analysis of why 0 of the 6,402 tested rules produced no significant resultsThe Bottom LineAronson's book reminds me of that masked-magician on TV who has given away the secrets to all the best stage illusions.Novice magicians and apprentice conjurers will undoubtedly be "pissed-off."But true professionals are liberated.The best in the field can focus on new and potentially MORE exciting illusions--not the same old tricks.

David Aronson's Evidence Based Technical Analysis ("EBTA") is a fantastic book, and one which our industry has sorely needed. It is a "How to Do Research" book that details the scientific method with regard to the markets. Everyone in the field should both read the book and practice what it preaches. But that won't happen, which is both bad news and good news. The bad news is that the vast majority of market traders who do not practice what the book preaches will lose money. The good news is that those who do will most certainly prosper. As the numbers of the former outnumber those of the latter, the few will earn a lot from the many.The long (over 100 pages) psychology "preface" is extremely important to Aronson's body of work. I found it hugely interesting, but fear that others may not, or worse. In fact, the psychology preface itself indicates that this work will be reviled (my words) by the multitude. People do not like their sacred cows criticized.The problem is that most market practitioners use methods with little or no scientific basis. Even if shown evidence of faulty logic, people continue to believe its validity. This is also true in the medical profession as Aronson illustrated and which scared the daylights out of me.For anything to be scientifically testable, it must be possible to prove it wrong. However, many of the technical analysis disciplines cannot be defined. Thus they cannot be disproved. Consequently they have no scientific validity. They may have some anecdotal importance, but true science is lacking.Let us say that one of the market gurus espouses that when the chart of XYZ resembles "Pattern A", the stock is destined to rally. To test that we have to define Pattern A and we have to define "rally", and we should provide some time parameters in which to work or fail. The trouble is that the guru cannot define any of that. But the guru still believes in his work, and all of the investors who pay monthly fees for his expertise believe it also. Anyone who criticizes the guru or the validity of Pattern A is looking to get flamed.EBTA preaches that technical analysis research should be conducted like quantitative analysis research. Those who treat TA as a casual discipline will get casual results. The book is not an easy read, but it is an easier and much more interesting read than the "bible" of the CFA community, Quantitative Methods for Investment Analysis (DeFusco, et al.). I own both books and certainly consider EBTA more valuable than the CFA manual, worshipped by thousands. Don't expect to download all of Aronson's knowledge the first time - read it again. I did and learned more the second time through.Aronson is meticulous and provides "service after the sale". I had recently traded emails with him about an article he had cited. He was prompt to respond and discus the implications of our expanded research. I have the feeling that he is like this with everyone.In conclusion I have to say, that if you cannot do what EBTA preaches, at least get yourself a money manager who does.Bill RafterPresidentMathematical Investment Decisions, Inc.