How to Invest Systematically and Gain Alpha – Part III

Share This Post

By Vinesh Jha, CEO & founder of ExtractAlpha

In Part I and Part II, we talked about the advantages and pitfalls of systematic investment research. In this post, we’ll discuss some best practices we’ve picked up over the years and which we apply at ExtractAlpha.

Practice makes perfect

We all know how to get to Carnegie Hall, right? Practice, practice, practice. As a young music student, I learned the importance of repetition in gaining the necessary muscle memory needed to move beyond pure mechanics. After sufficient amounts of practice, a musician can start to think about phrasing, structure, and nuance rather than focusing on being able to play the notes.

It’s not so different with quant research. We have a set of analytical tools and through their repeated use, we are able to more fluidly answer deep research questions without expending as much time and energy on the mechanics, which in this case, involves a lot of rote data analysis. Data manipulation is still a big part of the job – when we get a new dataset, we need to line it up and get familiar with it, like a musician sight-reading a new piece for the first time – but over time and with repetition we get better at it.

Checks and balances

Having a fantastic teacher or mentor helps, of course … in this regard, I was fortunate as a musician, but not in quant, where I’m largely an autodidact. And having a handbook or checklist, whether it’s written down or part of your routine, is also helpful. Here are a few of the things we do as part of our research routine at ExtractAlpha.

We start each research project with a list of hypotheses and ideas. This can be a long list – anything that’s sort of reasonable and which pertains to the questions at hand. We’ll brainstorm these ideas over the course of many days, often logging the ideas in a task management system. Then it’s time for triage, by sorting to the top those ideas we definitely want to test, and for which we have the resources – usually this means we have or can acquire the relevant data. In the middle go the “maybes,” and then we’ll throw out ideas that seem a bit too outlandish or for which data isn’t available.

Next, we pay careful attention to the datasets we’re analyzing. An often overlooked step is opening up the data in a spreadsheet-like format and scrolling through each of the fields (assuming we are talking about structured, tabular data). We’ll often notice things like odd ways null values are stored, or oddly repeating values, when we take this simple but crucial step.

We also look at the distribution of the fields of interest. Which are well populated? Which categorical variables’ values are common versus sparse? Which ones have outliers we need to take into consideration, for example by Winsorization?

Be strict about in- and out-of-sample testing. Now that we’ve got an understanding of the look and feel of the data, we can begin our in-sample hypothesis testing. For the majority of our research process, we will be in sample – that is, we will spend our time analyzing a pre-specified testing dataset, and the remainder of the data will remain out of sample, for verification at the very end of the process. It’s vital to be completely strict about in and out of sample testing, as tempting as it might be to “peek” out of sample midway through your testing to ease your discomfort. Choosing an in-sample and out-of-sample split is also quite important. You want your out-of-sample period to be long enough to be meaningful, i.e., to encompass more than one type of market condition, but you also want your in-sample period to be representative of the current time period; things have changed in the last decade, as noted in the previous post. We have a few techniques we use to address these issues, and they are worth thinking about before beginning the research process.

Practical matters

Use the right tool for the job. Hypothesis testing doesn’t necessarily mean portfolio backtesting. Event studies are helpful, and sometimes we’re trying to predict something other than returns; for example, models which predict company fundamentals such as revenues can often end up being more robust.

When it comes to backtests, we do a lot of cross sectional tests. The goal is to come up with a score across a wide swath of stocks at each time period (say, day), and determine whether the high-scoring stocks outperform the low-scoring stocks. The advantage of this approach, versus say a trading simulation, is that we get a very rich set of information. We can learn how the factor performs across sectors, capitalization ranges, time, and other slices. Furthermore the results are not as sensitive to the particular choice of portfolio construction parameters, and they are more indicative of how this factor might add into an existing multifactor model.

We also look at a factor’s exposure to common risk factors, its turnover and autocorrelation, and a plot of the time series of its (in sample!) returns, before and after transaction cost assumptions. All of this together gives us a holistic view of the efficacy of a factor, or of a variant of a factor as we try applying different hypotheses. And as we test, it allows us to understand the sensitivity of the idea to various formulations – the more robust, the better, lest we find the next butter in Bangladesh.

Use a realistic universe construction. We see a lot of backtests from commercial vendors that include something like 5,000 stocks in the US. Even for a retail investor, a small trade can move the price of the 5,000th-most-liquid stock, and for institutional investors these stocks aren’t tradable at all, even at very long horizons. We use a universe designed to mimic what institutions look at, but we’re also always careful to split results by capitalization range, lest we find something that only has value among very small, illiquid stocks. 


Hopefully some of these pointers were helpful. With a lot of quant research, it doesn’t take huge amounts of resources to do it right. It’s more about being careful and finding the right tool for the job, and being aware of some common pitfalls. For many of us quants, there are few things more satisfying than finding value in a new dataset or anomaly; and it’s all the more satisfying if, having followed some of these best practices, we can have more confidence that we’re right.

And we’re here to help! If you’d like to learn more about the services we provide, please schedule a call

More To Explore

Proprietary Trading Firms in Vermont

Introduction As the financial landscape continues to evolve, proprietary trading firms have found an unexpected home in Vermont. This article explores the burgeoning presence of

Chloe Miao

Chloe joined ExtractAlpha in 2023. Prior to joining, she was an associate director at Value Search Asia Limited. She earned her Masters of Arts in Global Communications from the Chinese University of Hong Kong.

Matija Ratkovic

Matija is a specialist in software sales and customer success, bringing experience from various industries. His career, before sales, includes tech support, software development, and managerial roles. He earned his BSc and Specialist Degree in Electrical Engineering at the University of Montenegro.

Jack Kim

Jack joined ExtractAlpha in 2022. Previously, he spent 20+ years supporting pre- and after-sales activities to drive sales in the Asia Pacific market. He has worked in many different industries including, technology, financial services, and manufacturing, where he developed excellent customer relationship management skills. He received his Bachelor of Business in Operations Management from the University of Technology Sydney.

Perry Stupp

Perry brings more than 20 years of Enterprise Software development, sales and customer engagement experience focused on Fortune 1000 customers. Prior to joining ExtractAlpha as a Technical Consultant, Perry was the founder, President and Chief Customer Officer at Solution Labs Inc. a data analytics company that specialized in the analysis of very large-scale computing infrastructures in place at some of the largest corporate data centers in the world.

Perry Stupp

Perry brings more than 20 years of Enterprise Software development, sales and customer engagement experience focused on Fortune 1000 customers. Prior to joining ExtractAlpha as a Technical Consultant, Perry was the founder, President and Chief Customer Officer at Solution Labs Inc. a data analytics company that specialized in the analysis of very large-scale computing infrastructures in place at some of the largest corporate data centers in the world.

Janette Ho

Janette has 22+ years of leadership and management experience in FinTech and analytics sales and business development in the Asia Pacific region. In addition to expertise in quantitative models, she has worked on risk management, portfolio attribution, fund accounting, and custodian services. Janette is currently head of relationship management at Moody’s Analytics in the Asia-Pacific region, and was formerly Managing Director at State Street, head of sales for APAC Asset Management at Thomson Reuters, and head of Asia for StarMine. She is also a board member at Human Financial, a FinTech firm focused on the Australian superannuation industry.

Leigh Drogen

Leigh founded Estimize in 2011. Prior to Estimize, Leigh ran Surfview Capital, a New York based quantitative investment management firm trading medium frequency momentum strategies. He was also an early member of the team at StockTwits where he worked on product and business development.  Leigh is now the CEO of StarKiller Capital, an institutional investment management firm in the digital asset space.

Andrew Barry

Andrew is the CEO of Human Financial, a technology innovator that is pioneering consumer-led solutions for the superannuation industry. Andrew was previously CEO of Alpha Beta, a global quant hedge fund business. Prior to Alpha Beta he held senior roles in a number of hedge funds globally.

Natallia Brui

Natallia has 7+ years experience as an IT professional. She currently manages our Estimize platform. Natallia earned a BS in Computer & Information Science in Baruch College and BS in Economics from BSEU in Belarus. She has a background in finance, cybersecurity and data analytics.

June Cook

June has a background in B2B sales, market research, and analytics. She has 10 years of sales experience in healthcare, private equity M&A, and the tech industry. She holds a B.B.A. from Temple University and an M.S. in Management and Leadership from Western Governors University.

Steven Barrett

Steve worked as a trader at hedge funds and prop desks in Hong Kong and London for 15+ years. He also held roles in management consultancy, internal audit and business management. He holds a BA in Business Studies from Oxford Brookes University and an MBA from Hong Kong University of Science & Technology.

Jenny Zhou, PhD

Jenny joined ExtractAlpha in 2023. Prior to that, she worked as a quantitative researcher for Chorus, a hedge fund under AXA Investment Managers. Jenny received her PhD in finance from the University of Hong Kong in 2023. Her research covers ESG, natural language processing, and market microstructure. Jenny received her Bachelor degree in Finance from The Chinese University of Hong Kong in 2019. Her research has been published in the Journal of Financial Markets.

Kristen Gavazzi

Kristen joined ExtractAlpha in 2021 as a Sales Director. As a past employee of StarMine, Kristen has extensive experience in analyst performance analytics and helped to build out the sell-side solution, StarMine Monitor. She received her BS in Business Management from Cornell University.

Triloke Rajbhandary

Triloke has 10+ years experience in designing and developing software systems in the financial services industry. He joined ExtractAlpha in 2016. Prior to that, he worked as a senior software engineer at HSBC Global Technologies. He holds a Master of Applied Science degree from Ryerson University specializing in signal processing.

Jackie Cheng, PhD

Jackie joined ExtractAlpha in 2018 as a quantitative researcher. He received his PhD in the field of optoelectronic physics from The University of Hong Kong in 2017. He published 17 journal papers and holds a US patent, and has 500 citations with an h-index of 13. Prior to joining ExtractAlpha, he worked with a Shenzhen-based CTA researching trading strategies on Chinese futures. Jackie received his Bachelor’s degree in engineering from Zhejiang University in 2013.

Yunan Liu, PhD

Yunan joined ExtractAlpha in 2019 as a quantitative researcher. Prior to that, he worked as a research analyst at ICBC, covering the macro economy and the Asian bond market. Yunan received his PhD in Economics & Finance from The University of Hong Kong in 2018. His research fields cover Empirical Asset Pricing, Mergers & Acquisitions, and Intellectual Property. His research outputs have been presented at major conferences such as AFA, FMA and FMA (Asia). Yunan received his Masters degree in Operations Research from London School of Economics in 2013 and his Bachelor degree in International Business from Nottingham University in 2012.

Willett Bird, CFA

Prior to joining ExtractAlpha in 2022, Willett was a sales director for Vidrio Financial. Willett was based in Hong Kong for nearly two decades where he oversaw FIS Global’s Asset Management and Commercial Banking efforts. Willett worked at FactSet, where he built the Asian Portfolio and Quantitative Analytics team and oversaw FactSet’s Southeast Asian operations. Willett completed his undergraduate studies at Georgetown University and finished a joint degree MBA from the Northwestern Kellogg School and the Hong Kong University of Science and Technology in 2010. Willett also holds the Chartered Financial Analyst (CFA) designation.

Julie Craig

Julie Craig is a senior marketing executive with decades of experience marketing high tech, fintech, and financial services offerings. She joined ExtractAlpha in 2022. She was formerly with AlphaSense, where she led marketing at a startup now valued at $1.7B. Prior to that, she was with Interactive Data where she led marketing initiatives and a multi-million dollar budget for an award-winning product line for individual and institutional investors.

Jeff Geisenheimer

Jeff is the CFO and COO of ExtractAlpha and directs our financial, strategic, and general management operations. He previously held the role of CFO at Estimize and two publicly traded firms, Multex and Market Guide. Jeff also served as CFO at private-equity backed companies, including Coleman Research, Ford Models, Instant Information, and Moneyline Telerate. He’s also held roles as advisor, partner, and board member at Total Reliance, CreditRiskMonitor, Mochidoki, and Resurge.

Vinesh Jha

Vinesh founded ExtractAlpha in 2013 with the mission of bringing analytical rigor to the analysis and marketing of new datasets for the capital markets. Since ExtractAlpha’s merger with Estimize in early 2021, he has served as the CEO of both entities. From 1999 to 2005, Vinesh was the Director of Quantitative Research at StarMine in San Francisco, where he developed industry leading metrics of sell side analyst performance as well as successful commercial alpha signals and products based on analyst, fundamental, and other data sources. Subsequently, he developed systematic trading strategies for proprietary trading desks at Merrill Lynch and Morgan Stanley in New York. Most recently he was Executive Director at PDT Partners, a spinoff of Morgan Stanley’s premiere quant prop trading group, where in addition to research, he also applied his experience in the communication of complex quantitative concepts to investor relations. Vinesh holds an undergraduate degree from the University of Chicago and a graduate degree from the University of Cambridge, both in mathematics.

Subscribe to the ExtractAlpha monthly newsletter