Business Analytics - The Right Tools For The Job

Whether your analytic tool of choice is Excel or R or Access or SQL Server or ... whatever,  if you've worked a reasonable range of analytic problems I will guarantee that at some point you have tried to make your preferred tool do a job it is not intended for or that it is ill-suited for.  The end result is an error-prone, maintenance nightmare and there is a better way.

We all know when we are pushing it too far - the system starts to "creak".  Symptoms vary but include at least some of the following:

• Model calculations run slowly (if they complete at all).
• Model calculations give, apparently, inconsistent results.
• Making minor changes becomes a major headache
• Errors ("bugs") are routine.  When you need to do a demo, you pray first.
• You have learned to avoid certain operations because the likelihood of success is slim.
• If you must come back to your model/application after 6 months to update it, you feel physically ill.
• and perhaps most importantly, you are far from sure that using your model generates the right results.

This isn't just an academic issue or one of preference.  Poor models can cost real money in lost opportunity or bad decisions that get implemented.  (See this post for a few examples)

So why do we do this ?  I think it's because for many analysts their tool box looks like this.

Why is their toolbox so empty?  In some cases, corporate IT restrictions may make it very difficult to  acquire/install the right tools;  I've been there, it's a real challenge.

For many people though, it just feels easier to take a tool they know well and try and "make it work" than to learn something new.  That's rather like thinking  "I need to chop down a tree... I'll sharpen my hammer" 

Last week, I gave you my nominations for "The Worst use of Excel Ever!"  I could easily cite similar abuses for other tools and over the next few months I will.   

This blog post is the first in a series around using "The Right Tool For The Job".   I'm going to encourage you to add a few more tools to your analytic tool-box and learn to wield them effectively. (Or, alternatively, to at least recognize when you need someone who can do that for you.)  Do you need: 

• A application programming environment ? 
• A database (of varying capability, Access, SQL or perhaps a newer column storage databases) ?
• A reporting tool ?
• An optimization modeling language ?
• A visualization tool ?
• A statistics or data-mining package ?
• A simulation package ?

Which tools do you think are the most mis-used?  What skills/tools should a business analyst consider adding to their toolbox ?