Excel “Pull” Function: Creating dynamic links to closed workbooks

I was working on a project this week that required aggregation of data from many, many different Excel workbooks (and various tabs within those workbooks) into one unified dashboard. Now, this can be accomplished with traditional linking, but anyone who has tried to link more than a few workbooks together will tell you this is a pain to maintain and often not very reliable.

One useful function for creating dynamic links within the same workbook is INDIRECT. Instead of hardcoding cell and worksheet references into a formula, INDIRECT allows pieces of a formula to be dynamically updated using variables. Sounds pretty good right? Well, INDIRECT has an Achilles’ heel. It only works if you are linking within the same workbook.

Need dynamic links to a closed workbook? Bill Gates says, “You’re out of luck.”

Fortunately, after searching around, I found an old Usenet post from 2004 by a fellow named Harlan Grove. He wrote an Excel function in VB called PULL which essentially fixes INDIRECT’s fatal flaw and allows the user to create links to closed workbooks. Just copy the code below into a Module within the workbook of your choice. Then you can use the PULL function just like any other.

Example:
A1 =C:\work\
B1 = data\
C1 = [ClosedBook.xls]Sheet1′!$A1
E1 = PULL(“‘”&A1&B1&C1)

'----- begin VBA -----
Function pull(xref As String) As Variant
'inspired by Bob Phillips and Laurent Longre
'but written by Harlan Grove
'-----------------------------------------------------------------
'Copyright (c) 2003 Harlan Grove.
'
'This code is free software; you can redistribute it and/or modify
'it under the terms of the GNU General Public License as published
'by the Free Software Foundation; either version 2 of the License,
'or (at your option) any later version.
'-----------------------------------------------------------------
'2004-05-30
'still more fixes, this time to address apparent differences between
'XL8/97 and later versions. Specifically, fixed the InStrRev call,
'which is fubar in later versions and was using my own hacked version
'under XL8/97 which was using the wrong argument syntax. Also either
'XL8/97 didn't choke on CStr(pull) called when pull referred to an
'array while later versions do, or I never tested the 2004-03-25 fix
'against multiple cell references.
'-----------------------------------------------------------------

'2004-05-28
'fixed the previous fix - replaced all instances of 'expr' with 'xref'
'also now checking for initial single quote in xref, and if found
'advancing past it to get the full pathname [dumb, really dumb!]
'-----------------------------------------------------------------
'2004-03-25
'revised to check if filename in xref exists - if it does, proceed;
'otherwise, return a #REF! error immediately - this avoids Excel
'displaying dialogs when the referenced file doesn't exist
'-----------------------------------------------------------------
Dim xlapp As Object, xlwb As Workbook
Dim b As String, r As Range, C As Range, n As Long
'** begin 2004-05-30 changes **

'** begin 2004-05-28 changes **
'** begin 2004-03-25 changes **
n = InStrRev(xref, "\")
If n > 0 Then
If Mid(xref, n, 2) = "\[" Then
b = Left(xref, n)
n = InStr(n + 2, xref, "]") - n - 2
If n > 0 Then b = b & Mid(xref, Len(b) + 2, n)
Else
n = InStrRev(Len(xref), xref, "!")
If n > 0 Then b = Left(xref, n - 1)
End If

'** key 2004-05-28 addition **
If Left(b, 1) = "'" Then b = Mid(b, 2)
On Error Resume Next
If n > 0 Then If Dir(b) = "" Then n = 0
Err.Clear
On Error GoTo 0
End If

If n <= 0 Then
pull = CVErr(xlErrRef)
Exit Function
End If
'** end 2004-03-25 changes **
'** end 2004-05-28 changes **
pull = Evaluate(xref)

'** key 2004-05-30 addition **
If IsArray(pull) Then Exit Function
'** end 2004-05-30 changes **

If CStr(pull) = CStr(CVErr(xlErrRef)) Then
On Error GoTo CleanUp 'immediate clean-up at this point

Set xlapp = CreateObject("Excel.Application")
Set xlwb = xlapp.Workbooks.Add 'needed by .ExecuteExcel4Macro

On Error Resume Next 'now clean-up can wait

n = InStr(InStr(1, xref, "]") + 1, xref, "!")
b = Mid(xref, 1, n)

Set r = xlwb.Sheets(1).Range(Mid(xref, n + 1))

If r Is Nothing Then
pull = xlapp.ExecuteExcel4Macro(xref)

Else
For Each C In r
C.Value = xlapp.ExecuteExcel4Macro(b & C.Address(1, 1, xlR1C1))
Next C

pull = r.Value

End If

CleanUp:
If Not xlwb Is Nothing Then xlwb.Close 0
If Not xlapp Is Nothing Then xlapp.Quit
Set xlapp = Nothing

End If

End Function
'----- end VBA -----

The problem with college

Much of the discussion surrounding the Occupy Wall Street protest movement has turned towards the rising costs of higher education. This is likely because many of the protesters are young, college educated and saddled with enormous federal debt.

The Los Angeles Times had a piece this weekend discussing the issue. It was good article, but I’ve noticed something missing in all the coverage. Something that colleges, students and even parents don’t seem to want to talk about.

Not all college degrees are created equal.

Have you noticed that almost all of the protesters interviewed have undefined liberal arts degrees? Where are the chemical engineers or the nurses or the accountants? Working, that’s where. The stark reality is there are more college majors than fields with actual paying jobs.

I never heard this when I was 18. Like most, I was led to believe that just having a college degree was enough. But it’s not. I had to figure it out on my own through a series of missteps and major changes. However, I was fortunate to see the reality before it was too late.

I’m not saying it’s the protesters’ fault.

The higher education system is primarily run by people with liberal arts degrees so it’s natural they give high regard to these fields. However, colleges are doing students a disservice by leading them to believe that simply by graduating with any degree, employers will be knocking at their door. Employers are looking for marketable skills. They don’t care what you know; they care what you can do.

Colleges need to do a better job of informing students about job prospects in their chosen fields. “Oh, you chose electrical engineering. The next four years are going to stink, but you can afford to live on your own when you graduate. Oh, you chose Irish literature. Hope your parents have a spare bedroom.” It sounds harsh, but I wish someone had been that frank with me as an 18 year old. It would have saved me from stumbling around until I found a field I liked AND actually paid a salary.

A friend recently shared this career planning Venn Diagram. It’s funny, but also wise in its simplicity. I think the protesters angry about their student loan debt are well-intentioned but misinformed. As students, we are are told to follow our passions and find our calling, but we are told to ignore money. As these protesters are finding out, that is bad advice. College is an investment, with returns that can be measured. It’s time we started doing a better job informing students about life after graduation.

The Tulip Bubble of 1636

With all the news coverage on the recent “Great Recession”, you might think bursts of speculation and the subsequent collapse in prices are unique to the modern age. They’re not.

During the latter half of 1636 Tulip prices in Holland skyrocketed. By the peak of the bubble in February 1637, a single tulip bulb was selling for more than the average laborer made in an entire year. There was a strong belief that tulip prices would only continue to rise. Then, just as suddenly, the tulip market collapsed. Those who had bought in at the top were left destitute wondering where their fortunes had gone.

Sound familiar?

As much as economists would like to explain markets with clean, mathematical formulas, there is a human element that can never be fully predicted. The cycle of booms and busts is linked to human psychology as much as actual macroeconomic conditions. The older I get the more I think the efficient-market hypothesis is a pipe dream and real markets are far more complex and unpredictable than its creators would have envisioned.

The Batman Equation

There is a geometric equation going around the internet that supposedly generates the Batman symbol. Best I can tell, it started at this site, and blew up from there.

A user at Stack Exchange took it upon himself to test whether it worked. He mapped out the various functions and looks like it does. Pretty cool.

Sino-Forest, Short Sellers Allege Ponzi Scheme

There is an interesting alleged accounting fraud unfolding in China. Sino-Forest Corporation, a Chinese company in the business of selling timber, has been accused by short sellers of being a massive Ponzi scheme.

Like Enron, they seem to be running their business through a complex system of subsidiaries in order to obfuscate transactions.

One of the funnier quotes from the short sellers’ report is below:

Despite TRE’s opacity on the revenue side, we have overwhelming evidence that the $231.1 million in Yunnan province timber TRE claimed to sell is largely fabricated. Such amount exceeds TRE’s real timber holdings in Yunnan province. It exceeds the applicable harvesting quotas by six times. Transporting the harvested logs would have required over 50,000 trucks driving on two-lane roads winding through the mountains from this remote region, which is far beyond belief (and likely road capacity).

In other words, the amount of timber they claimed to sell in this one transaction not only exceeded their holdings, but the provinces’ road capacity. Oops. They should have sandbagged a little.

Sino-Forest is fighting back today against the allegations. It will interesting to watch this story unfold. Thanks to @bargles for the tweet that piqued my interest.

R for SAS and SPSS Users, A Review

If you’ve followed me for any length of time, you’ll know that I’m bullish on good data analysis. I am not a programmer by training but, through a variety of projects, I’ve found a solid understanding of data manipulation has served me well in my work as an accountant. My primary tool for 4 years has been SAS. It is an awesome, easy to learn language that has allowed me to look like a hero on numerous projects. However, starting next week, I’m moving to a company that doesn’t offer SAS as an option.

I didn’t want to lose the edge that a good analytics package offers, so I started looking around for an open source alternative. That’s when I found “R”.

Like SAS, R is a statistics package. It’s designed for statisticians to perform t-tests, calculate standard deviations, and all the other black-box stuff that gives auditors the warm-fuzzies. However, like SAS, it also has strong data manipulation abilities. There is even a package called sqldf, that allows users to write SQL statements in the R command line.

Not surprisingly, because the name “R” is so generic, it is nearly impossible to find good tutorials online through tradition searching. So, I decided it was time to buy a book. Fortunately, Robert Muenchen from University of Tenn, has written a book for people exactly like me: R for SAS and SPSS Users (ISBN-10: 0387094172).

Unlike a lot of programming books, Muenchen’s is packed with examples and real world applications. He walks readers through a particular scenario, then shows how to accomplish with SAS, SPSS and R. It’s like a Rosetta Stone for users needing to move from one language to the other. Overall a great help and I’d highly recommend it.

Bitcoin: A Money Launderer’s Dream Come True

I first heard about Bitcoin when Steve Gibson did a podcast on it a few months ago. Essentially, it’s a medium of exchange just like any other currency. The major differences are 1. “coins” are simply bits in the system with no physical reality (actually, most of the US money supply has no physical reality either, but that’s a post for another day), 2. transfers can’t be traced, and 3. there is little to no transaction cost.

Jason Calacanis, a well known tech entrepreneur, recently made the following statements about the open source project.

1. Bitcoin is a technologically sound project.
2. Bitcoin is unstoppable without end-user prosecution.
3. Bitcoin is the most dangerous open-source project ever created.
4. Bitcoin may be the most dangerous technological project since the internet itself.
5. Bitcoin is a political statement by technotarians (technological libertarians).
6. Bitcoins will change the world unless governments ban them with harsh penalties.

There are currently a number of exchanges where it is possible to buy and sell bitcoins using real-world currencies. It’s a money launderer’s dream come true; just like anything else using encryption, it’s math, it exists and there’s not much authorities will be able to do about it.