Goizueta in the news

Some nice press for the undergraduate program here from the Fiske Guide to Colleges. 

Time series graph

There are lots of time series graphs available, but sometimes there are more issues that you want to include.  This is sometimes accomplished with bubble charts, but here's a nice example using geography as the "x-axis":

Graphs can say one thing, or another

First, I am not posting this to make any statement about Paul Krugman, or anything like that.  But when comparing time series data, choosing the starting point can be more imporant than it seems, as this blogger demonstrates.  You could imagine investing strategies being compared in a similar fashion.  One graph shows strategy A is better, but with the same data and a different start date, another graph would show that strategy B is better. 

Another issue is the choice of the other data sets - they can also be chosen to look "fair" but those time series contain special features that make the analysis less than fair.  For example, compare company X with specially chosen company Y, because of the special charges for Y in a particular quarter make some observation about X seem more true.

Simple comparisons are not always so simple.

A post about something I know little about

One of the questions I like to pose in my classes is "how do you know?"  Doing analysis of data and possible decision leads to lots of wild speculation, and this question sometimes brings us back to recognizing that widely held theories are actually quite tenuous.  One observation can bring down a whole theory, a "black swan" event.  I like to use examples from physics, even though I am not a physicist, because I find that students often think that we "know" stuff there.  Science is settled, even if other areas are not.

Here is an interesting (to me) article about measuring light and comparing the measurements to accepted physical theories.  At the end, something to think about:

There’s also a possibility that the explanation could be even more far-reaching, such as that the universe is not expanding and that the big bang theory is wrong.

How certain are you of the things you know?

Subprime mortgages and CDS

Here is an interesting interview with Michael Lewis about the financial meltdown.  Part of it has to do with how so many people made so many dumb mistakes.  I hope this makes people more skeptical of their decisions, and do a bit more analysis and think about how it could go wrong.

Costs and risks

The "Top Gear" guy had a nice rant in the Times Online about laws to mitigate risks.  Can we think of a way to connect the risk with the costs?  Suppose that we could eliminate a risk entirely if everyone just paid a small price.  Think polio, for example.  Well, then what about this risk, or that risk.  Surely the same solution applies, right?  As Jeremy points out, we all take our shoes off at the airport now, so the terrorists decided to use their underwear.  What will we have to do now, if risk elimination is the goal.

Or are there risks that we just have to live with?  Where do we draw the line?  Can we use probabilities and decision analysis to help make policies a little more bearable?

Rankings

Here is the most admired companies list for this year.  There are quite a few ways to set up the mechanics for the 'scoring' and there is not much in the way of confidence intervals in the article.  Would the article be more or less effective if there were statistical ties in the rankings?  How many ties would there be?

Here are the undergraduate business school rankings for 2010.

Christian statistics

Not statistics that act in a Christian way, but rather trying to use statistics to understand the state of Christianity.  Is it growing?  Shrinking?  Healthy?  Dying?  Turns out that describing statistics collected and making inferences is tricky, and the link describes some nice examples of where things can go wrong.  Thanks to Brad for the link.

When is an average or not an average?

First, I have not followed up on the reference in this Instapundit blog post, so I do not know whether this is really the way the British Met Office really computes average temperatures:

In fact, the Met still asserts we are in the midst of an unusually warm winter — as one of its staffers sniffily protested in an internet posting to a newspaper last week: “This will be the warmest winter in living memory, the data has already been recorded. For your information, we take the highest 15 readings between November and March and then produce an average. As November was a very seasonally warm month, then all the data will come from those readings.”

But suppose for a minute that an average for a whole season (year) really is fifteen days, and that the highest fifteen all (most) came from November. Would that really represent the average for the winter? Can you think about why they would not use every day's reading from November through March? Should they use just the high for the day, or the high and the low for each day? And where should the reading come from? How many locations around England would be enough for what would seem like a good average calculation? What if averages in the past were computed differently? Could you make comparisons?

Average seems like such an easy concept - but it often is quite tricky. To say nothing about trying to understand the variability of temperatures (do they compute standard deviations?)

For example, the fifteen day record could occur in a season where the other one hundred and thirty five days were pretty cold...

Something a bit different

Most of the posts here are about what I do in the classroom, mostly geek stuff. But I am also a Christian who believes God plays an active role in my life. I have experienced too much to doubt this.

But it is also true that as a professor, there is a dynamic that I have to be careful about with students. Since I have the rights to assign grades, I try to be careful not to make students feel that their beliefs might impact their grades. I work with students from too many different backgrounds and I never want them to worry that their faith will be an issue with me.

On the other hand, I also try not to hide my Christian beliefs, and the way that affects how I live my life and treat others.

The dust-up over Brit Hume's comments about Tiger Woods at first seemed to be the kind of thing I sometimes worry about: how can a Christian tell a Buddhist what to do?

This article from Michael Gerson in the Washington Post is a great way of thinking about this (I think). In the absence of coercive power, why wouldn't someone offer a life preserver that worked for them to another person in pain?

UPDATE: Brit's comments.

This is not how I do it

I could never get a good distribution this way, so I just read them one page at a time.

What do we know?

I am naturally somewhat of a skeptic, but it is hard to not read something and think about how smart the scientists are, and how much they know, and what will soon be possible. One of the things that is really interesting is brain research. Suppose we could know how we work?

All kinds of new tools allow researchers to "watch" your brain work. Or can they? How do they "know"?

Here some interesting reading about the difficulties in measuring, knowing and statistics.

Here is another that makes you think twice when we assume that science is a clean process that follows a straight line to the truth, getting it right along the way.

And here's a cartoon view of how this works (or doesn't work).

Cartoon with statistics

This cartoon site has quite a bit of "geek" humor, so I like it. It also has an interesting feature where if you move the mouse over the cartoon, a hidden message appears. This one has a comment about "the mother of all sampling biases." It may not seem funny, but it is a really great example. Some of his other cartoons are pretty good. This one makes me wonder about teaching statistics.

Complex software

One of the areas I have worked in is thinking about modeling complex manufacturing systems to try to analyze the cost functions that could be used to approximate the behavior of the system. This required lots of machinery, and reviewers were at a huge disadvantage, wondering what was going on with all my software. I tried to think of ways to "open" the code and the methods, but it was quite difficult. Getting the paper published was not picnic.

The global warming software debate is very similar. I don't know whose software is more complicated (maybe not mine?). Here's an article that talks about the lack of (and need for) software reviewing. Academe needs to confront this issue seriously.

Simpson's paradox

Averages seem like simple things, but not always. Consider this simple baseball example:

Tony and Joe are competitive friends and so they compare batting averages. At the All-Star break, Tony is batting .300 and Joe is only batting .290. Joe mentions that batting in the second half of the season is more important, and so he and Tony agree to compare their batting averages for the second half of the season (and only the second half). When they finally meet, it turns out that Tony batted .390 in the second half of the season. Joe did better, too, but only batted .375. Tony wins both halves of the season.

Question: who's batting average was higher for the entire year? Turns out we don't know, and it could very easily be Joe! (I'll post an example later)

What the paradox states is that averages for subgroups can demonstrate relationships that are inconsistent with averages for different subgroups or the overall averages. So Tony could win both halves of the season, but have a lower batting average for the entire season.

I do not know the details about Climategate, but it is very interesting to me, with all the statistics. Are average temperatures going up or down or whatever. Here's an article that about midway through mentions this batting average paradox. I wonder if I have a new example of the paradox involving averages.

Telling a story with a graph

This website has some interesting graphs, and I especially like this one. You have to think for a second before it makes sense, but once it does, it provides a nice explanation for some of the mess people have gotten themselves into (they believed the fiction).

Financial collapse

First, know that I am not especially savvy about financial instruments. I sort of understand puts and calls, but not some of the financial instruments that are all over the news. And you can read lots of articles from people who think these people or those people are responsible for the mess. And some that civilization is coming to an end in deflation or hyperinflation.

Second, know that I am not a stockholder in JPMorgan Chase, although I do have one of their credit cards.

But I read this (long) letter from the CEO of JPMorgan Chase to his shareholders and came away feeling like it was one of the more honest assessments I've read in a while. The part that interested me starts on page nine. I think I will come back to it in a year or two and see how Mr. Dimon's analysis has held up.

Bubble charts

New charts that are popular these days are called bubble charts. They show data over time, usually over a couple of axes by using bubbles of different sizes. Now I don't know about the source data for this page, but the graph is pretty cool. And a little scary.

Means and medians

Here is an article from the Wall Street Journal about the housing market in Detroit. Here's how the story starts:

On a grassy lot on a quiet block on a graceful boulevard stands the answer to a perplexing question: Why does the typical house in Detroit sell for $7,100?

The brick-and-stucco home at 1626 W. Boston Blvd. has watched almost a century of Detroit's ups and downs, through industrial brilliance and racial discord, economic decline and financial collapse. Its owners have played a part in it all. There was the engineer whose innovation elevated auto makers into kings; the teacher who watched fellow whites flee to the suburbs; the black plumber who broke the color barrier; the cop driven out by crime.

The last individual owner was a subprime borrower, who lost the house when investors foreclosed.

Then the article sites some statistics:

And the median selling price for a home stood at a paltry $7,100 as of July, according to First American CoreLogic Inc., a real-estate research firm -- down from $73,000 three years earlier. A typical house in Cleveland sells for $65,000. One in St. Louis goes for $120,000.

Now I understand that median house prices are the standard way of talking about what is "typical." Using the mean creates problems when most houses are of one value, but there are some outlier, high-priced houses.

But in this article, is median misleading in a different way? Did the house on Boston Blvd sell for $7,100? How many houses were sold in Detroit? Voluntarily? Can the dynamics associated with foreclosures make the median a poor choice to report?

Often we try to summarize with a few statistics, and that can be useful. But here I think there is need for more information to really understand this situation. The reporter probably had access to that data, if they wanted to report it.

Decision making when you know you will be second guessed

I teach quite a bit about structuring problems to help make decisions. Usually models can help illuminate the tradeoffs, and sometimes it becomes much more clear what the right strategy is.

But sometimes the downside of the "right" decision comes to dominate the thinking of the decision maker. This is a fundamental part of the dilemma facing the contestant on the game show Let's Make a Deal when they are offered the chance to change their original decision. Even if they are better off changing, the emotional pain of changing and then losing makes them stay with their first choice. They can hear their friends saying "You won the car and then you gave it away."

Here's an interesting article about football coaches going for it on fourth down. I am not sure about all the details of the study, but it is probably true that the thought of what the sports writers and talk radio people will say play an important part of the calculations of when to try or punt.