The article "Monitoring the U.S. Economy Using R", written by Kevin Farnham, was published at oreillynet.com on (date). This page provides sample raw data, the Perl & R code used in developing the article, and related materials.

S&P 500 Input Data and Conversion

The raw Standard and Poors 500 index data was downloaded from the Yahoo! Finance site and converted into a simple two-column table using the csv2CloseTable.pl script:

Listing: csv2CloseTable.pl

#!/usr/bin/perl -w
# csv2CloseTable.pl - 5aug2005 KF
# http://www.MathematicalAnalysis.com 
# convert S&P500 csv file into date & close table

use strict;


if ($#ARGV != 1) {
  die "usage: $0  \n";
}

my $inCsv  = shift;
my $outTab = shift;

my $inLine;
my @items;
my @closes;
my @dates;
my $iDay;
my $outLine;
my $close;
my $date;




# read in all the data

open IN, $inCsv or die "could not open input CSV file $inCsv!!\n";
$inLine = ;  #header line

while () {
  $inLine = $_;
  chomp $inLine;
  @items = split(",", $inLine);
  push @dates,$items[0];
  push @closes,$items[6];
  }
    
close IN;



# write date and adjusted close (in reversed date order)

open OUT, ">$outTab" or 
      die "could not open output table file $outTab!!\n";

print OUT "   Date    SP500\n";

for ($iDay = $#dates; $iDay >= 0; $iDay--) {
  $date = $dates[$iDay];
  $close = $closes[$iDay];
  $outLine = sprintf("%9s %7.2f",$date,$close);
  print OUT "$outLine\n";
}

close OUT;

The resultant output file (SP500.txt) was used as the input into R for all of the S&P 500 analyses demonstrated in the article.

OFHEO House Price Index Data

House price index data published by the Office of Federal Housing Enterprise Oversight listing the percent price increase for homes in the nine OFHEO-defined regions of the U.S. in the past five years was copied into a simple text file:

File: ofheo5yr.txt

83.03
40.22
37.71
24.94
29.00
23.38
55.22
63.63
70.28

The input data was then converted into a grid that represents the house price increases as a function of geographical location (using an approximation of the OFHEO regions for mapping the data points) by script makeOfheoRegionGridInput.pl, using a grid points multiplier setting of 3.

Listing: makeOfheoRegionGridInput.pl

#!/usr/bin/perl
# makeOfheoRegionGridInput.pl - Kevin Farnham, 12 August 2005
# make input data grid for OFHEO data images

use strict;

if ($#ARGV != 2) {
    die "usage: $0   \n";
}

my $inFile      = shift;
my $outFile     = shift;
my $gridPtsMult = shift;

my @regions = ("P", "M", "WNC", "WSC", "ENC", "ESC", "SA", "MA", "NE");

my %xmin = qw (
    P   4
    M   3
    WNC 6
    WSC 2
    ENC 6
    ESC 3
    SA  1
    MA  7
    NE  8
);

my %xmax = qw (
    P   10
    M   10
    WNC 10
    WSC 5
    ENC 9
    ESC 5
    SA  6
    MA  8
    NE  10
);

my %ymin = qw (
    P   1
    M   3
    WNC 7
    WSC 7
    ENC 11
    ESC 12
    SA  14
    MA  14
    NE  17
);

my %ymax = qw (
    P   2
    M   6
    WNC 10
    WSC 11
    ENC 13
    ESC 13
    SA  15
    MA  16
    NE  17
);


my @data;
my $nRec = $gridPtsMult * $gridPtsMult;


open DAT, $inFile or die "cannot open input file $inFile!!\n";

while () {
    push @data, $_;
}

close DAT;

if ($#data < $#regions) {
    die "Problem: $#data data items entered for $#regions regions!!\n";
}


open OUT, ">$outFile" or die "cannot open output file $outFile!!\n";

for (my $x = 1; $x <= 10; $x++) {
    for (my $iMult = 0; $iMult < $gridPtsMult; $iMult++) {
        for (my $y = 1; $y <= 17; $y++) {
            my $iRegion = -1;
            for (my $jRegion = 0; ($jRegion <= $#regions) && 
	             ($iRegion == -1); $jRegion++) {
                my $region = $regions[$jRegion];
                if (($x >= $xmin{$region}) && ($x <= $xmax{$region}) &&
                    ($y >= $ymin{$region}) && ($y <= $ymax{$region})) {
                    $iRegion = $jRegion;
                }
            }

            if ($iRegion > -1) {
                for (my $jMult = 0; $jMult < $gridPtsMult; $jMult++) {
                    print OUT "$data[$iRegion]";
                }
            } else {
                for (my $jMult = 0; $jMult < $gridPtsMult; $jMult++) {
                    print OUT "NA\n";
                }
            }
        }
    }
}

close OUT

Script makeOfheoRegionGridInput.pl was executed as follows:

   perl makeOfheoRegionGridInput.pl ofheo5yr.txt ofheo5yr.gridFine 3

The resultant output file (ofheo5yr.gridFine) was used as the input into R to generate the image and three-dimensional plot of the U.S. Housing "Bubble" presented in the article.

Related Sites