Technical Difficulties from on Top of the Mountain
2006-02-26
  sparklines sparklines and more sparklines
I've been doing a lot of reporting and analysis on various collections of data where the number of samples ranges from the tens of thousands to the hundreds of millions and I just can't see how normal graphs could have remotely coped with this. At best they could have summarized the data, pointed out one or two facts, at worst they would just have obscured all meaning. The one answer I keep applying over and over is ...

Sparklines.
It shows the littlest detail, while at the same time lets you step back and see the forest for the trees (or the trend from the data). I'm worried I've found the golden hammer and will forever reduce every problem ahead of me into a nail, but it seems in this case to be a good fit.

At the same time, I'm also trying to refine my rendering of these things. The original method with an entire <TABLE> in every cell was a little heavy handed and of course didn't work at all on my phone. For the example up above, several of the strips are just single height bars of different colors which I was able to do with <TD> blocks using width and color control in the TD tag. Even creating the entire strip ends up being child's play in perl:

$chrt .= "<TR height=$ht>". join('', map { "<TD width=1 bgcolor=$color[$_]></TD>" } map { $_ || 0 } @{$h->{$_}} ) "</TR><TR height=2><TD></TD></TR>\n" ; Where $h->{$_} contains the array reference for one strip, and $color[$_] contains the color for the integer value at that point. Could have used names in the hash array, and a hash for the colors, but several years of assembly programming back in ancient times still leave their mark on me today.
I even got more daring, and did some variable height stuff using rowspan. The tick marks under the graphs are a trivial example. You want something like this:


Which you can do with a couple of <TR> where the larger ticks just stretch across rows. (I've magnified the effect here a bit, but its easy to change just by changing the heights of the three different TR's. Not wanting to type much, I generated all this with a little perl as well.

my $ticks= "<TR height=4>". join('', map { "<TD rowspan=$_ colspan=2 bgcolor=black></TD><TD colspan=28></TD>\n" } ( 3, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, 3, 1, 1, 1, 1, 1, 2, 1, 1, 1, 1, 1, ) ). "</TR><TR height=1><TD></TD></TR><TR height=2><TD></TD></TR>" ;
Growing the bars up is a little more tricky. I tried a couple of things for the battery life chart, and finally realized that you sort of had to fill in from the top. First you treat the top row as special, and you fill in every row with the inverse of the height of the bar you want (with a special case for bars that go all the way to the top). So if your graph has 10 steps, you go across the top row filling in blank bars of height 10-f(x) for each spot (and filling in solid bars for those places where f(x) = 10). Then after that you just step each row filling in just the bars that are that high.

So if for instance, my data points were: 6, 8, 2, 3, 1, 0, 3
My first row would have six blank bars of height, 4, 2, 8, 9, 10, 7; then the next row (row 9 counting down from the top) would have nothing in it, the next row (row 8) would have one bar of height 8, next row would be empty, next row (row 6) would have one, and on and on:

<TABLE border=0 cellSpacing=0 cellPadding=0> <TR height=4> <TD rowspan=4 width=4 bgcolor=white></TD> <TD rowspan=2 width=4 bgcolor=white></TD> <TD rowspan=8 width=4 bgcolor=white></TD> <TD rowspan=7 width=4 bgcolor=white></TD> <TD rowspan=9 width=4 bgcolor=white></TD> <TD rowspan=10 width=4 bgcolor=white></TD> <TD rowspan=7 width=4 bgcolor=white></TD> </TR> <TR height=4><TD> </TD></TR> <TR height=4><TD rowspan=8 bgcolor=blue></TD></TR> <TR height=4><TD> </TD></TR> <TR height=4><TD rowspan=6 bgcolor=blue></TD></TR> <TR height=4><TD> </TD></TR> <TR height=4><TD ></TD></TR> <TR height=4><TD rowspan=3 bgcolor=blue></TD><TD rowspan=3 bgcolor=blue></TD></TR> <TR height=4><TD rowspan=2 bgcolor=blue></TD></TR> <TR height=4><TD bgcolor=blue></TD></TR> </TABLE>
And once again, I found the solution to implement this was rather simple:
  ## first row pass $txt= "<TR height=1>" ; foreach (@$data) { $txt .= ($_ >= $steps) ? "<TD rowspan=$steps bgcolor=$fg></TD>" : "<TD rowspan=". ($steps-$_) . " bgcolor=$bg></TD>" ; } $txt .= "</TR>\n" ;   ## fill in pass foreach my $level (reverse 1..($steps-1)) { $c= $cols->[$level ] || $fg ; $txt .= " <TR height=1>". join('', map { "<TD rowspan=$l bgcolor=$c></TD>" } grep { $_ == $level } @$data ). "</TR>\n" ; }
Sigh. Now that I can see what I'm doing, I see I have a lot to do. Better get to it.
 
Comments: Post a Comment

<< Home
Life in the middle of nowhere, remote programming to try and support it, startups, children, and some tinkering when I get a chance.

ARCHIVES
January 2004 / February 2004 / March 2004 / April 2004 / May 2004 / June 2004 / July 2004 / August 2004 / September 2004 / October 2004 / November 2004 / December 2004 / January 2005 / February 2005 / March 2005 / April 2005 / May 2005 / June 2005 / July 2005 / August 2005 / September 2005 / October 2005 / November 2005 / December 2005 / January 2006 / February 2006 / March 2006 / April 2006 / May 2006 / June 2006 / July 2006 / August 2006 / September 2006 / October 2006 / November 2006 / December 2006 / January 2007 / February 2007 / March 2007 / April 2007 / June 2007 / July 2007 / August 2007 / September 2007 / October 2007 / November 2007 / December 2007 / January 2008 / May 2008 / June 2008 / August 2008 / February 2009 / August 2009 / February 2010 / February 2011 / March 2011 / October 2011 / March 2012 / July 2013 / August 2013 / September 2013 / October 2013 / November 2013 / December 2013 / December 2014 / February 2015 / March 2015 / July 2016 / September 2016 / December 2016 / April 2017 /


Blogroll
Paul Graham's Essays
You may not want to write in Lisp, but his advise on software, life and business is always worth listening to.
How to save the world
Dave Pollard working on changing the world .. one partially baked idea at a time.
SnowDeal
Eric Snowdeal IV - born 15 weeks too soon, now living a normal baby life.
Land and Hold Short
The life of a pilot.

The best of?
Jan '04
The second best villain of all times.

Feb '04
Oops I dropped by satellite.
New Jets create excitement in the air.
The audience is not listening.

Mar '04
Neat chemicals you don't want to mess with.
The Lack of Practise Effect

Apr '04
Scramjets take to the air
Doing dangerous things in the fire.
The Real Way to get a job

May '04
Checking out cool tools (with the kids)
A master geek (Ink Tank flashback)
How to play with your kids

Powered by Blogger