Today I was playing with cygwin, and decided to cat all the source code on my machine to the screen. Something like this:
find . -name '*.c' -exec cat {} \;
No particular reason, and it didn't take that long since it was all coming off a solid state drive, though it did pause a couple times ... But the very last lines were:
int main(void) {
printf("hello world\n");
return 0;
}
Some pretty old code there.
Communicating between programs usually means across computers, especially when its a client program ( or web browser ), talking to a server somewhere else. For that you almost universally use a inet socket ( TCP, or UDP if you're a glutton for punishment).
There used to be a bunch of other protocols, but IP pretty much crushed them all. Ungerman Bass had its own SNA back when the internet was starting to form, Digital Equipment Corp had DECnet, IBM had SNA, and Novell Netware used IPX. But TCP/IP was good enough, pretty darn simple, and as we went from 1mbit to 10,000mbit, other things became the bottleneck. Sure if you run a telco, you may still wax on about ATM, but even the ATM network is carrying TCP traffic.
On a server however, you have a fair amount of traffic staying on the same machine, passing back and forth between different programs; usually a result of dividing a problem down into smaller parts that are hopefully harder to mess up. So you need a mechanism for setting up connections and passing messages.
Even in this case you could use INET sockets, but that's not your only option, nor is it the best choice for a number of reasons. First, there's a lot of overhead to INET sockets. Even if your packet isn't going to cross great distances, the operating system still does all the packet overhead like it would. This puts a limit on the number of packets you can read and write, especially a problem if your communications is a bunch of short messages. Secondly, when you create a INET service, it is visible to the network beyond your computer, allowing anyone running a portscan to find it. So speed and security are both good reasons to look elsewhere.
As the workstation market began to grow, and AT&T decided to wade back into the unix market; they added new kernel services to System V, called IPC ( inter process communication ). There were three parts: semaphores, shared memory and message queues. Semaphores allowed passing access or control between processes to a shared resource, shared memory seemed like a good way to avoid having to pass around large data sets when disk access was expensive, and message queues gave you both an orderly mechanism passing data, as well as atomically handling a message. Unfortunately at the time, all these data structures existed in kernel memory, and they were fixed in size ( originally compiled into the kernel settings ), so on a typical machine they were ridiculously small. One one HP machine with 64MB, the limits shown were 64 semaphores, 4k message queue, and 64k shared memory. Even today on a machine with 2GB of RAM, ipcs -l shows a queue size limit of 16k.
Moving on.
On unix, creating the raw socket() itself is protocol neutral. Its just that most everyone in the universe uses INET. You can also use sockets for RAW packets, ATM, Appletalk, IPX, X25; and one more format called AF_UNIX ( although its now refered to as AF_LOCAL for POSIX reasons, but the structs are still all un_ ). A AF_UNIX socket is for communicating locally on the machine. Originally, like INET sockets, there was a private namespace, using 32bit numbers which you used for the "port" number, but then they expanded it to also allow you to map the socket into the filesystem, so you would get a file that showed up like this:
% ls -lF total 0 srw------- 1 woolstar users 0 2009-06-26 23:46 agent.16975=ssh-agent uses this to allow ssh processes to authenticate against a stored key. Back when this first showed up, on some OS's you could actually just talk to this entry like it was an ordinary device. That was sort of the spirit of unix, everything was a file. You could open up /dev/serial0 the same way you would open up foo.txt. So back in the day, you could open this file mapped socket, send it some data, and then close it, and the interaction on the server side would look just like you had telnet'd in. Sadly, AF_UNIX sockets don't work like this any more. Even though they're sitting right there in the filesystem, you can't just echo "hi" > mytest.sock You have to use socket functions to connect to it and read and write to it.
Still, if you control both sides, the upside is definitely worth it. In some tests I did around 2001, for smaller packets on a single processor machine, AF_UNIX connections could out-run AF_INET by over ten to one. Also AF_UNIX allows some funky *magic* data to be passed between machines. Like one process can pass an open file handle over to another process. You can also authenticate your user id and group across an AF_UNIX connection and the kernel will validate you to the recipient. And as I mentioned before, AF_UNIX connections are only available on the machine, so there's no outside hacking into these services.
But for my current project, the lack of transparently using AF_UNIX sockets was a bummer, because I have a project I'm working on where a process runs and then opens up an external file to write log entries to. I want to have those entries go immediately into another process for processing, and so I wanted to throw a named socket into the file system and have the first program log to that "file". The first process is this big third part server that I didn't want to have to mess with, so spoofing a file would have been ideal. Luckily there's something else available now that will do it.
fifo(7) or named pipes
A fifo is like a queue, where you put several things in, and then pull them out. In this case the first thing you put in is the first thing that comes out ( First In First Out ... fifo). There's also FILO, but it doesn't make as good an acronym in my opinion. So the modern linux kernel allows you to create a named pipe in the file system, and as an improvement, you don't even have to have any active processes attached to either end. It could just be sitting there. Then when you want, you attach and try to read from it, which doesn't do much cause there's nothing in it. When someone else comes along and write to it, then the message shows up to the reader.
There are some caveats of course. If no one is hanging around waiting for the message, you can't write to the named pipe. The kernel isn't going to save things up for you. Also, if several processes are reading to a named pipe, you can't tell the difference. Its not like sockets where each connection will have its own file handle and you can tell the lifespan of each client. But for the purposes of log processing it will do just fine. At least I hope it will. I will have to get back to you on that.
There was a mouse down in the basement this weekend. The kids got all excited, until the mouse was smushed, and then they were all like, "papa, time to cleanup." Note for future adventures, smashing things are the most effective when hunting mice. Very hard to stab a mouse.
In the process of cornering the criter, I had to disturb a pile of wood pellet sacks from last winter's burning operation. I was saving those to count them, but I've done a terrible job of keeping track of how much I actually burnt per month.
I have some notes in twitter account, which turns out to be a terrible place to put anything you want to lookup later. Best I can find is numbers for January and March, 1,400 and 1,600; and 1,320 for April. Overall I had six tons, and there's about 18 bags left out of 300.
It seems I burn as much pellets as I have. So the lesson there for conservation, buy less pellets. Temperatures already getting down to 40s at night, so we'll see how that goes.
I get calls from my alumni association every once in a while for a donation, and once in a while I'll give a few dollars to be counted as a supporter. But I'm not an active alumni. Its not that I don't think the college is a good one, it is. There's just issues that have cut me off. One of which is long over, but the other, while completely solvable, sits between me and active participation.
My time at college was excellent. I was surrounded by a bunch of really smart people, had access to millions of dollars in equipment (what it took back in the stone age to have great computers--now you can get a really great computer at Sams club), and even had access to staff and faculty where I could just sit down and launch into a tirade, only to have my lack of understanding explained to me.
But the end of the eighties was also the ascendancy of Reagan and extreme conservatism, and it hit at college as well, with the Master of Student Houses running all over the student body, including banning the once a year all-house party (Interhouse). Students were no longer trusted young adults and future leaders, they were troublesome kids that needed to be policed. Luckily I got out of there just as it was getting bad.
Fast forward nearly twenty years later, and things have returned more towards the middle. Interhouse is back, and things seem a bit normal. The problem is, I've moved on, and need somehow to reconnect with things. To see what's happening on campus, with the students, with the faculty, with the institute. Something like the university's streaming theater, wich interesting lectures from people like Harry Gray ( a professor I actually liked when I was there ).
But I can't watch it, cause its in Real Video format.
Yes, I could go get the Real Video player and install it and all its associated spyware on my computer. But I'm not, and nobody else is going to anymore because Realvideo is so 90s. We moved past that to Flash video about ten years ago. And in fact, the future is to just let the user watch the file however he wants with a avi/mp4/ipod download. Its doesn't have to be HD. I'd watch Caltech events at 320x240, just like my John Stewart.
I even sent off a email to the director of the PR/Media department (who made this unfortunate video format choice in the first place), but I'm not holding my breath. The last media update on mr.caltech.edu was a entry about a radio broadcast in 1998; and a note about a lecture in the Biology department on October 2003.
I hope those memories from twenty years ago are not the last positive memories I have from college.
When I first discovered the treo, I was very excited. I almost went out and got a 600, but came to my senses and waited for the 650. The 600 was low tech compared to the 650. The 650 had a 320x320 screen with lots of colors, a decent processor, a SD card slot, an improved keyboard and so on and so forth. When I got it I immediately set out to figure out how I could use it.
I was able to stuff quite a few programs on it, and tried to use it as often as I could. One problem was that cell reception wasn't very good at my house, so I couldn't use it much in the house, which was a little annoying, because I worked at home. So it wasn't very useful as a home unit, but it was fairly kick ass when traveling. Sort of.
The web browser was quite slow. It could easily take five minutes to login and display my home page on ebay. (It was also frightening that one page on ebay was over a megabyte of data.) The email client was ok, but the sms notification that I had new email didn't display anything about the message itself ( and was often larger than the message itself ), and deleting the text messages often caused the phone to reset.
Then, there was this ancient free AOL client, but it had an annoying habit that if you left it logged in, then switched to another app long enough, the connection went away, and then the AOL client corrupted its preference page; which on the next reset would cause the phone to infinitely reboot. That was loads of fun to cleanup after. Thank goodness I knew engineers that worked at Palm, and so I could get free tech support from someone who actually knew what was going on.
So things were a mixed bag at first. Then TCPMP came out. Suddenly I had a movie player that fit in my pocket. With that, travelling was a joy. I'd be sitting there watching the Daily Show, laughing myself silly, completely forgetting how tiny my seat was. (I had long ago given up trying to open a laptop in the small silver of space between my face and the seat in front of me.)
Over time I settled into using the Treo as a media player and as a cell modem for my laptop with June Fabrices PdaNet. Once in a while I'd use the web browser to read coding horrors or another lo cal web site, and I did get regular gurgles from the email when I had a posting to moderate for Flagstaff Freecycle; but those weren't killer apps. Google maps came out for Palm, which was cool, though I didn't need a map every day, but on those occasions that I did need it, it worked well.
So I settled at that point. The treo wasn't an internet device proper, but it could get me there in an emergency, and it meant that more powerful devices could get online when I needed to.
Newer treos came out, but they didn't really add anything useful. Same screen, about the same keyboard. Little faster cpu, maybe a little more memory, but nothing radical, like going from the 600 to the 650. I let it slide.
Then, after getting un-married, somehow my ex-wife decided that she needed to get her own cell phone plan, and after staring into space at the Sprint store for an hour, the service droid declaired that this would only be mostly possible if I kept five lines on my account by move a new phantom line onto a new real cell phone. I didn't really buy the explanation, but as I imagined spending another several hours at this store with my three kids running around destroying displays and digging ancient dustballs from beneath the counters, I gave into fate and let them sell me a Treo 755p.
It was a little bit cooler, a little bit faster. One nice thing was its non-slip case. My Treo 650 had to live inside a non-slip sleeve in order for me to keep a grip on it. But that bulked it up somewhat. The 755p could also read SDHC cards, though it didn't use full size cards, but instead used mini-sdhc. However they already had 4gb cards, and soon would have 8gb; so that was progress. Finally, it had next generation data support, which in Flagstaff didn't do me any good, but when I went traveling to a real city, it could reach speeds of over 500kb/s.
So I used my new phone and stuck the old one under a pile of papers on my desk.
Until a few weeks ago. When I went to pull the phone out of my bag and it had a giant crack across the screen and was displaying all kinds of funky colors, instead of something resembling a normal screen.
So I went back to my 650 for the time being. The major functions are all there, but surprisingly its the small things that I miss.
The 755p would blink a green light if there was a missed call/voicemail/text message. You could look at it, even with the screen off, and see if something had happened. Just a little software thing, but the 650 doesn't have it, and even the new Centro doesn't do it either.
The 755p has a better radio. The 650 doesn't work at all in a lot of spots in my house, and even sometimes will show two bars, but when I start a call it drops to no bars, or even no service at all. Sometimes it will show a bar, but no calls or text messages will come in, then I'll move it around on my desks and a stack of messages will suddenly arrive.
I'm back to having the messaging app reset my phone when I delete messages. Its still annoying. And it was doing it before I loaded any other 3rd party apps on there.
And the 650 is just slower. It lags significantly when doing simple things. They did some cleanup two generations down the line, and a faster processor and more memory probably helps; so the 650 feels like going back to a ancient processor with not enough memory to get anything done. I really don't want to try using the web browser at all.
Also, I have a Nokia N800 internet pad, which is the device I thought the Treo was going to be originally. The Nokia has a huge screen, two full size SD card slots ( so I have 12 GB of space for my music and movies ), wifi, a real processor, and a full featured browser that can do Flash and AJAX pages like Google docs. Plus I got off my butt and setup RSS feeds for my most read blogs on it. The treo makes a good internet tether for the N800 when there's no wifi around, so I don't think I'll try to browse pages on the Treo except in an extreme case.
I don't know if you can combine a Treo and an N800 into a single device. The N810 has a sliding keyboard, but still no phone. There are other N devices with phones, but they're mostly GSM, so its not going to replace my Sprint phone any time soon. I don't think I'll buy another modern phone right away. Maybe I'll look again with Palm completes the switch over to Linux and can make a phone with wifi that works at the same time as the phone.
For now I have a bag of tricks. Some tasks are handled with excellence, others are just barely covered, but I have reached an understanding of how my current pieces fit together and I'm cool with it. Its up to the consumer electronics industry to shake my out of complacency with the next great thing. Or not.
For the last several years, I've saved a few days in the summer to head off into the far reaches of Arizona ( yes, there are places even more remote than Flagstaff, just ask my cell phone company ), and spend some time teaching welding at summer camp. Its been a organic process, we started off with whatever I could borrow and stuff into the back of my car, but after several years of doing it, I can make my way through the classes with a minimum of fuss and damage. It also helps that a number of the campers are repeat offenders, and thus have some prior experience with my laid back style of teaching.
But no matter how smoothly things are going at camp, it seems there's always some excitement going on back at home.
I get out to camp the first day, get everything setup, and am relaxing on the bunkhouse porch before bed, when my friend tells me he has a message on his cell phone from my ex-wife. ( My cell company doesn't have any service out there.)
So I call, and the water's out. Its not raining, and the underground equipment area isn't flooded, so its not the obvious thing. My dad's there, and he's trying to figure out if there's power, but I have three levels of sub-panels cross wired into everything, and none of the breakers are labeled, so he's struggling a bit, but my memory can only go so far. I tell him to turn everything on, and then go plug something in down in the crawl space plug.
Unfortunately the first two work lights he grabs are burnt out, but I would have expected that, but he wastes time going around in a circle a little, and finally grabs my big work like, which does work, and confirms that there is indeed power down in the basement.
This is good news in that the power shouldn't have gone out, but bad news in that the pump is now the suspect, and replacing the pump could be a challenge. Give how non-linearly things have gone so far, I decide to call for backup.
The nice thing about having friends, is that you can call on them in the most desperate of times, and they'll come over and help. My life is so hectic, that I don't have very many people call on me for help that much, but I do try and do my part. Mr. Hess is retired, and his kids are all grown, so its easier to get ahold of him. He says he'll come over and take a look.
Thankfully, the next report I get, after I give the cell phone a chance to recharge, is that the pressure switch is burnt out, and so they just have to get a new one of those and things should be back in business. For some reason, that takes three hours the next day, but I'm hoping that its because they sat down in the cave yakking most of the time, not because it was a major engineering effort. The water gets restored, and the crisis is over.
I wonder what's going to happen next year.
I'd like to say that the burning season is over. I'd like it to be over. But the fact that its May doesn't seem to keep the nights from dropping down into the mid twenties. Gads. The 2007 season pellets are gone, they went pretty fast:
| Month | This year | 2007 | 2006 |
|---|---|---|---|
| September | 160 | 360 | - |
| October | 960 | 1,360 | - |
| November | 1,120 | 1,640 | 1,200 |
| December | 2,080 | 1,920 | 1,720 |
| January | 2,240 | 2,360 | 2,000 |
| February | 1,800 | 1,920 | 1,280 |
| March | 840 | 1,320 | 1,080 |
| April | 240 | 480 | 440 |
You can see we burnt a lot less in March, but that was mostly due to the fact that there wasn't much left to burn. A lot more fires that much. April wasn't too bad, and I was home most of the month watching the kids while JM went overseas, so I let the temperatures in the house drop below 70°rees;F at night. But that's over now too.
Just when I was getting ready to enjoy having that third parking spot left, I got a call from a friend who had spotted a sale sign over at Ace.
2006 was a mess for wood pellet users. Nobody in town had consistent stock, prices was all over the place, and I ended up buying pellets from two hardware stores, Walmart, and the local gas station (who shipped in a supply from Canada). Part of the problem was a supply issue from a local pellet plant in eastern Arizona. So for 2007, the stores placed monster orders all over the place. Now it was May 2008, and they were sitting on top of a couple million tons still.
Ace blinked first. Normally a more expensive option (over priced only by the local grocery store), they dropped their price to $3/bag or $150/ton. There went my parking space. For about the same price as four tons before, I picked up six tons this time, and actually managed to get them all in the garage, in one parking spot. And because we now have what looks like an infinite quantity, we've relaxed a bit and started burning them again. Granted otherwise my basement would drop below 60°rees;F, but it still seems absurd.
I need to start setting up for next year with some solar plans. Maybe by 2009 I might not need to hand carry 12,000 pounds of wood pellets down into my basement.
At last week's green building meeting, there were a number of design students attending the meeting ( it was on campus at the university, go figure ), and one of them asked us "old folk" what advise we would give them, nominally about how to work renewable and sustainable technologies into their projects.
I gave half a good answer: when it comes to choosing projects, choose the customer who's willing to be more adventurous. NAU had just given us a tour of their new building which they built with un-tested concrete mixes, and where they were refining the process as they went. There's actually a difference in the final product between the first floor and the third floor. That's just an astounding thing, to have a customer spending millions of dollars, and still walk out on a ledge to try 40% fly ash content in concrete and use the project as a laboratory to work out the problems.
While it would be nice to have daring clients on every project, an even better strategy might be to help every client take a step, no matter how small, away from the status quo. And to do that requires a special kind of promotion. One of the biggest stumbling blocks to progress is not lack of promotion, its the opposite: hype. Hype has done more to stall the implementation of sound technical ideas than anything else. Every new idea has its rough edges, and running a reluctant customer right into one of those edges will cause more problems, bad will, and general long term loss of credibility; than loosing out trying a new idea two times out of three.
The other way to help around this, is find ways to take small steps before pushing the customer to take big ones. Don't expect the customer to spend thousands or more on some new thin film solar panel if you've never even seen the device in person yourself. When I want to implement new technology in my projects, I play with the language/database/operating system first personally before I claim it should be used on a project. Even in metal working, I spend my own money on something, and check it out before I expect the shop/school/customer to use it. Doesn't mean you have to go as large, or go as nice as what you would use for the project. Just get your hands on anything, in whatever state so you can say, "Yes this is real, and I believe in it." Ebay is great for that.
Going with the way things were before is always going to be the least effort, so putting effort into moving in a new direction is how things will get changed. Do what you can to take on some of that effort, and help everyone you meet put in what effort that they can. You won't turn things around in one instant, but if you can look back at the end of your career and see how you were part of a larger effort that pushed society slowly in a new direction, that's something to be proud of.
