ADVENT #8 — pcmcia
Today’s post was going to be about concurrency in Alice and some of the esoteric Lisp stuff that keeps you coming back. Instead, it‘s about my home water heater.
Before we get there, this is also a series about removable media. Today’s removable media is the PCMCIA card. PCMCIA is a terrible name. A handy way to remember it is to just say “Personal Computer Memory Card International Association” and then take the first letter of each word! A more popular mnemonic was “People Can’t Memorize Computer Industry Acronyms”. As cutting as that is, I don’t like it since PCMCIA itself is an initialism, not an acronym.
PCMCIA’s heydey was roughly the same years Seinfeld was in prime time — about ten years starting from 1989. In that time, the major removeable media in personal computers was the 3.5 inch floppy disk. As convenient as they were, disks were too large to fit inside a wave of palmtop PCs, like the Sharp PC-3000, that proliferated in the early 1990s. These machines used battery-sipping CMOS versions of the 8086 processor family popularized by IBM’s PC. Palmtops often had a version of MS-DOS built in along with a shrunken version of a desktop productivity app like Lotus 1-2-3. The PCMCIA memory card format was really built for these machines. The format had an electrical interface that was essentially a hot-plug version of the original IBM PC bus. the cards was smaller than a floppy but thicker, sitting in that awkward middle space between a stack of business cards and seven Virginia Slims in a trench coat. The standard originally addressed SRAM cards, then flash memories, modems, Ethernet cards, and other peripherals suitable for these tiny machines. The slots were a bit in laptop computers and a variety of embedded applications as well. The famous ‘Fortezza’ card is typically fitted to a special telephone with a dedicated PCMCIA slot and removed when the phone is not in use. In its original application, PCMCIA cards could function more like hard drives that lived in a permanently in a device but gave the user some control over what could be a substantial part of the total device cost. As digital cameras were popularized, the PCMCIA card found a new role as an early form of digital film. PCMCIA begat the Compact Flash format that continued the format’s run for another decade. PCMCIA was often the way a new type of peripheral was made available to otherwise unexpandable laptops. Until around 2000, most of my laptops had Ethernet this way. At around the same time, the first widely-used 802.11b WiFi cards turned up. one of the most popular of the early lines was Lucent’s Orinoco. Here, the same card was used in laptops, in the base stations, and in desktops through a PCMCIA to ISA adapter. when apple adopted 802.11 as ‘AirPort’, the hardware was built in to laptops. Their first base station was a repackaged Lucent AP1000 with the Orinoco WiFi card.
My water heater is leaking. I guess water is a media after a fashion and anyway it seems like it’s removing itself. The heater is part of a combined water heater / hot water boiler for my radiators. Each of the ten radiators in my house are controlled by a valve at a central manifold and each of those is controlled by an arduino-based PLC. The reason I chose a combined hot water heater / boiler is sort of the same reason you would choose a Xircom PCMCIA combination Ethernet card and dialup modem. It fits in the small space available in my small house. The other reason is really about my core design principle as a working programmer. If you’re building a system that has a special party trick, that trick had better be required for even “Hello World“ to work or the special sauce will have stopped working years before you notice. I want to get a check on the health of my seasonal boiler every time I turn on my hot water tap, and not the first cold day in November.
Today’s leak turned out to not be that bad and was easily fixed for the moment. That principle that the trickiest piece should always be in the critical path is part of how, and why, you would build a system like Alice. If you’re going to generate a trace, then the system had better be dependent on the trace itself in order to function. My furnace control system runs an Alice precursor. A temperature record from every sensor in the house is dumped to a flash memory log every two seconds. Every piece of state associated with the thermostat is dumped into the log. The state of the valves is dumped into the log. After four years of that, the log is only a few tens of gigabytes. For the price of a month’s gas bill, I can provide this system with enough flash to outlive me and my children without having to ever delete a record. The only drag with this particular system is that I restart it every couple of years and I do that by replaying the entire log into it. It now takes several hours to reboot it. Multiple agents interact with the log and flock() provides all the consistency I need. A downside of this as an experimental platform is that it’s the program that keeps my family warm on cold nights, so I only restart it every couple of years. It has changed very little from the first version able to reliably keep us warm. Several copies of this program run concurrently against the same log, each monitoring a different sensor and thermostat. Each controlling a different relay.
My plan is to upgrade the system to Alice and enable sound, fast restart. One feature that I put in this system is sensor syndication — anybody who consumes information on the tape, like a route to a sensor or relay, is expected to republish it from time to time. A practical Alice requires the same thing — a way to trade space for locality in the pathological case. We’ll look at how the compiler manages that now and how I think tape post processors can manage that after the fact.
Here it is:
LineStream = require('./line_stream.js');
let port_name = false;
UnendingReadStream = require('./stream_readable_unending.js');
fs = require('fs');
const {get_route} = require('./route.js');
// our AC default
let ac_valve_zone = {
last: false, // the most recent relay status record
prev: false, // the last status record of the prior relay status
min_cycle_minutes: 20,
max_cycle_minutes: 60,
inter_cycle_minutes: 20,
relay_addr: "33:05",
mode: "cool"
};
/* // a heat default for on-demand boiler with cast iron
let valve_zone = {
last: false, // the most recent relay status record
prev: false, // the last status record of the prior relay status
min_cycle_minutes: 15,
max_cycle_minutes: 60,
inter_cycle_minutes: 20,
relay_addr: "xx:xx",
mode: "heat"
};
*/
let basement_heat_valve_zone = {
last: false, // the most recent relay status record
prev: false, // the last status record of the prior relay status
min_cycle_minutes: 15,
max_cycle_minutes: 60,
inter_cycle_minutes: 20,
relay_addr: "33:08",
mode: "heat"
};
let kitchen_heat_valve_zone = {
last: false, // the most recent relay status record
prev: false, // the last status record of the prior relay status
min_cycle_minutes: 15,
max_cycle_minutes: 60,
inter_cycle_minutes: 20,
relay_addr: "22:34",
mode: "heat"
};
// our default house heat valve zone for the mixed-cast/steel radiators
let heat_valve_zone = {
last: false, // the most recent relay status record
prev: false, // the last status record of the prior relay status
min_cycle_minutes: 15,
max_cycle_minutes: 60,
inter_cycle_minutes: 20,
relay_addr: "33:07",
mode: "heat"
};
let hallway_sense_zone = {
sense_addr: "28:68:37:E2:08:00:00:C2",
sense_location: "upstairs hallway"
};
let basement_kitchen_sense_zone = {
sense_addr: "28:0C:1E:E3:08:00:00:59",
sense_location: "basement kitchen"
};
let kitchen_sense_zone = {
sense_addr: "28:9B:82:E3:08:00:00:FB",
sense_location: "kitchen"
};
let kitchen_heat_tstat_zone = {
set_temp: 23.2, // default temperature in C
thermostat_addr: "44:35",
can_syndicate: false,
};
let basement_heat_tstat_zone = {
set_temp: 22.2, // default temperature in C
thermostat_addr: "44:34",
can_syndicate: false,
};
let upstairs_heat_tstat_zone = {
set_temp: 23.2, // default temperature in C
thermostat_addr: "44:37",
can_syndicate: false,
};
let ac_tstat_zone = {
set_temp: 26.5, // default temperature in C
thermostat_addr: "44:36",
can_syndicate: false,
};
let action_thresh = 10000; // horizon in milliseconds beyond which a read record
// is not considered contemporary
let command_status_latency_thresh = 10000; // how long to wait for a command to be reflected in status
let last_command = false;
let sense_count = 0;
let earliest_seen = false;
let debug = false;
const {flock, constants} = require('fs-ext');
let send_message = function(msg, next) {
fs.open(port_name, 'w', function(er, fd) {
if(er) {
return next('unable to open port');
}
let s = msg;
let b = Buffer.from(s, 'utf8')
flock(fd, 'ex', function (er) {
if(er) {
fs.close(fd, function(er) {
return next('unable to lock port');
});
}
fs.write(fd, b, function(er, res) {
if(er) {
fs.close(fd, function(er) {
return next('unable to write to port');
});
}
if(res != b.length) {
return fs.close(fd, function(er) {
next('short write');
});
}
return fs.close(fd, function(er) {
next(false, true);
});
});
});
});
};
let send_off = function(addr, next) {
if(addr == "22:34") {
send_message("{c}" + '\n', next);
} else {
send_message(JSON.stringify({type: "record", date: new Date().toISOString(), command: "off", addr: addr}) + '\n', next);
}
};
let send_on = function(addr, next) {
if(addr == "22:34") {
send_message("{C}" + '\n', next);
} else {
send_message(JSON.stringify({type: "record", date: new Date().toISOString(), command: "on", addr: addr}) + '\n', next);
}
};
let journal_message = function(msg, next) {
console.log('journaling message');
fs.open('temps.out', 'a', function(er, fd) {
if(er) {
return next('unable to open journal');
}
let s = msg;
let b = Buffer.from(s, 'utf8')
flock(fd, 'ex', function (er) {
if(er) {
fs.close(fd, function(er) {
return next('unable to lock journal');
});
}
fs.write(fd, b, function(er, res) {
if(er) {
fs.close(fd, function(er) {
return next('unable to write to journal');
});
}
if(res != b.length) {
return fs.close(fd, function(er) {
next('short write');
});
}
return fs.close(fd, function(er) {
next(false, true);
});
});
});
});
};
let program_start_time = Date.now();
let run_sense_tstat_valve = function(sense_zone, tstat_zone, valve_zone) {
get_route(valve_zone.relay_addr, function(er, route) {
if(er) {
process.exit(1);
}
port_name = route;
console.log('using port ' + route + ' for valve ' + valve_zone.relay_addr);
fs.open('temps.out', 'r', function(er, fd) {
if(er) {
throw('unable to open temps');
}
fs.fstat(fd, function(er, s) {
let start = 0;
let rs = UnendingReadStream(null, { fd: fd, start: start});
console.log('d: start: ' + start);
var ln = new LineStream(rs, {delim_start: true});
ln.on('data', function(lrec) {
let rec = false;
try {
try {
rec = JSON.parse(lrec);
} catch(e) {
// JSON parsing error
return;
}
if(typeof(rec) != "object") return;
if(rec.type != "record") return;
if(rec.status) {
if(rec.addr != valve_zone.relay_addr) return;
let now = Date.now();
let then = Date.parse(rec.date);
if(!earliest_seen || (then < earliest_seen))
earliest_seen = then;
if(!valve_zone.last) {
} else if(rec.status != valve_zone.last.status) {
valve_zone.prev = valve_zone.last;
console.log("(" + rec.date + ")" + "SAW STATUS CHANGE " + valve_zone.prev.status + " -> " + rec.status);
} else {
}
rec.date_parsed = then;
valve_zone.last = rec;
if((now - then) < action_thresh) {
console.log("(" + rec.date + ")" + "status: " + rec.status);
}
} else if(rec.temp) {
if(typeof(rec.temp) != "number") return;
if(isNaN(rec.temp)) return;
if(rec.addr == tstat_zone.thermostat_addr) {
if(rec.syndicate) {
// skip syndicate addresses
} else {
console.log('got thermostat record: ' + rec.temp);
tstat_zone.set_temp = rec.temp;
tstat_zone.can_syndicate = true; // we cannot syndicate until we have seen a non-syndicate temperature
}
} else if(rec.addr == sense_zone.sense_addr) {
if(debug) console.log('got sense record: ' + rec.temp);
let now = Date.now();
let then = Date.parse(rec.date);
if((now - then) < action_thresh) {
console.log("(" + rec.date + ")" + "temp: " + rec.temp);
if((now - valve_zone.last.date_parsed) > action_thresh) {
console.log("(" + rec.date + ")" + "actionable temperature record but valve_zone status is too old");
} else if(((valve_zone.mode == "cool") && (rec.temp <= tstat_zone.set_temp)) ||
((valve_zone.mode == "heat") && (rec.temp >= tstat_zone.set_temp))) {
if(valve_zone.last.status == "on") {
console.log("(" + rec.date + ")" + "Should shut off");
if(valve_zone.prev) {
if(last_command && ((now - last_command.date) < command_status_latency_thresh)) {
console.log("(" + rec.date + ")" + "I should maybe turn off but I'm still possibly waiting for a command of mine to settle");
} else if((now - valve_zone.prev.date_parsed) < (valve_zone.min_cycle_minutes * 60 * 1000)) {
console.log("(" + rec.date + ")" + "I should turn off but I'm in the minimum cycle period. My last status " + valve_zone.prev.status + " was " + (now - valve_zone.prev.date_parsed)/(1000 * 60) + " minutes ago ");
} else {
console.log("(" + rec.date + ")" + "My last status " + valve_zone.prev.status + " was " + (now - valve_zone.prev.date_parsed)/(1000 * 60) + " minutes ago ");
last_command = { status: "off", date: now};
send_off(valve_zone.relay_addr, function(er, res) {
if(er) {
throw(er);
}
});
}
} else {
console.log("Should shut off but I don't have a previous status so I don't know how long we've been in this cycle.");
if((now - program_start_time) < command_status_latency_thresh) {
console.log("A previous thermostat could have issued a command");
} else {
console.log("A previous thermostat could not have issued a command. turning it off.");
last_command = { status: "off", date: now};
send_off(valve_zone.relay_addr, function(er, res) {
if(er) {
throw(er);
}
});
}
}
} else {
console.log("(" + rec.date + ")" + "already off");
}
} else if(((valve_zone.mode == "cool") && (rec.temp > tstat_zone.set_temp)) ||
((valve_zone.mode == "heat") && (rec.temp < tstat_zone.set_temp))) {
if(valve_zone.last.status == "off") {
console.log("(" + rec.date + ")" + "Should turn on");
if(valve_zone.prev) {
if(last_command && ((now - last_command.date) < command_status_latency_thresh)) {
console.log("(" + rec.date + ")" + "I should maybe turn on but I'm still possibly waiting for a command of mine to settle");
} else if((now - valve_zone.prev.date_parsed) < (valve_zone.inter_cycle_minutes * 60 * 1000)) {
console.log("(" + rec.date + ")" + "I should turn on but I'm in the inter-cycle hyst period. My last status " + valve_zone.prev.status + " was " + (now - valve_zone.prev.date_parsed)/(1000 * 60) + " minutes ago ");
} else {
console.log("(" + rec.date + ")" + "My last status " + valve_zone.prev.status + " was " + (now - valve_zone.prev.date_parsed)/(1000 * 60) + " minutes ago ");
last_command = { status: "on", date: now};
send_on(valve_zone.relay_addr, function(er, res) {
if(er) {
throw(er);
}
});
}
} else {
console.log("(" + rec.date + ")" + "but there is no previous value for the valve.");
if((now - program_start_time) < command_status_latency_thresh) {
console.log("A previous thermostat could have issued a command");
} else {
console.log("A previous thermostat could not have issued a command. turning it on");
last_command = { status: "on", date: now};
send_on(valve_zone.relay_addr, function(er, res) {
if(er) {
throw(er);
}
});
}
}
} else {
console.log("(" + rec.date + ")" + "already on");
if((now - valve_zone.prev.date_parsed) > (valve_zone.max_cycle_minutes * 60 * 1000)) {
console.log("(" + rec.date + ")" + "already on but I'm past the max cycle period. My last status " + valve_zone.prev.status + " was " + (now - valve_zone.prev.date_parsed)/(1000 * 60) + " minutes ago ");
last_command = { status: "off", date: now};
send_off(valve_zone.relay_addr, function(er, res) {
if(er) {
throw(er);
}
});
}
}
}
if(sense_count == 100) {
if(tstat_zone.can_syndicate) {
journal_message(JSON.stringify({type: "record", date: new Date().toISOString(), temp: tstat_zone.set_temp, addr: tstat_zone.thermostat_addr, syndicate: "66:01" }) + '\n', function(er, res) {
if(er) console.log('failed to journal message');
else console.log('journaled message');
});
}
sense_count = 0;
} else {
sense_count++;
}
}
}
}
} catch(e) {
console.log('BAR: that exploded: ' + lrec + ' ' + e);
}
});
});
});
});
};
if((process.argv.length < 3)) {
console.error("no target");
process.exit(1);
}
switch(process.argv[2]) {
case "ac":
run_sense_tstat_valve(hallway_sense_zone, ac_tstat_zone, ac_valve_zone);
break;
case "heat":
run_sense_tstat_valve(hallway_sense_zone, upstairs_heat_tstat_zone, heat_valve_zone);
break;
case "basement_heat":
run_sense_tstat_valve(basement_kitchen_sense_zone, basement_heat_tstat_zone, basement_heat_valve_zone);
break;
case "kitchen_heat":
run_sense_tstat_valve(kitchen_sense_zone, kitchen_heat_tstat_zone, kitchen_heat_valve_zone);
break;
default:
console.error("bad target");
process.exit(1);
}