August 8, 2004 notes
On Tuesday we did a very successful set of hover tests with
the big vehicle. I had two changes that
I wanted to test: an optional PWM of
the throttle movement to make it change position slower when it was in
hunt-for-an-acceleration mode, and testing a 50% gain increase which I might
enable during high speed flights if it looks like it is having a hard time
controlling the attitude. I had these
set up momentary overrides on the joystick, so I could lift the vehicle up,
engage the change, let go real fast if it isnt working, then try the other
one, all on a single propellant load.
When we tipped the vehicle up, several catalyst rings fell
out of the engine nozzle. We looked up
the engine with a boroscope and found that the screen at the bottom had pulled
past one section of the support plate, allowing some rings to escape. This had also happened on the previous 12
engine after a few runs (you could see a couple red hot catalyst rings fly out
in one of the static test videos). It
didnt seem to be progressive last time, so we went ahead and left it alone,
expecting the test run to squash the rings down into an interference fit again.
Because this was set up to be a 25 second hover (tethered),
which would be our longest hover test, we decided to make this a no-direct-view
test, with my flying it from behind a concrete wall looking at a monitor
instead of directly viewing it. The
engine warmed up fine and lifted off and hovered fine. I was about to engage the first test when
the vehicle just set itself back down on the ground. It took me a few moments to figure out what happened I had
moved the computer and wireless antenna behind the wall with me, so the
telemetry link was very ratty, dropping quite a few packets. Eventually it dropped enough in a row to hit
the internal limit and triggered a loss-of-telemetry abort, which is an auto
I moved the antenna back in view of the vehicle, and we
completed both of the control system tests without incident. We used our new propellant disposal burner
to catalyze the remaining propellant, which worked pretty well. The foam coming out was probably still 10%
peroxide or so, but a little water was fine for washing it away. We might consider adding a spark ignition
system to it so it would completely burn everything away, but that would be a
more complex system, and would leave us with a red hot propellant burner.
When we set the vehicle back down on the cradle, a few more
catalyst rings came out, but the engine still seemed to be working perfectly.
Based on these results, I changed the flight control code to
use the PWM valve movement when it is hunting back and forth past a desired acceleration. If it hasnt crossed it in 500 msec, or the
desired valve position is fully open or closed, it goes back to full speed.
We also weighed the vehicle, and surprisingly found it
lighter than we had estimated, right at 1000 pounds.
Complete Loss of Vehicle
Saturday was a perfect day for flying, so we went out to the
100 acres for a boosted hop. We had
high expectations for success, since the vehicle had been operating perfectly
on all tests so far.
After we loaded up the propellant and pressurized the
vehicle, we ran into a problem. When I
opened it up to 20% throttle for the warmup it looked like it cleared up fine,
but the telemetry was only reading 100C, as if the hot pack hadnt started
heating. We were a long way from the
vehicle, so we couldnt really tell what was going on. I gave it a bunch of slugs of propellant
until it finally started going up in temperature properly, but we had blown a
lot of propellant out on the ground.
It finally reached operating temperature and we
launched. We had only been operating
this engine at hover thrust levels, so we had been a little concerned that it
might be rough at full throttle. It
was. It flew fine through the
roughness, but when it started to throttle down after the two second boost to a
0.5 G positive acceleration level for the stabilization phase, the rough pulses
kept passing both above and below the desired acceleration, keeping the engine
from throttling down at full speed, resulting in it going a lot higher than intended
(just under 600 feet high). It did
finally get out of the rough stability zone into clear stabilization, but a
couple seconds later, everything got quiet.
It ran out of propellant.
It had not hit apogee yet, so the unstable vehicle immediately
started rotating, hitting about 50 degrees / second. If the vehicle had been past apogee when it ran out, it probably
would have just dropped feet first.
We had telemetry all the way to the time of impact, which
matched the video perfectly, landing eight meters from the launch point. The vehicle hit the ground basically
sideways, a little tail first. The bottom
manway flange broke off the tank, and the 450 pound tank with 180 psi pressure
still in it got punted about 200 yards away by the gas release. $35,000 of rocket is now a whole lot of
Droppings. There are a few pipe
fittings that survived, but thats about it. Amazingly, even though the on-board camera was destroyed, the tape
did survive with only some scuffed sections.
Its a good thing Doom 3 is selling very well
From analyzing the telemetry (integrating the chamber
pressure during the flight), it looks like it wasted two thirds of the
propellant on the warmup. If it had
lifted off with a normal warmup, it would have landed ok even with the rough
throttling, but we would have been in violation of the 15 second burn time
limit by the time it landed. There was
twice as much propellant loaded as this flight should have required, which I
thought was enough to cover any off-nominal conditions, but we obviously should
have scrubbed when the warmup didnt catch after the second or third try. We are going to look into getting a continuous
capacitive level sensor next time so we can have a firm no-go line for
liftoff. If anyone knows of a peroxide
compatible (316 SS / Teflon / viton / eetc) capacitive sensor that runs off of
12v or 5v DC and can handle 300 psi (we may be willing to run past rated
pressure if nexessary), let me know.
Ideally we would want a 5V or 10V analog out, but we could live with a
current sensor, or (with some begrudging) a serial port. We would like to mount it on the bottom of
the tank instead of the conventional top location, but we dont think that will
be a problem.
The failure did give us some demonstration data that we
always sort of wanted to get (but not that bad). The vehicle is absolutely, positively, NOT going to continue flying
nose first when it loses active control.
This should be blatantly obvious from the CG, but we had a WSMR engineer
pushing us towards a NASA consultant to prove it. When it fails in the air, it just drops like a rock, landing very
near the launch site. Rupturing a
fiberglass tank doesnt produce shrapnel, but it does drop kick the tank pretty
good. This looked pretty close to an
optimal 45 degree launch angle for the tank, so we have a pretty good idea what
our safe distances should be.
We probably would have been able to save the vehicle if we
had a rocket drawn parachute on board, but we are trying to have a pyro-free
vehicle. A pneumatic drogue cannon
might have been able to deploy a chute fast enough, but it would be a lot more
We cut the engine open with the plasma cutter to do a
post-mortem, and found what had been causing the engine issues. The combination of the bottom catalyst
retaining plate bowing down because it was only welded on the bottom and some
catalyst escaping both out the bottom and some out the top (the top screen was
burned through in a couple places) left the bottom catalyst not even completely
covering the diameter of the engine.
When we had the nozzle and cold pack cut off and the engine on its side,
you could see right through the hot pack at the top. This explains the apparently clear exhaust at the start while the
thermocouple was still reading only 100C, because the thermocouple was fairly
short (we used to use a longer one, but the bowing of the retaining plate
forced us to use a shorter one so we could still insert it) so it was in a
stream around the edges that bypassed most or all of the hot pack catalyst
(driving down the highway probably also settled the catalyst on the opposite
side from the sensors), while much of the main flow was still being
burned. The loosening catalyst is also
almost certainly why this engine got rough after we had been using it for a
The support plate bowing can be fixed by either making a
full depth angle on the sides of the plate so the weld gets full side coverage,
or actually weld the plate between two chamber sections, instead of inside a
single chamber section. We are making
new plates that are made with 1300 quarter inch holes instead of large water
jet cut squares that are bridged by screens.
This will let us completely avoid the screens altogether, and we are
also going to tie the top and bottom plates around the hot pack together by putting
quarter inch bolts through some of the quarter inch holes, and welding them
together as a unit with the catalyst in between. This should fix the engine behavior.
Everything else operated perfectly, so we still feel good about
the general configuration, but we have a number of improvements for robustness
and operability that we will be making in the next vehicle we put together. A couple of the necessary items are fairly
long lead times, so we are probably grounded for five weeks.