March 25 and 29, April 1 and 5 Meeting Notes
The big tank was delivered this week. It is an 850 gallon fiberglass tank from
Structural North America, which will be our early test model for the full size X-Prize vehicle tank. The plan is that we can have a longer 1600
gallon tank fabricated with carbon fiber, which will have the same mass as the
850 gallon tank in fiberglass. There is
an outside chance that the 850 gallon tank may be the final vehicle tank if
we continue to have problems securing a high volume supplier of 90% peroxide,
we may be forced to a bipropellant solution with a lower concentration peroxide
oxidizer. That would be more
development work and technical risk for us, but it would make the entire
vehicle smaller, and allow the use of the off-the-shelf tanks.
I finally got a quote from Alliant Techsystems, the big
military conglomerate, for carbon filament winding the full size tank. I emphasized that it doesnt need to meet
any mil-specs, it just needs to pass a proof test. I was fearing a completely ridiculous price quote, but it didnt
turn out all that bad. They dont have
liner tooling for anything in our tank size range, but if we can provide the
liners, they could wind them for $15,600 per tank. Great, I thought!
However, they want $40,000 in non-recurring engineering expenses before
the first one is produced. That put a
bit of a damper on the enthusiasm, but if we make two or three tanks, it is
still in the ballpark of what I would be willing to pay. I have a couple more places to try before
nailing down an order.
The hanging weight of the fiberglass tank was 920 pounds,
but 44 pounds of that was in the manway flange, and another dozen pounds or so
in our rigging, so the tank is pretty much right on our expectations they are
all about one pound per gallon of capacity.
Our cabin weighs 200 pounds at the moment, but that doesnt include the
hatch, electronics, and air bottle. The
crush cones are only 37 pounds. The
parachute canopies that we were quoted for (before the vendor got cold feet and
backed out on us) had a weight of around 150 pounds for a fully redundant
system. The big unknown is large engine
thrust to weight. We are specifying
2400 pounds under the parachutes, which assumes three X-Prize standard
passengers, so if we go overweight on our test vehicle, we will just delete
some of the ballast for the phantom passengers.
Last week, one of the major goals was to finalize the
seating arrangements for the other two passengers in the X-Prize cabin, because
once we bond the cabin onto the big tank, working on it is going to be much
more difficult. We made a frame
arrangement in the cabin with two by fours and plywood, and tested entry and
exit for three people, but that was just on our boilerplate 2:1 eliptical tank
end. We werent sure if the filament
wound tank end would have similar contours, so final fabrication was held off
until we could sit it on the tank.
Happily, it still works out fine on the tank the center dome with the
flange goes higher, but the side slope away more, so it still works out. We will be doing final cabin fabrication
with aluminum and honeycomb boards in the coming weeks, then permanently
bonding the cabin to the tank.
We also did some timings of propellant venting through 2
ball valves, which is data we need to determine the dangerous times in the
flight profile where you may not be able to vent enough propellant for a good
parachute landing. A half full 60
gallon tank with 100 psi ullage pressure took five seconds to vent. A ¾ full tank with 200 psi ullage pressure
also took about five seconds to vent.
These numbers should scale linearly to the larger tanks, and by the
square root of pressure, so the 850 gallon tank half full with 400 psi would
take 35 seconds to vent completely. We
also tried putting one of our little cameras actually inside the tank to see if
we could see a swirl pattern as the tank was emptying, but the rapid expansion
cooled things so quickly that a dense fog was formed in the tank as it neared
We are resuming some work on alternate propellant
combinations, because it is looking like we will run completely dry of 90%
peroxide very soon now. We expect to
have more in two months, but that will leave us with enough of a dead spot that
we will want to at least be trying some other options. We are still focusing on lower concentration
hydrogen peroxide as the oxidizer, although we havent completely ruled out a
switch to a different oxidizer. Various
50% peroxide / fuel combinations can still run with uncooled nozzles, and should
be able to hit around 160s Isp at sea level.
70% peroxide / fuel combinations should get about 20 seconds more Isp,
but would require a cooled nozzle / chamber.
Our last set of tests seemed to indicate that 50% peroxide
doesnt decompose very well at all through our silver screen catalyst packs,
and we were unable to get combustion started with a fuel mix post-catalyst
pack. We had enough problems trying to
use solid rocket propellant for an ignition source that upcoming experiments
will use electrical means.
We tried using a finely atomized spray nozzle with a 50%
mixed propellant to see if it could be made to decompose or light in an open
air situation, but that didnt work either.
The next step was to try 50% peroxide with separate injection
of a catalyst fluid dissolved in a fuel.
Calcium permanganate solution was used as a catalyst in some of the
German peroxide mechanisms, reportedly because it is much more soluble in water
than other catalysts. We tried to track
down a supplier, but the two quotes we got were for several hundred dollars for
TEN GRAM quantities! No thanks. We have manganese acetate and potassium permanganate
on hand, which seems plenty reactive enough.
We know from previous tests that catalyst dissolved in fuel
will autoignite with 90% peroxide, but not with anything lower, so we installed
a glow plug between the two spray nozzles as an additional ignition source. The open-air tests didnt do anything
exciting. You could hear the peroxide
mist decomposing in the air, but it evidently didnt happen fast enough, close
enough to the glow plug to get anything started.
The next step will be to try the same tests, but move to lower
flow / finer atomization for the ignition system, and contain everything in a
chamber. If we get that working, then
we can stage in higher flow. We are
currently experimenting with high flow spiral nozzles from Bete, but we will
move to impingement nozzles for the ignition system.
We tried yet again to fly the 2 diameter vehicle, but we
still had problems.
The telemetry problem we were having, that I thought I had
fixed by changing the RF channel, came back.
The problem behavior is that sometimes when the flight computer starts
transmitting the UDP telemetry packets, the telnet TCP connection locks up, and
pings stop getting through. I thought
for sure I had found the root cause when I noticed that there was an IP conflict
between the laptop and one of the internal bridge IP addresses, but that didnt
change a thing. Cutting down the rate or
size of the telemetry packets didnt fix it either, but when I changed the
telemetry packet from a UDP broadcast to a UDP packet only addressed to the
laptop, the problem did disappear, and it didnt show up again the rest of the
day on Saturday. This is probably a
firmware problem in the Esteem bridges, but if the workaround holds, I can live
We got the damaged engine bulkhead fixed up and back in the
vehicle. All of the bolts that broke
off came right out with an easy-out extractor.
I still found it surprising that the bolts broke instead of zippering
through the 1/8 thick filament wound tube, but it sure made the repair
easier. One of the legs is still bent a
fair amount out of shape, but it doesnt hurt anything. We replaced all four manifold-to-engine
hoses. We were very careful not to kink
the short hoses during installation, but we still have some concern about
that. I ordered a set of convolute core
hoses that should be much more flexible (at the expense of more flow losses),
but they were a three week lead time item.
We tested a little 7 LCD from Xenarc with the flight
computer, but we didnt work out a good mounting position for it. We replaced the A/D board again, and removed
the currently unused four port serial board from the PC104 stack to reduce the
number of things we need to worry about.
I tested new abort modes for the things that failed the last
two times crossbow failure, and A/D converter failure. I also added an abort if the vehicle tips
more than 10 degrees.
Because it had always felt like one of the engines wasnt
making nearly as much thrust as the others while warming up, we manually set
all the valves to a specific position by using a drill bit as a feeler gauge
before we installed the hoses and put everything back together. When we powered on the flight computer, I
collected some A/D samples of the current feedback values. I had to do some averaging and conversions,
so I just saved the data off and we went about the testing, but I really wish I
had done it right then, because the data didnt make sense later three of the
valve positions werent even in the calibrated open / closed range. My current theory is that the servo valves
dont have any manual stops, relying only on their limit switches, and we
turned them the wrong way when setting them.
This may also have caused some of the problems we experienced later. This may also explain one of the other
mysteries with the current setup: one of the valve feedbacks is in a higher
range than the others. It may have been
rotated past its limit switch at some point.
I currently clamp the converted throttle positions to the 0.0 to 1.0
range, but I will start letting them go out of range, so we will clearly know
what happened if we see something in the 1 to 0 or 1 to 2 range.
We pulled the vehicle out and lifted up fairly high for the
water test of the new plumbing, but after loading some water and pressurizing, we
found that we had left the master cutoff valve closed again. I had added a warning light for this, but I
hadnt recorded the proper range values to make it accurate yet. Since we knew this position was closed, I
noted that, then we hooked up the manual valve switchbox and opened the master cutoff,
and I noted that value. The warning
light now functions correctly. However,
when we hooked everything back up to the computer and I ran water through all
the engines, engine 0 wouldnt move at all.
We had to take a break to wait for a few tornados and a hail
storm to move past our section of Dallas, so we pulled the vehicle back inside
to do some other work. We initially
thought that we must have bent a pin hooking up the manual switchbox for
opening the cutoff valve, but when we hooked the manual switchbox back up, that
couldnt move valve 0 either.
Eventually, Russ cranked it a little bit with a wrench, and it went back
to working. Seems like a limit switch
issue. Today, I went back over the data
logs (one of the smartest things I have done this project is make the remote
pilot application archive absolutely every telemetry packet, even if it isnt
in flight or manually saved, which makes this type of investigation a whole lot
easier) and was able to determine that valve 0 had stopped working before the
master cutoff had been opened. It
operated fine for the tests we did indoors, but when it closed after the final
test, it ramped down to a below-zero level on the pot feedback. This seems to have caused some problem with
the limit switches. I tried to
replicate this behavior with an older valve servo that I have taken apart at
home, but it didnt seem sensitive to it.
The internals have changed a bit in the more recent KZCO valves to allow
a visual position indicator on top, so the new ones might have an issue the old
After the nasty weather cleared off, we took it back out and
water tested again. Everything worked
properly, but we couldnt tell clearly if the engines were all flowing
correctly. We loaded up 2.5 gallons of
peroxide, hoisted it well off the ground, and slowly warmed the engines
up. Engine 2 was clearly not making as
much thrust as the others during the warmup pulses, but the warm-ups only go to
30% throttle, which is just barely opening for ball valves, so it might have
been a minor difference in initial calibration. After finally clearing all the water from the packs and getting
them up to operating temperature, I slowly throttled up for a liftoff. Unfortunately, the peroxide ran out just as
it was getting light.
When I looked at the telemetry, it showed engine 2, which we
thought was weak, throttling a lot higher than the others, which I took as more
evidence that we had a problem with one of the engines. Now that I look back more closely at the
data, it turns out I was wrong. It wasnt
just one engine going higher, it was a pair of engines going higher to attempt
to counteract the roll of the vehicle as it was hanging on the line under the
crane. I didnt notice the other
engine, trace because it was almost on top of the other one, and I was drawing
that number in yellow, which is very hard to see on the white graph
background. I have since changed yellow
graphs to purple, which shows up a lot better.
My current view of the data is that during throttle up, all the engines
were moving close to the same rate, and there isnt actually any real problem,
which helps explain the rest of our tests.
We did full-system measured water flow tests at both 100%
and 40% throttle. There was no radical
difference in flow rate, but engines 0 and 2 flowed about 20% less than engines
1 and 3. We could not come up with any
reason why engine 2 felt week, but I now believe it to just be an initial
position slightly more closed than the others, making it seem week at the 30%
After the 40% throttle test, I thought that two of the
valves stuck open at 40%, which had us highly confused, suspecting a blown
motor or blown driver board, but when we got it inside, it still responded fine
to the manual control, and when I talked to it on the computer again, it seemed
to be working fine. The really odd
thing is that the stored telemetry shows all the valves closing just as they
were supposed to, so now I am suspecting we may have hallucinated the problem.
In summary, I think that I now understand all of our issues,
and if we had just tried to fly it again on Saturday, it probably would have
The valve with the weirdly out of range feedback position
almost certainly overrated past the limit switch. This could happen on a shock landing, because we default to
leaving the valves commanded to close, relying on the limit switch to stop
them. The limit switches are reed
springs, so if the were banged hard enough to bounce off their contact, the
motor would have started rotating underneath it. We are using the very fastest valves available, which may
contribute to being able to run away from a bounced limit switch. I bet if we manually crank it a few degrees
past close, it will cycle back to the same range as the others.
It seems to be possible to stick the valve at the closed
limit switch. A reasonable argument
could be made that we should completely remove the limit switches, because the
computer can stop it properly with just the existing pot feedback. However, this would make controlling the
valves with the manual switchbox very difficult. Instead, I am going to change the code to have the computer only
go from 5% to 95% of the range of motion, so it should never actually hit the
limit switches in operation, except when we calibrate the valves.
The weak feeling engine is most likely an initial cracking
calibration issue. Instead of taking
everything apart and doing the feeler-gauge job, I will make a little
calibration utility that lets us creep the valves open with the computer, and
someone can just listen for when pressure starts escaping from the tank through
the engine. If we took out the limit
switches, we could also find the final closing point, which would let us
accurately determine the wide-open point halfway between. Instead, I think I am going to just assume
that only the initial calibration varies from engine to engine, while the total
range of motion is the same, which should be the case, given that they use
identical pots, and the valves are inherently 90 degree devices.