I'm surprised that the low fuel warning light doesn't come on when the key is at ON but the engine isn't yet started, but given the number of confirmations here, I guess that must be correct for the 2nd generation OBs. (It does come on in my 2007.)
The warning light is controlled by a cpu that's part of the combination meter. It measures the resistance of the two fuel tank level sensors, and when the total resistance rises to a predetermined level, the light is turned on.
How interested are you in testing the function of the bulb without risking running out of fuel. I think there is a way. (Appreciate comment on this from others.)
I've attached the wiring diagram for the fuel gauge system; this is for 2002, but I have verified it's the same back to 2000.
The technique would be to lift the rear seat bottom to access the floor hatches to the fuel tank. Remove the left side hatch, which is over the sub-level sensor. Disconnect the two-wire connector (R59 in the diagram) and connect a variable resistor with a range of 0 to 100 Ohms across the connector. (Make sure there's no chance of shorting or grounding any connections.) Then turn on the ignition and increase the variable resistor slowly. As the resistance increases, the fuel gauge should go down. At some point the combined resistance of the variable resistor and the other fuel level sensor will reach the point where the gauge is at E and the light should come on. (The system might delay triggering the light to avoid false positives due to normal fuel shifting whe driving, so the test should be done slowly.)
Explanation: The main fuel level sensor varies from near zero Ohms when full to around 53 Ohms when empty. The sub-level sensor varies from near zero Ohms when full to about 40 Ohms when empty. When the tank is "empty" the total (maximum) resistance of the two sensors in series would be in the 90 to 95 Ohm range. This should be the point where the light is (or has already been) turned on. Assuming that the test is done when the tank in fact is not empty, the main sensor will be somewhere above zero Ohms, so it shouldn't take the full 100 Ohms of the variable resistor for the gauge to drop to E, and for the light to come on.
If the light doesn't come on when the variable resistor is at it's highest resistance (100 Ohms) it might indicate the light isn't working. The bulb would be the first thing to check.