Here's how. Make sure the oven is cooled off and not hot to the touch. Create a thick paste using baking soda mixed with water. Spread it on the glass using a sponge, rag, or even your hands (but be sure to put on cleaning gloves first). Let the paste sit for 15 to 20 minutes, depending on how dirty the glass is.