An approach for a perception system for autonomous vehicle navigation is presented. The approach relies on low-cost electro-optical (EO) sensors for terrain classification, 3D environment modeling, and object/obstacle recognition. Stereo vision is used to generate real-time range maps which are populated into a hybrid probabilistic environment model. Textural and spectral cues are utilized for terrain classification and spatial contextual knowledge is proposed to augment object recognition performance.