A thermodynamics-based Vehicle Thermal Management System (VTMS) model for a heavy-duty, off-road vehicle with a series hybrid electric powertrain is developed to analyze the thermal behavior of the powertrain system and investigate the power consumption under different vehicle driving conditions. The simulation approach consists of two steps: first, a Series Hybrid Electric Vehicle (SHEV) powertrain is modeled; the output data of the powertrain system simulation are then fed into a cooling system model to provide the operating conditions of the powertrain components. Guidelines for VTMS configuration was developed based on the vehicle simulation results and the operating conditions of powertrain components. Based on the guidelines, a VTMS configuration for the hybrid vehicle was created and used for designs of experiments to identify the factors that affect the performance and power consumption of each cooling system. Design space exploration techniques are then applied to investigate trade-offs and determine near-optimal size of components such that power consumed by fans and pumps is minimized. Finally, gradient-based optimization is used to fine-tune the component sizing subject to performance and geometry constraints. The cooling system design study demonstrates that the configuration and sizing of an SHEV cooling system is different from that of a conventional cooling system because of additional heat sources, increased complexity of component operations and interactions, and the dependency of parasitic power consumption on driving modes.