Target representation and localization is a central component in visual object tracking. In this paper a new approach for target representation and localization is presented. This approach tackles two of the most important causes of failure in object tracking: changes of object appearance and occlusions. We propose a modified template matching approach which does not require a priori learning of object views. This method allows to track non-rigid objects in real-time by employing a weighted template which is dynamically updated, and a hierarchical framework that integrates all the components of the tracker. Our hierarchical tracker allows tracking of multiple objects with low increase of computational time. The capability of the tracking system to handle occlusions and target distortions is demonstrated for several video sequences.