Human can easily see and read the document word by word, character by character whether it is written clearly or not. He can also easily guess the next word after the current word. But machine or computer cannot easily predict the handwritten words until they are not trained enough. Optical character recognition is such a system which recognizes the characters both printed and handwritten. It works very well for English characters but the results are not that good for handwritten Devanagari characters, that is why the experimentsare going on to train the machine with new features and classification techniques. In this paper, handwritten Devanagari characters recognition system has been presented using discrete cosine transform zigzag features,geometric features (open end points, intersection points, number of horizontal lines, length of horizontal lines, number of right diagonal lines, length of right diagonal lines, number of vertical lines, length of vertical lines, number of left diagonal lines, length of left diagonal lines, foreground area) and the hue moments features. For classification purpose, linear discriminant analysis (LDA), support vector machine with Quadratic kernel, Subspace Discriminant analysis, Subspace K-nearest neighbours and Weighted K-nearest neighbours techniquesare considered. Authors have created a database of 3877 Devanagari characters from the scanned handwritten documents of the Marathi notebooks of 5th–9th class students which is very young age group to get variety of characters. Authors have achieved 93.6% recognition accuracy using a combination of all the features with Linear Discriminant Analysis (LDA) classifier.