Computer vision applications for real-world problems include handwriting recognition. Nowadays handwritten character recognition is commonly used for reading postal addresses, bank check amounts, and forms. A few works have also been done for recognizing the Tamil character. The Tamil character syllable is a segment of a Tamil word that has a single vowel sound and is spoken as a single unit. Recently, Tamil character recognition has been implemented through deep learning techniques. The hyperparameters used in the deep learning techniques signify the training algorithms’ behavior, which has an impact on model performance in predicting the characters. One of the major challenges in training the model is overfitting the data. This has to be overcome by choosing the values of hyperparameters appropriately. However, the proposed system uses the bayesian optimization algorithm for tuning the values of the hyperparameters. In addition to the hyperparameter values, the proposed system classifies the given Tamil character using a convolutional neural network. The proposed system performance is assessed using tuned hyperparameters and the standard accuracy metric. The proposed system produces an accuracy of 87.3% when compared to the existing Tamil character recognition system.