Alloys of different materials are extensively used in many fields of our day-to-day life. Several studies are performed at a microscopic level to analyze the properties of such alloys. Manually evaluating these microscopic structures (microstructures) can be time-consuming. This thesis attempts to build different models that can automate the identification of an alloy from its microstructure. All the models were developed, with various supervised and unsupervised machine learning algorithms, and results of all the models were compared. The best accuracy of 92.01 ± 0.54% and 94.31 ± 0.59% was achieved, for identifying the type of an alloy from its microstructure (Task 1) and classifying the microstructure as belonging to either Ferrous, Non-Ferrous or Others class (Task 2), respectively. The model, which gave the best accuracy, was then used to build an Image Search Engine (ISE) that can predict the type of an alloy from its microstructure, search the microstructures by different keywords and search for visually similar microstructures.