In recent years a lot of data has been generated owing to the exponential increase in internet usage. People are overloaded with information, spanning over multiple domains. This helps people obtain knowledge and come to informed decisions. If we consider purchasing a product as a use case, the buyer can visit multiple websites to find strengths and weaknesses of the product as well as the opinions of other purchasers. To make this process easier and faster for the buyer we propose a robust and scalable hybrid recommendation system which is implemented as a combination of Content Based Recommendation System and Collaborative Filtering Techniques. As a test case, this system has been used to help purchase smartphones based on all features and sentiments of previous purchasers. System gets data using a self-learning web crawler that gathers data from a number of relevant websites irrespective of their different structures. The sentiments of users who purchased the same items previously have also been analyzed to aid the current buyers’ decisions. In this paper, we have provided detailed survey and comparison of multiple techniques we reviewed before implementing each module.

Artificial intelligence, Decision support systems, Knowledge based systems, Hybrid intelligence systems, Data Mining, Web Mining, Supervised Learning, Recommender systems, Content based retrieval, Information retrieval, Clustering algorithms, Classification algorithms, Natural language processing, Sentiment analysis, Recurrent Neural Networks.

[1] P. Srinivasan,F. Menczer, G. Pant“A General Evaluation Framework for Topical Crawlers”, 2004.
[2] Ziyan Zhou and Muntasir Mashuq, “Web Content Extraction Through Machine Learning”, Stanford University.
[3] Amit Gupte, Sourabh Joshi, Pratik Gadgul, Akshay Kadam, “Comparative Study of Classification Algorithms used in Sentiment Analysis”, 2014.
[4] Yun Y., Hooshyar D., Jo J., Lim H., “Developing a hybrid collaborative filtering recommendation system with opinion mining on purchase review.”, 2017.
[5] Wenxian Wang, Xingshu Chen, Yongbin Zou, “A Focused Crawler Based on Naive Bayes Classifier”, Third International Symposium on Intelligent Information Technology and Security Informatics, 2010.
[6] Duygu Taylan, Mitat Poyraz, Selim Akyokuş, Murat Can Ganiz, “Intelligent Focused Crawler: Learning which Links to Crawl”, IEEE, 2011.
[7] Youngki Park, Sungchan Park, Sang-goo Lee, “Fast Collaborative Filtering with a k- Nearest Neighbor Graph”, Journal of Machine Learning Research, 2014.
[8] Claudio Adrian Levinas., “An Analysis of Memory Based Collaborative Filtering Recommender Systems with Improvement Proposals”, 2014.
[9] Koji Miyahara†, Michael J. Pazzani, “Improvement of Collaborative Filtering with the Simple Bayesian Classifier”, 2013.
[10] J. Herlocker, J. A. Konstan, J. Riedl,“An Empirical Analysis of Design Choices in Neighborhood-Based Collaborative Filtering Algorithms”, Information Retrieval, Vol 5, Issue 4, 2002.
[11] Smyth B, “Case-Based Recommendation”, The Adaptive Web. Lecture Notes in Computer Science, Vol. 4321, Springer, Berlin, 2007.
[12] Dima S. Mahmoud, Robert I. John, “Enhanced Content-based Filtering Algorithm using Artificial Bee Colony Optimisation”, SAI Intelligent Systems Conference, London, 2015.
[13] Shivaprasad T K, J. Shetty, “Sentiment Analysis of Product Reviews:A Review”, 2017.
[14] Suiki Li, “Sentiment analysis on Amazon Unlocked mobile phones”, 2017.
[15] Doaa Mohey, El-Din, “Enhancement Bag-of-Words Model for Solving the Challenges of Sentiment Analysis”, 2016.
[16] Xin Rong, “word2vec Parameter Learning Explained”, 2016.
[17] H. Palangi, Li Deng, Y. Shen, J. Gao, Xiaod, “Deep Sentence Embedding Using Long Short-Term Memory Networks: Analysis and Application to Information Retrieval”, 2016.