17 September, 2018
What is Mahout?
Apache Mahout, a project developed by Apache Software Foundation is meant for Machine Learning. It enables machines learn without being overtly programmed.It produces scalable machine learning algorithms, extracts recommendations and relationships from data sets in a simplified way.
Apache Mahout is an open-source project, which is free to use under the Apache license.vIt runs on Hadoop, using the MapReduce paradigm.
With its data Science tools, Mahout enables:
-Frequent item-set mining
What is the History of Apache Mahout? When did it start?
The Mahout project was started by several people involved in the Apache Lucene (open source search) community with an active interest in machine learning and a desire for robust, well-documented, scalable implementations of common machine-learning algorithms for clustering and categorization. The community was initially driven by Ng et al.’s paper “Map-Reduce for Machine Learning on Multicore” (see Resources) but has since evolved to cover much broader machine-learning approaches. Mahout also aims to:
-Build and support a community of users and contributors such that the code outlives any particular contributor’s involvement or any particular company or university’s funding.
-Focus on real-world, practical use cases as opposed to bleeding-edge research or unproven techniques.
-Provide quality documentation and examples.
Interested in mastering Apache Mahout Training? Enroll now for FREE demo on Apache Mahout Online Training.
Although relatively young in open source terms, Mahout already has a large amount of functionality, especially in relation to clustering and CF. Mahout’s primary features are:
-Taste CF. Taste is an open source project for CF started by Sean Owen on SourceForge and donated to Mahout in 2008.
-Several Mapreduce enabled clustering implementations, including k-Means, fuzzy k-Means, Canopy, Dirichlet, and Mean-Shift.
-Distributed Naive Bayes and Complementary Naive Bayes classification implementations.
-Distributed fitness function capabilities for evolutionary programming.
-Matrix and vector libraries.
-Examples of all of the above algorithms.
Unless you are highly proficient in Java, the coding itself is a big overhead. There’s no way around it, if you don’t know it already you are going to need to learn Java and it’s not a language that flows! For R users who are used to seeing their thoughts realized immediately the endless declaration and initialization of objects is going to seem like a drag. For that reason I would recommend sticking with R for any kind of data exploration or prototyping and switching to Mahout as you get closer to production.
The next major version, Mahout 1.0, will contain major changes to the underlying architecture of Mahout, including:
-Scala: In addition to Java, Mahout users will be able to write jobs using the Scala programming language. Scala makes programming math-intensive applications much easier as compared to Java, so developers will be much more effective.
-Spark & h2o: Mahout 0.9 and below relied on MapReduce as an execution engine. With Mahout 1.0, users can choose to run jobs either on Spark or h2o, resulting in a significant performance increase.
Learn more about Apache Mahout Tutorials in this blog post.
The main difference will came from underlying frameworks. In case of Mahout it is Hadoop MapReduce and in case of MLib it is Spark. To be more specific – from the difference in per job overhead If Your ML algorithm mapped to the single MR job – main difference will be only startup overhead, which is dozens of seconds for Hadoop MR, and let say 1 second for Spark. So in case of model training it is not that important.
Things will be different if your algorithm is mapped to many jobs. In this case we will have the same difference on overhead per iteration and it can be game changer.
Let’s assume that we need 100 iterations, each needed 5 seconds of cluster CPU.
-On Spark: it will take 100*5 + 100*1 seconds = 600 seconds.
-On Hadoop: MR (Mahout) it will take 100*5+100*30 = 3500 seconds.
In the same time Hadoop MR is much more mature framework then Spark and if you have a lot of data, and stability is paramount – I would consider Mahout as serious alternative.
-Item-based Collaborative Filtering
-Matrix Factorization with Alternating Least Squares
-Matrix Factorization with Alternating Least Squares on Implicit Feedback
-Complementary Naive Bayes
-Principal Component Analysis
-Latent Dirichlet Allocation
-Frequent Pattern Matching