Error: mdb_status == 0 (2 vs. 0) No such file or directory

Error: 

mdb_status == 0 (2 vs. 0) No such file or directory

Solution:

check for lambda files location in prototext file and if files are there. 

./data/mnist/get_mnist.sh

./examples/mnist/create_mnist.sh

Library not loaded: /usr/local/opt/libpng/lib/libpng16.16.dylib

Referenced from: /usr/local/lib/libopencv_highgui.2.4.dylib

Reason: Incompatible library version: libopencv_highgui.2.4.dylib requires version 37.0.0 or later, but libpng16.16.dylib provides version 34.0.0

make: *** [runtest] Trace/BPT trap: 5

solution:

brew reinstall libpng

Error: The "brew link" step did not complete successfully

Error: 

The `brew link` step did not complete successfully

The formula built, but is not symlinked into /usr/local

Could not symlink include/google

/usr/local/include is not writable.

solution:

Permission issue 

sudo chown -R 'jwithanawasam':admin /usr/local/lib

sudo chown -R 'jwithanawasam':admin /usr/local/include

sudo chown -R 'jwithanawasam':admin /usr/local/share

sudo chown -R 'jwithanawasam':admin /usr/local/bin

You can try again using:

brew link protobuf

Error: No nodes loaded. Quitting (on error).

Command: netconvert --node-files=hello.nod.xml --edge-files=hello.edg.xml --output-file=hello.net.xml

Error: No nodes loaded. Quitting (on error).

Solution:
Check whether the hello.nod.xml file is corrupted. Check for formatting issues.

Reminiscence of writing a book on Machine Learning: Challenges, Lessons Learnt, Experiences and Insights...

I have been working on writing a book on Machine Learning, namely “Apache Mahout Essentials" for about 6 months, which was published recently by Packt Publishing - UK.

I’m sharing my experience in this article, as it may help others who want to pursue the same.

So, I got an invitation to write a book, what’s next?

When I got an email from Shaon (Acquisition Editor at Packt Publishing) to write a book, I immediately replied saying that I’m currently occupied (if not overloaded) with MSc and office work and I won’t be able to do that. Then, Shaon again approached me saying they can give flexible timelines for chapter deliverables and asked me to give it a second thought.

Then I spoke to Abi, with three possible options in my hand and one was “not writing the book” which she straight away eliminated saying that “even writing a bed time story book itself is something she won’t miss out”.

Also, I spoke to Rafa, who was the Head of Research at Zaizi sometime back. He assured that I can do this and gave an advice which was just three words but helped me vastly though out the journey of writing the book. “Step by step!”

So, I want to emphasis the fact  that, even though I’m getting some recognition on writing a book, if it wasn’t for these people it would have been just a rejected invitation. I have no words to explain my gratitude for them for the motivation they provided.

From my side, the steady and compelling reason to start writing this book is my unquenchable curiosity about machine learning and the desire to learn more.

Yup, decided to go ahead and try out, But still…

So, I started writing and within no time I realised that this is not as easy as I imagined.

One reason is I was following MSc in Artificial Intelligence where we had to follow 4 modules in 8 weeks (and the following week exams! - no study leave) and we had lectures during entire week end 8-5 (Those who went through this struggle will realise the pain ;)). Apart from that, I was working full time as well. To make the situation even worse, I had to travel for 2 hours daily as I stayed out of Colombo.

So, I decided to utilise the travel time effectively and I was reading the required content using my smart phone even if I’m standing in a crowded train. There was a time which I worked almost all the hours continuously. As a result, I got stressed out and most of the time I was sick.

This is where "focusing one thing at a time" helped me, as it was so overwhelming to think about all the items in my “things-to-do” list. Also, I planned out the structure and the content before start writing, with fresh mindset. And then I spent all night before the deadline finalising everything.

However, regardless of the problems that came along my way, I was determined to complete what I started. I remember one day I was having a terrible ear infection and still I was struggling to meet a chapter deadline until 3 a.m.

Shaon and Nikhil (Content editor at Packt Publishing) were working with me during this time and they were kind enough to give me flexible chapter deadlines which will not overlap with my university exams.

Finally, it all worth the effort!

The book went through several stages of reviews/ revisions etc. before publishing and the happiest of all was the time I completed all the first drafts.
And the next may be getting the opportunity to decide an image with n-Shades of Grey as the cover page. ;)

Reading has been my favourite and consistent hobby since my childhood, yet I was unaware of the publishing process a book has to go through before it reaches reader’s hands. So, getting to know the process itself was another exciting factor.

In addition to learning and writing about ML concepts, planning out on how to structure and present the content to ensure others can understand was a novel experience as well.

Finally, writing a book was one of the bucket list item in my life and it turned out to be immensely rewarding that exceeded my expectations.

However, this is just one milestone in the long journey of machine learning. There is lot to learn, lot to experience and lot of things that needs to get better :)

Recap on WiML and NIPS 2015

I presented in WiML 2015 and attended NIPS 2015 which was held in Canada. I thought of sharing my experience in this blog on that. I know it's little late, but better late than never... ;)

 WiML was founded by two women researchers (Hanna Wallach and Jenn Wartmen) from Microsoft research when there were sharing room for NIPS. Very few women engage in machine learning research (Specially in our region, very few women engage in machine learning research when compared to men). WiML was form to provide women machine learning researchers an opportunity to collaborate and share their research experiences.

Few quick facts on WiML: 

• Support network for women researchers 
• Share knowledge about their research work
• Initiated for Grass hopper conference - proposal for grass hopper session 
• Co located with Grace hopper conference (women in Computing)
• 2008 co located with NIPS

What did I present there?

In WiML, I presented an approach to analyse and retrieve different content forms such as image, video, text, etc., which are embedded in different content forms in a collective manner. I have given more information on this in the link given below:

http://jayaniwithanawasam.blogspot.com/2012/12/content-extraction-and-context.html

Few notes I took from WiML Invited Talks

The slides from the speakers for the invited talks are available at: http://www.thetalkingmachines.com/blog/2016/1/15/real-human-actions-and-women-in-machine-learning

Super human multi tasking  - Raia Hadsell  - DeepMind

• Games as platform to implement and test AI applications
• Why? difficult and interesting for humans/ huge variety of games/  built in evaluation criteria and reward 
• Atari 2600 games
• Reinforcement Learning 
• Deep Q- Learning
• Knowledge/ policy distillation - model distillation (model compression/ compress the knowledge in an ensemble into single model) 
• Create intelligent agents that can learn many tasks > multiple Atari games 

Structured data/ facts at scale (and bit of machine learning at Google) - Corinna Cortes - Google Research 

• Structured snippets - Extracting structure from unstructured content - Less clicking, more convenient 
• Problem - How do we find good tables on the web?
• Feature design - 
• semantics of the table is often determined by surrounding text
• detecting subject columns - other columns contain properties of the subject
• Determining column classes using Google knowledge graph 

Is it all in the phrasing? - Lillian Lee - Cornell University

• Does phrasing affects memorability? 
• Memorable and non-memorable movie quotes
• Memorable quotes use less common word choices
• Memorable quotes tend to be more general in ways that make them easy to apply in new contexts
• “These aren’t the droids you are looking for” :)
• http://www.cs.cornell.edu/~cristian/memorability.html 

Interactive and Interpretable Machine Learning Models for Human Machine Collaboration - Been Kim, AI2/University of Washington

• Communication from machine to human - provide intuitive explanation 
• Basin case model - proto type and subspaces to help humans understand machine learning results
• BCM on recipe data
• Subspaces, the sets of features that play important roles in the characterization of the prototypes
• Learns prototypes, the ``quintessential observations that best represent clusters in a dataset
• Prototype clustering and subspace learning. In this model, the prototype is the exemplar that is most representative of the cluster.

Other events I attended

• Lean in Circles
• Dedicated to helping all women achieve their ambitions.
• Founded by Sheryl Sandersberg - COO Facebook
• Nvidia 
• GPU computing/ speed up deep learning matrix calculations 
• NVidia digits - interactive deep learning GPU training system 
• Demo that shows how GPUs can speed up training operation in deep neural networks 
• Career advice session 
• Helpful not specifically for machine learning but for any career 
• http://wiml2015.weebly.com/list-of-careeradvice-tables.html

Finally NIPS!

NIPS is one of the top machine learning conferences in the world. I have mentioned few important deep learning techniques that got highlighted in the conference.

CNN  

• Recognise images, used in computer vision 
• Object proposal generation, image segmentation 
• Feed forward neural networks 

RNN 

• Networks with recurrent connections which forms circles (signals travelling in both directions)
• Used in NLP
• Designed to recognise sequences such as speech signal or text 
• Process arbitrary sequence of input 
• Speech recognition, hand writing recognition 
• LTSM - question answering 
• Type of RNN
• LTSM outperforms other sequence learning methods such as conventional RNNs and HMMs 
• Grammer as a foreign language 

So, that’s it for now. :) I might write a detailed blog on NIPS, if I get some free time in future. NIPS is somewhat overwhelming and I need to go through the ideas presented there again to have a clear grasp on cutting edge technologies in machine learning.

I have given some thoughts on this in Zaizi blog as well. 

http://www.zaizi.com/blog/wiml-and-nips-conference-montreal-2015

Also, I gave my thoughts on 2nd Colombo Machine Intelligence Meetup which was held in WSO2 on Feb, 2016.

http://www.meetup.com/colombo-machine-intelligence/events/228052923/

I have given my slides at, 

http://www.slideshare.net/JayaniWithanawasam/thoughts-from-wiml-2015-and-nips-2015

Issues while setting up Jade (Java Agent Development Framework)

Use the following command to avoind the errors given below:

java -cp "jade.jar:(path to your classes)" jade.Boot -agents nickName:(fully qualified name for the agent class E.g., packageName:className)

E.g.,
java -cp "jade.jar:Control-0.0.1-SNAPSHOT.jar" jade.Boot -agents buy:Examples.BookBuyerAgent

Possible errors due to issues in class path or class name:

Error creating the Profile [Can't load properties: Cannot find file buyer:BookBuyerAgent]
jade.core.ProfileException: Can't load properties: Cannot find file buyer:BookBuyerAgent
    at jade.core.ProfileImpl.(ProfileImpl.java:129)
    at jade.Boot.main(Boot.java:76)

jade.Boot: No such file or directory

SEVERE: Cannot create agent buyer: Class BookBuyerAgent for agent ( agent-identifier :name buyer@172.20.10.2:1099/JADE ) not found - Caused by:  BookBuyerAgent

Issues when setting up DeepLearning4J in Mac OSX

I got the following errors when I was trying out DeepLearning4J example with Deep Belief Nets (DBNs) in Mac OSX 10.10 (Yosemite)

It is being said that Jblas is already available in Mac OSX, but still I got some errors with that. 

Jblas is a pre-requisite for setting up DeepLearning4J. 

DeepLearning4J uses ND4J to enable scientific computing with N-Dimentional arrays for Java. ND4J works on several backend linear algebra libraries (execution support with CPU or GPU). Jblas is one Java backend used in  DeepLearning4J for the required matrix operations. 

NoAvailableBackendException ND4J 

Solution: Add the following dependancy

org.nd4j

nd4j-jblas

0.4-rc0

java.lang.ClassNotFoundException: org.jblas.NativeBlas 

Solution: Add the following dependancy

  org.jblas

  jblas

  1.2.4

Latent Dirichlet Allocation (LDA) with Apache Spark MLlib

Latent Dirichlet allocation is an scalable machine learning algorithm for topic annotation or topic modelling. It is available in Apache Spark MLlib. I will not explain the internals of the algorithm in detail here.

Please visit the following link for more information about LDA algorithm.
http://jayaniwithanawasam.blogspot.com/2013/12/infer-topics-for-documents-using-latent.html

Here’s the code for LDA algorithm in Spark MLlib.
import scala.Tuple2;

import org.apache.spark.api.java.*;

import org.apache.spark.api.java.function.Function;

import org.apache.spark.mllib.clustering.DistributedLDAModel;

import org.apache.spark.mllib.clustering.LDA;

import org.apache.spark.mllib.linalg.Vector;

import org.apache.spark.mllib.linalg.Vectors;

import org.apache.spark.SparkConf;

public class lda {

  public static void main(String[] args) {



// Spark configuration details

    SparkConf conf = new SparkConf().setAppName("LDA");

    JavaSparkContext sc = new JavaSparkContext(conf);



    // Load and parse the data (sample_lda_data.txt is available with Spark installation)

    // word count vectors (columns: terms [vocabulary], rows [documents])

    String path = "data/mllib/sample_lda_data.txt";

   

    // Read data

    // creates a RDD with each line as an element

    // E.g., 1 2 6 0 2 3 1 1 0 0 3

    JavaRDD data = sc.textFile(path);

   

    // Map is a transformation that passes each data set element through a function

    // It returns a new RDD representing the results

    // Prepares input as numerical representation

    JavaRDD parsedData = data.map(

        new Function() {


public Vector call(String s) {

            String[] sarray = s.trim().split(" ");

            double[] values = new double[sarray.length];

            for (int i = 0; i < sarray.length; i++)

              values[i] = Double.parseDouble(sarray[i]);

            return Vectors.dense(values);

          }

        }

    );

   

    // Index documents with unique IDs

    // The transformation 'zipWithIndex' provides a stable indexing, numbering each element in its original order.

    JavaPairRDD corpus = JavaPairRDD.fromJavaRDD(parsedData.zipWithIndex().map(

        new Function, Tuple2>() {



public Tuple2 call(Tuple2 doc_id) {

            return doc_id.swap();

          }

        }

    ));

    corpus.cache();



    // Cluster the documents into three topics using LDA

    // number of topics = 3

    DistributedLDAModel ldaModel = new LDA().setK(3).run(corpus);



    // Topic and its term distribution

    // columns = 3 topics/ rows = terms (vocabulary)

    System.out.println("Topic-Term distribution: \n" + ldaModel.topicsMatrix());

    // document and its topic distribution

    // [(doc ID: [topic 1, topic 2, topic3]), (doc ID: ...]

    JavaRDD> topicDist = ldaModel.topicDistributions().toJavaRDD();

    System.out.println("Document-Topic distribution: \n" + topicDist.collect());

    sc.close();

  }

}

Output:

Topic-Term distribution

Document-Topic distribution

Market Basket Analysis with Apache Spark MLlib FP-Growth

Market Basket Analysis 

source: http://www.noahdatatech.com/solutions/predictive-analytics/

Market basket analysis is identifying items in the supermarket which customers are more likely to buy together.
e.g., Customers who bought pampers also bought beer

      
This is important for super markets to arrange their items in a consumer convenient manner as well as to come up with promotions taking item affinity in to consideration.

Frequent Item set Mining and Association Rule Learning  

Frequent item set mining is a sub area in data mining that focuses on identifying frequently co-occuring items. Once, the frequent item set is ready, we can come up with rules         to derive association between items.
        e.g., Frequent item set = {pampers, beer, milk}, association rule = {pampers, milk ---> beer}

        There are two possible popular approaches for frequent item set mining and association rule learning as given below:

Apriori algorithm 
FP-Growth algorithm

To explain above algorithms, let us consider example with 4 customers making 4 transactions in supermarket that contain 7 items in total as given below:

    Transaction 1: Jana’s purchase: egg, beer, pampers, milk
    Transaction 2: Abi’s purchase: carrot, milk, pampers, beer
    Transaction 3: Mahesha’s purchase: perfume, tissues, carrot
    Transaction 4: Jayani’s purchase: perfume, pampers, beer

    Item index
    1: egg, 2: beer, 3: pampers, 4: carrot, 5: milk, 6: perfume, 7: tissues

Using Apriori algorithm

Apriori algorithm identifies frequent item sets by starting individual items and  extending item set by one at a time. This is known as candidate generation step.
This algorithm makes the assumption that any sub set of item within a frequent item set is also frequent.

Transaction: Items
1: 1, 2, 3, 5
2: 4, 5, 3, 2
3: 6, 7, 4
4: 6, 3, 2

Minimum Support 

Minimum support is used to prune the associations that are less frequent.

Minimum support = number of times item occur in transactions/ number of transactions

For example, lets say we define minimum support as 0.5.
Calculating support for egg is 1/4 = 0.25 (0.25 < 0.5), so that is eliminated. Support for beer is 3/4 = 0.75 (0.75 > 0.5) is it is considered for further processing.

Calculation of support for all items

size of the candidate itemset = 1

itemset: support
1: 0.25: eliminated
2: 0.75
3: 0.75
4: 0.5
5: 0.5
6: 0.5
7: 0.25: eliminated

remaining items: 2, 3, 4, 5, 6

extend candidate itemset by 1
size of the items = 2

itemset: support
2, 3: 0.75
2, 4: 0.25: eliminated
2, 5: 0.5
2, 6: 0.25: eliminated
3, 4: 0.25: eliminated
3, 5: 0.5
3, 6: 0.25: eliminated
4, 5: 0.25: eliminated
4, 6: 0.25: eliminated
5, 6: 0.25: eliminated

remaining items: {2,3},{ 2, 5}, {3, 5}

extend candidate itemset by 1
size of the items = 3

2, 3, 5: 0.5

Using FP-Growth algorithm

In FP-Growth algorithm, frequent patterns are mined using a tree approach (construction of Frequent Patter Tree)
FP-Growth algorithm has been proven to execute much faster than the Apriori algorithm.

Calculate support for frequent items and sort in degreasing order of the frequency as given below:

item: frequency
1: 1 - eliminated
2: 3
3: 3
4: 2
5: 2
6: 2
7: 1 - eliminated

Decreasing order of the frequency
2 (3), 3 (3), 4 (2), 5 (2), 6 (2)

Construction of FP-Tree

A) Transaction 1
 1, 2, 3, 5 > 2 (1), 3 (1), 5 (1)

B) Transaction 2
4, 5, 3, 2 > 2 (2), 3 (2), 4 (1), 5 (1)

C) Transaction 3
6, 7, 4 > 4 (1), 6 (1)

D) Transaction 4
6, 3, 2 > 2 (3), 3 (3), 6 (1)

Once the FP-tree is constructed, frequent item sets are calculated using depth first strategy along with divide and conquer mechanism.
This enables algorithm is be computationally more effective and parallelizable (using map-reduce).

Code Example with Apache Spark MLlib

    public static void main(String[] args) {
   
        SparkConf conf = new SparkConf().setAppName("Market Basket Analysis");
        JavaSparkContext sc = new JavaSparkContext(conf);
       
        // Items
        String item1 = "egg";
        String item2 = "beer";
        String item3 = "pampers";
        String item4 = "carrot";
        String item5 = "milk";
        String item6 = "perfume";
        String item7 = "tissues";
       
        // Transactions
        List transaction1 = new ArrayList();
        transaction1.add(item1);
        transaction1.add(item2);
        transaction1.add(item3);
        transaction1.add(item5);

        List transaction2 = new ArrayList();
        transaction2.add(item4);
        transaction2.add(item5);
        transaction2.add(item3);
        transaction2.add(item2);
       
        List transaction3 = new ArrayList();
        transaction3.add(item6);
        transaction3.add(item7);
        transaction3.add(item4);
       
        List transaction4 = new ArrayList();
        transaction4.add(item6);
        transaction4.add(item3);
        transaction4.add(item2);
       
        List> transactions = new ArrayList>();
        transactions.add(transaction1);
        transactions.add(transaction2);
        transactions.add(transaction3);
        transactions.add(transaction4);
       
        // Make transaction collection parallel with Spark
        JavaRDD> transactionsRDD = sc.parallelize(transactions);

        // Set configurations for FP-Growth
        FPGrowth fpg = new FPGrowth()
          .setMinSupport(0.5)
          .setNumPartitions(10);
       
        // Generate model
        FPGrowthModel model = fpg.run(transactionsRDD);

        // Display frequently co-occuring items
        for (FPGrowth.FreqItemset itemset: model.freqItemsets().toJavaRDD().collect()) {
           System.out.println("[" + Joiner.on(",").join(itemset.javaItems()) + "], " + itemset.freq());
        }
        sc.close();
    }