Pages

Showing posts with label Eclipse. Show all posts
Showing posts with label Eclipse. Show all posts

Saturday, July 25, 2015

How to set up Apache Spark (Java) - MLlib in Eclipse?

Apache Spark version: 1.3.0

Download Apache Spark required pre-built version from the following link:
http://spark.apache.org/downloads.html

Create Maven project in Eclipse
File > New > Maven Project

Add following dependencies in pom.xml.



  org.apache.spark
  spark-mllib_2.10
  1.3.0
  provided
 

org.apache.spark
spark-core_2.10
1.3.0
provided


We have mentioned scope as “provided” as those dependancies are already available in Spark server.

Create new class and add you Java source code for required MLlib algorithm

Run as > Maven Build… > package

Verify .jar file is created in ‘target' folder of Maven project

Change the location to Spark installation you downloaded and unpacked and try following command:
./bin/spark-submit —class --master local[2]
E.g.,

./bin/spark-submit --class fpgrowth --master local[2] /Users/XXX/target/uber-TestMLlib-0.0.1-SNAPSHOT.jar

Tuesday, July 29, 2014

"Updating Maven Project". Unsupported IClasspathEntry kind=4

To fix the issue,

  1. Right click on the project
  2. Maven > Remove maven nature
  3. Right click on the project
  4. Run as > Run configurations
  5. eclipse : clean
  6. Right click on the project
  7. Maven > Convert to maven projects

Sunday, February 2, 2014

Better approach to load resources using relative paths in Java

FileInputStream (Absolute path)


To load a resource file such as x.properties for program use, first thing that we would consider will be specifying the absolute file path as given below:

InputStream input = new FileInputStream("/Users/jwithanawasam/some_dir/src/main/resources/
config.properties”);

However, when ever we moved the project to another location, this path has to be changed, which is not acceptable.

FileInputStream (Relative path)


So, the next option would be to use the relative file path as given below, instead of giving absolute file path:

InputStream input = new FileInputStream("src/main/resources/config.properties”);

This approach seems to solve the above mentioned concern.

However, problem with this is the relative path is depending on the current working directory, which JVM is started. In this scenario, it is "/Users/jwithanawasam/some_dir". But, in a different deployment setting this may change, which leads to change the given relative path accordingly. Moreover, we, as developers do not have much control over JVMs current working directory.


In any of the above cases, we will get java.io.FileNotFoundException error, which is a familiar exception for most java developers.


class.getResourceAsStream


At runtime, JVM checks the class path to locate any user defined  classes and packages. (In Maven, build artifacts and dependancies are stored under path given for M2_REPO class path variable. E.g., /Users/jwithanawasam/.m2/repository) The .jar file which is the deployable unit of the project will be located here.

JVM uses class loader to load java libraries specified in class path.

So, best thing we can do is load the resource specifying a path relative to its class path using class loader. Here, specified relative path will work  irrespective of the actual disk location the package is deployed.

Following methods reads the file using class loader.

InputStream input = Test.class.getResourceAsStream("/config.properties");

Usually, in Java projects resources such as configuration files, images etc. are located in src/main/resources/ path. So, if we add a resource immediately inside this folder, during packaging, the file will be located in the immediate folder in .jar file.

We can verify this using the following command to extract content of jar file:

jar xf someproject.jar

If you place the resources in another sub folder, then you have to specify the path relative to src/main/resources/ path.

So, using this approach we can load resources using relative paths in a hard disk location independent manner. Once we package the application, it is ready to be deployed anywhere, as it it is, without the overhead of having to validate resource file paths, thus improving the portability of the application.

ServletContext.getResourceAsStream for web applications


For web applications, use the following method:

ServletContext context = getServletContext();
    InputStream is = context.getResourceAsStream("/filename.txt");
 
Here, file path is taken relative to your web application folder. (The unzipped version of the .war file)
E.g., mywebapplication.war (unzipped) will have a hierarchy similar to the following.  
 
mywebapplication
    META-INF
    WEB-INF
        classes
   filename.txt
 
So, "/" means the root of this web application folder.  
This method allows servlet containers to make a resource available to a servlet from any location, without using a class loader. 


 

Wednesday, January 29, 2014

Eclipse F5, F6 keyboard shortcuts do not work in Mac OSX?

Option 1
Use F5, F6 with fn key

Option 2
Mac > System Preferences > Keyboard > Keyboard > check "Use all F1, F2 etc. keys as standard function keys


Wednesday, December 18, 2013

How to resolve "import java.neo.file cannot be resolved" error?

Ypu will get the "import java.neo.file cannot be resolved" error with following imports:
import java.nio.file.Files;
import java.nio.file.Paths;

To resolve that do the following:
Right click on the project > Properties > Java Compiler > Set Compiler compliance level as 1.7

Refresh the project

Thursday, November 28, 2013

Install Maven Integration plugin (m2eclipse) for eclipse

  • Open Eclipse
  • Go to Help > Install New Software
  • Under available software, give the following URL as Work with: site
  • http://download.eclipse.org/technology/m2e/releases
  • Hit Enter

Monday, November 25, 2013

How to troubleshoot "Missing artifacts" issues in Maven

Check whether the artifacts given in the pom.xml file matches with the artifacts installed in the local repository (.m2/repositories).

I encountered this error once due to last update date appending to the artifact name as given below:

org.apache.stanbol.enhancer.servicesapi-0.11.0-20131021.093418-197.jar
org.apache.stanbol.enhancer.servicesapi-0.11.0-20131021.093418-197.jar.sha1
org.apache.stanbol.enhancer.servicesapi-0.11.0-20131021.093418-197.pom
org.apache.stanbol.enhancer.servicesapi-0.11.0-20131021.093418-197.pom.sha1
org.apache.stanbol.enhancer.servicesapi-0.11.0-SNAPSHOT.jar
org.apache.stanbol.enhancer.servicesapi-0.11.0-SNAPSHOT.pom

Where as pom.xml expected a different set of dependancies as given below:

   
      org.apache.stanbol
      org.apache.stanbol.enhancer.servicesapi
      0.11.0-SNAPSHOT
   


To resolve the issue, delete all the files in the local repository for the .jar file in question using following command.
rm /*

Then install the artifacts again using the goal clean install.
E.g, in Eclipse right click on the project > Run as > Maven build... > Goal "clean install" > Run

Wednesday, November 6, 2013

How to fix java.net.ConnectException: Connection refused?

A "connection refused" error happens when you attempt to open a TCP connection to an IP address / port where there is nothing currently listening for connections. Check whether the IP and the port connected is correct.

E.g., Connection to http://localhost:8765 refused

Debug Java Applications using JPDA in Eclipse

JVM tools interface defines that program being debugged should provide VM for debugging.

IDEs such as Eclipse, implement JDI (Java Debug Interface). JDI is a high level interface to define information related to remote debug requests.

-Xdebug - Enables debugging information

JDWP (Java Debug Wire protocol): format of debug information and requests (communication channel)
Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n

Xrunjdwp -
Loads JDWP in the target VM (communicate with debugger application)

transport -
The socket being used

server-
y - yes for target application to listen to a debugger to attach

address -
transport address for the connection

suspend -
n - no  for target VM to be suspended until debugger application connects

Remote debugging in Eclipse 

Run > Debug Configurations > Remote Java Application > Create new debug configuration and specify the port (address given above)





java.lang.NoSuchFieldError: DEF_CONTENT_CHARSET in Apache Stanbol Integration Tests

Check for multiple versions of httpcore .jar in the class path and remove if any. If there are duplicates, it reads the one which appears first in your classpath.

- Check classpath in Eclipse: Run > Run Configurations > Classpath

- Leave only project dependencies: Select the project > Edit > Edit Runtime classpath > check "Only include exported entries" > Ok > Run