File Operations in Camel (ftp,file components)

In this tutorial, we are going to learn how to do file operations with Apache Camel. We will use two camel components for this purpose – File and Ftp. Generally, File component is used for local file input/output operations and Ftp for remote file operations as an ftp client for Ftp servers.

Example Scenario:

Case 1 : File component

We have an input folder which has a sub-folder and files with different extensions. We want to copy all the text files from it and its sub folders to another folder “output” and  rename them and change their contents.

Case 2: Ftp component

We want to copy a file from our local to a remote ftp server with few configurations for ftp connection.


Project artifacts:

1. Technologies used –

a. Eclipse IDE for Java EE developer 4.5.0.20150621-1200 (Mars)
b. Preinstalled Maven with Eclipse Mars
c. Camel 2.16.0
d. Java 1.8

2. Eclipse project folder structure:

CamelFileOperationsProjStruc

3. Jars Used

If you are not using Maven, here is the list of jars you would need for this project:

camel-core-2.16.0.jar
slf4j-api-1.6.6.jar
jaxb-core-2.2.11.jar
jaxb-impl-2.2.11.jar
camel-spring-2.16.0.jar
spring-context-4.1.6.RELEASE.jar
spring-beans-4.1.6.RELEASE.jar
spring-core-4.1.6.RELEASE.jar
commons-logging-1.2.jar
spring-expression-4.1.6.RELEASE.jar
spring-aop-4.1.6.RELEASE.jar
aopalliance-1.0.jar
spring-tx-4.1.6.RELEASE.jar
commons-net-2.0.jar
camel-ftp-2.16.0.jar
jsch-0.1.53.jar

4. Maven Dependencies

file:pom.xml . Paste the below content to pom.xml. It has camel and spring dependencies. It can pass two arguments – “java” for running camel in java DSL route and “xml” for running as Spring routes.

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>com.javanbeyond</groupId>
	<artifactId>CamelFileOperations</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<dependencies>
		<dependency>
			<groupId>org.apache.camel</groupId>
			<artifactId>camel-core</artifactId>
			<version>2.16.0</version>
		</dependency>
		<dependency>
			<groupId>org.apache.camel</groupId>
			<artifactId>camel-spring</artifactId>
			<version>2.16.0</version>
		</dependency>
		<dependency>
			<groupId>commons-net</groupId>
			<artifactId>commons-net</artifactId>
			<version>2.0</version>
		</dependency>
		<dependency>
			<groupId>org.apache.camel</groupId>
			<artifactId>camel-ftp</artifactId>
			<version>2.16.0</version>
		</dependency>
	</dependencies>
	<build>
		<plugins>
			<plugin>
				<groupId>org.codehaus.mojo</groupId>
				<artifactId>exec-maven-plugin</artifactId>
				<version>1.1.1</version>
				<executions>
					<execution>
						<phase>test</phase>
						<goals>
							<goal>java</goal>
						</goals>
						<configuration>
							<mainClass>com.javanbeyond.MainClass</mainClass>
							<arguments>
								<!--<argument>java</argument>  -->
								<argument>xml</argument>
							</arguments>
						</configuration>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</build>
</project>

5. Main Class

file: MainClass.java . Main class is our starting point and is called by maven. It can run Java DSL or Spring DSL based on argument passed. Also, we have to comment the line based on the component we don’t want to run.

package com.javanbeyond;

import org.apache.camel.CamelContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class MainClass {

  public static void main(String[] argsthrows Exception {
    if (args.length < 1) {
      System.err.println("Please pass java or xml as argument");
      System.exit(1);
    }
    // Pass route id to execute the particular route
    if (args[0].equalsIgnoreCase("java")) {
      // executeJavaDSL("executeFileComponent");
      executeJavaDSL("executeFtpComponent");
    else if (args[0].equalsIgnoreCase("xml")) {
      // executeSpringDSL("executeFileComponent");
      executeSpringDSL("executeFtpComponent");

    else {
      System.out.println("Argument not supported.");
      System.exit(1);
    }
    // Wait 10 seconds for camel to finish file operation
    Thread.sleep(10000);

  }

  @SuppressWarnings("resource")
  public static void executeSpringDSL(String routeIdthrows Exception {
    new ClassPathXmlApplicationContext("camel-context.xml")
        .getBean("camel", CamelContext.class).startRoute(routeId);

  }

  public static void executeJavaDSL(String routeIdthrows Exception {
    new CamelJavaDSL().getCamelContext().startRoute(routeId);
  }

}

6a. Java DSL class

file: CamelJavaDSL.java . The class where Camel routes are defined  for different components. In our case, file and ftp.

Case 1: “executeFileComponent

I have used camel file expressions to rename the output files and camel simple expressions to change the content of each file.

Case 2: “executeFtpComponent”

I have used filegenie server as a remote ftp server. You need to register in their site http://www.filegenie.com/ before you can use it for ftp connections. At the time of writing, storage less than 20 MB can be used freely. Use your userid and password provided by them through email.

package com.javanbeyond;

import org.apache.camel.CamelContext;
import org.apache.camel.builder.RouteBuilder;
import org.apache.camel.impl.DefaultCamelContext;
import org.apache.camel.util.jndi.JndiContext;
import org.apache.commons.net.ftp.FTPClientConfig;

public class CamelJavaDSL extends RouteBuilder {
  private static CamelContext context;

  public CamelJavaDSL() throws Exception {
    JndiContext jndiContext = new JndiContext();
    // Will be used in ftp component
    jndiContext.bind("ftpConfig", getFtpConfig());
    context = new DefaultCamelContext(jndiContext);
    context.addRoutes(this);
    context.start();
  }

  @Override
  public void configure() throws Exception {
    // noop - don't delete file in the source.
    // recursive - look into subfolders.
    // include - pick the files whose names matches the regex.
    // autoCreate - create folders in output if not available.
    // fileName - rename the output files based on the pattern mentioned by
    // file expression.
    // keepLastModified - preserves the Last Modified date of source files
    // in the destination folder.
    // setBody method replaces the message body with the content provided.
    from("file:input?noop=true&recursive=true&include=.*\\.txt").autoStartup(false).routeId("executeFileComponent")
      .setBody(simple("${body} Javanbeyond"))
      .to("file:output?autoCreate=true&fileName=${file:name.noext}_cp.${file:name.ext}&keepLastModified=true");
    
    from("file:input/sub-input?noop=true").autoStartup(false).routeId("executeFtpComponent")
        // We can configure additional options on the ftpClient
        // from the URI directly by using the "ftpClient." prefix
        // Additionally, FTPClientConfig class can be referred
        // by its name in Jndi Context binding
      .to("ftp://ftp.filegenie.com?username=javanbeyond&password=XXXXXX&ftpClient.dataTimeout=30000&ftpClientConfig=#ftpConfig");

  }

  public CamelContext getCamelContext() {
    return context;
  }

  public FTPClientConfig getFtpConfig() {
    // Configuration for ftp component
    FTPClientConfig ftpConfig = new FTPClientConfig();
    ftpConfig.setServerLanguageCode("en");
    return ftpConfig;
  }
}

 

6b. Camel Spring XML DSL

file:camel-context.xml. Spring XML equivalent to Java DSL .

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="
         http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
         http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd">

	<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
	<!--         noop - keeps file in the source
				 recursive - looks into subfolders
				 include - picks the files whose names matches the regex
				 autoCreate - create folders in output if the available
				 fileName - rename the output files based on the pattern mentioned by file expression
				 keepLastModified - preserves the Last Modified date of source files in the destination folder.
				 setBody method replaces the message body with the content provided
	 -->
		<route id="executeFileComponent" autoStartup="false">
			<from uri="file:input?noop=true&amp;recursive=true&amp;include=.*\.txt" />
			<setBody>
			<!-- Using simple expression to append string in message body -->
				<simple>${body} Javanbeyond</simple>
			</setBody>
			<to
				uri="file:output?autoCreate=true&amp;fileName=${file:name.noext}_cp.${file:name.ext}&amp;keepLastModified=true" />

		</route>
		<!--  We can configure additional options on the ftpClient
						 from the URI directly by using the "ftpClient." prefix.
						 Additionally, FTPClientConfig class can be referred
						 by its id in Spring context -->
		<route id="executeFtpComponent" autoStartup="false">
			<from uri="file:input/sub-folder?noop=true" />
			<to
				uri="ftp://ftp.filegenie.com?username=javanbeyond&amp;password=XXXXXX&amp;ftpClient.dataTimeout=30000&amp;ftpClientConfig=#ftpConfig" />
		</route>
	</camelContext>
	<bean id="ftpConfig" class="org.apache.commons.net.ftp.FTPClientConfig">
		<property name="serverLanguageCode" value="en" />
	</bean>

</beans>

 


Example Execution:

Case 1: File component

  1. Make changes to Main class by calling either executeJavaDSL or executeSpringDSL with “executeFileComponent” as argument and commenting other methods.
  2. Run the example by running maven clean and install and passing either “xml” or “java” to the Main Class.
  3. Wait for the program to end. Refresh the project.
  4. An output folder containing text files should get created with file names changed and “javanbeyond” added to the content.

 

CamelFileOperationsOutputFolder

Case 2: Ftp component

  1. Make changes to Main class by calling either executeJavaDSL or executeSpringDSL with “executeFtpComponent” as argument and commenting other methods.
  2. If needed, configure ftp server name and username and password in ftp URIs.
  3. Run the example by running maven clean and install and passing either “xml” or “java” to the Main Class.
  4. You should see the input1.txt file inside ftp server. Using Filezilla, following is the view:

FtpCompFilegenieServer

 

 

Leave a Reply

Back to Top
%d bloggers like this: