Sftp with Spring Integration

Posted: December 30, 2016 in Uncategorized
Tags: , , , , , ,

Spring integration makes it easy to monitor an sftp server for new files and inject those files into your application for processing.  I like to configure my spring integration applications with annotations and @Bean configurations.  I found the documentation, and online examples for doing this annotations a bit lacking.

 

This post demonstrates the basics on how to connect get a spring-boot (1.3.x or 1.4.x) application to monitor a sftp server.

 

You need add the spring-integration-sftp library to your project. With maven you can do this with the following in your pom.xml

 

 <dependency>
   <groupId>org.springframework.integration</groupId>
   <artifactId>spring-integration-sftp</artifactId>
 </dependency>

Next you need to define a set of beans in one of your @Configuration classes

 

 

Default Poller

@Bean(name=PollerMetadata.DEFAULT_POLLER)
 public PollerMetadata defaultPoller()  {
   PollerMetadata pollerMetadata = new PollerMetadata();
   pollerMetadata.setTrigger(new PeriodicTrigger(600000));
   return pollerMetadata;
 }

The spring-integration documentation (and error messages) allude to a default poller but the section on the sftp inbound adapter doesn’t provide an example with annotations.  The default poller controls how often the sftp server will be scanned for new files

 

 

SftpSessionFactory

The SftpSessionFactory creates the sftp sessions.  This is where you define the host , user and key information for your sftp server.

 

@Value("file:///tmp/private_key")
 Resource privateKey;

@Bean
 DefaultSftpSessionFactory sftpSessionFactory() {
   DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
   factory.setPrivateKey(privateKey);

    factory.setHost("localhost");
    factory.setUser("sftpuser");
    factory.setPort(22);
    factory.setAllowUnknownKeys(true);
    return factory;
 }

 

SftpInboundFileSynchronizer

The SftpInboundFileSynchronizer uses the session factory that we defined above. Here we set information about the remote directory to fetch files from.  We could also set filters here to control which files get downloaded.

 

@Bean
 SftpInboundFileSynchronizer sftpInboundFileSynchronizer() {
   SftpInboundFileSynchronizer fileSync = new SftpInboundFileSynchronizer(sftpSessionFactory());
   fileSync.setDeleteRemoteFiles(false);
   fileSync.setRemoteDirectory("/tmp/filedir");
   return fileSync;
 }

MessageSource

The Message source bean uses the @InboundChannelAdapter annotation.  This message source connects the hfile synchronizer we defined above to a message queue (sftpChannel).  The adapter will take files from the sftp server and place them in the message queue as messages.

 

 @Bean
 @InboundChannelAdapter("sftpChannel")
 public MessageSource<File> sftpMessageSource() {
   SftpInboundFileSynchronizingMessageSource source = new SftpInboundFileSynchronizingMessageSource(sftpInboundFileSynchronizer());
   source.setLocalDirectory(new File("/tmp/local_inbound"));
   source.setAutoCreateLocalDirectory(true);
   return source;
 }

 

Message Consumer

The message consumer is where you get to process the files that are downloaded.  A real application would either have a much more complicated handleMessage method or would pass the message to more elaborate spring integration message flow.

 

The @ServiceActivator annotation is used to tell the service which message queue to  monitor (this must match the name on the InboundChannelAdapter annotation).  The message handler handles messages from this queue while the inbound channel adapter places messages from the sftp server into this queue.

 @Bean
    @ServiceActivator(inputChannel="sftpChannel")
    MessageHandler messageHandler() {
        return new MessageHandler() {

            public void handleMessage(Message<?> arg0) throws MessagingException {
                File f = (File) arg0.getPayload();
                System.out.println(f.getName());
                //do something usefull
            }

        };
    }

 

 

This example will refetch all of the files from the sftp server each time the application restarts.  Techniques to keep track of which files have already been successfully been processed  are a topic for another post.

 

 

Advertisements
Comments
  1. joe says:

    Great example. Do you have an example using the SFTP Outbound Channel Adapter with annotations only?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s