If you are here I will assume you know what akka is and what an akka-stream is. In this particular use case we want to move stuff from say an S3 bucket to an SFTP server or another S3 bucket
If you do not know what alpakka is, please go their repo and read on, it will help you on a lot of common use cases.
Before we start, a couple of considerations:
In most of the documentation or examples you’ll find that when you create your stream, you know both the name of the file on the source and the name of the output file.
That alone can make you just drop alpakka and use the AWS Java Client on a regular Flow/Sink, then you can grab information from an item flowing thru the stream to get the desired name on the output file.
But there is a way to keep using alpakka and its benefits and still get the information from within the stream.
Lazy Sinks
This behaves like a lazy val in Scala, it delays the actual construction of the Sink, using the first function on the code, until the first element of the stream arrives, if no element arrives then it uses a fallback function, the second on the code above.
Bonus: How do you know the name of the S3 file at the Source
You can use bucket events and SQS, or you can list the content of the bucket using a prefix. Either way you get the actual name later on. So how do you add Sources in the middle of the stream?
Use flatMapConcat
This stage allows you to transform items into Sources!
Conclusion
Alpakka is a great companion when building akka streams, it gives you tools to interact with a huge number of resources without having to build the code from scratch if you ever need to, it gives you a great starting point, its most valuable resources are the unit test on the repo, even more than the documentation I would say.
