This article is brought to you by our Senior Director of Product Management, Craig Stewart.
In the Fall 2014 release, SnapLogic added a number of new features across the broad range of Snaps. Amongst those was the ability for a REST GET operation to accept gzip-encoded data. When combined with a triggered pipeline in another Snaplex, this can add significant performance and reliability (the less time you spend moving data over the wire, the less total packets moved, the less scope there is for network errors, and the less time it should take).
As an example, I created a simple pipeline which outputs a set of data, in this case just an Oracle database query returning 101,000 rows of data:
For this, I created a task so I could call it using the REST GET Snap in the other pipeline:
To call it, I created a pipeline using the REST GET snap, which would call this URL:
As the URLs for triggered pipelines require authentication, I created and assigned a Basic Auth account with my credentials, and associated it with the REST GET Snap. The URL is copied and pasted from the task created previously. This was all possible in earlier versions of SnapLogic. The change in this version, is the ability to add the content-type accept headers:
Now what will happen is that the Snap, if it gets data in gzip format, will automatically uncompress and process that data received (even when not from a SnapLogic triggered pipeline). No additional Snaps required. The clever bit is that the the triggered pipeline will also note that the caller is able to accept gzip format, so it will automatically send the data in that format.
In summary, you just need to add the HTTP Headers to the REST Get.
As an aside, the Task Execute Snap will do this compression automatically, to be covered in a future post. For more SnapLogic Integration Cloud best practices and tips and tricks, be sure to check out our TechTalk webinars and recordings.
