The SnapLogic August Release is here to help you connect your enterprise more securely with support for GitHub Enterprise and Hashicorp Vault. You can now store all SnapLogic pipeline metadata along with your organization’s IP in GitHub Enterprise Server and use your HashiCorp Vault to store all organization’s secrets. We are releasing a number of new Snap Packs in this release such as Databricks, Azure Synapse Analytics, Amazon S3, API Suite, Workday REST along with a number of Snap Pack enhancements. We are introducing AutoPrep, for faster automated data preparation for every data project. We are enhancing our public APIs to support granular copy, API Management with SSO support for the Developer Portal and new dashboards. Do you want to learn more about these and other enhancements? Please read on!
Connect your enterprise more securely with GitHub Enterprise support
Many organizations use GitHub Enterprise Server installed in their own VPC or data center to store all their organization’s code. With the August release, you will be able to store metadata associated with all your SnapLogic pipelines in GitHub Enterprise, enabling your IT teams a unified view of all their IP. The IT team no longer needs to manage multiple code repositories, build pipelines to create and struggle with maintaining a uniform set of governance policies. SnapLogic is an official GitHub Technology Partner! That means, whether you use GitHub directly through GitHub.com (GitHub’s cloud offering) or through SnapLogic’s native integration with GitHub, you will enjoy a seamless, reliable, and secure experience.
To learn more about SnapLogic’s GitHub Enterprise Support, please check out this demo video.
Centralize all your secrets with support for HashiCorp Vault
When you connect to any SaaS or on-premises account, you have to provide username, password, and secret key, among other things. This information is encrypted and saved in the SnapLogic Control Plane. But today, more and more organizations are relying on 3rd party specialist software to store and manage secrets across the entire organization. In fact, new information security mandates in some organizations ask for a centralized location for all secrets. With the August release, we are adding support for Hashicorp Vault! Now, your Snaplex can fetch secrets – such as passwords and secret keys – at runtime, directly from the Vault, without SnapLogic Control Plane involvement. SnapLogic is an official HashiCorp Technology Partner! You can use HashCorp Community, Enterprise or Cloud Vault to store credentials. Use of a secret’s manager also allows you to easily migrate accounts between DEV, TEST, PROD environments which leads to an improved CI/CD experience for your integration pipelines.
To learn how you can use HashiCorp Vault with the SnapLogic platform to handle your secrets, please check out this demo video.
New Snap Packs and Snap Pack enhancements
We have added a number of new Snap Packs in the August release so that you can build integrations faster and deliver automations and data for insights quicker.
New Amazon S3 Snap Pack has been added to meet rapid growth in usage of Amazon S3 on the SnapLogic platform. This new Snap Pack is optimized for large and small file transfers. With a new Browser Snap, you can browse and filter files in an S3 bucket easily. You can also easily connect to AWS S3 with enabled VPC endpoints with this dedicated Snap Pack for Amazon S3.
With the new Databricks Snap Pack you can leverage SnapLogic’s visual ETL for code-free transformations to deliver data to Databricks Lakehouse Platform. This Snap Pack provides typical Cloud Data Warehouse Snaps such as Select, Insert, Delete, Bulk Load, Unload, Merge Into, and Multi Execute. This is in addition to SnapLogic’s support for ELT that pushdown transformations to Databricks.
Soon to be released, the new Azure Synapse Snap Pack will provide all standard data warehouse Snaps such as Select, Insert, Update, Delete, Execute, Stored Procedure and Bulk Load among others and enables you to connect to this powerful data warehouse and analytics platform from Microsoft.
With the August release, we are releasing our first version of our new Workday REST Snap Pack to connect to Workday HRM, Financials, and Student systems. The Workday REST Snaps provide faster query response and better object modeling. It also provides better security for organizations with support for OAuth & private key certificates.
Another new Snap Pack this August is the API Suite Snap Pack. HTTP Client is the first Snap in this pack. HTTP Client is the next generation of our REST Snaps for you to connect to any custom web based application and API endpoint. It provides a better (Postman-like) experience for all HTTP operations and simplifies debugging as it allows you to track each request and response. It also provides in-built support for pagination in the response data.
Additionally, we have improved a number of Snap Packs. Here are some key Snap enhancements:
- Google BigQuery Snap Pack
- Enhanced with 5 new Snaps to manage datasets in Google BigQuery – Dataset Create, Dataset Delete, Dataset List, Table Delete, and Table List
- Snowflake Snap Pack
- Now supports asymmetric key pair authentication (public-private key pair) for account access
- DynamoDB Snap Pack, Redshift Snap Pack
- Now support AWS Security Token Service (STS) which allows organizations to give temporary access to DynamoDB/Redshift tables
- Postgres Snap Pack
- Provides SSH tunnel for Postgres accounts, providing an end-to-end encryption between the client and the Postgres Server
- OpenAPI Snap
- Now captures the schema from a file referenced in an OpenAPI specification
Aside from these Snap updates, we are also making available to our customers a catalog of custom Snaps that have been developed by the SnapLogic Professional Services team. Through this catalog, you can get access to Dremio, Docusign, Google Pub/Sub, Salesforce Marketing Cloud, SAP Ariba, and popular communication protocols/parsers such as EDI, HL7, and MARC21. To get access to any of these custom Snap Pack, please contact your customer success manager.
To get a complete list of Snap updates, please visit our documentation here.
Finish projects faster with automated data preparation
When building data integration pipelines, data preparation is omnipresent. Data preparation is also quite time consuming. Reasons for that are numerous. Even if two data silos contain similar data, data formats can be different, or data hierarchy (schema) can be different. With the August 2022 release of the SnapLogic Intelligent Integration Platform, we are introducing AutoPrep. A tool for you to automatically flatten hierarchical JSON or XML objects, do type conversions or drop fields. AutoPrep can flatten hierarchical schemas without any restriction on the number of levels. AutoPrep leverages Iris AI to suggest data types based on the target schema and the type of data and it provides a guided interface for you to easily drop or include fields/columns to fit your target schema. AutoPrep is going to significantly improve user productivity and we will be adding more capabilities in the upcoming months.
To get details on how you can accelerate time to value with AutoPrep, check out this extended demo video.
ELT Updates: Automatic Schema Inference and more
Schemas in cloud data warehouses can be complex since they power all analytical workloads. When you are creating a new table with hundreds or thousands of columns, it can easily take 3-5 hours, or even more. Now you can do it in less than a minute, with Automatic Schema Inference, a new feature added to the ELT Load Snap. ELT Load Snap looks at the first 400 records in the input CSV and JSON files and detects the column names and data types for each column of the target table. It can also infer schema from column metadata provided by input file in Parquet, Avro, or ORC file format. The table is then created automatically in the target CDW if it does not already exist. Automatica Schema Inference with ELT Load is currently available for Redshift with support for other CDWs coming soon.
Additionally, with the August release, we have added support all the latest JDBC driver for your ELT workloads for all Cloud Data Warehouses i.e. Snowflake, Redshift, Azure Synapse, Databricks, and Google BigQuery.
New API dashboards, public APIs, and SSO for API Management
We have enhanced our API Management offering with new dashboards that give you insights into API requests from API consumers. You can track subscription activity through these charts and can get detailed logs on subscription activity and authentication details. You also can export these logs via public APIs so that you can view them through an external tool of your choice.
We are also enhancing SnapLogic Developer Portal with the support for Single Sign On (SSO). With this SSO enabled Developer Portal, your internal users can access the Developer Portal through the same seamless SSO experience that they enjoy on the SnapLogic platform.
Easy debugging with the new Infrastructure View in SnapLogic Studio
We have introduced a new Infrastructure View in the SnapLogic Studio so that you can easily debug infrastructure issues. The Infrastructure View provides a visual view of all your nodes and color codes for each based on their memory/CPU/disk consumption level. Clicking on a node takes you to a detailed view including OS details, uptime details, disk and memory details along with a list of pipeline execution details.
New Cloudplex Regions
SnapLogic serves customers all across the globe. To provide superior product experience and to support data residency requirements for customers we are adding two more Cloudplex regions: Canada and the United Kingdom. This is in additions to the existing Cloudplex regions that SnapLogic supports i.e. US West, Ireland, Germany, Singapore, and Australia.
The August 2022 release introduces a number of new features and Snap Packs, so try them out in the product and let us know your feedback! If you missed the August 2022 product release webinar, you can watch it here and hear from our product team. You can always learn more about these updates by visiting the SnapLogic Documentation.
If you are not yet part of the SnapLogic Community, join the conversation here. Engage with others and get your questions answered, and for upcoming product releases check out the ‘Product Updates’ section in the community to get your first glimpse into what is coming next.
