TENGU Platform update: Taking data environment management to the next level.

Posted on 17-sep-2021 15:22:58

After our latest update, DataOps for data analytics. We continued our further development of the TENGU platform for DataOps orchestration. While this update isn't focused on a specific aspect such as data analyst support, it will improve on many quality-of-life aspects of the platform and improve its ease of use.

We've also had an update supporting data visualisation from the TENGU platform, which is featured in its own separate blog.


Let's take a look at the new and improved features.

Table of contents:

1. New overview tab

2. Element grouping and collapsing

3. Extra element-based actions

4. New buckets, file resources, and databases support

5. Two Modes of deployment

6. Improved Tengu assistant

7. More extractor and loader options for pipelines

8. Further quality-of-life improvements

1. The new Overview tab

If you wanted to look up info on an integrated resource kind in the TENGU platform graph view, or elements as we call them, you'd have to look into the code tab to find what you need. We improved this by adding a tab that has all the basic info and metadata on it, as well as its status and conditions, which is colour coded as to indicate its status, so you can easily see if everything is working as intended without diving into the code tab.

Element grouping and collapsing

More functionalities for the grouping function were long due, and we can finally present groups the way we originally meant them to be, as a way to better organise and navigate your data environment.

First off, we've moved groups to a separate tab in the side nav for easier management of your groups. Also, allowing more control over your groups in a way that feels more natural.

Secondly, adding multiple elements to a group allows you to find each component or element in the graph even faster by highlighting them in the graph overview and a more detailed list when selecting a group.

But the part we're most excited to announce, is that you can now collapse groups, hiding all smaller elements in one big group element. Enabling a better overview by hiding elements without the need to use a filter. This is as simple as grouping your elements and right-clicking any element in that cluster to collapse them into our new group element.

This way we can further differentiate tags and groups into their respective purposes, honing in on groups as a way to contextualise, organise and navigate the graph overview of your data environment. Tags will of course stay relevant as a way to label, and also quickly find all relevant elements to that label.

Our next steps forwards, will be to allow the TENGU user to add comments and documentation to these groups, further improving on its functionality.

Extra element-based actions

One of our main goals, is to offer both data specialists and non data specialists an intuitive way to start up, execute and manage their data projects. That's why we aim to enable you to perform crucial actions from within the platform interface, which should be automated to make it as straightforward as possible.

Because of this, we've heavily extended the actions you can perform from the side nav interface of many elements, making sure you can take any relevant action right away without searching for other tools or elements.

For example, when a Kubernetes integration is selected, the next step would be to add a namespace which is now possible from the Kubernetes element. You can now more easily add tables to databases. And as a final example, you can now more easily connect a database to a notebook as pictured below.

New Buckets, file resources, and databases support

To support even more projects, we've added additional resource kinds for you to use in TENGU. These include:

  • Google Cloud Bucket, AWS S3 Bucket on top of the MinIO bucket already supported
  • FTP file server support
  • CSV file object support
  • Parquet File object support
New Database and Data store integrations:
  • MariaDB support
  • MSSQL - database support
  • AWS Redshift - cloud data warehouse support
  • Snowflake support

We hope, that with these extra resource kinds support, you'll be able to tackle any data project with the TENGU platform.

Two modes of Deployment

With our goal of enabling specialists and non-specialists alike in their data projects, we provided the option to pick between simple or advanced deployment when deploying a resource kind as an element in the graph. Adding to the flexility and customizability of using the TENGU platform, without sacrificing the ease of use.

The Simple Deployment is the deployment that was available as the only type of deployment up until now. Its focus was always to enable the user to work as efficiently as possible, by simply providing a couple of parameters, and letting the platform do the rest through its automated deployment process.

What's new is the option to pick Advanced Deployment, allowing you to provide a Deployment Manifest yourself. This option offers more customisability to the user, allowing their own JSON code or other valid alternatives.

Improved Tengu assistant

Great news for those that are new to our platform. We've added a lot of additional instruction to the TENGU assistant, our little built-in helping bot, which gives you relevant information and instructions based on what you're doing in the platform.

No matter what you're deploying or selecting, the Tengu assistant will be ready with additional info, instructions, and as you can see below, even an example of a Deployment Manifest when using the advanced deployment function. 

We're also working on many how-to tutorials, and we'll be including links to these in the TENGU assistant so using the TENGU platform will be easier than ever.

More extractor and loader options for pipelines

An important part of any data project, are its pipelines. That's why we've expanded upon the options when deploying a data pipeline in the TENGU platform. We've extended the list of potential extractors and loaders when setting it up. 

We're happy to share with you, the updated list of supported extractors:

  • Hubspot
  • Stripe
  • ArangoDB
  • Google Analytics
  • Google sheet
  • PostgreSQL
  • File object
  • Jupyter Notebook*

As well as the current list of loader options for pipelines:

  • ArangoDB
  • Google sheet
  • PostgreSQL
  • File object
  • Jupyter Notebook*

As you might've noticed, you can now use a Jupyter notebook as both an extractor and loader. This allows you to connect any technology which isn't integrated in our platform through API yet. While you have to code your own API, you only have to add a specific formatted code block to make it compatible with the TENGU platform (how-to knowledge base article coming up). This way you can create streams that you can map to an integrated loader or extractor. Essentially allowing you almost infinite possibilities to extract and load data in the TENGU platform.

Further quality-of-life improvements

Now onto some smaller, but still important quality-of-life improvements.

We've further upgraded the filter functionality for better ease of use. The graph will now update itself while you're setting up your filter, this makes creating your personalised filter more intuitive, as you can see what your settings will filter out, and you're able to quickly toggle different elements on and off.

As of now, there are multiple small UI improvements, such as visual elements in an element's side nav, which indicate it resource type, as well its status next to the name of the element.

In the same side nav, we've made the managing of tags easier, making it easier to add and remove tags.

We've also improved the search function, allowing more ways to search an element in the graph, now you can:

  • Find groups
  • Find a resource kind to add to the graph, allowing you to quickly deploy the resource kind you want through the search bar as wel

  • Find an element through its UUID (the unique code that every element in the graph has)

Last but not least, we've improved the System status & notifications overview tab. You can now find more and better detailed information in the overview, where you can track events from your entire environment in 1 place.

We hope you look forward to getting the most out of your data with the TENGU platform!

Want to see if the TENGU platform is something for your team? Don't hesitate to contact us below, and we'll gladly set up a demo.

Schedule Your Demo Now!

Topics: Update

Jaco Van Laere

Written by Jaco Van Laere

Jaco is a marketeer and content creator at TENGU, and has a profound love for all things data, tech and media.

Join our 100+ subscribers!

Stay informed about topic related to DataOps, data management, Tengu, interesting data events.