Today we've released version 0.19 of Vyne, and it's a good 'un.

If you haven't heard of Vyne yet, it's a Data Gateway that makes it easy to connect, combine and transform data across your enterprise, automatically handling all the messy integration code - including automatically updating as data sources & APIs change.  Vyne provides cool things like:

  • A federated API for all your data, automatically integrating and stitching APIs together
  • Dynamic data pipelines which transform and enrich your data declaratively without requiring complex transformation code
  • No-code data stores, that turn offline data (like CSVs and batch files) into powerful REST microservices.  (We call these "Casks").
  • A BYO Data Model approach to serving and consuming large datasets on your own terms.
  • Fully automated data catalog and lineage for all your enterprise data.

Now, onto the new stuff - what's in 0.19?

Parallel work distribution

This one's really exciting.  For large workloads where customers are running reports that have heavy transformations & enrichments, those workloads used to take some time.  

Now, Vyne can distribute the work transparently over a cluster of nodes.  This gives near-linear reduction in time.  (ie., running a cluster of 3 nodes reduces the time to complete to roughly 1/3rd).

Release Notes   |   Documentation

Continuous Queries

Vyne has always been great at fetching data over REST from HTTP services, transforming and enriching it as required.

However, we now support streaming results from continuous queries - queries that have infinite results.  Using Taxi's new Stream<T> syntax, Vyne can fetch data from any services that expose a Stream of data.  

Currently, we support services that expose text/event-stream response types, but support for message brokers (such as Kafka) is coming soon.

Of course, continuous queries still have the full power of Vyne's transformations and enrichments.  This means  you can subscribe for a feed of data, and seamlessly transform to the shape you require, including asking for data stitched in from other services - all with a single taxiQL query - no integration code required - neat huh?

stream { CustomerOrders } as {
  // mixing and matching data from an event...
  orders : PurchasedItem[]
  orderTotal : OrderCost
  // ... and data enriched from other services...
  customer {
  	 id : CustomerId
     address : CustomerAddress
     customerName : String as CustomerFirstName + ' ' + CustomerLastName
  }
}

Release Notes | Documentation

Continuous feeds from Casks

Casks are our lightweight data store - a simple way to take data, and expose it as a fully fledged RESTful microservice.

We now support issu streaming updates from Casks as they come in, allowing you to turn RESTful API into a full event stream.

We also correctly auto-generate the schema information required to expose these streaming endpoints as Stream endpoints, so that Casks can be used as a source for a Continuous Query.

Release Notes | Documentation

Cask data retention policies

Casks are a great place to turn offline data into queryable content.  However, like all data stores, sometimes you want to only keep data for a short term.

Cask retention policies have you covered - letting you optionally specify a duration to keep data.

Release Notes | Documentation

New docs site

We have a new documentation site, over at docs.vyne.co.  We've migrated our documentation infrastructure away from Gitbook, and onto a Netlify Gatsby stack.  This allows us to keep our docs in our code, and maintain documentation as we go.

We're working on improving our docs all the time, and this is a big step in the right direction.

Support for HTTP Authentication headers

Vyne can now be configured to pass additional HTTP Authentication headers when calling services.

Configuration is defined per-service, and can be a mix of explicitly configured values, or values passed into Vyne through config and environment variables, to help keep your credentials secure.

Release Notes | Documentation

Take it for a spin

Vyne 0.19 is out today, and we can't wait for people to give it a spin.  It's easy to get started with our docker images - simply start up Vyne:

docker run -p 9022:9022 vyneco/vyne

As always, we're available on Slack to answer any questions - just reach out and say Hello!