Vyne is an automated data platform, and a fundamentally different way to approach integration  - fully autonomous data.

If you're spending time pulling data from different APIs, databases, streaming topics, and more, than Vyne can help.

Vyne replaces resolvers, integration code, and drag & drop UI’s.  Instead, data producers push schemas (enriched with Semantic Metadata) to Vyne, which uses metadata to integrate services on-the-fly, as needed.

As things change, Vyne automatically adapts.

In this post and series of videos, we’ll take a look at Vyne, and some of it’s capabilities.

Here's an outline, if you wanna just skip to the bit that piques your interest:

Introduction

Demo Overview

Our demo takes place in a fictitious Films company.  We have data spread across a SQL Database, REST Microservices, and a Kafka topic serving Protobuf.

We'll be building a data feed that pulls data from across all of these sources. We'll then introduce some breaking changes in our data sources, and see how Vyne automatically adapts.

Querying for data

With Vyne, consumers write queries for data, specifying the data they would like, in the way that suits them.

Querying feels a lot like writing a GraphQL query - in that consumers get to define the data they want.  The differences are that consumers get to define the schema they want, inline - so if you want data expressed in a flat columnar structure, or nested structure - that's up to you.  

There isn't a single graph schema, so you don't end up with lots of consumers attached to a single fixed GraphQL schema.  And, there's no resolvers in the back-end to maintain either.

Vyne uses schemas published by data producers to work out where data is, and fetch from different sources, dynamically building integrations on-the-fly.

In this example, we built the following query:

// Grab film data from the database
find { Film[] } as {

    // select only the properties we're interested in
    id : FilmId
    title : Title
    description: Description
    
    // Enrich with data from other services ...
    // ... where can I watch this film?
    streamingProviderName: StreamingProviderName
    price: StreamingProviderPrice

    // ... what are the reviews?
    review: FilmReviewScore
    reviewText: ReviewText
}[]

Vyne automatically created multiple integrations to deliver this query:

  • A SQL Query to fetch film data
  • A series of REST API calls to show where we can watch the film
  • A series of REST API calls to fetch reviews

Adapting to breaking changes

With Vyne, because there's no integration code (no resolvers, or field mappings buried in a UI somewhere), when producer services change, Vyne automatically adapts.

In the next two videos, we'll introduce a couple of breaking changes to our producer services.  Vyne's integration automatically adapts.

This means no breaking changes for consumers, and no having to co-ordinate change.

Refactoring a service

First, we'll introduce breaking changes to one of our services - changing from a GET to a POST, and refactoring the request and response objects.

As the service restarts, it pushes it's updated schema data to Vyne.

Query consumers can continue unaffected, as Vyne's integration adapts automatically.

Changing the Id scheme

In this video, we're replacing our Reviews Service with a new service that uses a different Id Schema (eg., perhaps we're now buying Review data from an external source).

Previously, we could use the FilmId we got from the database could be used.  Now, there's an additional Id resolution step that needs to be performed.

We make this change by changing the Id type consumed in the new Reviews service.  We'll push this as a breaking change, and re-run the query.

Vyne automatically adapts, detecting that the Ids are incompatible (because they have different types), and calls a new service to resolve the Ids.

Querying streaming data

Finally, let's link our data with a real-time streaming service - a New Releases message written in Protobuf to Kafka topic.

This time, we adjust the query, because we want a stream of data, triggered each time a NewFilmReleaseAnnouncement message is written to our Kafka topic.

stream { NewFilmReleaseAnnouncement } as {
    announcement: NewFilmReleaseAnnouncement

    id : FilmId
    title : Title
    description: Description

    streamingProviderName: StreamingProviderName
    price: StreamingProviderPrice

    review: FilmReviewScore
    reviewText: ReviewText
}[]

Summary

Vyne uses schemas (enriched with Taxi metadata) to automatically integrate services.

This gives us rich federated data access, without any of the manual integration, resovlers, field mapping required by other approaches.

For consumers - this means fewer protocols to learn, fewer dependencies to manage, and fewer data sources to connect to.

In this post, we saw a single query that linked data from across multiple sources:

  • A SQL Database
  • 3 RESTful Microservices
  • A Kafka topic, writing Protobuf

And, we saw Vyne automatically adapt to breaking chagnes - keeping data consumers protected.

If you're interested in exploring Vyne (it's free), try our Getting Started tutorial, join us on Slack, or book a call with one of our engineers to chat more.